Regression analysis method, regression analysis system, and regression analysis program

Abstract

The present disclosure provides a more general technique for constructing a regression model having a correspondence between a variation in an explanatory variable and a variation in a target variable. The regression analysis method is performed by a computer by: reading out training data from a storage device that stores training data serving as a target variable and an explanation variable of a regression model; and performing regression model-based machine learning using the training data in a manner that minimizes a cost function including a regularization term. The regularization terms include: a first term that increases cost in an interval in which the coefficient is positive compared to an interval in which the coefficient is negative; and a second term that increases cost in an interval in which the coefficient is negative compared to an interval in which the coefficient is positive.

Description

Technical Field

[0001] The present disclosure relates to a regression analysis method, a regression analysis system, and a regression analysis program. Background Art

[0002] Conventional methods for estimating regression model parameters using the least squares method have encountered the following problems: for example, when the number of data samples is small, it is difficult to obtain appropriate least squares estimates. Therefore, methods that impose a constraint called the L1 norm have been proposed (for example, see Non-Patent Document 1). LASSO (Least Absolute Shrinkage and Selection Operator), a parameter estimation method that uses the L1 norm as a constraint, simultaneously selects explanatory variables suitable for explaining the target variable and determines their coefficients.

[0003] In addition, a regression analysis device has been proposed, which includes: a data acquisition unit that reads out training data and constraints from a storage device, the storage device storing training data of target variables and explanatory variables used as a regression model, and pre-defined constraints that determine whether the explanatory variables should be changed in a positive or negative direction in order to change the target variable in a positive or negative direction; and a coefficient updating unit that repeatedly updates the coefficients of the explanatory variables in the regression model using the training data in a manner that minimizes a cost function including a regularization term (sign constraint regularization), the regularization term increasing the cost when the constraint is violated (for example, Patent Document 1).

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: International Publication No. 2021 / 157670

[0007] Non-patent literature

[0008] Non-patent literature 1: Robert Tibshirani, "Regression Shrinkage and Selection via the Lasso", Journal of the Royal Statistical Society.Series B (Methodological) Vol.58, No.1 (1996), pp.267-288 Summary of the Invention

[0009] Problems to be solved by the invention

[0010] Previously, regression analysis devices have been proposed that predefine constraints, such as whether to cause an explanatory variable to change in a positive or negative direction in order to cause the target variable to change. This allows for high-precision machine learning even with relatively little training data. However, preparing appropriate constraints based on the subject of regression analysis is laborious. Therefore, the present technology aims to provide a more general technique for constructing a regression model that establishes a correspondence between changes in the explanatory variables and changes in the target variable.

[0011] Solutions for solving problems

[0012] (Scheme 1)

[0013] The regression analysis method includes a computer that performs the following actions: reading training data from a storage device storing training data of target variables and explanatory variables used as a regression model; and using the training data to perform machine learning based on the regression model in a manner that minimizes a cost function including a regularization term. The regularization term includes a first term that increases the cost in intervals where the coefficient of the explanatory variable is positive compared to intervals where the coefficient is negative; and a second term that increases the cost in intervals where the coefficient is negative compared to intervals where the coefficient is positive.

[0014] (Scheme 2)

[0015] In the above-mentioned solution 1, the first term and the second term may include parameters for making either the first term or the second term zero or approximately zero, and in machine learning, the regression coefficients and parameters in the regression model may be repeatedly updated.

[0016] (Scheme 3)

[0017] In the above-mentioned scheme 2, the parameter may be a binary variable, and when the parameter takes one value, it constitutes a factor that becomes zero in the first term, and when it takes another value, it constitutes a factor that becomes zero in the second term.

[0018] (Scheme 4)

[0019] In the above-mentioned solution 3, it is also possible to set the binary variables to values ​​that approximate the growth curve, and calculate the regression coefficients and parameters in a manner that minimizes the cost function.

[0020] (Scheme 5)

[0021] In the above-mentioned scheme 3 or 4, the binary variable may be set to a value determined based on the magnitude relationship between a predetermined value represented by a regression coefficient and the random variable, and the regression coefficient and parameter may be calculated so as to minimize the cost function.

[0022] The object to which the regression analysis method is applied may also be equipment in the manufacturing industry or the non-manufacturing industry. The explanatory variable may also be the sensor data output by the production equipment. In addition, the target variable may also be the operating condition of the equipment, or a specified characteristic value or abnormality. It should be noted that the contents recorded in the solution for solving the problem can be combined as much as possible without departing from the scope of the problem and technical idea of ​​the present disclosure. In addition, the above-mentioned solutions 1 to 5 can be provided as a system including a device such as a computer or a plurality of devices, a method executed by a computer or a system, or a program for causing a computer or a system to execute. It should be noted that it is also possible to provide a recording medium for maintaining the program.

[0023] Effects of the Invention

[0024] According to the disclosed technology, a more general technology for constructing a regression model having a corresponding relationship between changes in explanatory variables and changes in target variables can be provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a diagram showing an example of observation values ​​used to create a regression equation.

[0026] Figure 2 It is used to describe the factor "R + (w)” schematic diagram.

[0027] Figure 3 It is used to describe the factor "R - (w)” schematic diagram.

[0028] Figure 4 is used to describe the spin variable S k An approximate diagram of .

[0029] Figure 5 This is a block diagram showing an example of the configuration of a regression analysis device.

[0030] Figure 6 This is a processing flowchart showing an example of regression analysis processing.

[0031] Figure 7 It is a graph showing the verification results. DETAILED DESCRIPTION

[0032] Hereinafter, embodiments of the regression analysis device will be described with reference to the drawings.

[0033] <First embodiment>

[0034] The regression analysis device of this embodiment constructs a regression equation (regression model) representing the relationship between one or more explanatory variables (independent variables) and one target variable (dependent variable). The process of determining the regression coefficients of the regression equation is called "learning." Furthermore, in this embodiment, the regression equation is constructed by appropriately applying constraints (referred to as "sign constraints") such that the direction of change (positive or negative) of the explanatory variables and the direction of change (positive or negative) of the target variable have a predetermined correspondence. In other words, learning is performed so that the signs of the regression coefficients are appropriate. It should be noted that, generally speaking, if there is a sufficient amount of training data, the regression coefficients of the regression equation constructed using this data converge so that the directions of change of the explanatory variables and the direction of change of the target variable have a predetermined correspondence. On the other hand, when the amount of training data is small, a regression equation may be constructed in which the correspondence is reversed. The "sign constraints" in this embodiment are applied so that the generated regression equation has an appropriate correspondence even when the amount of training data is small.

[0035] Figure 1 This is a diagram showing an example of observation values ​​used to create a regression equation. Figure 1 The table includes K input x (x1~x K ) columns and output y columns. Input x is used as the explanatory variable and output y is used as the target variable. In addition, it is assumed that the data points t (t1 to t T , ...) to create a regression equation, and input data point t into the created regression equation. T+1 The value of is used to predict the unknown output y.

[0036] The regression equation is represented by the following equation (1), for example.

[0037] [Formula 1]

[0038]

[0039] The regression equation is a linear model where the output y is expressed as the linear sum of the input x. k is the regression coefficient, w0 is the constant term, and they are determined by learning.

[0040] For determining the regression coefficient and the constant term, a cost function (error function) E(w, s) represented by the following equation (2) can be used.

[0041] [Formula 2]

[0042]

[0043] The cost function E(w, s) is the squared error L(w) plus the regularization term (penalty term) S(w, s).

[0044] Furthermore, the regularization term S(w, s) is expressed by the following equation (3).

[0045] [Formula 3]

[0046]

[0047] R ASL (w k , s k ) includes two items. That is, R ASL (w k , s k ) is derived from the first term “(1 / 2)(1+s k ) R + (w k )" and the second term "(1 / 2)(1-s k ) R - (w k )". In addition, s k is the spin variable, which can take the values ​​of +1 (up) or -1 (down). k = +1, the factor of the second term above is "(1-s k )" becomes zero, so only the first term remains, which becomes R ASL (w k , s k ) = R + (w k ). In S k = -1, the factor of the first term above is "(1+s k )" becomes zero, so only the second term remains, which becomes R ASL (w k , s k ) = R - (w k ). In other words, the first term and the second term include a parameter s for making either of the first term and the second term zero. k In addition, the parameter s is a binary variable k When it takes one value, it constitutes a factor that becomes zero in the first term, and when it takes another value, it constitutes a factor that becomes zero in the second term. In addition, L is set to an approximate value of infinity, that is, a finite sufficiently large value.

[0048] Figure 2 It is used to describe the factor "R + (w)" schematic diagram. Figure 2 In the graph, the vertical axis represents R ＋(w), the horizontal axis represents w. In the above formula (3), when input x k The coefficient w k When it is above zero, R ＋ (w) = w, and E(w, s) also increases with the increase of w. On the other hand, when the input x k The coefficient w k When it is less than zero, R ＋ (w) = +∞, and the cost diverges toward positive infinity. That is, the regularization term of this embodiment is in the spin variable S k = +1, when input x k The coefficient w k When it is less than zero, the cost will be infinitely divergent. In the process performed by the regression analysis device, it is sufficient to set it to a sufficiently large value. On the other hand, when the input x k The coefficient w k When it is greater than zero, the cost increases according to the magnitude of the regression coefficient w. + The first term of (w) increases the cost in the interval with negative coefficients compared to the interval with positive coefficients. k When it is above zero, the input x of the regression equation shown in formula (1) is k The more it increases, the more the predicted value y obtained by the regression equation increases. k = +1, in the case of input x k The more the value of increases, the more the predicted value y increases. The regularization term becomes smaller. k The cost function is defined in such a way that the regularization term becomes larger as the value of increases and the value of the predicted value y decreases.

[0049] Figure 3 It is used to describe the factor "R - (w)" schematic diagram. Figure 3 In the graph, the vertical axis represents R - (w), the horizontal axis represents w. In the above formula (3), when input x k The coefficient w k When it is above zero, R - (w) = +∞, and the cost diverges to positive infinity. In the process performed by the regression analysis device, it is sufficient to set it to a sufficiently large value. On the other hand, when the input x k The coefficient w k When it is less than zero, R - (w) = -w, and E(w, s) increases accordingly with the decrease of w. That is, including R - The second term of (w) increases the cost in the interval with a positive coefficient compared to the interval with a negative coefficient. kWhen it is less than zero, the input x of the regression equation shown in formula (1) is k The more it increases, the more the predicted value y obtained by the regression equation decreases. That is, when the spin variable S k = -1, in the case of input x k The more the value of increases, the smaller the value of the predicted value y decreases. k The cost function is defined in such a way that the regularization term becomes larger as the value of increases, and the value of the predicted value y increases.

[0050] That is, in the regression coefficient w k According to the spin variable S k When the value of is non-negative or non-positive, the equation (3) takes a sufficiently large value. K} and s={S1, S2, ..., S K} Minimize equation (2) to achieve the spin variable S k Optimization. For example, the update of parameters w, etc., such as minimizing E(w, s), can be performed using a gradient method. That is, based on the gradient of the variable w, etc. with respect to the cost function E(w, s) in one step, the variable w, etc. in the next step is updated, and this process is repeated until w, etc. converge. If this is set, even if the coefficient w k The sign of is not determined in advance, and the spin variable S is also determined appropriately. k The value of is used to adaptively impose a sign constraint. According to the regularization term described above, it is possible to impose a constraint such as a certain correspondence between the direction of change of the explanatory variable and the direction of change of the target variable to perform regression analysis.

[0051] Spin variable S k Through the continuous variable u k (-∞ k <+∞). The spin variable S k It can be approximated by the following formula (4).

[0052] [Formula 4]

[0053]

[0054] Spin variable S k It is represented by the hyperbolic tangent function. β (>0) is the "inverse temperature parameter". Figure 4 is used to describe the spin variable S k An approximate diagram of . Figure 4 In the graph, the vertical axis represents the spin variable S k , the horizontal axis represents the continuous variable u k ​In addition, the dotted curve has β = 0.25, the single-dot dash curve has β = 0.5, the double-dot dash curve has β = 1.0, and the solid curve has β = 2. When β is +∞, the spin variable S in equation (4) k Takes two values: +1 or -1. When β is finite, the spin variable S k It continuously takes values ​​between -1 and +1 around the origin.

[0055] The error function shown in formula (3) can be expressed as follows: w and u = {u1, u2, ..., u k} is rewritten as follows.

[0056] [Formula 5]

[0057]

[0058] The value of the parameter α indicating the strength of regularization can be determined by cross-validation, etc. Formula (5) can be differentiated with respect to w and u, and the following formula of the gradient method (steepest gradient method) can be derived.

[0059] [Formula 6]

[0060]

[0061] η is the step size that determines the size of the updated coefficient w in one step (one iteration). ± (w k ) in w k = 0. Therefore, the subdifferential value g is used as shown below. + (w k ) and g - (w k ). It should be noted that L is a sufficiently large value.

[0062] [Formula 7]

[0063]

[0064] The calculation based on the steepest gradient method is repeated k and u k Updates, in w k and u k When the value of no longer changes (minimization is achieved), the iteration ends. At this time, the value of the inverse temperature parameter β is adjusted according to the "simulated annealing method". That is, in the iterative calculation of the steepest gradient method, Figure 4 The β shown in the figure is first set to a sufficiently small value, and then gradually increases with the repetition. Finally, the spin variable S in formula (4) is kOnly two values, roughly ±1, are used. To increase the value of β (simulated annealing schedule), for example, a method can be used, where each iteration is multiplied by a fixed number γ greater than 1 (β←γβ). It is known that if the value of β is increased sufficiently slowly, the optimal solution can be obtained with probability 1 without becoming trapped in a local solution.

[0065] If set as above, the coefficient w will be subject to appropriate sign constraints and converge to a value that contributes to the target variable. If there is no such value, the coefficient w approaches zero. That is, if there is no value that satisfies the sign constraint, as Figure 2 and Figure 3 As shown in , the penalty effect based on regularization works, pulling back the values ​​that violate the sign constraint, and thus the result converges to zero. Therefore, it is possible to estimate part of the regression coefficients to zero in the same way as the so-called LASSO. In this way, s is obtained. k (k=1, 2, ..., K) and w k (k=1, 2, ..., K) as the solution to minimize the error function (5).

[0066] In addition, the obtained w k Used as the regression coefficient of the regression equation (1), it can be predicted for new input x K+1 The output y of . It should be noted that it can also be set as s obtained based on the solution of minimizing the error function (5) k (k=1, 2, ..., K) to determine the sign constraint, and use the sign constraint regularization of the existing technology to find w k (k=1, 2, ..., K). That is, if the spin variable S k >0, then apply Figure 2 The non-negativity constraint shown is that if the spin variable S k <0, then apply Figure 3 The non-positive constraint shown in the figure is used to find the regression coefficient w k .

[0067] <Device Configuration>

[0068] Figure 5This is a block diagram illustrating an example of the configuration of a regression analysis device 1 that performs the aforementioned regression analysis. The regression analysis device 1 is a computer comprising a communication interface (I / F) 11, a storage device 12, an input / output device 13, and a processor 14. These components are connected via a bus or other means. The communication I / F 11 may be, for example, a network card or a communication module, and communicates with other computers based on a predetermined protocol. The storage device 12 may also be a primary storage device such as RAM (Random Access Memory) or ROM (Read Only Memory), or an auxiliary storage device (secondary storage device) such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory. The primary storage device temporarily stores programs read by the processor 14 and information processed by these programs. The auxiliary storage device stores programs executed by the processor 14 and information processed by these programs. In this embodiment, the storage device 12 temporarily or permanently stores training data and information representing constraint conditions. The input / output device 13 is a user interface such as an input device such as a keyboard or pointing device, an output device such as a monitor, or an input / output device such as a touch panel. The processor 14 is a calculation processing device such as a CPU (Central Processing Unit) and performs various processes of this embodiment by executing programs. Figure 1 In the example of FIG, functional blocks are shown in the processor 14. That is, the processor 14 functions as a data acquisition unit 141, a regression analysis unit 142, a verification processing unit 143, and an operation processing unit 144 by executing a predetermined program.

[0069] The data acquisition unit 141 acquires the training data and information indicating the constraint conditions from the storage device 12. The regression analysis unit 142 updates the regression coefficient to the solution that minimizes the above-mentioned error function, and determines whether the value of the updated regression coefficient has converged. It should be noted that, if it is determined that it has not converged, the regression analysis unit 142 repeatedly updates the coefficient. If it is determined that it has converged, for example, the regression analysis unit 142 stores the finally generated coefficient in the storage device 12. In addition, the verification processing unit 143 evaluates the generated regression equation based on a prescribed evaluation index. For example, the verification processing unit 143 calculates the determination coefficient R based on the result of the cross-validation. 2 The processing unit 144 uses the generated regression equation and, for example, newly acquired observation values ​​(e.g., Figure 1The predicted value is calculated using the K+1th data of the generated regression equation. Furthermore, the operation processing unit 144 can also use the generated regression equation and arbitrary values ​​to calculate the predicted value when the conditions are changed. Here, the arbitrary value can be, for example, a value input by the user via the communication interface 11 or the input / output device 13. For example, when the operating conditions of the unit are changed, the process state after the change can be predicted.

[0070] <Regression Analysis Processing>

[0071] Figure 6 1 is a processing flow chart showing an example of regression analysis processing performed by the regression analysis device. The data acquisition unit 141 of the regression analysis device 1 reads the training data and the information indicating the constraint conditions from the storage device 12 ( Figure 6 :S1). In this step, for example, Figure 1 The values ​​of input x and output y as shown are used as training data. It should be noted that the input x is treated as the explanatory variable and the output y is treated as the target variable. Figure 1 In , the positive or negative sign registered in correspondence with the input x is read as information indicating the constraint condition. The regression analysis device 1 uses the read sign as the above-mentioned constraint sign. It should be noted that in this embodiment, a regression equation such as that shown in equation (1) is used.

[0072] Furthermore, the regression analysis unit 142 of the regression analysis device 1 updates the regression coefficient ( Figure 6 : S2). In this step, the regression analysis unit 142 updates the coefficient w and the parameter s (specifically, the parameter "u" in equation (5)) in a manner that minimizes the cost function E(w, s) shown in equation (2). Specifically, the regression analysis unit 142 can update the coefficient w and the parameter s based on the equation of the steepest gradient method described above.

[0073] In this embodiment, the regularization term in the cost function E(w, s) is defined to increase the cost if the adaptively applied constraints are not satisfied. Specifically, the regularization term reduces the value of the cost function E(w, s) when the direction of change in the explanatory variable and the direction of change in the target variable have a likely correspondence. Furthermore, the regression analysis unit 142 sets the coefficient w to zero if it does not converge to a value that satisfies the constraints. This approach can also be used to select explanatory variables.

[0074] Furthermore, the regression analysis unit 142 of the regression analysis device 1 determines whether all coefficients w and the like have converged or become zero ( Figure 6: S3). In this step, the regression analysis unit 142 determines convergence when the gradient of the updated coefficient w and the like is sufficiently close to zero. Specifically, the regression analysis unit 142 determines convergence when the values ​​of the coefficient w and the parameter u in the equation of the steepest gradient method described above no longer change.

[0075] If it is determined that the coefficient w and the like have not converged and have not become zero (S3: No), the process returns to S2 and the process is repeated. On the other hand, if it is determined that the coefficient w and the like have converged or become zero (S3: Yes), the regression analysis unit 142 stores the regression equation in the storage device 12 ( Figure 6 : S4). In this step, the regression analysis unit 142 stores the updated coefficient w and the like in the storage device 12.

[0076] Alternatively, the verification processing unit 143 of the regression analysis device 1 may verify the accuracy of the generated regression equation ( Figure 6 : S5). In this step, the verification processing unit 143 verifies the accuracy of the regression equation using test data, for example, through cross-validation. The verification processing unit 143 can perform verification based on a predetermined evaluation index such as a correlation coefficient or a coefficient of determination. It should be noted that this step can also be omitted.

[0077] Furthermore, the operational processing unit 144 of the regression analysis device 1 performs operational processing using the generated regression equation ( Figure 6 : S6). In this step, the processing unit 144 is used, for example, Figure 1 The data number shown is t T＋1 , the predicted value of the output y for the new input x is calculated as recorded in . It should be noted that this step may also be performed by a device (not shown) other than the regression analysis device 1 using the regression equation stored in S4.

[0078] Example

[0079] The regression analysis process of this technology was applied to sensor data obtained from a chemical process unit. The number of learning data N was set to 14, and 14 sets of input x and output y were prepared. Each set consisted of 12 input x and one output y. In addition, the constructed regression equation was evaluated by leave-one-out cross validation (LOOCV). That is, N-1 of the N sets were used as training data, and the remaining 1 was used as evaluation data. The validation was repeated N times in such a way that all 14 sets of data were used as evaluation data once. Then, the coefficient of determination (R) was calculated based on the results of the N times. 2 ). Coefficient of determination (R 2 ) is an indicator of the prediction accuracy and is given by the following formula (8).

[0080] [Formula 8]

[0081]

[0082] w0(n), w1(n), ..., wK(n) are regression coefficients determined by learning when the n-th data is used as evaluation data.

[0083] For various values ​​of α, the determination coefficient R is calculated by LOOCV using this embodiment. 2 In addition, the same verification is performed by L1 regularization (Lasso) and least squares method. Figure 7 The solid line represents the coefficient of determination R of the regression equation created by this embodiment. 2 The dotted line represents the coefficient of determination R of the regression equation produced by L1 regularization. 2 The horizontal dashed line represents the coefficient of determination R obtained by the least squares method (excluding the parameter α). 2 In the case of this embodiment, when α is 0.0001 or less, a high R of 0.6 or more is obtained. 2 In the case of L1 regularization, when α is 0.0001, the same R as that of the present embodiment is obtained. 2 However, when α is greater than 0.0001 or less than 0.0001, the R obtained by L1 regularization is 2 The value will drop sharply to near zero. As can be seen from the above, this embodiment can stably maintain high prediction performance (generalization performance) compared to L1 regularization. In addition, this embodiment and L1 regularization achieve higher prediction performance than the least squares method.

[0084] It should be noted that if Figure 7 As shown, the prediction performance based on L1 regularization is good at a specific value of α (in Figure 7 In the example, it reaches its maximum at α (around 0.00011), and drops sharply when the value of α exceeds or falls below this specific value. This means that the prediction performance when using L1 regularization is sensitive to the value of α. Therefore, it can be said that adjusting the parameter α through cross-validation or other methods is essential to achieve the effects of L1 regularization.

[0085] On the other hand, the prediction performance in the case of using this embodiment is improved when α is a specific value ( Figure 7While the prediction performance is maximized at α (around 0.00011 in the example), even if the value of α is reduced to a certain value, it does not decrease as much as with L1 regularization. Therefore, even if α is set to zero a priori without cross-validation, sufficiently high prediction performance can be maintained. Cross-validation is a computationally intensive process, but according to this embodiment, by setting α to zero from the outset, sufficient prediction performance can be achieved without cross-validation.

[0086] <Second embodiment>

[0087] In this embodiment, the spin variable s is not represented by a continuous variable. k Instead of approximating the spin variable s k That is, instead of equation (4) in the first embodiment, let the spin variable s k Strictly, only +1 or -1 is taken, and which value is taken is determined randomly. The following describes the differences from the first embodiment.

[0088] In learning using the steepest descent method, a quantity determined by the following equation (9) is defined.

[0089] [Formula 9]

[0090]

[0091] Here, β (>0) represents the “inverse temperature parameter”.

[0092] In addition, the spin variable s is updated according to the following rules k (k=1, 2,…, K).

[0093] (i) Generate a uniform random number r in the range [0,1].

[0094] (ii) If ρ ≥ r, s k =1;

[0095] (iii) else if ρ <r,S k =-1.

[0096] That is, first, a uniform random number r is generated within the range of 0 to 1. Second, when the value of ρ determined by equation (9) is greater than r, the spin variable s is k Set to +1. Third, when the value of ρ determined by equation (9) is less than r, set the spin variable s to k Set to -1.

[0097] Furthermore, the error function shown in equation (2) is minimized (learned) according to the following process. kThe minimization of can be performed by the re-accelerated descent method. In each step, the regression coefficient w k Update using the following formula (10). This is the same as the usual steepest descent method.

[0098] [Formula 10]

[0099]

[0100] Here, g + (w k ) and g - (w k ) is the “subdifferential” shown in the first embodiment. Next, the spin variable s is updated based on the above rules (i) to (iii). k The regression coefficient w is obtained by equation (10). k Update of and spin variable s by rules (i) to (iii) k The update is a one-step process ( Figure 6 In step S2 in the example, learning is performed by repeatedly performing this step.

[0101] Furthermore, similarly to the first embodiment, the value of the inverse temperature parameter β is adjusted according to the “simulated annealing method”.

[0102] It can be said that the second embodiment described above can also provide a more general technique for constructing a regression model having a correspondence relationship between changes in explanatory variables and changes in target variables without predetermining constraints.

[0103] <Modification>

[0104] Figure 5 The computer configuration shown is an example and is not limited to such an example. For example, it is also possible to provide a regression analysis system in which at least a portion of the functions of the regression analysis device 1 is distributed across multiple devices to achieve the above-mentioned processing as a whole. In addition, it is also possible to provide a regression analysis system in which at least a portion of the functions of the regression analysis device 1 is performed in parallel by multiple devices. In addition, it is also possible to provide at least a portion of the functions of the regression analysis device 1 on the so-called cloud. In addition, the regression analysis device 1 may not include a portion of the components such as the verification processing unit 143. In addition, it is also possible to provide a system that only includes the application processing unit 144 and stores the created learned model.

[0105] Furthermore, the cost function shown in equation (2) is assumed to be L1 regularized on the positive or negative side, but it can also be operated based on the L2 norm or other convex functions. In other words, it is also possible to use the sum of the squares of the coefficients or other penalty terms that apply a penalty on the positive or negative side instead of the sum of the absolute values ​​of the coefficients.

[0106] The parameter S included in the regularization term k It is not limited to the spin variable, as long as it is used to make the first term "(1 / 2)(1+s k ) R + (w k )" and the second term "(1 / 2)(1-s k ) R - (w k )” can be a binary variable of zero or approximately zero. For example, parameter S k It can also be a binary variable that takes either 0 or +1. In this case, just replace R in equation (3) with ASL (w k , s k ) can be defined as follows.

[0107] [Formula 11]

[0108]

[0109] In this case, a sigmoid function can be used to approximate a continuous variable that represents a binary variable. Furthermore, in the case of binary variables that take on other two values, for example, a function representing an S-shaped growth curve can be used for approximation. However, when using a spin variable that takes on values ​​of +1 or -1, as in the above-mentioned embodiment, formula modification offers a better prospect and also improves the readability of the code used to implement this function.

[0110] Furthermore, the content of the data analyzed by the regression analysis device 1 is not particularly limited. In addition to the prediction of characteristic values ​​such as quality in the manufacturing industry of chemicals, as described in the examples, it can also be applied to process control and diagnosis in machining, as well as to non-manufacturing industries such as power generation and wastewater treatment, and various other fields. For example, it is possible to use information such as sensor data output by production equipment as explanatory variables, and to use specified characteristic values ​​representing mechanical, physical, or chemical properties, preferred operating conditions corresponding to the situation, and abnormality levels as target variables for prediction. Furthermore, in non-manufacturing industries, it is also possible to use various information as explanatory variables to predict preferred operating conditions for equipment.

[0111] Furthermore, the present disclosure includes a method or a computer program for executing the above-described processing, and a computer-readable recording medium recording the program. The recording medium recording the program can cause a computer to execute the program to perform the above-described processing.

[0112] Here, a computer-readable recording medium refers to a recording medium that can store information such as data and programs and then be read from a computer through electrical, magnetic, optical, mechanical, or chemical action. Examples of such recording media that can be removed from a computer include floppy disks, magneto-optical disks, optical disks, magnetic tapes, and memory cards. Furthermore, examples of recording media that are fixed to a computer include HDDs, SSDs (Solid State Drives), and ROMs.

[0113] The various configurations and combinations thereof in the various embodiments are merely examples, and additions, omissions, substitutions, and other modifications may be made as appropriate without departing from the spirit of this disclosure. This disclosure is not limited to the embodiments but only to the claims. Furthermore, each aspect disclosed in this specification may be combined with any other feature disclosed in this specification.

[0114] Description of Reference Numerals

[0115] 1: Regression analysis device;

[0116] 11: Communication I / F;

[0117] 12: storage device;

[0118] 13: Input / output device;

[0119] 14: Processor;

[0120] 141: data acquisition unit;

[0121] 142: Regression Analysis Department;

[0122] 143: Verification Processing Department;

[0123] 144: Operations processing department.

Claims

1. A regression analysis method, wherein a computer performs the following steps: Reading the training data from a storage device storing the training data of the target variable and the explanatory variable used as the regression model; as well as performing machine learning based on the regression model using the training data in a manner that minimizes a cost function including a regularization term, In the regression analysis method, The regularization term includes: The first term increases the cost in the interval where the coefficient of the explanatory variable is positive compared to the interval where the coefficient is negative; and a second term that increases the cost in an interval where the coefficient is negative compared to an interval where the coefficient is positive.

2. The regression analysis method according to claim 1, wherein: The first term and the second term include parameters for making either the first term or the second term zero or approximately zero, In the machine learning, the regression coefficients and the parameters in the regression model are repeatedly updated.

3. The regression analysis method according to claim 2, wherein: The parameter is a binary variable, and when the parameter takes one value, it constitutes a factor that becomes zero in the first term, and when it takes another value, it constitutes a factor that becomes zero in the second term.

4. The regression analysis method according to claim 3, wherein: The binary variables are set to values ​​that approximate the growth curve, and the regression coefficients and the parameters are calculated in such a manner as to minimize the cost function.

5. The regression analysis method according to claim 3 or 4, wherein: The binary variable is set to a value determined based on a magnitude relationship between a predetermined value represented by the regression coefficient and a random variable, and the regression coefficient and the parameter are calculated so as to minimize the cost function. 6 . A regression analysis system comprising one or more computers for executing the regression analysis method according to claim 1 . 7 . A regression analysis program for causing one or more computers to execute the regression analysis method according to claim 1 .

Images