Regression analysis method, regression analysis system, and regression analysis program

MYPI2025004243A0Undetermined Publication Date: 2024-07-19DAICEL CORP +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
MYPI2025004243
Authority / Receiving Office
MY · MY
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-01-19
Filing Date
2024-01-18
Publication Date
2024-07-19

AI Technical Summary

Technical Problem

Conventional regression analysis methods face challenges in constructing a regression model with a correspondence relationship between explanatory and objective variables, especially when the number of data samples is small, requiring time-consuming and effort-intensive preparation of constraint conditions.

Method used

A regression analysis method that uses a cost function with regularization terms to adaptively impose sign constraints on regression coefficients, ensuring the direction of variation in explanatory variables corresponds to the direction of variation in objective variables, even with limited data, by employing a binary spin variable that adjusts to minimize the cost function.

Benefits of technology

This approach enables the construction of a regression model with a predetermined correspondence relationship between explanatory and objective variables, improving prediction accuracy and stability, especially in scenarios with limited data, without the need for extensive parameter tuning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure PI2025004243_DRAWING_001
    Figure PI2025004243_DRAWING_001
  • Figure PI2025004243_DRAWING_002
    Figure PI2025004243_DRAWING_002
  • Figure PI2025004243_DRAWING_003
    Figure PI2025004243_DRAWING_003
Patent Text Reader

Abstract

A more versatile technique is provided for constructing a regression model having a correspondence relationship with variation of an explanatory variable and variation of a response variable. A regression analysis method includes: retrieving, by a computer, training data from a storage device storing the training data, the training data being used as a response variable and an explanatory variable of a regression model; and performing, by the computer, machine learning by the regression model using the training data to minimize a cost function including a regularization term. The regularization term includes a first term that increases a cost more in an interval where a coefficient is positive than in an interval where the coefficient is negative, and a second term that increases the cost more in the interval where the coefficient is negative than in the interval where the coefficient is positive. Fig. 6
Need to check novelty before this filing date? Find Prior Art

Description

Regression analysis method, regression analysis system, and regression analysis program

[0001] The present disclosure relates to a regression analysis method, a regression analysis system, and a regression analysis program.

[0002] Conventionally, when estimating parameters of a regression model using the least squares method, there has been a problem in that an appropriate least squares estimator cannot be obtained when the number of data samples is small. To address this problem, a method of imposing a constraint condition called the L1 norm has been proposed (see, for example, Non-Patent Document 1). LASSO (Least Absolute Shrinkage and Selection Operator), a parameter estimation method that uses the L1 norm as a constraint condition, simultaneously selects explanatory variables suitable for explaining the target variable and determines coefficients.

[0003] Furthermore, a regression analysis device has also been proposed that includes a data acquisition unit that reads out training data and constraint conditions from a storage device that stores training data used as the objective variable and explanatory variable of the regression model, and constraint conditions that define in advance whether the explanatory variable should be changed in a positive or negative direction in order to change the objective variable in a positive or negative direction, and a coefficient update unit that repeatedly updates the coefficients of the explanatory variables in the regression model using the training data so as to minimize a cost function that includes a regularization term that increases the cost when the constraint conditions are violated (sign constraint regularization) (for example, Patent Document 1).

[0004] International Publication No. 2021 / 157670

[0005] Robert Tibshirani, “Regression Shrinkage and Selection via the Lasso”, Journal of the Royal Statistical Society. Series B (Methodological) Vol. 58, No. 1 (1996), pp. 267-288

[0006] Previously, regression analysis devices have been proposed that can perform accurate machine learning even when there is a relatively small amount of training data by defining in advance constraints that determine whether explanatory variables should be varied positively or negatively in order to vary the objective variable in a positive or negative direction. However, preparing appropriate constraints depending on the target of regression analysis is time-consuming. Therefore, the present technology aims to provide a more versatile technology for building a regression model that has a correspondence between the variation of explanatory variables and the variation of the objective variable.

[0007] (Aspect 1) In a regression analysis method, a computer reads training data from a storage device that stores training data used as a response variable and explanatory variables of a regression model, and performs machine learning using the training data to minimize a cost function including a regularization term. The regularization term includes a first term that increases the cost in a range where the coefficient of the explanatory variable is positive compared to a range where the coefficient is negative, and a second term that increases the cost in a range where the coefficient is negative compared to a range where the coefficient is positive.

[0008] (Aspect 2) In the above aspect 1, the first term and the second term may include a parameter for making either the first term or the second term zero or approximating it to zero, and in the machine learning, the regression coefficients and the parameter in the regression model may be repeatedly updated.

[0009] (Aspect 3) In the above aspect 2, the parameter may be a binary variable, and the parameter may constitute a factor in the first term that is zero when it takes one value, and a factor in the second term that is zero when it takes the other value.

[0010] (Aspect 4) In the aspect 3, the binary variables may be values ​​that approximate a growth curve, and the regression coefficients and parameters may be calculated so as to minimize the cost function.

[0011] (Aspect 5) In the above aspect 3 or 4, the binary variable may be a value determined based on the magnitude relationship between a predetermined value expressed using a regression coefficient and a random variable, and the regression coefficient and the parameter may be calculated so as to minimize the cost function.

[0012] The regression analysis method may be applied to equipment in the manufacturing or non-manufacturing industry. The explanatory variables may be sensing data output by production equipment. The objective variables may be the operating conditions of the equipment, or a predetermined characteristic value or abnormality level. The contents described in the "Means for Solving the Problems" section may be combined as much as possible without departing from the problem and technical idea of ​​the present disclosure. Furthermore, the above-mentioned aspects 1 to 5 may be provided as a device such as a computer or a system including multiple devices, a method executed by a computer or system, or a program for causing a computer or system to execute the method. A recording medium storing the program may also be provided.

[0013] The disclosed technology can provide a more versatile technology for constructing a regression model having a correspondence between the variation of explanatory variables and the variation of response variables.

[0014] FIG. 1 is a diagram showing an example of observed values ​​used to create a regression equation. + FIG. 3 is a schematic diagram for explaining the factor "R - FIG. 4 is a schematic diagram for explaining the spin variable S k Fig. 5 is a block diagram illustrating an example of a configuration of a regression analysis device. Fig. 6 is a processing flow diagram illustrating an example of regression analysis processing. Fig. 7 is a diagram illustrating verification results.

[0015] Hereinafter, an embodiment of a regression analysis device will be described with reference to the drawings.

[0016] First Embodiment A regression analysis device according to this embodiment constructs a regression equation (regression model) that represents the relationship between one or more explanatory variables (independent variables) and one response variable (dependent variable). The process of determining the regression coefficients of the regression equation is called "learning." In this embodiment, the regression equation is constructed so that a constraint (called a "sign constraint") is adaptively imposed so that the direction of change (positive or negative) of the explanatory variables and the direction of change (positive or negative) of the response variable have a predetermined correspondence. In other words, learning is performed so that the signs of the regression coefficients become appropriate. Generally, if there is a sufficient amount of training data, the regression equation constructed using this data will converge such that the direction of change of the explanatory variables and the direction of change of the response variable have a predetermined correspondence. On the other hand, if the amount of training data is relatively small, a regression equation may be created in which the correspondence is reversed. The "sign constraint" in this embodiment is imposed so that the created regression equation has an appropriate correspondence even when the amount of training data is relatively small.

[0017] FIG. 1 is a diagram showing an example of observed values ​​used to create a regression equation. The table in FIG. 1 shows K inputs x(x 1 ~x K ) and a column of output y. The input x is used as an explanatory variable, and the output y is used as a target variable. Also, the individual sample data points t (t 1 ~t T , ...), a regression equation is created using T records (training data), and the created regression equation is applied to data point t T+1 Suppose we input the value of y and predict the unknown output y.

[0018] The regression equation is expressed, for example, by the following equation (1). The regression equation is a linear model, where the output y is expressed as a linear sum of the input x. k is the regression coefficient, w 0 are constant terms, each determined by learning.

[0019] The regression coefficients and constant term can be determined using a cost function (error function) E(w, s) expressed by the following equation (2). The cost function E(w, s) is obtained by adding a regularization term (penalty term) S(w, s) to the squared error L(w).

[0020] The regularization term S(w, s) is expressed by the following equation (3). R ASL (w k , s k ) contains two terms: R ASL (w k , s k ) is the first term "(1 / 2)(1+s k ) R + (w k ) and the second term "(1 / 2)(1-s k ) R - (w k It is expressed as the sum of s k is a spin variable, which can take two values: +1 (up) or -1 (down). k If = +1, the factor of the second term above, "(1-s k ) becomes zero, so only the first term remains, and R ASL (w k , s k ) = R + (w k ) becomes. k If = -1, the factor of the first term above, "(1 + s k ) becomes zero, so only the second term remains, R ASL (w k , s k ) = R - (w k In other words, the first and second terms are determined by the parameter s k Also, the parameter s k constitutes a factor in the first term that is zero when it takes one value, and a factor in the second term that is zero when it takes the other value, and L is a finite, sufficiently large value that is an approximation of infinity.

[0021] Figure 2 shows the factor "R + 2 is a schematic diagram for explaining the relationship between the vertical axis and the horizontal axis of the graph. +(w), and the horizontal axis represents w. The above equation (3) is k Coefficient of w k When is greater than or equal to zero, R + (w) = w, and E(w, s) increases as w increases. On the other hand, the input x k Coefficient of w k When is less than zero, R + (w) = +∞, which causes the cost to diverge to positive infinity. That is, the regularization term in this embodiment is the spin variable S k If = +1, then input x k Coefficient of w k When x is less than zero, the cost is made to diverge to infinity. For the purposes of processing by the regression analysis device, it is sufficient to set it to a sufficiently large value. On the other hand, when the input x k Coefficient of w k When is greater than or equal to zero, the cost is increased according to the magnitude of the regression coefficient w. + The first term containing (w) increases the cost in intervals where the coefficient is negative more than in intervals where the coefficient is positive. k When is greater than or equal to zero, the input x of the regression equation shown in Eq. (1) k As the spin variable S increases, the predicted value y by the regression equation also increases. k If = +1, input x k As the value of increases, the value of the predicted value y also increases, and the regularization term becomes smaller, and the input x k The cost function is defined such that the regularization term increases as the value of the predicted value y decreases with increasing values ​​of .

[0022] Figure 3 shows the factor "R - 3 is a schematic diagram for explaining the relationship between the vertical axis and the horizontal axis of the graph. - (w), and the horizontal axis represents w. The above equation (3) is k Coefficient of w k When is greater than or equal to zero, R - (w) = +∞, which causes the cost to diverge to positive infinity. For the purposes of processing by the regression analysis device, it is sufficient to set this to a sufficiently large value. On the other hand, the input x k Coefficient of w k When is less than zero, R - (w) = -w, and E(w, s) increases as w decreases. That is, R- The second term, which includes (w), increases the cost in the interval where the coefficient is positive more than in the interval where the coefficient is negative. k When is less than zero, the input x of the regression equation shown in Eq. (1) k As the spin variable S increases, the predicted value y by the regression equation decreases. k If = -1, input x k As the value of increases, the regularization term becomes smaller when the value of the predicted value y decreases, and the input x k The cost function is defined such that the regularization term increases as the value of the predicted value y increases.

[0023] That is, equation (3) is a function of the spin variable S k Depending on the value of k is non-negative or non-positive. k The optimization of equation (2) is 0 , W 1 , ..., W K} and s = {S 1 , S 2 , ..., S K}. Updating the parameters w etc. to minimize E(w, s) can be performed by, for example, a gradient method. That is, based on the gradient of the cost function E(w, s) for the variables w etc. in a certain step, the variables w etc. in a later step are updated, and this process is repeated until w etc. converges. In this way, the coefficient w k Even if the sign of the spin variable S k The value of is appropriately determined, and a sign constraint is adaptively imposed. By using the regularization term described above, it is possible to perform regression analysis by imposing a constraint that ensures a certain correspondence between the direction of variation of the explanatory variables and the direction of variation of the response variable.

[0024] Spin variable S k is a continuous variable u k (-∞<u k < +∞). The spin variable S k can be approximated by the following equation (4). Spin variable Sk is expressed as a hyperbolic tangent function. β (>0) is the "inverse temperature parameter." Figure 4 shows the spin variable S k The graph in FIG. 4 is a diagram for explaining the approximation of the spin variable S k , the horizontal axis is the continuous variable u k The dashed line graph represents β = 0.25, the dashed line graph represents β = 0.5, the two-dot chain line graph represents β = 1.0, and the solid line graph represents β = 2. When β is +∞, the spin variable S in equation (4) k takes two values, +1 or -1. When β is finite, the spin variable S k takes on values ​​continuously between -1 and +1 near the origin.

[0025] The error function shown in equation (3) is 1 , u 2 , ..., u k} can be rewritten as follows: The value of the parameter α, which indicates the strength of regularization, can be determined by cross-validation, etc. Equation (5) can be differentiated with respect to w and u, and the following equation for the gradient method (steepest gradient method) can be derived. η is the step size that determines the size of the update of the coefficient w in one step (one iteration). ± (w k ) is w k = 0 and is not differentiable. Therefore, the "subdifferential" value g + (w k ) and g - (w k ) is used, where L is a sufficiently large value.

[0026] Based on the steepest gradient method formula, k and u k Repeat the update of w k and u kThe repetition ends when the value of stops changing (minimization is achieved). At this time, the value of the inverse temperature parameter β is adjusted according to the "simulated annealing" method. That is, in the iterative calculation of the steepest gradient method, β shown in FIG. 4 is initially set to a sufficiently small value, and as the repetition progresses, the value is gradually increased, and finally the spin variable S in Equation (4) is k takes on only two values, approximately ±1. The value of β can be increased (annealing schedule) by multiplying it by a fixed number γ greater than 1 at each iteration step (β←γβ). It is known that if the value of β is increased sufficiently slowly, an optimal solution can be obtained with a probability of 1 without falling into a local minimum.

[0027] By doing the above, an appropriate sign constraint is imposed on the coefficient w, and it converges to a value that contributes to the objective variable, and if there is no such value, the coefficient w approaches zero. In other words, if there is no value that satisfies the sign constraint, the penalty effect of regularization will come into play as shown in Figures 2 and 3, and values ​​that violate the sign constraint will be pulled back, resulting in convergence to zero. Therefore, like the so-called LASSO, some of the regression coefficients can be estimated to be zero. In this way, as a solution to minimize the error function (5), s k (k=1, 2, ..., K) and w k (k=1, 2, . . . , K) is obtained.

[0028] Also, it was requested lol k By using as the regression coefficients of the regression equation (1), the new input x K+1 It is possible to predict the output y for the error function (5). k (k=1, 2, ..., K), and by the conventional sign constraint regularization, w k (k=1, 2, ..., K). That is, the spin variable S k If >0, the non-negativity constraint shown in Figure 2 is imposed, and the spin variable S k If < 0, the non-positive constraint shown in Figure 3 is imposed, and the regression coefficient w k Ask for.

[0029] <Device Configuration> FIG. 5 is a block diagram showing an example of the configuration of a regression analysis device 1 that performs the above-described regression analysis. The regression analysis device 1 is a computer and includes a communication interface (I / F) 11, a storage device 12, an input / output device 13, and a processor 14. These components are connected via a bus or the like. The communication I / F 11 may be, for example, a network card or a communication module, and communicates with other computers based on a predetermined protocol. The storage device 12 may be a main storage device such as a random access memory (RAM) or a read-only memory (ROM), or an auxiliary storage device (secondary storage device) such as a hard-disk drive (HDD), a solid-state drive (SSD), or a flash memory. The main storage device temporarily stores programs read by the processor 14 and information processed by the programs. The auxiliary storage device stores programs executed by the processor 14 and information processed by the programs. In this embodiment, the storage device 12 temporarily or permanently stores training data and information representing constraints. The input / output device 13 is a user interface, such as an input device such as a keyboard or a pointing device, an output device such as a monitor, or an input / output device such as a touch panel. The processor 14 is an arithmetic processing device such as a CPU (Central Processing Unit), and performs each process according to this embodiment by executing a program. In the example of FIG. 1, functional blocks are shown within the processor 14. That is, by executing a predetermined program, the processor 14 functions as a data acquisition unit 141, a regression analysis unit 142, a verification processing unit 143, and an operation processing unit 144.

[0030] The data acquisition unit 141 acquires information representing training data and constraint conditions from the storage device 12. The regression analysis unit 142 updates the regression coefficients as a solution to the minimization of the error function described above, and determines whether the updated regression coefficient values ​​have converged. If it is determined that they have not converged, the regression analysis unit 142 repeats updating the coefficients. If it is determined that they have converged, for example, the regression analysis unit 142 stores the finally generated coefficients in the storage device 12. Furthermore, the verification processing unit 143 evaluates the regression equation created based on a predetermined evaluation index. For example, the verification processing unit 143 calculates the coefficient of determination R for the results of cross-validation. 2 The operation processing unit 144 calculates a predicted value using the created regression equation and, for example, a newly acquired observation value (for example, the K+1th data in FIG. 1 ). The operation processing unit 144 may also calculate a predicted value when conditions are changed using the created regression equation and an arbitrary value. Here, the arbitrary value may be, for example, a value input by a user via the communication I / F 11 or the input / output device 13. For example, when the operating conditions of the plant are changed, it becomes possible to predict the process state after the change.

[0031] <Regression Analysis Processing> FIG. 6 is a processing flow diagram showing an example of regression analysis processing executed by the regression analysis device. The data acquisition unit 141 of the regression analysis device 1 reads training data and information representing constraint conditions from the storage device 12 (FIG. 6: S1). In this step, for example, values ​​of input x and output y as shown in FIG. 1 are read as training data. Note that the input x is treated as an explanatory variable, and the output y is treated as a target variable. Also, in FIG. 1, a positive or negative sign registered in association with the input x is read as information representing the constraint conditions. The regression analysis device 1 uses the read sign as the constraint sign described above. Note that in this embodiment, a regression equation such as that shown in Equation (1) is used.

[0032] Furthermore, the regression analysis unit 142 of the regression analysis device 1 updates the regression coefficients under the sign constraint described above ( FIG. 6 : S2). In this step, the regression analysis unit 142 updates the coefficient w and the parameter s (specifically, for example, the parameter “u” in equation (5)) so as to minimize the cost function E(w, s) shown in equation (2), for example. Specifically, the regression analysis unit 142 can update the coefficient w, etc. based on the steepest gradient method described above.

[0033] The regularization term of the cost function E(w, s) according to this embodiment is defined so that the cost increases when adaptively imposed constraints are not satisfied. That is, the regularization term decreases the value of the cost function E(w, s) when the direction of variation of the explanatory variables and the direction of variation of the objective variable have a plausible correspondence. Furthermore, the regression analysis unit 142 sets the coefficient w to zero if it does not converge to a value that satisfies the constraints. In this manner, explanatory variables may be selected.

[0034] The regression analysis unit 142 of the regression analysis device 1 also determines whether all of the coefficients w, etc. have converged or been set to zero ( FIG. 6 : S3). In this step, the regression analysis unit 142 determines that convergence has occurred when the gradient of the updated coefficients w, etc. has sufficiently approached zero. Specifically, the regression analysis unit 142 determines that convergence has occurred when the values ​​of the coefficient w and parameter u in the steepest gradient method equation described above no longer change.

[0035] If it is determined that the coefficients w etc. have not converged or been set to zero (S3: NO), the process returns to S2 and repeats. On the other hand, if it is determined that the coefficients w etc. have converged or been set to zero (S3: YES), the regression analysis unit 142 stores the regression equation in the storage device 12 (FIG. 6: S4). In this step, the regression analysis unit 142 stores the updated coefficients w etc. in the storage device 12.

[0036] Furthermore, the verification processing unit 143 of the regression analysis device 1 may verify the accuracy of the created regression equation (FIG. 6: S5). In this step, the verification processing unit 143 verifies the accuracy of the regression equation using test data, for example, by cross-validation. The verification processing unit 143 can perform verification based on a predetermined evaluation index, such as a correlation coefficient or a coefficient of determination. Note that this step may be omitted.

[0037] The operation processing unit 144 of the regression analysis device 1 performs operation processing using the created regression equation (FIG. 6: S6). In this step, the operation processing unit 144 performs operation processing using the data number t T+1 A predicted value of the output y for a new input x is calculated, as in the record of step S1. Note that this step may be performed by a device (not shown) other than the regression analysis device 1, using the regression equation stored in step S4.

[0038] Example: The regression analysis process according to the present technology was applied to sensing data obtained from a chemical process plant. The number of training data, N, was set to 14, and 14 sets of inputs x and outputs y were prepared. Each set included 12 inputs x and one output y. The constructed regression equation was evaluated using leave-one-out cross validation (LOOCV). That is, N-1 of the N sets were used as training data, and the remaining one was used as evaluation data. The validation was repeated N times so that all 14 sets of data were used once as evaluation data. Then, the coefficient of determination (R 2 ) is calculated. 2 ) is an index representing the accuracy of prediction, and is given by the following equation (8). w0(n), w1(n), ..., wK(n) are regression coefficients determined by learning when the nth data is used as evaluation data.

[0039] This embodiment is applied to various values ​​of α, and the coefficient of determination R 2 The same verification was performed using L1 regularization (Lasso) and the least squares method. Figure 7 shows the verification results. The solid line indicates the coefficient of determination R of the regression equation created by this embodiment. 2The dashed line indicates the coefficient of determination R of the regression equation created by L1 regularization. 2 The horizontal dashed line indicates the coefficient of determination R by the least squares method (without the parameter α). 2 In the case of this embodiment, when α is 0.0001 or less, a high R 2 In the case of L1 regularization, when α is 0.0001, the R 2 However, when α is larger or smaller than this, the R 2 The value drops sharply to near zero. From the above, it can be seen that this embodiment can stably guarantee higher prediction performance (generalization performance) than L1 regularization. Furthermore, this embodiment and L1 regularization obtained higher prediction performance than the least squares method.

[0040] As shown in Figure 7, the prediction performance with L1 regularization is maximized when α is a specific value (near 0.0001 in the example of Figure 7), and drops sharply when the value of α becomes larger or smaller than that. This means that the prediction performance when using L1 regularization is sensitive to the value of α. For this reason, tuning the parameter α by cross-validation or the like is essential to obtain the desired effect from L1 regularization.

[0041] On the other hand, when using this embodiment, the prediction performance is maximized when α is a specific value (near 0.00011 in the example of FIG. 7 ), but even if the value of α is set smaller than this, the performance does not decrease as much as in the case of L1 regularization. Therefore, even if the value of α is set to zero a priori without undergoing cross-validation, sufficiently high prediction performance can be ensured. While cross-validation is a procedure that requires a large amount of calculation, according to this embodiment, by setting the value of α to zero from the beginning, it can be said that sufficient prediction performance can be obtained without undergoing cross-validation.

[0042] Second Embodiment In this embodiment, the spin variable s k is not approximated by a continuous variable but is treated as a random variable. That is, instead of equation (4) in the first embodiment, the spin variable s kis strictly limited to the values ​​+1 or -1, and which value it will take is determined probabilistically. The differences from the first embodiment will be described below.

[0043] In learning by the steepest descent method, the quantity given by the following equation (9) is defined. Here, β (>0) represents the "inverse temperature parameter."

[0044] Also, the spin variable s k (k = 1, 2, ..., K). (i) Generate a uniform random number r in the range [0, 1]. (ii) If ρ ≥ r, s k =1; (iii) else if ρ<r, s k = -1. That is, first, a uniform random number r is generated in the range of 0 to 1. Second, when the value of ρ defined by the formula (9) is equal to or greater than r, the spin variable s k Third, when the value of ρ defined in equation (9) is less than r, the spin variable s k Let be -1.

[0045] The error function shown in equation (2) is minimized (learned) in the following manner: k The minimization of can be done by the method of steepest descent. At each step, the regression coefficient w k is updated by the following equation (10): This is the same as the normal steepest descent method. Here, g + (w k ) and g - (w k ) is the "subdifferential" shown in the first embodiment. Next, the spin variable s k is updated based on the above rules (i) to (iii). The regression coefficient w k and the spin variable s according to rules (i) to (iii) k The update of the parameter .theta. is counted as one step (step S2 in FIG. 6), and learning is performed by repeating this process.

[0046] As in the first embodiment, the value of the inverse temperature parameter β is adjusted according to the "simulated annealing method."

[0047] The second embodiment as described above also does not require any constraint conditions to be defined in advance, and can provide a more versatile technique for constructing a regression model having a correspondence between the variation in the explanatory variables and the variation in the response variable.

[0048] <Modifications> The computer configuration shown in FIG. 5 is an example, and the present invention is not limited to this example. For example, a regression analysis system may be provided in which at least some of the functions of the regression analysis apparatus 1 are distributed across multiple devices, and the above-described processing is realized as a whole. Furthermore, a regression analysis system may be provided in which multiple devices execute at least some of the functions of the regression analysis apparatus 1 in parallel. Furthermore, at least some of the functions of the regression analysis apparatus 1 may be provided on a so-called cloud. Furthermore, the regression analysis apparatus 1 may not include some components, such as the verification processing unit 143. Furthermore, a system may be provided that stores a created trained model and includes only the operation processing unit 144.

[0049] Furthermore, while the cost function shown in Equation (2) is based on L1 regularization on either the positive or negative side, it can also work with an L2 norm or other convex functions. That is, instead of the sum of the absolute values ​​of the coefficients, a sum of squares of the coefficients on either the positive or negative side or other penalty terms can be used.

[0050] The parameter S included in the regularization term k is not limited to the spin variable, but includes the first term "(1 / 2)(1+s k ) R + (w k ) and the second term "(1 / 2)(1-s k ) R - (w k Any binary variable may be used to set one of the parameters S to zero or to approximate zero. For example, k may be a binary variable that takes either 0 or +1. In this case, R in Equation (3) ASL (w k , s k ) can be defined as follows: In this case, a sigmoid function can be used as a continuous variable to approximate the binary variable. Other binary variables that take two values ​​can also be approximated using a function that represents an S-shaped growth curve, for example. However, using a spin variable that takes +1 or -1, as in the above-described embodiment, provides better predictability in formula transformation and improves the readability of the code for implementing this function.

[0051] Furthermore, the content of data analyzed by the regression analysis device 1 is not particularly limited. In addition to predicting characteristic values ​​such as quality in the manufacturing industry that produces chemicals and other products as described in the embodiments, the regression analysis device 1 may be applied to process control and diagnosis in processing and other fields, as well as to non-manufacturing industries such as power generation and wastewater treatment, and various other fields. For example, it is possible to predict, as objective variables, predetermined characteristic values ​​representing mechanical, physical, or chemical properties, preferable operating conditions depending on the situation, the degree of abnormality, etc., using information including sensing data output by production equipment as explanatory variables. Furthermore, in non-manufacturing industries, it is also possible to predict preferable operating conditions of equipment using various information as explanatory variables.

[0052] The present disclosure also includes a method and a computer program for executing the above-described process, and a computer-readable recording medium having the program recorded thereon. The recording medium having the program recorded thereon enables the above-described process by causing a computer to execute the program.

[0053] Here, a computer-readable recording medium refers to a recording medium that stores information such as data and programs electrically, magnetically, optically, mechanically, or chemically and can be read by a computer. Among such recording media, those that can be removed from a computer include flexible disks, magneto-optical disks, optical disks, magnetic tapes, memory cards, etc. Furthermore, recording media that are fixed to a computer include HDDs, SSDs (Solid State Drives), ROMs, etc.

[0054] The configurations and combinations thereof in each embodiment are merely examples, and additions, omissions, substitutions, and other modifications of the configurations are possible as appropriate without departing from the spirit of the present disclosure. The present disclosure is not limited by the embodiments, but is limited only by the scope of the claims. Furthermore, each aspect disclosed in this specification can be combined with any other feature disclosed in this specification.

[0055] 1: Regression analysis device 11: Communication I / F 12: Storage device 13: Input / output device 14: Processor 141: Data acquisition unit 142: Regression analysis unit 143: Verification processing unit 144: Operation processing unit

Claims

1. A regression analysis method in which a computer reads out training data used as a response variable and explanatory variables of a regression model from a storage device that stores the training data, and performs machine learning using the regression model using the training data so as to minimize a cost function including a regularization term, wherein the regularization term includes a first term that increases the cost in a range in which the coefficient of the explanatory variable is positive compared to a range in which the coefficient is negative, and a second term that increases the cost in a range in which the coefficient is negative compared to a range in which the coefficient is positive.

2. The regression analysis method according to claim 1, wherein the first term and the second term include a parameter for making either the first term or the second term zero or approximating it to zero, and in the machine learning, the regression coefficients in the regression model and the parameter are repeatedly updated.

3. The regression analysis method according to claim 2, wherein the parameter is a binary variable, and the parameter constitutes a factor in the first term that is zero when it takes one value, and constitutes a factor in the second term that is zero when it takes the other value.

4. The regression analysis method according to claim 3, wherein the regression coefficients and the parameters are calculated so as to minimize the cost function, with the binary variables being values ​​approximating a growth curve.

5. The regression analysis method according to claim 3 or 4, wherein the regression coefficients and the parameters are calculated so as to minimize the cost function, with the binary variable being a value determined based on the magnitude relationship between a predetermined value expressed using the regression coefficients and a random variable.

6. A regression analysis system comprising one or more computers that executes the regression analysis method according to any one of claims 1 to 5.

7. A regression analysis program for causing one or more computers to execute the regression analysis method according to any one of claims 1 to 5.