Information processing apparatus, information processing method, and program

Abstract

To manage a regression model more efficiently.SOLUTION: An information processing apparatus includes a processing unit. The processing unit is configured to: detect whether one or more conditions defining timings to perform learning of a regression model configured to predict one or more objective variables for a plurality of explanatory variables are satisfied; determine priorities of the plurality of explanatory variables according to a condition detected to be satisfied; and perform learning of the regression model by using an objective function and learning data, the objective function including a regularization term having a regularization strength changing according to the priorities.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] An embodiment of the present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Factories (such as semiconductor factories) and plants (such as chemical plants) are using big data related to manufacturing to monitor quality characteristics, identify trends or abnormalities, and consider countermeasures, thereby improving productivity, yield, and reliability.

[0003] For example, a regression model is used as an analysis method for big data. A regression model is a model in which process data such as a sensor value, a control value, and a set value are used as explanatory variables, and quality characteristics are used as objective variables. In factories and plants, the regression model may need to be updated in response to events such as equipment maintenance and changes in trends in process data, etc., and it is therefore desirable to efficiently manage the regression model. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6480022 [Patent Document 2] Patent No. 7135025 [Non-patent literature]

[0005] [Non-Patent Document 1] Tibshirani, R., & Friedman, J. (2020). “A pliable Lasso.” Journal of Computational and Graphical Statistics, 29(1), 215-225. [Non-Patent Document 2] Tibshirani , R. ( 1996 ). Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1), 267-288.

Outdoor Tools3

Outdoor Tools 4

Direct Environment 5

Outdoor Configuration 6

Direct Environment 7

[0006] An object of the present invention is to provide an information processing device, an information processing method, and a program capable of managing regression models more efficiently. [Means for solving the problem]

[0007] An information processing device according to an embodiment includes a processor. The processor detects whether one or more conditions that define a timing for learning a regression model that predicts one or more objective variables for a plurality of explanatory variables are satisfied. The processor determines a priority of each of the plurality of explanatory variables according to the condition that is detected to be satisfied. The processor trains the regression model using an objective function including a regularization term whose strength of regularization changes according to the priority and learning data. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram of an information processing apparatus according to an embodiment. [Diagram 2] FIG. 4 is a diagram showing an example of a data structure of event information. [Diagram 3] FIG. 4 is a diagram showing an example of a data structure of correspondence information. [Figure 4] FIG. 4 is a diagram showing an example of a data structure of sensor data. [Diagram 5]FIG. 4 is a diagram for explaining an example of measurement points. [Figure 6] FIG. [Figure 7] 4 is a flowchart of a learning process in the embodiment. [Figure 8] FIG. 4 is a diagram showing an example of a data structure of sensor data. [Figure 9] FIG. 4 is a diagram showing an example of a data structure of event information. [Figure 10] FIG. 4 is a diagram showing an example of a data structure of correspondence information. [Figure 11] FIG. 2 is a hardware configuration diagram of the information processing apparatus according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of an information processing apparatus according to the present invention will be described in detail with reference to the accompanying drawings.

[0010] The information processing device of the present embodiment can be applied to, for example, management (including construction, updating, learning, etc.) of models used in systems that perform quality control in factories and plants. Applicable systems are not limited to these.

[0011] In factories and plants, regression models are used for improving productivity, yield, and reliability, for example, in the following four applications: (1) Defect cause analysis: It is possible to understand the effect that processes have on quality characteristics and identify the causes of yield loss and quality variation. (2) Soft sensors: Quality characteristics that are difficult to measure or cannot be physically measured can be inferred from process data. (3) Control and Adjustment: A particular process can be controlled and adjusted to result in desired quality attributes. (4) Detection of anomalies and changes: It is possible to detect anomalies and changes by monitoring the prediction error (prediction residual) of the regression model or changes in the regression model itself.

[0012] A regression model is a model that predicts one or more objective variables for multiple explanatory variables. The regression model may be any model that is a parametric regression model, such as a linear regression model, a logistic regression model, a Poisson regression model, a generalized linear regression model, a spline regression model, a generalized additive model, or a neural network. Here, prediction refers to estimating an objective variable from an explanatory variable, and is not necessarily limited to forecasting the future, but may also be estimating past values.

[0013] The objective variable is, for example, a quality characteristic, a defect rate, and a variable indicating whether the product is good or defective. The objective variable may be a sensor value detected by a sensor. The explanatory variable is other sensor values, a set value, a control value, and the like. The explanatory variable may be pre-processed in advance. The pre-processing may be, for example, standardization, scaling, conversion by a specific function, addition of an interaction term, time lag, time lead, dummy variable conversion, encoding, outlier processing, and missing value processing.

[0014] Data (such as process data) including the objective variables and explanatory variables is stored in a data management system, a database, or the like.

[0015] First, we will explain the case where there is one objective variable in the regression model. Suppose there is a total of n data (n is an integer equal to or greater than 2), and each data contains p explanatory variables and one objective variable. In other words, the data is (x i ,y i ), x i ∈R p , x i ∈R, i = 1, , n, where x i is a p-dimensional vector of explanatory variables, y i is a scalar response variable, and R is the set of real numbers.

[0016] In this case, the regression models β0^∈R, β^∈R pcan be estimated by the least squares method as shown in the following equation (1). Note that variables marked with a hat symbol "^" indicate that they are estimated values. The same applies to the following variables. The symbol "T" also indicates transposition.

number

[0017] Furthermore, when p>>n, the regression models β0^ and β^ can be estimated by minimizing the squared error (an example of a loss function) with L1 regularization, as shown in the following equation (2), where λ represents the regularization parameter.

number

[0018] The method shown in equation (2) is called Lasso (see, for example, Non-Patent Document 2), in which many regression coefficients are estimated to be zero to obtain sparse regression coefficients. This can improve the generalization performance in high-dimensional data and the interpretability of the model.

[0019] The loss function is not limited to the above-mentioned function representing the squared error, and may be any other loss function. Moreover, the regularization is not limited to the L1 regularization, and may be any other regularization.

[0020] In quality control of factories and plants, the regression model is not something that is completed once it is constructed. When a predetermined condition is met, the regression model needs to be reconstructed (updated). The condition may be, for example, one or more conditions that determine the timing for learning the regression model, such as the following conditions: - Occurrence of an event (equipment maintenance, change of processing conditions, etc.) Changes in process data trends (increasing values, decreasing values, increasing variability, etc.) -Changes in the trend of prediction results from regression models (e.g., prediction errors become larger)

[0021] One method for updating a regression model is to collect new data and estimate the regression model from scratch, but this method can have problems such as an increased computational load, large changes in the results of variable selection, unstable changes in regression coefficients, and an increased load for checking the regression model.

[0022] Therefore, a method of updating the model only where necessary (e.g., Non-Patent Document 3, hereinafter referred to as Method MA) can be used. Method MA uses the regression model β before updating ~ ∈R p This method estimates and updates the regression model using the regularization parameter α as shown in the following equation (3).

number

[0023] This formulation makes it possible to realize an update method that updates only the minimum necessary parameters and does not update other parameters.

[0024] When using the MA method, the updated parameters are automatically determined from the data. Therefore, when the data contains a lot of noise, the data has strong collinearity, or the data is high-dimensional, inappropriate parameters may be updated.

[0025] For example, when a specific machine undergoes maintenance, only the parameters (regression coefficients) related to that machine should be updated, but parameters not related to that machine may be updated. Also, when specific processing conditions are changed, it is desirable to update only the parameters related to the processing conditions, but other parameters may be updated.

[0026] Therefore, in this embodiment, when a condition (such as the occurrence of an event) that determines the timing for learning (updating) the regression model is satisfied, parameters (corresponding to regression coefficients and explanatory variables) that need to be updated according to the condition are identified, and the identified parameters are updated preferentially. This makes it possible to more appropriately identify and update only the necessary parts (parameters). In other words, the regression model can be managed more efficiently.

[0027] More specifically, in this embodiment, the priority of parameters (explanatory variables) is changed according to the conditions, and the regression model is updated based on the priority. This enables stable and efficient management (operation) of the regression model while realizing updating of the regression model that matches the domain knowledge. In the following, an example using a condition indicating the occurrence of an event will be mainly described, but the same method can be applied to conditions other than the occurrence of an event.

[0028] This embodiment can be applied not only to updating an already constructed regression model but also to constructing a new regression model. Hereinafter, learning of a regression model includes both updating an already constructed (trained) regression model and constructing a new regression model.

[0029] Fig. 1 is a block diagram showing an example of a configuration of an information processing device 100 according to an embodiment. As shown in Fig. 1, the information processing device 100 includes a model storage unit 121, an event storage unit 122, a corresponding information storage unit 123, a sensor data storage unit 124, a generated data storage unit 125, a model acquisition unit 101, an event detection unit 102 (an example of a detection unit), a priority determination unit 103 (an example of a determination unit), a data generation unit 104, a model learning unit 105 (an example of a learning unit), a prediction unit 111, and an output control unit 112.

[0030] The model storage unit 121 stores information about a regression model such as parameters of the regression model (regression coefficients, etc.) For example, the model storage unit 121 stores information about regression models that have been learned (constructed or updated) in the past.

[0031] The event storage unit 122 stores event information indicating an event that has occurred. FIG. 2 is a diagram showing an example of a data structure of event information. As shown in FIG. 2, the event information includes a date and time, an event, and an event type. The date and time indicates the date and time (year, month, day, time, etc.) when the event occurred. The event indicates the event that has occurred. The event type indicates the type of event. The event type is set in advance by, for example, an administrator. In FIG. 2, for example, an event type of "related to processing step 1" is associated with an event of "change in processing conditions for processing step 1."

[0032] One event may be associated with multiple event types. Similarly, one event type may be associated with multiple events. That is, events and event types may have one-to-one, one-to-many, many-to-one, and many-to-many relationships.

[0033] Returning to Fig. 1, the correspondence information storage unit 123 stores correspondence information that the priority determination unit 103 refers to when determining the priority. The correspondence information is information that associates a condition, an explanatory variable, and a priority. The correspondence information may be expressed in any structure as long as these are associated with each other.

[0034] Fig. 3 is a diagram showing an example of a data structure of correspondence information. Fig. 3 shows an example in which the correspondence information is managed by dividing it into two pieces, correspondence information 123a and correspondence information 123b. The correspondence information may be integrated into one piece and managed without dividing it into two pieces.

[0035] 3, the correspondence information 123a includes an event type, a variable type, a selection priority, and an update priority. In this example, the priority includes two priorities: a selection priority and an update priority. The selection priority represents the priority of selecting an explanatory variable. The update priority represents the priority of updating an explanatory variable.

[0036] The selection priority corresponds to the priority for the second term on the right side of equation (3) for the MA method. The second term is the priority for the p explanatory variables x ij(1≦j≦p) The p regression coefficients β to be estimated j The update priority corresponds to the priority for the third term on the right side of equation (3), for example. The third term is the p regression coefficients β j and the p regression coefficients β ~ j This is because it is expressed by the difference between and.

[0037] The priority may be either the selection priority or the update priority. For example, when formula (2) is used instead of formula (3), only the selection priority corresponding to the second term on the right side of formula (2) may be used.

[0038] In the example of the correspondence information 123a in FIG. 3, the event type “processing step 1 related” and the variable type “processing step 1 related” are associated with each other, and the combination is associated with a selection priority of “0.5 (high)” and an update priority of “0.5 (high)”.

[0039] Each priority may be expressed in any manner, including a number of levels such as "high, medium, low," or a large or small numerical value.

[0040] 3, the correspondence information 123b includes a variable and a variable type. The variable represents one of the explanatory variables. The variable type of each variable (explanatory variable) is set in advance by, for example, an administrator. In this example, a variable type "related to processing step 1" is associated with a variable "sensor 1."

[0041] 3, it is possible to determine, for example, explanatory variables corresponding to the event type of a detected event and priorities (selection priority, update priority). Note that the correspondence information may be interpreted as including the event information in the event storage unit 122. This is because the event type corresponding to an event is specified by the event information.

[0042] The event types and variable types in the correspondence information 123a may have one-to-one, one-to-many, many-to-one, or many-to-many relationships. Similarly, the variables and variable types in the correspondence information 123b may have one-to-one, one-to-many, many-to-one, or many-to-many relationships.

[0043] The correspondence information shown in FIG. 3 is an example and is not limited thereto. For example, the event type and the variable type may be the same as the event and the variable, respectively. In this case, one correspondence information in which the correspondence information 123a and the correspondence information 123b are integrated, for example, correspondence information in which the event, the variable (explanatory variable), and the priority are associated with each other, may be used.

[0044] Also, correspondence information that does not explicitly define priority may be used. For example, correspondence information that associates event types with variable types may be used instead of the correspondence information 123a. Or, correspondence information that associates events with variables may be used instead of the correspondence information 123a and 123b. In this case, different priorities may be determined depending on whether they are described in the correspondence information. For example, a high (or low) priority may be determined for variables described in the correspondence information, and a low (or high) priority may be determined for variables not described in the correspondence information.

[0045] 1, the sensor data storage unit 124 stores a plurality of pieces of sensor data detected by various sensors. The sensor data can also be interpreted as data indicating the history of production in a factory, a plant, or the like.

[0046] For example, the sensor data includes one sensor data corresponding to the objective variable and multiple sensor data corresponding to the explanatory variables. As described above, the explanatory variables are not limited to the sensor data (sensor values) and may be setting values ​​and control values. In the following, an example in which the sensor data is used as the explanatory variables will be mainly described.

[0047] Fig. 4 is a diagram showing an example of a data structure of sensor data. In the example of Fig. 4, Y represents sensor data corresponding to the objective variable. Sensor 1 is sensor data measured by a sensor identified as "sensor 1" and represents sensor data corresponding to the explanatory variable. Although Fig. 4 shows only sensor data from one sensor, sensor data from two or more sensors may be measured.

[0048] Measurement points W1 and W2 are information indicating the positions corresponding to the sensor data. In the example of Fig. 4, a value of "1" indicates that the corresponding sensor data (sensor 1) was measured at the measurement point, and a value of "0" indicates that the corresponding sensor data (sensor 1) was not measured at the measurement point.

[0049] FIG. 5 is a diagram for explaining an example of measurement points. FIG. 5 shows an example of measuring the size (trench dimension) of a trench at multiple measurement points when processing a trench (groove or hole) structure in a semiconductor factory. FIG. 5 also shows a diagram of the cross section of the trench as observed from the side. Wk represents the depth at which the width of the trench is measured.

[0050] 4, a value obtained by multiplying the value of sensor 1 by the value of measurement point W1 is set as sensor 1×measurement point W1. Similarly, a value obtained by multiplying the value of sensor 1 by the value of measurement point W2 is set as sensor 1×measurement point W2. In this embodiment, the value obtained by such multiplication is used as an explanatory variable when learning the regression model.

[0051] 4 is an example, and the data structure of the sensor data is not limited to this. For example, the sensor data may have a data structure in which the sensor data measured at each measurement point is set in a column corresponding to each measurement point.

[0052] Returning to FIG. 1, the generated data storage unit 125 stores the learning data generated by the data generation unit 104. The learning data is data used for learning a regression model. For example, the learning data is data including a plurality of explanatory variables and one objective variable. Therefore, the sensor data shown in FIG. 4 can be used as the learning data.

[0053] Each memory unit (model memory unit 121, event memory unit 122, corresponding information memory unit 123, sensor data memory unit 124, generated data memory unit 125) can be configured using any commonly used storage medium such as a flash memory, a memory card, a RAM (Random Access Memory), a HDD (Hard Disk Drive), and an optical disk.

[0054] Each storage unit may be realized as a physically different storage medium, or as different storage areas of the same physically same storage medium.Furthermore, each storage unit may be realized by a plurality of physically different storage media.

[0055] The model acquisition unit 101 acquires, from the model storage unit 121, information on the regression model to be learned (constructed and updated).

[0056] The event detection unit 102 detects whether one or more conditions that determine the timing for learning the regression model are satisfied. For example, the event detection unit 102 detects whether a predetermined event has occurred. Any detection method may be used, but for example, the following method may be used. Detects input of information indicating the occurrence of an event by a user, etc. - Detect notifications of event occurrences from other systems. - Queries other systems to see if an event has occurred, and detects it based on the response to the query.

[0057] For example, the event detection unit 102 may detect the occurrence of an event when event information is stored in the event storage unit 122 by another system, etc. The event detection unit 102 may read out the event information stored in the event storage unit 122 periodically or when specified, and detect a predetermined event from the read out event information.

[0058] The priority determination unit 103 determines the priority of each of the multiple explanatory variables according to the condition that is detected to be satisfied. For example, when an event is detected by the event detection unit 102, the priority determination unit 103 determines explanatory variables related to the event and the priority of the explanatory variables according to the detected event.

[0059] The priority determination unit 103 can determine the priority of explanatory variables using the event information stored in the event storage unit 122 and the correspondence information (correspondence information 123a, 123b) stored in the correspondence information storage unit 123. First, the priority determination unit 103 refers to the event information and extracts an event type corresponding to the detected event. The priority determination unit 103 refers to the correspondence information 123a and extracts a variable type and priority (selection priority, update priority) corresponding to the extracted event type. The priority determination unit 103 refers to the correspondence information 123b and extracts variables (explanatory variables) corresponding to the extracted variable type. The priority determination unit 103 determines a priority corresponding to the variable type of the extracted variable for the extracted variable.

[0060] In this way, the priority determination unit 103 uses correspondence information that associates conditions (events), explanatory variables, and priorities to determine a priority corresponding to the detected condition (event) for the explanatory variables corresponding to the detected condition (event).

[0061] The priority determination unit 103 may further determine the priority by considering factors other than the detected event. For example, the priority determination unit 103 obtains a change in each of a plurality of explanatory variables between the learning data and the test data. The change in the explanatory variables can also be interpreted as a change in the distribution of the explanatory variables. The priority determination unit 103 determines the level of the priority depending on the magnitude of the obtained change. For example, the priority determination unit 103 determines the priority of an explanatory variable whose obtained change is larger than other explanatory variables to a value larger than the priorities of the other explanatory variables. The priority determination unit 103 may determine the priority of an explanatory variable whose obtained change is larger than other explanatory variables to a value smaller than the priorities of the other explanatory variables.

[0062] The test data is data that is used as input for prediction using a trained regression model. The test data is, for example, sensor data different from the training data (such as sensor data detected after training using the training data).

[0063] The data generating unit 104 generates learning data. For example, the data generating unit 104 generates, as learning data, a part of the sensor data stored in the sensor data storage unit 124. When the sensor data is used as the learning data as it is, the generated data storage unit 125 and the data generating unit 104 may not be provided.

[0064] The data generating unit 104 may generate learning data taking priority into consideration. For example, the data generating unit 104 generates learning data including one or more explanatory variables having a higher priority than other explanatory variables among a plurality of explanatory variables and a target variable. For the priority, for example, a value determined by the priority determining unit 103 before generating learning data to be used in the next learning can be referenced.

[0065] The model learning unit 105 learns a regression model using the learning data. When the learning data is generated by the data generating unit 104, the model learning unit 105 learns the regression model using the generated learning data. In addition, the model learning unit 105 learns the regression model so as to optimize an objective function including a regularization term (penalty term) whose regularization strength changes according to the priority. The model learning unit 105 stores information about the learned regression model in the model storage unit 121.

[0066] The details of the regression model learning performed by the model learning unit 105 will be described below. i ,y i ), x i ∈R p , x i ∈R, i = 1, , n. Also, let β be a previously trained regression model. ~ ∈R p The weight based on the selection priority and the weight based on the update priority are denoted as u j , v j (j = 1, , p).

[0067] When the selection priority and the update priority are expressed as a numerical value, the model learning unit 105 uses the numerical value itself or a value calculated using the numerical value as the weight u j and the weight v j When the selection priority and the update priority are not expressed as numerical values, the model learning unit 105 converts the selection priority and the update priority into numerical values ​​and uses the converted values ​​as the weights u j and the weight v j The weight is a real number equal to or greater than 0, and the larger the weight, the lower the priority.

[0068] FIG. 6 is a diagram showing an example of the weight. In the example of FIG. 6, the weight u j is set to one of the following values: 1 / 4, 1 / 2, 1, 2, or 4. These values ​​correspond to the explanatory variable being "almost always selected", "actively selected", "business as usual", "actively not selected", or "almost always not selected", respectively.

[0069] Similarly, the weight v based on the update priority j is set to one of the following values: 1 / 4, 1 / 2, 1, 2, or 4. These values ​​correspond to the explanatory variable being "almost always changing", "positively changing", "business as usual", "positively constant", or "almost always constant", respectively.

[0070] The model learning unit 105 uses the learning data (x i ,y i ), regression model β ~ , weight u j , and the weight v j The regression model is trained using the above. In this embodiment, a method in which a weight based on the priority is added to the method MA (hereinafter, referred to as method MB) is adopted. The method MB is expressed by the following formula (4).

number

[0071] In method MB, the weight u based on the selection priority is smaller than that in method MA. j and the weight v based on the update priority j are added to the regularization terms. These weights adjust the priorities for variable selection and variable updates.

[0072] Thus, in scheme MB, the objective function includes the following three terms: The term that evaluates the compatibility between the prediction result and the correct data: the first term on the right side of equation (4) Regularization term (first regularization term) obtained by multiplying the term for regularizing the parameters corresponding to the explanatory variables by the weight based on the selection priority: the second term on the right-hand side of equation (4) Regularization term (second regularization term) that regularizes the change in the parameter before the update by multiplying it by a weight based on the update priority: the third term on the right side of equation (4)

[0073] Note that for all explanatory variables (all j), u j = 1, v j When = 1, scheme MB coincides with scheme MA.

[0074] u j When the value of v is large, there is a large penalty for non-zero regression coefficients, which discourages the selection of variables. j When the value of is large, the regression coefficient is ~ This imposes a large penalty on updating the regression coefficients from

[0075] Since the priority is adjusted according to the detected event (condition), it is possible to select and update appropriate explanatory variables for the event. Since only the necessary explanatory variables are selected or updated, for example, the access load to each storage unit, the network load, or the calculation processing load can be reduced. Furthermore, even when there is a lot of noise, when the data has high collinearity, or when the number of data is small, the model can be stably updated. Furthermore, since the model update is consistent with the event information, the management cost required for the model update can be reduced.

[0076] When building a regression model for the first time, use the trained regression model β ~ Since there is no such model, the model learning unit 105 uses a trained regression model β ~ It is also possible to train a regression model using a method that does not use

[0077] The prediction unit 111 executes a prediction process using the trained regression model. For example, the prediction unit 111 executes a prediction process for predicting a response variable for test data using the trained regression model.

[0078] The prediction unit 111 may use the prediction result (predicted value) of the regression model to estimate whether an object related to the test data is in a specific state (e.g., abnormality). The object is, for example, a specific facility of a factory or plant. The prediction unit 111 may detect an abnormality in the object based on a prediction error of a dependent variable predicted using the regression model. For example, when the prediction error becomes larger than a threshold value, the prediction unit 111 estimates that an abnormality has occurred in the object.

[0079] The prediction process may be executed by a device (prediction device) external to the information processing device 100. In this case, the information processing device 100 does not need to include the prediction unit 111.

[0080] The output control unit 112 controls the output of various information used in the information processing device 100. For example, the output control unit 112 outputs the result of the prediction process by the prediction unit 111. Any output method may be used, but for example, a method of transmitting the result to an external device via a network and a method of displaying the result on a display device such as a liquid crystal display can be applied.

[0081] At least a part of each of the above units (model acquisition unit 101, event detection unit 102, priority determination unit 103, data generation unit 104, model learning unit 105, prediction unit 111, and output control unit 112) may be realized by one processing unit. Each of the above units is realized, for example, by one or more processors. For example, each of the above units may be realized by having a processor such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit) execute a program, that is, by software. Each of the above units may be realized by a processor such as a dedicated IC (Integrated Circuit), that is, by hardware. Each of the above units may be realized by using a combination of software and hardware. When multiple processors are used, each processor may realize one of the units, or two or more of the units.

[0082] The information processing device 100 may be physically configured as one device, or may be physically configured as multiple devices. For example, the information processing device 100 may be built on a cloud environment.

[0083] Next, a description will be given of a learning process performed by the information processing device 100 according to the embodiment. Fig. 7 is a flowchart showing an example of the learning process according to the embodiment.

[0084] When the event detection unit 102 detects an event, a learning process is started (step S101). The priority determination unit 103 determines the priority of explanatory variables according to the detected event using the corresponding information (step S102).

[0085] The data generating unit 104 generates learning data used for learning the regression model (step S103). The model acquiring unit 101 acquires information on the learned regression model, for example, from the model storage unit 121 (step S104). The model learning unit 105 learns parameters of the regression model using the learning data generated in step S103 so as to minimize the objective function using the priority determined in step S102 (step S105).

[0086] In this way, in the information processing device of the embodiment, when a condition that determines the timing for learning a regression model is satisfied, the priority of explanatory variables is determined according to the condition, and the regression model is learned using an objective function that includes a regularization term according to the priority, thereby making it possible to manage the regression model more efficiently.

[0087] (Modification) Factories and plants may require many regression models. For example, in semiconductor factories, not just one product but various products are produced on the same line, and the trends differ for each manufactured product (variety, model number). In addition, there are various quality characteristics for each product. Examples of quality characteristics in semiconductors that are used as objective variables in regression models include electrical characteristic values ​​and trench dimensions. Furthermore, the trends of these quality characteristics differ depending on where they are measured: in the equipment, chamber, wafer, or chip. For this reason, a regression model that represents each quality characteristic is required.

[0088] Therefore, a technology has been proposed for constructing an integrated model that integrates multiple regression models. Such a technology makes it possible to estimate the integrated model using multiple data. However, in conventional technology, there is no means for updating the constructed model, so when an update is required, it is necessary to reconstruct the integrated model. The method of reconstructing the integrated model may cause problems such as the integrated model changing unstably, the construction process (reconstruction process) taking a long time, and the burden on workers involved in construction or verification increasing.

[0089] An integrated model that integrates multiple regression models can be interpreted as a model that predicts multiple objective variables. Therefore, in the modified example, an example of managing regression models that predict multiple objective variables will be described. Basically, the regression model can be learned (constructed and updated) in the same manner as in the above embodiment.

[0090] Let us explain an example of multiple regression models. If there are K dependent variables (K is an integer equal to or greater than 2), the data including the dependent variables and explanatory variables is (x i ,y i ), x i ∈R p , y i ∈R K , i = 1, . . . , n. The regression model is estimated as shown in equation (5) below.

number

[0091] There are K regression models, and the number of regression coefficients (number of parameters) including the intercept is (p+1) × K. In this way, if multiple regression models are constructed individually, the number of regression coefficients will become enormous, making the construction and management of the models cumbersome.

[0092] Therefore, by introducing sparsity, the number of parameters is reduced, thereby improving and stabilizing generalization performance.

[0093] For example, if there are K types of objective variables, the K-dimensional dummy variable z representing the type of the i-th data i ∈{0,1} K This dummy variable is a dummy variable called one-hot encoding of categorical variables, but K-1 dimensional dummy variable encoding or other encoding methods may be used.

[0094] A categorical variable is a variable that can take multiple levels (values) for a given category. Any combination of categories and levels is acceptable, but the following combinations can be used: (A1) Category: Measurement position in the trench, Level: Depth (in the example of Figure 5, levels are W1 to W44) (A2) Category: Measurement position within the wafer, level: 3 levels: top, center, bottom (A3) Category: Measurement object, Level: Equipment, chamber, etc.

[0095] The dummy variables are generated, for example, as follows. - Each level is made into a dummy variable for one of several categories. For combinations of multiple categories, each level is converted into a dummy variable. -Create dummy variables for each of the multiple categories.

[0096] In this case, if we assume that there is a common regression coefficient for each type and a different intercept for each type, the regression model is expressed by the following equation (6).

number

[0097] Here, γ T ∈R K and the kth element γ k represents the intercept when the type of the objective variable is k. This results in the number of parameters being (1+p+K). Furthermore, if each type has a different regression coefficient, the regression model can be considered as expressed by the following equation (7).

number

[0098] In this case, the number of parameters is (1+p+K+pK). Although the number of parameters will be large if left as is, the number of effective parameters can be reduced by adding appropriate regularization and estimating many parameters to zero, as shown in the following formula (8) (for example, Non-Patent Document 1).

number

[0099] Here, the first term of the objective function corresponding to the right-hand side of equation (8) is the squared error, the second term is a term (regularization term) that makes the common regression coefficient and individual regression coefficients sparsity for each variable j, and the third term is a term (regularization term) that makes each individual regression coefficient sparsity.

[0100] In particular, the second term is "β j γ is nonzero if and only if jk can be nonzero (γ jk is nonzero, but β jThe formulation is one that restricts the number of effective (non-zero) parameters to be smaller, improving generalization performance, stabilization, and interpretability.

[0101] This type of method is also called multi-task learning since there are multiple objective variables, and various other methods have been proposed in addition to those mentioned above (for example, Patent Document 1, Non-Patent Documents 4 to 7).

[0102] In addition, such a method does not use the original training data (x ij ), we use dummy variables to create new training data (x ij ,z ik ,x ij z ik ) and estimating an integrated regression model using sparse regularization on the expanded training data.

[0103] For example, z i1 is a variable that expresses whether the objective variable is the quality characteristic of chamber 1 as 1 or 0, and x i1 Let x be the variable of sensor 1. In this case, i1 z i1 This new variable represents "sensor 1 x chamber 1", and if the chamber is not 1, the value is 0, and if the chamber is 1, the value represents sensor 1 itself.

[0104] In other words, by expanding the data, it is possible to build an integrated model that integrates multiple regression models. When implementing the algorithm, if you expand the training data as a preprocessing step, you can use general methods and packages for regularized optimization problems.

[0105] Fig. 8 is a diagram showing an example of the data structure of sensor data used in the modified example Fig. 8 shows an example of sensor data expanded by the above-mentioned method.

[0106] Y represents sensor data (such as quality characteristics) that corresponds to the objective variable. Sensor 1 is sensor data measured by a sensor identified by "Sensor 1." "Chamber 1" and "Chamber 2" correspond to dummy variables that represent the type of objective variable Y. For example, when the value of "Chamber 1" is 1, it indicates that a semiconductor product was manufactured in a chamber corresponding to "Chamber 1." When the value of "Chamber 2" is 1, it indicates that a semiconductor product was manufactured in a chamber corresponding to "Chamber 2." In other words, this example can be interpreted as an example of data for an integrated model for predicting two quality characteristics of products manufactured in two different chambers. Note that in this integrated model, the two quality characteristics are represented by one objective variable Y.

[0107] The data expansion is performed by, for example, the data generating unit 104. That is, the data generating unit 104 calculates a plurality of explanatory variables by multiplying one or more variables VA (first variables) by a plurality of dummy variables corresponding to a plurality of objective variables, respectively. In the example of FIG. 8, for example, the sensor data measured by "sensor 1" corresponds to the variable VA. Also, "chamber 1" and "chamber 2" correspond to the plurality of dummy variables. The data generating unit 104 generates data including the calculated plurality of explanatory variables.

[0108] In the modified example, the explanatory variables and the regression model (integrated model) of the expanded data are regarded as p-dimensional, and the same integrated model learning method as in the above embodiment is applied. For example, in the example of Fig. 8, the variable "sensor 1 x chamber 1" expanded as a variable specific to chamber 1 and the variable "sensor 1 x chamber 2" expanded as a variable specific to chamber 2 are interpreted as at least a part of the p-dimensional explanatory variables.

[0109] The integrated model constructed as described above may also require model updating. In particular, since the integrated model targets multiple quality characteristics, the model needs to be updated more frequently than a regression model for a single quality characteristic.

[0110] For updating the integrated model, the MA method can be used to update only the necessary parts. To apply the MA method to the integrated model, ij ,z ik ,x ij z ik Let (j=1, ,p, k=1, ,K) be the vector x i It is sufficient to extend the data in advance.

[0111] However, since the integrated model includes multiple quality characteristics, noise is likely to be large, and the dimensionality of the data is often large, so automatic parameter updating using the MA method may not always be appropriate.

[0112] Therefore, in this modified example, when a condition (such as the occurrence of an event) that determines the timing for learning (updating) the integrated model is satisfied, parameters that need to be updated according to the condition are identified, and the identified parameters are updated preferentially.

[0113] An example of a method for determining priority according to a modified example will be described.

[0114] Even in the modified example, the priority determination unit 103 can determine the priority of the explanatory variables using the event information stored in the event memory unit 122 and the correspondence information (correspondence information 123a, 123b) stored in the correspondence information memory unit 123.

[0115] Fig. 9 is a diagram showing an example of a data structure of event information used in a modified example. In Fig. 9, for example, an event type "related to chamber 1" is associated with an event "maintenance of chamber 1".

[0116] Fig. 10 is a diagram showing an example of a data structure of correspondence information used in a modified example. In the example of correspondence information 123a in Fig. 10, an event type "chamber 1 related" is associated with a variable type "chamber 1 related", and a selection priority of "0.5 (high)" and an update priority of "0.5 (high)" are associated with the combination. In the example of correspondence information 123b in Fig. 10, a variable type "chamber 1 related" is associated with a variable "sensor 1 x chamber 1" that is extended as a variable specific to chamber 1.

[0117] The correspondence information 123b between variables and variable types may be set manually by, for example, an administrator, etc. The correspondence information 123b may be generated by, for example, the priority determination unit 103 (which may be the data generation unit 104, the model learning unit 105, etc.) with reference to the extended data.

[0118] For example, the priority determination unit 103 identifies the type of objective variable corresponding to a dummy variable. In the example of FIG. 8, for the dummy variable "Chamber 1", the priority determination unit 103 identifies the type of objective variable as "Chamber 1 related". The priority determination unit 103 identifies the identified type of objective variable as the type of explanatory variable (variable type). The priority determination unit 103 generates correspondence information 123b (first correspondence information) that associates the identified types of explanatory variables with explanatory variables.

[0119] As a result, the priority determination unit 103 can use the generated correspondence information 123b and the correspondence information 123a (second correspondence information) to determine a priority corresponding to the type of the detected condition for the explanatory variables included in the variable type corresponding to the type of the detected condition (event type).

[0120] The priority determination unit 103 may further determine the priority by taking into consideration the prediction error of the prediction process by the trained model. For example, the priority determination unit 103 determines the priority of the corresponding explanatory variable according to the magnitude of the prediction error by the trained regression model among the multiple objective variables. For example, when the prediction error of the quality characteristic (objective variable) related to the chamber 2 is larger than the other quality characteristics (for example, the quality characteristic related to the chamber 1), the priority determination unit 103 determines the priority of the explanatory variable (for example, sensor 1 × chamber 2) related to the quality characteristic (objective variable) related to the chamber 2 to a value larger than the other explanatory variables. Conversely, the priority determination unit 103 may determine the priority of the explanatory variable having a larger prediction error than the other explanatory variables to a value smaller than the priority of the other explanatory variables.

[0121] The model learning unit 105 learns a regression model so as to optimize an objective function including a regularization term (penalty term) whose strength of regularization changes according to the determined priority. As described above, in the modified example, the explanatory variables of the expanded data are rewritten as p-dimensional learning data x i By assuming that the above-described embodiment is the same as the above-described embodiment, a learning method for an integrated model (regression model) can be applied.

[0122] In this manner, in the modified example, even for an integrated model having multiple quality characteristics, the model can be updated by changing the priority depending on the event, thereby enabling the model to be updated stably and also reducing the load associated with updating the model for efficient management.

[0123] Next, a hardware configuration of the information processing device of the embodiment (and the modified example) will be described with reference to Fig. 11. Fig. 11 is an explanatory diagram showing an example of the hardware configuration of the information processing device of the embodiment.

[0124] The information processing device of the embodiment includes a control device such as a CPU 51, a storage device such as a ROM (Read Only Memory) 52 and a RAM 53, a communication I / F 54 that connects to a network and performs communication, and a bus 61 that connects each unit.

[0125] The programs executed by the information processing apparatus according to the embodiment are provided in a state that they are pre-installed in the ROM 52 or the like.

[0126] The program executed by the information processing device of the embodiment may be configured to be provided as a computer program product by being recorded in an installable or executable format on a computer-readable recording medium such as a CD-ROM (Compact Disk Read Only Memory), a flexible disk (FD), a CD-R (Compact Disk Recordable), or a DVD (Digital Versatile Disk).

[0127] Furthermore, the program executed by the information processing apparatus of the embodiment may be configured to be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the program executed by the information processing apparatus of the embodiment may be configured to be provided or distributed via a network such as the Internet.

[0128] The programs executed by the information processing device of the embodiment can cause the computer to function as each unit of the information processing device described above. In this computer, the CPU 51 can read the programs from a computer-readable storage medium onto a main storage device and execute the programs.

[0129] A configuration example of the embodiment will be described below. (Configuration example 1) Detecting whether one or more conditions are met that determine when to train a regression model that predicts one or more objective variables for a plurality of explanatory variables; determining a priority for each of the plurality of explanatory variables in response to the conditions detected to be satisfied; training the regression model using an objective function including a regularization term whose strength of regularization changes according to the priority and training data; Processing section, An information processing device comprising: (Configuration example 2) The regression model predicts a plurality of dependent variables; The processing unit includes: calculating a plurality of explanatory variables by multiplying one or more first variables by a plurality of dummy variables corresponding to the plurality of objective variables, respectively, and generating the learning data including the calculated plurality of explanatory variables; training the regression model using the generated training data; The information processing device according to configuration example 1. (Configuration example 3) The processing unit includes: identifying a type of the objective variable corresponding to the dummy variable as a type of the explanatory variable, and generating first correspondence information that associates the identified type of the explanatory variable with the explanatory variable; determining the priority corresponding to the type of the detected condition for the explanatory variables included in the type of the explanatory variables corresponding to the type of the detected condition by using the first correspondence information and second correspondence information in which the type of the condition, the type of the explanatory variables, and the priority are associated with each other; The information processing device according to configuration example 2. (Configuration Example 4) The processing unit includes: determining a priority of the explanatory variable corresponding to the objective variable having a prediction error by the trained regression model larger than the other objective variables among the plurality of objective variables to be a value higher than the priority of the other explanatory variables; The information processing device according to configuration example 2. (Configuration Example 5) The processing unit includes: determining the priority corresponding to the detected condition for the explanatory variable corresponding to the detected condition using correspondence information that associates the condition, the explanatory variable, and the priority; The information processing device according to any one of configuration examples 1 to 4. (Configuration Example 6) The processing unit includes: generating the learning data including one or more explanatory variables having a higher priority than the other explanatory variables among the plurality of explanatory variables and one or more objective variables; training the regression model using the generated training data; 6. The information processing device according to any one of configuration examples 1 to 5. (Configuration Example 7) The processing unit includes: determining a change in each of the plurality of explanatory variables between the learning data and test data that is an input for prediction using the learned regression model, and determining a priority of the explanatory variable whose change is greater than that of other explanatory variables to be a value greater than the priorities of the other explanatory variables; The information processing device according to any one of configuration examples 1 to 6. (Configuration Example 8) the priority includes a selection priority indicating a priority for selecting the explanatory variable and an update priority indicating a priority for updating the explanatory variable, The objective function is: A term for evaluating the compatibility of the prediction result with the correct data; a first regularization term obtained by multiplying a term for regularizing a parameter corresponding to the explanatory variable by a weight based on the selection priority; a second regularization term obtained by multiplying a term for regularizing a change in the parameter before the update by a weight based on the update priority; Including, The information processing device according to any one of configuration examples 1 to 7. (Configuration Example 9) The processing unit includes: predicting one or more of the objective variables for test data including a plurality of the explanatory variables using the trained regression model; The information processing device according to any one of configuration examples 1 to 8. (Configuration Example 10) The processing unit includes: Inferring whether a subject associated with the test data is in a particular state based on a prediction error of the predicted dependent variable; The information processing device according to configuration example 9. (Configuration Example 11) The processing unit includes: training the regression model using the training data to optimize the objective function; The information processing device according to any one of configuration examples 1 to 10. (Configuration Example 12) The processing unit includes: a detection unit that detects whether the condition is satisfied; A determination unit that determines the priority; A learning unit that learns the regression model; Equipped with The information processing device according to any one of configuration examples 1 to 11. (Configuration Example 13) An information processing method executed by an information processing device, a detection step of detecting whether or not one or more conditions that define a timing for learning a regression model that predicts one or more objective variables for a plurality of explanatory variables are satisfied; determining a priority of each of the explanatory variables in response to the conditions detected to be satisfied; a learning step of learning the regression model using an objective function including a regularization term whose strength of regularization changes according to the priority and learning data; An information processing method comprising: (Configuration Example 14) On the computer, a detection step of detecting whether or not one or more conditions that define a timing for learning a regression model that predicts one or more objective variables for a plurality of explanatory variables are satisfied; determining a priority of each of the explanatory variables in response to the conditions detected to be satisfied; a learning step of learning the regression model using an objective function including a regularization term whose strength of regularization changes according to the priority and learning data; A program for executing the above.

[0130] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0131] 100 Information processing device 101 Model Acquisition Department 102 Event detection unit 103 Priority determination section 104 Data Generation Division 105 Model Learning Department 111 Prediction Department 112 Output control section 121 Model Memory Section 122 Event Memory Unit 123 Corresponding information storage unit 124 Sensor data storage unit 125 Generated data storage unit

Claims

1. Detecting whether one or more conditions are satisfied that define the timing for training a regression model that predicts one or more objective variables for a plurality of explanatory variables; determining a priority for each of the plurality of explanatory variables in response to the conditions detected to be satisfied; training the regression model using an objective function including a regularization term whose strength of regularization changes according to the priority and training data; Processing section, An information processing device comprising:

2. The regression model predicts a plurality of dependent variables; The processing unit includes: calculating a plurality of explanatory variables by multiplying one or more first variables by a plurality of dummy variables corresponding to the plurality of objective variables, respectively, and generating the learning data including the calculated plurality of explanatory variables; training the regression model using the generated training data; The information processing device according to claim 1 .

3. The processing unit includes: identifying a type of the objective variable corresponding to the dummy variable as a type of the explanatory variable, and generating first correspondence information that associates the identified type of the explanatory variable with the explanatory variable; determining the priority corresponding to the type of the detected condition for the explanatory variables included in the type of the explanatory variables corresponding to the type of the detected condition by using the first correspondence information and second correspondence information in which the type of the condition, the type of the explanatory variables, and the priority are associated with each other; The information processing device according to claim 2 .

4. The processing unit includes: determining a priority of the explanatory variables corresponding to the plurality of objective variables according to a magnitude of a prediction error by the trained regression model; The information processing device according to claim 2 .

5. The processing unit includes: determining the priority corresponding to the detected condition for the explanatory variable corresponding to the detected condition using correspondence information that associates the condition, the explanatory variable, and the priority; The information processing device according to claim 1 .

6. The processing unit includes: generating the learning data including one or more explanatory variables having a higher priority than the other explanatory variables among the plurality of explanatory variables and one or more objective variables; training the regression model using the generated training data; The information processing device according to claim 1 .

7. The processing unit includes: determining a change in each of the plurality of explanatory variables between the training data and test data that is an input for prediction using the trained regression model, and determining a priority of the explanatory variables based on the magnitude of the change; The information processing device according to claim 1 .

8. the priority includes a selection priority indicating a priority for selecting the explanatory variable and an update priority indicating a priority for updating the explanatory variable, The objective function is: A term for evaluating the compatibility of the prediction result with the correct data; a first regularization term obtained by multiplying a term for regularizing a parameter corresponding to the explanatory variable by a weight based on the selection priority; a second regularization term obtained by multiplying a term for regularizing a change in the parameter before the update by a weight based on the update priority; Including, The information processing device according to claim 1 .

9. The processing unit includes: predicting one or more of the dependent variables for test data including a plurality of the explanatory variables using the trained regression model; The information processing device according to claim 1 .

10. The processing unit includes: Inferring whether a subject associated with the test data is in a particular state based on a prediction error of the predicted dependent variable. The information processing device according to claim 9.

11. The processing unit includes: training the regression model using the training data to optimize the objective function; The information processing device according to claim 1 .

12. The processing unit includes: a detection unit that detects whether the condition is satisfied; A determination unit that determines the priority; A learning unit that learns the regression model; Equipped with The information processing device according to claim 1 .

13. An information processing method executed by an information processing device, a detection step of detecting whether or not one or more conditions that define a timing for learning a regression model that predicts one or more objective variables for a plurality of explanatory variables are satisfied; determining a priority of each of the explanatory variables in response to the conditions detected to be satisfied; a learning step of learning the regression model using an objective function including a regularization term whose strength of regularization changes according to the priority and learning data; An information processing method comprising:

14. On the computer, a detection step of detecting whether or not one or more conditions that define a timing for learning a regression model that predicts one or more objective variables for a plurality of explanatory variables are satisfied; determining a priority of each of the explanatory variables in response to the conditions detected to be satisfied; a learning step of learning the regression model using an objective function including a regularization term whose strength of regularization changes according to the priority and learning data; A program for executing the above.

Images