Early experiment stopping for batch bayesian optimization in industrial processes

Abstract

Early experiment stopping for batch Bayesian optimization in industrial processes. Real-time intervention in an industrial process can include searching a batch of candidate configurations for use by the industrial process, the batch of candidate configurations searched by performing batch Bayesian optimization (BBO). The batch of candidate configurations is sent to the industrial process for use in running the industrial process. A result of the running is received from the industrial process. The result is used in the BBO to search a next batch of candidate configurations. Based on the next batch of candidate configurations and by applying a function to a BBO harvest score, a determination is made whether a stopping criterion is met. Responsive to determining that the stopping criterion is met, the search for the next batch of candidate configurations is terminated.

Description

TECHNICAL FIELD

[0001] The present application relates generally to computers and computer applications, and more specifically to searching for desired outcomes of production trials and early stopping of trial production. BACKGROUND

[0002] Techniques based on Bayesian optimization have shown strong performance for tasks related to optimization of functions that are costly to evaluate. Examples of such functions include synthesis of molecules for medical discovery, configuration of machine learning systems, and design of electronic chips. Batch Bayesian optimization (BBO) extends the technique to allow simultaneous evaluation of multiple candidates. In BBO, the optimization is considered complete when the allocated budget is exhausted or a selection criterion fails to identify a new candidate to sample. However, it can be observed that the best candidate is observed before the budget is exhausted, which results in unnecessary expense to the entity performing the experiment. SUMMARY

[0003] A computer-implemented method and system of real-time intervention of an industrial process can be provided. In one aspect, the method can include searching a batch of candidate configurations used by the industrial process, the batch of candidate configurations searched by performing batch Bayesian optimization (BBO). The method can also include sending the batch of candidate configurations to the industrial process for use in running the industrial process. The method can also include receiving results of the running from the industrial process. The method can also include searching a next batch of candidate configurations in the BBO using the results. The method can also include determining, based on the next batch of candidate configurations and by applying a function to a BBO acquisition score, whether a stopping criterion is satisfied. The method can also include terminating the searching of the next batch of candidate configurations in response to determining that the stopping criterion is satisfied. The method can also include sending the next batch of candidate configurations to the industrial process for use in running the industrial process and repeating the searching of the next batch of candidate configurations in the BBO using the results and the determining of whether the stopping criterion is satisfied in response to determining that the stopping criterion is not satisfied.

[0004] In one aspect, a system for real-time intervention of an industrial process can include a hardware processor and a storage device coupled with the hardware processor. The hardware processor can be configured to search a batch of candidate configurations used by the industrial process, the batch of candidate configurations searched by performing batch Bayesian optimization (BBO). The hardware processor can also be configured to send the batch of candidate configurations to the industrial process for use in running the industrial process. The hardware processor can also be configured to receive a result of the running from the industrial process. The hardware processor can also be configured to search a next batch of candidate configurations in the BBO using the result. The hardware processor can also be configured to determine, based on the next batch of candidate configurations, whether a stopping criterion is satisfied. The hardware processor can also be configured to terminate the search of the next batch of candidate configurations in response to determining that the stopping criterion is satisfied. The hardware processor can also be configured to send the next batch of candidate configurations to the industrial process for use in running the industrial process and repeat the searching of the next batch of candidate configurations in the BBO using the result and the determining of whether the stopping criterion is satisfied in response to determining that the stopping criterion is not satisfied.

[0005] A computer readable storage medium storing program instructions executable by a machine for performing one or more of the methods described herein can also be provided.

[0006] The foregoing features and other aspects of different embodiments are described in detail herein with reference to the accompanying drawings. In the drawings, like numerals refer to like or functionally similar elements. BRIEF DESCRIPTION OF DRAWINGS

[0007] Figure 1 FIG. 1 is a diagram illustrating a system implementing early stopping of a BBO search in one embodiment;

[0008] Figure 2 FIG. 2 is a flow diagram illustrating a method of determining whether to early terminate a BBO search in one embodiment;

[0009] Figure 3 FIG. 3 is a block diagram illustrating system components in one embodiment;

[0010] Figure 4 FIG. 4 is a flow diagram illustrating a method in one embodiment;

[0011] Figure 5 FIG. 5 is a diagram illustrating components of a system providing real-time early stopping intervention of a process in one embodiment;

[0012] Figure 6 FIG. 6 illustrates a schematic diagram of an example computer or processing system that can implement a system in accordance with one embodiment;

[0013] Figure 7 A cloud computing environment in one embodiment is shown; and

[0014] Figure 8 A set of functional abstraction layers provided by a cloud computing environment in one embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0015] In one aspect, early stopping or real-time intervention of experimental or trial production runs, e.g., producing a product or generating a process in an actual physical system (e.g., a manufacturing system or another system), can be provided. In one or more embodiments, methods and systems for real-time intervention in a batch Bayesian optimization (BBO) system (e.g., running a real physical system) can be provided so that if a predetermined threshold is reached (e.g., if a particular criterion is met), it is automatically determined or decided whether to early terminate the BBO search. For example, a manufacturing system can be controlled to early stop its experimental or trial production runs when looking for the best results in production. Experiments show that such a method reduces the average cost associated with, e.g., a BBO search that can involve iteratively running a real physical machine or system.

[0016] For example, a method can include determining whether a threshold is met by applying a function to a batch Bayesian optimization (BBO) harvest score, the function determining a likelihood that evaluating a next batch will provide a significant value. The function includes a target variable representing a statistical significance level required to terminate the search and a batch percentage variable determining how many of a batch need to fail to terminate the search. The method can also include sending a signal to the batch Bayesian optimization system to early terminate the search if the threshold is met or reached.

[0017] Bayesian optimization is used to optimize a black-box function or unknown function. Bayesian optimization guides the search to find a minimum or maximum of an unknown objective function. A surrogate model or probabilistic representation (e.g., a probabilistic model) of the black-box function can be constructed to approximate the true function based on available parameter values and associated observations, e.g., a Gaussian process is fitted using available data. The mean from the Gaussian process can be used to model the black-box function. An acquisition function (e.g., of the surrogate model) determines the best location to sample next in the modeled parameter space. For example, the next sample can be located at a location where the acquisition function is maximized. A corresponding output is obtained corresponding to the new sample. The corresponding output can be based on an actual or true observation obtained using the new sample. For example, the actual or true observation can be obtained from running a real experiment or trial manufacturing process. The new sample and corresponding output (observation) are then used to update the surrogate model. This process of finding or searching for the next sample, obtaining an actual observation using the next sample, and updating the surrogate model can be repeated, e.g., until a budget is met. Batch Bayesian optimization (BBO) finds the best sample in batches.

[0018] The methods disclosed herein include intervening based on a criterion to early terminate a BBO search. In one embodiment, this criterion can be determined by applying an additional function to the BBO acquisition score that determines the likelihood that evaluating the next batch (which can be expensive) will provide a significant value. For example, in one embodiment, the criterion for terminating the BBO search can be determined from the BBO acquisition function or BBO acquisition function value.

[0019] In one embodiment, the criterion for early termination of the BBO search considers two variables, a target criterion and a batch percentage criterion. The target criterion represents the statistical significance required to terminate the search (e.g., the target criterion can take a binary form, e.g., 0: full significance, 1: no significance), while the percentage criterion determines how many of a batch need to fail to terminate the search (e.g., 0% - none in the batch, 100% - all in the batch). In one embodiment, these criteria can be set or predetermined based on the cost of acquiring a batch (e.g., a more aggressive strategy can be used when the cost is high) and the importance of full optimization (e.g., a more aggressive strategy can be used when full optimization is not necessary). If the termination criterion is met, a signal can be sent to the BBO engine to early terminate the search, thereby saving the cost of evaluating additional batches.

[0020] Such methods can be integrated with, but are not limited to, laboratory-based experimental design, manufacturing processes, high-performance computing, and / or other physical or industrial processes. Figure 1is a diagram illustrating a system implementing early stopping of BBO search in one embodiment. The illustrated components include computer-implemented components, e.g., computer-implemented components implemented and / or running on one or more processors, such as hardware processors, or coupled with one or more hardware processors. For example, the one or more hardware processors can include components such as programmable logic devices, microcontrollers, storage devices, and / or other hardware components, which can be configured to perform respective tasks described in this disclosure. The coupled storage devices can be configured to selectively store instructions that are executable by the one or more hardware processors.

[0021] The processor can be a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), another suitable processor or processing component, or one or more combinations thereof. The processor can be coupled with a storage device. The storage device can include random access memory (RAM), read-only memory (ROM), or another storage device, and can store data and / or processor instructions for implementing different functions associated with the methods and / or systems described herein. The processor can execute computer instructions stored in the storage or received from another computer device or medium.

[0022] The Bayesian optimization system 104 can be implemented on a computer system including one or more processors, for example, and can include computer-implemented modules such as a communication module 108, a decision module 112, a termination module 110, and a storage module 114. The communication module 108 can include a network interface or the like that can send and receive data to and from one or more computer systems and / or processors, e.g., to communicate with a user or client computer, to communicate with one or more controllers that control a manufacturing or similar process (also referred to as a data acquisition source). The storage module 114 can be coupled with a storage device for storing data, and can include a database for storing and accessing data.

[0023] A user computer can have a user interface or front-end application 102 that can communicate with the Bayesian optimization system 104. An optimization user (e.g., via the user interface or front-end application 102) can communicate optimization criteria to the Bayesian optimization system 104, e.g., to initiate experimental runs or a process of trial and error with different parameter values in a process, such as a manufacturing process, in order to obtain a result or resulting product. Other examples can include a process of discovering molecules, in pharmaceutical applications (e.g., discovering medicine), and any other process in which an optimization solves or determines one or more parameters or values to use in a process. Another example can include training a machine learning model, such as training a neural network using different hyperparameter values obtained with Bayesian optimization.

[0024] Examples of the user interface 102 can include a web portal or web browser interface, a front-end user interface implemented for the Bayesian optimization system 104, and the like. An optimization user can run such a computer-implemented user interface 102 to communicate with the Bayesian optimization system 104 to provide optimization criteria and to start a process such as an experimental design process, a manufacturing process, and / or another process. Examples of optimization criteria can include a budget (e.g., a number of iterations to perform in the optimization), a batch size, an early termination aggression. For example, the "early termination aggression" can be expressed in numerical form (e.g., a floating point number from 0 to 1, another numerical value, a binary value (e.g., 0 (not aggressive) or 1 (aggressive)), or multiple value settings (e.g., 0, 1, 2, 3, 4, 5) representing different levels of aggression, or any other representation). In one embodiment, such a setting can be transformed or converted into one or more values (e.g., mapped to a target criteria and a batch percentage criteria) to apply a function to a BBO acquisition function score in determining the criteria for early termination of a BBO search. In one embodiment, the "early termination aggression" entered by a user can include specific values, e.g., a target criteria and a batch percentage criteria. The "early termination aggression" can be entered or indicated in other ways. The communication module 108 can receive the communication from the user interface 102.

[0025] In another aspect, the Bayesian optimization system 104 can be automatically programmed or triggered to start, e.g., with a set of initial criteria (e.g., a budget, a batch size, an early termination aggression), without an explicit command from an optimization user to start via the user interface.

[0026] The decision module 112 of the Bayesian optimization system 104 generates a batch of configurations and communicates the configurations to, e.g., the data acquisition source 106, makes a decision about which candidate configurations to try next. For example, the decision module 112 runs or performs a batch Bayesian optimization, which generates the batch of configurations. The batch of configurations includes different parameter values that the data acquisition source 106 can try or experiment with in order to obtain a desired result or characteristic. For example, the communication module 108 sends the batch of configurations to the data acquisition source 106 to perform a process or run (e.g., an experimental process or trial run) to produce an actual output (e.g., a manufactured product) using the communicated parameters.

[0027] The data acquisition source 106 can include a physical system that performs or activates the actual process. Examples of such data acquisition sources can include a robot or controller that performs a physical process, such as a manufacturing process, like an industrial process for manufacturing a product such as paint, a robot or controller that performs a task such as assay and / or molecular discovery; a machine learning system that trains a machine learning model (e.g., a neural network). Examples of configurations or parameters received from the decision model for experimentation can include: concentration of ingredients, mixing rate of ingredients, temperature at which ingredients are mixed, mixing time or duration for example in chemical manufacturing to mix them. Examples of results or return values can include yield of the process resulting from performing the chemical manufacturing process using these configurations. Another example can be a batch of hyperparameter values used in training a neural network.

[0028] The data acquisition source 106 generates a result of the configuration (e.g., as a result of performing the actual process) and transmits the result to the Bayesian optimization system 104, for example, via the communication module 108. The result includes one or more characteristics of the result obtained from the actual run of the process. The characteristics can indicate how well the data acquisition source produces its product or end product using the configuration received from the decision module 112. The storage module 114 of the Bayesian optimization system 104 stores the result of the configuration, for example, in a storage device.

[0029] The decision module 112 of the Bayesian optimization system 104 uses the result and uses the BBO method to generate another (next) batch of configurations. For example, the result is used to update the surrogate model, while the activation function is used to find the next batch of configurations. The decision module 112 transmits the values obtained from the BBO for the batch to the termination module 110. Using these values, the termination module 110 evaluates whether to terminate the search for better configurations before the budget is exhausted. For example, the termination module 110 applies a function to the BBO acquisition function, for example, using one or more values associated with an “early termination aggressiveness” that can be set based on user input or pre-defined, to determine whether the search is early to end. Different acquisition functions can be used, for example, improvement probability, expected improvement, and / or others. If the termination module 110 determines that the criteria for stopping are met, the termination module 110 signals the decision module 112 to stop performing the optimization to stop the search for more configurations. The Bayesian optimization system 104 also signals to the data acquisition source 106, for example, via the decision module 112 and the communication module 108, that the search for the best parameters has stopped. In one embodiment, the sending of such a stop signal can control the data acquisition source 106 to stop its process, for example, a trial run of a process that produces a product. The decision module 112 can also send to the data acquisition source 106 via the communication module 108 and / or to the optimization user via the user interface 102 the final configuration that reflects the best configuration from the multiple sets or batches of configurations from different BBO iteration runs.

[0030] If the termination module 110 determines that the stopping criteria is not satisfied, the decision module 112 performs another iteration of the BBO, sends the parameter values (a batch of configurations) obtained from the batch of BBO to the termination module 110 for another evaluation of whether the stopping criteria is satisfied before the batch is exhausted. These values are also sent to the data acquisition source 106 to perform its process again using these parameter values. The optimization iteration or repetition, where the data acquisition source 106 iterates its execution of experiments using the configurations determined from the iteration, until the termination module 110 determines that the criteria is satisfied or the budget is exhausted. If the budget is exhausted, the final optimized configuration (e.g., the best set of configurations) can be sent to the optimization user via the user interface 102 by the communication module 108.

[0031] Figure 2 is a flowchart illustrating a method of determining whether to early terminate a BBO search in one embodiment. The method can be implemented, executed, or run on one or more hardware processors.

[0032] At 202, a target criteria and a batch percentage criteria for the search can be set or predetermined. In one embodiment, the target criteria can be represented by a floating point between 0 and 1 (floating point value). In one embodiment, the batch percentage criteria can be represented by a percentage (%) from 0-100. The target criteria and the batch percentage criteria can be predetermined or predefined based on the cost of performing modeling and experiments based on the modeling performed, and are configurable. For example, if the cost of experiments (e.g., running a real system) is high or expensive, the target criteria and / or the batch percentage criteria can be set more aggressively, e.g., to stop the search earlier than when set to less aggressive values. For example, the target criteria and the batch percentage criteria can be user-defined parameters received from a user and / or mapped from user input data that indicate the degree of aggressiveness in pursuing early stopping.

[0033] At 204, a batch of candidates for evaluation is obtained using a batch Bayesian optimization (BBO) method. For example, a proxy model (also referred to as a response surface model) or a probabilistic model can be constructed based on available parameter values and associated observations (e.g., results or outputs). A batch of candidates can be found using an acquisition function associated with the proxy model.

[0034] At 206, for example, for each candidate within the batch, a contextual improvement probability (cPI) is evaluated, where the cPI is formulated as follows:

[0035]

[0036] where

[0037]

[0038] wherein, is the mean of the variance contained within the sampled posterior distribution, and is distinguished from σ, which is the individual variance of the prediction for a particular point in the posterior, y pred represents the predicted value from the candidate (the value predicted by the Bayesian model for the candidate with the given configuration), f * represents the best candidate found so far (e.g., the value generated by the data acquisition source for the best candidate evaluated so far), while Φ is the cumulative distribution function (CDF) of the standard normal distribution. In one embodiment, all of these values are extracted from the BBO method.

[0039] At 208, the computed cPI values are then compared to a target criterion. For example, the cPI score or value of each candidate is compared to the target criterion. The number of candidates in the batch whose cPI score is less than the target criterion is determined. The percentage or proportion of all candidates in the batch whose cPI score is less than the target criterion is determined.

[0040] At 210, if the percentage of candidates whose cPI score is less than the stop criterion (target criterion) is greater than a predetermined percentage (batch percentage criterion), a signal can be generated and / or sent to the BBO engine (which, for example, performed the BBO at 204) to terminate, for example, the search to obtain more candidates to perform the BBO.

[0041] Figure 3is a block diagram illustrating system components in one embodiment. The illustrated components include computer-implemented components, e.g., implemented and / or running on one or more hardware processors, or coupled with one or more hardware processors. The batch Bayesian optimization engine 304 can receive optimization criteria, such as a budget, a batch size, and termination settings, from a user, e.g., via the user interface 302. The batch Bayesian optimization engine 304 performs a batch Bayesian optimization method, and determines a batch of configurations to be tried by the data collection source 308 in a process, such as a manufacturing process. The data collection source 308 uses the batch of configurations to perform its experiments or tests, and sends results to the batch Bayesian optimization engine 304. For example, in one embodiment, the sending of the batch of configurations automatically controls a manufacturing process to perform its manufacturing, e.g., activates manufacturing equipment to be run. As another example, in one embodiment, the sending of the batch of configurations automatically controls or activates a robot to perform its tasks, such as assays. The batch Bayesian optimization engine 304 receives results from the data collection source 308, e.g., one or more properties of a product or process resulting from performing a process, such as a manufacturing process. For example, the results can be how well a product produced using the configurations eliminates bacteria. As another example, the results can be how accurately a neural network trained using the configurations performs a classification task. The batch Bayesian optimization engine 304 stores the results and associated configurations. Based on the results of different batch configurations, the batch Bayesian optimization engine 304 is able to determine which configurations are the best of the configurations that have been tested.

[0042] The batch Bayesian optimization engine 304 obtains a next batch of configurations, e.g., by updating a surrogate model using the results received from the data collection source 308 and applying a collection function according to the updated surrogate model. The batch Bayesian optimization engine 304 sends the next batch of configurations to the stopping criteria evaluator 306. The stopping criteria evaluator 306 determines whether to stop searching for a next batch of configurations, e.g., by applying a function to the collection function of the batch Bayesian optimization engine 304. For example, the stopping criteria evaluator 306 can perform the method described with reference to Figure 2 The stopping criteria evaluator 306 returns or sends a signal to the batch Bayesian optimization engine 304 indicating whether to stop searching. For example, if the stopping criteria are met, the stopping criteria evaluator 306 sends a signal indicating that the search is to be stopped. Otherwise, the stopping criteria evaluator 306 can send a signal or indication that the search is to continue.

[0043] If no termination or stop signal is received, the batch Bayesian optimization engine 304 sends the next batch of configurations to the data collection source 308 for use in its real or actual process. The data collection source 308 uses the next batch of configurations to perform its process, returning results to the batch Bayesian optimization engine 304. The results are stored. The Bayesian optimization engine 304 again updates the surrogate model based on the results and applies the acquisition function to find the next batch of configurations.

[0044] At 304, the next batch of configurations is found, at 306, the stop criteria are evaluated, and the next batch of configurations is sent to the data collection source 308, where the data collection source 308 performs its test process, and the results received from the data collection source 308 are stored are repeated until the stop criteria evaluator 306 determines that the stop criteria are satisfied or a specified budget (e.g., number of iterations) is exhausted. In response to the optimization search being completed (e.g., the stop criteria are satisfied or the specified budget is exhausted), the batch Bayesian optimization engine 304 can send the batch of configurations determined to be optimal (e.g., those that produced the best results) to the user, e.g., via the user interface 302. In one aspect, the batch Bayesian optimization engine 304 can also control the data collection source 308 to stop its experiment or trial process. For example, the batch Bayesian optimization engine 304 can control or deactivate a physical process, such as a manufacturing process, to stop a process in manufacturing. For example, the batch Bayesian optimization engine 304 can control a physical process or device to stop or continue its process. As another example, the batch Bayesian optimization engine 304 can also control a neural network to stop a process of training the neural network, e.g., stop an experiment using different hyperparameters in training the neural network.

[0045] Figure 4 is a flowchart illustrating a method in one embodiment. The method can be a computer- implemented method running on one or more hardware processors. The method performs real-time intervention in a batch Bayesian optimization (BBO) system to automatically determine whether to early terminate a BBO search if a predetermined threshold is reached. In one aspect, running a data collection source (such as a manufacturing process and / or a robot), for example, in real-time can be expensive. For example, iteratively running the data collection source with new or next configurations can consume power, equipment, and computer resources. The method in one embodiment intervenes in real-time to early stop the iterative running of the data collection source (e.g., a manufacturing process and / or a robot performing a task). For example, the method can include controlling and / or causing such control of a physical machine in its process in real-time. Such early stopping can reduce power consumption, reduce consumption of equipment and computer resources (including but not limited to memory equipment and storage device space).

[0046] At 402, termination settings can be received, such as data indicative of an aggressiveness of early stopping of pursuit of BBO search. For example, a target criterion and a batch percentage criterion can be received. As another example, information that can be mapped to such criteria can be received. In one embodiment, such data (e.g., target criterion and batch percentage criterion) is configurable. In one embodiment, the target criterion and the batch percentage criterion can be received from a user. In one embodiment, the target criterion represents a level of statistical significance required to terminate the search, and the batch percentage criterion represents how many failures in a batch are required to terminate the search.

[0047] At 404, a batch of candidate configurations is searched for use by the industrial process. In one embodiment, the batch of candidate configurations is searched by performing batch Bayesian optimization (BBO), e.g., as described above.

[0048] At 406, the batch of candidate configurations is sent to the industrial process for use in running the industrial process. In one embodiment, the transmission of the batch of candidate configurations controls a machine (e.g., a robot) associated with the industrial process to perform its tasks or run, e.g., to manufacture or produce a product. At 408, results of the running or production are received from the industrial process. For example, the results can include one or more characteristics of the manufactured product.

[0049] At 410, the results are used in the BBO to search for a next batch of candidate configurations. At 412, whether a stopping criterion is satisfied is determined, e.g., based on the next batch of candidate configurations and by applying a function to a BBO acquisition score (e.g., using the received data indicative of an aggressiveness of early stopping of pursuit of BBO search).

[0050] In one embodiment, applying the function includes evaluating data associated with the next batch of candidates based on the target criterion and the batch percentage criterion. In one embodiment, determining whether the stopping criterion is satisfied based on the next batch of candidate configurations can include, for each of the candidate configurations in the batch, computing a contextual improvement probability (cPI) score, and determining that a percentage of candidate configurations having a cPI score less than the target criterion is greater than the batch percentage criterion, e.g., as described above with reference to Figure 2 For example, the cPI score can be determined as a cumulative distribution function CDF of a standard normal distribution of a combination of: a mean of variances contained within a sampled posterior distribution, a predicted value from a candidate, and a best candidate value found among iterations of the BBO, e.g., as described above with reference to Figure 2 For example, the cPI score can be determined as a cumulative distribution function CDF of a standard normal distribution of a combination of: a mean of variances contained within a sampled posterior distribution, a predicted value from a candidate, and a best candidate value found among iterations of the BBO, e.g., as described above with reference to

[0051] At 414, in response to determining that the stop criterion is satisfied, the search for the next batch candidate is stopped or terminated. In one embodiment, the industrial process can be controlled to stop running. For example, in response to determining that the stop criterion is satisfied, the processor executing the BBO can send a signal to the industrial process or automatically control the industrial process to stop its running iteration. The best batch configuration among the discovered batch configurations can be sent to the industrial process. For example, the industrial process can be controlled to use the best batch configuration in its production. Such best batch configuration can also be sent to a user, e.g., via a user interface.

[0052] At 416, in response to determining that the stop criterion is not satisfied, the next batch candidate configuration is sent to the industrial process for use in running the industrial process. For example, in response to determining that the stop criterion is not satisfied, the processor executing the BBO can send the next batch candidate configuration to the industrial process for another iteration of running. For example, sending the next batch candidate configuration automatically controls the industrial process to run in real time. The process or steps of using the result in the BBO to search for the next batch candidate configuration and determining whether the stop criterion is satisfied can be repeated or iterated. In one aspect, the above-described method can provide early stop intervention in real time for actual running processes such as manufacturing or industrial processes. In one aspect, the method in one embodiment can automatically control physical machines associated with manufacturing or industrial processes, such as robots, to continue and / or stop their production iterations in real time.

[0053] Figure 5is a diagram illustrating components of a system that provides real-time early stopping intervention of a process in one embodiment. One or more hardware processors 502, such as central processing units (CPUs), graphics processing units (GPUs), and / or field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), and / or another processor, can be coupled with a storage device 504, and perform batch Bayesian optimization and recommend candidate configurations for use in the process. The storage device 504 can include random access memory (RAM), read-only memory (ROM), or another storage device, and can store data and / or processor instructions for implementing various functions associated with the methods and / or systems described herein. The one or more processors 502 can execute computer instructions stored in the memory 504 or received from another computer device or medium. The storage device 504 may, for example, store instructions and / or data for running the one or more hardware processors 502, and can include an operating system and other instruction and / or data programs. The one or more hardware processors 502 can receive input, which can include data for performing batch Bayesian optimization, batch size, target criteria, batch percentage criteria, budget (e.g., number of iterations). For example, the at least one hardware processor 502 can generate a recommendation (candidate configuration) and send the candidate configuration in real-time for a trial run in the actual process. The candidate configuration is also used to determine whether a stopping criterion is met. Based on the stopping criterion being met, the experiment run can be early stopped. The recommendation and results of the trial experiment run based on the recommendation can be stored in storage 506. In one embodiment, input data can be received from a remote device via network interface 508, and can be temporarily loaded into storage device 504 for performing batch Bayesian optimization. The one or more hardware processors 502 can be coupled with interface devices such as network interface 508 for communicating with remote systems, e.g., via a network, and input / output interface 510 for communicating with input and / or output devices such as a keyboard, a mouse, a display, etc.

[0054] Figure 6 A schematic diagram illustrating an example computer or processing system in which a system can be implemented in one embodiment is shown. The computer system is only one example of a suitable processing system and is not intended to limit the scope of use or functionality of embodiments of the methods described herein. The processing system shown can operate as a standalone device or can be connected, e.g., networked to other processing systems or devices, in a networked deployment, for example, an internal network, an external network (e.g., the Internet), or combinations thereof. In a networked deployment, the processing system can operate in the capacity of a server or as a client user computer in server-client user computer network environments. Figure 6Examples of well known computing systems, environments, and / or configurations that can be employed by the processing system described herein include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems or devices, and the like.

[0055] A computer system can be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules can include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks or implement particular abstract data types. Computer systems can be implemented in a distributed cloud computing environment with remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules can be located in both local and remote computer system storage media including memory storage devices.

[0056] Components of computer system can include, but are not limited to, one or more processors or processing units 12, a system memory 16, and a bus 14 that couples various system components including system memory 16 to processor 12. Processor 12 can include modules 30 that perform the methods described herein. Modules 30 can be programmed into integrated circuits of processor 12, or loaded from memory 16, storage device 18, or network 24, or a combination thereof.

[0057] Bus 14 can represent one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0058] Computer system can include a variety of computer system readable media. Such media can be any available media that is accessible by computer system and it can include both volatile and non-volatile media, removable and non-removable media.

[0059] The system memory 16 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory, etc. The computer system can further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 18 can be provided for reading from and writing to non-removable, non-volatile magnetic media (e.g., a "hard drive"). Although not shown, a magnetic disk drive can also be provided for reading from and writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive can be provided for reading from or writing to a removable, non-volatile optical disk (e.g., a "CD-ROM" or "DVD-ROM"), among others. In such instances, each can be connected to the bus 14 by one or more data media interfaces.

[0060] The computer system can also communicate with one or more external devices 26 such as a keyboard, a pointing device, a display 28, etc.; one or more devices that enable a user to interact with the computer system; and / or any devices (e.g., network card, modem, etc.) that enable the computer system to communicate with one or more other computing devices. Such communication can occur via Input / Output (I / O) interface(s) 20.

[0061] Still yet, the computer system can communicate with one or more networks 24 such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet) via network adapter 22. As depicted, network adapter 22 communicates with the other components of the computer system via bus 14. It should be understood that, although not shown, other hardware and / or software components could be used in conjunction with the computer system. Examples, include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0062] It should be understood that, although the present disclosure can include detailed discussion of cloud computing, aspects of the technologies described herein can be implemented without cloud computing. Cloud computing is a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g. networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a provider of the service. This cloud model can include at least five characteristics, at least three service models, and at least four deployment models.

[0063] The characteristics are as follows:

[0064] On-demand self-service: a consumer can unilaterally provision computing capabilities, such as server time and network storage, as needed automatically (without human interaction with each service provider).

[0065] Broad network access: capabilities are available over a network that is typically the Internet and user can access software and data from various devices using a variety of protocols.

[0066] Resource pooling: the provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to consumer demand. There is a sense of location independence in that the consumer generally has no control or knowledge over the exact location of the provided resources but can be able to specify location at a higher level of abstraction (e.g., country, state, or data center).

[0067] Rapid elasticity: capabilities can be rapidly and elastically provisioned, in some cases automatically, to quickly scale out and in to match demand. The consumer can have no knowledge of the underlying cloud infrastructure that is supporting the provided capability, and can not need to take any of that into consideration (no concern about the total

[0068] Measured service: cloud systems automatically control and optimize resource use by leveraging a metering capability at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported providing transparency for both the provider and consumer of the service.

[0069] The business model of a cloud system is as follows:

[0070] Software as a Service (SaaS): the capability provided to the consumer is to use the provider's applications running on a cloud infrastructure. The applications are accessible from various client devices through a thin client interface such as a web browser (e.g., web-based e-mail). The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0071] Platform as a Service (PaaS): the capability provided to the consumer is to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including networks, servers, operating systems, or storage, but has control over the deployed applications and possibly application hosting environment configurations.

[0072] Infrastructure as a Service (IaaS): This provides consumers with the capability to deploy and run any software, including operating systems and applications, on the underlying cloud infrastructure, including processing, storage, networking, and other basic computing resources. Consumers neither manage nor control the underlying cloud infrastructure, but they have control over the operating system, storage, and the applications deployed thereon, and may have limited control over chosen network components (such as host firewalls).

[0073] The deployment model is as follows:

[0074] Private cloud: The cloud infrastructure runs exclusively for a single organization. The cloud infrastructure can be managed by that organization or a third party and can exist inside or outside the organization.

[0075] Community cloud: A cloud infrastructure shared by several organizations that supports a specific community with common interests (such as mission, security requirements, policy, and compliance considerations). A community cloud can be managed by multiple organizations within the community or by third parties and can exist inside or outside the community.

[0076] Public cloud: Cloud infrastructure provided to the public or large industrial groups and owned by organizations that sell cloud services.

[0077] Hybrid cloud: A cloud infrastructure consisting of two or more cloud deployment models (private cloud, community cloud, or public cloud) that remain distinct entities but are bound together by standardized or proprietary technologies that enable data and applications to be ported together (such as cloud burst traffic balancing for load balancing between clouds).

[0078] Cloud computing environments are service-oriented, characterized by statelessness, loose coupling, modularity, and semantic interoperability. The core of cloud computing is its infrastructure, which comprises a network of interconnected nodes.

[0079] Now for reference Figure 7 The figure illustrates an exemplary cloud computing environment 50. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10 that can communicate with local computing devices used by cloud computing consumers. Local computing devices may be, for example, personal digital assistants (PDAs) or mobile phones 54A, desktop computers 54B, laptops 54C, and / or automotive computer systems 54N. The cloud computing nodes 10 can communicate with each other. The cloud computing nodes 10 can be physically or virtually grouped (not shown) in one or more networks, including but not limited to private clouds, community clouds, public clouds, or hybrid clouds or combinations thereof, as described above. In this way, cloud consumers can request Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and / or Software as a Service (SaaS) provided by the cloud computing environment 50 without maintaining resources on their local computing devices. It should be understood that... Figure 7The various types of computing devices 54A-N shown are only illustrative and that cloud computing nodes 10 and cloud computing environment 50 are not limited to

[0080] Referring now to FIG. 1, illustrative cloud computing environment 50 is depicted. Figure 8 wherein cloud computing environment 50 provides a set of functional abstraction layers. Figure 7 It should be understood that any of the components, layers, and functions described herein can be implemented in hardware, software, or a combination of both hardware and software. Figure 8 The components shown in FIG. 1 are merely illustrative and are not meant to limit the scope of embodiments of the present application. One of ordinary skill in the art will Figure 8 As shown, the following layers and corresponding functions are provided:

[0081] Hardware and software layer 60 includes hardware and software components. Examples of hardware components include: mainframes 61; RISC (Reduced Instruction Set Computer) architecture based servers 62; servers 63; blade servers 64; storage devices 65; and networks and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0082] Virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers 71; virtual storage 72; virtual networks 73, including virtual private networks; virtual applications and operating systems 74; and virtual clients 75.

[0083] In one example, management layer 80 can provide the functions described below. Resource provisioning 81: providing dynamic procurement of computing resources and other resources that are utilized to perform tasks within the cloud computing environment. Metering and Pricing 82: providing costs associated with using the cloud computing resources, for example, in a

[0084] Workloads layer 90 provides examples of functionality for which the cloud computing environment can be utilized. Examples of workloads and functions which can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analysis processing 94; transaction processing 95; and early experiment stoppage for batch Bayesian optimization processing 96.

[0085] The present application can be a system, a method, and / or a computer program product at any possible technical detail level of integration. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present application.

[0086] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch cards or raised structures in grooves of a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0087] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0088] Computer readable program instructions for carrying out operations of the present application can be assembly instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, configuration data for an integrated circuit, or source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0089] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0090] These computer readable program instructions can be provided to a processor of a computer, or other programmable data processing apparatus, to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored

[0091] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0092] The computer program product of the second aspect of the present application can comprise a computer readable medium. The computer readable medium can comprise a transitory signal. The computer program product of the second aspect of the present application can comprise a computer readable medium. The computer readable medium can comprise a non-transitory computer-readable medium. The computer program product of the second aspect of the present application can comprise a computer readable medium. The computer readable medium can comprise a transitory signal.

[0093] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein the phrase "in an embodiment" does not necessarily refer to the same embodiment, although it may. As used herein the phrase "in one embodiment" does not necessarily refer to the same embodiment, although it may. As used herein the phrase "in another embodiment" does not necessarily refer to a different embodiment, although it may. Furthermore, examples and / or embodiments of the present application can be combined freely with one another.

[0094] The corresponding structures, materials, acts, and equivalents of all means or step plus functions elements in the claims below are intended to include any structure, material, or act for performing the functions in combination with other claimed elements as specifically claimed. The description of the present application has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the application in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the application. The embodiment was chosen and described in order to best explain the principles of the application and the practical application, and to enable others skilled in the art to understand the application for various embodiments with various modifications as are suited to the particular use contemplated.

Claims

1. A computer-implemented method of real-time intervention for an industrial process, comprising: searching for a batch of candidate configurations for use by the industrial process, the batch of candidate configurations being searched for by performing batch Bayesian optimization (BBO), the candidate configurations including concentrations of ingredients, mixing rates of ingredients, temperatures at which ingredients are mixed, mixing times; sending the batch of candidate configurations to the industrial process for use in running the industrial process, wherein the industrial process includes at least one robot associated with the industrial process, the robot performing a task in accordance with controls of the candidate configurations; receiving results of the running from the industrial process; using the results in the BBO to search for a next batch of candidate configurations; determining, based on the next batch of candidate configurations and by applying a function to a BBO acquisition score, whether a stopping criterion is satisfied; in response to determining that the stopping criterion is satisfied, terminating the search for the next batch of candidate configurations and controlling the robot to stop performing the task; and in response to determining that the stopping criterion is not satisfied, sending the next batch of candidate configurations to the industrial process for use in running the industrial process, and repeating using the results in the BBO to search for a next batch of candidate configurations and determining whether the stopping criterion is satisfied.

2. The method of claim 1, wherein, the function includes evaluating data associated with the next batch of candidate configurations based on a target criterion and a batch percentage criterion, the target criterion representing a statistical significance level required to terminate the search, the batch percentage criterion representing how many of a batch need to fail to terminate the search.

3. The method of claim 2, wherein, determining, based on the next batch of candidate configurations, whether a stopping criterion is satisfied includes: for each of the candidate configurations in the batch, computing a contextual improvement probability (cPI) score; and determining that a percentage of candidate configurations having the cPI score less than the target criterion is greater than the batch percentage criterion.

4. The method of claim 3, wherein, the cPI score is determined as a cumulative distribution function (CDF) of a standard normal distribution of a combination of: a mean of variances contained within a sampled posterior distribution, a predicted value from a candidate, and an optimal candidate value found among iterations of the BBO.

5. The method of claim 1, further comprising: in response to determining that the stopping criterion is satisfied, sending, to the industrial process, an optimal one of the batches of configurations found in the search.

6. The method of claim 1, further comprising: in response to determining that the stopping criterion is satisfied, sending, to a user via a user interface, an optimal one of the batches of configurations found in the search.

7. The method of claim 2, wherein, the target criterion and the batch percentage criterion are configurable.

8. A system for real-time intervention for an industrial process, comprising: a hardware processor; and a storage device coupled with the hardware processor; the hardware processor is configured to perform at least the following operations: searching for a batch of candidate configurations for use by the industrial process, the batch of candidate configurations being searched for by performing batch Bayesian optimization (BBO), the candidate configurations including concentrations of ingredients, mixing rates of ingredients, temperatures at which ingredients are mixed, mixing times; ​ sending the batch of candidate configurations to the industrial process for use in running the industrial process, wherein the industrial process includes at least one robot associated with the industrial process, the robot performing a task in accordance with the batch of candidate configurations; receiving results of the running from the industrial process; searching, in the BBO, for a next batch of candidate configurations using the results; determining, based on the next batch of candidate configurations and by applying a function to a BBO harvest score, whether a stopping criterion is met; in response to determining that the stopping criterion is met, terminating the searching for the next batch of candidate configurations and controlling the robot to stop performing the task; and in response to determining that the stopping criterion is not met, sending the next batch of candidate configurations to the industrial process for use in running the industrial process, and repeating searching, in the BBO, for a next batch of candidate configurations using the results and determining whether the stopping criterion is met.

9. The system of claim 8, wherein, the hardware processor applying the function includes evaluating data associated with the next batch of candidate configurations based on a target criterion and a batch percentage criterion, the target criterion representing a statistical significance level required to terminate the searching, the batch percentage criterion representing how many of a batch need to fail to terminate the searching.

10. The system of claim 9, wherein, to determine, based on the next batch of candidate configurations, whether the stopping criterion is met, the hardware processor is configured to: for each of the candidate configurations in the batch, compute a contextual improvement probability, cPI, score; and determine that a percentage of candidate configurations having the cPI score less than the target criterion is greater than the batch percentage criterion.

11. The system of claim 10, wherein, the cPI score is determined as a cumulative distribution function, CDF, of a standard normal distribution of a combination of: a mean of variances contained within a sampled posterior distribution, a predicted value from a candidate, and a best candidate value found among iterations of the BBO.

12. The system of claim 8, wherein, the hardware processor is further configured to: in response to determining that the stopping criterion is met, send, to the industrial process, a best one of the batches of configurations found in the searching.

13. The system of claim 8, wherein, the hardware processor is further configured to: in response to determining that the stopping criterion is met, send, to a user via a user interface, a best one of the batches of configurations found in the searching.

14. The system of claim 9, wherein, the target criterion and the batch percentage criterion are configurable.

15. A computer program product comprising a computer readable storage medium storing program instructions readable by a device to cause the device to perform the operations of: searching for a batch of candidate configurations for use by an industrial process, the batch of candidate configurations searched for by performing batch Bayesian optimization, BBO, the candidate configurations including concentrations of ingredients, mixing rates of ingredients, temperatures at which to mix ingredients, mixing times; sending the batch of candidate configurations to the industrial process for use in running the industrial process, wherein the industrial process includes at least one robot associated with the industrial process, the robot performing a task in accordance with the batch of candidate configurations; receiving results of the running from the industrial process; searching, in the BBO, for a next batch of candidate configurations using the results; determining whether a stopping criterion is met based on the next batch of candidate configurations and by applying a function to BBO harvest scores; in response to determining that the stopping criterion is met, terminating the search for the next batch of candidate configurations and controlling the robot to stop performing the task; and in response to determining that the stopping criterion is not met, sending the next batch of candidate configurations to the industrial process for use in running the industrial process, and repeating the searching for a next batch of candidate configurations in the BBO using the results and determining whether the stopping criterion is met.

16. The computer program product of claim 15, wherein, so that the device applies the function by evaluating data associated with the next batch of candidate configurations based on a target criterion and a batch percentage criterion, the target criterion representing a level of statistical significance required to terminate the search, the batch percentage criterion representing how many out of a batch need to fail to terminate the search.

17. The computer program product of claim 16, wherein, To determine whether a stopping criterion is met based on the next batch of candidate configurations, the device is caused to: for each of the candidate configurations in the batch, compute a contextual improvement probability, cPI, score; and determine that a percentage of candidate configurations having the cPI score less than the target criterion is greater than the batch percentage criterion.

18. The computer program product of claim 17, wherein, The cPI score is determined as a cumulative distribution function, CDF, of a standard normal distribution of a combination of: a mean of variances contained within a sampled posterior distribution, a predicted value from a candidate, and a best candidate value found among iterations of the BBO.

19. The computer program product of claim 15, wherein, The device is also caused to: in response to determining that the stopping criterion is met, send to the industrial process a best one of the multiple batches of configurations found in the search.

20. The computer program product of claim 16, wherein, The target criterion and the batch percentage criterion are configurable.

Images