Integrating machine learning into control systems

Abstract

Methods, systems, apparatus, and computer program products for implementing machine learning within a control system are disclosed. An industrial facility settings board can be received from a machine learning system, and a determination can be made as to whether to adopt settings in the industrial facility settings board. The machine learning model can be a neural network, e.g., a deep neural network, that has been trained, e.g., using reinforcement learning, to predict data settings boards that are predicted to optimize efficiency of a data center.

Description

TECHNICAL FIELD

[0001] This specification relates to integrating machine learning into control systems. BACKGROUND

[0002] A machine learning model receives input and generates output based on the input received by the model and values of parameters of the model.

[0003] A neural network is a type of machine learning model that uses one or more layers of nonlinear units to predict an output for a received input. Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as input to the next layer in the network, i.e., the next hidden layer or the output layer. Each layer of the network generates an output from a received input based on current values of a respective set of parameters.

[0004] A neural network can be trained to generate predicted outputs using reinforcement learning. Typically, in reinforcement learning training techniques, a reward is received and used to adjust the parameter values of the neural network.

[0005] For example, a neural network trained using reinforcement learning can propose settings for an industrial facility, such as a data center, which is a facility that houses computer servers for the remote storage, processing, or distribution of large amounts of data. SUMMARY

[0006] This specification generally describes techniques for machine learning systems and, in particular, to systems and methods for directly controlling the physical infrastructure of an industrial facility using a machine learning system.

[0007] Generally, one innovative aspect of the subject matter described in this specification can be embodied in methods for controlling the physical infrastructure of an industrial facility using machine learning.

[0008] Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods. The systems of one or more computers to be configured to perform particular operations or actions mean that the systems have installed on them software, firmware, hardware, or a combination thereof that, when operated in conformance with the software, firmware, or hardware, causes the system to perform the operations or actions. The one or more computer programs to be configured to perform particular operations or actions mean that the one or more programs include instructions that, when executed by a data processing apparatus, cause the apparatus to perform the operations or actions.

[0009] The foregoing and other embodiments can each optionally include one or more of the following features, alone or in combination. In particular, one embodiment includes all of the following features in combination.

[0010] An example method includes: receiving from a machine learning system an industrial facility setup predicted by the machine learning system to optimize the efficiency of an industrial facility, wherein an industrial facility setup board defines a corresponding setup for each of a plurality of industrial facility controls; determining whether the industrial facility setup defined by the industrial facility setup board can be safely adopted by the industrial facility (e.g., whether the industrial facility will operate according to predetermined criteria for determining the safety environment of the industrial facility if the setup predicted by the predictive model is adopted); and adopting the industrial facility setup defined by the industrial facility setup board in response to determining that the industrial facility setup can be safely adopted.

[0011] Here, the term "optimization" is used to mean improving the efficiency of an industrial facility relative to an efficiency criterion. "Efficiency optimization" does not necessarily mean that the setting defined by the industrial facility setup board provides the absolute maximum efficiency relative to all possible values ​​of the setting; rather, the term may mean that the efficiency according to the efficiency criterion is higher for this industrial facility setup board than for at least one other possible industrial facility setup board. Specifically, the term "efficiency optimization" can mean that the industrial facility setup board provides an efficiency according to the efficiency criterion that is not less than the corresponding efficiency derived for several other possible industrial facility setup boards.

[0012] The term "industrial facility" can be defined as a physical entity ("physical infrastructure") that includes one or more physical units (e.g., machines, computer equipment, or other equipment) arranged to act on (e.g., generate, modify, or rearrange) any of the following: (i) data, (ii) at least one communication signal, (iii) at least one power signal, and / or (iv) multiple physical elements. An industrial facility can generate data / signals / physical elements for a large number of individuals (e.g., at least 100, typically thousands) who typically do not own the industrial facility. Industrial facility control can include control parameters for modifying the physical operation of physical units and / or control parameters for modifying the operation of additional equipment (e.g., cooling equipment) used to maintain the operational state of physical units. Physical units can be located in a geographical location (e.g., within a building), but may also be geographically distributed. An "industrial facility" can be, for example, a data center, and a physical unit can include servers for processing data received by the data center to generate modified data sent from the data center. Alternatively, an "industrial facility" can be a manufacturing or distribution center, and physical units can include means for acting on physical elements to modify them and / or assemble them and / or distribute them to locations outside the industrial facility. Alternatively, an "industrial facility" can be a station for generating remote communication signals (e.g., broadcast or multicast), a power generation facility for generating power signals, or a laboratory for inspecting and / or modifying physical elements to generate data.

[0013] The second embodiment may be a controller that performs the corresponding operations of the example method.

[0014] The second embodiment can be expressed as a system including the following parts:

[0015] One or more computers; and

[0016] One or more storage devices that store instructions, which, when executed on one or more computers, are operable to cause one or more computers to perform the following actions:

[0017] The machine learning system receives an industrial facility setup board that predicts the efficiency of the industrial facility to be optimized, wherein the industrial facility setup board defines the corresponding settings for each of the multiple industrial facility controls.

[0018] Determine whether the industrial facility setup defined by the industrial facility setup board can be safely adopted by the industrial facility; and

[0019] In response to determining that the industrial facility settings defined by the industrial facility settings panel can be safely adopted, the industrial facility settings defined by the industrial facility settings panel are adopted.

[0020] The second embodiment can also be represented as a computer program product (e.g., one or more non-transitory computer-readable storage media) including instructions (e.g., stored on a medium), which can be executed by a processing device and, upon such execution, cause the processing device to perform the following operations:

[0021] The machine learning system receives an industrial facility setup board that predicts the efficiency of the industrial facility to be optimized, wherein the industrial facility setup board defines the corresponding settings for each of the multiple industrial facility controls.

[0022] Determine whether the industrial facility setup defined by the industrial facility setup board can be safely adopted by the industrial facility; and

[0023] In response to determining that the industrial facility settings defined by the industrial facility settings board can be safely adopted, the industrial facility settings defined by the industrial facility settings board are adopted.

[0024] The second embodiment can also be represented as a device for controlling physical infrastructure in an industrial facility, the device comprising:

[0025] The controller that performs the following corresponding operations:

[0026] The machine learning system receives an industrial facility setup board that predicts the efficiency of the industrial facility to be optimized, wherein the industrial facility setup board defines the corresponding settings for each of the multiple industrial facility controls.

[0027] Determine whether the industrial facility settings defined in the industrial facility setting board can be safely adopted by the industrial facility; and

[0028] In response to determining that the industrial facility settings defined by the industrial facility settings board can be safely adopted, the industrial facility settings defined by the industrial facility settings board are adopted.

[0029] The third embodiment may be a system comprising: a machine learning system that receives state data characterizing the state of an industrial facility board and predicts an industrial facility setup board that will optimize the efficiency of the industrial facility, wherein the industrial facility setup board defines a corresponding setup for each of a plurality of industrial facility controls; a controller that determines whether the industrial facility setup defined by the industrial facility setup board can be adopted by the industrial facility; and, in response to determining that the industrial facility setup defined by the industrial facility setup board can be safely adopted, adopting the industrial facility setup defined by the industrial facility setup board; and an agent that facilitates (e.g., makes possible) a communication path between the machine learning system and the controller.

[0030] A proxy can include physical components (e.g., a physical interface to a communication network) and / or communication protocols. In some implementations, the proxy is part of the system. In other implementations, the proxy communicates with the system.

[0031] In response to the determination that the industrial facility cannot be safely equipped with industrial facility settings, settings provided by the default control system can be used for the industrial facility.

[0032] The default control system can be a rule-based control system. Determining whether an industrial facility setting defined by the industrial facility setting board can be safely adopted may include: determining whether each industrial facility setting defined by the industrial facility setting board falls within the acceptable range or rate of variation of that setting.

[0033] Determining whether an industrial facility setup defined by an industrial facility setup board can be used safely may include determining whether predictions received from a machine learning system have become unstable or erroneous.

[0034] Determining whether a prediction received from a machine learning system has become unstable or erroneous may include determining, for each industrial facility control, whether the rate of change of the most recently predicted setting used for that control has met a threshold. The term "most recently predicted" may refer to a setting generated less than a certain (e.g., predefined) period prior to the determination.

[0035] Determining whether predictions received from a machine learning system have become unstable or erroneous may include determining, for each industrial facility control, whether the variance of the settings for the most recent predictions used for that industrial facility control has met a threshold.

[0036] Determining whether predictions received from a machine learning system have become unstable or erroneous can include determining that the predictions are completely incorrect, leading to inefficient or malfunctioning operations of industrial facilities.

[0037] Before adopting an industrial facility setup, status data characterizing the current state of the industrial facility can be received. This status data can be used to determine whether it is safe to adopt the industrial facility setup defined by the industrial facility setup board. This determination may include determining whether the current state of the industrial facility is suitable for adopting the industrial facility setup.

[0038] Determining whether the current state of an industrial facility is suitable for industrial facility setup may include determining whether any sensor readings identified in the status data fall outside the (e.g., predetermined) acceptable range of the sensors.

[0039] It can be determined that no communication has been received from the machine learning system for a period of time exceeding a threshold, and in response, the industrial facility can be controlled using the default control system for the industrial facility.

[0040] After using industrial facility settings to generate new predictive data dashboards, state data characterizing the updated state of the industrial facilities can be sent to a machine learning system.

[0041] Machine learning systems can include machine learning models, which are neural networks. These models can be deep neural networks. Reinforcement learning can be used to train neural networks based on the efficiency of measurements or computations at industrial facilities.

[0042] The subject matter described herein may be implemented in certain embodiments to achieve one or more of the following advantages.

[0043] Machine learning systems can automatically select process control setpoints within industrial facilities to optimize a desired objective function. For example, in a data center, setpoints can be selected to optimize power or other resource (e.g., water in the system) utilization efficiency, machine health, and central processing unit utilization. For power plants, setpoints can be selected to optimize total power output and heat rate. For manufacturing facilities, setpoints can be selected to optimize output, revenue, and product quality.

[0044] Although this specification provides examples in the context of data centers, the described techniques are equally applicable to any type of industrial facility, such as data centers, power plants, and manufacturing facilities. Therefore, the described techniques can generally be used to improve the operation of industrial facilities.

[0045] Using machine learning models that predict favorable safety settings, the system can select settings without user input or extensive testing. As the operating status or configuration of industrial facilities and their operating conditions change, the system can continuously optimize efficiency over time.

[0046] Details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages of this subject matter will become apparent from the specification, drawings, and claims. Attached Figure Description

[0047] Figure 1 An example efficiency management system is shown.

[0048] Figure 2 An example of a control system with machine learning integration is shown.

[0049] Figure 3 This is a flowchart of an example process for controlling the setup of industrial facilities.

[0050] The same reference numerals and names in the accompanying drawings represent the same elements. Detailed Implementation

[0051] This specification generally describes control systems integrated with machine learning systems that provide direct control over industrial facility infrastructure to improve facility efficiency. For example, the machine learning system can select settings for resources within the industrial facility that optimize one or more of factors such as power usage efficiency, machine health, central processing unit utilization, and thermal margin. These settings can optimize the efficiency of all facilities or only a specified portion of the facility (e.g., a subset of machinery within the facility).

[0052] In industrial facilities, many possible combinations of hardware (such as mechanical and electrical equipment) and software (such as control strategies and setpoints) contribute to facility efficiency. For example, one of the primary sources of energy use in industrial facility environments is cooling. Industrial facilities generate heat, which must be removed to keep the facilities running. Cooling is typically accomplished by large industrial equipment such as pumps, coolers, and cooling towers.

[0053] However, a simple change to the cold aisle temperature setpoint will introduce load variations in the cooling infrastructure of an industrial facility, such as coolers, cooling towers, heat exchangers, and pumps. These load variations lead to non-linear changes in equipment efficiency. The number of possible operating configurations between equipment in an industrial facility, the various feedback loops, equipment operation, and the industrial facility environment make it difficult to optimize energy efficiency. Testing every combination of features to maximize industrial facility efficiency is impractical due to time constraints, frequent fluctuations in industrial facility sensor information and weather conditions, and the need to maintain a stable industrial facility environment. Traditional engineering formulas used for predictive modeling often produce large errors because they fail to capture the complex interdependencies of systems within an industrial facility.

[0054] A machine learning system receives state data characterizing the current state of an industrial facility and provides updated industrial facility settings to a control system that manages the settings of the industrial facility. The machine learning system can be, for example, a machine learning system described below: U.S. Patent Application No. 15 / 410,547, filed January 19, 2017, entitled "Optimizing Data Center Controls Using Neural Networks," the entire contents of which are incorporated herein by reference.

[0055] Figure 1 An exemplary efficiency management system 100 is shown. The efficiency management system 100 is an example of a system in which the systems, components and technologies described below can be implemented as computer programs on one or more computers in one or more locations.

[0056] The efficiency management system 100 receives status data 140 representing the current status of the data center (or other industrial facility) 104 and provides updated settings 120 to the control system 102 that manages the settings of the data center 104.

[0057] The efficiency management system 100 can take in status data 140, representing the current status of a data center (or other industrial facility) 104, as input. This status data can come from sensor readings from sensors within the data center 104 and operational scenarios within the data center 104. The status data may include any one or more data points such as temperature, power, pump speed, and setpoint.

[0058] The efficiency management system 100 uses this data to determine the data center settings 120 that should be changed in the data center 104 to make the data center 104 more efficient.

[0059] Once the efficiency management system 100 determines a data center setting 120 that will make data center 104 more efficient, the efficiency management system 100 provides the updated data center setting 120 to the control system 102. The control system 102 uses the updated data center setting 120 to set one or more data center values ​​(control values) used to control the data center. For example, if the efficiency management system 100 determines that additional cooling towers should be turned on in data center 104, the efficiency management system 100 can provide the updated data center setting 120 to the user or control system 102, which automatically adopts the setting without user interaction. The control system 102 can then send a signal to the data center to increase the number of cooling towers powered and operational in data center 104.

[0060] If a specific data center setup is used, the efficiency management system 100 can use the model training subsystem 160 to train a set of machine learning models 132A-132N to predict the resource efficiency of the data center 104. In some cases, if a specific data center setup is used, the efficiency management system 100 can train a single machine learning model to predict the resource efficiency of the data center.

[0061] Specifically, each machine learning model 132A-132N in the set is configured by training to receive a state input characterizing the current state of the data center 104 and a data center setup board defining combinations of possible data center settings, and if the data center settings defined by the data center setup board are adopted, the state input and the data center setup board are processed to generate an efficiency score characterizing the predicted resource efficiency of the data center.

[0062] In some implementations, if data center 104 uses a specific board configuration, the efficiency score represents the data center's predicted power usage efficiency (PUE). PUE is defined as the ratio of total building energy usage to information technology energy usage.

[0063] In some implementations, if data center 104 uses a specific board configuration, the efficiency score represents the data center's projected water consumption. In other implementations, the efficiency score represents the projected monetary amount spent on electricity. In still other implementations, the efficiency score represents the projected load that the data center can achieve.

[0064] In some implementations, each machine learning model (132A-132N) is a neural network, such as a deep neural network, which the efficiency management system 100 can train to produce efficiency scores.

[0065] A neural network is a machine learning model that uses one or more layers to generate outputs, such as one or more classifications, for a given input. In addition to the output layer, a deep neural network includes one or more hidden layers. The output of each hidden layer is used as input to the next layer in the network (i.e., the next hidden layer or output layer). Each layer of the neural network generates an output from the received input based on the current values ​​of its corresponding parameter set.

[0066] The model training subsystem 160 uses historical data from data center 104 to create different datasets of sensor data from the data center. Each machine learning model 132A-132N in the collection of machine learning models can be trained on one of the datasets of historical sensor data.

[0067] If data center 104 adopts certain data center settings 102, the efficiency management system 100 can use the model training subsystem 160 to train an additional set of constrained machine learning models 112A-112N to predict the operational attributes of the data center corresponding to operational constraints.

[0068] If the efficiency management system determines that the constraint model predicts that a given data center setting will violate the data center's constraints, the efficiency management system will discard the violating setting.

[0069] Each constraint model 112A-112N is a machine learning model, such as a deep neural network, trained to predict certain values ​​of operational attributes of the data center over a period of time if the data center is given input settings. For example, model training subsystem 160 can train a constraint model to predict the future water temperature of the data center in the next hour, given input state data 140 and potential settings. Given state data 140 and potential settings, model training subsystem 120 can train another constraint model to predict the water pressure in the next hour.

[0070] The setting management subsystem 110 within the efficiency management system 100 preprocesses the status data 140 and constructs a set of setting boards, which represent one or more (typically multiple) data center setting values ​​that can be set for various parts of the data center given the known operating conditions and current state of the data center 104. Each setting board defines a corresponding combination of possible data center settings that affect the efficiency of the data center 104.

[0071] For example, the efficiency management system 100 can determine the most resource-efficient settings for the cooling system of the data center 104. The cooling system may have the following architecture: (1) the server heats the air on the server base plate; (2) the air circulates, and the heat is transferred to the treatment water system; (3) the treatment water system circulates and is thermally connected to the condensate system; and (4) the condensate system obtains heat from the treatment water system and uses a cooling tower or large fan to transfer the heat to the outside air.

[0072] To effectively control the cooling system, the efficiency management system 100 can construct various potential setting boards, including settings for different temperatures at the cooling tower setpoint, cooling tower bypass valve positions, cooling unit condenser pump speeds, the number of operating cooling units, and / or handling water pressure differential setpoints. As an example, a setting board might include the following values: a cooling tower setpoint temperature of 68 degrees Celsius, a cooling tower bypass valve position of 27 degrees Celsius, a cooling unit condenser pump speed of 500 rpm, and a number of operating cooling units of 10.

[0073] Other examples of board settings that affect the efficiency of data center 104 include: potential power usage across different parts of the data center; certain temperature settings throughout the data center; given water pressure; specific fan or pump speeds; and the number and type of data center equipment such as cooling towers and water pumps in operation.

[0074] During preprocessing, the setup board management subsystem 110 can modify the status data 140. For example, it can remove data within invalid power usage efficiency, replace missing data for a given data setting with the average value of that data setting, and / or remove a certain proportion of data settings. The setup board management system 110 discretizes all action dimensions and generates an exhaustive set of possible action combinations. For any continuous action dimension, the system converts the action into a set of discrete possible values. For example, if one of the action dimensions is a valve with values ​​from 0.0 to 1.0, the system can discretize the values ​​into a set [0.0, 0.05, 0.1, 0.15, ..., 1.0]. The system can discretize each dimension, and the complete set of actions is every possible combination of values. The system then removes all actions that violate the constraint model.

[0075] The setup board management subsystem 110 sends a constructed set of setup boards and the current state of data center 104 to constraint models 112A-112N. The setup board management subsystem then determines whether certain data center setup boards (if selected by the system) are predicted to cause a violation of the data center's operational constraints. The setup board management subsystem 100 removes any data center setup boards from the set of setup boards predicted to violate the data center's constraints.

[0076] The efficiency management system 100 sends an updated set of settings boards and status data 140 to a collection of machine learning models 132A-132N, which use the status data and settings boards to generate an efficiency score as output.

[0077] Because each machine learning model in the ensemble of models 132A-132N is trained on a different dataset than the other models, each model may potentially provide different predicted PUE outputs when all machine learning models in the ensemble are run with the same data center settings as input. Additionally or alternatively, each machine learning model may have a different architecture, which could also potentially allow each model to provide different predicted PUE outputs.

[0078] The efficiency management system 100 can select data center settings that focus on the long-term efficiency of the data center. For example, some data center settings provide long-term power usage efficiency for the data center, such as ensuring that power usage in the data center is effective for a long, predetermined period of time after the data center enters a state characterized by an input state. Long-term power usage efficiency can be a duration of at least ten minutes (e.g., thirty minutes, an hour, or longer) from the time the data center enters the input state, while short-term power usage efficiency focuses on a short period of time (e.g., less than ten minutes) after the data center enters the input state (e.g., immediately after five seconds).

[0079] This system can optimize machine learning models for long-term efficiency so that the models can predict based on the dynamics of the data center and are less likely to provide board settings that produce good results in the short term but are bad for long-term efficiency. For example, assuming that optimal actions will continue to be taken every hour, the system can predict the PUE for the next day. Then, even if the PUE for a given hour is worse than the previous hour, the system can take actions that it knows will result in the optimal PUE across the entire dataset.

[0080] The efficiency management system 100 determines the final efficiency score of a given setup based on the efficiency score of each machine learning model in the model set of a given setup, in order to produce a total efficiency score for each setup.

[0081] The efficiency management system 100 then recommends or selects new values ​​for data center settings based on the efficiency scores assigned to each board from machine learning models 132A-132N. The efficiency management system can, for example, send recommendations to the data center operator by presenting them on the user's computer, or it can automatically set the settings without sending them to the data center operator.

[0082] In some implementations, the machine learning system can be a cloud-based artificial intelligence system. An agent can exist between the machine learning system and the control system, allowing the control system to communicate with the cloud-based AI, for example, via a telecommunications system. The agent sends recommended industrial facility settings from the machine learning system to the control system. The agent can use communication protocols (such as Modbus) to facilitate communication.

[0083] In some cases, the use of Figure 1 Predictions generated by machine learning systems can lead to instability or hazardous situations in data centers. In such cases, control systems should be used to determine safe settings that can be adopted in data centers or industrial facilities without causing the data center or industrial facility environment to become hazardous or unstable.

[0084] Figure 2 An example of such a control system 202 is shown. Control system 202 is an example of a system in which the systems, components, and techniques described below can be implemented as computer programs on one or more computers at one or more locations.

[0085] The control system 202 receives status data 240 describing the industrial facility 204. The status data 240 may come from sensor readings from sensors within the industrial facility 204 and operational scenarios within the industrial facility 204. The status data may include data such as temperature, power, pump speed, and setpoint. Figure 2 As shown, the control system is located within industrial facility 204. However, in some implementations, the control system may be separate from industrial facility 204 and communicate with the industrial facility via an agent or some other communication mechanism.

[0086] The control system 202 can send status data 240 to the machine learning system 200. The system 202 receives proposed industrial facility settings 220 and, optionally, receives a heartbeat signal 260 from the machine learning system 200. The industrial facility settings 220 are settings determined by the machine learning system 200 to make the industrial facility 204 more efficient relative to one or more metrics that the machine learning system has been trained and optimized for. These settings can be in the form of a settings panel defining setting values ​​for each industrial facility control setting.

[0087] For example, a cooling system can be configured for industrial facility 204, as described above. The cooling system may have the following architecture: (1) a server heats the air on the server base plate; (2) the air is circulated, and the heat is transferred to the treatment water system; (3) the treatment water system circulates and is thermally connected to the condensate system; (4) the water condenser system obtains heat from the treatment water system and transfers that heat to the outside air using a cooling tower or large fan. The configuration may include various temperatures for the cooling tower setpoint, cooling tower bypass valve positions, cooling unit condenser pump speeds, the number of operating cooling units, and / or the treatment water differential pressure setpoint.

[0088] Efficiency can be measured according to one of several cost functions, including: optimized power or other resource (e.g., water in the system) utilization efficiency, machine health, central processing unit utilization, and thermal margin. The heartbeat signal 260 verifies communication between the control system 202 and the machine learning system 200's real-time data network.

[0089] The control system 202 can use updated industrial facility settings 220 from the machine learning system 200 to set values ​​for the industrial facility 204. For example, if the machine learning system 200 determines that another cooling tower should be turned on in the industrial facility 204, the machine learning system 200 can provide the updated industrial facility settings 220 to the control system 202. Figure 2 As shown, the control system 202 communicates directly with the machine learning system 200. However, in some implementations, the control system communicates with the machine learning system through an agent or some other communication mechanism.

[0090] The controller 222 of the control system 202 determines whether the settings are safe to be adopted automatically without user interaction 202. If the settings are safe to adopt, the control system 202 sets the control of the industrial facility 204 to the same settings 225 as the industrial facility settings 220 received by the control system 202 from the machine learning system 200. For example, the control may be electronically configurable and communicatively coupled to the control system 202, for example, via a wired or wireless connection. The control system 202 may then send a signal that causes the control to be set to the same settings 225 as the industrial facility settings 220.

[0091] In this example, if the control system 202 determines that the setup is safe to adopt, the control system 202 may send a signal to the industrial facility 204 to increase the number of cooling towers that are powered and functioning in the industrial facility 204.

[0092] However, if controller 222 determines that the settings received from machine learning system 200 are unsafe to adopt, control system 202 can keep the industrial facility in the last known good industrial facility settings and begin adopting the settings provided by default control system 232 for industrial facility 204. The controller then sends the default control system 232 settings as industrial facility settings 225 to industrial facility 204.

[0093] Figure 3 An example flowchart is shown showing how controller 222 determines whether to use industrial facility settings from machine learning system 200 or default control and settings from default control system. In some cases, default control system uses rules and heuristics to set industrial facility values; that is, the default control system selects settings in a hard-coded manner and does not use machine learning.

[0094] Controller 222 uses one or more criteria to determine whether to use the industrial facility settings from the machine learning system or the default control and settings. The controller can check if the machine learning mode, which distinguishes between the industrial facility settings using the machine learning system and the default settings from the default control system, is disabled. The machine learning mode can be disabled manually by the industrial facility operator or by the machine learning system through mode settings. If the machine learning mode is disabled, the controller will enter the default mode and use the default control system to set the industrial facility settings. The controller will also enter the default mode and use the default control system settings when equipment failure occurs in the industrial facility. The controller may also revert to default control if communication between the controller and other controllers or devices is lost.

[0095] When the controller 222 is in machine learning mode, the controller can determine whether the industrial facility setup proposed by the machine learning system is safe to implement in the industrial facility.

[0096] like Figure 3 As shown, the example controller 222 receives industrial facility settings 220 from the machine learning system and status data including sensor data 345 from the industrial facility. The controller 222 may also optionally receive a heartbeat signal 260 from the machine learning system.

[0097] Controller 222 determines whether the industrial facility can safely adopt the industrial facility settings proposed by machine learning system 302. The controller can perform this determination by comparing each received industrial facility setting with a predefined acceptable value or range of values ​​for that setting. The controller determines whether each setting is an appropriate value or falls within the acceptable range of the setting. If the setting is within the acceptable value or range, the controller determines that the industrial facility can safely adopt the settings proposed by the machine learning system. For example, the industrial facility settings proposed by the machine learning system may include settings indicating that additional cooling towers should be turned on in the industrial facility. The acceptable number of cooling towers to be turned on in the industrial facility at a given time could be 10. If 10 are currently turned on, the setting indicating that additional cooling towers should be turned on would result in 11 cooling towers being turned on in the industrial facility at a given time. Since 11 is greater than the defined appropriate value of 10, the controller determines that the industrial facility cannot safely adopt the industrial facility settings.

[0098] Additionally or alternatively, in some implementations, the industrial facility determines whether the industrial facility setup proposed by the machine learning system is safe to adopt by determining whether the predictions received from the machine learning model have become unstable. For example, stability can be determined by the rate of change or variance of the setup values. If the industrial facility setup changes and / or varies by at least a predetermined threshold amount, the setup value is considered unstable. The system calculates the rate of change or variance between the most recent predicted values ​​of the setup, as calculated by the machine learning system. If the rate of change or variance exceeds a defined threshold, the system determines that the predictions from the machine learning system regarding the setup values ​​have become unstable.

[0099] When it is determined that the industrial facility setup is unstable, controller 222 can keep the industrial facility at the last known good value and switch to default control system 232 for the new industrial facility setup.

[0100] Controller 222 may additionally or alternatively receive state data 240, including data characterizing the current state of the industrial facility, and determine whether the current state of the industrial facility is suitable for adopting an industrial facility setup proposed by the machine learning system. In some cases, given the state of the industrial facility, the proposed industrial facility setup would be an unsafe implementation. This suitability determination can be made by determining whether any sensor readings identified in the state data fall outside the acceptable values ​​or acceptable ranges of the sensors. For example, one of the sensor readings could be the current temperature at a point within the facility. If the current temperature reading exceeds a threshold, the proposed industrial facility is considered to be in an unsafe state for direct control by the machine learning system. Therefore, before adopting the industrial facility setup, the controller maintains the industrial facility at the last known good value and transitions to the default control system for the new industrial facility setup.

[0101] The controller 222 can use an optional heartbeat signal 260 to determine whether communication exists between the controller and the machine learning system. If it is determined that the controller has not received communication from the machine learning system for a predetermined threshold amount of time 304, the controller will use the default control system 232 of the industrial facility to control the industrial facility 314 and can disable the machine learning control mode 312.

[0102] Embodiments of the subject matter and functional operation described herein may be implemented in digital electronic circuits, in tangibly embodied computer software or firmware, in computer hardware (including the structures disclosed herein and their equivalents), or in one or more combinations thereof. Embodiments of the subject matter described herein may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or one or more combinations thereof. Alternatively or additionally, program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, generated to encode information for transmission to a suitable receiving device for execution by the data processing apparatus.

[0103] The term "data processing apparatus" refers to data processing hardware and includes all kinds of devices, apparatuses, and machines for processing data, such as programmable processors, computers, or multiple processors or computers. The apparatus may also be or further include special-purpose logic circuitry, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). In addition to hardware, the apparatus may optionally include code that creates an execution environment for computer programs, such as code constituting processor firmware, protocol stacks, database management systems, operating systems, or combinations thereof.

[0104] A computer program, also referred to or described as a program, software, software application, app, module, software module, script, or code, can be written in any programming language, including compiled or interpreted languages ​​or declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but does not need to, correspond to a file in a file system. A program may be stored as a portion of a file that holds other programs or data, such as in a markup language document, in a single file dedicated to said program, or in one or more scripts within multiple co-located files (e.g., files storing one or more modules, subroutines, or code portions). A computer program can be deployed to execute on a single computer or on multiple computers located in one location or distributed across multiple locations and interconnected by a communication network.

[0105] The processes and logic flows described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform functions by manipulating input data and generating outputs. The processes and logic flows can also be executed by special-purpose logic circuitry (e.g., FPGA or ASIC), or by a combination of special-purpose logic circuitry and one or more programmable computers.

[0106] A computer suitable for executing computer programs can be based on a general-purpose or special-purpose microprocessor, or both, or any other type of central processing unit (CPU). Typically, the CPU receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are the CPU for executing or implementing instructions and one or more memory devices for storing instructions and data. The CPU and memory may be supplemented by or incorporated into special-purpose logic circuitry. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or operatively coupled to receive data from or transfer data to, or both, said mass storage devices. However, a computer does not necessarily need to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), to name just a few.

[0107] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example: semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0108] To provide interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing devices such as a mouse or trackball through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual, auditory, or tactile feedback; and input from the user can be received in any form, including sound, speech, or tactile input. Additionally, the computer can interact with the user by sending and receiving documents to and from the device used by the user; for example, by sending a webpage to a web browser in response to a request received from a web browser on the user's client device. Furthermore, the computer can interact with the user by sending text messages or other forms of messages to a personal device (e.g., a smartphone), running a messaging application, and subsequently receiving response messages from the user.

[0109] Embodiments of the subject matter described herein can be implemented in a computing system that includes back-end components such as data servers, middleware components such as application servers, front-end components such as client computers with graphical user interfaces, web browsers, or apps, or any combination of one or more such back-end, middleware, or front-end components through which a user can interact with an implementation of the subject matter described herein via the graphical user interface, web browser, or app. Components of the system can be interconnected via digital data communication (e.g., a communication network) of any form or medium. Examples of communication networks include local area networks (“LANs”) and wide area networks (“WANs”) such as the Internet.

[0110] A computing system may include clients and servers. Clients and servers are typically geographically separated and usually interact via a communication network. The client-server relationship is established by means of computer programs running on respective computers and having a client-server relationship with each other. In some embodiments, the server sends data (e.g., HTML pages) to a user device, for example, for the purpose of displaying data to a user interacting with the device as a client and receiving user input from that user. Data generated on the user device, such as the result of user interaction, may be received at the server from the user device.

[0111] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather as descriptions of features specific to particular embodiments of a particular invention. Certain features described in the context of individual embodiments in this specification may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, while features may be described above as functioning in certain combinations and even initially claimed in this way, one or more features from a claimed combination may be removed from the combination in some cases, and the claimed combination may involve sub-combinations or variations thereof.

[0112] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order or sequential sequence shown, or that all of the shown operations be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of the various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0113] Therefore, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. For example, the actions described in the claims can be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific order or sequence shown to achieve the desired result. In some cases, multitasking and parallel processing can be advantageous.

Claims

1. A method comprising: The control system of the industrial facility receives from a machine learning system a first industrial facility setting board generated by the machine learning system for the industrial facility in a first state at a first time, wherein the machine learning system includes a set of machine learning models, each of which has been trained on a different dataset, and the machine learning system is configured to determine a total efficiency score for a given setting board based on the efficiency score generated by each machine learning model in the set of machine learning models, and wherein the first industrial facility setting board includes a first value for setting multiple industrial facility controls; The control system of the industrial facility determines that the first value in the first industrial facility setting plate, once adopted by the industrial facility in the first state, will not cause instability in the industrial facility; In response to determining that the first value will not cause instability in the industrial facility, the first value in the first industrial facility setting panel is used to control the industrial facility. The control system of the industrial facility receives from the machine learning system a second industrial facility setting board generated by the machine learning system for the industrial facility in a second state at a second time, wherein the second industrial facility setting board includes second values ​​for setting the control of the plurality of industrial facilities; The control system of the industrial facility determines that the second value in the second industrial facility setting plate, once adopted by the industrial facility in the second state, will lead to an unstable condition in the industrial facility; and In response to determining that the second value would lead to an unstable condition in the industrial facility, settings provided by the default control system are used to control the industrial facility.

2. The method according to claim 1, wherein, The first industrial facility setup panel is predicted to optimize the efficiency of the industrial facility over a period of time starting from the first time point.

3. The method according to claim 1, wherein, The default control system is a rule-based control system.

4. The method according to claim 1, wherein, Determining that the second value in the second industrial facility setting panel will lead to an unstable situation includes: Determine whether each of the second values ​​falls within the acceptable range.

5. The method according to claim 1, wherein, Determining that the second value in the second industrial facility setting plate will lead to an unstable condition further includes: Determine whether the predictions received from the machine learning system have become unstable.

6. The method according to claim 5, wherein, Determining whether a prediction received from the machine learning system has become unstable includes: For each of the plurality of industrial facility controls, determine whether the most recently predicted rate of change of the settings of the plurality of industrial facility controls has met a threshold.

7. The method according to claim 5, wherein, Determining whether a prediction received from the machine learning system has become unstable includes: For each of the plurality of industrial facility controls, determine whether the variance of the most recently predicted setting of the plurality of industrial facility controls has met a threshold.

8. The method according to claim 5, wherein, Determining that the second value has led to an unstable situation includes: Determine whether the second state of the industrial facility is suitable for adopting the second value.

9. The method of claim 1, further comprising: Receiving status data characterizing the current state of the industrial facility, the status data including one or more sensor readings obtained by one or more sensors, wherein determining whether the second state of the industrial facility is suitable for adopting the second value includes: Determine whether any sensor readings identified in the status data fall outside the acceptable range of the sensor.

10. The method of claim 1, further comprising: The machine learning system was found to have received no communication for an amount of time exceeding a threshold. as well as In response, the industrial facility is controlled using its default control system.

11. The method of claim 1, further comprising: After the first value has been adopted to generate a new predictive data dashboard, status data characterizing the updated state of the industrial facility is sent to the machine learning system.

12. The method according to any one of claims 1-11, wherein, The machine learning system includes a machine learning model, which is a neural network.

13. The method according to claim 12, wherein, The machine learning model is a deep neural network.

14. The method according to claim 12, wherein, The neural network has been trained using reinforcement learning based on the measured or calculated efficiency of the industrial facility.

15. A system comprising: One or more computers; as well as One or more storage devices storing instructions that, when executed on one or more computers, cause the one or more computers to perform operations, the operations including: The control system of the industrial facility receives a first industrial facility setting panel generated by the machine learning system for the industrial facility in a first state at a first time, wherein the machine learning system includes a set of machine learning models, each of which has been trained on a different dataset. The machine learning system is configured to determine a total efficiency score for a given setting board based on the efficiency score generated by each machine learning model in the set of machine learning models, and wherein the first industrial facility setting board includes a first value for setting multiple industrial facility controls. The control system of the industrial facility determines that the first value in the first industrial facility setting plate, once adopted by the industrial facility in the first state, will not cause instability in the industrial facility; In response to determining that the first value will not cause instability in the industrial facility, the first value in the first industrial facility setting panel is used to control the industrial facility. The control system of the industrial facility receives from the machine learning system a second industrial facility setting board generated by the machine learning system for the industrial facility in a second state at a second time, wherein the second industrial facility setting board includes second values ​​for setting the control of the plurality of industrial facilities; The control system of the industrial facility determines that the second value in the second industrial facility setting plate, once adopted by the industrial facility in the second state, will lead to an unstable condition in the industrial facility; and In response to determining that the second value would lead to an unstable condition in the industrial facility, settings provided by the default control system are used to control the industrial facility.

16. The system according to claim 15, wherein, The first industrial facility setup panel is predicted to optimize the efficiency of the industrial facility over a period of time starting from the first time point.

17. The system according to claim 15, wherein, Determining that the second value in the second industrial facility setting plate will lead to an unstable condition further includes: Determine whether the predictions received from the machine learning system have become unstable.

18. A non-volatile computer-readable storage medium storing instructions, said instructions causing the processing device to perform operations when executed by a processing device, said operations including: The control system of the industrial facility receives from a machine learning system a first industrial facility setting board generated by the machine learning system for the industrial facility in a first state at a first time, wherein the machine learning system includes a set of machine learning models, each of which has been trained on a different dataset, and the machine learning system is configured to determine a total efficiency score for a given setting board based on the efficiency score generated by each machine learning model in the set of machine learning models, and wherein the first industrial facility setting board includes a first value for setting multiple industrial facility controls; The control system of the industrial facility determines that the first value in the first industrial facility setting plate, once adopted by the industrial facility in the first state, will not cause instability in the industrial facility; In response to determining that the first value will not cause instability in the industrial facility, the first value in the first industrial facility setting panel is used to control the industrial facility. The control system of the industrial facility receives from the machine learning system a second industrial facility setting board generated by the machine learning system for the industrial facility in a second state at a second time, wherein the second industrial facility setting board includes second values ​​for setting the control of the plurality of industrial facilities; The control system of the industrial facility determines that the second value in the second industrial facility setting plate, once adopted by the industrial facility in the second state, will lead to an unstable condition in the industrial facility; and In response to determining that the second value would lead to an unstable condition in the industrial facility, settings provided by the default control system are used to control the industrial facility.

19. The non-volatile computer-readable storage medium according to claim 18, wherein, The first industrial facility setup panel is predicted to optimize the efficiency of the industrial facility over a period of time starting from the first time point.

20. The non-volatile computer-readable storage medium according to claim 18, wherein, Determining that the second value in the second industrial facility setting plate will lead to an unstable condition further includes: Determine whether the predictions received from the machine learning system have become unstable.

21. An apparatus for controlling physical infrastructure in an industrial facility, the apparatus comprising: The controller that performs the following corresponding operations: The control system of the industrial facility receives a first industrial facility setting panel generated by the machine learning system for the industrial facility in a first state at a first time, wherein the machine learning system includes a set of machine learning models, each of which has been trained on a different dataset. The machine learning system is configured to determine a total efficiency score for a given setting board based on the efficiency score generated by each machine learning model in the set of machine learning models, and wherein the first industrial facility setting board includes a first value for setting multiple industrial facility controls. The control system of the industrial facility determines that the first value in the first industrial facility setting plate, once adopted by the industrial facility in the first state, will not cause instability in the industrial facility; In response to determining that the first value will not cause instability in the industrial facility, the first value in the first industrial facility setting panel is used to control the industrial facility. The control system of the industrial facility receives from the machine learning system a second industrial facility setting board generated by the machine learning system for the industrial facility in a second state at a second time, wherein the second industrial facility setting board includes second values ​​for setting the control of the plurality of industrial facilities; The control system of the industrial facility determines that the second value in the second industrial facility setting plate, once adopted by the industrial facility in the second state, will lead to an unstable condition in the industrial facility; and In response to the determination that a second value would lead to an unstable condition in the industrial facility, the settings provided by the default control system are used to control the industrial facility.

Images