Computer system and method for providing operating instructions for thermal control of a blast furnace

Abstract

A computer system (100), a computer-implemented method, and a computer program product are provided for training a reinforcement learning model (130) to provide operation instructions for thermal control of a blast furnace. A domain-adaptive machine learning model (110) generates a first domain-invariant data set (22) from historical operation data (21) acquired as a multivariate time series and reflecting the thermal state of each blast furnace (BF1-BFn) in a plurality of domains. A transient model (121) of a generic blast furnace process is used to generate artificial operation data (24a) as a multivariate time series reflecting the thermal state of the generic blast furnace (BFg) for a specific thermal control action (26a). A generative deep learning network (122) generates a second domain-invariant data set (23a) by transferring features learned from the historical operation data (21) to the artificial operation data (24a). The reinforcement learning model (130) processes the combined first and second domain-invariant datasets (22, 23a) to determine (1400) rewards (131) for specific thermal control actions (26a) taking into account a given objective function. In response to the rewards (131), the second domain-invariant dataset is regenerated based on the modified parameters (123-2), and the reward determination is repeated to learn optimized operating instructions for optimized thermal control actions to be applied for each operating state of the one or more blast furnaces.

Description

[Technical Field]

[0001] The present invention relates generally to systems for controlling blast furnaces, and more particularly to methods, computer program products and systems for using machine learning methods to generate operating instructions for blast furnaces. [Background technology]

[0002] Blast furnaces are used to produce molten iron, the raw material for steel. Blast furnaces have highly complex processes that must be modeled due to their dependency on multivariable process inputs and disturbances. The objective is to optimize overall furnace efficiency and stability, molten iron quality, and reduce material and fuel consumption to improve furnace life. Therefore, it is desirable to provide optimized operating instructions for complex production goal definitions. Summary of the Invention [Means for solving the problem]

[0003] This technical problem is solved by the features of the independent claims by training a reinforcement learning (RL) model implemented by a recurrent neural network to provide operation instructions for thermal control of a blast furnace. The operation instructions relate to corresponding thermal control actions. As used herein, a thermal control action refers to any action that affects an actuator for the purpose of providing thermal control to the blast furnace process. Depending on the level of control automation, the operation instructions may be targeted to a human operator to provide guidance for precise control of the blast furnace, or may directly command the blast furnace's thermal controller, which can execute such instructions without human interaction.

[0004] Thereby, real-world (measured) operational data from multiple blast furnaces are used together with a simulation model (transient model) of the blast furnace process to train a recurrent neural network model by reinforcement learning. This can be understood as offline RL model training at the data level and the simulation model level. From the historical recorded data, multiple additional features can be generated to provide better insights for the characterization of the blast furnace process. These features are either phenomena defined by rules implemented from the recorded raw data or predictions of process phenomena available in the form of forecasts provided by machine learning models.

[0005] Once trained, the RL model provides operational command recommendations to the blast furnace's main actuators, such as tuyere and blast setpoints, e.g., pulverized coal injection (PCI) rate (kg / s), blast flow rate (Nm3 / s), and oxygen enrichment (%), and / or blast furnace burden composition and charge setpoints, e.g., coke rate (kg / cycle), basicity, and blast furnace burden distribution. The recommendations, either by a virtual operator (maximum level of autonomy, level 5) or manually by a human operator, ensure that the objective function is optimized when the process is at its thermal equilibrium after implementing the recommendations. The objective is defined by a blast furnace expert and can consist of multiple objectives, such as (1) minimizing fuel consumption, (2) maximizing blast furnace life, (3) minimizing CO2 emissions, and (4) iron quality and quantity to stabilize furnace operation. Each objective is weighted (e.g., by the expert) to define the global objective employed to train the RL model. When the model is trained and deployed during production, it can continuously learn from the deviation between the global objective and the actual objective reached after the recommended operating instructions for thermal control of each blast furnace are executed (online RL model training).

[0006] In one embodiment, a computer-implemented method for training a reinforcement learning model to provide operational instructions for thermal control of a blast furnace is provided. For example, the reinforcement learning model may be implemented by a recurrent neural network.

[0007] A domain-adaptive machine learning model trained by transfer learning processes historical operational data acquired as multivariate time series from multiple blast furnaces across multiple domains. The historical operational data reflects the thermal state of each blast furnace across the multiple domains. Typically, thousands of sensors per blast furnace measure operational parameters, e.g., temperature, pressure, chemical content, etc. Such measured parameters at a particular point in time define the respective thermal state of the blast furnace at that time. Due to the multiple characteristics of each blast furnace (e.g., operation mode, size, input materials (material composition), etc.), two blast furnaces (source blast furnace and target blast furnace) cannot be directly compared without applying a dedicated transformation of the multivariate time series data.

[0008] The domain-adaptive machine learning model generates, as an output, a first domain-invariant dataset that represents any thermal state of a blast furnace regardless of the domain. Historical operation data was typically collected in the past from multiple different blast furnaces (e.g., blast furnaces of different sizes, blast furnaces operated under different conditions, etc.) in response to their respective thermal control operations. Typically, each blast furnace corresponds to a specific domain, but a domain can also be a specific operation of a blast furnace. The domain-adaptive machine learning model is trained to perform a kind of normalization operation on the data obtained from different domains so that the data are ultimately comparable.

[0009] Different transfer learning techniques may be used. For example, a domain-adaptive machine learning model may be implemented by a deep learning neural network using convolutional and / or recurrent layers trained to extract domain-invariant features from historical operation data as a first domain-invariant dataset. In this embodiment, transfer learning is implemented to extract domain-invariant features from historical operation data. Features in deep learning are abstract representations of the characteristics of a specific blast furnace extracted from multivariate time series data generated by the operation of this specific blast furnace. By applying transfer learning, it is possible to extract domain-invariant features from multiple real-world blast furnaces that are independent of a specific furnace (i.e., independent of various domains).

[0010] In an alternative approach, a domain-adaptive machine learning model is trained to learn multiple mappings of corresponding raw data from multiple blast furnaces to a reference blast furnace. The reference blast furnace can be a virtual blast furnace representing a certain average blast furnace, or a real blast furnace. Each mapping is a representation of the transformation of each specific blast furnace to the reference blast furnace. In this approach, the multiple mappings correspond to a first domain-invariant dataset. For example, such a domain-adaptive machine learning model can be implemented by a generative deep learning architecture based on the CycleGAN architecture, which is popular in fake image generation. CycleGAN is an extension of the GAN architecture that involves simultaneously training two generator models and two discriminator models. One generator takes data from a first domain as input and outputs data for a second domain, while the other generator takes data from the second domain as input and generates data for the first domain. The discriminator model is then used to determine how plausible the generated data is and update the generator model accordingly. CycleGAN uses an additional extension of the architecture called cycle consistency. The idea behind this is that the data output by a first generator can be used as input to a second generator, and the output of the second generator should match the original data. The reverse is also true: the output from the second generator can be fed as input to the first generator, and the result should match the input to the second generator.

[0011] Cycle consistency is a concept from machine translation that states that a phrase translated from English to French, translated back from French to English, should be identical to the original phrase. The reverse process should also be true. CycleGAN promotes cycle consistency by adding an additive loss to measure the difference between the generated output of the second generator and the original image, and vice versa. This acts as a regularizer for the generator model, guiding the image generation process in new domains to image translation. To adapt the original CycleGAN architecture to process multivariate time series data from image processing to obtain the first domain-invariant dataset, the following modifications can be implemented by using recurrent layers (LSTM, as an example) combined with convolutional layers to learn the time dependencies of the multivariate time series data, as described in detail in C. Schockaert, H. Hoyez, (2020) "MTS-CycleGAN: An Adversarial-based Deep Mapping Learning Network for Multivariate Time Series Domain Adaptation Applied to the Ironmaking Industry", arXiv:2007.07518.

[0012] The obtained first domain-invariant dataset represents the thermal state of the blast furnace that existed after each thermal control action was applied to each furnace. After domain adaptation, this representation is no longer tied to a specific blast furnace (either in the form of a learned mapping to a reference blast furnace or in the form of extracted common features).

[0013] In parallel, a transient model of the generic blast furnace process is used to generate artificial operating data as a multivariate time series for a specific thermal control action, which reflects the thermal state the generic blast furnace transitions through after the application of that specific control action. The generic blast furnace is a virtual device (similar to the reference blast furnace). The transient model is a transient-based numerical model with appropriate physical, chemical, thermal, and flow conditions to generate plausible artificial data representing the thermal state of the generic blast furnace. The transient model reflects the respective physical, chemical, thermal, and flow conditions of the generic blast furnace and provides a solution for the upward gas flow and downward movement of the structured solid layer in the generic blast furnace while exchanging heat, mass, and momentum transfer.

[0014] The model receives as input parameters the quantity and chemical analysis of the blast furnace charge material, along with hot blast conditions such as temperature, pressure, PCI rate, and oxygen enrichment. The transient model has an energy equation to predict the hot metal temperature, a species equation to calculate the hot metal chemical composition, and gas-phase equations to predict the maximum gas temperature, gas efficiency (Eta CO), and gas pressure. Due to the transient nature of the model, artificial dynamic time series data can be generated by changing the input parameters over time, which resembles the operation of a real-world blast furnace. Advantageously, the transient model can use data ranges for the input parameters that exceed the data ranges covered by historical operating data of real-world blast furnaces. In other words, the parameter ranges for a generic blast furnace can be extended into the operating parameter space that real-world blast furnace operating data cannot cover.

[0015] A typical blast furnace is divided into a finite number of layers across its height. Each layer consists of a single batch of raw materials (e.g., iron ore and coke). These layers represent computational cells over which equations are numerically solved. Boundary conditions for gas-phase properties, such as composition, velocity, and temperature, are defined using a raceway submodel, while boundary conditions for the solid phase are defined as the charge composition at room temperature. Such a raceway model is described, for example, in "Deepak Sau et al., A Reduced-Order Mathematical Model of the Blast Furnace Raceway with and Without Pulverized Coal Injection for Real-Time Plant Application," International Journal of Modeling and Simulation, DOI: 10.1080 / 02286203.2018.1435759, February 2018." Pulverized coal is injected into the blast furnace tuyere to reduce coke consumption and hot metal production costs. Knowledge of the pulverized coal combustion behavior and unburned char accumulation in the blast furnace raceway zone is important. This paper describes a blast furnace low-order raceway model for real-time plant applications. The model is capable of predicting the radial temperature and gas composition profiles in the raceway zone with and without pulverized coal injection (PCI). The effects of all key operating process parameters, such as PCI rate, blast temperature, blast rate, oxygen enrichment, and steam addition, on the raceway combustion behavior, temperature and gas composition profiles, and raceway depth were investigated and verified, where possible, with literature and plant databases.

[0016] Complete decomposition of the gas and solid phases is computationally extremely expensive. Therefore, according to one embodiment, to conserve computational resources (and therefore energy), the gas phase can be considered steady-state because its gas-withstand time (approximately 3 seconds) is much shorter than the time step (approximately 2 minutes). However, the solid phase is considered a transient phase. The solution algorithm first solves the gas-phase equations iteratively and continuously to satisfy the relative parameter tolerances at each time step. When the gas-phase parameters converge to the defined tolerances, the solid-phase equations are continuously solved at the same time step. The time loop continues until the end of the simulation. Gas and solid parameters, as well as transfer parameters such as heat and mass transfer, are updated at the beginning of each time step. Continuously, when one parameter is solved, the other parameters are considered known, meaning that the old values are used. In this way, nonlinear terms and coupling parameters can be solved, avoiding complex and expensive block solvers.

[0017] In one implementation, the transient model has multiple computational cells, each representing a different layer of a typical blast furnace consisting of a single batch of raw materials. Each computational cell iteratively and continuously solves the gas-phase equations to satisfy relative gas-phase parameter tolerances at each time step. When the gas-phase parameters converge to the predetermined tolerances, the solid-phase equations are continuously solved at the same time step.

[0018] Iteratively solving the gas-phase equations involves calculating gas, solid and liquid properties, calculating reaction rates and heat transfer coefficients, and calculating gas temperatures, gas species, gas velocities and gas pressure drops for each iteration of the pressure-velocity correction loop.

[0019] Once the gas phase parameters have converged to the predetermined tolerances, the calculation continues by successively solving the solid phase equations at the same time step, which includes calculating the solid temperature and species, calculating the liquid temperature and species, and calculating the solid velocity.

[0020] The artificial operation data obtained from the transient model is then processed by a generative deep learning network trained on the multivariate time series of historical operation data. This allows the artificial operation data to be augmented with features of real-world operation data to make them more realistic. A properly trained generative deep learning network can augment the artificial data so that the augmented synthetic operation data is indistinguishable from real-world operation data to experts. This is advantageous for training the reinforcement learning model using data with characteristics similar to the real-world test inputs expected when operating the reinforcement learning model in the prediction phase. That is, processing the artificial operation data generates a second domain-invariant dataset augmented with features learned from the historical operation data. The second domain-invariant dataset is simply a synthetic dataset based on calculations of the transient model, but it is nevertheless a domain-invariant dataset that exhibits the unique features present in the time series of real-world historical operation data.

[0021] Next, a reinforcement learning model is trained using the combined first and second domain-invariant datasets. If training relies solely on the first dataset, the reinforcement learning model will not be able to learn optimized control instructions that have not been applied to multiple blast furnaces. By combining such real-world training datasets with artificially generated datasets, the transient model can be used to simulate the response of a general blast furnace to alternative control actions applied to a given thermal state of the general blast furnace under various optimization objectives. When processing the combined first and second domain-invariant datasets, the reinforcement learning model determines the reward for a specific thermal control action used by the transient model to calculate the second invariant dataset given the given objective function and the current state of the blast furnace. Reward functions describe how the reinforcement learning model (i.e., the agent) should behave. In other words, they have normative content and specify what the agent is supposed to achieve. While there are no absolute restrictions, the agent will learn better if the reward function is "better behaved." In practice, this means that the convergence rate increases and the agent does not get stuck in a local minimum. As an example, a reward function can measure "how far" from the Pareto front of a multi-objective function, to which a particular thermal control action is guiding the process. By definition, the Pareto front is the set of non-dominated solutions that are selected as optimal if an objective cannot be improved without sacrificing at least one other objective. For a given objective, a measure of the improvement delta of another objective can be measured, for example, by gradient analysis. The reward function can be a function of those measures that characterize the characteristics of the Pareto front. Those skilled in the art can use other suitable reward functions.

[0022] If the determined reward is below a predetermined minimum reward, the recommended thermal control action (control command) was not optimal with respect to its intended effect on the thermal state of the blast furnace. In such cases, alternative control actions can be simulated by a transient model. To this end, a genetic search and / or Bayesian optimization algorithm guides the search for modified parameters (i.e., input parameters for the transient model) for further (alternative) thermal control actions based on the reinforcement learning model's current environment and the thermal control action output of the current learning step (i.e., the control action that led to too low a reward). The transient model then regenerates a second domain-invariant dataset (an updated second dataset) based on the modified parameters. The updated second dataset is then fed to the input layer of the reinforcement learning model, and a new reward is determined for the updated second dataset. This process is performed iteratively until the reinforcement learning model learns to output optimized operating commands for optimized thermal control actions for any foreseeable situation.

[0023] Once the reinforcement learning model has been trained as described, it can be operated to predict optimized operation commands for at least one actuator of a particular blast furnace in production based on current operational state data for that particular blast furnace. In other words, the trained reinforcement learning model receives test input data that includes operational data consistent with the input layer of the reinforcement learning model and specifies the current (thermal) state of the blast furnace. The model processes the test input data and provides as output a prediction of optimized operation commands corresponding to thermal control actions to be applied to the blast furnace to achieve an optimized result given a given objective function.

[0024] Advantageously, each prediction data set can be used to further improve the training of the reinforcement learning model. To this end, the model applies a thermal control action according to the optimized operation command (predicted output) to at least one actuator, and then determines a reward based on the new state of the particular blast furnace after the thermal control action is performed. If the reward falls below a predetermined threshold, the transient model regenerates second domain-invariant data for one or more alternative operation commands for retraining the reinforcement learning model. This retraining can be applied after the application of any thermal control action according to the respective predicted optimized operation command.

[0025] Advantageously, the reinforcement learning model is trained to learn optimized operational instructions such that the associated objective measures are within a predetermined range from the Pareto front for the corresponding multidimensional objective function.

[0026] In one embodiment, the transient model has multiple computational cells, each representing a different layer of a typical blast furnace consisting of a single batch of raw materials. Each computational cell iteratively and continuously solves the gas-phase equations to satisfy relative gas-phase parameter tolerances at each time step. When the gas-phase parameters converge to the predetermined tolerances, the computational cell continuously solves the solid-phase equations at the same time step.

[0027] [Table 1]

[0028] Table 1 describes five automation levels for blast furnace control. A combination of a recommended reinforcement learning model and additional associated machine learning models that generate advanced context information for process characterization (e.g., process phenomenon prediction, hot metal temperature prediction, etc.) can be used to achieve Level 4 or 5 automation, while the associated machine learning models alone can contribute to Level 2 or 3 automation. Training a recommended model without associated machine learning models can lead to Level 3 automation. The techniques disclosed herein for training a reinforcement learning model to recommend (predict) optimal thermal control actions can be used to achieve Level 4 or 5 automation, assuming the process is accurately represented by a machine learning model for process characterization and advanced context data generated by additional sensors, as described in more detail in the detailed description. Such associated machine learning models can be used to add further data augmentation capabilities that improve the training dataset for reinforcement learning, in that the received operational data (raw sensor data) that serves as additional input for training the reinforcement learning model provides predictions based on more than domain-invariant process data. With such additional "context" information, the reinforcement learning model gains knowledge about new dimensions that can be used to more accurately learn the best behavior for thermal control.

[0029] When using such an associated machine learning model to predict information about the future thermal evolution of a particular blast furnace state based on historical operation data and / or further measured environmental data related to the blast furnace environment, the associated machine learning model needs to be trained accordingly to complement the historical operation data (obtained from sensors) with future multivariate time series data related to future time points. The generated future multivariate time series can then be processed by the domain-adaptive machine learning model in the same way as the historical operation data to extend the first domain-invariant data set with data related to future time points.

[0030] The associated machine learning models may be trained in the following manner: In a first training step, multiple base models are trained using one or more machine learning algorithms with different selections of operational and / or environmental data to provide base model-specific future multivariate time series data as training inputs to specific ones of the machine learning models, whereby each base model focuses on a single, specific aspect of the blast furnace process (e.g., predicting hot metal temperature trends over a given future time interval); In a second training step, the associated machine learning models are trained using the base model-specific future multivariate time series data to learn which combinations of base models are best suited to which states of the blast furnace.

[0031] Additional aspects of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as so described. [Brief explanation of the drawings]

[0032] [Figure 1] FIG. 1 is a simplified diagram of one embodiment of a computer system for training a reinforcement learning model to provide operating instructions for thermal control of a blast furnace. [Figure 2] 1 is a simplified flowchart of a computer-implemented method that may be performed by an embodiment of a computer system. [Figure 3A] 1 is a simplified flowchart reflecting the processing of a transient model having multiple computational cells, each cell representing a respective layer of a generic blast furnace, according to one embodiment. [Figure 3B] FIG. 1 illustrates a computational cell in a visual representation of a blast furnace. [Figure 4] FIG. 1 illustrates an exemplary embodiment of a reward calculation for a reinforcement learning model. [Figure 5] FIG. 1 illustrates the Pareto front as the boundary of a point cloud in objective space for a reward function. [Figure 6A] FIG. 10 shows an example Pareto front in the objective space of the two-dimensional objective function for each blast furnace for visualization purposes. [Figure 6B] FIG. 10 shows an example Pareto front in the objective space of the two-dimensional objective function for each blast furnace for visualization purposes. [Figure 7] FIG. 1 illustrates the use of additional machine learning models for training data augmentation, according to one embodiment. [Figure 8] FIG. 10 illustrates the use of tuyere images for training data augmentation by using additional deep learning models, according to one embodiment. [Figure 9] FIG. 1 illustrates the use of additional sensors to characterize the state of the blast furnace for training a reinforcement learning model. [Figure 10] 1 illustrates an example of a general computing device and a general mobile computing device that may be used with the techniques described herein. DETAILED DESCRIPTION OF THE INVENTION

[0033] Figure 1 shows a simplified diagram of a basic embodiment of a computer system 100 for training a reinforcement learning model 130 to provide operating instructions for thermal control of a blast furnace. Figure 1 is described in the context of Figure 2, which is a simplified flowchart of a computer-implemented method 1000 that may be performed by an embodiment of the computer system 100. Accordingly, the following description of Figure 1 in the context of Figure 2 will refer to reference numerals from both figures.

[0034] In one embodiment, computer system 100 is communicatively coupled to multiple blast furnaces BF1-BFn. Blast furnaces BF1-BFn may provide historical operating data 21, belonging to different domains and captured as multivariate time series, reflecting the thermal state of each blast furnace. Examples of such historical operating data include, but are not limited to, blast furnace charge material quantities and chemical analysis, temperature, pressure, PCI rate, and oxygen enrichment, using an energy equation to predict hot metal temperature, one or more species equations to calculate hot metal chemical composition, and one or more gas-phase equations to predict maximum gas temperature, gas efficiency (Eta CO), and gas pressure.

[0035] In a real-world blast furnace, different domains may be associated with different combinations of parameter values in the historical operating data 21 that describe the thermal state of the blast furnace in the different domains, but similarities exist between such thermal states. Accordingly, the system 100 includes a domain-adaptive machine learning model DAM 110 for generating 1100 a first domain-invariant dataset 22 that represents the thermal state of any of the blast furnaces BF1-BFn, regardless of domain. The DAM 110 is trained using a transfer learning method TL111. In one implementation, the DAM 110 may be implemented by a generative deep learning neural network GDL1 113 using convolutional and / or recurrent layers trained to extract domain-invariant features from the historical operating data 21 as the first domain-invariant dataset 22.

[0036] In an alternative implementation, the DAM 110 may be implemented by a generative deep learning architecture (e.g., based on the CycleGAN architecture previously described) trained to learn multiple mappings 112 of corresponding raw data from multiple blast furnaces BF1-BFn to a reference blast furnace BFr, whereby each mapping is a representation of a transformation of a respective blast furnace (e.g., BF1) to the reference blast furnace BFr. In this implementation, the multiple mappings correspond to a first domain-invariant dataset 22.

[0037] The system 100 includes an artificial data generator module ADG 120 configured to generate 1200 artificial operating data 24a as multivariate time series reflecting the thermal state of a generic blast furnace BFg for a particular thermal control action 26a. To this end, the ADG 120 uses a transient model 121 of the generic blast furnace process. The transient model 121 is a simulation model that reflects the respective physical, chemical, thermal, and flow conditions of the generic blast furnace and provides a solution for the upward gas flow and downward movement of the structured solid layer in the generic blast furnace while exchanging heat, mass, and momentum transfer. Generally, the simulation model is based on simulation parameters that correspond to such real-world state parameters monitored in historical operating data.

[0038] Referring briefly to FIG. 3A, the transient model 121 includes a plurality of computational cells, each representing a different layer of a general blast furnace BFg consisting of a single batch of raw materials. Each computational cell iteratively and continuously solves the gas-phase equations to satisfy the relative gas-phase parameter tolerances at each time step (iteration time interval). Once the gas-phase parameters have converged to the predetermined tolerances, the solid-phase equations are continuously solved at the same time step. The iterative solution of the gas-phase equations includes, for each iteration of the pressure-velocity correction loop: - calculating gas, solid and liquid properties 3300; - calculating reaction rates and heat transfer coefficients 3400; - Calculate gas temperature, gas species, gas velocity, and gas pressure drop may include:

[0039] Solving the solid-state equation continuously is - Calculating the solid temperature and solid species 3600; - Calculating the liquid temperature and liquid type 3700; - Calculate solid velocity 3800 and may include:

[0040] 3B, which shows a computational cell CC in a visual representation 300 of a blast furnace, the transient model 121 may receive one or more of the following input parameters 302: the quantity and chemical analysis of the blast furnace charge material 302-1, the temperature, the pressure, the PCI rate 302-2, and the oxygen enrichment. Furthermore, the furnace profile 302-3 represents the shape of the blast furnace and thus affects the transport time of the charged material (e.g., a tall blast furnace may have an 8-hour transport time, while a short one may only have a 6-hour transport time). The furnace profile 302-3 is a fixed parameter for each blast furnace used for artificial data generation. It will be apparent to those skilled in the art that the transient model takes the blast furnace shape into account. The transient model generates output 303 using, for example, an energy equation to predict hot metal temperature, one or more species equations to calculate hot metal chemistry 303-2, and one or more gas phase equations to predict maximum gas temperature, gas efficiency (Eta CO), and gas pressure (see maximum gas conditions 303-1).

[0041] In other words, a transient (simulation) model is a numerical model with appropriate physical, chemical, thermal, and flow conditions used to generate plausible artificial data. Due to the transient nature of the model, artificial dynamic time series data can be generated by changing input parameters over time, which resembles real-world furnace operation. As a result, the (parameter) data range can be extended to a wide operating space that actual blast furnace data obtained from real-world blast furnaces cannot cover.

[0042] In the transient model, the furnace is divided into a finite number of layers across the furnace height. In FIG. 3B, the various layers are separated by solid horizontal lines 301. Each layer consists of a batch of raw material, in this case, iron ore and coke. These layers represent the previously described computational cells CC 310 over which the equations are numerically solved. In one embodiment, boundary conditions for gas-phase properties, such as composition, velocity, and temperature, are defined using a raceway submodel 320, while boundary conditions for the solid phase are defined as the charge composition at room temperature. The internal state 304 of the blast furnace 300 includes substates for the gas, solid, and liquid phases. The substate for the gas phase can be characterized by temperature (Tg, K), pressure (p, Pa), velocity (Vg, m / s), and species (CO, CO2, H2, HO, N2). Substates for the solid phase can be characterized by temperature (Ts, K), velocity (Vs, m / s), and species (Fe2O3, Fe3O4, FeO, Fe, slag, CokeC, Cokeash), and substates for the liquid phase can be characterized by temperature (Tl, K) and species (Fe, slag, FeO).

[0043] Completely resolving the gas and solid phases is computationally extremely expensive (and time-consuming). Therefore, to save time and energy, the gas phase can be considered steady-state because its gas-tightness time (approximately 3 seconds) is much shorter than the time step defined as the iteration interval (approximately 2 minutes). However, the solid phase is considered a transient phase. The solution algorithm (see Figure 3A) first solves the gas-phase equations iteratively and continuously to satisfy the relative parameter tolerances at each time step. When the gas-phase parameters converge to the predetermined tolerances, the solid-phase equations are solved continuously at the same time step. The time loop continues until the end of the simulation. Gas and solid parameters, as well as transfer parameters such as heat and mass transfer, are updated at the beginning of each time step. After solving one parameter continuously, the other parameters are considered known, meaning that the old values are used. In this way, nonlinear terms and coupling parameters can be solved, avoiding complex and expensive block solvers.

[0044] As described above, the artificially manipulated data 24a generated by the transient model 121 is generated according to a mathematical formula that results in clean data in that the artificially manipulated data 24a does not exhibit real-world characteristics, such as noise or shifts caused by the respective measurement / sensor devices. To train the reinforcement learning model RILM 130 to make more accurate predictions, it is desirable to provide the model 130 with training data that reflects the characteristics of the real-world operational data used as test input for RILM 130. Therefore, the ADG 120 uses a generative deep learning network GDL2 122 with recurrent layers to generate 1300 a second domain-invariant dataset 23a by transferring features learned from the historical operational data 21 to the artificially manipulated data 24a. GDL2 122 was trained on a multivariate time series of the historical operational data 21 to learn the real-world characteristics from the historical operational data and apply the learned patterns to the simulated artificially manipulated data 24a. This results in a purely synthetic dataset 23a reflecting the thermal state of a generic blast furnace BFg in response to thermal control actions 26a. Note that using a generative deep learning network with recurrent layers to learn the natural characteristics of each signal in the context of other signals is similar to well-known techniques applied to images to learn the style of a particular set of drawings and apply that style to any other images. Similar techniques can be applied to multivariate time series, an approach that can be solved, for example, by the CycleGAN-based algorithm mentioned above, when adapted to multivariate time series data.

[0045] Both datasets, i.e., the first domain-invariant dataset 22 and the synthetic second invariant dataset 23, are provided to the RILM 130 as training data. The RILM 130 processes the combined first and second domain-invariant datasets 22, 23a to determine 1400 rewards 131 for specific thermal control actions 26a given a given objective function. Based on such training data, the RILM 130 learns thermal control actions depending on the (general) blast furnace conditions (environment). For example, the environment may be defined by, illustratively, the blast furnace operation, material composition, etc.

[0046] In response to the reward 131, the ADG 120 regenerates (1300) a second domain-invariant dataset based on the modified parameters 123-2. The parameter generator PG 123 uses a genetic search and / or Bayesian optimization algorithm 123-1 to guide the search for modified parameters for further thermal control actions based on the RILM 130's current environment 25a and the thermal control action 26a output of the current learning step. Using the modified parameters, the transient model simulates thermal conditions for further control actions. The regenerated second domain-invariant dataset is then provided to the RILM 130 as a new training input, and the reward is again determined for the new training input. This process is repeated until the current reward exceeds a predetermined reward threshold (1500) to learn optimized operating instructions for optimized thermal control actions.

[0047] Below, an example real-world scenario for reward calculation is described as shown in Figure 4. Note that one skilled in the art may use other suitable reward functions to implement a reinforcement learning model. The following example scenario describes optimization to identify optimal actuator values for a simple two-objective function to be maximized by using a genetic search algorithm. Objective: Maximize quality (constant Si content) and maximize production Actuator: PCI rate (kg / s), air flow rate (Nm3 / s), coke rate (kg / cycle) Rewards: =1 / (Euclidian_dist_to_pareto_front) As an example, the improvement of each objective can be approximated by a delta analysis: 1 / eucl_dist((quality_prev,prod_prev),(quality_new,prod_new))

[0048] In this example, the definition of reward is only valid for a genetic search algorithm that guarantees that there is convergence towards the Pareto front, i.e., in this example of two objective functions to be maximized, both the improvement in quality and production are certain between two successive iterations.

[0049] Initial furnace thermal state (current environment): S_init -Iteration 1: Actuator values = [PCI_1,blast_flow_rate_1,coke_rate_1] objective measure=quality_1;prod_1 -Iteration 2: Actuator values = [PCI_2,blast_flow_rate_2,coke_rate_2] objective measure=quality_2;prod_2 Reward=R_2=1 / eucl_dist((quality_1,prod_1),(quality_2,prod_2)) -Iteration 3: Actuator values = [PCI_3,blast_flow_rate_3,coke_rate_3] objective measure=quality_3;prod_3 Reward=R_3=1 / eucl_dist((quality_2,prod_2),(quality_3,prod_3)) ............................. -Iteration i: Actuator value = [PCI_i,blast_flow_rate_i,coke_rate_i] Objective measure = quality_i;prod_i Reward=R_i=1 / eucl_dist((quality_i-1,prod_i-1),(quality_i,prod_i)) ............................. -Iterative opt: (reaching the Pareto front) Actuator values=[PCI_opt,blast_flow_rate_opt,coke_rate_opt] objective measure=quality_opt;prod_opt Reward=R_opt=1 / eucl_dist((quality_opt-1,prod_opt-1),(quality_opt,prod_opt))

[0050] A time-consuming random search can be performed without using a genetic search algorithm. In Figure 5, the Pareto front in such a case is characterized by points (quality_i, prod_i) with a dotted pattern (at the boundary of the point cloud in the objective space). The reward for each point (quality_i, prod_i) can then be calculated as the inverse of the Euclidean distance, and is calculated after the Pareto front is identified (rather than during the random search process).

[0051] In summary, the reinforcement learning model 130 is trained to learn optimized operational instructions such that the associated objective measures are within a predetermined range from the Pareto front for the corresponding multidimensional objective function.

[0052] Once learning is complete, RILM 130 has been trained to provide 1600 optimized operating instructions for thermal control of a real-world blast furnace in response to test inputs having current operating data describing the current state of said blast furnace (see FIG. 2). Optionally, training of RILM 130 can continue in an online mode while the blast furnace is operating.

[0053] In the online mode, the reinforcement learning model 130 predicts 1700 an optimized operation command for at least one actuator of a particular blast furnace in production based on the current operating state data of the blast furnace (see FIG. 2 ). Assume that a thermal control action for the optimized operation command is applied to the blast furnace (either by an operator or automatically via a corresponding control system). After the thermal control action is applied to the at least one actuator according to the optimized operation command, a reward is then determined based on the new state of the blast furnace reached after the thermal control action is executed. Again, the determined reward is compared 1500 with a predetermined reward threshold. If the reward is below this threshold, the ADG 120 regenerates second domain-invariant data (using the transient model 121) for one or more alternative operation commands to retrain the reinforcement learning model 130.

[0054] Figures 6A and 6B show (for visualization purposes) the Pareto front (dashed line) in the objective space of a two-dimensional objective function (with two objectives O1 and O2) for each blast furnace state BFS1 and BFS2. The RILM model needs to learn optimal control instructions for the blast furnace such that the associated objective measurements lie on the Pareto front. In these figures, the objectives were calculated for each historical and artificial data sample. The figures illustrate the limitation of historical data, which is often limited to only a few operating modes of the blast furnace, leading to clusters in the objective space. Therefore, type 22-2 filled circles are associated with domain-invariant datasets obtained from historical data. Type 23a squares are associated with domain-invariant datasets based on artificial (simulated) data. Type 22-1 filled circles are associated with data generated by a deep generative model trained from historical data 21 (raw data) or 22 (domain-invariant raw data). This deep generative model acts as an advanced interpolation algorithm that provides new raw data generated from historical data. Therefore, the generated data may simply be relatively close to existing historical data. The generation of such data associated with type 23a is described in more detail in FIGS. 1 and 2. In FIG. 6B, the type 22-3 triangle is associated with online data acquired during blast furnace operation and used for the online training mode of RILM 130. Because the type 22-3 triangle results from a trained model that provides optimized operation instruction recommendations (see predicting 1700 in FIG. 2), it is naturally closer to the Pareto front. However, online retraining of RILM 130 is triggered to further optimize the operation instruction recommendations for those data.

[0055] In one embodiment, the system 100 may include a data augmentation module DA140 for augmenting the raw operational data 21 measured by sensors on the blast furnace by using one or more specially trained machine learning models ML1-MLn to predict information about the future thermal evolution of the blast furnace state or any other information related to the current thermal state (e.g., process phenomenon predictions, e.g., virtual sensors providing measurements at a higher frequency than the actual sensors). Such predictions serve the same purpose as the raw data for training the RILM130 model and are used in the same way as the raw data 21 (historical operational data). An example of such a specially trained machine learning model is a model that predicts the temperature of the hot metal after three hours. This prediction of the hot metal temperature can then be used to train RILM130. Such data augmentation further improves the training dataset used for reinforcement training of RILM130, leading to improved prediction accuracy of the reinforcement learning model. Alternatively, new sensors can be added, as illustrated in FIG. 9 . This allows for a more accurate characterization of the blast furnace state for training RILM130. As an example, if some properties of the raw material charged to the furnace are missing (e.g., porosity, humidity), they can be measured (using additional sensors) or potentially estimated using machine learning models ML1 to MLn.

[0056] Listed below is a list of machine learning (ML) models that would be advantageous for data augmentation to enable more accurate characterization of the blast furnace condition and, consequently, more accurate training of RILM130. a) ML for advanced data validation: Anomalies in the raw data provided by blast furnace sensors can be detected before training a machine learning model or can be used as input during the production of a deployed machine learning model. b) ML for predicting blast furnace thermal conditions and hot metal production KPIs (Key Performance Indicators) c) ML for charge matrix optimization d) ML for tuyere camera-based process inspection e) ML for recommending taphole openings for optimal operation f) ML for TMT SOMA-based event detection and KPI calculation / prediction g) ML for phenomenon labeling by process rules defined by process engineers or by supervised or unsupervised machine learning or pattern detection models (potentially using outputs generated by machine learning models). h) ML for predicting phenomena from the labels generated in g) i) ML for process prediction j) ML for predictive and prescriptive maintenance k) ML for Advanced Context Representation Learning: Environmental sensors can be used to train unsupervised deep learning models to learn representations that are used to extend datasets for the required use cases described above.

[0057] Figure 7 illustrates in more detail an approach for implementing DA 140 to train a machine learning model to predict the temperature of the hot metal after three hours. Figure 7 shows the procedure for training 706 a machine learning model MLT to predict the temperature of the hot metal at a future time point (e.g., after three hours) from the predictive BMP 704 of multiple machine learning models, called base models. The base models are trained 703 to generate predictions to augment the measured raw data.

[0058] For this purpose, multiple base models are trained (703) using different selections of variables (process variables 701 and / or context variables 702) and / or machine learning algorithms. The process variables 701 are raw data (operational data) measured by respective sensors directly on the blast furnace. The context variables 702 are measured by any other sensors measuring environmental variables such as noise, images, etc. The process variables and context variables are variables that are available to train the machine learning models.

[0059] Each base model provides outputs 704, 705 that can be used to train an MLT (706) to make better predictions of that parameter than any of the base model's predictions (generally to train any machine learning model to predict parameters other than the hot metal temperature after three hours). The purpose of the base models is to generate additional information for training a more accurate predictive model (i.e., a meta-model such as the MLT). The predictive model MLT also uses the process variables 701 and context variables 702 as inputs to learn which combination of base models is most appropriate for which state of the blast furnace. That is, the meta-model learns how to combine the outputs of all the base models to make more precise and accurate predictions of specific blast furnace state parameters. Some base models may not predict the hot metal temperature after three hours, but they may predict the hot metal temperature trend, for example, whether the temperature is likely to increase, decrease, or stabilize, or the occurrence of a specific process event in the near future. In other words, the base models produce as output additional information that is either related to the process (process information PI 705) or is already a base model prediction BMP 704 about the hot metal temperature or a hot metal temperature attribute (e.g., trend prediction). When the MLT is trained 706 based on the outputs of the various base models, it provides a more accurate prediction MLTP 705 than any of the base models (BMP 704).

[0060] In this example, the process information PI705 may provide input information for the MLT, such as feature predictions in the range [0,6h], including but not limited to clusters, process phenomena, process / context variables, or features. These outputs provide new inputs that are process-related and potentially have a higher correlation to the hot metal temperature predicted by the MLT. The base model prediction BMP704 for hot metal temperature may provide information such as the hot metal temperature trend after 3 and 6 hours (e.g., high increase, moderate increase, low increase, stable, low decrease, moderate decrease, high decrease), or the predicted hot metal production quality during the above time range. The BMP704 is an output of the base model that is directly related to the output of the MLT and is either the same output or a characteristic thereof. An example of the same output would be "hot metal temperature after 3 hours," and an example of a characteristic of that output would be "temperature trend" predicted by the base model.

[0061] Below, some examples from the above list of machine learning models are described in more detail.

[0062] Advanced Data Validation:

[0063] The data validation pyramid may be defined by multiple data validation levels, described below, starting with the lowest level pyramid and ending with the highest level pyramid. - Sensor maintenance and calibration: Procedures for sensor maintenance and calibration can be implemented. Artificial intelligence (AI) can be required to optimally schedule maintenance operations and determine the best actions to be taken to keep the sensors in operational mode as long as possible. -Process min / max for individual sensor signals: The first level of anomaly detection is the definition of the minimum and maximum values allowed for each sensor signal in the raw data. The minimum and maximum values are constant and therefore independent of the process operation. The min / max values for condition-based processes can be configured in rules defined by process experts to provide some context. Outlier and anomaly detection for individual sensor signals: Below, typical methods are listed with increasing complexity. i) Statistical Amplitude Outliers: A data analysis method for detecting point anomalies, which by definition are amplitude-shifted values from the mean within a moving time window of length L, specified by process experts and based on the typical autocorrelation depth of the time series recorded by the sensor. ii) Supervised Anomaly Detection: A supervised algorithm learns known patterns in the sensor signal to detect anomalies. iii) Unsupervised outlier detection: Methods in this category detect outliers by applying clustering algorithms after features have been computed from the sensor signals, and thus such approaches are not limited to unusual amplitude values for a given context, but can also take into account spectral information or any other characteristic defined by the features. Anomaly detection for multi-sensor signals: With a large number of sensors, manual matching between redundant sensor signals is not sufficient to detect complex contextual anomalies in the data. Rule-based approaches are usually limited because only known relationships are verified. The same limitation applies to supervised data-driven models trained to detect known anomalies. Unsupervised data-driven approaches are a complementary validation step to ensure the detection of known and unknown anomalies. Contextual anomalies can be detected by data-driven models that have learned correlations between sensor signals and can therefore detect whether sensor measurements deviate from their normal operation in a given context defined by the process. Machine learning causality discovery combined with unsupervised data-driven anomaly detection enables root cause analysis. - Matching sensor to simulation model results: When a simulation model describing the process is available, matching model results to raw sensor data provides expert-level autonomous data validation. However, this validation is limited to the operating conditions specific to the simulation model assumptions.

[0064] The data validation pyramid aims to detect anomalies in the received operational data (raw data). Anomalies can be related to a faulty sensor, but can also be process-related. In the case of process anomalies, rare process events can be properly labeled for the development of specific machine learning models, such as "few-shot learning" (FSL), for their proper detection or prediction. FSL is a known machine learning paradigm that uses supervised information to learn from a limited number of examples. Root cause analysis is a method aimed at distinguishing process anomalies from those related to faulty sensors. The analysis of causal relationships leading to the detection of anomalies can classify them as process-related or sensor-related anomalies. To this end, process engineers define rules or machine learning models, and a semi-supervised classifier is trained from the causal relationships and labels generated from the above rules.

[0065] Blast furnace thermal condition forecast:

[0066] This concerns the machine learning model MLT used by example in Figure 7. The MLT provides insight into the future of the blast furnace thermal state and hot metal production characteristics. From relevant process variables and other context variables that are useful for predicting the blast furnace thermal state or hot metal production characteristics, the MLT is trained to predict the following metrics in a given time period: -Hot metal temperature trends after 3 and 6 hours: high increase, medium increase, low increase, stable, low decrease, medium decrease, high decrease -Prediction of hot metal silicon content for multiple time periods from 1 hour to 6 hours into the future -Hot metal quality over multiple time periods ranging from 1 hour to 6 hours in the future

[0067] The model can be trained using hot metal temperatures measured manually after each cast or autonomously and continuously by dedicated sensors. An ensemble modeling approach can be implemented by combining predictions of multiple base models as new inputs to train the metamodel MLT, which results in predictions with reduced prediction bias or prediction variance.

[0068] Charge matrix optimization:

[0069] Blast furnace burden distribution is one of the most important actuators available to operators to optimize gas utilization (etaCO) to minimize coke rate and mitigate CO2 emissions. Blast furnace burden distribution must always be adapted to the blast furnace operation and is a compromise between optimal gas utilization, smooth blast furnace burden descent, and wall / stave (skin flow) temperatures.

[0070] As of today, some plants use blast furnace burden distribution models to assess the effect of a given charge matrix on the blast furnace charge profile and to determine the C / (O+C) ratio across the throat diameter of the blast furnace. This information is valuable and gives a reasonably good hint about the temperature profile in the cohesive zone. Nevertheless, defining the charge matrix in a model is not straightforward, and the model provides only limited help in finding the optimal charge matrix for a given operation.

[0071] The charging matrix is defined by the operator to optimally distribute materials on the blast furnace. To that end, the charging matrix includes parameters such as the inclination of the chute and the number of rotations for each material type. A machine learning model can be trained to predict the optimal charging matrix depending on the furnace's current thermal state, its expected evolution, and its production KPIs. The charging matrix prediction model can be trained from the raw data of multiple blast furnaces if there is not enough variation in the charging matrix elements for a single blast furnace to train the machine learning model.

[0072] Tuyere camera-based process inspection:

[0073] This example relates to Figure 8. Images 801 provided by a tuyere camera are analyzed by a combination of a convolutional neural network (CNN) and computer vision 803 that aims to detect phenomena 804 by applying computer vision on areas (e.g., circles, lances, blowing areas in classified image 801c) detected by a CNN-based region classifier 802. The detected phenomenon labels together with the tuyere image 801 can serve as input to a further deep learning model 805 trained to predict process phenomena.

[0074] Another application of tuyere image sequence analysis using machine learning is to encode spatiotemporal features to enrich the representation of blast furnace states that define the environment for reinforcement learning models. To that end, multimodal learning 808 can be used as a method to learn a representation 809 of the environment from heterogeneous data such as images 801, multivariate time series 806, and sound 807. This allows for more sophisticated approaches compared to unimodal machine learning, which makes the assumption of modal independence.

[0075] Taphole opener recommendations for optimal operation:

[0076] The scheduling of tapping as well as its parameterization (e.g. clay type) can be recommended by machine learning models.

[0077] TMT SOMA-based event detection and KPI calculation:

[0078] SOMA is an instrument for providing 2D information on the temperature distribution over the top of a blast furnace. The temperature maps can be processed by machine vision algorithms, potentially combined with machine learning models for predictive purposes. A processing pipeline such as that described in Figure 8 for camera-based tuyere inspection can also be applied to SOMA.

[0079] Phenomenon Labeling and Prediction:

[0080] Generating labels for process phenomena ensures the creation of rich information for improving learning of the relationship between the RILM's 130 operation and the environment. Labels can be generated by rules defined by the process engineer or by pattern detection models trained on patterns selected by the process engineer in historical data. The occurrence of patterns can be detected by algorithms such as dynamic time wrapping for univariate or multivariate time series data, or by defining features to train the respective machine learning models. In addition to providing high-level contextual information to the RILM model, these labels can be used to train machine learning models to detect the occurrence of combinations of phenomena or to predict the occurrence of single phenomena or combinations of phenomena. Training a supervised machine learning model from the generated labels requires sufficient labels with sufficient variance.

[0081] Predictive and prescriptive maintenance:

[0082] Machine learning models can be trained to predict maintenance and recommend actions to be taken to defer maintenance, thereby extending the life of a blast furnace or any asset related to a blast furnace. Several approaches are known for this, such as applying supervised learning to predict the "remaining useful life" or "time to failure" of an asset. Unsupervised learning models can be trained to detect rare events and to temporally cluster training data sets that are used to train supervised models that anticipate those rare events. Predictive root cause analysis allows an autonomous system trained with maintenance actions recorded from past maintenance to determine actions that are best known to defer maintenance.

[0083] Advanced contextual representation learning:

[0084] The reinforcement learning model needs a representation of the context to better model the environment and learn optimal actions to take with respect to that environment. To that end, multiple sensors may be deployed around the furnace 90 at different locations to record images (camera sensor 91), sound waves (sound sensor 92), vibrations (vibration sensor 93), and analyze the air (gas sensor 94), as shown in Figure 9. Each multi-modal time series may be analyzed by a deep learning network to extract a meaningful representation of the context, which may potentially be combined with process data or material descriptive data of the blast furnace. The material descriptive data corresponds to the chemical analysis of the material and other properties that may affect the thermal conditioning of the blast furnace.

[0085] FIG. 10 illustrates an example of a generic computing device 900 and a generic mobile computing device 950 that may be used with the technology described herein. Computing device 900 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Generic computing device 900 may correspond to computer system 100 of FIG. 1. Computing device 950 is intended to represent various forms of mobile devices, such as personal digital assistants, mobile phones, smartphones, driver assistance systems or on-board computers of vehicles (e.g., vehicles 401, 402, and 403, see FIG. 1), and other similar computing devices. For example, computing device 950 may be used as a front end by a user (e.g., a blast furnace operator) to interact with computing device 900. The components, their connections and relationships, and their functions illustrated herein are merely exemplary and do not limit the implementation of the invention(s) described and / or claimed herein.

[0086] Computing device 900 includes a processor 902, memory 904, a storage device 906, a high-speed interface 908 connecting to memory 904 and a high-speed expansion port 910, and a low-speed interface 912 connecting to a low-speed bus 914 and storage device 906. Each of the components 902, 904, 906, 908, 910, and 912 are interconnected using various buses and may be mounted on a common motherboard or otherwise as needed. Processor 902 can process instructions for execution within computing device 900, including instructions stored in memory 904 or storage device 906 for displaying graphical information for a GUI on an external input / output device, such as a display 916 coupled to high-speed interface 908. In other implementations, multiple processors and / or multiple buses may be used, along with multiple memories and multiple types of memory, as needed. Multiple computing devices 900 may also be connected, each providing a portion of the required operations (e.g., as a bank of servers, a group of blade servers, or a multiprocessor system).

[0087] The memory 904 stores information within the computing device 900. In one implementation, the memory 904 is one or more volatile memory units. In another implementation, the memory 904 is one or more non-volatile memory units. The memory 904 may also be another form of computer-readable medium, such as a magnetic or optical disk.

[0088] The storage device 906 can provide mass storage for the computing device 900. In one implementation, the storage device 906 can be or include a computer-readable medium, such as a floppy disk device, a hard disk device, an optical disk device, or an array of devices, including a tape device, flash memory or other similar solid-state memory device, or devices in a storage area network or other configuration. A computer program product can be tangibly embodied in an information carrier. The computer program product can also include instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer- or machine-readable medium, such as memory 904, the storage device 906, or memory on the processor 902.

[0089] The high-speed controller 908 manages bandwidth-intensive operations for the computing device 900, and the low-speed controller 912 manages low-bandwidth-intensive operations. Such an allocation of functionality is merely exemplary. In one implementation, the high-speed controller 908 is coupled to the memory 904, the display 916 (e.g., via a graphics processor or accelerator), and a high-speed expansion port 910 that may accept various expansion cards (not shown). In this implementation, the low-speed controller 912 is coupled to the storage device 906 and the low-speed expansion port 914. The low-speed expansion port, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), may be coupled, for example, via a network adapter, to one or more input / output devices, such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or router.

[0090] Computing device 900, as shown in the figure, may be implemented in several different forms. For example, it may be implemented as a standard server 920 or multiple times in a group of such servers. It may also be implemented as part of a rack server system 924. Furthermore, it may be implemented in a personal computer, such as a laptop computer 922. Alternatively, components from computing device 900 may be combined with other components in a mobile device (not shown), such as device 950. Each such device may include one or more of computing devices 900, 950, and the entire system may be composed of multiple computing devices 900, 950 in communication with each other.

[0091] Computing device 950 includes, among other components, a processor 952, memory 964, input / output devices such as a display 954, a communications interface 966, and a transceiver 968. Device 950 may also include a storage device, such as a microdrive or other device, to provide additional storage. Each of components 950, 952, 964, 954, 966, and 968 are interconnected using various buses, and some of the components may be mounted on a common motherboard or otherwise as desired.

[0092] The processor 952 can execute instructions within the computing device 950, including instructions stored in the memory 964. The processor may be implemented as a chipset of chips including separate analog and digital processors. The processor may provide, for example, control of a user interface, applications run by the device 950, and coordination of other components of the device 950, such as wireless communication by the device 950.

[0093] The processor 952 may communicate with a user via a control interface 958 and a display interface 956 coupled to a display 954. The display 954 may be, for example, a TFT LCD (thin film transistor liquid crystal display) or an OLED (organic light emitting diode) display, or other suitable display technology. The display interface 956 may comprise appropriate circuitry for driving the display 954 to present graphical and other information to the user. The control interface 958 may receive commands from the user and convert them for submission to the processor 952. Additionally, an external interface 962 may be provided in communication with the processor 952 to enable short-range communication of the device 950 with other devices. The external interface 962 may provide, for example, wired communication in some implementations or wireless communication in other implementations, although multiple interfaces may also be used.

[0094] Memory 964 stores information within computing device 950. Memory 964 may be implemented as one or more computer-readable media, one or more volatile memory units, or one or more non-volatile memory units. Expansion memory 984 may also be provided and connected to device 950 via expansion interface 982, which may include, for example, a SIMM (single in-line memory module) card interface. Such expansion memory 984 may provide additional storage space for device 950 or may store applications or other information for device 950. Specifically, expansion memory 984 may include instructions for performing or supplementing the processes described above and may also include secure information. Thus, for example, expansion memory 984 may function as a security module for device 950 and be programmed with instructions that enable secure use of device 950. Additionally, secure applications may be provided via SIMM cards, along with additional information, such as hackably disposing of identifying information on the SIMM card.

[0095] The memory may include, for example, flash memory and / or NVRAM memory, as described below. In one implementation, a computer program product is tangibly embodied in an information carrier. The computer program product includes instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer- or machine-readable medium, such as memory 964, expansion memory 984, or memory on processor 952, which may be received, for example, via transceiver 968 or external interface 962.

[0096] Device 950 may communicate wirelessly via communication interface 966, which may include digital signal processing circuitry as needed. Communication interface 966 may provide for communication under various modes or protocols, such as GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others. Such communication may occur, for example, via radio frequency transceiver 968. Additionally, short-range communication may occur using Bluetooth, WiFi, or other such transceivers (not shown). Additionally, GPS (Global Positioning System) receiver module 980 may provide device 950 with additional navigation- and location-related wireless data that may be appropriately used by applications running on device 950.

[0097] Device 950 may also communicate audibly using audio codec 960, which may receive verbal information from a user and convert it into usable digital information. Audio codec 960 may also generate audible sounds for the user, such as through a speaker in the handset of device 950. Such sounds may include sounds from voice telephone calls, may include recorded sounds (e.g., voice messages, music files, etc.), and may also include sounds generated by applications running on device 950.

[0098] The computing device 950, as shown in the figure, may be implemented in several different forms. For example, it may be implemented as a mobile phone 980. It may also be implemented as part of a smartphone 982, personal digital assistant, or other similar mobile device.

[0099] Various implementations of the systems and techniques described herein may be realized in digital electronic circuitry, integrated circuits, specially designed ASICs (application-specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs executable and / or interpretable on a programmable system including at least one programmable processor, which may be special purpose or general purpose, coupled to receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0100] These computer programs (also known as programs, software, software applications, or code) include machine instructions for a programmable processor and may be implemented in a high-level procedural and / or object-oriented programming language and / or in an assembly / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., magnetic disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0101] To provide for user interaction, the systems and techniques described herein may be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) by which the user can provide input to the computer. Other types of devices may also be used to provide for user interaction; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user may be received in any form, including acoustic, speech, or tactile input.

[0102] The systems and techniques described herein may be implemented in computing devices that include back-end components (e.g., as data servers), middleware components (e.g., application servers), or front-end components (e.g., client computers having a graphical user interface or web browser through which a user can interact with an implementation of the systems and techniques described herein), or any combination of such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include a local area network ("LAN"), a wide area network ("WAN"), and the Internet.

[0103] Computing devices may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0104] Although several embodiments have been described, it will be understood that various modifications may be made without departing from the spirit and scope of the invention.

[0105] Furthermore, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. Furthermore, other steps may be provided or steps may be eliminated from the described flows, and other components may be added to or removed from the described systems. Accordingly, other embodiments are within the scope of the following claims.

Claims

1. A computer-implemented method (1000) for training a reinforcement learning model (130) to provide operating instructions for thermal control of a blast furnace, the method comprising: processing (1100) historical operation data (21) acquired as a multivariate time series and reflecting the thermal state of each of the blast furnaces (BF1-BFn) in a plurality of domains by a domain-adaptive machine learning model (110) trained by transfer learning to generate a first domain-invariant dataset (22) representing the thermal state of any of the blast furnaces (BF1-BFn) regardless of domain; generating (1200) artificial operating data (24a) as multivariate time series reflecting the thermal state of a general blast furnace (BFg) for a specific thermal control operation (26a) by using a transient model (121) of a general blast furnace process, wherein the transient model (121) reflects the respective physical, chemical, thermal, and flow conditions of the general blast furnace and provides a solution for the upward gas flow and downward movement of a structured solid layer in the general blast furnace while exchanging heat, mass, and momentum transfer; generating (1300) a second domain-invariant dataset (23 a) by processing the artificially manipulated data (24 a) with a generative deep learning network (122) trained on the multivariate time series of the historical manipulation data (21) to transfer features learned from the historical manipulation data (21) to the artificially manipulated data (24 a); the reinforcement learning model (130) processes the combined first and second domain-invariant data sets (22, 23a) to determine (1400) a reward (131) for the particular thermal control action (26a) given a given objective function; and regenerating (1300) the second domain-invariant dataset based on correction parameters (123-2) in response to the reward (131), wherein a genetic search and / or Bayesian optimization algorithm (123-1) guides the search for the correction parameters for further thermal control actions based on a current environment (25a) of the reinforcement learning model (130) and the thermal control action (26a) output of a current learning step, and repeating the determining (1400) step to learn and regenerate (1300) optimized operation instructions for optimized thermal control actions to be applied for each operating state of one or more blast furnaces.

2. The reinforcement learning model (130) predicts (1700) an optimized operation command for at least one actuator of a particular blast furnace in production based on current operating state data of the particular blast furnace; determining (1400) the compensation based on a new state of the particular blast furnace after applying a thermal control action according to the optimized operation command to the at least one actuator; 10. The method of claim 1, further comprising: if the reward is below a predetermined threshold, using the transient model to regenerate the second domain-invariant data for one or more alternative manipulation instructions to retrain the reinforcement learning model.

3. 3. The method of claim 1 or 2, wherein the domain-adaptive machine learning model (110) is implemented by a generative deep learning neural network using convolutional and / or recurrent layers trained to extract domain-invariant features from the historical operational data (21) as the first domain-invariant dataset.

4. 3. The method of claim 1, wherein the domain-adaptive machine learning model (110) is trained to learn a plurality of mappings of corresponding raw data from the plurality of blast furnaces (BF1 to BFn) to a reference blast furnace (BFr), each mapping being a representation of a transformation of the respective blast furnace to the reference blast furnace, and the plurality of mappings correspond to the first domain-invariant dataset.

5. The method of claim 4 , wherein the domain-adaptive machine learning model (110) is implemented by a generative deep learning architecture based on a CycleGAN architecture.

6. 6. The method of claim 1, wherein the reinforcement learning model is trained to learn the optimized operating instructions such that associated objective measures are within a predetermined range from a Pareto front for a corresponding multidimensional objective function.

7. 7. The method of claim 1, wherein the transient model (121) includes a plurality of computational cells, each representing a respective layer of the general blast furnace consisting of a batch of raw materials, and wherein each computational cell iteratively and continuously solves the gas phase equations to satisfy relative gas phase parameter tolerances at each iteration time interval, and continuously solves the solid phase equations at the same iteration time interval when the gas phase parameters converge to predetermined tolerances.

8. Iteratively solving the gas phase equations involves solving, for each iteration of the pressure-velocity correction loop, Calculating gas, solid and liquid properties (3300); Calculating reaction rates and heat transfer coefficients (3400); Calculating (3500) gas temperature, gas species, gas velocity, and gas pressure drop; and solving the solid-state equation continuously is Calculating the solid temperature and solid species (3600); Calculating the liquid temperature and liquid type (3700); and calculating (3800) the solids velocity.

9. 9. The method according to any one of claims 1 to 8, wherein the transient model (121) receives one or more of the following input parameters: quantity and chemical analysis of blast furnace charge material, temperature, pressure, PCI rate, and oxygen enrichment, using an energy equation to predict hot metal temperature, one or more species equations to calculate hot metal chemical composition, and one or more gas phase equations to predict maximum gas temperature, gas efficiency (Eta CO), and gas pressure.

10. The method of claim 1 , wherein the reinforcement learning model is implemented by a recurrent neural network.

11. - predicting information about the future thermal evolution of a particular blast furnace state based on said historical operating data (21) and / or further measured environmental data related to the environment of the blast furnace by using one or more respectively trained associated machine learning models (ML1 to MLn) to complement said historical operating data (21) with future multivariate time series data related to future time points; 11. The method of claim 1, further comprising processing the future multivariate time series with the domain-adaptive machine learning model to augment the first domain-invariant dataset with data relating to future time points.

12. Training a particular one (MLT) of the associated machine learning models (ML1 to MLn) includes: training (703) a plurality of base models with different selections of operational data (701) and / or environmental data (702) using one or more machine learning algorithms to provide base model-specific future multivariate time series data as training inputs to the particular one of the machine learning models; and training (706) the particular one of the relevant machine learning models with future multivariate time series data specific to the base models to learn which combination of base models is best suited for which state of the blast furnace.

13. 13. The method of claim 12, wherein the particular one of the machine learning models (ML1-MLn) is trained to predict, at a future time, one of the following parameters: anomalies in the blast furnace process; thermal state of the blast furnace and hot metal production KPIs; charging matrix optimization; blast furnace phenomena based on tuyere camera based process inspection; taphole opener recommendations for optimal operation; TMT SOMA based phenomena and KPIs; phenomena based on phenomenon labeling by process rules.

14. A computer program product which, when loaded into the memory of a computer system and executed by at least one processor of said computer system, performs the steps of the computer-implemented method of any one of claims 1 to 13.

15. A computer system (100) comprising a plurality of functional modules that, when executed by the computer system, perform the steps of the computer-implemented method of any one of claims 1 to 13.

Images