Machine learning device and waste heat recovery system

Abstract

Provided are a machine learning device and a waste heat recovery system. The waste heat recovery system has: a waste gas line that guides a portion of waste gas generated in a cement sintering device to a release device via a boiler; a bypass line that bypasses the boiler for the remaining portion of the waste gas generated in the waste gas generation section; at least one damper provided in at least one of the waste gas line and the bypass line; a controller that operates the at least one damper in accordance with an operation value; and a steam turbine generator. The machine learning device has: an operation value output section; a state observation section that acquires state values including a boiler inlet gas temperature, a steam flow rate from the boiler, and a power generation output value of the steam turbine generator; a reward calculation section that calculates a reward based on a prescribed energy value associated with the power generation output value included in the state values acquired after the at least one damper is operated in accordance with the operation value; and a learning section that learns to determine, from a current state value, an operation value to be output to the controller based on the operation value, the state values, and the reward.

Description

Technical Field

[0001] This invention relates to a machine learning device and a waste heat recovery system used in a waste heat recovery system for recovering heat from waste gas in cement manufacturing processes. Background Technology

[0002] The cement manufacturing process mainly consists of the following steps: the raw material stage, which involves drying, crushing, and blending cement raw materials; the sintering stage, which involves sintering clinker from the raw materials as an intermediate product; and the finishing stage, which involves adding gypsum to the clinker and crushing it to finally produce cement. In the sintering stage, the cement raw materials are first preheated in a preheater (hereinafter also referred to as "PH"), then calcined in a calcining furnace, then sintered in a kiln, and finally cooled in an airquenching cooler (hereinafter also referred to as "AQC"). Previously, waste heat recovery systems were known to recover the heat generated in the waste gases produced in the PH and AQC processes and use the recovered heat for power generation.

[0003] For example, patent document 1 Figure 3 The disclosed waste heat recovery system comprises: an exhaust gas line equipped with an AQC boiler for recovering heat from exhaust gas generated in the AQC, extending from the high-temperature section of the AQC to the chimney; and a bypass line extending from the low-temperature section of the AQC and bypassing the AQC boiler. The exhaust gas line discharges exhaust gas with a relatively high temperature (e.g., an average of 360°C) from the AQC, while the bypass line discharges exhaust gas with a relatively low temperature (e.g., an average of 110°C) from the AQC. The exhaust gas discharged from the AQC via the exhaust gas line is guided to the AQC boiler. In the AQC boiler, superheated steam is generated using the heat from the exhaust gas, and this superheated steam is used for power generation in a steam turbine generator. After the heat-recovered exhaust gas in the AQC boiler is combined with the exhaust gas discharged from the AQC in the bypass line, it is released into the atmosphere from the chimney via a dust collector.

[0004] In addition, the waste heat recovery system has a waste gas line through the AQC boiler and a bypass line that bypasses the AQC boiler, as well as a waste gas line through the PH boiler and a bypass line that branches off from the PH waste gas line and bypasses the PH boiler.

[0005] Patent Document 1: Japanese Patent No. 5897302

[0006] The waste heat recovery system described above is optimized to generate the maximum possible energy. One approach is to adjust, for example, the opening of dampers installed in bypass lines to allow high-temperature, high-volume waste gas to be sent to the boiler. However, the flow rate and temperature of the waste gas flowing into the boiler are in a relationship where increasing one will decrease the other.

[0007] For example, if a bypass line bypassing the AQC boiler is equipped with an adjustable damper, reducing the damper opening decreases the amount of exhaust gas discharged from the AQC via the bypass line, while increasing the amount discharged through the exhaust gas line. However, since it becomes more difficult to discharge lower-temperature exhaust gas from the AQC via the bypass line, the temperature distribution within the AQC changes, resulting in a decrease in the temperature of the exhaust gas flowing into the AQC boiler through the exhaust gas line. Conversely, increasing the damper opening of the bypass line reduces the flow rate of exhaust gas into the AQC boiler, but increases its temperature. Consequently, the amount of clinker supplied from the kiln to the AQC constantly changes, resulting in fluctuations in the temperature distribution of the exhaust gas within the AQC. Therefore, achieving optimal system performance by adjusting the flow rates of exhaust gas in the exhaust gas line and bypass line is extremely difficult. Furthermore, the need for system optimization may also arise in the exhaust gas line through the PH boiler and the bypass line bypassing the PH boiler. Summary of the Invention

[0008] Therefore, the object of the present invention is to provide a machine learning device and a waste heat recovery system for optimizing the flow rate of waste gas flowing in the waste gas line and the bypass line in a waste heat recovery system having a waste gas line that directs waste gas generated in a cement sintering plant to a boiler and a bypass line that bypasses the boiler and merges with the waste gas line.

[0009] One aspect of the present invention provides a machine learning apparatus for determining operating values ​​in a waste heat recovery system, wherein the waste heat recovery system comprises: a cement sintering equipment having a waste gas generating section that generates waste gas during the sintering of cement raw materials; a release device that releases the waste gas to the atmosphere outside the cement sintering equipment; a waste gas line equipped with a boiler and an exhaust fan, which guides a portion of the waste gas generated in the waste gas generating section to the release device via the boiler; a bypass line that allows the remaining portion of the waste gas generated in the waste gas generating section to bypass the boiler and be guided between the boiler and the release device in the waste gas line; at least one damper disposed on at least one of the waste gas line and the bypass line; and a controller that activates the at least one damper according to the operating values. The machine learning device includes: an operation value output unit that outputs the operation value to the controller; a state observation unit that acquires a state value including the temperature of the exhaust gas flowing into the boiler, the flow rate of the steam supplied from the boiler to the steam turbine generator, and the power generation output value of the steam turbine generator; a reward calculation unit that calculates a reward based on a predetermined energy value associated with the power generation output value included in the state value acquired after actuating the at least one damper according to the operation value; and a learning unit that performs machine learning on the determination of the operation value to be output to the controller based on the current state value, the state value, and the reward.

[0010] In addition, one aspect of the waste heat recovery system of the present invention is the waste heat recovery system having the aforementioned machine learning device.

[0011] Based on the aforementioned machine learning device and waste heat recovery system, machine learning can be used to determine the operating value that generates the maximum possible energy. Therefore, a system can be implemented whereby, using the learning results from the machine learning device and based on the operating value determined from the current state of the waste heat recovery system, at least one damper located on at least one of the exhaust gas line and bypass line is activated, thereby generating the maximum possible energy from the exhaust gas.

[0012] According to the present invention, for a waste heat recovery system having a waste gas line that guides waste gas generated in a cement sintering plant to a boiler and a bypass line that bypasses the boiler and merges with the waste gas line, a machine learning device and a waste heat recovery system for optimizing the flow rate of waste gas flowing in the waste gas line and the bypass line can be proposed. Attached Figure Description

[0013] Figure 1 This is a schematic structural diagram of a waste heat recovery system according to one embodiment of the present invention.

[0014] Figure 2 This is a block diagram of the control system in a waste heat recovery system.

[0015] Figure 3 This diagram illustrates the basic concepts of reinforcement learning algorithms.

[0016] Figure 4 This is a functional block diagram of a machine learning device.

[0017] Figure 5 This is a flowchart illustrating the process of machine learning (reinforcement learning) performed by a machine learning device.

[0018] Label Explanation

[0019] 1: Waste heat recovery system; 2: Cement manufacturing equipment; 3: Power generation equipment; 7: Control device; 15: Exhaust fan; 16: Release device; 20: Sintering equipment (cement sintering equipment); 21: Preheater (PH); 22: Calcining furnace; 22a: Exhaust line for calcining furnace; 22b: Fuel supply line; 23: Rotary kiln; 24: Air quick-cooling cooler (AQC); 30: PH boiler; 35: Exhaust fan; 40: AQC boiler; 47: Exhaust fan; 48: Release equipment; 50: Steam turbine generator; 61: Exhaust gas line; 61a: Inlet damper; 62: Bypass line; 62a: Bypass ventilation damper; 63: Exhaust gas line; 63a: Inlet damper; 64: Bypass line; 64a: Bypass ventilation damper; 71: Controller; 72: Machine learning machine (machine learning device); 91: State observation unit; 92: State value storage unit; 93: Report calculation unit; 94: Learning unit; 95: Learning result storage unit; 96: Operation value output unit. Detailed Implementation

[0020] Waste heat recovery system

[0021] Hereinafter, a waste heat recovery system according to one embodiment of the present invention will be described with reference to the accompanying drawings. Waste heat recovery system 1 is a system for recovering heat from waste gas generated during the cement manufacturing process in cement manufacturing equipment 2. The waste heat recovery system 1 of this embodiment includes cement manufacturing equipment 2 and power generation equipment 3.

[0022] Cement manufacturing equipment

[0023] As mentioned above, the cement manufacturing process consists of raw material processing, sintering, and finishing processes. Figure 1 As shown, the cement manufacturing equipment 2 has a raw material generating device 10 that undertakes the raw material process and a sintering device 20 that undertakes the sintering process.

[0024] The raw material generating equipment 10 is used to generate powdered raw materials from cement raw materials such as limestone and clay by drying, crushing, and blending them. The raw material generating equipment 10 includes a raw material mill 11, a separator 12, and a silo 13. In the raw material mill 11, limestone, clay, etc., are crushed and dried. Additionally, exhaust gas from the PH boiler 30 (described later) flows into the raw material mill 11 and is used for drying the cement raw materials. The cement raw materials crushed and dried in the raw material mill 11 are discharged along with the exhaust gas and flow into the separator 12. In the separator 12, the cement raw materials are separated from the exhaust gas and classified. The cement raw materials separated and classified by the separator 12 are mixed and stored in the silo 13 and supplied from the silo 13 to the sintering equipment 20. Furthermore, as described later, the exhaust gas in the separator 12 is conveyed to the dust collector 14.

[0025] The sintering equipment 20 is a device for sintering clinker, an intermediate product, from powder raw materials obtained from the raw material generating equipment 10. The sintering equipment 20 includes a preheater (PH) 21, a calcining furnace 22, a rotary kiln 23, and an AQC 24.

[0026] PH 21 has a multi-stage cyclone dust collector connected in series. In PH 21, waste heat from the rotary kiln 23 moves sequentially from the bottom cyclone dust collector to the top cyclone dust collector, while cement raw materials move sequentially from the top cyclone dust collector to the bottom cyclone dust collector. The bottom cyclone dust collector of PH 21 is connected to the calcining furnace 22.

[0027] In the calcining furnace 22, the cement raw material from PH 21 is calcined in an atmosphere of approximately 900°C. The calcining furnace 22 is connected to an exhaust line 22a for supplying waste heat from AQC 24 and a fuel supply line 22b for supplying fuel and other materials to the calcining furnace 22. The outlet of the calcining furnace 22 is connected to the inlet of the rotary kiln 23.

[0028] Rotary kiln 23 is a horizontally elongated cylindrical rotary kiln, arranged at a slight downward slope from the raw material inlet to the raw material outlet. In rotary kiln 23, the cement raw materials, preheated and calcined in PH 21 and calcining furnace 22, are sintered using waste heat from AQC 24 and combustion gases from burner 25. The outlet of rotary kiln 23 is connected to the inlet 24a of AQC 24.

[0029] In AQC 24, the high-temperature (e.g., approximately 1400°C) sinter exiting rotary kiln 23 is rapidly cooled. Specifically, the sinter fed into AQC 24 from the outlet of rotary kiln 23 is conveyed towards outlet 24b via a conveyor (not shown) within AQC 24. During conveying, the sinter is cooled by cooling air blown from below the conveyor. After being cooled by AQC 24, the sinter, i.e., clinker, exits from outlet 24b and is then conveyed to a clinker silo via a clinker conveyor (not shown).

[0030] The cooling air blown onto the high-temperature sinter within the AQC 24 becomes high-temperature exhaust gas. The gas temperature within the AQC 24 forms a distribution that decreases as it approaches the outlet 24b along the conveying direction of the sinter. For example, the gas temperature near the inlet 24a of the AQC 24 is approximately 1350°C, and the gas temperature near the outlet 24b of the AQC 24 is approximately 100°C. However, due to fluctuations in the amount of clinker supplied to the AQC 24, the temperature of the exhaust gas within the AQC 24 will vary. Furthermore, a thermometer 81 for measuring the temperature of the exhaust gas is installed within the AQC 24 (more specifically, at the upstream side of the higher temperature section 24c within the AQC 24 along the conveying direction of the sinter).

[0031] [Power generation equipment]

[0032] The power generation equipment 3 includes a preheater boiler (hereinafter also referred to as "PH boiler") 30, a kiln head waste heat boiler (hereinafter also referred to as "AQC boiler") 40, and a steam turbine generator 50.

[0033] (PH boiler)

[0034] The PH boiler 30 is a boiler that uses the exhaust gas generated in PH 21 as a heating medium. The PH boiler 30 has a boiler body 31, which has a gas inlet and a gas outlet. Within the boiler body 31, a superheater 32 and an evaporator 33, serving as heat exchangers, are arranged sequentially from the gas inlet to the gas outlet. Additionally, a steam drum 34 is attached to the boiler body 31.

[0035] The PH boiler 30 is located on the exhaust gas line 61 (PH exhaust gas line) extending from PH 21 to the release device 16. Specifically, the high-temperature exhaust gas from the rotary kiln 23 flows in the order of calcining furnace 22 and PH 21, and is then guided to the PH boiler 30 through the exhaust gas line 61.

[0036] An exhaust fan 35 is installed in the exhaust gas line 61 downstream of the PH boiler 30. Additionally, a bypass line 62 (PH bypass line) is connected to the exhaust gas line 61, bypassing the PH boiler 30. The upstream end of the bypass line 62 is connected to the exhaust gas line 61 upstream of the PH boiler 30, and the downstream end of the bypass line 62 is connected between the PH boiler 30 and the exhaust fan 35 in the exhaust gas line 61.

[0037] An inlet damper 61a is provided between the upstream end of the bypass line 62 in the exhaust gas line 61 and the PH boiler 30. A bypass damper 62a is provided in the bypass line 62. Furthermore, a thermometer 82 for measuring the temperature of the exhaust gas is provided in the exhaust gas line 61 at a position upstream of the PH boiler 30, and more specifically, upstream of the upstream end of the bypass line 62. Additionally, a thermometer 83 for measuring the temperature of the exhaust gas is provided in the exhaust gas line 61 at a position downstream of the PH boiler 30, and more specifically, between the downstream end of the PH boiler 30 and the bypass line 62.

[0038] The exhaust gas after heat exchange in the PH boiler 30 and the exhaust gas flowing in the bypass line 62 are conveyed to the raw material mill 11 by the exhaust fan 35 and used as a heat source for the raw material mill 11. The cement raw material (dried material) supplied to the raw material mill 11 and the cement raw material (dried material) discharged from the raw material mill 11 are periodically sampled and their respective masses are measured. The moisture content (moisture percentage) of the cement raw material supplied to the raw material mill 11 can be determined based on the difference between the mass of the dried material and the mass of the dried material in the raw material mill 11, that is, the mass reduction in the raw material mill 11. In this embodiment, the mass measuring device for these cement raw materials is a moisture measuring device 17 for measuring the moisture content of the cement raw materials. However, the moisture measuring device 17 is not limited to the above case. In addition, a thermometer 87 for measuring the temperature of the exhaust gas flowing out of the raw material mill 11 is provided between the raw material mill 11 and the separator 12 in the exhaust gas line 61.

[0039] In the exhaust gas line 61, a dust collector 14 and an exhaust fan 15 are sequentially arranged from upstream to downstream of the exhaust gas flow between the separator 12 and the release device 16. Exhaust gas used for drying cement raw materials in the raw material mill 11 is conveyed to the dust collector 14 after being transported to the separator 12. After dust is separated by the dust collector 14, the exhaust gas conveyed from the separator 12 to the dust collector 14 is then transported by the exhaust fan 15 to the chimney, which serves as the release device 16, and released into the atmosphere.

[0040] (AQC Boiler)

[0041] The AQC boiler 40 is a boiler that uses the exhaust gas generated in the AQC 24 as a heating medium. The AQC boiler 40 has a boiler body 41, which has a gas inlet and a gas outlet. Within the boiler body 41, from the gas inlet to the gas outlet, are arranged sequentially a superheater 42 (serving as a heat exchanger), an evaporator 43, and a preheater (eco-friendly unit) 44. Additionally, a steam drum 45 is attached to the boiler body 41.

[0042] The AQC boiler 40 is connected to an exhaust gas line 63 (AQC exhaust gas line) extending from the high-temperature section 24c of the AQC 24 to the release device 48. The high-temperature section 24c of the AQC 24 refers to the part of the AQC 24 where the exhaust gas temperature is relatively high (e.g., approximately 350°C). For example, the average temperature of the high-temperature section 24c of the AQC 24 is within the allowable temperature range of the AQC boiler 40. The relatively high-temperature exhaust gas is guided from the AQC 24 to the AQC boiler 40 via the exhaust gas line 63, located at the upstream end of the high-temperature section 24c of the AQC 24, and is used as the heating medium of the AQC boiler 40.

[0043] Additionally, the aforementioned calcining furnace extraction line 22a is connected to AQC 24. The calcining furnace extraction line 22a is connected to AQC 24 at a position upstream in the direction of sinter transport, compared to the connection point between exhaust gas line 63 and AQC 24. Therefore, exhaust gas at a temperature higher than the exhaust gas discharged through exhaust gas line 63 (e.g., approximately 600°C) is extracted from the calcining furnace extraction line 22a.

[0044] In the exhaust gas line 63, a dust collector 46 and an exhaust fan 47 are sequentially installed from upstream to downstream of the exhaust gas flow between the AQC boiler 40 and the release device 48. A bypass line 64 (AQC bypass line) extends from the low-temperature section 24d of the AQC 24, bypassing the AQC boiler 40. The low-temperature section 24d of the AQC 24 refers to the part of the AQC 24 where the exhaust gas temperature is relatively low (e.g., approximately 150°C), located downstream of the high-temperature section 24c of the AQC 24 in the sinter conveying direction. The upstream end of the bypass line 64 connects to the low-temperature section 24d of the AQC 24, and the downstream end of the bypass line 64 connects between the AQC boiler 40 and the dust collector 46 in the exhaust gas line 63. Through the upstream end of the bypass line 64 located in the low-temperature section 24d of the AQC 24, the relatively low-temperature exhaust gas is discharged from the AQC 24 via the bypass line 64.

[0045] An inlet damper 63a is installed upstream of the AQC boiler 40 in the exhaust gas line 63. A bypass damper 64a is installed in the bypass line 64. A thermometer 84 for measuring the temperature of the exhaust gas is installed between the upstream end of the exhaust gas line 63 and the AQC boiler 40 (the inlet of the boiler body 41). A thermometer 88 for measuring the temperature of the exhaust gas is installed between the downstream end of the AQC boiler 40 (the outlet of the boiler body 41) in the exhaust gas line 63 and the downstream end of the bypass line 64.

[0046] The exhaust gas flowing from the AQC boiler 40 and the exhaust gas flowing in the bypass line 64 are conveyed to the dust collector 46 by the exhaust fan 47, and then to the chimney, which serves as the release device 48, and released into the atmosphere.

[0047] (Steam turbine generator)

[0048] The steam turbine generator 50 includes a steam turbine 51 and a generator 52. The steam turbine 51 is driven by supplied steam. The steam turbine 51 is a multi-stage steam turbine, for example, supplying high-pressure steam to the high-pressure stage and low-pressure steam to the low-pressure stage. The generator 52 converts the kinetic energy of the rotating shaft of the steam turbine 51 into electrical energy. The electricity generated by the steam turbine generator 50 is used for the operation of auxiliary equipment (e.g., exhaust fans 15, 35, 47) in the waste heat recovery system 1.

[0049] The steam used in the steam turbine 51 is condensed in the condenser 54 by cooling water supplied from the cooling tower 53. The condensate generated by the condenser 54 is pumped to the AQC boiler 40 via the pump 55. The water supplied to the AQC boiler 40 is heated by the preheater 44, which imparts heat from the exhaust gas. The water heated by the preheater 44 is distributed to the steam drum 45 of the AQC boiler 40, the steam drum 34 of the PH boiler 30, and the high-pressure flash evaporator 56.

[0050] The water supplied to the steam drum 45 of the AQC boiler 40 is heated by the evaporator 43 and then returned to the steam drum 45 for gas-liquid separation. The steam in the steam drum 45 is heated to above the saturation temperature in the superheater 42 and then guided to the steam turbine 51.

[0051] The water supplied to the steam drum 34 of the PH boiler 30 is heated by the evaporator 33 and then returned to the steam drum 34 for gas-liquid separation. The steam in the steam drum 34 is heated to above the saturation temperature in the superheater 32 and then guided to the steam turbine 51.

[0052] The remaining water from the water heated by preheater 44 that is not supplied to steam drums 34 and 45 is conveyed to the high-pressure flash evaporator 56. The high-pressure flash evaporator 56 performs gas-liquid separation on the water conveyed from preheater 44. The steam generated by the high-pressure flash evaporator 56 is supplied to steam turbine 51 (more specifically, the intermediate-pressure section of steam turbine 51), and the remaining hot water is conveyed to the low-pressure flash evaporator 57. The low-pressure flash evaporator 57 performs gas-liquid separation on the water conveyed from high-pressure flash evaporator 56. The steam generated by the low-pressure flash evaporator 57 is supplied to steam turbine 51 (more specifically, the low-pressure stage of steam turbine 51), and the remaining water, along with condensate from condenser 54, is conveyed to preheater 44.

[0053] In addition, the power generation equipment 3 includes: a steam flow meter 85, which measures the flow rate of steam supplied from the PH boiler 30 to the steam turbine 51; and a steam flow meter 86, which measures the flow rate of steam supplied from the AQC boiler 40 to the steam turbine 51.

[0054] (Control device)

[0055] The waste heat recovery system 1 of this embodiment has a control device 7. Figure 2 This is a block diagram of the control system in waste heat recovery system 1. (Example:) Figure 2 As shown, the temperatures measured by thermometers 81, 82, 83, 84, and 88, the steam flow rates measured by steam flow meters 85 and 86, and the power output values ​​from the steam turbine generator 50 are sent to the control device 7. Power consumption meters 15a, 35a, and 47a are respectively installed on exhaust fans 15, 35, and 47 to measure their power consumption. The power consumption values ​​measured by the power consumption meters 15a, 35a, and 47a are sent to the control device 7.

[0056] Furthermore, the control device 7 includes a controller 71 and a machine learning unit 72. The controller 71 and the machine learning unit 72 each have an arithmetic processing unit such as a CPU and a non-volatile and volatile storage unit, respectively. The controller 71 and the machine learning unit 72 are electrically connected in a manner capable of transmitting and receiving information. Alternatively, the machine learning unit 72 may be disposed separately from the controller 71 and located outside the controller 71; in this case, the machine learning unit 72 can be connected to the controller 71 via a network or the like.

[0057] Controller 71 controls dampers 61a, 62a, 63a, 64a and exhaust fans 15, 35, 47. Alternatively, controller 71 may include control units for controlling dampers 61a, 62a, 63a, 64a and control units for controlling exhaust fans 15, 35, 47. Machine learning unit 72 performs machine learning to generate the maximum possible energy in waste heat recovery system 1. Machine learning unit 72 performs machine learning to determine the operating value of at least one of the opening degrees of dampers 61a, 62a, 63a, 64a. That is, a portion of the dampers 61a, 62a, 63a, 64a and exhaust fans 15, 35, 47, which are the controlled objects, are controlled based on the operating values ​​output by machine learning unit 72, and the remaining portions are controlled based on programs and / or data pre-stored in the storage unit.

[0058] In the examples described below, the control of bypass ventilation dampers 62a and 64a is explained based on operating values ​​output by the machine learning unit 72. In this case, for example, inlet dampers 61a and 63a are controlled by the controller 71 to a predetermined set opening degree (e.g., 100%) during the operation of the cement manufacturing equipment 2. Additionally, exhaust fans 15, 35, and 47 are controlled by the controller 71 to operate at a predetermined set speed (exhaust volume) during the operation of the cement manufacturing equipment 2.

[0059] Regarding the state of the waste heat recovery system 1, it is difficult to explicitly express using simple formulas how the optimal system can be achieved simply by adjusting the opening degrees of the bypass ventilation dampers 62a and 64a. Therefore, the machine learning device 72, as an artificial intelligence, employs a reinforcement learning algorithm that automatically learns actions to achieve the goal simply by being given a reward. The machine learning performed by the machine learning device 72 will be explained below.

[0060] -Machine Learning Methods-

[0061] Figure 3 This is a diagram illustrating the basic concepts of reinforcement learning algorithms. (For example...) Figure 3 As shown, in reinforcement learning, the agent learns by exchanging information with the environment (cement manufacturing equipment 2 and power generation equipment 3) as the object of control. More specifically, the following steps (1) to (4) are repeated during the learning process.

[0062] (1) The state of the environment s at time t observed by the agent t .

[0063] (2) The agent selects the actions it can take based on observation results and past learning. t and carry out action a t .

[0064] (3) By implementing action a t The state of the environment s t Towards the next state s t+1 Changes occur, and based on these changes in state, the agent receives a reward r. t+1 .

[0065] (4) The agent is based on state s t Action a t , return r t+1 Learn from the results of past learning.

[0066] In the learning process described in (4) above, the agent acquires state s. t Action a t , return r t+1 The mapping serves as a benchmark for determining the amount of future reward *r* that can be obtained. For example, when the number of states obtainable at each time point is set to *m*, and the number of actions obtainable is set to *n*, by repeatedly performing actions, the storage and state *s* can be obtained. t and action a t The relative return of the group r t+1 We have an m×n 2D array. Then, based on the mapping obtained above, we use a value function (evaluation function) that represents the current state and the degree of action. We update the value function by repeatedly taking actions, thereby learning the best action relative to the state.

[0067] The known state-action value function Q(s) is one of the value functions. t a t ) represents a state s t Next action a t What level of action is it? State-action value function Q(s) t a t The state-action value function Q(s) is expressed as a function with state and action as independent variables. t a t During the learning process of repeated actions, the value is updated based on the reward obtained from an action in a certain state, the value of actions in future states resulting from that action, etc. The state-action value function Q(s) is... t a t The update formula for ) is defined according to the reinforcement learning algorithm. For example, in Q-learning, which is one of the representative reinforcement learning algorithms, the state-action-value function Q(s) is defined as follows: t a t The updated formula for ) is defined by the following formula 1.

[0068] [Formula 1]

[0069]

[0070] in,

[0071] Q: State-action value function

[0072] s t The state at time t

[0073] a t Actions relative to the state at time t

[0074] r t+1 The reward obtained from the state at time t+1

[0075] α: Learning coefficient (0 < α ≤ 1)

[0076] γ: Discount rate (0 < γ ≤ 1)

[0077] Then, in action a of (2) above t In the selection, a value function generated through past learning is used in the current state s. t Choose the return for the entire future (r) t+1 +r t+2 +…) Best action a t (in state s) t The most valuable action a t In Q-learning, the reward (r) t+1 +r t+2 +…) Best action a t Able to be equivalent to a return (r) t+1 +r t+2 +…) The biggest action a t .

[0078] As reinforcement learning algorithms, various methods such as Q-learning, SARSA, TD-learning, and AC-learning are well-known. However, any reinforcement learning algorithm can be used as the method applied to this invention. Since these reinforcement learning algorithms are well-known, further detailed descriptions of each algorithm in this specification are omitted.

[0079] Alternatively, as a method for updating the value function, it is also possible to avoid updating the current state s every time. t Apply a certain action a t And make state s t Transition to new state t+1 This refers to a learning method that updates the value function in real time (so-called online learning). For example, it can be achieved by repeatedly updating the value function based on the current state s. t Apply a certain action a t And make state s t Transition to new state t+1This action involves storing these states and actions as learning data and using the stored learning data to update the value function—a process known as batch learning or mini-batch learning.

[0080] Figure 4 This is a functional block diagram of machine learning machine 72. Figure 4 The machine learning machine 72 shown has a state observation unit 91, a state value storage unit 92, a reward calculation unit 93, a learning unit 94, a learning result storage unit 95, and an operation value output unit 96.

[0081] The status observation unit 91 acquires various values ​​related to the status of the cement manufacturing equipment 2 and the power generation equipment 3 as status values. The status value storage unit 92 stores the status values ​​acquired by the status observation unit 91 and outputs the stored status values ​​to the feedback calculation unit 93 and the learning unit 94.

[0082] Status values ​​may include, for example, the measured value of thermometer 81 (temperature of exhaust gas on the upstream side of AQC 24), the measured value of thermometer 82 (temperature of exhaust gas flowing into PH boiler 30), the measured value of thermometer 83 (temperature of exhaust gas flowing out of PH boiler 30), the measured value of thermometer 84 (temperature of exhaust gas flowing into AQC boiler 40), the measured value of thermometer 88 (temperature of exhaust gas flowing out of AQC boiler 40), the measured value of thermometer 87 (exhaust gas temperature at the outlet of raw material mill 11), the measured value of steam flow meter 85 (steam flow rate of PH boiler 30), the measured value of steam flow meter 86 (steam flow rate of AQC boiler 40), the power output value of steam turbine generator 50, and the power consumption values ​​of power consumption meters 15a, 35a, and 47a. Status values ​​may also include the temperature of exhaust gas at the inlet or outlet of exhaust fans 15, 35, and 47. The status values ​​may include not only values ​​obtained in the latest operation but also values ​​obtained in past operations. For example, the status values ​​may also include values ​​pre-stored in the control device 7. For example, the status values ​​may include the moisture content of the cement raw materials measured by the moisture measuring device 17, the set opening degree of the inlet dampers 61a and 63a, and the set rotational speed (exhaust volume) of the exhaust fans 15, 35, and 47. The status values ​​may also include the exhaust gas temperature at the inlet of the exhaust fans 15, 35, and 47.

[0083] In addition, Figure 4 Examples of state values ​​are shown, but state values ​​are not limited to these. That is, the state values ​​stored in the state value storage unit 92 may not include... Figure 4 All the values ​​shown can contain the following: Figure 4 The values ​​shown are different.

[0084] The reward calculation unit 93 analyzes the state values ​​obtained from the state value storage unit 92 according to the preset reward conditions and calculates the reward. The reward calculation unit 93 outputs the calculated reward to the learning unit 94.

[0085] Reward conditions are the conditions for assigning rewards in reinforcement learning. These reward conditions are pre-set in the machine learning device 72 by the operator or others. For example, if it is determined that the energy generated in the waste heat recovery system 1 is large, the reward condition is set to assign a larger reward compared to a case where the energy generated is small. Rewards can be positive, negative, or zero.

[0086] In this embodiment, the reward conditions are set in a manner corresponding to an indicator that evaluates the energy generated in the waste heat recovery system 1. Specifically, the evaluation indicator is the value obtained by subtracting the power consumption of the auxiliary equipment in the waste heat recovery system 1 from the power generation output value of the steam turbine generator 50. That is, the energy value generated in the waste heat recovery system 1 is not the power generation output value of the steam turbine generator 50 itself, but rather the net energy value obtained by subtracting the power consumption of the auxiliary equipment from the power generation output value, which is set as the evaluation indicator.

[0087] The reward calculation unit 93 calculates a positive reward when the energy value, which serves as an evaluation indicator, exceeds a predetermined threshold, and calculates a negative or zero reward when the energy value is below the predetermined threshold. For example, the reward calculation unit 93 can calculate a zero reward when the energy value is lower than a predetermined first threshold but higher than other second thresholds, and calculate a negative reward when the energy value is lower than the second threshold. The thresholds are determined by simulation, past operating results, etc., and are pre-stored in the storage unit of the control device 7.

[0088] Here, in the calculation of the evaluation index, i.e., the calculation of the return, the power consumption of all auxiliary machines operating using the power generated by the steam turbine generator 50 may not be used. For example, among all auxiliary machines operating using the power generated by the steam turbine generator 50, only the devices with higher power consumption may be used in the return calculation. In this example, the power consumption of exhaust fans 15, 35, and 47, which consume more power than other auxiliary machines, is used in the return calculation.

[0089] The learning unit 94 performs machine learning (reinforcement learning) on ​​the determination of the operation value based on the state values ​​obtained from the state value storage unit 92 and the rewards obtained from the reward calculation unit 93. More specifically, the learning unit 94 uses a value function (state-action value function Q(s)). t a t The value function is updated in a manner that maximizes the reward obtained, wherein the value function is determined based on the state values ​​obtained by the state observation unit 91 (the state s is defined by the combination of state values).t The determination of the operating value under this state is represented by the independent variable.

[0090] The learning result storage unit 95 stores the learning results learned by the learning unit 94. Furthermore, methods for storing the value function as the learning result generally use methods employing approximate functions or arrays. However, besides these methods, for example, when considering a large number of states, methods utilizing states s can also be used. t Action a t Methods such as SVMs, neural networks, and teacher learners, which use multiple-valued outputs as inputs and outputs, are examples of approaches that employ this approach.

[0091] The operation value output unit 96 determines the operation value based on the learning result of the learning unit 94 and the current state value, and outputs the determined operation value to the controller 71.

[0092] In this example, the operating value is the opening degree of the bypass ventilation dampers 62a and 64a, but the operating value is not limited to this. As mentioned above, the operating value only needs to include at least one of the opening degrees of dampers 61a, 62a, 63a, and 64a. For example, as... Figure 4 As shown in the example of the operating values, the operating values ​​may include, in addition to or instead of the openings of the bypass ventilation dampers 62a and 64a, the openings of the inlet dampers 61a and 63a and a portion or all of the rotational speeds of the exhaust fans 15, 35, and 47.

[0093] Next, refer to Figure 5 The process of machine learning (reinforcement learning) performed by machine learning machine 72 is explained.

[0094] When machine learning begins, the state observation unit 91 acquires information for determining the environment (state s). t The information is used as a state value (step S01). The state value storage unit 92 stores the acquired state value (step S02). The learning unit 94 determines the current state s based on the state value acquired by the state observation unit 91. t (Step S03). The learning unit 94, based on past learning results and the state s determined in step S03, t Choose action a t (Step S04). Action a t It is related to the defined state s t The corresponding operating value is determined. For example, multiple operating values ​​can be prepared as selectable actions, and the action a that yields the greatest future reward r is selected based on past learning results. The operating value output unit 96 outputs the action a selected in step S04. t The system determines the operating value to be output and outputs the determined operating value to the controller 71. The controller 71 then executes action a. t(Step S05) Correspondingly, the cement manufacturing equipment 2 and the power generation equipment 3 are put into operation.

[0095] After executing action a t After the state transition, the state observation unit 91 obtains the information used to determine the environment (state s). t+1 The information (step S06) is stored as a state value by the state value storage unit 92 (step S07). During this stage, the states of the cement manufacturing equipment 2 and the power generation equipment 3 change according to the action a performed as time progresses from time t to time t+1. t The reward calculation unit 93 calculates the reward r based on the state value at time t+1, according to the set reward conditions. t+1 (Step S08). The learning unit 94, based on the state s determined in step S03... t Action a selected in step S04 t and the return r calculated in step S08 t+1 Machine learning is then performed (step S09). The value function used in the learning process is determined based on the applied learning algorithm. The learning result storage unit 95 stores the learning results from the learning unit 94 (step S10), and the process returns to step S03.

[0096] As described above, machine learning (reinforcement learning) is repeatedly performed in machine learning machine 72. The learning of machine learning machine 72 may also end at a stage where it is confirmed that the optimal system has been achieved through energy generation (e.g., a stage where the specified energy is generated within a specified period).

[0097] When using the learned data to actually determine the operating value of the waste heat recovery system 1, the machine learning machine 72 can also repeatedly run the system using the learned data from the completed learning process without performing new learning. In this case, the operating value output unit 96 determines the operating value based on the learning result of the learning unit 94 (i.e., the learning result stored in the learning result storage unit 95) and the current state value, and outputs it to the controller 71.

[0098] As described above, the waste heat recovery system 1 of this embodiment has a machine learning device 72, which includes: an operation value output unit 96 that outputs operation values ​​to a controller 71; a status observation unit 91 that acquires status values ​​including the temperature of the exhaust gas flowing into the AQC boiler 40, the temperature of the exhaust gas flowing out of the AQC boiler 40, the temperature of the exhaust gas flowing into the PH boiler 30, the temperature of the exhaust gas flowing out of the PH boiler 30, and the power generation output value of the steam turbine generator 50; a reward calculation unit 93 that calculates a reward based on a predetermined energy value associated with the power generation output value included in the status value acquired after the bypass ventilation doors 62a and 64a are operated based on the operation value; and a learning unit 94 that performs machine learning on the determination of the operation value to be output to the controller 71 based on the current status value, the status value, and the reward.

[0099] According to this embodiment, machine learning can be used to determine the operating value that generates the maximum possible energy. Therefore, using the learning results of the machine learning machine 72, based on the operating value determined from the current state of the waste heat recovery system 1, the bypass ventilation dampers 62a and 64a respectively installed in the bypass lines 62 and 64 are activated, thereby enabling a system that generates the maximum possible energy from the waste gas.

[0100] Learning Department 94 can use the value function (state-action value function Q(s)). t a t The value function is updated in a way that maximizes the return, wherein the value function is represented by the independent variable by the state determined by the state value obtained by the state observation unit 91 and the determination of the operating value under that state.

[0101] Furthermore, in this embodiment, the machine learning machine 72 learns to determine the operating values ​​for adjusting the inflow of exhaust gas for both the AQC boiler 40 and the PH boiler 30. Therefore, overall system optimization can be achieved.

[0102] Furthermore, the return calculation unit 93 of the machine learning machine 72 uses the value obtained by subtracting the power consumption value of the auxiliary equipment in the waste heat recovery system 1 from the power generation output value of the steam turbine generator 50 as an evaluation index for calculating the return. Therefore, net power generation efficiency can be improved.

[0103] The preferred embodiments of the present invention have been described above. However, without departing from the spirit of the present invention, modifications to the specific structure and / or function of the above embodiments may also be included in the present invention. The structure of the above-described waste heat recovery system 1 may be modified, for example, as follows.

[0104] In the above embodiment, the machine learning machine 72 learns to determine the operating value for adjusting the amount of waste gas flowing in for both the AQC boiler 40 and the PH boiler 30, but the present invention is not limited thereto.

[0105] For example, in the above embodiment, the machine learning machine 72 learned to determine the operating values ​​including the opening degree of the PH bypass damper 62a and the AQC bypass damper 64a. However, the machine learning machine 72 can also perform machine learning on the determination of the operating values ​​including the opening degree of the PH inlet damper 61a and / or the opening degree of the PH bypass damper 62a. That is, machine learning can be performed on the determination of the operating values ​​that improve the heat recovery rate of the PH boiler 30 as a whole, without considering the AQC boiler 40 and the PH boiler 30 as a whole. In this case, the state values ​​may include the temperature of the exhaust gas flowing into the PH boiler 30, the flow rate of steam supplied from the PH boiler 30 to the steam turbine generator 50, and the power generation output value of the steam turbine generator 50. In addition, in this case, the state values ​​may include the temperature of the exhaust gas flowing out of the PH boiler 30.

[0106] Alternatively, for example, the machine learning machine 72 can also perform machine learning on the determination of operating values ​​including the opening degree of the AQC inlet damper 63a and / or the opening degree of the AQC bypass damper 64a. That is, machine learning can be performed on the determination of operating values ​​that improve the heat recovery rate of the AQC boiler 40 alone, without considering the AQC boiler 40 and PH boiler 30 as a whole. In this case, the state values ​​may include the temperature of the exhaust gas flowing into the AQC boiler 40, the flow rate of steam supplied from the AQC boiler 40 to the steam turbine generator 50, and the power generation output value of the steam turbine generator 50. Additionally, in this case, the state values ​​may include the temperature of the exhaust gas flowing out of the AQC boiler 40.

[0107] Furthermore, in the above embodiment, the energy value used as the evaluation index is obtained by subtracting the power consumption of the auxiliary equipment from the power output value of the steam turbine generator 50. However, the energy value used as the evaluation index is not limited to this. For example, the energy value used as the evaluation index may also be the power output value of the steam turbine generator 50 itself.

Claims

1. A machine learning device that machine learns determination of an operation value in a waste heat recovery system, wherein the waste heat recovery system has: a cement sintering apparatus that has a waste gas generation section that generates a waste gas in a process of sintering a cement raw material; a release apparatus that releases the waste gas to the atmosphere outside the cement sintering apparatus; a waste gas line that is provided with a boiler and an exhaust fan, and that guides a part of the waste gas generated at the waste gas generation section to the release apparatus through the boiler; a bypass line that guides a remaining part of the waste gas generated at the waste gas generation section to the waste gas line between the boiler and the release apparatus while bypassing the boiler; at least one damper that is provided in at least one of the waste gas line and the bypass line; a controller that actuates the at least one damper in accordance with the operation value; a steam turbine generator that generates electric power by steam generated at the boiler; and an auxiliary machine that operates using electric power generated by the steam turbine generator, the auxiliary machine including at least the exhaust fan, the machine learning device has: an operation value output section that outputs the operation value to the controller; a state observation section that acquires a state value that includes a temperature of the waste gas flowing into the boiler, a flow rate of steam delivered from the boiler to the steam turbine generator, and an electric power generation output value of the steam turbine generator; a reward calculation section that calculates a reward on the basis of a prescribed energy value associated with the electric power generation output value included in the state value acquired after the at least one damper is actuated in accordance with the operation value; and a learning section that machine learns determination of the operation value output to the controller in accordance with a current state value on the basis of the operation value, the state value, and the reward, the energy value is not the electric power generation output value of the steam turbine generator itself, but is a net energy value obtained by subtracting a consumed electric power value of the auxiliary machine from the electric power generation output value of the steam turbine generator, the reward calculation section calculates a positive reward in a case where the energy value exceeds a prescribed threshold value, and calculates a negative reward or a zero reward in a case where the energy value is lower than the prescribed threshold value.

2. The machine learning device according to claim 1, wherein the cement sintering apparatus has: a preheater that preheats a cement raw material; a kiln that sinters the cement raw material preheated by the preheater; and an air quenching type cooler that transports a sintered product coming out of the kiln while rapidly cooling the sintered product, the waste gas generation section is the air quenching type cooler, an upstream side end portion of the waste gas line is connected to a prescribed high temperature section of the air quenching type cooler, and an upstream side end portion of the bypass line is connected to a low temperature section of the air quenching type cooler, the low temperature section being located at a position downstream of the high temperature section in a transport direction of the sintered product.

3. The machine learning device according to claim 2, wherein ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ The release device is a release device for an air quenching cooler, the boiler is a boiler for an air quenching cooler, the exhaust fan is an exhaust fan for an air quenching cooler, the exhaust gas line is an exhaust gas line for an air quenching cooler, the bypass line is a bypass line for an air quenching cooler, the damper is an inlet damper, The waste heat recovery system further has: a release device for a preheater that releases the exhaust gas to the atmosphere outside the cement sintering facility; an exhaust gas line for a preheater that is provided with a boiler for a preheater and an exhaust fan for a preheater, and that guides a portion of the exhaust gas generated at the preheater to the release device for a preheater through the boiler for a preheater; a bypass line for a preheater that guides a remaining portion of the exhaust gas generated at the preheater to the exhaust gas line between the boiler for a preheater and the release device for a preheater of the exhaust gas line while bypassing the boiler for a preheater; and at least one damper for a preheater that is provided in at least one of the exhaust gas line for a preheater and the bypass line for a preheater, and that can change an opening degree, The steam turbine generator generates electric power by steam generated in the boiler for an air quenching cooler and the boiler for a preheater, The operation value includes an opening degree of the at least one damper for a preheater.

4. The machine learning device according to claim 1, wherein The cement sintering facility has: a preheater that preheats a cement raw material; a kiln that sinters the cement raw material preheated by the preheater; and an air quenching cooler that sharply cools a sintered material coming out of the kiln while conveying the sintered material, The exhaust gas generation section is the preheater, An upstream side end portion of the exhaust gas line is connected to the preheater, An upstream side end portion of the bypass line is connected at a position in the exhaust gas line that is on an upstream side of the boiler.

5. The machine learning device according to any one of claims 1 to 4, wherein The learning section updates the value function in a manner that maximizes the return using a value function that represents a state determined in accordance with the state value acquired by the state observation section and the operation value in the state as a determination argument.

6. A waste heat recovery system having the machine learning device according to any one of claims 1 to 4.

Images