Circulating fluidized bed boiler server for optimizing operating variable of circulating fluidized bed boiler and operating method thereof

Abstract

Disclosed are a circulating fluidized bed boiler server and a method by which the circulating fluidized bed boiler server optimizes an operating variable of a circulating fluidized bed boiler. The method may comprise the steps of: receiving state variables of the circulating fluidized bed boiler in real time; inputting, to a reinforcement learning algorithm, a past time series state variable corresponding to a first period among the received state variables, and outputting, from the reinforcement learning algorithm, a recommended operation variable of the circulating fluidized bed boiler for a second period in response thereto; and providing the recommended operation variable to the circulating fluidized bed boiler. The reinforcement learning algorithm is configured on the basis of a decision transformer, has applied thereto a time-shifting algorithm for outputting the recommended operating variable, and has a reward determined such that an emission amount of at least one of sulfur oxide (SOx) or nitrogen oxide (NOx) of the circulating fluidized bed boiler is reduced.

Description

Circulating fluidized bed boiler server for optimizing operating parameters of a circulating fluidized bed boiler and method of operation thereof

[0001] The following embodiments relate to a circulating fluidized bed boiler server for optimizing operating parameters of a circulating fluidized bed boiler and a method of operation thereof. More specifically, the following embodiments relate to a circulating fluidized bed boiler server for optimizing operating parameters of a circulating fluidized bed boiler to reduce the emission of environmental pollutants from the circulating fluidized bed boiler and to resolve combustion non-uniformity, and a method of operation thereof.

[0002] Coal accounts for approximately 44% of carbon dioxide emissions from fossil fuels worldwide. Efforts to minimize environmental pollutant emissions from coal-fired power plants are ongoing, and in particular, circulating fluidized bed boilers are widely used for combined heat and power generation and small to medium-scale power generation because they are a high-efficiency power generation method that produces steam from heat transfer through the circulation of a fluidized medium within a combustion chamber and can effectively utilize low-quality fuel.

[0003] Meanwhile, reinforcement learning is a type of machine learning that can be defined as a method in which an agent defined within an environment recognizes its current state and selects an action or sequence of actions from among available actions to maximize the reward.

[0004] Efforts are continuing to apply reinforcement learning algorithms to circulating fluidized bed boilers to predict combustion and environmental performance and to operate them in a way that minimizes environmental pollutants, thereby enabling operation using optimal operating variables.

[0005] A related prior art document is Korean Patent Publication No. 10-2023-0151413 (November 1, 2023).

[0006] The technical problem that the present disclosure aims to solve is to provide a circulating fluidized bed boiler server that optimizes the operating parameters of a circulating fluidized bed boiler and a method of operating the same.

[0007] The technical problems that this disclosure aims to solve are not limited to those described above, and problems not mentioned will be clearly understood by those skilled in the art from this specification and the accompanying drawings.

[0008] A method for optimizing operating variables of a circulating fluidized bed boiler in a circulating fluidized bed boiler server according to one embodiment of the present disclosure may include: receiving state variables of the circulating fluidized bed boiler in real time; inputting past time-series state variables corresponding to a first period among the received state variables into a reinforcement learning algorithm, and in response, the reinforcement learning algorithm outputs recommended operating variables of the circulating fluidized bed boiler for a second period; and providing the recommended operating variables to the circulating fluidized bed boiler. Here, the reinforcement learning algorithm is configured based on a Decision Transformer, a time-shifting algorithm for outputting the recommended operating variables is applied, and a reward may be determined in a direction that reduces the emission of at least one of sulfur oxides (SOx) and nitrogen oxides (NOx) of the circulating fluidized bed boiler.

[0009] According to an embodiment, the state variable may include at least one of the coal usage, coal calorific value, feedwater amount, and the ratio of coal usage to feedwater amount of the circulating fluidized bed boiler.

[0010] According to an embodiment, the circulating fluidized bed boiler comprises a plurality of zones, and the state variable may include at least one of oxygen concentration, loop seal differential pressure, cyclone differential pressure, windbox pressure, combustion furnace upper differential pressure, sulfur oxide (SOx) emission, nitrogen oxide (NOx) emission, and boiler bottom temperature for each of the plurality of zones.

[0011] According to an example, the recommended driving variable may include at least one of coal moving air, fuel spoon air, and limestone usage.

[0012] According to an embodiment, the recommended operating variable may include at least one of the following for each of the plurality of zones: lower combustion furnace pressure, preheating air ratio, primary air supply ratio at the lower combustion furnace, secondary air supply ratio at the upper combustion furnace, rotational speed of the coal screw feeder, air volume of the final superheater lift leg, and air volume of the final reheater lift leg.

[0013] According to an embodiment, the reinforcement learning algorithm may be an offline reinforcement learning algorithm that includes a Decision Transformer and is learned using state variables, driving variables, and rewards during a first period ((ta) minutes to t minutes) corresponding to a certain duration in the past based on a specific point in time (t) in the past, and a may be a natural number.

[0014] According to an embodiment, the reinforcement learning algorithm can be learned by masking the state variable, the driving variable, and the reward during a second period corresponding to a future duration based on the past specific time point (t), outputting a driving variable recommended at a first time within the second period, and by using a time shifting algorithm to repeatedly perform the operation of outputting the driving variable within the second period multiple times.

[0015] According to an embodiment, the reinforcement learning algorithm may determine a reward in a direction that reduces the temperature deviation between the left and right sides of the lower combustion chamber of the circulating fluidized bed boiler.

[0016] A circulating fluidized bed boiler server according to one embodiment of the present disclosure may include a memory and at least one processor configured to communicate with said memory. The processor may be configured to receive state variables of a circulating fluidized bed boiler in real time, input past time-series state variables corresponding to a first period among the received state variables into a reinforcement learning algorithm, and in response, for the reinforcement learning algorithm to output recommended operating variables of the circulating fluidized bed boiler for a second period and provide said recommended operating variables to the circulating fluidized bed boiler. The reinforcement learning algorithm may be configured based on a Decision Transformer, and a time-shifting algorithm for outputting said recommended operating variables may be applied, and a reward may be determined in a direction that reduces the emission of at least one of sulfur oxides (SOx) and nitrogen oxides (NOx) of the circulating fluidized bed boiler.

[0017] The technical solutions of the present disclosure are not limited to the technical solutions described above, and technical solutions not mentioned will be clearly understood by those skilled in the art to which the present disclosure pertains from the present specification and the attached drawings.

[0018] According to the embodiments of the present disclosure, by using a reinforcement learning algorithm to identify which operating variables have a significant impact on state variables and optimizing the operating variables of a circulating fluidized bed boiler, it is possible to maximize the reduction of environmental pollutants emitted from the circulating fluidized bed boiler and the elimination of combustion non-uniformity.

[0019] According to embodiments of the present disclosure, a high-performance reinforcement learning algorithm can be implemented by using appropriate state variables and driving variables and performing offline learning on time series data using a decision transformer, which is a model-free reinforcement learning model.

[0020] According to embodiments of the present disclosure, a more robust reinforcement learning algorithm can be implemented by appropriately preprocessing a state variable, which is time-series driving data, and using it as input to a reinforcement learning algorithm, and by learning to predict a driving variable for a certain period (e.g., 5 minutes) using a driving variable for a certain period (e.g., 25 minutes).

[0021] According to embodiments of the present disclosure, the emission of environmental pollutants can be minimized by recommending operating variables that maximize the compensation of a circulating fluidized bed boiler using a high-performance reinforcement learning algorithm.

[0022] The effects of the present disclosure are not limited to those described above, and unmentioned effects will become apparent to those skilled in the art from the present specification and the accompanying drawings.

[0023] FIG. 1 is a conceptual diagram of a circulating fluidized bed boiler system according to one embodiment of the present disclosure.

[0024] FIG. 2 is a block diagram of a circulating fluidized bed boiler system according to one embodiment of the present disclosure.

[0025] FIG. 3 is a conceptual diagram of a decision transformer included in a reinforcement learning algorithm according to one embodiment of the present disclosure.

[0026] FIG. 4 is a conceptual diagram of a time-shifting algorithm according to one embodiment of the present disclosure.

[0027] FIG. 5 is a flowchart illustrating a method for optimizing operating parameters of a circulating fluidized bed boiler in a circulating fluidized bed boiler server according to one embodiment of the present disclosure.

[0028] FIG. 6 is a flowchart illustrating a method for optimizing operating parameters of a circulating fluidized bed boiler in a circulating fluidized bed boiler server according to one embodiment of the present disclosure.

[0029] Figure 7 is an example of a graph showing sensing data over time before and after the application of an EMA filter.

[0030] Figure 8 is an example of a graph showing loess data over time after applying the STL decomposition method.

[0031] Figure 9 is an example of a graph showing sensing data over time before and after the application of the EMA filter and STL decomposition method.

[0032] FIG. 10 is an example of a graph showing a comparison between operating variables applied during the actual operation of a circulating fluidized bed boiler and recommended operating variables of a circulating fluidized bed boiler output using a reinforcement learning algorithm learned according to an embodiment of the present disclosure.

[0033] FIG. 11 is another example of a graph showing a comparison between operating variables applied during the actual operation of a circulating fluidized bed boiler and recommended operating variables of a circulating fluidized bed boiler output using a reinforcement learning algorithm learned according to an embodiment of the present disclosure.

[0034] FIG. 12 is an example of a user interface screen for monitoring a method for optimizing operating parameters of a circulating fluidized bed boiler according to one embodiment of the present disclosure.

[0035] Specific structural or functional descriptions of embodiments according to the concept of the present disclosure disclosed herein are provided merely for the purpose of explaining embodiments according to the concept of the present disclosure, and embodiments according to the concept of the present disclosure may be implemented in various forms and are not limited to the embodiments described herein.

[0036] Embodiments according to the concept of the present disclosure may be subject to various modifications and may take various forms; therefore, embodiments are illustrated in the drawings and described in detail in this specification. However, this is not intended to limit the embodiments according to the concept of the present disclosure to specific disclosed forms, and includes modifications, equivalents, or substitutions that fall within the spirit and scope of the present disclosure.

[0037] Terms such as "first" or "second" may be used to describe various components, but said components shall not be limited by said terms. For the sole purpose of distinguishing one component from another, for example, without departing from the scope of rights according to the concept of the present disclosure, the first component may be named the second component, and similarly, the second component may be named the first component.

[0038] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. Conversely, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between. Expressions describing the relationships between components, such as "between," "exactly between," or "directly adjacent to," should be interpreted in the same way.

[0039] The terms used herein are used merely to describe specific embodiments and are not intended to limit the disclosure. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “comprising” or “having” are intended to specify the existence of the described features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0040] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which this disclosure pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0041] In this specification, the term "processor" may refer to hardware capable of performing functions and operations according to each name described in this specification, computer program code capable of performing specific functions and operations, or an electronic recording medium loaded with computer program code capable of performing specific functions and operations.

[0042] In other words, the term "processor" may mean hardware for carrying out the technical concept of the present disclosure, software for driving said hardware, a functional combination of said hardware and said software, and / or a structural combination of said hardware and said software.

[0043] Hereinafter, embodiments will be described in detail with reference to the attached drawings. However, the scope of the patent application is not limited or restricted by these embodiments. Identical reference numerals in each drawing indicate identical components.

[0044] FIG. 1 is a conceptual diagram of a circulating fluidized bed boiler system according to one embodiment of the present disclosure.

[0045] The circulating fluidized bed boiler system may include a manager device (100), a circulating fluidized bed boiler server (200), a database (300) and / or a circulating fluidized bed boiler (400).

[0046] According to one embodiment, in a circulating fluidized bed boiler system, a circulating fluidized bed boiler server (200) can control the circulating fluidized bed boiler. For example, the circulating fluidized bed boiler server (200) can transmit various control signals for operating the circulating fluidized bed boiler (400). The circulating fluidized bed boiler server (200) can receive various state variables from the circulating fluidized bed boiler (400) and calculate operating variables for operating the circulating fluidized bed boiler (400) using the received state variables. In this specification, the state variables may also be referred to as 'state variable data' or 'sensing data'. According to an embodiment of the present disclosure, the circulating fluidized bed boiler server (200) can propose optimal operating variables for reducing environmental pollutants (e.g., SOx, NOx) emitted from the circulating fluidized bed boiler (400) and / or resolving combustion non-uniformity of the circulating fluidized bed boiler (400) by using a reinforcement learning algorithm. The manager of the circulating fluidized bed boiler (400) can constantly monitor the optimal operating variable recommendation value using the circulating fluidized bed boiler server (200) and / or the manager device (100) connected to the circulating fluidized bed boiler (400).

[0047] A circulating fluidized bed boiler (400) can be defined as a boiler capable of burning various fuels such as coal, lignite (brown coal), biomass, and petroleum coke (petcoke). The circulating fluidized bed boiler (400) can be operated as an environmentally friendly power generation facility by optimizing operating variables according to the embodiments of the present disclosure. The type of circulating fluidized bed boiler (400) does not limit the embodiments or scope of the present disclosure.

[0048] According to the method for optimizing operating variables of a circulating fluidized bed boiler according to one embodiment of the present disclosure, by optimizing the operating variables of a circulating fluidized bed boiler (400) using a reinforcement learning algorithm, the reduction of environmental pollutants emitted from the circulating fluidized bed boiler (400) and the resolution of combustion non-uniformity of the circulating fluidized bed boiler (400) can be maximized. Specifically, according to the method for optimizing operating variables of a circulating fluidized bed boiler according to one embodiment of the present disclosure, by using appropriate state variables and operating variables, and by performing offline learning on time-series state data using a Decision Transformer, which is a model-free reinforcement learning model, and by using the operating variables, which are time-series operating data, as input to the reinforcement learning algorithm and learning to predict operating variables for a certain period (e.g., 5 minutes) using operating variables for a certain period (e.g., 25 minutes), a high-performance reinforcement learning algorithm can be implemented. Furthermore, by appropriately recommending operating variables of the circulating fluidized bed boiler (400) using a high-performance reinforcement learning algorithm, the emission of environmental pollutants can be minimized.

[0049] Furthermore, according to an embodiment of the present disclosure, the circulating fluidized bed boiler server (200) can recommend optimal operating parameters for reducing environmental pollutants (e.g., SOx, NOx) emitted from the circulating fluidized bed boiler (400) and / or resolving combustion non-uniformity of the circulating fluidized bed boiler (400), and can compare past trends and current measured values ​​of environmental pollutants such as NOx and modify the recommended values ​​based on this.

[0050] More specifically, according to the method for optimizing operating variables of a circulating fluidized bed boiler according to an additional embodiment of the present disclosure, the operating variables of a circulating fluidized bed boiler recommended by the reinforcement learning algorithm may be modified by comparing the environmental pollutants (e.g., SOx, NOx) at a certain time with the measured values ​​of the environmental pollutants at that time and referring to this.

[0051] For example, if the NOx value measured at that time is higher than the average NOx value of the past 15 minutes of data, the recommended values ​​can be adjusted according to the reinforcement learning algorithm to decrease the supply air ratio at the bottom of the combustion chamber and increase the supply air ratio at the top of the combustion chamber among the operating variables of the circulating fluidized bed boiler recommended by the reinforcement learning algorithm.

[0052] FIG. 2 is a block diagram of a circulating fluidized bed boiler system according to one embodiment of the present disclosure.

[0053] The circulating fluidized bed boiler system may include a manager device (100), a circulating fluidized bed boiler server (200), a database (300) and / or a circulating fluidized bed boiler (400).

[0054] The circulating fluidized bed boiler server (200) can be implemented as a printed circuit board (PCB), such as a motherboard, an integrated circuit (IC), or a system on chip (SoC). For example, the circulating fluidized bed boiler server (200) can be implemented as an application processor.

[0055] The manager device (100) may be implemented in a PC (personal computer), server, or portable device. The portable device may be implemented as a laptop computer, mobile phone, smartphone, tablet PC, mobile internet device (MID), PDA (personal digital assistant), EDA (enterprise digital assistant), digital still camera, digital video camera, PMP (portable multimedia player), PND (personal navigation device or portable navigation device), handheld game console, e-book, or smart device. The smart device may be implemented as a smart watch, smart band, or smart ring.

[0056] In one embodiment, the method for optimizing operating variables of a circulating fluidized bed boiler (400) may be operated through a manager device (100) in the form of an application or program. The application for optimizing operating variables of the circulating fluidized bed boiler (400) may be an application program running on a PC or a portable device. The application for optimizing operating variables of the circulating fluidized bed boiler (400) may display a UI (user interface) or GUI (graphic user interface) through a display based on information processed by the manager device (100) or the circulating fluidized bed boiler server (200), and / or information processed by the manager device (100) or the circulating fluidized bed boiler server (200). The information processed by the circulating fluidized bed boiler server (200) may include information regarding sensing data, state variables, operating variables, and / or compensation of the circulating fluidized bed boiler (400). The application may include an interface linked to an existing system for operating the circulating fluidized bed boiler.

[0057] The database (300) may be a database connected to the circulating fluidized bed boiler server (200) via wired and / or wireless connections. In one embodiment, the database (300) may be a database that stores information regarding compensation, such as state variables, operating variables, and compensation functions, historical data, and / or temporary tables of the circulating fluidized bed boiler (400), which are updated in real time.

[0058] In one embodiment, the database (300) may be a cloud database that stores data related to the circulating fluidized bed boiler (400) that is updated in real time, including sensing data, state variables, operating variables, compensation functions, information on compensation, history data, and / or temporary tables. Here, history data refers to data representing the history of sensing data, state variables, operating variables, etc. Temporary tables refer to temporary data before the current day's data is transferred as history data, and refer to data representing the input variables of a reinforcement learning algorithm in a table format over time.

[0059] Additionally, the database (300) can store tag values ​​that are updated in real time and tag values ​​required for fuel injection inversion to display the status of the boiler (400) so that an administrator can check it in a user interface for monitoring the optimization method of operating variables of the circulating fluidized bed boiler (400).

[0060] The circulating fluidized bed boiler server (200) may include a processor (10) and / or memory (20). Although not shown in FIG. 2, the circulating fluidized bed boiler server (200) may further include a transceiver and / or interface.

[0061] The processor (10) may include a reinforcement learning algorithm (12).

[0062] The reinforcement learning algorithm (12) may consist of an agent suitable for the circulating fluidized bed, an environment in which the agent interacts, an operating state of the circulating fluidized bed boiler (400) perceived by the agent, a control action performed by the agent, and a reward received as a result of the agent's control action. The control action performed by the agent refers to an operating action that indicates which operating variable will be used to cause the circulating fluidized bed boiler (400) to operate, and can be used with the same meaning as the operating variable.

[0063] The reinforcement learning algorithm (12) can determine a reward in the direction of reducing the emission of at least one of sulfur oxides (SOx) and nitrogen oxides (NOx) of the circulating fluidized bed boiler (400). The reinforcement learning algorithm (12) can determine a reward in the direction of reducing the deviation of the lower temperature on the left and right sides of the combustion furnace of the circulating fluidized bed boiler (400).

[0064] The reinforcement learning algorithm (12) may include a decision transformer. For example, the reinforcement learning algorithm (12) may be an algorithm based on a time shifting algorithm. The processor (10) can recommend operating variables of the circulating fluidized bed boiler (400) at a specific point in the future using the reinforcement learning algorithm (12) including the decision transformer.

[0065] Detailed information regarding the decision transformer and time shifting algorithm included in the reinforcement learning algorithm (12) according to the embodiment of the present disclosure will be described later with reference to FIGS. 3 and FIGS. 4.

[0066] The processor (10) can process data stored in memory (20). The processor (10) can execute computer-readable code (e.g., software) stored in memory (20) and instructions triggered by the processor (10). The processor (10) may be a data processing device implemented in hardware having a circuit having a physical structure for executing desired operations. For example, the desired operations may include code or instructions included in a program. For example, the data processing device implemented in hardware may include a microprocessor, a central processing unit, a processor core, a multi-core processor, a multiprocessor, an Application-Specific Integrated Circuit (ASIC), or a Field Programmable Gate Array (FPGA).

[0067] The memory (20) can store various data used to optimize the operating variables of the circulating fluidized bed boiler (400).

[0068] Memory (20) can store state variables received from the circulating fluidized bed boiler (400). For example, memory (20) can store the coal usage, coal calorific value, feedwater amount, and / or the ratio of coal usage to feedwater amount of the circulating fluidized bed boiler (400). For another example, memory (20) can store the oxygen concentration, loop room differential pressure, cyclone differential pressure, windbox pressure, combustion furnace upper differential pressure, sulfur oxide (SOx) emissions, nitrogen oxide (NOx) emissions, and / or boiler bottom temperature for each of the multiple zones of the circulating fluidized bed boiler.

[0069] Memory (20) can store operating variables transmitted to the circulating fluidized bed boiler (400). For example, memory (20) can store coal transport air, fuel spoon air, and / or limestone usage. For another example, memory (20) can store the combustion furnace lower pressure, preheat air ratio, combustion furnace lower air ratio, combustion furnace upper air ratio, coal screw feeder rotation speed, final superheater lift leg air volume, or final reheater lift leg air volume for each of the multiple zones of the circulating fluidized bed boiler (400).

[0070] The memory (20) can store instructions (or programs) executable by the processor (10). For example, the instructions may include instructions for executing the operation of the processor and / or the operation of each component of the processor.

[0071] The memory (20) can be implemented as a volatile memory device or a non-volatile memory device.

[0072] Volatile memory devices can be implemented as DRAM (dynamic random access memory), SRAM (static random access memory), T-RAM (thyristor RAM), Z-RAM (zero capacitor RAM), or TTRAM (Twin Transistor RAM).

[0073] A non-volatile memory device can be implemented as an EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, MRAM (Magnetic RAM), Spin-Transfer Torque (STT)-MRAM, Conductive Bridging RAM (CBRAM), FeRAM (Ferroelectric RAM), PRAM (Phase change RAM), Resistive RAM (RRAM), Nanotube RRAM, Polymer RAM (PoRAM), Nano Floating Gate Memory (NFGM), holographic memory, Molecular Electronic Memory Device, or Insulator Resistance Change Memory.

[0074] FIG. 3 is a conceptual diagram of a decision transformer included in a reinforcement learning algorithm according to one embodiment of the present disclosure, and FIG. 4 is a conceptual diagram of a time shifting algorithm according to one embodiment of the present disclosure.

[0075] The reinforcement learning algorithm according to the embodiments of the present disclosure is an algorithm implemented through reinforcement learning without using prediction models such as CNN (Convolutional Neural Network) or DNN (Deep Neural Network). That is, the reinforcement learning algorithm according to the embodiments of the present disclosure can be implemented based on a model-free reinforcement learning model including a decision transformer.

[0076] The reinforcement learning algorithm according to the embodiment of the present disclosure can be updated to select the driving variable that gives the highest reward value by appropriately adjusting exploration and exploitation through the Epsilon-Greedy search method, while performing random exploration with the probability of the set Epsilon value. Accordingly, the reinforcement learning algorithm according to the embodiment of the present disclosure can be modified into an algorithm that is more robust and generalizes well, and it was confirmed that the difference between ValidationLoss and TrainLoss and the ValidationLoss value are lowered due to this method.

[0077] In the Decision Transformer, each modality, such as Return, State, and Action, can be passed to an embedding network. If the State and Action are time-series sequencing data, they can be passed to a linear layer. The vector values ​​containing the Return, State, and Action can be processed through a linear output layer by an autoregressive Transformer model trained to predict the action at the next time point from previous tokens.

[0078] For example, the reinforcement learning algorithm may be an offline reinforcement learning algorithm that learns using state variables, driving variables, and rewards during a first period ((ta) to t) corresponding to a certain duration in the past based on a specific point in time (t) in the past. An offline reinforcement learning algorithm means an algorithm that trains a model using a fixed dataset from the past. For example, the reinforcement learning algorithm may output driving variables recommended at a first time within the second period while masking the state variables, driving variables, and rewards during a second period (t to (t+b)) corresponding to a certain duration in the future based on a specific point in time (t) in the past, and may be learned by repeatedly performing the operation of outputting driving variables within the second period multiple times using a time shifting algorithm.

[0079] Referring to FIG. 3, past data, namely the reward value (1), state variable value (2), and driving variable value (3) at any point in the past (e.g., "t-24 minutes"), can be embedded as training data in the decision transformer. The trained decision transformer can output the driving variable value (4) at the next point in time (e.g., 1 minute after the current point in time, "t-25 minutes").

[0080] Figure 4 shows a time shifting algorithm when a=24 and b=5, assuming the current time point is To and a specific time point in the past is t.

[0081] In the embodiment of FIG. 4, the horizontal axis represents time [minutes], and the sliding window can be set to 1 minute. An episode refers to a sequence of states, actions, and rewards that an agent of a reinforcement learning algorithm goes through from an initial state to a final state. In the embodiment of FIG. 4, one episode can be set to 30 minutes. A series of interactions between the agent and the environment can be divided into episodes.

[0082] The aforementioned "first period (ta to t) corresponding to a certain duration in the past based on a specific point in time (t)" means the period from t-24 [minutes] to t [minutes]. That is, it means the period of 25 minutes in the past based on t.

[0083] The aforementioned "second period (t to t+b) corresponding to a certain future duration based on a specific point in the past (t)" refers to the period from t [minute] to t+5 [minute]. That is, it refers to a period of 5 minutes in the future based on t. The decision transformer can be trained to output recommended driving variable values ​​for the second period while masking the driving variable values ​​that were actually output during the second period.

[0084] The reinforcement learning algorithm according to the embodiment of FIG. 4 can take state variables (or state variable data) from the past 25 minutes as input based on the current time point To, and output recommended driving variables at 1-minute intervals for 5 minutes in the future. Accordingly, the reinforcement learning algorithm according to the embodiment of the present disclosure can be implemented as a more robust algorithm by learning to recommend driving variables up to 5 minutes in the future, rather than simply considering values ​​1 minute in the future.

[0085] Meanwhile, according to the method for optimizing operating variables of a circulating fluidized bed boiler (400) according to an additional embodiment of the present disclosure, the operating variables of the circulating fluidized bed boiler (400) recommended according to the reinforcement learning algorithm may be modified by comparing the environmental pollutant (e.g., SOx, NOx) at a certain time with the measured value of the environmental pollutant at that time and referring to it.

[0086] For example, if the NOx value measured at that time is higher than the average NOx value of the past 15 minutes of data, the recommended values ​​can be adjusted according to the reinforcement learning algorithm to decrease the supply air ratio at the bottom of the combustion chamber and increase the supply air ratio at the top of the combustion chamber among the operating variables of the circulating fluidized bed boiler recommended by the reinforcement learning algorithm.

[0087] FIG. 5 is a flowchart illustrating a method for optimizing operating parameters of a circulating fluidized bed boiler in a circulating fluidized bed boiler server according to one embodiment of the present disclosure.

[0088] Referring to FIG. 5, the circulating fluidized bed boiler server (200) can receive status variables of the circulating fluidized bed boiler (400) in real time (S502).

[0089] State variables of the circulating fluidized bed boiler (400) may include the amount of coal used, the calorific value of the coal, the amount of feedwater (output), and / or the ratio of the amount of coal used to the amount of feedwater (Fuel / Water Ratio).

[0090] According to an embodiment, the circulating fluidized bed boiler (400) may include a plurality of zones. For example, the plurality of zones may include a left zone and a right zone. As another example, the plurality of zones may include Zone 1, Zone 2, Zone 3, and Zone 4.

[0091] State variables of the circulating fluidized bed boiler (400) may include oxygen concentration, room room differential pressure, cyclone differential pressure, windbox pressure, combustion furnace upper differential pressure, sulfur oxide (SOx) emissions, nitrogen oxide (NOx) emissions, and / or boiler bottom temperature for each of the plurality of zones (Zone) of the circulating fluidized bed boiler (400).

[0092] The circulating fluidized bed boiler server (200) can output recommended operating variables of the circulating fluidized bed boiler (400) using a learned reinforcement learning algorithm that takes the above state variables as input (S504).

[0093] The circulating fluidized bed boiler server (200) can convert data by inputting the recommended operating variable into the Inverse Transform to return it to the original scale for the recommended operating variable.

[0094] Recommended operating variables for the circulating fluidized bed boiler (400) may include coal moving air, fuel spoon air, and / or limestone usage.

[0095] Recommended operating variables for the circulating fluidized bed boiler (400) may include, for each of the multiple zones of the circulating fluidized bed boiler, the lower combustion chamber pressure, the preheating air ratio, the lower combustion chamber supply ratio, the upper combustion chamber supply ratio, the rotational speed of the coal screw feeder, the air volume of the final superheater lift leg, and / or the air volume of the final reheater lift leg. The operating variables may be set to include variables related to both SOx emissions and NOx emissions related to compensation. Alternatively, they may be set to include limestone usage, which is a variable related only to SOx, through experimentation.

[0096] The reinforcement learning algorithm can determine a reward in the direction of reducing the emission of at least one of sulfur oxides (SOx) and nitrogen oxides (NOx) of the circulating fluidized bed boiler (400). The reinforcement learning algorithm can determine a reward in the direction of reducing the deviation of the lower temperature on the left and right sides of the combustion furnace of the circulating fluidized bed boiler.

[0097] The circulating fluidized bed boiler server (200) can provide the recommended operating variables to the circulating fluidized bed boiler (400) (S506).

[0098] According to an embodiment, the circulating fluidized bed boiler server (200) can set the episode of the reinforcement learning algorithm to 30 minutes, receive 25 minutes of past data as input, and output recommended operating variables at the corresponding time at 1-minute intervals for 5 minutes in the future.

[0099] Meanwhile, although not separately illustrated in FIG. 5, a circulating fluidized bed boiler server (200) according to an additional embodiment of the present disclosure may modify outputted recommended operating variables. Modification of recommended operating variables may be processed by comparing the environmental pollutants (e.g., SOx, NOx) of a certain period of time with the measured values ​​of environmental pollutants at that time. For example, if the NOx value measured at that time is higher than the average NOx value of the past 15 minutes of data, the circulating fluidized bed boiler server (200) may adjust the recommended values ​​according to a reinforcement learning algorithm so as to decrease the supply air ratio at the bottom of the combustion chamber and increase the supply air ratio at the top of the combustion chamber among the outputted operating variables of the circulating fluidized bed boiler (400).

[0100] FIG. 6 is a flowchart illustrating a method for optimizing operating parameters of a circulating fluidized bed boiler in a circulating fluidized bed boiler server according to one embodiment of the present disclosure.

[0101] Referring to FIG. 6, the circulating fluidized bed boiler server (200) can preprocess the sensing data of the circulating fluidized bed boiler (400) using an EMA (Exponential Moving Average) filter and a time series decomposition method (S602).

[0102] Preprocessing is required to input the sensing data (i.e., unprocessed data) sensed by each sensor installed in the circulating fluidized bed boiler server (200) into the model. If the sensing data is a tag related to air, it is used as a ratio to the total air rather than an actual value, so the circulating fluidized bed boiler server (200) can calculate the 'total air volume' to calculate the ratio. Additionally, the circulating fluidized bed boiler server (200) can group and sort the tags based on the left / right side of the boiler. Since the scale of each sensing data is different, the circulating fluidized bed boiler server (200) can perform scaling and convert the data into an episode format.

[0103] According to an embodiment, the circulating fluidized bed boiler server (200) can preprocess the sensing data of the circulating fluidized bed boiler (400) using an EMA filter. Specifically, the circulating fluidized bed boiler server (200) can smooth the sensing data using an EMA filter during the shoot blowing time of the circulating fluidized bed boiler (400).

[0104] Referring to FIG. 7, a graph (700) showing sensing data over time before the application of the EMA filter and a graph (702) showing sensing data over time after the application of the EMA filter are shown. According to the embodiment, the sensing data may refer to sensing data measuring NOx emissions or total sensing data. Multiple sensing data measured inside the circulating fluidized bed boiler (400) may be referred to as total sensing data.

[0105] When generating thermal power using a circulating fluidized bed boiler (400), the NOx sensor measuring NOx may be excessively affected during the soot blowing time, which corresponds to the time for cleaning the inside of the boiler by applying pressure to remove dust, etc., and the NOx sensing data may tend to fluctuate excessively. However, if the NOx sensing data (70) during the soot blowing time is removed, the continuity of the time-series state data (i.e., NOx sensing data) cannot be maintained. Therefore, according to an embodiment of the present disclosure, the sensing data can be preprocessed by using an EMA filter to maintain meaningful information in the time-series state data without removing the sensing data during the soot blowing time. According to an embodiment, if one refers to the NOx sensing data (72) during the soot blowing time of the graph (702) showing the sensing data measuring NOx emissions over time after applying the EMA filter, it can be confirmed that the noise in the soot blowing data has been removed.

[0106] As another example, the circulating fluidized bed boiler server (200) can preprocess the sensing data of the circulating fluidized bed boiler (400) using a time series decomposition method. In this case, the time series decomposition method may include the STL (Seasonal and Trend decomposition using Loess) decomposition method. Specifically, the circulating fluidized bed boiler server (200) can decompose the sensing data into seasonal data, trend data, and / or loess data, and smooth the loess data using an EMA filter so as to be adjusted within a preset threshold value.

[0107] Referring to FIG. 8, a graph (800) showing the LOES data over time after applying the STL decomposition method is shown.

[0108] The sum of seasonal data, trend data, and LOES data results in the total sensing data. LOES data may refer to noise data, outlier data, or data excluding seasonal data or trend data. The circulating fluidized bed boiler server (200) may pre-set an arbitrary area (80) for the LOES data. The arbitrary area (80) may be set based on a threshold value (or outlier) of the sensing data. The circulating fluidized bed boiler server (200) may smooth the LOES data using an EMA filter so that the LOES data is adjusted within a pre-set threshold value.

[0109] Referring to FIG. 9, a graph (900) showing sensing data over time before and after the application of the EMA filter and STL decomposition method is shown.

[0110] Unlike the graph (90) before the application of the EMA filter and STL decomposition method, according to the graph (92) after the application of the EMA filter and STL decomposition method, meaningful information was extracted from the sensing data during the shoot blowing time, and it can be confirmed that outliers were removed in terms of time series and the data was preprocessed into a form suitable for continuous data modeling.

[0111] Referring again to FIG. 6, the circulating fluidized bed boiler server (200) can learn a reinforcement learning algorithm using the preprocessed sensing data (S604).

[0112] The circulating fluidized bed boiler server (200) can output recommended operating variables of the circulating fluidized bed boiler (400) using the learned reinforcement learning algorithm (S606).

[0113] FIGS. 10 and 11 are graphs showing a comparison between the operating variables (action_targets) applied during the operation of the actual circulating fluidized bed boiler (400) and the recommended operating variables (action_preds) of the circulating fluidized bed boiler (400) output using a learned reinforcement learning algorithm.

[0114] FIG. 10 is a graph showing a comparison of the Final Super Heater (FSH) and Final Reheater (FRH) Lift leg air volumes for Zone 1 of a circulating fluidized bed boiler. It can be seen that the recommended operating variables of the circulating fluidized bed boiler (400), FSH and FRH Lift leg air volumes (action_preds), output using a learned reinforcement learning algorithm, closely follow the trends of the operating variables applied during the actual operation of the circulating fluidized bed boiler (400), FSH and FRH Lift leg air volumes (action_targets).

[0115] FIG. 11 is a graph showing a comparison of the limestone flow of the circulating fluidized bed boiler (400). It can be seen that the limestone flow (action_preds), which is the recommended operating variable of the circulating fluidized bed boiler (400) output using a learned reinforcement learning algorithm, follows the trend of the limestone flow (action_targets), which is the operating variable applied during the actual operation of the circulating fluidized bed boiler (400).

[0116] FIG. 12 is an example of a user interface screen for monitoring a method for optimizing operating parameters of a circulating fluidized bed boiler according to one embodiment of the present disclosure.

[0117] Referring to FIG. 12, a user interface screen for monitoring the method of optimizing the operating variables of a circulating fluidized bed boiler (400) is shown in a manager device (100) that runs an application or program for optimizing the operating variables of a circulating fluidized bed boiler (400).

[0118] On the user interface screen, the real-time operating status, operating direction, and prediction graph of the circulating fluidized bed boiler (400) can be displayed.

[0119] In the real-time operation status section of the circulating fluidized bed boiler (400), information related to the boiler bottom temperature ('Bed Temperature' in FIG. 12, 90), NOx emissions ('NOx Emission' in FIG. 12, 92), and limestone usage ('LimeStone Usage' in FIG. 12, 94) can be updated and displayed in real time.

[0120] In the operation direction section of the circulating fluidized bed boiler (400), items such as combustion air, fluidized air, bed material, limestone, and fuel, detailed operation variable items for each item, current values ​​for each operation variable, and recommended values ​​for each operation variable may be displayed in a table format (96). Here, the recommended values ​​correspond to recommended operation variables output using a reinforcement learning algorithm according to an embodiment of the present disclosure and may be updated every minute. In the case of 'Fuel Feeding', which is a detailed operation variable for the 'Fuel' item among the items, the circulating fluidized bed boiler server (200) may replace and display the value by calculating it in reverse through real-time data in a form that an actual manager can input.

[0121] In the prediction graph section of the circulating fluidized bed boiler (400), operating variable data corresponding to the boiler bed temperature, which is output at 1-minute intervals from the reinforcement learning algorithm according to the embodiment of the present disclosure, can be displayed in a graph format (98) separated into left and right.

[0122] A circulating fluidized bed boiler server (200) according to one embodiment may include a computer program stored on a non-transient computer-readable recording medium to perform S502 to S506 of FIG. 5 described above and / or S602 to S606 of FIG. 6 described above, combined with a computer or computing device which is hardware.

[0123] The circulating fluidized bed boiler server (200) may be implemented as a computing device comprising at least one processor (10) that executes instructions of programs loaded into memory (20). Memory (20) may be loaded with a program including instructions described to execute S502 to S506 of FIG. 5 described above. Furthermore, memory (20) may be loaded with a program including instructions described to execute S602 to S606 of FIG. 6 described above.

[0124] The device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include a plurality of processing elements and / or a plurality of types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are also possible.

[0125] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0126] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware device may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0127] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results can be achieved even if the described techniques are performed in a different order than described, and / or the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0128] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

[0129] The circulating fluidized bed boiler server and the method of operation thereof for optimizing the operating parameters of the circulating fluidized bed boiler as described above can be applied to fields where circulating fluidized bed boilers are used, such as power plants, chemical processes, waste-to-energy facilities, and industrial steam production.

Claims

1. A step of receiving state variables of a circulating fluidized bed boiler in real time; A step of inputting a past time-series state variable corresponding to a first period among the received state variables into a reinforcement learning algorithm, and in response, the reinforcement learning algorithm outputting a recommended operating variable of a circulating fluidized bed boiler for a second period; and The method includes the step of providing the recommended operating variables to the above-mentioned circulating fluidized bed boiler, and The above reinforcement learning algorithm is, It is configured based on the Decision Transformer, and A time-shifting algorithm is applied to output the above recommended driving variables, and A reward determined in a direction that reduces the emission of at least one of sulfur oxides (SOx) and nitrogen oxides (NOx) of the above-mentioned circulating fluidized bed boiler, Method for optimizing operating parameters of a circulating fluidized bed boiler in a circulating fluidized bed boiler server.

2. In Paragraph 1, The above state variable is, at least one of the coal usage, coal calorific value, feedwater amount, and the ratio of coal usage to feedwater amount of the above-mentioned circulating fluidized bed boiler, Method for optimizing operating parameters of a circulating fluidized bed boiler in a circulating fluidized bed boiler server.

3. In Paragraph 2, The above-mentioned circulating fluidized bed boiler includes a plurality of zones, and The above state variable is, For each of the plurality of the above zones, comprising at least one of oxygen concentration, loop room differential pressure, cyclone differential pressure, windbox pressure, combustion furnace upper differential pressure, sulfur oxide (SOx) emissions, nitrogen oxide (NOx) emissions, and boiler bottom temperature, Method for optimizing operating parameters of a circulating fluidized bed boiler in a circulating fluidized bed boiler server.

4. In Paragraph 3, The above recommended driving variables are, Comprising at least one of coal moving air, fuel spoon air, and limestone usage, Method for optimizing operating parameters of a circulating fluidized bed boiler in a circulating fluidized bed boiler server.

5. In Paragraph 4, The above recommended driving variables are, For each of the plurality of the above zones, comprising at least one of the combustion furnace lower pressure, preheating air ratio, primary supply air ratio of the lower combustion furnace, secondary supply air ratio of the upper combustion furnace, coal screw feeder rotation speed, final superheater lift leg air volume, and final reheater lift leg air volume, Method for optimizing operating parameters of a circulating fluidized bed boiler in a circulating fluidized bed boiler server.

6. In Paragraph 1, The above reinforcement learning algorithm is, Includes Decision Transformer, An offline reinforcement learning algorithm learned using state variables, driving variables, and rewards during a first period ((ta) minutes to t minutes) corresponding to a certain past duration based on a specific past point in time (t), wherein a is a natural number. Method for optimizing operating parameters of a circulating fluidized bed boiler in a circulating fluidized bed boiler server.

7. In Paragraph 6, The above reinforcement learning algorithm is, Outputs a driving variable recommended at a first time within the second period, with the state variable, the driving variable, and the compensation masked during a second period corresponding to a certain duration in the future based on the aforementioned specific point in time (t) in the past, and Learning by repeatedly performing the operation of outputting driving variables within the second period multiple times using a time-shifting algorithm, Method for optimizing operating parameters of a circulating fluidized bed boiler in a circulating fluidized bed boiler server.

8. In Paragraph 1, The above reinforcement learning algorithm is, A reward determined in a direction that reduces the temperature deviation between the left and right sides of the lower combustion chamber of the above-mentioned circulating fluidized bed boiler, Method for optimizing operating parameters of a circulating fluidized bed boiler in a circulating fluidized bed boiler server.

9. Memory; and It includes at least one processor configured to communicate with the memory, and The above processor is, Receive state variables of the circulating fluidized bed boiler in real time, and Among the received state variables, the past time series state variable corresponding to the first period is input into the reinforcement learning algorithm, and In response, the above reinforcement learning algorithm outputs recommended operating variables for the circulating fluidized bed boiler during the second period, and It is configured to provide the recommended operating parameters to the above-mentioned circulating fluidized bed boiler, and The above reinforcement learning algorithm is, It is configured based on the Decision Transformer, and A time-shifting algorithm is applied to output the above recommended driving variables, and A reward determined in a direction that reduces the emission of at least one of sulfur oxides (SOx) and nitrogen oxides (NOx) of the above-mentioned circulating fluidized bed boiler, Circulating fluidized bed boiler server.

10. In Paragraph 9, The above-mentioned circulating fluidized bed boiler includes a plurality of zones, and The above state variable is, It includes at least one of the coal usage, coal calorific value, feedwater amount, and the ratio of coal usage to feedwater amount of the above-mentioned circulating fluidized bed boiler, and For each of the plurality of the above zones, comprising at least one of oxygen concentration, loop room differential pressure, cyclone differential pressure, windbox pressure, combustion furnace upper differential pressure, sulfur oxide (SOx) emissions, nitrogen oxide (NOx) emissions, or boiler bottom temperature, Circulating fluidized bed boiler server.

11. In Paragraph 10, The above recommended driving variables are, For each of the plurality of the above zones, comprising at least one of the combustion furnace lower pressure, preheating air ratio, primary supply air ratio of the lower combustion furnace, secondary supply air ratio of the upper combustion furnace, coal screw feeder rotation speed, final superheater lift leg air volume, and final reheater lift leg air volume, Circulating fluidized bed boiler server.

12. In Paragraph 11, The above reinforcement learning algorithm is, Includes Decision Transformer, An offline reinforcement learning algorithm learned using state variables, driving variables, and rewards during a first period ((ta) minutes to t minutes) corresponding to a certain duration in the past based on a specific point in the past (t), wherein a is a natural number. Circulating fluidized bed boiler server.

13. In Paragraph 12, The above reinforcement learning algorithm is, Outputs a driving variable recommended at a first time within the second period, with the state variable, the driving variable, and the compensation masked during a second period corresponding to a certain duration in the future based on the aforementioned specific point in time (t) in the past, and Learning by repeatedly performing the operation of outputting driving variables within the second period multiple times using a time-shifting algorithm, Circulating fluidized bed boiler server.

14. A non-transient computer-readable recording medium comprising instructions for executing a method for optimizing operating variables of a circulating fluidized bed boiler in a circulating fluidized bed boiler server according to paragraph 1.

Images