Boiler control method based on adaptive multi-scale reward deep reinforcement learning algorithm
Optimizing boiler control through an adaptive multi-scale reward deep reinforcement learning algorithm solves the problem that the PID controller cannot accurately control the steam boiler, and improves the stability and efficiency of boiler operation.
Patent Information
- Application Number
- CN202411431268.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-14
- Publication Date
- 2024-12-10
AI Technical Summary
The existing PID controllers are difficult to achieve precise control of the operating status of the steam boiler, resulting in unstable and inefficient boiler operation.
The boiler control method based on the adaptive multi-scale reward deep reinforcement learning algorithm is adopted. Through real-time interactive learning, the control strategy is optimized to adapt to the dynamic changes and uncertainties of the boiler, and the steam data is adjusted to determine the boiler control parameters.
It improves the adaptability and robustness of the boiler control model, realizes precise control of the operating status of the steam boiler, and improves the stability of the operation process and energy supply efficiency.
Smart Images
Figure CN119103527A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of automatic control technology, and in particular to a boiler control method based on an adaptive multi-scale reward deep reinforcement learning algorithm. Background Art
[0002] A steam boiler is a thermal device that uses the thermal energy of fuel or other energy sources to heat water into steam. It is an important energy supply device in various fields. Steam boilers have a wide range of applications and are mainly used to provide steam to meet the needs of various equipment and systems. How to adjust the input parameters such as the amount of water, natural gas and / or cooling water input to the steam boiler to achieve the regulation of the boiler's operating status plays a very critical role in controlling costs and saving energy and reducing consumption during boiler operation.
[0003] At present, PID controller is usually used to adjust the input parameters during the operation of the boiler. However, since the steam boiler is a nonlinear system with fast dynamic changes and uncertainty, the method of adjusting the input parameters by using PID controller cannot achieve accurate control of the steam boiler operation state, and there are problems such as poor stability and low efficiency in the boiler operation process. Summary of the invention
[0004] The present invention provides a boiler control method based on an adaptive multi-scale reward deep reinforcement learning algorithm to achieve precise control of the operating state of a steam boiler, thereby improving the stability and energy supply efficiency of the boiler operation process.
[0005] In a first aspect, an embodiment of the present invention provides a boiler control method based on an adaptive multi-scale reward deep reinforcement learning algorithm, the method comprising:
[0006] The steam data corresponding to the target boiler at the current moment is input into the boiler control model corresponding to the current moment, and the output is used to adjust the boiler control parameters of the target boiler at the next moment; the boiler control model is determined by optimizing the boiler control model corresponding to the previous moment based on the control parameter equity attributes and model evaluation attributes at the previous moment, and the control parameter equity attributes and the model evaluation attributes are determined based on the historical steam data at the historical moment and the preset target information;
[0007] The equipment parameters of the target boiler at the next moment are adjusted based on the boiler control parameters, so that the target boiler works based on the boiler control parameters at the next moment, and the next moment is taken as the current moment.
[0008] In a second aspect, an embodiment of the present invention further provides a boiler control device based on an adaptive multi-scale reward deep reinforcement learning algorithm, the device comprising:
[0009] A control parameter determination module, used to input the steam data corresponding to the target boiler at the current moment into the boiler control model corresponding to the current moment, and output the boiler control parameters for adjusting the target boiler at the next moment; the boiler control model is determined after optimizing the boiler control model corresponding to the previous moment based on the control parameter equity attributes and model evaluation attributes at the previous moment, and the control parameter equity attributes and the model evaluation attributes are determined based on the historical steam data at the historical moment and the preset target information;
[0010] The boiler control module is used to adjust the equipment parameters of the target boiler at the next moment based on the boiler control parameters, so that the target boiler works based on the boiler control parameters at the next moment, and the next moment is used as the current moment.
[0011] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:
[0012] one or more processors;
[0013] a storage device for storing one or more programs,
[0014] When one or more programs are executed by one or more processors, the one or more processors implement a boiler control method based on an adaptive multi-scale reward deep reinforcement learning algorithm as described in any of the embodiments of the present invention.
[0015] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute a boiler control method based on an adaptive multi-scale reward deep reinforcement learning algorithm as described in any of the embodiments of the present invention.
[0016] The technical solution of the embodiment of the present invention is to input the steam data corresponding to the target boiler at the current moment into the boiler control model corresponding to the current moment, and output the boiler control parameters for adjusting the target boiler at the next moment, wherein the boiler control model is determined after optimizing the boiler control model corresponding to the previous moment based on the control parameter equity attribute and model evaluation attribute of the previous moment, and the control parameter equity attribute and model evaluation attribute are determined based on the historical steam data and preset target information of the historical moment, thereby adjusting the equipment parameters of the target boiler at the next moment based on the boiler control parameters, so that the target boiler works based on the boiler control parameters at the next moment, and the next moment is taken as the current moment. The technical solution of the present invention is that in the process of the boiler control model controlling the target boiler, the boiler control model gradually optimizes the control strategy through continuous real-time interactive learning, adapts to the dynamic changes and uncertainties of the boiler, improves the adaptability and robustness of the boiler control model, and realizes precise control of the operating state of the steam boiler, thereby improving the stability and energy supply efficiency of the boiler operation process. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required to describe the embodiments. Obviously, the drawings introduced are only drawings of a part of the embodiments to be described in the present invention, rather than all the drawings. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0018] Figure 1 A schematic flow chart of a boiler control method based on an adaptive multi-scale reward deep reinforcement learning algorithm provided by an embodiment of the present invention;
[0019] Figure 2 A schematic flow chart of another boiler control method based on an adaptive multi-scale reward deep reinforcement learning algorithm provided by an embodiment of the present invention;
[0020] Figure 3 A schematic diagram of the structure of a boiler control device based on an adaptive multi-scale reward deep reinforcement learning algorithm provided by an embodiment of the present invention;
[0021] Figure 4 The present invention is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0022] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.
[0023] Embodiment 1
[0024] Figure 1 A flow chart of a boiler control method based on an adaptive multi-scale reward deep reinforcement learning algorithm provided in an embodiment of the present invention. This embodiment is applicable to the situation where the input parameters of a steam boiler are regulated. The method can be executed by a boiler control device based on an adaptive multi-scale reward deep reinforcement learning algorithm. The device can be implemented in the form of software and / or hardware. The hardware can be an electronic device, such as a mobile terminal, a PC or a server.
[0025] like Figure 1 As shown, the boiler control method based on the adaptive multi-scale reward deep reinforcement learning algorithm includes:
[0026] S110, inputting the steam data corresponding to the target boiler at the current moment into the boiler control model corresponding to the current moment, and outputting the boiler control parameters for adjusting the target boiler at the next moment.
[0027] The target boiler is a boiler device for which control parameters are to be adjusted. The current moment is the current control moment of the target boiler. For example, it is pre-set that the boiler control parameters are determined every preset time period, and the time difference between the current moment and the next moment of the current moment is the preset time period.
[0028] The steam data is the steam status information in the target boiler, and the steam data specifically includes steam load data, steam pressure data, and steam temperature data. The boiler control parameters are boiler input parameters used to control the working status of the boiler, and the boiler control parameters specifically include boiler water inlet, boiler natural gas input, and / or desuperheating water input.
[0029] In this embodiment, the boiler control model is a reinforcement learning model that processes and analyzes steam data and outputs boiler control parameters. In the specific application process, the boiler control model can be updated in real time, that is, the model parameters and / or model structure in the boiler control model can be optimized. For example, the model optimization method can include adjusting the number of hidden layer neurons, adjusting the learning rate, and adjusting the discount factor and other parameters to improve the adaptability and learning efficiency of the model, so as to achieve the purpose of model updating, so that the boiler control parameters determined by the real-time updated boiler control model have stronger adaptability and higher accuracy.
[0030] More specifically, the boiler control model is determined after optimizing the boiler control model corresponding to the previous moment based on the control parameter equity attributes and model evaluation attributes of the previous moment. The control parameter equity attributes and model evaluation attributes are determined based on historical steam data and preset target information at historical moments.
[0031] Among them, the control parameter equity attribute can be understood as the reward information corresponding to the boiler control parameter at the previous moment. The model evaluation attribute is the evaluation value corresponding to the operation result of the boiler control model at the previous moment, which is used to characterize the performance of the boiler control model at the previous moment. For example, the larger the evaluation value, the better the performance of the boiler control model, and the smaller the evaluation value, the worse the performance of the boiler control model. The historical moment is the moment before the current moment, for example, the historical moment is the moment before the current moment. The historical steam data is the steam data corresponding to the target boiler before the current moment. The historical steam data can specifically include historical steam load data, historical steam pressure data, and historical steam temperature data. The preset target information is the ideal value corresponding to the steam data. Specifically, the preset target information can include: a preset pressure target value corresponding to the steam pressure, a preset temperature target value corresponding to the steam temperature, and a preset load target value corresponding to the steam load.
[0032] In this embodiment, the control parameter equity attribute and the model evaluation attribute can be determined based on the error information between the historical steam data and the preset target information. Then, the model parameters of the boiler control model are optimized based on the control parameter equity attribute, and the number of hidden layers in the boiler control model is adjusted based on the model evaluation attribute, thereby realizing the optimization process of the boiler control model.
[0033] In this embodiment, before starting to adjust the control parameters of the target boiler, a set of initial boiler control parameters (i.e., initial boiler water intake, boiler natural gas input, and / or desuperheating water input) can be randomly determined. After the input of the target boiler is controlled according to the initial boiler control parameters, the initial steam data after the initial boiler control parameters act on the target boiler is obtained. When the control parameters of the target boiler are adjusted, i.e., at the initial control moment (corresponding to the first control moment), a boiler control model pre-trained by a large number of historical training samples can be obtained. The boiler control model at this time can be called an initial boiler control model. Thus, the initial steam data can be input into the initial boiler control model, and the initial boiler control model can output the first boiler control parameter. The steam error information can be determined according to the first boiler control parameter and the preset target information, and thus, the first control parameter equity attribute and the first model evaluation attribute can be determined according to the steam error information. Further, the initial boiler control model is subjected to model optimization processing through the first control parameter equity attribute and the first model evaluation attribute to obtain the first boiler control model. The first boiler control model can be saved so that the first boiler control model can be called at the next control moment.
[0034] On this basis, the technical solution provided by this implementation is further described with the second control moment as the current moment. At the second control moment, the boiler control model corresponding to the current moment is the first boiler control model that completes the model optimization at the first control moment, and the steam data corresponding to the target boiler at the current moment (i.e., the second control moment) is obtained, wherein the steam data can be collected by a temperature sensor, a pressure sensor, and / or a load sensor arranged inside the target boiler. Thus, these steam data are input into the first boiler control model, and the first boiler control model analyzes and processes these steam data and outputs the boiler control parameters for adjusting the target boiler at the next moment (i.e., the third control moment). Based on the same control parameter determination method, the boiler control parameters corresponding to the third control moment, the fourth control moment, ..., and the Nth control moment for adjusting the target boiler at the next moment can be determined.
[0035] More specifically, the boiler control model may include an action value mapping layer and an action decision layer, wherein the action value mapping layer is used to use steam data as state information and determine the value parameter corresponding to at least one preset action through a value function. The action decision layer is used to determine the boiler control parameter to be finally executed according to each value parameter.
[0036] Among them, at least one preset execution action includes: a first preset action of increasing the natural gas intake based on a first preset value and increasing the water intake based on a second preset value; a second preset action of decreasing the natural gas intake based on the first preset value and decreasing the water intake based on the second preset value; a third preset action of increasing the natural gas intake based on a third preset value and increasing the water intake based on a fourth preset value; a fourth preset action of decreasing the natural gas intake based on the third preset value and decreasing the water intake based on the fourth preset value; a fifth preset action of increasing the cooling water amount based on a fifth preset value; and a sixth preset action of decreasing the cooling water amount based on a sixth preset value. The first preset value, the second preset value, the third preset value, the fourth preset value, the fifth preset value and the sixth preset value are input amount adjustment amplitude information, and the specific value of each preset value can be adjusted according to actual needs.
[0037] The specific implementation method of inputting the steam data corresponding to the target boiler at the current moment into the boiler control model corresponding to the current moment and outputting the boiler control parameters for adjusting the target boiler at the next moment may include:
[0038] S1101. Input steam data corresponding to the target boiler at the current moment into the action value mapping layer to obtain a value parameter corresponding to at least one preset execution action.
[0039] In this embodiment, the steam data corresponding to the current moment can be input into the action value mapping layer, and the action value mapping layer can output the value parameters corresponding to the first preset action, the second preset action, the third preset action, the fourth preset action, the fifth preset action and the sixth preset action.
[0040] Exemplarily, the output of the action-value mapping layer can be represented in the form of a vector, namely {Q1, Q2, Q3, Q4, Q5, Q6}, wherein Q1 represents the value parameter corresponding to the first preset action, Q2 represents the value parameter corresponding to the second preset action, Q3 represents the value parameter corresponding to the third preset action, Q4 represents the value parameter corresponding to the fourth preset action, Q5 represents the value parameter corresponding to the fifth preset action, and Q6 represents the value parameter corresponding to the sixth preset action.
[0041] S1102. Input each value parameter into the action decision layer to obtain a target value parameter that meets preset conditions.
[0042] Among them, the preset condition is to determine the maximum value parameter value as the target value parameter.
[0043] In this embodiment, the value parameters corresponding to the first preset action, the second preset action, the third preset action, the fourth preset action, the fifth preset action and the sixth preset action can be input into the action decision layer, and the action decision layer outputs the target value parameter, which is the maximum value parameter value.
[0044] S1103. Determine the target preset execution action corresponding to the target value parameter as a boiler control parameter for adjusting the target boiler at the next moment.
[0045] Based on the above example, the value parameter Q5 has the largest value, Q5 is used as the target value parameter, and the fifth preset action is used as the target preset execution action, that is, the fifth preset action (i.e. increasing the amount of cooling water based on the fifth preset value) is used as the boiler control parameter for adjusting the target boiler at the next moment.
[0046] S120. Adjust the equipment parameters of the target boiler at the next moment based on the boiler control parameters, so that the target boiler works based on the boiler control parameters at the next moment, and take the next moment as the current moment.
[0047] The equipment parameters are boiler equipment adjustment information for adjusting boiler water inlet, boiler natural gas input and / or desuperheating water input. For example, the equipment parameters may include: water inlet caliber, water inlet flow information, etc., for adjusting boiler water inlet; gas inlet caliber, for adjusting boiler natural gas input; desuperheating water inlet caliber, desuperheating water inlet flow information, etc., for adjusting desuperheating water input.
[0048] In this embodiment, a correspondence between boiler control parameters and target boiler equipment parameters is established in advance. Based on the boiler control parameters, the equipment parameters corresponding to the target boiler can be determined, so that the target boiler can be adjusted according to the equipment parameters to adjust the boiler water intake, boiler natural gas input and / or desuperheating water input, so that the target boiler can work based on these boiler control parameters at the next moment. Then, the next moment is taken as the current moment, and the above steps S110-S120 are repeatedly executed.
[0049] Based on the above embodiments, the boiler control method provided in this embodiment may further include: recording key data in the regulation process, such as input quantities such as boiler water inlet, boiler natural gas input and / or cooling water input, and output quantities such as steam load, steam pressure, steam temperature, etc., and creating dynamic charts through graphical tools for visual display, which intuitively reflects the degree of consistency between the regulation effect and the regulation target value.
[0050] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of this application are in compliance with the relevant provisions of national laws and regulations. For example, network requests are intercepted and processed with the authorization of the user.
[0051] The technical solution of the embodiment of the present invention is to input the steam data corresponding to the target boiler at the current moment into the boiler control model corresponding to the current moment, and output the boiler control parameters for adjusting the target boiler at the next moment, wherein the boiler control model is determined after optimizing the boiler control model corresponding to the previous moment based on the control parameter equity attribute and model evaluation attribute of the previous moment, and the control parameter equity attribute and model evaluation attribute are determined based on the historical steam data and preset target information of the historical moment, thereby adjusting the equipment parameters of the target boiler at the next moment based on the boiler control parameters, so that the target boiler works based on the boiler control parameters at the next moment, and the next moment is taken as the current moment. The technical solution of the present invention is that in the process of the boiler control model controlling the target boiler, the boiler control model gradually optimizes the control strategy through continuous real-time interactive learning, adapts to the dynamic changes and uncertainties of the boiler, improves the adaptability and robustness of the boiler control model, and realizes precise control of the operating state of the steam boiler, thereby improving the stability and energy supply efficiency of the boiler operation process.
[0052] Embodiment 2
[0053] Figure 2A schematic diagram of a boiler control method based on an adaptive multi-scale reward deep reinforcement learning algorithm provided by an embodiment of the present invention is provided. On the basis of the above-mentioned embodiment, a detailed description is given of how the boiler control model corresponding to the current moment is determined. The specific implementation method can refer to the technical solution of this embodiment. Among them, the technical terms that are the same or corresponding to the above-mentioned embodiment are not repeated here.
[0054] like Figure 2 As shown, the method specifically comprises the following steps:
[0055] S210: Obtain at least one historical steam data and corresponding preset target information.
[0056] The historical steam data is the steam data corresponding to the target boiler at the last moment, and at least one historical steam data includes a historical steam pressure, a historical steam temperature and / or a historical steam load.
[0057] In this embodiment, the preset target information may include: a preset pressure target value corresponding to the steam pressure, for example, the preset pressure target value is 1.3Mpa; a preset temperature target value corresponding to the steam temperature, for example, the preset temperature target value is 300°C; a preset load target value corresponding to the steam load, for example, the preset load target value is the steam load demand value (a value that can be set artificially according to actual production needs).
[0058] S220: Determine the control parameter equity attribute and model evaluation attribute at the previous moment based on the error information between each historical steam data and the corresponding preset target information.
[0059] In this embodiment, the pressure error information can be determined based on the historical steam pressure and the preset pressure target value, the temperature error information can be determined based on the historical steam temperature and the preset temperature target value, and the load error information can be determined based on the historical steam load and the preset load target value. Furthermore, based on the pressure error information, the temperature error information and the load error information, the control parameter equity attributes and the model evaluation attributes at the previous moment can be determined.
[0060] Specifically, based on the error information between each historical steam data and the corresponding preset target information, the specific implementation steps of determining the control parameter equity attribute at the previous moment may include:
[0061] (1) Determine the pressure error information based on the historical steam pressure and the preset pressure target value.
[0062] The pressure error information includes a pressure error weight and a pressure difference.
[0063] In this embodiment, the pressure difference value may be determined by performing a difference operation between the historical steam pressure and the preset steam target value; and the pressure error weight may be determined based on the pressure difference value and the preset weight function.
[0064] Among them, the preset weight function is:
[0065] P scale =1+0.1*P error
[0066] Among them, P scale Represents the pressure error weight value, P error Indicates historical pressure difference.
[0067] In the specific implementation process, the pressure difference is substituted into the above-mentioned preset weight function, and the pressure error weight can be obtained after calculation.
[0068] (2) Determine the temperature error information based on the historical steam temperature and the preset temperature target value.
[0069] The temperature error information includes the temperature error weight and the temperature difference.
[0070] In this embodiment, the temperature difference may be determined by performing a difference operation between the historical steam temperature and the preset steam target value; and the temperature error weight may be determined based on the temperature difference and the preset weight function.
[0071] Among them, the preset weight function is:
[0072] T scale =1+0.1*T error
[0073] Among them, T scale Represents the temperature error weight value, T error Indicates the historical temperature difference.
[0074] In the specific implementation process, the temperature difference is substituted into the above-mentioned preset weight function, and the temperature error weight can be obtained after calculation.
[0075] (3) Determine load error information based on historical steam load and preset load target value.
[0076] Specifically, the absolute value of the difference between the historical steam load and the preset load target value may be used as the load error information.
[0077] Exemplarily, the load error information may be determined by the following formula, including:
[0078] L error = abs(L history -L target )
[0079] Among them, L error Indicates load error information, L history Represents the historical steam load, L target Indicates the preset load target value.
[0080] (4) Determine the control parameter equity attribute based on the pressure error information, temperature error information and steam load information.
[0081] In this embodiment, the control parameter equity attribute can be determined by the following formula, including:
[0082] reward=-(L error +P error ×P scale +T error ×T scale )
[0083] Among them, reward represents the equity attribute of the control parameter, L error Indicates load error information, P error ×P scale Indicates pressure error information, T error ×T scale Indicates temperature error information.
[0084] Specifically, based on the error information between each historical steam data and the corresponding preset target information, the specific implementation steps of determining the model evaluation attribute at the previous moment may include:
[0085] (1) For each historical steam data, based on the mean absolute error between the historical steam data and the corresponding preset target information, determine the to-be-used evaluation value corresponding to each historical steam data.
[0086] Among them, the historical steam data includes historical steam pressure and historical steam temperature.
[0087] In this embodiment, the first mean absolute error can be calculated based on the historical steam pressure and the preset pressure target value, and the second mean absolute error can be calculated based on the historical steam temperature and the preset pressure target value. The first mean absolute error and the second mean absolute error are the evaluation values to be used.
[0088] (2) Sum all the evaluation values to be used to obtain the model evaluation attribute.
[0089] In this embodiment, the model evaluation attribute is obtained by summing the first mean absolute error and the second mean absolute error.
[0090] S230. Based on the control parameter equity attribute, the model parameters of the boiler control model corresponding to the previous moment are optimized, and based on the model evaluation attribute, the model structure of the boiler control model corresponding to the previous moment is optimized to obtain the boiler control model corresponding to the current moment.
[0091] In this embodiment, the model parameters of the boiler control model may be optimized once every preset control moment, or the model parameters of the boiler control model may be optimized once at each control moment. The specific implementation method of optimizing the model parameters of the boiler control model may be: using the control parameter equity attribute as the reward value in the reinforcement learning technology, calculating the target Q value through the Bellman equation and the reward value, and updating the model parameters in the boiler control model by minimizing the difference between the predicted Q value (i.e., the boiler control parameter output by the boiler control model at the previous moment) and the target Q value.
[0092] In this embodiment, the boiler control model includes at least one hidden layer. Based on the model evaluation attribute, the specific implementation method of optimizing the model structure of the boiler control model corresponding to the previous moment may include: if the model evaluation attribute is greater than or equal to the first model evaluation threshold, then adding a preset number of hidden layers to the boiler control model corresponding to the previous moment; if the model evaluation attribute is less than or equal to the second model evaluation threshold, then reducing a preset number of hidden layers to the boiler control model corresponding to the previous moment.
[0093] Among them, the first model evaluation threshold and the second model evaluation threshold are both preset model evaluation thresholds, and the specific value of the first model evaluation threshold is different from that of the second model evaluation threshold.
[0094] Exemplarily, if the model evaluation attribute is greater than or equal to the first model evaluation threshold A, a preset number (for example, the preset number is 4) of hidden layers are added to the boiler control model corresponding to the previous moment to obtain the boiler control model corresponding to the current moment; if the model evaluation attribute is less than or equal to the second model evaluation threshold A / 2, a preset number (for example, the preset number is 4) of hidden layers are reduced from the boiler control model corresponding to the previous moment to obtain the boiler control model corresponding to the current moment. It should be noted that the total number of hidden layers in the boiler control model is not less than 16. By continuously optimizing the structure and parameters of the boiler control model, the generalization ability and adaptability of the model are improved.
[0095] S240: Input the steam data corresponding to the target boiler at the current moment into the boiler control model corresponding to the current moment, and output the boiler control parameters for adjusting the target boiler at the next moment.
[0096] S250. Adjust the equipment parameters of the target boiler at the next moment based on the boiler control parameters, so that the target boiler works based on the boiler control parameters at the next moment, and take the next moment as the current moment.
[0097] The technical solution of the embodiment of the present invention is to first obtain at least one historical steam data and corresponding preset target information when the boiler control model corresponding to the current moment is obtained, so as to determine the control parameter equity attribute and model evaluation attribute of the previous moment based on the error information between each historical steam data and the corresponding preset target information, and further, based on the control parameter equity attribute, optimize the model parameters of the boiler control model corresponding to the previous moment, and based on the model evaluation attribute, optimize the model structure of the boiler control model corresponding to the previous moment, and obtain the boiler control model corresponding to the current moment. The technical solution provided by this embodiment improves the generalization ability and adaptability of the model by continuously optimizing the structure and parameters of the boiler control model, and further realizes the precise control of the operating state of the steam boiler, thereby improving the stability and energy supply efficiency of the boiler operation process.
[0098] Embodiment 3
[0099] Figure 3 A schematic structural diagram of a boiler control device based on an adaptive multi-scale reward deep reinforcement learning algorithm provided in an embodiment of the present invention, the device includes: a control parameter determination module 310 and a boiler control module 320.
[0100] Among them, the control parameter determination module 310 is used to input the steam data corresponding to the target boiler at the current moment into the boiler control model corresponding to the current moment, and output the boiler control parameters for adjusting the target boiler at the next moment; the boiler control model is determined after optimizing the boiler control model corresponding to the previous moment based on the control parameter equity attributes and model evaluation attributes at the previous moment, and the control parameter equity attributes and the model evaluation attributes are determined based on the historical steam data and preset target information at the historical moment;
[0101] The boiler control module 320 is used to adjust the equipment parameters of the target boiler at the next moment based on the boiler control parameters, so that the target boiler works based on the boiler control parameters at the next moment, and the next moment is used as the current moment.
[0102] On the basis of the above device, optionally, the boiler control model includes an action value mapping layer and an action decision layer, and the control parameter determination module 310 includes:
[0103] A value parameter determination unit, used to input the steam data corresponding to the target boiler at the current moment into the action value mapping layer to obtain a value parameter corresponding to at least one preset execution action;
[0104] A target value determination unit, used to input each of the value parameters into the action decision layer to obtain a target value parameter that meets a preset condition;
[0105] A boiler control parameter determination unit, used to determine the target preset execution action corresponding to the target value parameter as a boiler control parameter for adjusting the target boiler at the next moment;
[0106] Among them, the at least one preset execution action includes: a first preset action of increasing the natural gas intake based on a first preset value and increasing the water intake based on a second preset value; a second preset action of decreasing the natural gas intake based on the first preset value and decreasing the water intake based on the second preset value; a third preset action of increasing the natural gas intake based on a third preset value and increasing the water intake based on a fourth preset value; a fourth preset action of decreasing the natural gas intake based on the third preset value and decreasing the water intake based on the fourth preset value; a fifth preset action of increasing the cooling water amount based on the fifth preset value; and a sixth preset action of decreasing the cooling water amount based on the sixth preset value.
[0107] On the basis of the above device, optionally, the boiler control device based on the adaptive multi-scale reward deep reinforcement learning algorithm further includes: a boiler control model optimization module; the boiler control model optimization module specifically includes:
[0108] A historical data acquisition unit, used to acquire at least one historical steam data and corresponding preset target information; wherein the historical steam data is the steam data corresponding to the target boiler at the previous moment, and the at least one historical steam data includes historical steam pressure, historical steam temperature and / or historical steam load;
[0109] an attribute determination unit, configured to determine a control parameter equity attribute and a model evaluation attribute at a previous moment based on error information between each of the historical steam data and corresponding preset target information;
[0110] A model optimization unit is used to optimize the model parameters of the boiler control model corresponding to the previous moment based on the control parameter equity attribute, and to optimize the model structure of the boiler control model corresponding to the previous moment based on the model evaluation attribute to obtain the boiler control model corresponding to the current moment.
[0111] Based on the above-mentioned device, optionally, the attribute determination unit includes: a control parameter equity attribute determination subunit, which is specifically used to determine the pressure error information based on the historical steam pressure and the preset pressure target value; determine the temperature error information based on the historical steam temperature and the preset temperature target value; determine the load error information based on the historical steam load and the preset load target value; determine the control parameter equity attribute based on the pressure error information, the temperature error information and the steam load information.
[0112] On the basis of the above device, optionally, the control parameter equity attribute determination subunit is further used to perform a difference operation between the historical steam pressure and the preset steam target value to determine the pressure difference; based on the pressure difference and a preset weight function, determine the pressure error weight; wherein the preset weight function is:
[0113] P scale =1+0.1*P error
[0114] Among them, P scale represents the pressure error weight value, P error Indicates the historical pressure difference.
[0115] On the basis of the above device, optionally, the control parameter equity attribute determination subunit is further configured to use the absolute value of the difference between the historical steam load and the preset load target value as the load error information.
[0116] Based on the above-mentioned device, optionally, the attribute determination unit includes: a model evaluation attribute determination subunit, which is specifically used to determine, for each of the historical steam data, an evaluation value to be used corresponding to each of the historical steam data based on the mean absolute error between the historical steam data and the corresponding preset target information; and to sum up each of the evaluation values to be used to obtain the model evaluation attribute.
[0117] Based on the above-mentioned device, optionally, the model optimization unit includes: a model structure optimization subunit; a model structure optimization subunit, which is specifically used to add a preset number of hidden layers to the boiler control model corresponding to the previous moment if the model evaluation attribute is greater than or equal to the first model evaluation threshold; if the model evaluation attribute is less than or equal to the second model evaluation threshold, then reduce a preset number of hidden layers for the boiler control model corresponding to the previous moment.
[0118] The technical solution of the embodiment of the present invention is to input the steam data corresponding to the target boiler at the current moment into the boiler control model corresponding to the current moment, and output the boiler control parameters for adjusting the target boiler at the next moment, wherein the boiler control model is determined after optimizing the boiler control model corresponding to the previous moment based on the control parameter equity attribute and model evaluation attribute of the previous moment, and the control parameter equity attribute and model evaluation attribute are determined based on the historical steam data and preset target information of the historical moment, thereby adjusting the equipment parameters of the target boiler at the next moment based on the boiler control parameters, so that the target boiler works based on the boiler control parameters at the next moment, and the next moment is taken as the current moment. The technical solution of the present invention is that in the process of the boiler control model controlling the target boiler, the boiler control model gradually optimizes the control strategy through continuous real-time interactive learning, adapts to the dynamic changes and uncertainties of the boiler, improves the adaptability and robustness of the boiler control model, and realizes precise control of the operating state of the steam boiler, thereby improving the stability and energy supply efficiency of the boiler operation process.
[0119] The boiler control device based on the adaptive multi-scale reward deep reinforcement learning algorithm provided in an embodiment of the present invention can execute the boiler control method based on the adaptive multi-scale reward deep reinforcement learning algorithm provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0120] It is worth noting that the various units and modules included in the above system are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present invention.
[0121] Embodiment 4
[0122] Figure 4 The present invention is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 4 A block diagram of an exemplary electronic device 40 suitable for implementing exemplary embodiments of the present invention is shown. Figure 4 The electronic device 40 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0123] like Figure 4 As shown, the electronic device 40 is in the form of a general computing device. The components of the electronic device 40 may include but are not limited to: one or more processors or processing units 401, a system memory 402, and a bus 403 connecting different system components (including the system memory 402 and the processing unit 401).
[0124] Bus 403 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.
[0125] The electronic device 40 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 40, including volatile and non-volatile media, removable and non-removable media.
[0126] System memory 402 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 404 and / or cache memory 405. Electronic device 40 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 406 may be used to read and write non-removable, non-volatile magnetic media ( Figure 4 not shown, usually called a "hard drive"). Although Figure 4 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 403 via one or more data medium interfaces. Memory 402 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of various embodiments of the present invention.
[0127] A program / utility 408 having a set (at least one) of program modules 407 may be stored, for example, in memory 402, such program modules 407 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 407 generally perform the functions and / or methods of the embodiments described herein.
[0128] The electronic device 40 may also communicate with one or more external devices 409 (e.g., keyboards, pointing devices, displays 810, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 40, and / or may communicate with any device that enables the electronic device 40 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface 411. Furthermore, the electronic device 40 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 412. As shown, the network adapter 412 communicates with other modules of the electronic device 40 via the bus 403. It should be understood that although Figure 4 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 40, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0129] The processing unit 401 executes various functional applications and page processing by running the program stored in the system memory 402, such as implementing the boiler control method based on the adaptive multi-scale reward deep reinforcement learning algorithm provided in the embodiment of the present invention.
[0130] Embodiment 5
[0131] An embodiment of the present invention further provides a storage medium containing computer executable instructions, which, when executed by a computer processor, is used to execute a boiler control method based on an adaptive multi-scale reward deep reinforcement learning algorithm, the method comprising:
[0132] The steam data corresponding to the target boiler at the current moment is input into the boiler control model corresponding to the current moment, and the output is used to adjust the boiler control parameters of the target boiler at the next moment; the boiler control model is determined by optimizing the boiler control model corresponding to the previous moment based on the control parameter equity attributes and model evaluation attributes at the previous moment, and the control parameter equity attributes and the model evaluation attributes are determined based on the historical steam data at the historical moment and the preset target information;
[0133] The equipment parameters of the target boiler at the next moment are adjusted based on the boiler control parameters, so that the target boiler works based on the boiler control parameters at the next moment, and the next moment is taken as the current moment.
[0134] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.
[0135] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0136] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
[0137] Computer program code for performing the operation of embodiments of the present invention may be written in one or more programming languages or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0138] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims
1. A boiler control method based on an adaptive multi-scale reward deep reinforcement learning algorithm, characterized in that: include: The steam data corresponding to the target boiler at the current moment is input into the boiler control model corresponding to the current moment, and the output is used to adjust the boiler control parameters of the target boiler at the next moment; the boiler control model is determined by optimizing the boiler control model corresponding to the previous moment based on the control parameter equity attributes and model evaluation attributes at the previous moment, and the control parameter equity attributes and the model evaluation attributes are determined based on the historical steam data at the historical moment and the preset target information; The equipment parameters of the target boiler at the next moment are adjusted based on the boiler control parameters, so that the target boiler works based on the boiler control parameters at the next moment, and the next moment is taken as the current moment.
2. The method according to claim 1, characterized in that The boiler control model includes an action value mapping layer and an action decision layer, wherein the steam data corresponding to the target boiler at the current moment is input into the boiler control model corresponding to the current moment, and the output is used to adjust the boiler control parameters of the target boiler at the next moment, including: Inputting the steam data corresponding to the target boiler at the current moment into the action value mapping layer to obtain a value parameter corresponding to at least one preset execution action; Input each of the value parameters into the action decision layer to obtain a target value parameter that meets the preset conditions; Determining the target preset execution action corresponding to the target value parameter as a boiler control parameter for adjusting the target boiler at the next moment; Among them, the at least one preset execution action includes: a first preset action of increasing the natural gas intake based on a first preset value and increasing the water intake based on a second preset value; a second preset action of decreasing the natural gas intake based on the first preset value and decreasing the water intake based on the second preset value; a third preset action of increasing the natural gas intake based on a third preset value and increasing the water intake based on a fourth preset value; a fourth preset action of decreasing the natural gas intake based on the third preset value and decreasing the water intake based on the fourth preset value; a fifth preset action of increasing the cooling water amount based on the fifth preset value; and a sixth preset action of decreasing the cooling water amount based on the sixth preset value.
3. The method according to claim 1, characterized in that The boiler control model corresponding to the current moment is determined in the following manner: Acquire at least one historical steam data and corresponding preset target information; wherein the historical steam data is the steam data corresponding to the target boiler at the last moment, and the at least one historical steam data includes historical steam pressure, historical steam temperature and / or historical steam load; Determining the control parameter equity attribute and the model evaluation attribute at the last moment based on the error information between each of the historical steam data and the corresponding preset target information; Based on the control parameter equity attribute, the model parameters of the boiler control model corresponding to the previous moment are optimized, and based on the model evaluation attribute, the model structure of the boiler control model corresponding to the previous moment is optimized to obtain the boiler control model corresponding to the current moment.
4. The method according to claim 3, characterized in that The determining of the control parameter equity attribute at the previous moment based on the error information between each of the historical steam data and the corresponding preset target information includes: Determining pressure error information based on the historical steam pressure and a preset pressure target value; Determining temperature error information based on the historical steam temperature and a preset temperature target value; Determining load error information based on the historical steam load and a preset load target value; A control parameter equity attribute is determined based on the pressure error information, the temperature error information, and the steam load information.
5. The method according to claim 4, characterized in that The pressure error information includes a pressure error weight and a pressure difference. The pressure error information is determined based on the historical steam pressure and the preset pressure target value, including: Performing a difference operation on the historical steam pressure and the preset steam target value to determine the pressure difference; The pressure error weight is determined based on the pressure difference and a preset weight function; wherein the preset weight function is: P scale =1+0.1*P error Among them, P scale represents the pressure error weight value, P error Indicates the historical pressure difference.
6. The method according to claim 4, characterized in that The step of determining load error information based on the historical steam load and the preset load target value includes: The absolute value of the difference between the historical steam load and the preset load target value is used as load error information.
7. The method according to claim 3, characterized in that The determining of the model evaluation attribute at the previous moment based on the error information between each of the historical steam data and the corresponding preset target information includes: For each of the historical steam data, determining an evaluation value to be used corresponding to each of the historical steam data based on a mean absolute error between the historical steam data and corresponding preset target information; A sum operation is performed on each of the evaluation values to be used to obtain the model evaluation attribute.
8. The method according to claim 3, characterized in that The boiler control model includes at least one hidden layer, and the optimizing the model structure of the boiler control model corresponding to the previous moment based on the model evaluation attribute includes: If the model evaluation attribute is greater than or equal to the first model evaluation threshold, adding a preset number of hidden layers to the boiler control model corresponding to the previous moment; If the model evaluation attribute is less than or equal to the second model evaluation threshold, a preset number of hidden layers are reduced for the boiler control model corresponding to the previous moment.
9. A boiler control device based on an adaptive multi-scale reward deep reinforcement learning algorithm, characterized in that: The device includes: A control parameter determination module, used to input the steam data corresponding to the target boiler at the current moment into the boiler control model corresponding to the current moment, and output the boiler control parameters for adjusting the target boiler at the next moment; the boiler control model is determined after optimizing the boiler control model corresponding to the previous moment based on the control parameter equity attributes and model evaluation attributes at the previous moment, and the control parameter equity attributes and the model evaluation attributes are determined based on the historical steam data at the historical moment and the preset target information; The boiler control module is used to adjust the equipment parameters of the target boiler at the next moment based on the boiler control parameters, so that the target boiler works based on the boiler control parameters at the next moment, and the next moment is used as the current moment.
10. An electronic device, characterized in that: Electronic equipment includes: one or more processors; a storage device for storing one or more programs, When one or more programs are executed by one or more processors, the one or more processors implement the boiler control method based on the adaptive multi-scale reward deep reinforcement learning algorithm as described in any one of claims 1-8.