Intelligent control method and device for boiler

Abstract

One or more embodiments of the invention provide an intelligent control method and device for a boiler. The method comprises the following steps: acquiring an image feature vector of a boiler flame image, operation data of a boiler and a control action list; deciding from the control action list based on a reinforcement learning algorithm to obtain a target control action; wherein the state of the reinforcement learning algorithm is the image feature vector of the boiler flame image and the operation data of the boiler, and the action of the reinforcement learning algorithm is the control action selected from the control action list. The reward value of the reinforcement learning algorithm is obtained based on a control action selected from the control action list; and controlling the boiler by using the target control action. By increasing the image feature vector of the flame image, more comprehensive feature information can be provided for an automatic control system of the boiler, so that the target control action is more accurate, and the intelligent control effect is improved.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of intelligent control, and in particular, to a method and device for intelligent control of a boiler. Background Art

[0002] The combustion process of industrial boilers has characteristics such as large delay, large inertia, large time variation, and nonlinearity. The automatic control of its full working conditions has long been one of the difficult problems in the thermal industry. In recent years, relevant technologies have combined machine algorithms and combustion mechanisms to develop intelligent combustion optimization algorithms for industrial boilers. Based on the operation data of the boiler, a prediction model is built to achieve automatic control of the boiler. However, since the operation data of the boiler cannot directly reflect the actual state of the flame combustion in the boiler, the automatic control effect of the boiler is poor. Summary of the invention

[0003] In view of this, one or more embodiments of the present specification provide a method and device for intelligently controlling a boiler to solve the problems existing in the related art.

[0004] To achieve the above objectives, one or more embodiments of this specification provide the following technical solutions:

[0005] According to a first aspect of one or more embodiments of this specification, a boiler intelligent control method is proposed, including:

[0006] Obtaining an image feature vector of a boiler flame image, boiler operation data, and a control action list;

[0007] A target control action is obtained by decision-making from the control action list based on a reinforcement learning algorithm; wherein the state of the reinforcement learning algorithm is the image feature vector of the boiler flame image and the operation data of the boiler, the action of the reinforcement learning algorithm is the control action selected from the control action list, and the reward value of the reinforcement learning algorithm is obtained based on the control action selected from the control action list;

[0008] The boiler is controlled using the target control action.

[0009] According to a second aspect of one or more embodiments of this specification, an intelligent control device for a boiler is provided, comprising:

[0010] An acquisition module, used to obtain image feature vectors of boiler flame images, boiler operation data, and a control action list;

[0011] A decision module, configured to obtain a target control action by making a decision from the control action list based on a reinforcement learning algorithm; wherein the state of the reinforcement learning algorithm is the image feature vector of the boiler flame image and the operation data of the boiler, the action of the reinforcement learning algorithm is the control action selected from the control action list, and the reward value of the reinforcement learning algorithm is obtained based on the control action selected from the control action list;

[0012] A control module is used to control the boiler using the target control action.

[0013] According to a third aspect of one or more embodiments of this specification, an electronic device is provided, including:

[0014] processor;

[0015] a memory for storing processor-executable instructions;

[0016] The processor implements the method as described in the first aspect by running the executable instructions.

[0017] According to a fourth aspect of one or more embodiments of the present specification, a computer-readable storage medium is provided, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0018] In the above technical solution, when realizing intelligent control of the boiler, the image feature vector of the boiler flame image is used, and the image feature vector and the boiler operation data are used as the state of the reinforcement learning algorithm to determine the target control action from the control action list. By adding the image feature vector of the flame image, more comprehensive feature information can be provided for the automatic control system of the boiler, so that the target control action can be obtained more accurately, thereby improving the intelligent control effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of an intelligent control method of a boiler provided by an exemplary embodiment.

[0020] Figure 2 It is a schematic diagram of the structure of a variational autoencoder provided by an exemplary embodiment.

[0021] Figure 3 It is a structural diagram of a deep Q network model provided by an exemplary embodiment.

[0022] Figure 4 It is a training diagram of a deep Q network model provided by an exemplary embodiment.

[0023] Figure 5It is a schematic diagram of the structure of an intelligent control device for a boiler provided by an exemplary embodiment.

[0024] Figure 6 It is a schematic structural diagram of an electronic device provided by an exemplary embodiment. DETAILED DESCRIPTION

[0025] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0026] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0027] Related technologies The combustion process of industrial boilers has characteristics such as large delay, large inertia, large time variation, and nonlinearity. The automatic control of its full working conditions has long been one of the difficult problems in the thermal industry. At present, there are two mainstream methods for the automatic control model of industrial boilers at the application level and research level: mechanism model and data-driven model.

[0028] Mechanism model: Based on heat transfer and fluid mechanics equations, a model of the boiler combustion process is established, which is then simplified into a transfer function model and applied to the control system. Its advantages are strong interpretability, and its disadvantages are: the characteristics of different types of boilers vary greatly, and it is difficult to determine the coefficient parameters when establishing the model; the accuracy is low, and most of the results are qualitative and cannot obtain accurate quantitative prediction values; the boiler characteristics will change over time, and the mechanism model will gradually mismatch over time, and its generalization ability is weak. The method of using predictive control based on the mechanism model to achieve automatic control of the boiler cannot overcome the nonlinear and time-varying characteristics of the boiler, so the control performance decays quickly and the subsequent maintenance cost is very high.

[0029] Data-driven model: Based on machine learning methods, historical operating data is used to establish a data-driven black box model. Its model accuracy and generalization performance are significantly better than the mechanism model. However, in the industrial boiler scenario, in addition to operating data, the information of the flame image burning in the boiler is also an important reference for operation control. However, the input of all data-driven models is operating data, and the information of the flame image is not incorporated, resulting in the established model not containing all the characteristics of the combustion system. The optimal control quantity obtained by optimization solution or reinforcement learning based on this model may not be accurate.

[0030] In summary, there are still two key problems in the current boiler intelligent control system:

[0031] 1. Lack of image information: Currently, manual control of boilers is mainly based on observing the boiler’s flame monitor, supplemented by reference to the boiler’s operating data. The prediction models in intelligent control systems are mostly data-driven and lack key flame image information, resulting in poor automatic control results from predictions.

[0032] 2. Control performance deteriorates over time: Due to equipment aging or changes in raw materials, boiler operating conditions will change significantly over time. Correspondingly, the performance of the automatic control system based on the predictive model will also deviate greatly and deteriorate, requiring a large amount of manpower for operation, maintenance and optimization.

[0033] In view of this, this specification proposes a method for intelligent control of a boiler. The image feature vector of the flame image is obtained by extracting information from the collected flame image; based on the reinforcement learning model, the image feature vector of the flame image and the boiler operation data are used as the environment state of the reinforcement learning, and the target control action is obtained from the control action list to realize intelligent control of the boiler.

[0034] During implementation, an image feature vector of a boiler flame image, operating data of the boiler, and a control action list are obtained; the image feature vector and the operating data of the boiler are used as the state of a reinforcement learning algorithm, and a control action selected from the control action list is used as an action of the reinforcement learning algorithm, so as to decide a target control action from the control action list to control the boiler; wherein a reward value of the reinforcement learning algorithm is obtained based on the control action selected from the control action list.

[0035] In the above technical solution, when realizing intelligent control of the boiler, the image feature vector of the boiler flame image is used, and the image feature vector and the boiler operation data are used as the state of the reinforcement learning algorithm to determine the target control action from the control action list. By adding the image feature vector of the flame image, more comprehensive feature information can be provided for the automatic control system of the boiler, so that the target control action can be obtained more accurately, thereby improving the intelligent control effect.

[0036] In order to enable those skilled in the art to better understand the technical solutions in the present application, the technical solutions in this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0037] See also Figure 1 , Figure 1 An exemplary embodiment provides a method for intelligently controlling a boiler, which can be executed by an intelligent control system of the boiler, and the intelligent control system can include a control terminal or a control server of the boiler.

[0038] The boiler intelligent control method may include the following steps.

[0039] S110, obtaining an image feature vector of a boiler flame image, boiler operation data, and a control action list.

[0040] The intelligent control system can collect the flame image of the boiler through a pre-deployed image collection device, such as a flame monitor, and perform feature extraction on the flame image to obtain an image feature vector of the boiler flame image.

[0041] The obtained image feature vector can be used to represent the features of the entire flame image, and can also be used to represent the features of a portion of graphics or key points in the flame image, for example, used to represent the features of the flame edge in the flame image.

[0042] The feature extraction method can be various. For example, the color distribution in the flame image can be statistically analyzed to obtain a feature vector of a specific length to represent the color characteristics of the flame image; or, the key points in the flame image can be detected, the local features around the key points can be extracted, and a feature vector of a specific length can be generated to represent the image; or, the image can be converted to the frequency domain, taking Fourier transform as an example, the frequency information of the flame image can be provided, and a feature vector can be generated to represent the image; or, a deep learning model can be used to extract abstract features from the image, and then these features can be represented as vectors; and so on.

[0043] The intelligent control system can also collect various operating data of the boiler during operation by deploying various data acquisition devices on the system, such as sensors, for example, pusher speed, grate speed, primary fan frequency, secondary fan frequency, steam flow, furnace temperature, oxygen content after economizer, drum water level, etc.

[0044] The intelligent control system can collect flame images to obtain image feature vectors of the flame images and various boiler operation data based on a certain collection frequency. The collection frequency can be set according to actual needs, for example, once every 20 seconds (s), once every minute, etc.

[0045] The intelligent control system can pre-maintain a list of control actions for controlling the boiler. The control action list can include control actions for adjusting the operating states of several operating devices of the boiler respectively, such as control actions for adjusting the pusher speed, grate speed, primary fan frequency, secondary fan frequency, etc. The control action list can be obtained based on the control action combination of each control action.

[0046] S120. Obtaining a target control action from the control action list based on a reinforcement learning algorithm; wherein the state of the reinforcement learning algorithm is the image feature vector of the boiler flame image and the operating data of the boiler, the action of the reinforcement learning algorithm is the control action selected from the control action list, and the reward value of the reinforcement learning algorithm is obtained based on the control action selected from the control action list.

[0047] A preset reinforcement learning algorithm may be used to determine the target control action from the control action list A based on the image feature vector of the boiler flame image acquired at the current time t0 and the boiler operation data.

[0048] The intelligent control system can serve as an agent in the reinforcement learning algorithm, the image feature vector of the boiler flame image and the boiler operation data can serve as the state S of the environment of the reinforcement learning algorithm, and the reward value of the reinforcement learning algorithm can be calculated based on the control action selected from the control action list.

[0049] The environment can be based on the state S at time t t and a control action a selected from the control action list, predicting the state S at the next time t+1 after executing the selected control action a t+1 , and calculate the reward value R after executing the selected control action a. The reward value can be used to represent the state S at the next time t+1 after executing the selected control action a. t+1 The predicted values ​​of some key target operating data are in line with the preset safety value range.

[0050] The intelligent control system can decide from the control action list the target control action for the state by interacting with the environment, that is, based on the image feature vector of the acquired boiler flame image and the boiler operation data, decide from the control action list the target control action that meets the specific goal.

[0051] Among them, the specific goal can be set according to actual needs. For example, the target control action that meets the specific goal can be a control action that can obtain the highest reward value; or based on a preset value function, the target control action that meets the specific goal can be a control action that can obtain the highest value.

[0052] For example, in one embodiment, the state S at the current time t0 can be obtained. t0 , that is, the image feature vector of the boiler flame image and the boiler operation data at the current time t0. The reward values ​​corresponding to the respective control actions in the control action list are calculated, and the control action with the highest reward value is determined as the target control action.

[0053] Among them, the state S at the current time t0 t0 It may include the image feature vector of the boiler flame image and the boiler operation data acquired within the sampling period corresponding to the current time t0.

[0054] S130: Use the target control action to control the boiler.

[0055] After the target control action is determined from the control action list based on the reinforcement learning algorithm, the target control action can be used to control the boiler.

[0056] In the above embodiment, when realizing intelligent control of the boiler, the image feature vector of the flame image of the boiler is used, and the image feature vector and the boiler operation data are used as the state of the reinforcement learning algorithm to decide the target control action from the control action list. Accordingly, by adding the image feature vector of the flame image, more comprehensive feature information can be provided for the automatic control system of the boiler, so that the target control action can be obtained more accurately, thereby improving the intelligent control effect.

[0057] In one implementation, a variational autoencoder (VAE) may be used to extract features from the acquired boiler flame image to obtain an image feature vector of the boiler flame image.

[0058] Variational autoencoder is a generative model based on deep learning, such as Figure 2 As shown in the figure, the variational autoencoder includes an encoder and a decoder. The picture code of the original image is input into the encoder to perform the encoding process, so as to map the picture code of the original image to a low-dimensional latent space; then the decoder randomly samples in the latent space and uses the random sampling points to reconstruct a generated image with similar properties to the input boiler flame image.

[0059] The variational autoencoder needs to be trained in advance. During the training process, the training objectives of the variational autoencoder may include minimizing the deviation between the original image and the generated image, and minimizing the difference between the distribution of the latent variable in the latent space and the standard normal distribution. Taking minimizing the deviation between the original image and the generated image as an example, the deviation between the original image and the generated image may be defined as a loss function, such as the mean squared error (MSE) or binary cross entropy (BCE) may be used to calculate and train the variational autoencoder.

[0060] Historical boiler flame images can be exported from the database of the intelligent control system as training image samples for training the variational autoencoder.

[0061] In order to ensure the quality of training, each training image sample can be preprocessed before using the training image sample to train the variational autoencoder. The preprocessing can include filtering the training image samples based on the preset clarity requirements, deleting blurred images that do not meet the clarity requirements, and unifying the format and size of each training image sample.

[0062] There are various ways to filter the training image samples based on the preset clarity requirements, for example, the Laplacian operator of each historical boiler flame image is calculated, and the Laplacian operator can detect high-frequency detail information such as edges and textures in the image, which is equivalent to detecting the clarity of the image. Images whose calculation results of the Laplacian operator are lower than the preset threshold are considered to be blurred images, and blurred images whose calculation results are lower than the preset threshold need to be deleted from the training image samples.

[0063] The image encoding of the training image sample after preprocessing is input into the variational autoencoder for training. Taking the input image as an RGB color image as an example, the image encoding of the input training image sample can be a high-dimensional vector representation consisting of the numerical values ​​of the RGB three channels of each pixel.

[0064] The encoder is generally a neural network, for example, the encoder may include three fully connected layers, and the activation function adopts the LeakyRuLU function. The encoder may map the input image encoding to the mean vector z_mean and the variance vector z_log_var in the low-dimensional latent space, and then use these parameters to describe the Gaussian distribution in the latent space.

[0065] The latent variable output by the encoder may be obtained by performing a reparameterization trick based on the mean vector z_mean and the variance vector z_log_var. For example, the latent variable z may be calculated using the following formula:

[0066] z=z_mean+exp(z_log_variance)*epsilon, where epsilon can be a random tensor with a very small value, and z is a latent variable obtained by compressing the input image encoding, and the latent variable can be a low-dimensional representation of encoding the input image into the latent space.

[0067] The decoder is also usually a neural network, for example, the decoder may include three fully connected layers. The decoder may map the information in the latent space back to the data space corresponding to the input image code, and reconstruct a high-dimensional image code, and the reconstructed image code has similar characteristics to the input image code.

[0068] A loss function can be determined based on the difference between the reconstructed image encoding and the input image encoding, and the parameters of the variational autoencoder are updated based on the loss function.

[0069] The loss function may also include a regularization loss, which may be used to prevent the initial noise from being too small and to prevent the model from overfitting.

[0070] After completing the training of the variational autoencoder, the trained model can be saved and deployed.

[0071] In one embodiment, the image encoding of the acquired boiler flame image can be input into the encoder of the trained variational autoencoder, and the latent variables in the latent space output by the encoder can be considered as the low-dimensional expression obtained by compressing the key information of the boiler flame image, and the latent variables output by the encoder can be used as the image feature vector of the boiler flame image. Based on the reinforcement learning algorithm, the image feature vector of the boiler flame image output by the encoder and the acquired boiler operation data are used as the state of the reinforcement learning algorithm at the current time t0, and the target control action is determined from the control action list A to control the boiler.

[0072] In the above embodiment, by training the variational autoencoder, the key features of the boiler flame image can be learned to be expressed in a low-dimensional vector in the latent space, so that the obtained image feature vector of the boiler flame image can not only retain the key features of the image, but also reduce the difficulty of data processing by compressing and reducing the dimensionality of the image data.

[0073] It should be noted that the reinforcement learning algorithm used in the embodiments of the present disclosure may be replaced by other decision optimization methods, such as but not limited to model predictive control, nonlinear programming, particle swarm, genetic algorithm, ant colony algorithm, etc. However, for the sake of simplicity, the reinforcement learning algorithm is used as an example in the following embodiments.

[0074] The reinforcement learning algorithm can be varied, for example, it can include model-based and model-free learning methods, value-based and policy-based learning methods, etc.

[0075] In one embodiment, the reinforcement learning algorithm may be a deep Q-network (DQN) learning algorithm, which approximates the Q function through a pre-constructed deep Q-network model, and the Q function may be a preset action value function, and the calculated Q value may be used to represent the expected cumulative reward obtained after executing the control action a selected from the control action list under the state s including the image feature vector of the boiler flame image and the operation data of the boiler. The Q function may be expressed as Q(s, a).

[0076] In the above step S120, the target control action is obtained from the control action list based on the reinforcement learning algorithm, including:

[0077] Based on the deep Q network learning algorithm, a target control action with the maximum Q value is determined from the control action list.

[0078] The deep Q network model may be a deep neural network model, which may be constructed according to actual needs. For example, in one embodiment, a deep Q network model may be constructed based on an environment model for predicting future operation data of a boiler. Figure 3 As shown, the input of the environmental model may include the state acquired at the current moment and the control action selected from the control action list, the state may include the image feature vector of the boiler flame image obtained by using the encoder of the variational autoencoder and the acquired boiler operation data, and the output of the environmental model may be the operation data predicted for the future moment, and the predicted operation data may include the target operation data; based on the predicted target operation data, the reward value corresponding to the selected control action can also be calculated, and then the Q value corresponding to the selected control action can also be calculated.

[0079] After obtaining the Q values ​​corresponding to the respective control actions in the control action list through the deep Q network model, the target control action with the maximum Q value can be determined from the control action list, and then the boiler can be controlled using the target control action.

[0080] When training the constructed deep Q network model, it can include two parts: offline training and online training. Among them, the offline training can be the training of the environment model.

[0081] Among them, the environmental model can be constructed in various ways, and a machine learning model, a lightweight gradient boosting machine (Light Gradient Boosting Machine, LightGBM) model can be used, and an extreme gradient boosting (eXtreme Gradient Boosting, XGBoost) model, a classification boosting (Categorical Boosting, CatBoost) model, a convolutional neural network (Convolutional Neural Network, CNN), a long short-term memory network (Long Short-Term Memory, LSTM), a recurrent neural network (Recurrent Neural Network, RNN), etc. can also be used.

[0082] Offline training of the environment model may utilize historical data in a database to perform supervised training on the environment model, and the historical data may include historical images, historical operation data of the boiler, and historical control actions.

[0083] The historical image may be an image captured from a historical flame video according to a preset acquisition frequency, and the captured image may be used as a historical image after being marked according to the time of shooting.

[0084] The historical data can be preprocessed, which may include cleaning and filtering the historical data, removing null values, bad values, and abnormal values, performing time homogenization, and eliminating data from abnormal operating conditions. The preprocessing of historical images may include image cleaning based on the Lappacian operator as described above, as well as format and size unification.

[0085] For the preprocessed historical images, the historical image feature vector can be obtained based on the encoder of the variational autoencoder.

[0086] The historical image feature vectors of the previous period in the historical data, the historical operation data of the boiler, and the action data of the historical control action are used as training sample features, and the historical target operation data of the next period in the historical data are used as sample labels to perform supervised training on the environmental model. The duration of the previous period and the next period can be set according to actual needs. For example, taking five minutes as an example, based on the time mark associated with the historical data, the historical image feature vectors of the previous five minutes, the historical operation data of the boiler, and the action data of the historical control action can be spliced ​​as the input vector of the environmental model, and the historical operation status of the next five minutes can be used as the sample label of the input vector to perform supervised training on the environmental model. The trained environmental model can be used to predict the predicted value of the operation data of the next five minutes of the current moment using the image feature vectors, operation data, and action data of the selected control action obtained within five minutes before the current moment as input vectors.

[0087] Based on the trained environment model, the deep Q network learning algorithm is applied to build the initial deep Q network model and deploy it to the intelligent control system. Then, the target control action with the maximum Q value is determined from the control action list using the actual collected boiler flame image and operation data, and the deep Q network model is trained online to continuously optimize the deep Q network model online.

[0088] There are various ways to perform online training on the deep Q network model. For example, a temporal difference (TD) algorithm can be used to optimize the environment model based on changes in Q values ​​corresponding to target control actions calculated in adjacent rounds.

[0089] like Figure 4 As shown, when the deep Q network model is trained online, two networks can be used, namely a main network and a target network, where the main network is the deep Q network model actually used and the target network is the deep Q network model used for evaluation.

[0090] Among them, the main network is used to calculate the state S at the current time t t Make a decision on the target control action a t The maximum Q value is taken as the current Q value; the target network is used to calculate the state S at the next time t+1 t+1 Make a decision on the target control action a t+1 The maximum Q value is taken as the target Q value. Then the loss can be determined based on the difference between the target Q value and the current Q value, and the parameters of the main network are updated based on the loss.

[0091] In the above embodiment, the Q network model can be updated in real time through offline training and online training of the Q network model to maintain the accuracy of decision-making and the effect of intelligent control.

[0092] In the reinforcement learning algorithm, the environment can be based on the state S at time t t and a control action a selected from the control action list, predicting the state S at the next time t+1 after executing the selected control action a t+1 , and calculate the reward value R after executing the selected control action. The reward function used to calculate the reward value R can be set according to actual needs. For example, the reward value can be calculated based on the amplitude of the control action and / or the change of the state at the next moment.

[0093] For example, in one embodiment, the reward value may be used to represent the deviation between the predicted value of the target operating data in the operating data of the boiler at the next moment predicted after executing the control action selected from the control action list and the corresponding preset safety value range. The larger the deviation, the smaller the reward value may be.

[0094] In one embodiment, the environmental model as described above can be used. Based on the pre-trained environmental model, the image feature vector, boiler operation data, and action data corresponding to the control action selected from the control action list are used as input data of the environmental model, and the predicted value of the target operation data of the boiler predicted when executing the selected control action is output.

[0095] The target operating data may be key operating data in the operating data of the boiler. In one embodiment, the target operating data may include at least one of the following operating data:

[0096] Steam flow rate;

[0097] Furnace temperature;

[0098] Oxygen content.

[0099] Accordingly, the reward value can be used to represent the deviation of steam flow, furnace temperature and oxygen content from a set safety value range. The safety value range can be a certain safety value or a data range between a set maximum value and a set minimum value.

[0100] In one embodiment, the reward component r corresponding to each target operating data ai can be calculated based on the deviation between each target operating data ai and the corresponding safety data range. ai , the larger the deviation, the smaller the calculated reward component. Then count the reward components r corresponding to each target operation data aiai To obtain a reward value R, the reward value R can be expressed by the following formula:

[0101] R=∑ ai r ai

[0102] For the target operating data a1, the safety data range corresponding to the target operating data a1 is a certain safety value Y a1 , then the reward component r corresponding to the target operation data a1 is calculated a1 It can be expressed by the following formula:

[0103]

[0104] Among them, m a1，1 is the weight coefficient corresponding to the target operation data a1, To predict the predicted value of the target operating data at the next moment, the next moment may be the i-th moment after moment t.

[0105] In the case where the safety data range is a certain safety value, in order to avoid frequent fluctuations in the control amount, the reward component r corresponding to the target operating data a1 can be calculated. a1 , the consideration based on the adjustment amount of the target operation data can be increased. At this time, the reward component r corresponding to the target operation data a1 a1 It can be expressed by the following formula:

[0106]

[0107] Among them, m a1，2 is another weight coefficient corresponding to the target operation data a1, the ΔU a1 is the adjustment amount of the target operating data, that is, the value change of the target operating data a1 at adjacent moments.

[0108] For the target operating data a1, the safety data range corresponding to the target operating data a2 is within the maximum value max a2 and minimum value min a2 The reward component r corresponding to the target operation data a2 is calculated a2 It can be expressed by the following formula:

[0109]

[0110] Among them, m a2 is the weight coefficient corresponding to the target operation data a2, It is the predicted value of the target operating data a2 at the i-th moment after the moment t.

[0111] In one implementation, taking a1 as an example, where a2 represents the steam flow rate, a3 represents the furnace temperature, and a4 represents the oxygen content, the reward value is represented by the following formula:

[0112]

[0113] Wherein, a1, a2, a3 are used to represent target operation data respectively; m a1，1 、m a1，2 、m a2 、m a3 are the weight coefficients corresponding to the target operation data a1, a2, and a3, respectively. is the predicted value of the target operation data at the i-th moment after time t, and the Y a1 is the safety value corresponding to the target operating data a1, ΔU a1 is the adjustment amount corresponding to the target operation data a1; max a2 and min a2 is the maximum and minimum safety values ​​corresponding to the target operating data a2; max a3 and min a3 The maximum and minimum values ​​of the safety values ​​corresponding to the target operating data a3.

[0114] In the above embodiment, the reward value of the reinforcement learning algorithm is set to represent the deviation between the predicted value of the target operating data of the boiler predicted after executing the control action selected from the control action list and the corresponding preset safety value range, so that the optimization goal when deciding the target control action from the control action list is to make the target operating data as much as possible within the preset safety value range, thereby improving the effect of intelligent control.

[0115] Corresponding to the above-mentioned embodiment of the intelligent control method of the boiler, the present application also provides an embodiment of the intelligent control device of the boiler.

[0116] like Figure 5 As shown, the intelligent control device of the boiler includes: a collection module 501, a decision module 502 and a control module 503.

[0117] The acquisition module 501 is used to obtain the image feature vector of the boiler flame image, the boiler operation data, and the control action list;

[0118] The decision module 502 is used to determine the target control action from the control action list based on a reinforcement learning algorithm; wherein the state of the reinforcement learning algorithm is the image feature vector of the boiler flame image and the operation data of the boiler, the action of the reinforcement learning algorithm is the control action selected from the control action list, and the reward value of the reinforcement learning algorithm is obtained based on the control action selected from the control action list;

[0119] The control module 503 is used to control the boiler using the target control action.

[0120] In the above embodiment, when realizing intelligent control of the boiler, the image feature vector of the flame image of the boiler is used, and the image feature vector and the boiler operation data are used as the state of the reinforcement learning algorithm to decide the target control action from the control action list. Accordingly, by adding the image feature vector of the flame image, more comprehensive feature information can be provided for the automatic control system of the boiler, so that the target control action can be obtained more accurately, thereby improving the intelligent control effect.

[0121] Furthermore, the acquisition module 501 is used for:

[0122] The collected boiler flame image is input into a pre-trained variational autoencoder VAE, and an image feature vector of the boiler flame image is obtained from the encoder output of the variational autoencoder; wherein the variational autoencoder is a generative model obtained by unsupervised training of flame image samples.

[0123] Furthermore, the reward value is used to represent the deviation between a predicted value of the target operating data of the boiler predicted after executing the control action selected from the control action list and a corresponding preset safety value range.

[0124] Furthermore, the target operation data includes at least one of the following operation data:

[0125] Steam flow rate;

[0126] Furnace temperature;

[0127] Oxygen content.

[0128] Furthermore, the reward value is expressed by the following formula:

[0129]

[0130] Wherein, a1, a2, a3 are used to represent target operation data respectively; m a1，1 、m a1，2 、m a2 、m a3are the weight coefficients corresponding to the target operation data a1, a2, and a3, respectively. is the predicted value of the target operation data at the i-th moment after time t, and the Y a1 is the safety value corresponding to the target operating data a1, ΔU a1 is the adjustment amount corresponding to the target operation data a1; max a2 and min a2 is the maximum and minimum safety values ​​corresponding to the target operating data a2; max a3 and min a3 The maximum and minimum values ​​of the safety values ​​corresponding to the target operating data a3.

[0131] Furthermore, the decision module 502 is used to:

[0132] Based on the pre-trained environmental model, the image feature vector, the operating data of the boiler and the action data corresponding to the control action selected from the control action list are used as input data of the environmental model, and the predicted value of the target operating data of the boiler predicted when the selected control action is executed is output; wherein the environmental model is a machine learning model obtained by supervised training using the historical image feature vectors of the previous time period in the historical data, the historical operating data of the boiler and the action data of the historical control action as training sample features, and the historical target operating data of the latter time period in the historical data as sample labels.

[0133] Furthermore, the reinforcement learning algorithm is a deep Q network learning algorithm;

[0134] The decision module 502 is used to: decide the target control action with the maximum Q value from the control action list based on the deep Q network learning algorithm; wherein the Q value is used to represent the expected cumulative reward obtained after executing the control action selected from the control action list under the image feature vector of the boiler flame image and the operating data of the boiler.

[0135] Furthermore, the decision module 502 is also used to optimize the environment model based on changes in Q values ​​corresponding to target control actions calculated in adjacent rounds.

[0136] Figure 6 is a schematic structural diagram of a device provided by an exemplary embodiment. Figure 6At the hardware level, the device includes a processor 602, an internal bus 604, a network interface 606, a memory 608, and a non-volatile memory 610, and may also include other required hardware. One or more embodiments of this specification may be implemented based on software, such as the processor 602 reading the corresponding computer program from the non-volatile memory 610 into the memory 608 and then running it. Of course, in addition to the software implementation, one or more embodiments of this specification do not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0137] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.

[0138] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0139] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0140] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0141] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0142] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0143] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0144] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0145] It should be understood that although the terms first, second, third, etc. may be used to describe various information in one or more embodiments of this specification, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0146] The above description is merely a preferred embodiment of one or more embodiments of the present specification and is not intended to limit one or more embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present specification shall be included in the scope of protection of one or more embodiments of the present specification.

Claims

1. A boiler intelligent control method, characterized in that: include: Obtaining an image feature vector of a boiler flame image, boiler operation data, and a control action list; A target control action is obtained by decision-making from the control action list based on a reinforcement learning algorithm; wherein the state of the reinforcement learning algorithm is the image feature vector of the boiler flame image and the operation data of the boiler, the action of the reinforcement learning algorithm is the control action selected from the control action list, and the reward value of the reinforcement learning algorithm is obtained based on the control action selected from the control action list; The boiler is controlled using the target control action.

2. The method according to claim 1, characterized in that The step of obtaining the image feature vector of the boiler flame image includes: The collected boiler flame image is input into a pre-trained variational autoencoder VAE, and an image feature vector of the boiler flame image is obtained from the encoder output of the variational autoencoder; wherein the variational autoencoder is a generative model obtained by unsupervised training of flame image samples.

3. The method according to claim 1, characterized in that The reward value is used to represent the deviation between a predicted value of the target operating data of the boiler predicted after executing the control action selected from the control action list and a corresponding preset safety value range.

4. The method according to claim 3, characterized in that The target operation data includes at least one of the following operation data: Steam flow rate; Furnace temperature; Oxygen content.

5. The method according to claim 3, characterized in that: The reward value is expressed by the following formula: Wherein, a1, a2, a3 are used to represent target operation data respectively; m a1,1 、m a1,2 、m a2 、m a3 are the weight coefficients corresponding to the target operation data a1, a2, and a3, respectively. is the predicted value of the target operation data at the i-th moment after time t, and the Y a1 is the safety value corresponding to the target operating data a1, ΔU a1 is the adjustment amount corresponding to the target operation data a1; max a2 and min a2 is the maximum and minimum safety values ​​corresponding to the target operation data a2; max a3 and min a3 The maximum and minimum values ​​of the safety values ​​corresponding to the target operating data a3.

6. The method according to claim 3, characterized in that The predicting and obtaining a predicted value of the target operating data of the boiler after executing the control action selected from the control action list includes: Based on the pre-trained environmental model, the image feature vector, the operating data of the boiler and the action data corresponding to the control action selected from the control action list are used as input data of the environmental model, and the predicted value of the target operating data of the boiler predicted when the selected control action is executed is output; wherein the environmental model is a machine learning model obtained by supervised training using the historical image feature vectors of the previous time period in the historical data, the historical operating data of the boiler and the action data of the historical control action as training sample features, and the historical target operating data of the latter time period in the historical data as sample labels.

7. The method according to claim 6, characterized in that The reinforcement learning algorithm is a deep Q network learning algorithm; The step of obtaining a target control action from the control action list based on a reinforcement learning algorithm includes: Based on the deep Q network learning algorithm, a target control action with the maximum Q value is determined from the control action list; wherein the Q value is used to represent the expected cumulative reward obtained after executing the control action selected from the control action list under the image feature vector of the boiler flame image and the operating data of the boiler.

8. The method according to claim 7, characterized in that The method further comprises: The environment model is optimized based on changes in Q values ​​corresponding to target control actions calculated in adjacent rounds.

9. An intelligent control device for a boiler, characterized in that: include: An acquisition module, used to obtain image feature vectors of boiler flame images, boiler operation data, and a control action list; A decision module, configured to obtain a target control action by making a decision from the control action list based on a reinforcement learning algorithm; wherein the state of the reinforcement learning algorithm is the image feature vector of the boiler flame image and the operation data of the boiler, the action of the reinforcement learning algorithm is the control action selected from the control action list, and the reward value of the reinforcement learning algorithm is obtained based on the control action selected from the control action list; A control module is used to control the boiler using the target control action.

10. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor implements the method according to any one of claims 1 to 8 by running the executable instructions.

11. A computer-readable storage medium, characterized in that: Computer instructions are stored thereon, and when the instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images