A boiler heating surface abnormal image recognition method, marking method and system

Abstract

The embodiment of the present invention provides a method and system for identifying and marking abnormal images of heating surfaces of boilers, including: obtaining a large number of heating surface images in the furnace, the heating surface images have been artificially identified as normal heating surface images and different types of abnormal heating surface images, preprocessing each heating surface image to obtain a preprocessed heating surface image; using a feature learner to extract features from each preprocessed heating surface image, training the characteristics of each preprocessed heating surface image to obtain an image recognition model; identifying the collected heating surface images in the furnace through a graphic recognition model, and identifying abnormal heating surface images; drawing a three-dimensional model consistent with the actual size and spatial distribution of the boiler according to the boiler design drawings, marking the abnormal heating surface image on the boiler model, and forming a diagnostic report. The image recognition model is used to identify whether the heating surface in the furnace is abnormal, and the abnormal heating surface image is marked on the boiler model and sent to maintenance workers to reduce manpower input.

Description

Technical Field

[0001] The present invention relates to the field of coal-fired power generation, and in particular to an abnormal image recognition method, a marking method and a system for a boiler heating surface. Background Art

[0002] Coal-fired power plants produce a lot of ash particles, sulfur and nitrogen oxides in their daily production and operation. These substances are sometimes deposited on the surface of the heating surface in various forms during the operation of the boiler, causing coking, ash accumulation and corrosion of the heating surface. The coking, ash accumulation and corrosion of the boiler not only increase the heat transfer resistance of the boiler heating surface, deteriorate the heat transfer of the heating surface and reduce the thermal economy of the boiler, but may also cause blockage of the flue gas channel, affecting the safe operation of the boiler. In serious cases, equipment damage and personal injury accidents may occur.

[0003] In the process of realizing the present invention, the applicant found that there are at least the following problems in the prior art: due to the coking, ash accumulation and corrosion of the boiler heating surface, not only the heat transfer resistance of the boiler heating surface is increased, the thermal economy of the boiler is reduced, but also the flue gas passage may be blocked, affecting the safe operation of the boiler. Coal-fired power plants will use the opportunity of shutdown to regularly check the boiler heating surface based on their maintenance experience. Although the current maintenance position has fully considered the operating conditions of the boiler, it is limited by the field of vision and maintenance experience, and the efficiency is low. Summary of the invention

[0004] The embodiment of the present invention provides a method, a marking method and a system for abnormal image recognition of the heating surface of a boiler. An image recognition model is obtained through training. The image recognition model automatically extracts image feature information, can identify whether the heating surface in the furnace is abnormal, what kind of abnormality (coking, ash accumulation, corrosion), and mark the abnormal heating surface image on the boiler model and send it to maintenance workers, which can reduce a lot of manpower input.

[0005] To achieve the above-mentioned purpose, on the one hand, an embodiment of the present invention provides a method for identifying and marking abnormal images of a boiler heating surface, comprising:

[0006] A large number of heating surface images in a coal-fired power plant boiler are obtained, wherein the heating surface images have been artificially identified as normal heating surface images and different types of abnormal heating surface images, and each heating surface image is preprocessed to obtain a preprocessed heating surface image; wherein the types of the abnormal heating surface images include at least one of: a heating surface dusting image, a heating surface coking image, and a heating surface corrosion image;

[0007] A feature learner is used to extract features from each pre-processed heating surface image, and the characteristics of each pre-processed heating surface image are trained to obtain an image recognition model;

[0008] Multiple images of the heating surface of the coal-fired power plant boiler furnace and the spatial position of each image on the heating surface are collected by automatic image acquisition equipment, and the collected heating surface images are recognized by a graphic recognition model to identify abnormal heating surface images; and a three-dimensional model consistent with the actual size and spatial distribution of the boiler is drawn according to the boiler design drawings, and the abnormal heating surface images are marked on the boiler model to form a diagnosis report.

[0009] Preferably, the preprocessing of each heated surface image to obtain a preprocessed heated surface image specifically includes:

[0010] Performing augmentation processing on each heated surface image, wherein the augmentation processing operation includes: rotating the heated surface image, mirror flipping, changing the brightness of the entire image or a part of the image pixels, and / or adding random noise to the global image to simulate different real scenes;

[0011] Scaling the augmented heated surface image to a preset size;

[0012] The heating surface images of predetermined sizes are subjected to mean removal processing to highlight the contrast of the heating surface images of each predetermined size, thereby obtaining a preprocessed heating surface image.

[0013] Preferably, the feature learner is a convolutional neural network;

[0014] The feature learner is used to extract features from each pre-processed heating surface image, and the characteristics of each pre-processed heating surface image are trained to obtain an image recognition model, which specifically includes:

[0015] Extract the RGB components of each pre-processed heating surface image and input the RGB components of each pre-processed heating surface image into a convolutional neural network respectively. The convolutional layer of the convolutional neural network performs weighted calculation based on different convolution kernels to obtain a feature matrix; and perform non-linear activation on the feature matrix through an activation function and then input it into a pooling layer for feature compression; and repeat the convolutional layer and pooling layer of the convolutional neural network for multiple times to obtain a set of fixed-size feature matrices;

[0016] The fixed-size feature matrix is ​​input into the fully connected layer in the product neural network, and the fixed-size feature matrix structure is transformed through the fully connected layer to obtain a fixed-length feature vector;

[0017] The fixed-length feature vector is input into the classifier for spatial probability mapping, and the preprocessed heating surface image is classified into normal heating surface and different types of abnormal heating surface images.

[0018] Preferably, the RGB components of each pre-processed heated surface image are extracted and the RGB components of each pre-processed heated surface image are respectively input into a convolutional neural network, and the convolutional layer of the convolutional neural network performs weighted calculation based on different convolution kernels to obtain a feature matrix, specifically including:

[0019] The product kernel included in the convolution layer represents the weight of the RGB component of the preprocessed image;

[0020] The features of the layer of each preprocessed preheated image are extracted by performing dot multiplication with the convolution kernel and its corresponding RGB component, and the extracted features are formed into a feature matrix; and when extracting the features of the layer of each preheated preprocessed image, the spatial position of the preprocessed image eigenvalue where the feature is located is kept unchanged.

[0021] Preferably, the fixed-length feature vector is input into a classifier for spatial probability mapping, and the pre-processed heating surface image is classified into normal heating surface and different types of abnormal heating surface images, specifically including:

[0022] The classifier is a softmax activation function, which maps each vector in the fixed-length feature vector output by the fully connected layer to the probability space of the image category through the softmax activation function to obtain the true probability value of the feature vector under each category; then the loss function is used to calculate the true probability of classifying the preprocessed heated surface image as a certain type and the error of the real sample to feedforward update the weight to obtain the updated true probability value of the preprocessed heated surface image, and the updated true probability value is used as the probability that the preprocessed image is identified as a certain category in the current training round; according to the probability, the preprocessed heated surface image is matched as a normal heated surface or different types of abnormal heated surface images.

[0023] Preferably, the method of calculating the true probability of classifying the pre-processed heated surface image as a certain type and the error of the real sample by the loss function to feed-forward update the weight to obtain the true probability value of the pre-processed heated surface image update specifically includes:

[0024] The loss function of the convolutional neural network is set. When minimizing the loss function, the loss function is fed back to the weights of each pixel of the fully connected layer and the convolutional layer in turn through the back propagation algorithm. In the process of back propagation, the weights of each pixel of the convolutional layer and the fully connected layer are adjusted by the gradient descent method, and the parameter value of the cross entropy loss function is adjusted according to the error. The above process is continuously iterated until the cross entropy loss function converges, and the risk of the loss function of the convolutional neural network is minimized to obtain the loss of the true probability of classifying the preprocessed heated surface image as a certain type. The weight of the preprocessed heated surface image is updated according to the loss to obtain the updated true probability value of the preprocessed heated surface image.

[0025] Preferably, the automatic image acquisition device is a drone equipped with a high-definition camera;

[0026] The step of acquiring a large number of images of heating surfaces in the boiler of a coal-fired power plant specifically includes:

[0027] A large number of heating surface images of coal-fired power plant boilers are obtained through automatic image acquisition equipment.

[0028] Preferably, drawing a three-dimensional model consistent with the actual size and spatial distribution of the boiler according to the boiler design drawings, and marking the abnormal heating surface image on the boiler model specifically includes:

[0029] A three-dimensional simulation model of the boiler is constructed according to the actual size of the boiler using a three-dimensional drawing tool, and corresponding eye-catching marks are preset for different types of abnormal heating surface images; according to the spatial position of each abnormal heating surface image on the heating surface of the boiler, the corresponding preset eye-catching marks are marked on the corresponding abnormal area of ​​the three-dimensional simulation model, so that maintenance personnel can locate the maintenance position of the heating surface based on the abnormal heating surface image of the three-dimensional simulation model.

[0030] Preferably, drawing a three-dimensional model consistent with the actual size and spatial distribution of the boiler according to the boiler design drawings, and marking the abnormal heating surface image on the boiler model specifically includes:

[0031] The identified abnormal heating surface images are broadly marked in the corresponding areas of the three-dimensional simulation model, and a diagnostic report of abnormal heating surface conditions in different heating surface areas is generated based on the broadly marked three-dimensional simulation model, and the diagnostic report is pushed to maintenance personnel.

[0032] To achieve the above-mentioned purpose, on the other hand, an embodiment of the present invention further provides a boiler heating surface abnormal image recognition and marking system, comprising:

[0033] The furnace heating surface image acquisition unit is used to acquire a large number of heating surface images of the furnace of the coal-fired power plant boiler, wherein the heating surface images have been artificially identified as normal heating surface images and different types of abnormal heating surface images, and pre-process each heating surface image to obtain a pre-processed heating surface image; wherein the types of the abnormal heating surface images include at least one of: a heating surface dust image, a heating surface coking image, and a heating surface corrosion image;

[0034] An image recognition model forming unit is used to extract features from each pre-processed heating surface image using a feature learner, and train the characteristics of each pre-processed heating surface image to obtain an image recognition model;

[0035] The abnormal heating surface image recognition unit is used to collect multiple images of the heating surface in the furnace of the coal-fired power plant boiler and the spatial position of each image on the heating surface in the furnace through an automatic image acquisition device, recognize the collected heating surface images in the furnace through a graphic recognition model, and recognize abnormal heating surface images;

[0036] The abnormal heating surface image marking unit is used to draw a three-dimensional model consistent with the actual size and spatial distribution of the boiler according to the boiler design drawings, mark the abnormal heating surface image on the boiler model, and form a diagnosis report.

[0037] The above technical solution has the following beneficial effects: an image recognition model is obtained through training, and the image recognition model automatically extracts image feature information, can identify whether the heating surface in the furnace is abnormal, what kind of abnormality (coking, ash accumulation, corrosion), and mark the abnormal heating surface image on the boiler model and send it to maintenance workers, which can reduce a lot of manpower investment. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 is a flow chart of a method for identifying and marking abnormal images of a boiler heating surface according to an embodiment of the present invention;

[0040] Figure 2 is a structural diagram of a boiler heating surface abnormality image recognition and marking system according to an embodiment of the present invention;

[0041] Figure 3 is the training process of the image recognition model of the embodiment of the present invention;

[0042] Figure 4 is a convolutional layer of a convolutional neural network according to an embodiment of the present invention;

[0043] Figure 5 This is the feature learning algorithm workflow of an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] like Figure 1 As shown, in combination with an embodiment of the present invention, a method for identifying and marking abnormal images of a boiler heating surface is provided, comprising:

[0046] S101: acquiring a large number of heating surface images in a coal-fired power plant boiler, wherein the heating surface images have been manually identified as normal heating surface images and different types of abnormal heating surface images, and preprocessing each heating surface image to obtain a preprocessed heating surface image; wherein the types of the abnormal heating surface images include at least one of: a heating surface dusting image, a heating surface coking image, and a heating surface corrosion image;

[0047] S102: extracting features from each pre-processed heating surface image using a feature learner, and training the characteristics of each pre-processed heating surface image to obtain an image recognition model;

[0048] S103: using an automatic image acquisition device to acquire multiple images of the heating surface in the coal-fired power plant boiler furnace and the spatial position of each image on the heating surface in the furnace, recognizing the acquired heating surface images in the furnace through a graphic recognition model, and identifying abnormal heating surface images

[0049] S104: Draw a three-dimensional model that is consistent with the actual size and spatial distribution of the boiler according to the boiler design drawings, mark the abnormal heating surface image on the boiler model, and form a diagnosis report.

[0050] Preferably, in step 101, preprocessing each heated surface image to obtain a preprocessed heated surface image specifically includes:

[0051] S1011: performing augmentation processing on each heated surface image, wherein the augmentation processing operation includes: rotating the heated surface image, mirror flipping, changing the brightness of the entire image or a part of the image pixels, and / or adding random noise to the global image to simulate different real scenes;

[0052] S1012: scaling the augmented heated surface image to a preset size;

[0053] S1013: De-meaning the heating surface images of predetermined sizes to highlight the contrast of the heating surface images of each predetermined size, and obtain a pre-processed heating surface image.

[0054] Preferably, the feature learner is a convolutional neural network;

[0055] Step 102 specifically includes:

[0056] S1021: extracting the RGB components of each pre-processed heated surface image and inputting the RGB components of each pre-processed heated surface image into a convolutional neural network respectively, wherein the convolutional layer of the convolutional neural network performs weighted calculation based on different convolution kernels to obtain a feature matrix; and performing non-linear activation on the feature matrix through an activation function and then inputting the non-linear activation into a pooling layer for feature compression; and repeatedly performing multiple times of convolutional layer and pooling layer of the convolutional neural network to obtain a set of fixed-size feature matrices;

[0057] S1022: inputting the set of fixed-size feature matrices into a fully connected layer in a product neural network, and transforming the structure of the set of fixed-size feature matrices through the fully connected layer to obtain a fixed-length feature vector;

[0058] S1023: Inputting the fixed-length feature vector into the classifier for spatial probability mapping, classifying the preprocessed heating surface image into normal heating surface and different types of abnormal heating surface images.

[0059] Preferably, step 1021 specifically includes:

[0060] The product kernel included in the convolution layer represents the weight of the RGB component of the preprocessed image;

[0061] The features of the layer of each preprocessed preheated image are extracted by performing dot multiplication with the convolution kernel and its corresponding RGB component, and the extracted features are formed into a feature matrix; and when extracting the features of the layer of each preheated preprocessed image, the spatial position of the preprocessed image eigenvalue where the feature is located is kept unchanged.

[0062] Preferably, step 1023 specifically includes:

[0063] The classifier is a softmax activation function, which maps each vector in the fixed-length feature vector output by the fully connected layer to the probability space of the image category through the softmax activation function to obtain the true probability value of the feature vector under each category; then the loss function is used to calculate the true probability of classifying the preprocessed heated surface image as a certain type and the error of the real sample to feedforward update the weight to obtain the updated true probability value of the preprocessed heated surface image, and the updated true probability value is used as the probability that the preprocessed image is identified as a certain category in the current training round; according to the probability, the preprocessed heated surface image is matched as a normal heated surface or different types of abnormal heated surface images.

[0064] Preferably, the method of calculating the true probability of classifying the pre-processed heated surface image as a certain type and the error of the real sample by the loss function to feed-forward update the weight to obtain the true probability value of the pre-processed heated surface image update specifically includes:

[0065] The loss function of the convolutional neural network is set. When minimizing the loss function, the loss function is fed back to the weights of each pixel of the fully connected layer and the convolutional layer in turn through the back propagation algorithm. In the process of back propagation, the weights of each pixel of the convolutional layer and the fully connected layer are adjusted by the gradient descent method, and the parameter value of the cross entropy loss function is adjusted according to the error. The above process is continuously iterated until the cross entropy loss function converges, and the risk of the loss function of the convolutional neural network is minimized to obtain the loss of the true probability of classifying the preprocessed heated surface image as a certain type. The weight of the preprocessed heated surface image is updated according to the loss to obtain the updated true probability value of the preprocessed heated surface image.

[0066] Preferably, the automatic image acquisition device is a drone equipped with a high-definition camera;

[0067] In step 101, the acquisition of a large number of heating surface images in a coal-fired power plant boiler specifically includes:

[0068] S1014: Acquire a large number of images of heating surfaces in the boiler of a coal-fired power plant through automatic image acquisition equipment.

[0069] Preferably, step 104 specifically includes:

[0070] S1041: Use a three-dimensional drawing tool to construct a three-dimensional simulation model of the boiler according to the actual size of the boiler, and preset corresponding eye-catching marks for different types of abnormal heating surface images; according to the spatial position of each abnormal heating surface image on the heating surface of the boiler, mark the corresponding preset eye-catching marks on the corresponding abnormal area of ​​the three-dimensional simulation model, so that maintenance personnel can locate the maintenance position of the heating surface based on the abnormal heating surface image of the three-dimensional simulation model.

[0071] Preferably, step 104 specifically includes:

[0072] S1042: Broadly mark each abnormal heating surface image identified in the corresponding area of ​​the three-dimensional simulation model, and generate a diagnosis report of abnormal heating surface conditions in different heating surface areas based on the broadly marked three-dimensional simulation model, and the diagnosis report is used to push it to maintenance personnel.

[0073] like Figure 2 As shown, in combination with the embodiments of the present invention, a boiler heating surface abnormal image recognition and marking system is also provided, comprising:

[0074] The furnace heating surface image acquisition unit 21 is used to acquire a large number of heating surface images of the furnace of the coal-fired power plant boiler, wherein the heating surface images have been artificially identified as normal heating surface images and different types of abnormal heating surface images, and pre-process each heating surface image to obtain a pre-processed heating surface image; wherein the types of the abnormal heating surface images include at least one of: a heating surface dust image, a heating surface coking image, and a heating surface corrosion image;

[0075] An image recognition model forming unit 22 is used to extract features from each pre-processed heating surface image using a feature learner, and train the characteristics of each pre-processed heating surface image to obtain an image recognition model;

[0076] The abnormal heating surface image recognition unit 23 is used to collect multiple images of the heating surface in the furnace of the coal-fired power plant boiler and the spatial position of each image on the heating surface in the furnace through an automatic image acquisition device, recognize the collected heating surface images in the furnace through a graphic recognition model, and recognize abnormal heating surface images;

[0077] The abnormal heating surface image marking unit 24 is used to draw a three-dimensional model consistent with the actual size and spatial distribution of the boiler according to the boiler design drawings, mark the abnormal heating surface image on the boiler model, and form a diagnosis report.

[0078] Preferably, in the furnace heating surface image acquisition unit 21, the preprocessing of each heating surface image to obtain the preprocessed heating surface image specifically includes:

[0079] Performing augmentation processing on each heated surface image, wherein the augmentation processing operation includes: rotating the heated surface image, mirror flipping, changing the brightness of the entire image or a part of the image pixels, and / or adding random noise to the global image to simulate different real scenes;

[0080] Scaling the augmented heated surface image to a preset size;

[0081] The heating surface images of predetermined sizes are subjected to mean removal processing to highlight the contrast of the heating surface images of each predetermined size, thereby obtaining a preprocessed heating surface image.

[0082] Preferably, the feature learner is a convolutional neural network;

[0083] The image recognition model forming unit 22 specifically includes:

[0084] Extract the RGB components of each pre-processed heating surface image and input the RGB components of each pre-processed heating surface image into a convolutional neural network respectively. The convolutional layer of the convolutional neural network performs weighted calculation based on different convolution kernels to obtain a feature matrix; and perform non-linear activation on the feature matrix through an activation function and then input it into a pooling layer for feature compression; and repeat the convolutional layer and pooling layer of the convolutional neural network for multiple times to obtain a set of fixed-size feature matrices;

[0085] The fixed-size feature matrix is ​​input into the fully connected layer in the product neural network, and the fixed-size feature matrix structure is transformed through the fully connected layer to obtain a fixed-length feature vector;

[0086] The fixed-length feature vector is input into the classifier for spatial probability mapping, and the preprocessed heating surface image is classified into normal heating surface and different types of abnormal heating surface images.

[0087] Preferably, the RGB components of each pre-processed heated surface image are extracted and the RGB components of each pre-processed heated surface image are respectively input into a convolutional neural network, and the convolutional layer of the convolutional neural network performs weighted calculation based on different convolution kernels to obtain a feature matrix, specifically including:

[0088] The product kernel included in the convolution layer represents the weight of the RGB component of the preprocessed image;

[0089] The features of the layer of each preprocessed preheated image are extracted by performing dot multiplication with the convolution kernel and its corresponding RGB component, and the extracted features are formed into a feature matrix; and when extracting the features of the layer of each preheated preprocessed image, the spatial position of the preprocessed image eigenvalue where the feature is located is kept unchanged.

[0090] Preferably, the fixed-length feature vector is input into a classifier for spatial probability mapping, and the pre-processed heating surface image is classified into normal heating surface and different types of abnormal heating surface images, specifically including:

[0091] The classifier is a softmax activation function, which maps each vector in the fixed-length feature vector output by the fully connected layer to the probability space of the image category through the softmax activation function to obtain the true probability value of the feature vector under each category; then the loss function is used to calculate the true probability of classifying the preprocessed heated surface image as a certain type and the error of the real sample to feedforward update the weight to obtain the updated true probability value of the preprocessed heated surface image, and the updated true probability value is used as the probability that the preprocessed image is identified as a certain category in the current training round; according to the probability, the preprocessed heated surface image is matched as a normal heated surface or different types of abnormal heated surface images.

[0092] Preferably, the method of calculating the true probability of classifying the pre-processed heated surface image as a certain type and the error of the real sample by the loss function to feed-forward update the weight to obtain the true probability value of the pre-processed heated surface image update specifically includes:

[0093] The loss function of the convolutional neural network is set. When minimizing the loss function, the loss function is fed back to the weights of each pixel of the fully connected layer and the convolutional layer in turn through the back propagation algorithm. In the process of back propagation, the weights of each pixel of the convolutional layer and the fully connected layer are adjusted by the gradient descent method, and the parameter value of the cross entropy loss function is adjusted according to the error. The above process is continuously iterated until the cross entropy loss function converges, and the risk of the loss function of the convolutional neural network is minimized to obtain the loss of the true probability of classifying the preprocessed heated surface image as a certain type. The weight of the preprocessed heated surface image is updated according to the loss to obtain the updated true probability value of the preprocessed heated surface image.

[0094] Preferably, the automatic image acquisition device is a drone equipped with a high-definition camera;

[0095] The furnace heating surface image acquisition unit 21 specifically includes:

[0096] A large number of heating surface images of coal-fired power plant boilers are obtained through automatic image acquisition equipment.

[0097] Preferably, the abnormally heated surface image marking unit 24 specifically includes:

[0098] A three-dimensional simulation model of the boiler is constructed according to the actual size of the boiler using a three-dimensional drawing tool, and corresponding eye-catching marks are preset for different types of abnormal heating surface images; according to the spatial position of each abnormal heating surface image on the heating surface of the boiler, the corresponding preset eye-catching marks are marked on the corresponding abnormal area of ​​the three-dimensional simulation model, so that maintenance personnel can locate the maintenance position of the heating surface based on the abnormal heating surface image of the three-dimensional simulation model.

[0099] Preferably, the abnormally heated surface image marking unit 24 specifically includes:

[0100] The identified abnormal heating surface images are broadly marked in the corresponding areas of the three-dimensional simulation model, and a diagnostic report of abnormal heating surface conditions in different heating surface areas is generated based on the broadly marked three-dimensional simulation model, and the diagnostic report is pushed to maintenance personnel.

[0101] The above technical solution of the embodiment of the present invention is described in detail below in conjunction with specific application examples. For technical details not introduced during the implementation process, please refer to the relevant description in the previous text.

[0102] The present invention provides a method for identifying and marking abnormal images of boiler heating surfaces. It uses drones to collect large quantities of image information of heating surfaces in coal-fired power plants. It automatically extracts relevant image feature information from the image information of heating surfaces in the furnace through feature learning algorithms, identifies abnormal conditions of heating surfaces in boiler furnaces with certain differences, and manages abnormal positions based on a three-dimensional visualization platform, forming a complete set of intelligent detection and analysis methods and systems for boiler heating surfaces. It can effectively assist in determining whether there are coking, dust accumulation, and corrosion problems on the heating surfaces of boiler furnaces, and can perform maintenance work in a timely manner, overcoming the difficulties of "relying on maintenance experience and taking advantage of shutdown opportunities to regularly check the heating surface of boilers; although the current maintenance position has fully considered the operating conditions of the boiler, it is limited by vision and maintenance experience, and the efficiency is low". The technical solutions of the present invention are mainly summarized as follows:

[0103] 1. Use automatic image acquisition equipment, such as drones, to collect images inside the furnace to form a large number of samples, and use feature learners (convolutional neural networks) to automatically extract feature information of the images inside the furnace. Through feature training and fitting, it can still maintain good performance in different scenarios, thereby achieving effective identification of dust accumulation / coking / corrosion conditions inside the furnace;

[0104] 2. Based on 3D drawing tools, a 3D model of the same size as the boiler is constructed, and the spatial position of the image taken by the robot is marked in the model for highlighting, so that the abnormal area of ​​the boiler can be highlighted, allowing maintenance personnel to locate the maintenance position based on the image information and achieve efficient management of the boiler heating surface.

[0105] The technical solution of the application example of the present invention is as follows:

[0106] 1.1 Image acquisition: Image acquisition by drone

[0107] Based on the drone equipped with a high-definition camera, the image inside the furnace is collected. The collection pixel is required to be more than 200W, and the image shooting specification is 1*1m. A single shooting is expected to produce more than 1,000 picture samples, including various normal and abnormal samples. It overcomes the problem that "boiler equipment managers must set up scaffolding for on-site inspections, which not only poses safety risks but also has low efficiency; and can only conduct close-up inspections on some pipe sections each time; although the maintenance personnel can realize abnormalities such as ash coking and corrosion on the boiler heating surface based on their experience, due to low efficiency, they cannot complete the inspection of all heating surfaces during a shutdown maintenance period."

[0108] 1.2 Image Preprocessing

[0109] i Image augmentation: By rotating, mirroring, and changing the brightness of the heated surface image in the furnace, random noise is added to simulate more real scenes to improve the generalization ability of the model. Among them, the real scene: for example, an augmented new picture can be generated by combining random noise and brightness adjustment; compared with the picture generated by simple brightness adjustment, one more factor is introduced, which also forms a new picture.

[0110] ii Image scaling: The image needs to be scaled to a size suitable for model input.

[0111] The main method is bilinear interpolation method, which uses the surrounding points of the coordinates after the coordinates are reversed, and then obtains the weights based on the distance relationship, and composites the pixel points through the weights and multiple points.

[0112] iii Remove the mean (standardization):

[0113] Feature normalization means (independently) making each dimension of the data have zero mean and unit variance: first calculate the mean of the data in each dimension (calculated using all the data), and then subtract the mean in each dimension. The next step is to divide each dimension of the data by the standard deviation of the data in that dimension. For natural images, the image is more often zero-meaned, and there is no need to estimate the variance of the sample. That is: the heated surface images of predetermined sizes are processed by removing the mean to highlight the contrast of the heated surface images of each predetermined size, and a preprocessed heated surface image is obtained.

[0114] 1.3 Feature extraction technology

[0115] like Figure 4 , Figure 5 As shown, the RGB components of each preprocessed heated surface image are extracted and the RGB components of each preprocessed heated surface image are respectively input into the convolutional neural network, and the convolutional layer of the convolutional neural network performs weighted calculation based on different convolution kernels to obtain a feature matrix; and the feature matrix is ​​nonlinearly activated by an activation function and then input into the pooling layer for feature compression; and the convolutional layer and pooling layer of the convolutional neural network are repeated multiple times to obtain a set of fixed-size feature matrices.

[0116] The fixed-size feature matrix is ​​input into the fully connected layer in the product neural network, and the structure of the fixed-size feature matrix is ​​transformed through the fully connected layer to obtain a fixed-length feature vector.

[0117] The fixed-length feature vector is input into the classifier for spatial probability mapping, and the preprocessed heating surface image is classified into normal heating surface and different types of abnormal heating surface images.

[0118] The details are as follows:

[0119] i. Convolutional Neural Networks

[0120] The deep learning image recognition uses convolutional neural network. After the image is input, it is subjected to weighted sampling through multiple convolutional layers and pooling layers to extract features. After that, the fixed-size feature vector is input into the classifier through the fully connected layer for spatial probability mapping, and finally the classification result is obtained. Specifically:

[0121] The RGB components of the heating surface image in the furnace are input into the convolutional neural network respectively. The convolutional layer in the network performs weighted calculation based on different convolution kernels to obtain the feature matrix of this layer and performs nonlinear activation through the activation function. Then, it is input into the pooling layer for feature compression. After the above steps are repeated many times in the multi-layer convolutional neural network, a set of feature matrices of fixed size are finally obtained and input into the full connection. The fully connected layer undergoes structural transformation;

[0122] After structural transformation, the fully connected layer obtains a feature vector of fixed length, which is then mapped to the probability space of the actual category using the softmax activation function.

[0123] In the convolution layer, the main component is the convolution kernel, which represents the corresponding weight. The convolution operation between the convolution kernel and the image, that is, the dot product of the convolution kernel and the image, represents that the weight in the convolution kernel acts on the pixels at the corresponding position separately, extracting the features in the image while maintaining the spatial invariance on the eigenvalues.

[0124] Mathematical representation of convolution calculation:

[0125]

[0126] 1.4 Model Training

[0127] Training is mainly a process of minimizing the risk of the loss function. The back-propagation algorithm continuously feeds back to the weights of each layer, and the gradient descent method is used to adjust the weights of each layer, ultimately approaching the global optimum or a better local optimum.

[0128] The functions used in the training process are:

[0129] Cross Entropy Loss Function

[0130] Cross entropy can measure the difference between two different probability distributions in the same random variable. In machine learning, it is expressed as the difference between the true probability distribution and the predicted probability distribution. The smaller the cross entropy value, the better the model prediction effect; it is often used in conjunction with softmax. Softmax processes the output results so that the sum of the predicted values ​​of multiple categories is 1, and then calculates the loss through cross entropy.

[0131] Multi-classification cross entropy formula:

[0132]

[0133] like Figure 3 As shown, back propagation

[0134] (1) The training set data is input into the input layer of the neural network, passes through the hidden layer, and finally reaches the output layer and outputs the result. This is the forward propagation process of the neural network;

[0135] (2) Since there is an error between the output result of the neural network and the actual result, the error between the estimated value and the actual value is calculated, and the error is back-propagated from the output layer to the hidden layer until it reaches the input layer;

[0136] (3) During the back propagation process, the values ​​of various parameters are adjusted according to the error; the above process is continuously iterated until convergence.

[0137] 1.5 Image Recognition and Location Annotation

[0138] Based on PDMS and other 3D drawing software, a 1:1 3D model is constructed, and a 3D space coordinate system is constructed based on this. Based on the automatic image acquisition equipment (e.g., drone) equipped with a high-definition camera, the image of the heating surface in the furnace is collected. The collected data includes the image of the heating surface in the furnace and the location information of the captured image. The abnormal image is identified through the image recognition model, and the abnormal image position is displayed based on the 3D model, including:

[0139] A three-dimensional simulation model of the boiler is constructed according to the actual size of the boiler using a three-dimensional drawing tool, and corresponding eye-catching marks are preset for different types of abnormal heating surface images; according to the spatial position of each abnormal heating surface image on the heating surface of the boiler, the corresponding preset eye-catching marks are marked on the corresponding abnormal area of ​​the three-dimensional simulation model, so that maintenance personnel can locate the maintenance position of the heating surface based on the abnormal heating surface image of the three-dimensional simulation model.

[0140] The identified abnormal heating surface images are broadly marked in the corresponding areas of the three-dimensional simulation model, and a diagnostic report of abnormal heating surface conditions in different heating surface areas is generated based on the broadly marked three-dimensional simulation model, and the diagnostic report is pushed to maintenance personnel.

[0141] This allows boiler managers to quickly check the ash accumulation, coking and corrosion conditions on the entire furnace heating surface, thereby assisting boiler maintenance management.

[0142] The technical effects achieved by the present invention are as follows:

[0143] 1. Make full use of technologies such as robots to form a large number of boiler heating surface image samples, use big data technology to analyze and mine the massive image data and make use of it, so as to realize the in-depth application of the furnace image data.

[0144] 2. Automatically extract relevant image feature information through feature learning algorithm (convolutional neural network), effectively assist in judging whether there are coking, dust accumulation, corrosion and other problems on the heating surface of the furnace. It can judge which of the collected pictures have coking, dust accumulation and corrosion, and then manually confirm and correct them, which can reduce a lot of manpower input.

[0145] 3. Automatically push abnormal situations after identification, so that operation and maintenance personnel can perform corresponding maintenance work based on the actual fault location, avoiding the problem that traditional maintenance work relies too much on experience.

[0146] It avoids a series of problems in the prior art such as "relying on manual experience to judge whether there is an abnormality on the heating surface of the furnace, which involves high operational risks, high inspection costs, and poor working environment", and also avoids the problem of "using manual image recognition to judge the coking, ash accumulation, and corrosion conditions of the boiler, which involves a lot of work".

[0147] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of the present disclosure. The attached method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0148] In the above detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are clearly stated in each claim. On the contrary, as reflected in the appended claims, the invention is in a state of having less than all the features of the disclosed individual embodiments. Therefore, the appended claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

[0149] The disclosed embodiments are described above to enable any person skilled in the art to implement or use the present invention. Various modifications of these embodiments are obvious to those skilled in the art, and the general principles defined herein may also be applied to other embodiments without departing from the spirit and scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0150] The above description includes examples of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but it should be recognized by those skilled in the art that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of protection of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, the word is covered in a manner similar to the term "including", just as "including," is explained as a transitional word in the claims. In addition, any term "or" used in the specification of the claims is intended to mean "non-exclusive or".

[0151] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying and marking abnormal images of boiler heating surfaces. It is characterized in that include: A large number of heating surface images in a coal-fired power plant boiler are obtained, wherein the heating surface images have been artificially identified as normal heating surface images and different types of abnormal heating surface images, and each heating surface image is preprocessed to obtain a preprocessed heating surface image; wherein the types of the abnormal heating surface images include at least one of: a heating surface dusting image, a heating surface coking image, and a heating surface corrosion image; A feature learner is used to extract features from each pre-processed heating surface image, and the characteristics of each pre-processed heating surface image are trained to obtain an image recognition model; A plurality of images of the heating surface in the furnace of a coal-fired power plant boiler and the spatial position of each image on the heating surface in the furnace are collected by an automatic image collection device, and the collected heating surface images in the furnace are recognized by a graphic recognition model to identify abnormal heating surface images; Draw a three-dimensional model that is consistent with the actual size and spatial distribution of the boiler based on the boiler design drawings, mark the abnormal heating surface image on the boiler model, and form a diagnostic report; The method of drawing a three-dimensional model consistent with the actual size and spatial distribution of the boiler according to the boiler design drawings and marking the abnormal heating surface image on the boiler model specifically includes: Broadly mark each abnormal heating surface image identified on the corresponding area of ​​the three-dimensional simulation model, and generate a diagnosis report of abnormal heating surface conditions in different heating surface areas based on the broadly marked three-dimensional simulation model, and the diagnosis report is used to push it to maintenance personnel; use a three-dimensional drawing tool to build a three-dimensional simulation model of the boiler according to the actual size of the boiler, and preset corresponding eye-catching marks for different types of abnormal heating surface images; according to the spatial position of each abnormal heating surface image on the heating surface of the boiler, mark the corresponding preset eye-catching mark on the corresponding abnormal area of ​​the three-dimensional simulation model, so that maintenance personnel can locate the maintenance position of the heating surface based on the abnormal heating surface image of the three-dimensional simulation model; The preprocessing of each heated surface image to obtain a preprocessed heated surface image specifically includes: Performing augmentation processing on each heated surface image, wherein the augmentation processing operation includes: rotating the heated surface image, mirror flipping, changing the brightness of the entire image or a part of the image pixels, and / or adding random noise to the global image to simulate different real scenes; Scaling the augmented heated surface image to a preset size; De-averaging the heating surface images of predetermined sizes to highlight the contrast of the heating surface images of each predetermined size, thereby obtaining a pre-processed heating surface image; The feature learner is a convolutional neural network; The feature learner is used to extract features from each pre-processed heating surface image, and the characteristics of each pre-processed heating surface image are trained to obtain an image recognition model, which specifically includes: Extract the RGB components of each pre-processed heating surface image and input the RGB components of each pre-processed heating surface image into a convolutional neural network respectively. The convolutional layer of the convolutional neural network performs weighted calculation based on different convolution kernels to obtain a feature matrix; and perform non-linear activation on the feature matrix through an activation function and then input it into a pooling layer for feature compression; and repeat the convolutional layer and pooling layer of the convolutional neural network for multiple times to obtain a set of fixed-size feature matrices; The fixed-size feature matrix is ​​input into the fully connected layer in the product neural network, and the fixed-size feature matrix structure is transformed through the fully connected layer to obtain a fixed-length feature vector; The fixed-length feature vector is input into the classifier for spatial probability mapping, and the pre-processed heating surface image is classified into normal heating surface and different types of abnormal heating surface images; The fixed-length feature vector is input into the classifier for spatial probability mapping, and the pre-processed heating surface image is classified into normal heating surface and different types of abnormal heating surface images, specifically including: The classifier is a softmax activation function, which maps each vector in the fixed-length feature vector output by the fully connected layer to the probability space of the image category through the softmax activation function to obtain the true probability value of the feature vector under each category; then the loss function is used to calculate the true probability of classifying the preprocessed heated surface image as a certain type and the error of the real sample to feedforward update the weight to obtain the updated true probability value of the preprocessed heated surface image, and the updated true probability value is used as the probability that the preprocessed heated surface image is identified as a certain category in the current training round; according to the probability, the preprocessed heated surface image is matched as a normal heated surface or different types of abnormal heated surface images.

2. The method for identifying and marking abnormal images of a boiler heating surface according to claim 1, It is characterized in that The RGB components of each pre-processed heated surface image are extracted and the RGB components of each pre-processed heated surface image are respectively input into a convolutional neural network, and the convolutional layer of the convolutional neural network performs weighted calculation based on different convolution kernels to obtain a feature matrix, specifically including: The product kernel included in the convolution layer represents the weight of the RGB component of the preprocessed heated surface image; The features of the layer of each preprocessed preheated image are extracted by performing dot multiplication with the convolution kernel and its corresponding RGB component, and the extracted features are formed into a feature matrix; and when extracting the features of the layer of each preheated preprocessed image, the spatial position of the preprocessed image eigenvalue where the feature is located is kept unchanged.

3. The method for identifying and marking abnormal images of a boiler heating surface according to claim 1, It is characterized in that The method further calculates the true probability of classifying the pre-processed heated surface image as a certain type and the error of the real sample by the loss function to feed-forward update the weight, and obtains the true probability value of the pre-processed heated surface image update, specifically including: The loss function of the convolutional neural network is set. When minimizing the loss function, the loss function is fed back to the weights of each pixel of the fully connected layer and the convolutional layer in turn through the back propagation algorithm. In the process of back propagation, the weights of each pixel of the convolutional layer and the fully connected layer are adjusted by the gradient descent method, and the parameter value of the cross entropy loss function is adjusted according to the error. The above process is continuously iterated until the cross entropy loss function converges, and the risk of the loss function of the convolutional neural network is minimized to obtain the loss of the true probability of classifying the preprocessed heated surface image as a certain type. The weight of the preprocessed heated surface image is updated according to the loss to obtain the updated true probability value of the preprocessed heated surface image.

4. The method for identifying and marking abnormal images of a boiler heating surface according to claim 1, It is characterized in that The automatic image acquisition device is a drone equipped with a high-definition camera; The step of acquiring a large number of images of heating surfaces in the boiler of a coal-fired power plant specifically includes: A large number of heating surface images of coal-fired power plant boilers are obtained through automatic image acquisition equipment.

5. A boiler heating surface abnormal image recognition and marking system, It is characterized in that include: The furnace heating surface image acquisition unit is used to acquire a large number of heating surface images of the furnace of the coal-fired power plant boiler, wherein the heating surface images have been artificially identified as normal heating surface images and different types of abnormal heating surface images, and pre-process each heating surface image to obtain a pre-processed heating surface image; wherein the types of the abnormal heating surface images include at least one of: a heating surface dust image, a heating surface coking image, and a heating surface corrosion image; An image recognition model forming unit is used to extract features from each pre-processed heating surface image using a feature learner, and train the characteristics of each pre-processed heating surface image to obtain an image recognition model; The abnormal heating surface image recognition unit is used to collect multiple images of the heating surface in the furnace of the coal-fired power plant boiler and the spatial position of each image on the heating surface in the furnace through an automatic image acquisition device, recognize the collected heating surface images in the furnace through a graphic recognition model, and recognize abnormal heating surface images; The abnormal heating surface image marking unit is used to draw a three-dimensional model consistent with the actual size and spatial distribution of the boiler according to the boiler design drawings, mark the abnormal heating surface image on the boiler model, and form a diagnosis report; The abnormally heated surface image marking unit is specifically used for: A three-dimensional simulation model of the boiler is constructed according to the actual size of the boiler using a three-dimensional drawing tool, and corresponding eye-catching marks are preset for different types of abnormal heating surface images; according to the spatial position of each abnormal heating surface image on the heating surface of the boiler, the corresponding preset eye-catching marks are marked on the corresponding abnormal area of ​​the three-dimensional simulation model, so that maintenance personnel can locate the maintenance position of the heating surface based on the abnormal heating surface image of the three-dimensional simulation model; Broadly marking each abnormal heating surface image identified on the corresponding area of ​​the three-dimensional simulation model, and generating a diagnosis report of abnormal heating surface conditions in different heating surface areas based on the broadly marked three-dimensional simulation model, wherein the diagnosis report is pushed to maintenance personnel; The furnace heating surface image acquisition unit is specifically used for: Performing augmentation processing on each heated surface image, wherein the augmentation processing operation includes: rotating the heated surface image, mirror flipping, changing the brightness of the entire image or a part of the image pixels, and / or adding random noise to the global image to simulate different real scenes; Scaling the augmented heated surface image to a preset size; De-averaging the heating surface images of predetermined sizes to highlight the contrast of the heating surface images of each predetermined size, thereby obtaining a pre-processed heating surface image; The image recognition model forming unit is specifically used for: The feature learner is a convolutional neural network; Extract the RGB components of each pre-processed heating surface image and input the RGB components of each pre-processed heating surface image into a convolutional neural network respectively. The convolutional layer of the convolutional neural network performs weighted calculation based on different convolution kernels to obtain a feature matrix; and perform non-linear activation on the feature matrix through an activation function and then input it into a pooling layer for feature compression; and repeat the convolutional layer and pooling layer of the convolutional neural network for multiple times to obtain a set of fixed-size feature matrices; The fixed-size feature matrix is ​​input into the fully connected layer in the product neural network, and the fixed-size feature matrix structure is transformed through the fully connected layer to obtain a fixed-length feature vector; The fixed-length feature vector is input into the classifier for spatial probability mapping, and the pre-processed heating surface image is classified into normal heating surface and different types of abnormal heating surface images; The fixed-length feature vector is input into the classifier for spatial probability mapping, and the pre-processed heating surface image is classified into normal heating surface and different types of abnormal heating surface images, specifically including: The classifier is a softmax activation function, which maps each vector in the fixed-length feature vector output by the fully connected layer to the probability space of the image category through the softmax activation function to obtain the true probability value of the feature vector under each category; then the loss function is used to calculate the true probability of classifying the preprocessed heated surface image as a certain type and the error of the real sample to feedforward update the weight to obtain the updated true probability value of the preprocessed heated surface image, and the updated true probability value is used as the probability that the preprocessed heated surface image is identified as a certain category in the current training round; according to the probability, the preprocessed heated surface image is matched as a normal heated surface or different types of abnormal heated surface images.

Images