Method and system for monitoring abnormal state of thermoelectric power plant based on deep learning

Abstract

The invention discloses a thermoelectric power plant abnormal state monitoring method and system based on deep learning. The method comprises the following steps: S1, extracting a thermoelectric power plant image sequence frame X from a video; s2, constructing a thermoelectric power plant image multi-level spatial-temporal feature extraction model which comprises a multi-scale spatial-temporal feature extraction sub-model, a spatial-temporal feature enhancement sub-model and a spatial-temporal feature attention fusion sub-model; extracting the multi-level spatial-temporal characteristics of the thermoelectric power plant image; s3, predicting a category label of the multi-level spatial-temporal characteristics of the thermoelectric power plant image, and constructing a loss function to evaluate and predict a difference between the category label and a real label to obtain abnormal state information; and S4, establishing a multi-level early warning system, and triggering different levels of early warning according to the severity of the abnormal state. According to the invention, abnormity monitoring and early warning can be automatically carried out, and the safety performance is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of abnormal monitoring of thermal power plants, and specifically relates to a method and system for monitoring abnormal conditions of thermal power plants based on deep learning. Background Art

[0002] Coal mine production accidents occur from time to time. Among them, power plants are the key operating environment for coal transportation and processing. If their abnormal conditions are not monitored and handled in time, serious production accidents and even casualties may occur. Traditional abnormal state monitoring methods for thermal power plants rely on manual or simple sensors, which have problems such as low monitoring efficiency, low accuracy, and long response time. With the development of artificial intelligence and deep learning technologies, intelligent transformation has become an effective way to improve the safety level of thermal power industry. Through real-time monitoring and early warning systems, abnormal conditions of thermal power plants, such as coal blockage, foreign matter, cracks, etc., can be discovered and handled in time, thereby effectively reducing the risk of shutdown and production reduction, and ensuring the continuity and stability of coal production.

[0003] However, there are still the following main problems in monitoring abnormal conditions of thermal power plants:

[0004] 1) Existing monitoring technologies often rely on fixed rules or simple algorithms, which are difficult to cope with complex and changing field environments and have limited ability to identify subtle abnormal conditions.

[0005] 2) The amount of data generated by thermal power plants is huge. Traditional data processing methods are difficult to meet real-time requirements and it is difficult to extract valuable information from massive data.

[0006] 3) Most existing monitoring systems require human participation and are difficult to automate and intelligentize, which not only increases labor costs but also prolongs response time.

[0007] 4) There is a lack of effective early warning mechanisms. Even if anomalies are detected, it is difficult to provide timely local and remote warnings, and it is difficult to take prompt measures to prevent accidents. Summary of the invention

[0008] The purpose of the present invention is to provide a method and system for monitoring abnormal conditions of thermal power plants based on deep learning. The method constructs an advanced deep learning model to intelligently monitor various abnormal phenomena in thermal power plants. By real-time analysis of relevant video streams or image data, various abnormal conditions can be accurately and real-time identified, and early warnings can be issued in a timely manner, effectively improving the safety production level of the thermal power industry.

[0009] In order to achieve one of the above purposes, the present invention adopts the following technical solution:

[0010] A method for monitoring abnormal conditions of a thermal power plant based on deep learning comprises the following steps:

[0011] S1: extracting thermal power plant image sequence frames X from the video;

[0012] S2: Construct a multi-level spatiotemporal feature extraction model for thermal power plant images, including a multi-scale spatiotemporal feature extraction sub-model, a spatiotemporal feature enhancement sub-model, and a spatiotemporal feature attention fusion sub-model; including:

[0013] S201: extracting image spatiotemporal features in the thermal power plant image sequence frame X using the multi-scale spatiotemporal feature extraction sub-model;

[0014] S202: extracting temporal correlation and dependency features in the thermal power plant image sequence frame X using the temporal and spatial feature enhancement sub-model, and fusing them with the image temporal and spatial features to enhance the temporal and spatial features of the thermal power plant image;

[0015] S203: extracting representative multi-level spatiotemporal features of the thermal power plant image in the thermal power plant image sequence frame X by using the spatiotemporal feature attention fusion sub-model;

[0016] S3: predicting the category labels of the multi-level spatiotemporal features of the thermal power plant image, constructing a loss function to evaluate the difference between the prediction and the true label, performing abnormal state monitoring, and obtaining abnormal state information;

[0017] S4: Build a multi-level warning system to trigger different levels of warnings according to the severity of the abnormal status information in step S3.

[0018] Furthermore, the step S1 also includes preprocessing the thermal power plant image data.

[0019] Furthermore, the preprocessing of the thermal power plant image data specifically includes: performing Gaussian kernel denoising, size adjustment and normalization, grayscale conversion and edge detection processing on the thermal power plant image data.

[0020] Furthermore, the S201: multi-scale spatiotemporal feature extraction sub-model extracts the image spatiotemporal features in the thermal power plant image sequence frame X; specifically includes:

[0021] S2011: Multi-scale convolutional layers are used to capture the spatiotemporal features of effective thermal power plant images at different scales, by using convolution kernels of different sizes K in parallel. s The 3D convolution layer is used to obtain multi-scale image features, where small convolution kernels are used to capture local detail information of the thermal power plant image, and large convolution kernels are used to capture global comprehensive information. The output features of each scale are expressed as:

[0022] O s =Conv3D(X,K s )+b s

[0023] where b s It is the bias term of the convolution layer of the corresponding scale in the multi-scale 3D convolutional network;

[0024] S2012: Use multiple 3D convolutional layers in series to extract temporal and spatial features from multi-scale convolutional outputs to obtain temporal contextual information and dependencies between video frames;

[0025] S2013: Using dilated convolution to 3D Processing is performed to increase the receptive field by inserting spatial intervals in the convolution kernel:

[0026] S2014: The 3D convolution layer output feature maps with different dilation factors and different scales after the dilation convolution processing are converted into attention calculation matrices Q, K and V using a linear layer;

[0027] S2015: Fusion through attention mechanism, enhanced feature representation, and input into subsequent feature enhancement network;

[0028]

[0029] The network captures rich image spatiotemporal features from local details to global contexts in thermal power plant images.

[0030] Furthermore, the step S202: extracting the temporal correlation and dependency features in the thermal power plant image sequence frame X by using the temporal and spatial feature enhancement sub-model, and fusing them with the image temporal and spatial features to enhance the temporal and spatial features of the thermal power plant image, specifically includes:

[0031] The gated recurrent unit GRU is used to extract the temporal correlation and dependency features in the sequence, and is fused with the image spatiotemporal features O to enhance the spatiotemporal features of the thermal power plant image, identify abnormal states in the sequence, and obtain the feature map O. GRU , the feature map O GRU The long-term temporal dependency information of related frames and multi-level spatial features are already included.

[0032] Further, the step S203: extracting representative multi-level spatiotemporal features of the thermal power plant image in the thermal power plant image sequence frame X by using the spatiotemporal feature attention fusion sub-model specifically includes:

[0033] S2031: Focus on the important area of ​​the thermal power plant image, learn an attention weight, and compare it with the feature map O GRUMultiply to enhance key features in the image and suppress unimportant features:

[0034] A(x,y)=softmax(W·O GRU (x,y)+b)

[0035] F att (x,y,c)=A(x,y)⊙O GRU (x,y,c)

[0036] Among them, A(x,y) is the attention weight of the spatial position (x,y), O GRU is the feature map, W and b are learnable parameters, ⊙ represents the dot product operation, F att is the feature map weighted by attention;

[0037] S2032: In order to enhance the feature representation of thermal power plant images, a feature pyramid network FPN is used to combine different levels of attention feature maps F through top-down and lateral connections. k , and obtain a representative multi-scale feature representation.

[0038] Furthermore, in step S3, a loss function is constructed to evaluate the difference between the prediction and the true label, and abnormal state monitoring is performed, which specifically includes:

[0039] If the samples of the categories are balanced, a weighted cross entropy loss function is constructed to evaluate the difference between the prediction and the true label;

[0040] If the samples of some categories are unbalanced, the weighted cross entropy loss function is replaced by assigning different weights to each category. The loss function after replacement is:

[0041]

[0042] Where N is the number of categories, y c is the one-hot encoding of the true label, p c is the probability of the model detecting category c, w c is the weight of category c.

[0043] Furthermore, the step S4: building a multi-level warning system to trigger different levels of warnings according to the severity of the abnormal status information in the step 3, specifically includes:

[0044] S401: Set a confidence score for the detected abnormal state, expressed as:

[0045] P max =max(P)

[0046] Among them, P is the probability vector output by the softmax layer, Pmax is the maximum probability value, representing the confidence in the predicted abnormal state category;

[0047] S402: Setting dynamic thresholds for each warning level based on historical data and real-time monitoring requirements;

[0048] S403: Compare the predicted confidence score with the warning threshold, determine the warning level, and implement adaptive and timely warning;

[0049]

[0050] Among them, T high is the high warning threshold of the early warning system, T low It is the low warning threshold of the early warning system.

[0051] In order to achieve the second of the above objectives, the present invention adopts the following technical solution:

[0052] A thermal power plant abnormal state monitoring system based on deep learning, comprising: a data acquisition module, a multi-level feature extraction module, an abnormal state detection module and an abnormal state early warning module.

[0053] The data acquisition module is used to extract the thermal power plant image sequence frame X from the video;

[0054] The multi-level feature extraction module includes a multi-scale spatiotemporal feature extraction submodule, a spatiotemporal feature enhancement submodule, and a spatiotemporal feature attention fusion submodule, wherein the multi-scale spatiotemporal feature extraction submodule is used to extract the image spatiotemporal features in the image sequence frame X of the thermal power plant;

[0055] The spatiotemporal feature enhancement submodule is used to extract the temporal correlation and dependency features in the thermal power plant image sequence frame X, and fuse them with the image spatiotemporal features to enhance the spatiotemporal features of the thermal power plant image;

[0056] The spatiotemporal feature attention fusion submodule is used to extract representative multi-level spatiotemporal features of the thermal power plant image in the thermal power plant image sequence frame X;

[0057] The abnormal state detection module is used to predict the category labels of the multi-level spatiotemporal features of the thermal power plant image, construct a loss function to evaluate the difference between the prediction and the true label, perform abnormal state monitoring, and obtain abnormal state information;

[0058] The abnormal state warning module is used to build a multi-level warning system to trigger warnings of different levels according to the severity of the abnormal state.

[0059] Furthermore, it also includes a data preprocessing module, and the data preprocessing module is used to preprocess the image data of the thermal power plant.

[0060] Beneficial effects of the present invention:

[0061] This method constructs a corresponding data preprocessing scheme based on the characteristics of thermal power plant images, effectively improves the quality of thermal power plant images, and obtains representative abnormal image features of thermal power plants through a multi-level spatiotemporal feature extraction and fusion method of thermal power plant images based on a deep learning model, thereby achieving real-time monitoring of abnormal conditions such as coal blockage, foreign matter, and cracks in thermal power plants, and improving the safety of thermal power plant scenes. In addition, by constructing a multi-level early warning system, timely early warning of abnormal conditions at different levels is achieved. Therefore, the present invention has the advantages of high accuracy, real-time performance, and intelligence, and can significantly improve the safety management level of thermal power plants, reduce production risks, and ensure the efficient operation of thermal power plant production. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a flow chart of a specific embodiment 1 of the present invention;

[0063] Figure 2 This is a flow chart of feature extraction and abnormal state monitoring in specific embodiment 1 of the present invention;

[0064] Figure 3 This is a flowchart of preprocessing image data in specific embodiment 1 of the present invention;

[0065] Figure 4 This is a principle block diagram of specific embodiment 2 of the present invention. DETAILED DESCRIPTION

[0066] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Specific embodiment 1:

[0068] See also Figures 1 to 3 As shown, a method for monitoring abnormal conditions of a thermal power plant based on deep learning comprises the following steps:

[0069] Step S1: extracting a thermal power plant image sequence frame X from a video.

[0070] The step S1 also includes preprocessing the image data of the thermal power plant. Since the images in the special environment of the thermal power plant are affected by complex factors such as light, dust and smoke, the quality of the acquired thermal power plant images is poor, which affects the subsequent algorithm effect for the scene. Therefore, according to the characteristics of the special environment images of the thermal power plant, the relevant data is preprocessed and optimized to improve the image quality and lay the foundation for the subsequent related abnormal state monitoring. Specifically including:

[0071] S101: Denoising Thermal Power Plant Image Data

[0072] Since there is usually uneven lighting in power plants, the image brightness is low and the details are unclear. Using Gaussian filtering can effectively remove random noise in thermal power plant images, improve image quality, and make it easier to extract effective features of abnormal conditions in thermal power plants.

[0073] First, a Gaussian kernel is created, and then all elements in the Gaussian kernel are normalized. The Gaussian kernel is then applied to each pixel in the image, that is, each pixel is weighted summed with the Gaussian kernel elements at the corresponding position, thereby updating each pixel in the image and realizing the denoising of the thermal power plant image. For an N×N Gaussian kernel, the central element is located at (0,0), and each element G(xy( in the Gaussian kernel is obtained according to the following formula:

[0074]

[0075] Where 9x,y) are the pixel coordinates of the image, σ is the standard deviation of the Gaussian kernel, which controls the width of the Gaussian function, that is, the blurring degree of the filter. A larger σ value will produce a wider Gaussian curve, resulting in a stronger blurring effect; a smaller σ value will make the curve narrower and the blurring effect weaker.

[0076] S102: Resizing and Normalizing

[0077] A thermal power plant is a dynamic system, and there may be fast-moving objects in the image. Sizing and normalization can ensure that the image processing algorithm can adapt to dynamic scenes of different sizes and brightness while retaining as many image details as possible. Sizing can generally be achieved using bilinear interpolation. Normalization scales the pixel values ​​of the image to a specified range, such as [0,1], to accelerate the convergence of the network and improve the stability of the value. The normalization formula is expressed as:

[0078]

[0079] S103: Grayscale

[0080] Due to the influence of lighting and environmental conditions, the color of thermal power plant images may be distorted. Grayscale can remove the interference of color information and focus on the brightness and contrast of the image, which is helpful for subsequent target feature extraction and abnormal state identification. Grayscale is expressed as:

[0081] I gray (x,y)=w R ×I R (x,y)+w G ×I G (x,y)+w B ×I B )x,y(

[0082] where w R =0.2989,w G =0.5870,w B =0.1140, indicating the sensitivity of the human eye to different colors. The human visual system is most sensitive to green light, followed by red light, and least sensitive to blue light. Therefore, in the calculation of grayscale images, the green channel contributes the most, while the red and blue channels contribute less. R (x,y),I G (x,y and I B )x,y(refers to the intensity values ​​of the red, green, and blue channels of the pixel at position )x,y( in the image.

[0083] S104: Edge Detection

[0084] The dust and smoke in the environment of the thermal power plant will affect the clarity of the thermal power plant image. De-noising technology can help remove the interference of these granular materials, while edge detection can highlight key structures in the image, such as belt edges or material contours. Here, the Sobel operator is used to enhance the edge detection of the thermal power plant image.

[0085] The image data preprocessing method of the present invention adopts an image denoising method based on Gaussian filtering to target the influence of granular noise, illumination, etc. on the images of thermal power plants. In order to enhance the edge and contour information of the image, the Sobel operator is used to enhance the edge detection of the thermal power plant images, and a more comprehensive and effective thermal power plant image data preprocessing scheme is constructed, thereby improving the image quality.

[0086] Step S2: construct a multi-level spatiotemporal feature extraction model for thermal power plant images, including a multi-scale spatiotemporal feature extraction sub-model, a spatiotemporal feature enhancement sub-model, and a spatiotemporal feature attention fusion sub-model;

[0087] S201: extracting image spatiotemporal features in the thermal power plant image sequence frame X using the multi-scale spatiotemporal feature extraction sub-model, specifically including:

[0088] S2011: In order to extract rich image spatiotemporal features from thermal power plant video sequences, so as to deeply understand the dynamic thermal power plant images and identify abnormal states, a multi-scale convolutional layer is first used to capture the effective spatiotemporal features of thermal power plant images at different scales, and to improve the robustness of abnormal material features of different sizes under different contrast and low light conditions. By using convolution kernels K of different sizes in parallel, the multi-scale convolutional layer is used to capture the effective spatiotemporal features of thermal power plant images at different scales, and to improve the robustness of abnormal material features of different sizes under different contrast and low light conditions. s (where s∈{1,2,…,S}) is used to obtain multi-scale image features, where small convolution kernels are used to capture local detail information of thermal power plant images, and large convolution kernels are used to capture global comprehensive information. The output features of each scale are expressed as:

[0089] O s =Conv3D(X,K s )+b s

[0090] where b s It is the bias term of the convolution layer of the corresponding scale in the multi-scale 3D convolutional network;

[0091] S2012: Multiple 3D convolutional layers in series are used to extract temporal and spatial features from multi-scale convolutional outputs, obtain temporal contextual information and dependencies between video frames, and improve the capture of dynamically changing thermal power plant image features. For each scale of the 3D convolutional network layer, its output is expressed as:

[0092]

[0093] Where D is the 3D convolution kernel K 3D The size in the time dimension, H 3D and W 3D For its height and width, b 3D It is the bias term of the 3D convolutional network;

[0094] S2013: Using dilated convolution to 3D The processing is performed by inserting spatial intervals in the convolution kernel to increase the receptive field without increasing the number of parameters of the convolution kernel:

[0095]

[0096] Among them, H dilate and W dilate is the dilated convolution kernel K dilate The height and width of b dilateIt is the bias term for expanding the convolutional network; it increases the spatial and temporal receptive fields of the network, so that the model can learn more implicit knowledge of thermal power plant images.

[0097] S2014: The 3D convolution layer output feature maps with different dilation factors and different scales after the dilation convolution processing are converted into attention calculation matrices Q, K and V using a linear layer;

[0098] S2015: Fusion through attention mechanism, enhanced feature representation, and input into subsequent feature enhancement network;

[0099]

[0100] In this way, the dilated convolution is combined with the multi-scale 3D convolution. The network has a larger range of spatial and temporal receptive fields based on the spatiotemporal features of thermal power plant images of different scales, so that it can perceive richer features. The network can capture rich image spatiotemporal features O from local details to global context of thermal power plant images, which is very effective for obtaining abnormal states in complex monitoring scenes of thermal power plants, including but not limited to coal blockage, foreign matter in wood removal, and cracks in L-shaped chain plates of slot plates.

[0101] Possibly, before S2014, the 3D convolution layer output feature map with different dilation factors and different scales after the dilated convolution processing in step S2013 is nonlinearly transformed through an activation function. In this specific embodiment: the activation function is a ReLU activation function, and dimensionality reduction processing is performed through a pooling layer. The pooling layer adopts maximum pooling or average pooling to reduce the amount of calculation and extract main features.

[0102] S202: extracting the temporal correlation and dependency features in the thermal power plant image sequence frame X by using the temporal and spatial feature enhancement sub-model, and fusing them with the image temporal and spatial features to enhance the temporal and spatial features of the thermal power plant image, specifically including:

[0103] In order to make fuller use of the temporal context knowledge in the video frames of the thermal power plant, a spatiotemporal feature enhancement sub-model is constructed. The spatiotemporal feature enhancement sub-model uses the gated recurrent unit GRU to extract the temporal correlation and dependency features in the sequence, and fuses them with the above features to enhance the spatiotemporal features of the thermal power plant image and help identify abnormal states in the sequence. The feature map O is obtained. GRU , feature map O GRU The long-term temporal dependency information of related frames and multi-level spatial features are already included.

[0104] The image spatiotemporal feature O of each frame image in the multi-scale spatiotemporal feature extraction sub-model is used as the input x of the gated recurrent unit GRU t, the relevant temporal knowledge contained in the target image of the thermal power plant is further enhanced by mining the temporal context information. The process is expressed as:

[0105]

[0106] where r t is the reset gate, which is used to decide how much past information needs to be forgotten. t is the update gate, which determines how much new information will be stored in the current state. is a candidate hidden state, h t is a hidden state, the hidden state h t After the linear function is used to adjust the dimension and other transformations, the feature map O is obtained. GRU .

[0107] Through the self-attention calculation of spatiotemporal features, the spatial and temporal dependency information of the target image is mined to obtain more representative spatiotemporal features, which helps to identify abnormal states in the sequence, such as sudden changes in material flow or intermittent problems.

[0108] S203: extracting representative multi-level spatiotemporal features of the thermal power plant image in the thermal power plant image sequence frame X by using the spatiotemporal feature attention fusion sub-model, specifically including:

[0109] In order to strengthen the attention and capture of abnormal features of different sizes and directions in thermal power plant images, such as cracks, wear or foreign objects, a spatiotemporal feature attention fusion sub-model is constructed. The spatiotemporal feature attention fusion sub-module combines the spatial attention mechanism with the feature pyramid, so that the network can not only obtain important features at each scale, but also effectively fuse spatial features at different scales, thereby providing rich, multi-scale feature representation for subsequent abnormal state monitoring. Further mining the temporal context correlation information and realizing the fusion of spatiotemporal features at different scales, so that the network can learn more accurate knowledge of abnormal targets in the image.

[0110] S2031: In order to focus on the important area of ​​the thermal power plant image, learn an attention weight and combine it with the feature map O in S202 GRU Multiply to enhance key features in the image and suppress unimportant features:

[0111] A(x,y)=softmax(W·O GRU (x,y)+b)

[0112] F att (x,y,c)=A(x,y)⊙O GRU (x,y,c)

[0113] Among them, A(x,y) is the attention weight of the spatial position (x,y), W and b are learnable parameters, ⊙ represents the dot product operation, and F att is the attention-weighted feature map.

[0114] S2032: In order to enhance the feature representation of thermal power plant images, a feature pyramid network FPN is used to combine different levels of attention feature maps F through top-down and lateral connections. k :

[0115]

[0116] Here, k refers to the feature level, top-down refers to the process of combining features of different levels from top to bottom, and upsample refers to the upsampling operation. Through this process, spatial attention can be applied at each level of the feature pyramid network FPN to enhance the feature map of each scale. Then, the feature map enhanced by attention weighting is fused again through the FPN structure, and finally a representative multi-scale feature representation is obtained.

[0117] S3: First, the multi-level spatiotemporal features are further extracted through multiple fully connected network layers and information fusion is performed before classification, so that all the features of different levels are fused together, so that the final features used for abnormal state monitoring contain information of different levels of the network, thereby having stronger representation;

[0118] O fc =ReLU(W fc ·F k +b fc )

[0119] Among them, O fc is the output of the fully connected layer, W fc is the weight of the corresponding network layer, b fc is the bias term corresponding to the fully connected layer;

[0120] Then, a softmax activation function is used to output the probability of each category, and the category labels of the multi-level spatiotemporal features are predicted. Finally, a loss function is constructed to evaluate the difference between the prediction and the true label, and abnormal state monitoring is performed to obtain abnormal state information.

[0121] P = softmax(W P ·O fc +b P )

[0122] Among them, P is the probability vector output by the classification layer, W P is the bias of the corresponding network layer, b P It is the bias term corresponding to the network when calculating the classification probability;

[0123] If the samples of the categories are balanced, a weighted cross entropy loss function is constructed to evaluate the difference between the prediction and the true label;

[0124] If the samples of some categories are unbalanced, the weighted cross entropy loss function is replaced by assigning different weights to each category, thereby enhancing the ability to identify abnormal states. The loss function after replacement is defined as:

[0125]

[0126] Where N is the number of categories, y c is the one-hot encoding of the true label, p c is the probability of the model detecting category c, w c is the weight of category c.

[0127] S4: Building a multi-level warning system to trigger different levels of warnings according to the severity of the abnormal status information in step S3; specifically including:

[0128] S401: Set a confidence score for the detected abnormal state, expressed as:

[0129] P max =max(P)

[0130] Among them, P is the probability vector output by the softmax layer, P max is the maximum probability value, representing the confidence in the predicted abnormal state category;

[0131] S402: Setting dynamic thresholds for each warning level based on historical data and real-time monitoring requirements;

[0132] S403: Compare the predicted confidence score with the warning threshold, determine the warning level, and implement adaptive and timely warning Alert Level;

[0133]

[0134] Among them, T high is the high warning threshold of the early warning system, T low The present invention can adaptively trigger different levels of warnings according to the severity of the abnormal state monitored by the network, so as to directly make a judgment according to the warning situation and perform corresponding processing accordingly. Specific embodiment 2:

[0136] See also Figure 4As shown, a thermal power plant abnormal state monitoring system based on deep learning includes: a data acquisition module 1, a multi-level feature extraction module, an abnormal state detection module 5 and an abnormal state early warning module 6, the output end of the data acquisition module 1 is connected to the input end of the multi-level feature extraction module, the output end of the multi-level feature extraction module is connected to the input end of the abnormal state detection module 5, and the output end of the abnormal state detection module 5 is connected to the input end of the abnormal state early warning module 6.

[0137] The data acquisition module 1 is used to extract the thermal power plant image sequence frame X from the video;

[0138] The multi-level feature extraction module includes a multi-scale spatiotemporal feature extraction submodule 2, a spatiotemporal feature enhancement submodule 3, and a spatiotemporal feature attention fusion submodule 4.

[0139] The multi-scale spatiotemporal feature extraction submodule 2 is used to extract the image spatiotemporal features in the image sequence frame X of the thermal power plant;

[0140] The spatiotemporal feature enhancement submodule 3 is used to extract the temporal correlation and dependency features in the thermal power plant image sequence frame X, and fuse them with the image spatiotemporal features to enhance the spatiotemporal features of the thermal power plant image;

[0141] The spatiotemporal feature attention fusion submodule 4 is used to extract representative multi-level spatiotemporal features of the thermal power plant image in the thermal power plant image sequence frame X;

[0142] The abnormal state detection module 5 is used for abnormal state monitoring. First, the multi-level spatiotemporal features are further enhanced through multiple fully connected network layers to extract features and perform information fusion before classification. Then, a softmax activation function is used to output the probability of each category. Finally, a weighted cross entropy loss function is constructed to evaluate the difference between the prediction and the true label to obtain abnormal state information.

[0143] The abnormal state warning module 6 is used to build a multi-level warning system to trigger warnings of different levels according to the severity of the abnormal state.

[0144] Possibly, it further comprises a data preprocessing module 7, which is connected between the data acquisition module 1 and the multi-level feature extraction module and is used for preprocessing the thermal power plant image data.

[0145] The specific functions and functional features of each module are the same as those in the specific embodiment 1, so they are omitted here in this specific embodiment.

[0146] The technical solution provided by the present invention is described in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A method for monitoring abnormal state of thermal power plants based on deep learning, It is characterized in that The steps include: S1: extracting thermal power plant image sequence frames X from the video; S2: Construct a multi-level spatiotemporal feature extraction model for thermal power plant images, including a multi-scale spatiotemporal feature extraction sub-model, a spatiotemporal feature enhancement sub-model, and a spatiotemporal feature attention fusion sub-model; including: S201: extracting image spatiotemporal features in the thermal power plant image sequence frame X using the multi-scale spatiotemporal feature extraction sub-model; S202: extracting temporal correlation and dependency features in the thermal power plant image sequence frame X using the temporal and spatial feature enhancement sub-model, and fusing them with the image temporal and spatial features to enhance the temporal and spatial features of the thermal power plant image; S203: extracting representative multi-level spatiotemporal features of the thermal power plant image in the thermal power plant image sequence frame X by using the spatiotemporal feature attention fusion sub-model; S3: predicting the category labels of the multi-level spatiotemporal features of the thermal power plant image, constructing a loss function to evaluate the difference between the prediction and the true label, performing abnormal state monitoring, and obtaining abnormal state information; S4: Build a multi-level warning system to trigger different levels of warnings according to the severity of the abnormal status information in step S3.

2. The method for monitoring abnormal state of a thermal power plant based on deep learning according to claim 1, Features: The step S1 also includes preprocessing the thermal power plant image data.

3. The method for monitoring abnormal state of a thermal power plant based on deep learning according to claim 2, Features: The preprocessing of the thermal power plant image data specifically includes: performing Gaussian kernel denoising, size adjustment and normalization, grayscale conversion and edge detection processing on the thermal power plant image data.

4. The method for monitoring abnormal state of a thermal power plant based on deep learning according to claim 1, Features: The S201: a multi-scale spatiotemporal feature extraction sub-model extracts the image spatiotemporal features in the thermal power plant image sequence frame X; Specifically include: S2011: Multi-scale convolutional layers are used to capture the spatiotemporal features of effective thermal power plant images at different scales, by using convolution kernels of different sizes K in parallel. s The 3D convolution layer is used to obtain multi-scale image features, where small convolution kernels are used to capture local detail information of the thermal power plant image, and large convolution kernels are used to capture global comprehensive information. The output features of each scale are expressed as: THE s =Conv3D(X,K s )+b s where b s It is the bias term of the convolution layer of the corresponding scale in the multi-scale 3D convolutional network; S2012: Use multiple 3D convolutional layers in series to extract temporal and spatial features from multi-scale convolutional outputs to obtain temporal contextual information and dependencies between video frames; S2013: Using dilated convolution to 3D Processing is performed to increase the receptive field by inserting spatial intervals in the convolution kernel: S2014: The 3D convolution layer output feature maps with different dilation factors and different scales after the dilation convolution processing are converted into attention calculation matrices Q, K and V using a linear layer; S2015: Fusion through attention mechanism, enhanced feature representation, and input into subsequent feature enhancement network; The network captures rich image spatiotemporal features from local details to global contexts in thermal power plant images.

5. The method for monitoring abnormal state of a thermal power plant based on deep learning according to claim 1 or 4, Features: The step S202: extracting the temporal correlation and dependency features in the thermal power plant image sequence frame X by using the temporal and spatial feature enhancement sub-model, and fusing them with the image temporal and spatial features to enhance the temporal and spatial features of the thermal power plant image, specifically includes: The gated recurrent unit GRU is used to extract the temporal correlation and dependency features in the sequence, and is fused with the image spatiotemporal features O to enhance the spatiotemporal features of the thermal power plant image, identify abnormal states in the sequence, and obtain the feature map O. GRU , the feature map O GRU The long-term temporal dependency information of related frames and multi-level spatial features are already included.

6. The method for monitoring abnormal state of a thermal power plant based on deep learning according to claim 5, Features: The step S203: extracting representative multi-level spatiotemporal features of the thermal power plant image in the thermal power plant image sequence frame X by using the spatiotemporal feature attention fusion sub-model, specifically includes: S2031: Focus on the important area of ​​the thermal power plant image, learn an attention weight, and compare it with the feature map O GRU Multiply to enhance key features in the image and suppress unimportant features: A(x,y)=softmax(W·O GRU (x,y)+b) F att (x,y,c)=A(x,,y)☉O GRU (x,y,c) Among them, A(x, y) is the attention weight of the spatial position (x, y), O GRU is the feature map, W and b are learnable parameters, ⊙ represents the dot product operation, F att is the feature map weighted by attention; S2032: In order to enhance the feature representation of thermal power plant images, a feature pyramid network FPN is used to combine different levels of attention feature maps F through top-down and lateral connections. k , and obtain a representative multi-scale feature representation.

7. The method for monitoring abnormal state of a thermal power plant based on deep learning according to claim 1, Features: In step S3, a loss function is constructed to evaluate the difference between the prediction and the true label, and abnormal state monitoring is performed, specifically including: If the samples of the categories are balanced, a weighted cross entropy loss function is constructed to evaluate the difference between the prediction and the true label; If the samples of some categories are unbalanced, the weighted cross entropy loss function is replaced by assigning different weights to each category. The loss function after replacement is: Where N is the number of categories, y c is the one-hot encoding of the true label, p c is the probability of the model detecting category c, w c is the weight of category c.

8. The method for monitoring abnormal state of a thermal power plant based on deep learning according to claim 1, Features: The step S4: building a multi-level warning system to trigger different levels of warnings according to the severity of the abnormal status information in the step 3, specifically including: S401: Set a confidence score for the detected abnormal state, expressed as: P max =max(P) Among them, P is the probability vector output by the softmax layer, P max is the maximum probability value, representing the confidence in the predicted abnormal state category; S402: Setting dynamic thresholds for each warning level based on historical data and real-time monitoring requirements; S403: Compare the predicted confidence score with the warning threshold, determine the warning level, and implement adaptive and timely warning; Among them, T high is the high warning threshold of the early warning system, T low It is the low warning threshold of the early warning system.

9. A thermal power plant abnormal state monitoring system based on deep learning, It is characterized in that include: Data acquisition module, multi-level feature extraction module, abnormal state detection module and abnormal state warning module, The data acquisition module is used to extract the thermal power plant image sequence frame X from the video; The multi-level feature extraction module includes a multi-scale spatiotemporal feature extraction submodule, a spatiotemporal feature enhancement submodule, and a spatiotemporal feature attention fusion submodule, wherein the multi-scale spatiotemporal feature extraction submodule is used to extract the image spatiotemporal features in the image sequence frame X of the thermal power plant; The spatiotemporal feature enhancement submodule is used to extract the temporal correlation and dependency features in the thermal power plant image sequence frame X, and fuse them with the image spatiotemporal features to enhance the spatiotemporal features of the thermal power plant image; The spatiotemporal feature attention fusion submodule is used to extract representative multi-level spatiotemporal features of the thermal power plant image in the thermal power plant image sequence frame X; The abnormal state detection module is used to predict the category labels of the multi-level spatiotemporal features of the thermal power plant image, construct a loss function to evaluate the difference between the prediction and the true label, perform abnormal state monitoring, and obtain abnormal state information; The abnormal state warning module is used to build a multi-level warning system to trigger warnings of different levels according to the severity of the abnormal state.

10. The thermal power plant abnormal state monitoring system based on deep learning according to claim 9, Features: It also includes a data preprocessing module, which is used to preprocess the image data of the thermal power plant.

Images