Anomaly State Monitoring Method and System for Thermoelectric Power Plants Based on Deep Learning

Abstract

The present invention discloses an abnormal state monitoring method and system for a thermoelectric power plant based on deep learning, comprising the following steps: S1: Extracting a sequence of image frames X of the thermoelectric power plant from a video; S2: Constructing a multi-level spatio-temporal feature extraction model for the images of the thermoelectric power plant, including a multi-scale spatio-temporal feature extraction sub-model, a spatio-temporal feature enhancement sub-model, and a spatio-temporal feature attention fusion sub-model; Extracting multi-level spatio-temporal features of the images of the thermoelectric power plant; S3: Predicting the class labels of the multi-level spatio-temporal features of the images of the thermoelectric power plant, constructing a loss function to evaluate the difference between the prediction and the true labels, and obtaining abnormal state information; S4: Building a multi-level early warning system, and triggering early warnings of different levels according to the severity of the abnormal state. The present invention can automatically perform abnormal monitoring and early warning, and improves the safety performance.

Description

Technical Field

[0001] The present invention belongs to the technical field of abnormal monitoring of thermoelectric power plants, and particularly relates to a method and system for abnormal state monitoring of thermoelectric power plants based on deep learning. Background Technique

[0002] Coal mine safety production accidents occur from time to time. Among them, as a key operation environment for coal transportation and processing, if the abnormal state of the power plant is not monitored and processed in time, it may lead to serious production accidents and even casualties. Traditional methods for abnormal state monitoring of thermoelectric power plants rely on manual or simple sensors, and there are problems such as low monitoring efficiency, low accuracy, and long response time. With the development of artificial intelligence and deep learning technologies, intelligent transformation has become an effective way to improve the safety production level of the thermoelectric industry. Through a real-time monitoring and early warning system, abnormal states of thermoelectric power plants, such as coal blockage, foreign objects, cracks, etc., can be detected and processed in time, thereby effectively reducing the risks of production suspension and production reduction, and ensuring the continuity and stability of coal production.

[0003] However, there are still the following main problems in the current monitoring of the abnormal state of thermoelectric power plants:

[0004] 1) Existing monitoring technologies often rely on fixed rules or simple algorithms, and it is difficult to cope with complex and changeable on-site environments, and the ability to identify subtle abnormal states is limited.

[0005] 2) The amount of data generated by thermoelectric power plants is huge. Traditional data processing methods are difficult to meet the real-time requirements, and it is difficult to extract valuable information from the massive data.

[0006] 3) Most existing monitoring systems require manual participation, and it is difficult to achieve automation and intelligence, which not only increases labor costs but also prolongs the response time.

[0007] 4) There is a lack of an effective early warning mechanism. Even if an abnormality is detected, it is difficult to achieve timely local and remote early warnings, and it is difficult to quickly take measures to prevent accidents. Summary of the Invention

[0008] The object of the present invention is a method and system for abnormal state monitoring of thermoelectric power plants based on deep learning. This method constructs an advanced deep learning model to intelligently monitor various abnormal phenomena of thermoelectric power plants. By real-time analyzing relevant video streams or image data, it accurately and real-time identifies various abnormal states and issues early warnings in time, effectively improving the safety production level of the thermoelectric industry.

[0009] In order to achieve one of the above objects, the present invention adopts the following technical solutions:

[0010] A method for abnormal state monitoring of a thermoelectric power plant based on deep learning, comprising the following steps:

[0011] S1: Extract the image sequence frames X of the thermoelectric power plant from the video;

[0012] S2: Construct a multi-level spatio-temporal feature extraction model for the thermoelectric power plant images, including a multi-scale spatio-temporal feature extraction sub-model, a spatio-temporal feature enhancement sub-model, and a spatio-temporal feature attention fusion sub-model; including:

[0013] S201: Use the multi-scale spatio-temporal feature extraction sub-model to extract the spatio-temporal features of the thermoelectric power plant image sequence frames X;

[0014] S202: Use the spatio-temporal feature enhancement sub-model to extract the temporal correlation and dependence features in the thermoelectric power plant image sequence frames X, and fuse them with the spatio-temporal features of the images to enhance the spatio-temporal features of the thermoelectric power plant images;

[0015] S203: Use the spatio-temporal feature attention fusion sub-model to extract the multi-level spatio-temporal features of the thermoelectric power plant images with representativeness in the thermoelectric power plant image sequence frames X;

[0016] S3: Predict the class labels of the multi-level spatio-temporal features of the thermoelectric power plant images, construct a loss function to evaluate the difference between the prediction and the true labels, perform abnormal state monitoring, and obtain abnormal state information;

[0017] S4: Build a multi-level early warning system, and trigger early warnings at different levels according to the severity of the abnormal state information in step S3.

[0018] Further, step S1 further includes preprocessing the thermoelectric power plant image data.

[0019] Further, the preprocessing of the thermoelectric power plant image data specifically includes: performing Gaussian kernel denoising, size adjustment and normalization, grayscale conversion, and edge detection processing on the thermoelectric power plant image data.

[0020] Further, S201: The multi-scale spatio-temporal feature extraction sub-model extracts the spatio-temporal features of the thermoelectric power plant image sequence frames X; specifically includes:

[0021] S2011: Use a multi-scale convolutional layer to capture the effective spatio-temporal features of the thermoelectric power plant images at different scales, and use 3D convolutional layers with different sizes of convolutional kernels K s in parallel to obtain multi-scale image features, where small convolutional kernels are used to capture the local detail information of the thermoelectric power plant images, and large convolutional kernels are used to capture the global comprehensive information. The output feature of each scale is expressed as:

[0022] O s = Conv3D(X, K s ) + b s

[0023] where b s is the bias term of the convolutional layer at the corresponding scale in the multi-scale 3D convolutional network;

[0024] S2012: Use multiple cascaded 3D convolutional layers to extract temporal and spatial feature information from the multi-scale convolutional output, and obtain the temporal correlation context information and dependencies between video frames;

[0025] S2013: Use dilated convolution to process O 3D by inserting spatial gaps in the convolutional kernel to increase the receptive field:

[0026] S2014: The output feature maps of the 3D convolutional layers with different dilation factors and different scales after the dilated convolution processing are converted into the attention calculation matrices Q, K, and V using a linear layer;

[0027] S2015: Perform fusion through the attention mechanism to enhance the feature representation and input it into the subsequent feature enhancement network;

[0028]

[0029] The network captures rich spatio-temporal features of the thermoelectric power plant image from local details to global context O.

[0030] Furthermore, in S202: Use the spatio-temporal feature enhancer sub-model to extract the temporal correlation and dependence features in the sequence of frames X of the thermoelectric power plant image, and fuse them with the spatio-temporal features of the image to enhance the spatio-temporal features of the thermoelectric power plant image, specifically including:

[0031] Use a gated recurrent unit GRU to extract the temporal correlation and dependence features in the sequence, and fuse them with the spatio-temporal features O of the image to enhance the spatio-temporal features of the thermoelectric power plant image, identify abnormal states in the sequence, and obtain the feature map O GRU The feature map O GRU already contains the long-term temporal dependence information of the relevant frames and multi-level spatial features.

[0032] Furthermore, in S203: Use the spatio-temporal feature attention fusion sub-model to extract the multi-level spatio-temporal features of the thermoelectric power plant image with representativeness in the sequence of frames X of the thermoelectric power plant image, specifically including:

[0033] S2031: Focus on the important regions of the thermoelectric power plant image, learn an attention weight, and combine it with the feature map O GRUMultiply to enhance key features in the image and suppress unimportant features:

[0034] A(x,y) = softmax(W·O GRU (x,y) + b)

[0035] F att (x,y) = A(x,y)⊙O GRU (x,y)

[0036] where A(x,y) is the attention weight at spatial position (x,y), O GRU is the feature map, W and b are learnable parameters, ⊙ represents the dot product operation, and F att is the feature map weighted by attention;

[0037] S2032: To enhance the feature representation of the thermoelectric power plant image, a Feature Pyramid Network (FPN) is used to combine attention feature maps F at different levels through top-down and lateral connections k to obtain a representative multi-scale feature representation.

[0038] Furthermore, in step S3, a loss function is constructed to evaluate the difference between the prediction and the true label for abnormal state monitoring, which specifically includes:

[0039] If the samples of the classes are balanced, a weighted cross-entropy loss function is constructed to evaluate the difference between the prediction and the true label;

[0040] If the samples of some classes are unbalanced, the weighted cross-entropy loss function is replaced by assigning different weights to each class, and the replaced loss function is:

[0041]

[0042] where N is the number of classes, yc is the one-hot encoding of the true label, pc is the probability that the model detects class c, and w c is the weight of class c.

[0043] Furthermore, in step S4: Build a multi-level warning system, and trigger warnings at different levels according to the severity of the abnormal state information in step 3, which specifically includes:

[0044] S401: Set a confidence score for the detected abnormal state, which is expressed as:

[0045] P max = max(P)

[0046] where P is the probability vector output by the softmax layer, and P max is the maximum probability value, representing the confidence in predicting the abnormal state class;

[0047] S402: Set dynamic thresholds for each warning level according to historical data and real-time monitoring requirements;

[0048] S403: Compare the predicted confidence score with the warning threshold to determine the warning level and achieve adaptive and timely warning;

[0049]

[0050] where T high is the high warning threshold of the warning system, and T low is the low warning threshold of the warning system.

[0051] To achieve the second above object, the present invention adopts the following technical solutions:

[0052] An abnormal state monitoring system for a thermal power plant based on deep learning, comprising: a data acquisition module, a multi-level feature extraction module, an abnormal state detection module, and an abnormal state warning module,

[0053] The data acquisition module is used to extract the image sequence frames X of the thermal power plant from the video;

[0054] The multi-level feature extraction module includes a multi-scale spatio-temporal feature extraction sub-module, a spatio-temporal feature enhancement sub-module, and a spatio-temporal feature attention fusion sub-module. The multi-scale spatio-temporal feature extraction sub-module is used to extract the image spatio-temporal features in the image sequence frames X of the thermal power plant;

[0055] The spatio-temporal feature enhancement sub-module is used to extract the temporal correlation and dependence features in the image sequence frames X of the thermal power plant and fuse them with the image spatio-temporal features to enhance the spatio-temporal features of the thermal power plant images;

[0056] The spatio-temporal feature attention fusion sub-module is used to extract the multi-level spatio-temporal features of the thermal power plant images with representativeness in the image sequence frames X of the thermal power plant;

[0057] The abnormal state detection module is used to predict the class labels of the multi-level spatio-temporal features of the thermal power plant images, construct a loss function to evaluate the difference between the prediction and the true label, perform abnormal state monitoring, and obtain abnormal state information;

[0058] The abnormal state warning module is used to build a multi-level warning system and trigger different levels of warnings according to the severity of the abnormal state.

[0059] Furthermore, it further includes a data preprocessing module, and the data preprocessing module is used to preprocess the thermal power plant image data.

[0060] Beneficial effects of the present invention:

[0061] This method constructs a corresponding data preprocessing scheme based on the characteristics of thermal power plant images, effectively improves the quality of thermal power plant images, and obtains representative abnormal image features of thermal power plants through a multi-level spatiotemporal feature extraction and fusion method of thermal power plant images based on a deep learning model, thereby achieving real-time monitoring of abnormal conditions such as coal blockage, foreign matter, and cracks in thermal power plants, and improving the safety of thermal power plant scenes. In addition, by constructing a multi-level early warning system, timely early warning of abnormal conditions at different levels is achieved. Therefore, the present invention has the advantages of high accuracy, real-time performance, and intelligence, and can significantly improve the safety management level of thermal power plants, reduce production risks, and ensure the efficient operation of thermal power plant production. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a flow chart of a specific embodiment 1 of the present invention;

[0063] Figure 2 This is a flow chart of feature extraction and abnormal state monitoring in specific embodiment 1 of the present invention;

[0064] Figure 3 This is a flowchart of preprocessing image data in specific embodiment 1 of the present invention;

[0065] Figure 4 This is a principle block diagram of specific embodiment 2 of the present invention. DETAILED DESCRIPTION

[0066] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Specific embodiment 1:

[0068] See also Figures 1 to 3 As shown, a method for monitoring abnormal conditions of a thermal power plant based on deep learning comprises the following steps:

[0069] Step S1: extracting a thermal power plant image sequence frame X from a video.

[0070] The step S1 also includes preprocessing the image data of the thermal power plant. Since the images in the special environment of the thermal power plant are affected by complex factors such as light, dust and smoke, the quality of the acquired thermal power plant images is poor, which affects the subsequent algorithm effect for the scene. Therefore, according to the characteristics of the special environment images of the thermal power plant, the relevant data is preprocessed and optimized to improve the image quality and lay the foundation for the subsequent related abnormal state monitoring. Specifically including:

[0071] S101: Denoising of Image Data from Thermal Power Plants

[0072] Since there is usually uneven illumination in the power plant, the image brightness is low and the details are unclear. Using Gaussian filtering can effectively remove the random noise in the thermal power plant image, improve the image quality, and make the effective features of the abnormal state of the thermal power plant easier to be extracted.

[0073] First, create a Gaussian kernel, then normalize all elements in the Gaussian kernel, and then apply the Gaussian kernel to each pixel in the image, that is, each pixel is weighted and summed with the Gaussian kernel elements at the corresponding position, so as to update each pixel in the image and achieve denoising of the thermal power plant image. For an N×N Gaussian kernel, the central element is located at the position of (0,0), and each element G(x, y) in the Gaussian kernel is obtained according to the following formula:

[0074]

[0075] where (x,y) are the pixel coordinates of the image, and σ is the standard deviation of the Gaussian kernel, which controls the width of the Gaussian function, that is, the degree of blurring of the filter. A larger σ value will produce a wider Gaussian curve, resulting in a stronger blurring effect; a smaller σ value will make the curve narrower and the blurring effect weaker.

[0076] S102: Size Adjustment and Normalization

[0077] The thermal power plant is a dynamic system, and there may be fast-moving objects in the image. Size adjustment and normalization can ensure that the image processing algorithm can adapt to dynamic scenes of different sizes and brightnesses, while retaining as much image detail as possible. Size adjustment can generally be achieved by bilinear interpolation. Normalization scales the pixel values of the image to a specified range, such as [0,1], to accelerate the convergence of the network and improve the numerical stability. The normalization formula is expressed as:

[0078]

[0079] S103: Grayscale Conversion

[0080] Due to the influence of light and environmental conditions, the color of the thermal power plant image may be distorted. Grayscale conversion can remove the interference of color information, focus on the brightness and contrast of the image, and help the extraction of subsequent target features and the recognition of abnormal states. Grayscale conversion is expressed as:

[0081] I gray (x,y) = w R × I R (x,y) + w G × I G (x,y) + w B × I B (x,y)

[0082] where w R = 0.2989, w G = 0.5870, w B = 0.1140, representing the sensitivity of the human eye to different colors. The human visual system is most sensitive to green light, followed by red light, and least sensitive to blue light. Therefore, in the calculation of grayscale images, the green channel contributes the most, and the red and blue channels contribute less. I R (x, y), I G (x, y) and I B (x, y) refer to the intensity values of the red, green, and blue channels of the pixel at the (x, y) position in the image.

[0083] S104: Edge detection

[0084] Since dust and smoke in the environment of thermal power plants can affect the clarity of thermal power plant images. Denoising techniques can help remove the interference of these particulate matters, while edge detection can highlight key structures in the image, such as the edge of the belt or the contour of the material. Here, the Sobel operator is used to enhance the edge detection of thermal power plant images.

[0085] The image data preprocessing method of the present invention aims at the influence of particulate noise, illumination, etc. on thermal power plant images, adopts an image denoising method based on Gaussian filtering, and in order to enhance the edge and contour information of the image, uses the Sobel operator to enhance the edge detection of thermal power plant images, constructs a more comprehensive and effective preprocessing scheme for thermal power plant image data, thereby improving the quality of the image.

[0086] Step S2: Construct a multi-level spatio-temporal feature extraction model for thermal power plant images, including a multi-scale spatio-temporal feature extraction sub-model, a spatio-temporal feature enhancement sub-model, and a spatio-temporal feature attention fusion sub-model;

[0087] S201: Extract the spatio-temporal features of the thermal power plant image sequence frames X by using the multi-scale spatio-temporal feature extraction sub-model, specifically including:

[0088] S2011: In order to extract rich spatio-temporal features from the thermal power plant video sequence, so as to deeply understand the dynamic thermal power plant images and identify abnormal states, first use a multi-scale convolutional layer to capture the effective spatio-temporal features of thermal power plant images at different scales, and improve the robustness to the features of abnormal substances of different sizes under different contrast and low illumination conditions. By using convolutional kernels K of different sizes in parallel sA 3D convolutional layer (where s ∈ {1, 2, …, S}) is used to obtain multi-scale image features. Small convolutional kernels are used to capture local detailed information of the thermal power plant images, and large convolutional kernels are used to capture global comprehensive information. The output features at each scale are represented as:

[0089] O s = Conv3D(X, K s ) + b s

[0090] where b s is the bias term of the convolutional layer at the corresponding scale in the multi-scale 3D convolutional network;

[0091] S2012: Multiple 3D convolutional layers in series are used to extract temporal and spatial features from the multi-scale convolutional output, obtaining temporal correlation context information and dependencies between video frames, and enhancing the capture of dynamic thermal power plant image features. For each scale of the 3D convolutional network layer, its output is represented as:

[0092]

[0093] where D is the size of the 3D convolutional kernel K 3D in the time dimension, H 3D and W 3D are its height and width, and b 3D is the bias term of the 3D convolutional network;

[0094] S2013: Dilated convolution is used to process O 3D by inserting spatial gaps in the convolutional kernel to increase the receptive field without increasing the number of parameters of the convolutional kernel:

[0095]

[0096] where H dilate and W dilate are the height and width of the dilated convolutional kernel K dilate , and b dilate is the bias term of the dilated convolutional network; It increases the spatial and temporal receptive fields of the network, enabling the model to learn more sufficient implicit knowledge of thermal power plant images.

[0097] S2014: The output feature maps of the 3D convolutional layers with different dilation factors and different scales after the dilated convolution processing are converted into attention calculation matrices Q, K, and V using a linear layer;

[0098] S2015: Fusion is performed through the attention mechanism to enhance feature representation and input it into the subsequent feature enhancement network;

[0099]

[0100] In this way, by combining dilated convolution with multi-scale 3D convolution, the network has a larger range of spatial and temporal receptive fields on the basis of extracting spatio-temporal features of thermoelectric power plant images at different scales, so as to be able to perceive richer features. This enables the network to capture rich spatio-temporal features of thermoelectric power plant images from local details to global context O, which is very effective for obtaining abnormal states in complex scenes of thermoelectric power plant monitoring, including but not limited to coal blockage, foreign objects in the wood remover, cracks in the L-shaped chain plates of the trough plate, etc.

[0101] Possibly, before the step S2014, the output feature maps of the 3D convolution layers with different dilation factors and different scales after being processed by dilated convolution in the step S2013 are non-linearly transformed through an activation function. In this specific embodiment: the activation function is the ReLU activation function, and dimensionality reduction processing is performed through a pooling layer. The pooling layer adopts max pooling or average pooling to reduce the computational amount and extract main features.

[0102] S202: Use the spatio-temporal feature enhancement sub-model to extract the temporal correlation and dependence features in the sequence of frames X of the thermoelectric power plant images, and fuse them with the spatio-temporal features of the images to enhance the spatio-temporal features of the thermoelectric power plant images, specifically including:

[0103] In order to make more full use of the temporal context knowledge in the video frames of the thermoelectric power plant, a spatio-temporal feature enhancement sub-model is constructed. This spatio-temporal feature enhancement sub-model uses a gated recurrent unit GRU to extract the temporal correlation and dependence features in the sequence, and fuses them with the above-mentioned features to enhance the spatio-temporal features of the thermoelectric power plant images, helping to identify abnormal states in the sequence to obtain the feature map O GRU , feature map O GRU already contains the long-term temporal dependence information of the relevant frames and multi-level spatial features.

[0104] Take the spatio-temporal feature O of each frame image in the multi-scale spatio-temporal feature extraction sub-model as the input x of the gated recurrent unit GRU t , and further enhance the relevant temporal knowledge contained in the target image of the thermoelectric power plant through the mining of temporal context information. This process is expressed as:

[0105] z t =σ(W z ·[h t-1 ,x t +b z )

[0106] r t =σ(W r ·[h t-1 ,x t +br )

[0107]

[0108] where r t is the reset gate, which is used to determine how much past information needs to be forgotten, and z t is the update gate, which is used to determine how much new information will be stored in the current state, is the candidate hidden state, h t is the hidden state, and the hidden state h t is transformed through a linear function for dimensional adjustment and other transformations to obtain the feature map O GRU .

[0109] Through the self-attention calculation of spatio-temporal features, the dependency information of the target image in space and time is mined, so as to obtain more representative spatio-temporal features, which helps to identify abnormal states in the sequence, such as sudden changes or intermittent problems in material flow.

[0110] S203: Use the spatio-temporal feature attention fusion sub-model to extract the multi-level spatio-temporal features of the thermal power plant images with representativeness in the image sequence frames X of the thermal power plant, specifically including:

[0111] To strengthen the attention and capture of abnormal features of different sizes and directions in thermal power plant images, such as cracks, wear or foreign objects, a spatio-temporal feature attention fusion sub-model is constructed. This spatio-temporal feature attention fusion sub-module combines the spatial attention mechanism and the feature pyramid, enabling the network to not only obtain important features at each scale, but also effectively fuse spatial features at different scales, thus providing rich and multi-scale feature representations for subsequent abnormal state monitoring. Further mine the temporal context correlation information and achieve the fusion of spatio-temporal features at different scales, so that the network can learn more accurate abnormal target knowledge in the image.

[0112] S2031: To focus on the important regions of the thermal power plant images, learn an attention weight and multiply it with the feature map O in S202 GRU to enhance the key features in the image and suppress the unimportant features:

[0113] A(x,y) = softmax(W·O GRU (x,y) + b)

[0114] F att (x,y) = A(x,y) ⊙ O GRU (x,y)

[0115] where A(x,y) is the attention weight at the spatial position (x,y), W and b are learnable parameters, ⊙ represents the dot product operation, and Fatt is the attention-weighted feature map.

[0116] S2032: To enhance the feature representation of the thermoelectric power plant images, the Feature Pyramid Network (FPN) is adopted to combine the attention feature maps F at different levels through top-down and lateral connections. k :

[0117]

[0118] Among them, k refers to the feature level, top-down is the process of combining features at different levels from top to bottom, and upsample refers to the upsampling operation. Through this process, spatial attention can be applied at each level of the Feature Pyramid Network (FPN) to enhance the feature maps at each scale. Then, the attention-weighted enhanced feature maps are fused again through the FPN structure to finally obtain a representative multi-scale feature representation.

[0119] S3: First, the multi-level spatio-temporal features are further extracted through multiple fully connected network layers and information fusion before classification is performed, integrating all features at different levels together, so that the features finally used for abnormal state monitoring contain information from different levels of the network, thus having stronger representativeness.

[0120] O fc = ReLU(W fc ·F k + b fc )

[0121] Among them, O fc is the output of the fully connected layer, W fc is the weight of the corresponding network layer, and b fc is the bias term corresponding to the fully connected layer.

[0122] Then, the softmax activation function is used to output the probability of each category, predicting the category label of the multi-level spatio-temporal features. Finally, a loss function is constructed to evaluate the difference between the prediction and the true label for abnormal state monitoring to obtain abnormal state information.

[0123] P = softmax(W P ·O fc + b P )

[0124] Among them, P is the probability vector output by the classification layer, W P is the bias of the corresponding network layer, and b P is the bias term corresponding to the network when calculating the classification probability.

[0125] If the samples of the categories are balanced, a weighted cross-entropy loss function is constructed to evaluate the difference between the prediction and the true label;

[0126] If the samples of some categories are unbalanced, the weighted cross-entropy loss function is replaced by assigning different weights to each category, so as to enhance the ability to identify abnormal states. The replaced loss function is defined as:

[0127]

[0128] where N is the number of categories, y c is the one-hot encoding of the true label, p c is the probability that the model detects category c, and w c is the weight of category c.

[0129] S4: Build a multi-level early warning system, and trigger early warnings at different levels according to the severity of the abnormal state information in step S3; specifically including:

[0130] S401: Set a confidence score for the detected abnormal state, expressed as:

[0131] P max = max(P)

[0132] where P is the probability vector output by the softmax layer, and P max is the maximum probability value, representing the confidence in predicting the abnormal state category;

[0133] S402: Set dynamic thresholds for each early warning level according to historical data and real-time monitoring requirements;

[0134] S403: Compare the predicted confidence score with the early warning threshold to determine the early warning level and achieve an adaptive and timely early warning Alert Level;

[0135]

[0136] where T high is the high early warning threshold of the early warning system, and T low is the low early warning threshold of the early warning system. The present invention can adaptively trigger early warnings at different levels according to the severity of the abnormal state detected by the network, so as to directly make a judgment according to the early warning situation and perform corresponding processing accordingly. Specific Embodiment 2:

[0138] See Figure 4As shown in the figure, an abnormal state monitoring system for a thermoelectric power plant based on deep learning includes: a data acquisition module 1, a multi-level feature extraction module, an abnormal state detection module 5, and an abnormal state warning module 6. The output end of the data acquisition module 1 is connected to the input end of the multi-level feature extraction module. The output end of the multi-level feature extraction module is connected to the input end of the abnormal state detection module 5. The output end of the abnormal state detection module 5 is connected to the input end of the abnormal state warning module 6.

[0139] The data acquisition module 1 is used to extract the image sequence frames X of the thermoelectric power plant from the video.

[0140] The multi-level feature extraction module includes a multi-scale spatio-temporal feature extraction sub-module 2, a spatio-temporal feature enhancement sub-module 3, and a spatio-temporal feature attention fusion sub-module 4.

[0141] The multi-scale spatio-temporal feature extraction sub-module 2 is used to extract the spatio-temporal features of the image in the image sequence frames X of the thermoelectric power plant.

[0142] The spatio-temporal feature enhancement sub-module 3 is used to extract the temporal correlation and dependence features in the image sequence frames X of the thermoelectric power plant, and fuse them with the spatio-temporal features of the image to enhance the spatio-temporal features of the thermoelectric power plant image.

[0143] The spatio-temporal feature attention fusion sub-module 4 is used to extract the multi-level spatio-temporal features of the thermoelectric power plant image with representativeness in the image sequence frames X of the thermoelectric power plant.

[0144] The abnormal state detection module 5 is used to monitor the abnormal state. First, the multi-level spatio-temporal features are further enhanced and the features are extracted through multiple fully connected network layers and information fusion before classification. Then, the softmax activation function is used to output the probability of each category. Finally, a weighted cross-entropy loss function is constructed to evaluate the difference between the prediction and the true label, and the abnormal state information is obtained.

[0145] The abnormal state warning module 6 is used to build a multi-level warning system and trigger different levels of warnings according to the severity of the abnormal state.

[0146] Possibly, it further includes a data preprocessing module 7. The data preprocessing module 7 is connected between the data acquisition module 1 and the multi-level feature extraction module and is used to preprocess the thermoelectric power plant image data.

[0147] The specific functions, functional features of each module are the same as those in the specific embodiment 1, so the specific embodiment is omitted here.

[0148] The above has introduced the technical solution provided by the present invention in detail. Specific examples are used herein to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A method for monitoring abnormal conditions of a thermal power plant based on deep learning, characterized in that: The steps include: S1: extracting thermal power plant image sequence frames X from the video; S2: Construct a multi-level spatiotemporal feature extraction model for thermal power plant images, including a multi-scale spatiotemporal feature extraction sub-model, a spatiotemporal feature enhancement sub-model, and a spatiotemporal feature attention fusion sub-model; including: S201: extracting image spatiotemporal features in the thermal power plant image sequence frame X using the multi-scale spatiotemporal feature extraction sub-model; S202: extracting the temporal correlation and dependency features in the thermal power plant image sequence frame X by using the temporal and spatial feature enhancement sub-model, and fusing them with the image temporal and spatial features to enhance the temporal and spatial features of the thermal power plant image; specifically including: The gated recurrent unit GRU is used to extract the temporal correlation and dependency features in the sequence, and is fused with the image spatiotemporal features O to enhance the spatiotemporal features of the thermal power plant image, identify abnormal states in the sequence, and obtain the feature map O. GRU , the feature map O GRU The long-term temporal dependency information of related frames and multi-level spatial features are already included; S203: extracting representative multi-level spatiotemporal features of the thermal power plant image in the thermal power plant image sequence frame X by using the spatiotemporal feature attention fusion sub-model; specifically comprising: S2031: Focus on the important area of ​​the thermal power plant image, learn an attention weight, and compare it with the feature map O GRU Multiply to enhance key features in the image and suppress unimportant features: A(x,y)=softmax(W·O GRU (x,y)+b) F att (x,y)=A(x,y)⊙O GRU (x,y) Among them, A(x,y) is the attention weight of the spatial position (x,y), O GRU is the feature map, W and b are learnable parameters, ⊙ represents the dot product operation, F att is the feature map weighted by attention; S2032: In order to enhance the feature representation of thermal power plant images, a feature pyramid network FPN is used to combine different levels of attention feature maps F through top-down and lateral connections. k , and obtain a representative multi-scale feature representation; S3: predicting the category labels of the multi-level spatiotemporal features of the thermal power plant image, constructing a loss function to evaluate the difference between the prediction and the true label, performing abnormal state monitoring, and obtaining abnormal state information; S4: Build a multi-level warning system to trigger different levels of warnings according to the severity of the abnormal status information in step S3.

2. The method for monitoring abnormal state of a thermal power plant based on deep learning according to claim 1, characterized in that: The step S1 also includes preprocessing the thermal power plant image data.

3. The method for monitoring abnormal state of a thermal power plant based on deep learning according to claim 2 is characterized in that: The preprocessing of the thermal power plant image data specifically includes: performing Gaussian kernel denoising, size adjustment and normalization, grayscale conversion and edge detection processing on the thermal power plant image data.

4. The method for monitoring abnormal state of a thermal power plant based on deep learning according to claim 1, characterized in that: The S201: a multi-scale spatiotemporal feature extraction sub-model extracts the image spatiotemporal features in the thermal power plant image sequence frame X; Specifically include: S2011: Multi-scale convolutional layers are used to capture the spatiotemporal features of effective thermal power plant images at different scales, by using convolution kernels of different sizes K in parallel. s The 3D convolution layer is used to obtain multi-scale image features, where small convolution kernels are used to capture local detail information of the thermal power plant image, and large convolution kernels are used to capture global comprehensive information. The output features of each scale are expressed as: THE s =Conv3D(X,K s )+b s where b s It is the bias term of the convolution layer of the corresponding scale in the multi-scale 3D convolutional network; S2012: Use multiple 3D convolutional layers in series to extract temporal and spatial features from multi-scale convolutional outputs to obtain temporal contextual information and dependencies between video frames; S2013: Using dilated convolution to 3D Processing is performed to increase the receptive field by inserting spatial intervals in the convolution kernel: S2014: The 3D convolution layer output feature maps with different dilation factors and different scales after the dilation convolution processing are converted into attention calculation matrices Q, K and V using a linear layer; S2015: Fusion through attention mechanism, enhanced feature representation, and input into subsequent feature enhancement network; The network captures rich image spatiotemporal features from local details to global contexts in thermal power plant images.

5. The method for monitoring abnormal state of a thermal power plant based on deep learning according to claim 1, characterized in that: In step S3, a loss function is constructed to evaluate the difference between the prediction and the true label, and abnormal state monitoring is performed, specifically including: If the samples of the categories are balanced, a weighted cross entropy loss function is constructed to evaluate the difference between the prediction and the true label; If the samples of some categories are unbalanced, the weighted cross entropy loss function is replaced by assigning different weights to each category. The loss function after replacement is: Where N is the number of categories, y c is the one-hot encoding of the true label, p c is the probability of the model detecting category c, w c is the weight of category c.

6. The method for monitoring abnormal state of a thermal power plant based on deep learning according to claim 1, characterized in that: The step S4: building a multi-level warning system to trigger different levels of warnings according to the severity of the abnormal status information in the step 3, specifically including: S401: Set a confidence score for the detected abnormal state, expressed as: P max =max(P) Among them, P is the probability vector output by the softmax layer, P max is the maximum probability value, representing the confidence in the predicted abnormal state category; S402: Setting dynamic thresholds for each warning level based on historical data and real-time monitoring requirements; S403: Compare the predicted confidence score with the warning threshold, determine the warning level, and implement adaptive and timely warning; Among them, T high is the high warning threshold of the early warning system, T low It is the low warning threshold of the early warning system.

7. A thermal power plant abnormal state monitoring system based on deep learning, characterized in that: include: Data acquisition module, multi-level feature extraction module, abnormal state detection module and abnormal state warning module, The data acquisition module is used to extract the thermal power plant image sequence frame X from the video; The multi-level feature extraction module includes a multi-scale spatiotemporal feature extraction submodule, a spatiotemporal feature enhancement submodule, and a spatiotemporal feature attention fusion submodule, wherein the multi-scale spatiotemporal feature extraction submodule is used to extract the image spatiotemporal features in the image sequence frame X of the thermal power plant; The spatiotemporal feature enhancement submodule is used to extract the temporal correlation and dependency features in the thermal power plant image sequence frame X, and fuse them with the image spatiotemporal features to enhance the spatiotemporal features of the thermal power plant image; Specifically include: The gated recurrent unit GRU is used to extract the temporal correlation and dependency features in the sequence, and is fused with the image spatiotemporal features O to enhance the spatiotemporal features of the thermal power plant image, identify abnormal states in the sequence, and obtain the feature map O. GRU , the feature map O GRU The long-term temporal dependency information of related frames and multi-level spatial features are already included; The spatiotemporal feature attention fusion submodule is used to extract representative multi-level spatiotemporal features of the thermal power plant image in the thermal power plant image sequence frame X; specifically includes: S2031: Focus on the important area of ​​the thermal power plant image, learn an attention weight, and compare it with the feature map O GRU Multiply to enhance key features in the image and suppress unimportant features: A(x,y)=softmax(W·O GRU (x,y)+b) F att (x,y)=A(x,y)⊙O GRU (x,y) Among them, A(x,y) is the attention weight of the spatial position (x,y), O GRU is the feature map, W and b are learnable parameters, ⊙ represents the dot product operation, F att is the feature map weighted by attention; S2032: In order to enhance the feature representation of thermal power plant images, a feature pyramid network FPN is used to combine different levels of attention feature maps F through top-down and lateral connections. k , and obtain a representative multi-scale feature representation; The abnormal state detection module is used to predict the category labels of the multi-level spatiotemporal features of the thermal power plant image, construct a loss function to evaluate the difference between the prediction and the true label, perform abnormal state monitoring, and obtain abnormal state information; The abnormal state warning module is used to build a multi-level warning system to trigger warnings of different levels according to the severity of the abnormal state.

8. The thermal power plant abnormal state monitoring system based on deep learning according to claim 7 is characterized in that: It also includes a data preprocessing module, which is used to preprocess the image data of the thermal power plant.

Images