A method for detecting smoke and fire in a thermal power plant based on inter-frame difference method and improved YOLOv4

Abstract

The application claims a thermal power plant smoke and fire detection method based on interframe difference method and improved YOLOv4. First, the main network of the detection model is constructed by parallel DFC attention and residual, and the performance decline caused by a single detection head is jointly made up by using expansion convolution and receptive field supplement block to construct GS-YOLO model. Then, the video frames of smoke and flame accidents in thermal power plants are labeled to obtain a classification labeled data set and data enhancement. Then, GS-YOLO is trained for detecting smoke and fire, and attack noise is added to the network, which will help the network to distinguish the background and the detection target without changing the optimization target of the model. The test set with real labels is used to verify the target detection model, and the model with high detection performance is obtained. In actual detection, the moving part in the video is obtained by interframe difference method, and the irrelevant background image of the input model is reduced. The method has excellent smoke and fire detection performance and fast inference speed.

Description

Technical Field

[0001] This invention belongs to the field of video-based target detection, specifically a method for detecting smoke and fire in thermal power plants based on the inter-frame difference method and an improved YOLOv4. Background Technology

[0002] In recent years, convolutional neural networks have gradually become a mainstream deep learning model architecture and have become the most effective method in the fields of image processing and computer vision. The two major characteristics of weight sharing and local receptive fields reduce the number of weights, thereby reducing the computational complexity of the model; the translation invariance of image features also gives it good feature extraction capabilities and high stability.

[0003] Currently, a large number of studies have explored object detection methods using convolutional neural networks. Since R. Girshick et al. proposed the R-CNN deep learning model based on candidate regions in 2014, a number of classic two-stage object detection algorithms have emerged, such as Fast R-CNN and Faster R-CNN. In addition, one-stage detection methods, represented by YOLO, commonly seen in YOLOv3, YOLOv4, and the current DAMO-YOLO, attempt to balance accuracy and speed. Of course, the detection performance has been improved through various tricks, inevitably leading to an increase in network parameters. Generally, one-stage methods are inferior to two-stage methods in terms of accuracy. The development trend of detection models is generally to continuously deepen the number of network layers to obtain better feature extraction capabilities and to continuously increase the image scale to cover a wider range of features. However, the increasing complexity of the models also brings problems such as difficulty in network convergence, rapid parameter growth, and slow computation speed. The most significant problem is that with the increase in model complexity, it is difficult to deploy on resource-constrained devices. Therefore, some lightweight methods have been proposed, such as depthwise separable convolution, ShuffleNet, GhostNet, and distillation pruning techniques, to reduce the complexity of the network for easier deployment.

[0004] However, current detection networks are also vulnerable. Their performance can easily degrade significantly when subjected to external attacks. Common attack methods include Fast Gradient Sign Method (FGSD), Iterative Gradient Sign Method (IFGSD), Deepfool, and Optimization-based methods, which can lead to a decrease in classification scores or even misclassification. However, attacks can also improve network performance. Without changing the model's expected value, the addition of noise can help the network better distinguish between targets and the environment. This is often used for detection in harsh environments, such as low light, foggy weather, and underwater environments.

[0005] Meanwhile, at thermal power plant sites, various equipment faces high temperatures and is prone to insufficient heat dissipation, leading to overheating and accidents. Furthermore, employees may become less focused in high-temperature environments, resulting in operational errors and accidents. Currently, the supervision of smoke and fire detection at construction sites follows two approaches: one relies primarily on manual supervision, with dedicated personnel patrolling and inspecting, which is inefficient and labor-intensive; the other utilizes deep learning to build detection networks, but existing smoke and fire detection network models suffer from low accuracy, slow inference speed, and insufficient precision and real-time performance when deployed to edge computing devices. Therefore, utilizing lightweight end-to-end detection networks is the optimal choice for smoke and fire detection.

[0006] Therefore, a method for detecting smoke and fire in thermal power plants based on inter-frame difference and improved YOLOv4 is needed. Inter-frame difference alleviates the problem of detecting smoke and flames as small targets. Secondly, noise attack enables the network to better distinguish between smoke, fire and background, thereby reducing missed detections and false detections. Structurally, ResGhost is used to reduce network parameters and improve detection speed. Expanded convolution and receptive field supplement blocks are used in combination to increase the receptive field of a single detection head, enabling real-time detection of smoke and fire incidents and timely alarms.

[0007] Patent CN114821414A uses deformable convolution to enhance the detection capability of irregular smoke and fire, while area changes are used to refine the categories. However, the problem is that it adds parameters for learning the offset, which makes it difficult to handle large input images. It still suffers from slow inference speed and missed detection of small target smoke and fire.

[0008] This invention aims to improve inference speed and reduce missed detections of small targets. The difference between this invention and patent CN114821414A lies in the following: In terms of network structure, this invention uses parallel DFC to reduce network parameters and feature redundancy, further reducing the size and number of features for classification and regression, thus improving inference speed. To compensate for the decrease in accuracy caused by the receptive field, dilated convolution and receptive field supplementation blocks are employed. Patent CN114821414A, on the other hand, uses deformable convolution to enhance the detection of irregular fireworks; the network structure and starting point are different. Regarding small target detection, this invention uses inter-frame differencing to further crop and filter out changing fireworks regions, reducing irrelevant content input to the network, interpolating to a specified size, and increasing the proportion of small target fireworks in the input image. Patent CN114821414A does not include this aspect of the operation. In terms of category refinement, this invention uses noise attack to adjust the data distribution of the image, improving the distinction between fireworks and the background, thereby improving detection accuracy, while patent CN114821414A uses area change frequency for category refinement. Summary of the Invention

[0009] This invention aims to solve the problems of the prior art. It proposes a method for detecting smoke and fire in thermal power plants based on the inter-frame difference method and an improved YOLOv4. The technical solution of this invention is as follows:

[0010] A method for detecting smoke and fire in thermal power plants based on inter-frame difference and improved YOLOv4 includes the following steps:

[0011] Step 1: Obtain monitoring videos of smoke and fire accidents from the intelligent management and control system of the thermal power plant, label the areas where smoke and flames occur in the videos, use them as the training set for the detection network and perform data augmentation, and name the constructed dataset the smoke and fire detection dataset.

[0012] Step 2: Based on the YOLOv4 algorithm, the following improvements are used to construct the GS-YOLO detection model: 1) Construct a ResGhost module with parallel DFC attention and residuals, and use multiple ResGhosts to form the backbone network of the GS-YOLO detection model; 2) Convert YOLOv4 into a single detection head, and combine dilated convolutions with different dilation rates and receptive field enhancement blocks; At the same time, use the fireworks detection dataset from Step 1 to train GS-YOLO to detect whether fireworks accidents have occurred in surveillance videos;

[0013] Step 3: Add learnable attack noise to the GS-YOLO model trained in Step 2;

[0014] Step 4: Use the inter-frame difference method to obtain the approximate location of the flame and smoke in the video to be detected, crop it with a specific aspect ratio, name it Target-area, and use it as input for GS-YOLO;

[0015] Step 5: Input the Target-area from Step 4 into the GS-YOLO trained in Step 2 to detect smoke and flames in the surveillance video;

[0016] Step 6: Based on the detection results in Step 5, take appropriate action to issue an alarm for situations involving smoke and flames, thereby achieving timely early warning of smoke and fire accidents in thermal power plants.

[0017] Step 2 specifically involves: using GhostNetv2 as a foundation, constructing parallel attention modules using DFC attention modules with different downsampling rates, enabling the network to pay attention to features at different sizes; simultaneously constructing a ResGhost module based on a residual network, and using multiple ResGhost modules to form the backbone feature extraction network of GS-YOLO; abandoning the multi-scale detection of the FPN in the YOLO series, instead using a single detection head, i.e., the second largest feature C1 in the FPN, for prediction; to compensate for the reduction in receptive field caused by a single scale, five dilated convolutions are used in the detection head to increase the receptive field, with dilation coefficients of 2, 4, 6, 8, and 10 for each layer, and connected by a residual to combine the two different receptive fields; at the same time, a receptive field enhancement block is built at a lower layer, and the receptive fields of the dilated convolutions and the receptive field enhancement block are fused for classification; the GS-YOLO network optimizes localization loss, confidence loss, and classification loss until the model converges, saves the weights of the model with the best classification accuracy on the test set, and uses the Adam optimization algorithm of adaptive moments estimation to update the model weights.

[0018] Furthermore, step 1 specifically involves: traversing each frame of the surveillance video and saving it as an image, labeling the images containing smoke and fire, then performing enhancement methods including flipping, rotating, and adjusting brightness, and finally using Mosaic and MixUp data augmentation techniques to enrich the background and data diversity of the samples to construct a smoke and fire detection dataset.

[0019] Furthermore, the localization loss uses CIou as the loss function, the confidence loss is calculated by weighted summation of the binary cross-entropy loss for whether the target is in the prediction box, and the classification loss is the sum of the binary cross-entropy losses for each category.

[0020] ;

[0021] in, For location loss; B, B gt B represents the predicted bounding box and the ground truth bounding box. ctr , These represent the center points of the predicted bounding box and the ground truth bounding box, respectively. This represents the calculation of the Euclidean distance between two center points; w gt h gt Here, w and h represent the width and height of the target bounding box, respectively, while w and h represent the width and height of the predicted bounding box. The weight representing the consistency of aspect ratio between the ground truth bounding box and the predicted bounding box, the Intersection over Union (IOU) is the intersection-union ratio between the ground truth bounding box and the predicted bounding box, v is a parameter that measures the consistency of aspect ratio between the ground truth bounding box and the predicted bounding box, and c represents the diagonal distance of the smallest closure region that can simultaneously contain both the predicted bounding box and the ground truth bounding box.

[0022] ;

[0023] in, This represents the binary cross-entropy loss; y is the binary label 0 or 1, and p(y) is the probability that the output belongs to the label y;

[0024] ;

[0025] in Represents the classification loss function; This represents the confidence loss, where pred is the predicted value, C is the number of categories, and mask is the confidence score. Indicates the confidence level with a target; Indicates the confidence level without a target. The weight of the loss when there is no target in the detection box is reduced, thus reducing the proportion of loss when there is no target in the detection box.

[0026] Furthermore, step 3: adding learnable attack noise to the trained GS-YOLO model from step 2, specifically:

[0027] Learnable attack noise is added to the GS-YOLO model to disrupt the original image data distribution. The added noise is indistinguishable to the human eye, but does not modify the optimization objective of the detection network. Through continuous iteration, the addition of attack noise further satisfies the classifier's requirement to correctly identify smoke and fire, distinguish smoke from the background, and improve the confidence of classification.

[0028] Furthermore, step 4 specifically involves:

[0029] Inter-frame differencing involves subtracting the values ​​from two frames of a video. Frames with a non-zero difference are considered moving objects in the video, including fireworks. Inter-frame differencing can approximate the location of fireworks in the video being detected. GS-YOLO uses 1024 frames. 2 The size of the cropping window, i.e., the Target-area, is less than 1024. 2 The resolution is then upsampled to 1024 using padding and interpolation. 2 The purpose of truncating the size is to avoid the fireworks having too small a proportion in the input, causing the fireworks features to disappear during multiple convolutions.

[0030] The advantages and beneficial effects of this invention are as follows:

[0031] The current problems with smoke detection are: (1) difficulty in real-time detection on mobile devices; (2) detection of small targets; and (3) smoke is not easily distinguishable from the background. To address these problems, firstly, to improve inference speed and make the backbone network lightweight, the redundancy of features in traditional convolution is fully considered. Since feature maps have low-rank characteristics, calculating features in the horizontal and vertical directions of DFC is relatively inexpensive. However, DFC feature calculation is closely related to the downsampling rate, and a single downsampling rate results in information loss. Therefore, parallel DFCs with different downsampling are used to supplement the features. Furthermore, traditional FPN often requires multi-layer feature fusion, which actually causes a large amount of feature redundancy and even feature aliasing, resulting in some computational waste. Therefore, the multiple inputs of FPN are reduced to one, improving the network's effective expression. Secondly, to avoid the disappearance of features of small targets due to an overly deep network, using a moderately sized feature for regression classification is an effective approach. The method compensates for the loss of receptive field caused by single feature blocks by expanding convolution and receptive field. In addition, the approximate location of flames and smoke in the video to be detected is obtained by inter-frame difference method, which reduces the redundant computation of background in the network and avoids the degree to which smoke and flames are detected as small targets, effectively avoiding missed detections and false detections. The shape of smoke can lead to missed detections or false detections. Similar to the detection of harsh environments, attack noise is added to change the data distribution of video frames. At the same time, the optimization objective of the detection network is not modified, that is, minimizing the localization loss, confidence loss and classification loss. Through continuous iteration until the model converges, the addition of attack noise is beneficial to the network classification, improves the classification confidence, avoids missed detections and false detections caused by low distinction between smoke and background, and improves the detection performance and robustness.

[0032] Ultimately, with the help of a lightweight backbone network and a single detection head, a lightweight and high-performance smoke and fire target detection model was trained. It can directly detect targets in input videos and has a fast inference speed, which greatly reduces the difficulty of deployment to hardware and reduces the labor cost of monitoring smoke and fire accidents in thermal power plants. Attached Figure Description

[0033] Figure 1 This is a flowchart of a preferred embodiment of the present invention for detecting smoke and fire in thermal power plants based on the inter-frame difference method. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and thoroughly described below with reference to the accompanying drawings. The described embodiments are merely some embodiments of the present invention.

[0035] The technical solution of the present invention to solve the above-mentioned technical problems is:

[0036] As shown in the figure, the smoke and fire detection method for thermal power plants based on the inter-frame difference method provided in this embodiment includes the following steps:

[0037] Step 1: Obtain monitoring videos of smoke and fire accidents from the intelligent management and control system of the thermal power plant, label the areas where smoke and flames occur in the videos, use them as the training set for the detection network and perform data augmentation, and name the constructed dataset the Smoke-Fire detection dataset.

[0038] Step 2: Based on the YOLOv4 algorithm, the following improvements are used to construct the GS-YOLO detection model. (1) The ResGhost module is constructed with parallel DFC attention and residuals to improve the expressive power of the model. Multiple ResGhosts are used to form the backbone network of the GS-YOLO detection model. (2) YOLOv4 is converted into a single detection head, and the performance degradation caused by the combined use of dilated convolutions with different dilation rates and receptive field supplementation blocks is compensated. At the same time, the Smoke-Firedetection dataset in Step 1 is used to train GS-YOLO to detect whether a fire accident has occurred in the surveillance video.

[0039] Step 3: Use the input of the trained GS-YOLO from Step 2 to add learnable attack noise, so that the detection network can be better applied to the environment of thermal power plants and the scene of changing lighting, thereby improving the robustness of the detection model.

[0040] Step 4: Use the inter-frame difference method to obtain the approximate location of the flames and smoke in the video to be detected, crop it with a specific aspect ratio, name it Target-area, and use it as input for GS-YOLO, aiming to reduce the degree to which smoke and flames become small targets;

[0041] Step 5: Input the Target-area from Step 4 into the GS-YOLO trained in Step 3 to detect smoke and flames in the surveillance video.

[0042] Step 6: Based on the detection results in Step 5, take appropriate action to issue an alarm for situations involving smoke and flames, thereby achieving timely early warning of smoke and fire accidents in thermal power plants.

[0043] Preferably, step 1 involves acquiring monitoring videos of smoke and fire accidents occurring in the intelligent control system of a thermal power plant, labeling the areas in the videos where smoke and flames occur, using these areas as the training set for the detection network, and performing data augmentation. The constructed dataset is named the Smoke-Fire detection dataset. Specifically, this involves traversing each frame of the monitoring video and saving it as an image, labeling the images containing smoke and fire, performing conventional data augmentation operations such as flipping, rotating, and adjusting brightness, and then using Mosaic and MixUp data augmentation techniques to enrich the background and data diversity of the samples before constructing the Smoke-Fire detection dataset.

[0044] Preferably, in step 2: based on the YOLOv4 algorithm, the following improvements are used to construct the GS-YOLO detection model. (1) A ResGhost module is constructed using parallel DFC attention and residuals to improve the model's expressive power, and multiple ResGhosts form the backbone network of the GS-YOLO detection model; (2) YOLOv4 is converted into a single detection head, and the performance degradation caused by the combined use of dilated convolutions with different dilation rates and receptive field supplementary blocks is compensated. At the same time, the Smoke-Fire detection dataset in step 1 is used to train GS-YOLO to detect whether a smoke or fire accident has occurred in the surveillance video, specifically:

[0045] Based on GhostNetv2, parallel attention modules are constructed using DFC attention modules with different downsampling rates, enabling the network to pay attention to features at different sizes. Simultaneously, a ResGhost module based on a residual network is built, and multiple ResGhost modules form the backbone feature extraction network of GS-YOLO. To avoid the loss of small target features due to smaller feature maps generated by deeper networks, the multi-scale detection of the FPN in the YOLO series is abandoned. Instead, the traditional feature pyramid is replaced with a single detection head, using the second largest feature in the FPN (denoted as C1) for prediction. To compensate for the reduced receptive field caused by a single scale, five dilated convolutions are used in the detection head to increase the receptive field, with dilation coefficients of 2, 4, 6, 8, and 10 in each layer, connected by a residual to combine the two different receptive fields. A receptive field supplement block is also built at a lower layer, fusing the receptive fields of the dilated convolutions and the supplement block for classification. The GS-YOLO network optimizes localization loss, confidence loss, and classification loss until the model converges. It saves the weights of the model with the best classification accuracy on the test set and uses the Adam algorithm (Adaptive Moments Estimation) to update the model weights. The localization loss uses CIou as the loss function, while the others use binary cross-entropy as the loss function.

[0046]

[0047] Where B, B gt B represents the predicted bounding box and the ground truth bounding box. ctr , These represent the center points of the predicted bounding box and the ground truth bounding box, respectively. This represents the calculation of the Euclidean distance between two center points; w gt h gt Here, w and h represent the width and height of the target bounding box, respectively, while w and h represent the width and height of the predicted bounding box. IOU and v represent the weights for the consistency of aspect ratio between the ground truth box and the predicted box, respectively. IOU is the intersection-union ratio between the ground truth box and the predicted box, v is a parameter that measures the consistency of aspect ratio between the ground truth box and the predicted box, and c represents the diagonal distance of the smallest closure region that can simultaneously contain the predicted box and the ground truth box.

[0048]

[0049] Where y is a binary label 0 or 1, and p(y) is the probability that the output belongs to the label y.

[0050]

[0051] Where pred is the predicted value, C is the number of categories, and mask is the confidence score (1 if the bounding box contains an object, 0 otherwise). The weight of the loss when there is no target in the detection box is reduced, thus reducing the proportion of loss when there is no target in the detection box.

[0052] Preferably, step 3 involves adding learnable attack noise to the input of the GS-YOLO trained in step 2, making the detection network better applicable to power plant environments and scenarios with changing lighting, thereby improving the robustness of the detection model. Specifically:

[0053] Learnable attack noise is added to the input of GS-YOLO to disrupt the original image data distribution. The added noise can be indistinguishable to the human eye, but it does not modify the optimization objective of the detection network. Through continuous iteration, the addition of attack noise further satisfies the classifier's requirement to correctly identify smoke and fire, especially in distinguishing smoke from the background, thus improving the classification confidence. At the same time, it avoids false negatives and false positives caused by harsh environments such as low light, improving detection performance and robustness.

[0054] Preferably, step 4: obtaining the approximate location of the flame and smoke in the video to be detected using the inter-frame difference method, cropping it with a specific aspect ratio, naming it the Target-area, and using it as input to GS-YOLO, aims to reduce the degree to which smoke and flame become small targets, specifically:

[0055] Inter-frame difference (IF) involves subtracting the values ​​of two frames in a video. Frames with a non-zero difference are considered moving objects in the video, including fireworks. This effectively reduces the computational burden of detecting objects against a static background. IF can approximate the location of fireworks in the video and is adaptable to the aspect ratio of the network input, such as 1:1 (GS-YOLO uses 1024). 2 The size of the cropping window, i.e., the Target-area, is less than 1024. 2 The resolution is then upsampled to 1024 using padding and interpolation. 2 The purpose of truncating the size is to prevent the fireworks from having too small a proportion in the input, causing the fireworks features to disappear during multiple convolutions, thus effectively avoiding missed detections and false detections.

[0056] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0057] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0058] The above embodiments should be understood as illustrative only and not as limiting the scope of protection of the present invention. After reading the description of the present invention, those skilled in the art can make various alterations or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A method for detecting smoke and fire in thermal power plants based on inter-frame difference and improved YOLOv4, characterized in that, Includes the following steps: Step 1: Obtain monitoring videos of smoke and fire accidents from the intelligent management and control system of the thermal power plant, label the areas where smoke and flames occur in the videos, use them as the training set for the detection network and perform data augmentation, and name the constructed dataset the smoke and fire detection dataset. Step 2: Based on the YOLOv4 algorithm, the following improvements are used to construct the GS-YOLO detection model: 1) Construct a ResGhost module with parallel DFC attention and residuals, and use multiple ResGhosts to form the backbone network of the GS-YOLO detection model; 2) Convert YOLOv4 into a single detection head, and combine dilated convolutions with different dilation rates and receptive field enhancement blocks; At the same time, use the fireworks detection dataset from Step 1 to train GS-YOLO to detect whether fireworks accidents have occurred in surveillance videos; Step 3: Add learnable attack noise to the GS-YOLO model trained in Step 2; Step 4: Use the inter-frame difference method to obtain the approximate location of the flame and smoke in the video to be detected, crop it with a specific aspect ratio, name it Target-area, and use it as input for GS-YOLO; Step 5: Input the Target-area from Step 4 into the GS-YOLO trained in Step 2 to detect smoke and flames in the surveillance video; Step 6: Based on the detection results in Step 5, take appropriate action to issue an alarm for situations involving smoke and flames, thereby achieving timely early warning of smoke and fire accidents in thermal power plants. Step 2 specifically involves: using GhostNetv2 as a foundation, constructing parallel attention modules using DFC attention modules with different downsampling rates, enabling the network to pay attention to features at different sizes; simultaneously constructing a ResGhost module based on a residual network, and using multiple ResGhost modules to form the backbone feature extraction network of GS-YOLO; abandoning the multi-scale detection of the FPN in the YOLO series, instead using a single detection head, i.e., the second largest feature C1 in the FPN, for prediction; to compensate for the reduction in receptive field caused by a single scale, five dilated convolutions are used in the detection head to increase the receptive field, with dilation coefficients of 2, 4, 6, 8, and 10 for each layer, and connected by a residual to combine the two different receptive fields; at the same time, a receptive field enhancement block is built at a lower layer, and the receptive fields of the dilated convolutions and the receptive field enhancement block are fused for classification; the GS-YOLO network optimizes localization loss, confidence loss, and classification loss until the model converges, saves the weights of the model with the best classification accuracy on the test set, and uses the Adam optimization algorithm of adaptive moments estimation to update the model weights.

2. The method for detecting smoke and fire in thermal power plants based on inter-frame difference and improved YOLOv4 according to claim 1, characterized in that, Step 1 specifically involves: traversing each frame of the surveillance video and saving it as an image, labeling the images containing smoke and fire, then performing enhancement methods including flipping, rotating, and adjusting brightness, and finally using Mosaic and MixUp data augmentation techniques to enrich the background and data diversity of the samples to construct a smoke and fire detection dataset.

3. The method for detecting smoke and fire in thermal power plants based on inter-frame difference and improved YOLOv4 according to claim 1, characterized in that, The localization loss uses CIou as the loss function, the confidence loss is calculated by weighted summation of the binary cross-entropy loss for whether the target is in the prediction box, and the classification loss is the sum of the binary cross-entropy losses for each category. ； in, For location loss; B, B gt B represents the predicted bounding box and the ground truth bounding box. ctr , These represent the center points of the predicted bounding box and the ground truth bounding box, respectively. This represents the calculation of the Euclidean distance between two center points; w gt h gt Here, w and h represent the width and height of the target bounding box, respectively, while w and h represent the width and height of the predicted bounding box. The weight representing the consistency of aspect ratio between the ground truth bounding box and the predicted bounding box, the Intersection over Union (IOU) is the intersection-union ratio between the ground truth bounding box and the predicted bounding box, v is a parameter that measures the consistency of aspect ratio between the ground truth bounding box and the predicted bounding box, and c represents the diagonal distance of the smallest closure region that can simultaneously contain both the predicted bounding box and the ground truth bounding box. ； in, This represents the binary cross-entropy loss; y is the binary label 0 or 1, and p(y) is the probability that the output belongs to the label y; ； in Represents the classification loss function; This represents the confidence loss, where pred is the predicted value, C is the number of categories, and mask is the confidence score. Indicates the confidence level with a target; Indicates the confidence level without a target. The weight of the loss when there is no target in the detection box is reduced, thus reducing the proportion of loss when there is no target in the detection box.

4. The method for detecting smoke and fire in thermal power plants based on inter-frame difference and improved YOLOv4 according to claim 1, characterized in that, Step 3: Add learnable attack noise to the trained GS-YOLO model from Step 2, specifically as follows: Learnable attack noise is added to the GS-YOLO model to disrupt the original image data distribution. The added noise is indistinguishable to the human eye, but does not modify the optimization objective of the detection network. Through continuous iteration, the addition of attack noise further satisfies the classifier's requirement to correctly identify smoke and fire, distinguish smoke from the background, and improve the confidence of classification.

5. The method for detecting smoke and fire in thermal power plants based on inter-frame difference and improved YOLOv4 according to claim 4, characterized in that, Step 4 specifically involves: Inter-frame differencing involves subtracting the values ​​from two frames of a video. Frames with a non-zero difference are considered moving objects in the video, including fireworks. Inter-frame differencing can approximate the location of fireworks in the video being detected. GS-YOLO uses 1024 frames. 2 The size of the cropping window, i.e., the Target-area, is less than 1024. 2 The resolution is then upsampled to 1024 using padding and interpolation. 2 The purpose of truncating the size is to avoid the fireworks having too small a proportion in the input, causing the fireworks features to disappear during multiple convolutions.

Images