Smoke recognition and false alarm elimination method based on smoke color and negative sample similarity

Abstract

The application discloses a smoke identification and false alarm elimination method based on smoke color and negative sample similarity, which firstly trains an instance segmentation model by using a finely labeled data set, recognizes an input image, and obtains the category and contour of a target such as smoke and a chimney. Then, multi-level false alarm filtering is carried out: firstly, it is judged whether the smoke is discharged from the chimney, and the spatial position and shape correlation degree of the two are calculated for filtering; then, for the easily confused targets such as white smoke, the image features are extracted, and the similarity comparison is carried out with the historical negative sample features at the same point, and if the similarity is high, it is determined as normal interference such as fog and filtered. The application introduces multi-dimensional labels and space-time context information, effectively solves the problem of high false alarm rate of chimney exhaust, fog and cloud in the prior art, and significantly improves the accuracy and reliability of smoke identification.

Description

Technical Field

[0001] This invention relates to the field of power system safety operation and maintenance technology, and in particular to a smoke recognition and false alarm elimination method based on smoke color and negative sample similarity. Background Technology

[0002] In recent years, with the increasing frequency of extreme weather events and the expansion of human activity, the risk of wildfires around power transmission lines has risen significantly. Smoke detection along power transmission lines is a crucial component of intelligent monitoring in the power industry, primarily used for early warning and prevention of potential safety hazards such as wildfires beneath the lines, and is of great significance for the normal operation of the lines. Current smoke detection methods have evolved from early manual inspections to satellite-based remote sensing, and then to methods based on traditional image processing, further reducing the need for human intervention. The mainstream approach now is based on deep learning-based recognition methods, including common methods such as YOLO-based object detection, methods that obtain contours through semantic segmentation and then combine them with gradient changes, and methods that combine segmentation with smoke flow direction.

[0003] However, in current technologies, power transmission line scenarios are mostly deployed with ordinary cameras. Infrared devices and multi-sensor systems are costly and difficult to widely adopt. Vision-based methods are currently the mainstream recognition technology. Traditional image processing methods mainly utilize features such as color, texture, and gradient changes in images, combined with common classifiers or traditional segmentation methods to identify the changing patterns of contour pixels. This method is extremely sensitive to changes in the environment, such as lighting, and has poor versatility. YOLO series target detection methods mainly use labeled data from large databases for learning and recognition. However, because these methods only consider key information within the smoke area and do not accurately analyze the smoke contour, distinguishing between smoke and fog is difficult. Segmentation-based methods can better utilize the contour information of smoke, but they do not consider the changing characteristics of smoke. Methods combining detection and segmentation, and using the shape of smoke to determine its mobility, can better distinguish between smoke and fog, improving the detection rate and reducing the false detection rate to some extent. However, the identification of smoke from chimneys is still somewhat difficult to distinguish, and it does not consider historical image information from the same location, leaving room for improvement. Summary of the Invention

[0004] The purpose of this invention is to provide a smoke recognition and false alarm elimination method based on smoke color and negative sample similarity, thereby improving the smoke detection rate and further reducing false alarms of non-fire-generated smoke such as fog, clouds, and chimney smoke.

[0005] The technical solution adopted in this invention is:

[0006] The smoke recognition and false alarm elimination method based on smoke color and negative sample similarity includes the following steps:

[0007] Step 1: Obtain images from cameras patrolling overhead power transmission lines.

[0008] Step 2: After labeling the patrol images, the labeled data is generated. The labels should include at least four categories: black smoke, white smoke, chimney, and fire.

[0009] Step 3: Divide the labeled data into training set, validation set and test set according to the preset distribution ratio, and then use the instance segmentation model to train the model using the training data.

[0010] Step 4: Use the trained instance segmentation model to identify the input image to be processed, and obtain the recognition results including category labels, detection boxes, and segmentation masks. Based on the recognition results, perform the following logical judgment:

[0011] Step 4-1: When the document recognition result contains both black smoke and fire labels, calculate the cross-union ratio (CURBR) between black smoke and fire. If the CURBR is greater than 0, retain the current recognition result and proceed to step 5; otherwise, proceed to step 4-2.

[0012] Step 4-2: When the identification result contains both smoke and chimneys, calculate the cross-union ratio between smoke and chimneys; when the cross-union ratio between black smoke and fire is greater than 0, calculate the sum of the vertical projections (width ratio) of the smoke segmentation mask and the chimney segmentation mask in the intersecting ordinate interval and execute step 4-3; otherwise, execute step 5.

[0013] Step 4-3: Determine whether the ratio of the vertical projection of the smoke to the vertical projection of the chimney meets the first preset condition; if yes, determine that the smoke is non-fire smoke (released from the chimney); otherwise, proceed to step 5.

[0014] Step 5: Extract the feature vector of the image region that has been identified as smoke in the image to be processed; obtain the feature vector of at least one historical negative sample image without smoke that is taken at the same location as the image to be processed; calculate the similarity between the feature vector of the smoke region and the feature vector of each historical negative sample image; if all the calculated similarities meet the second preset condition, then the current smoke is determined to be a false alarm and filtered; otherwise, the current smoke is determined to be a real smoke alarm and output.

[0015] Furthermore, in step 2, labelme is used to label the rotating images.

[0016] Furthermore, the data labeled in step 2 includes the precise contour information of the target object.

[0017] Furthermore, in step 3, the instance segmentation model is the YOLOv12 instance segmentation model.

[0018] Furthermore, in step 3, the preset distribution ratio of the training set, validation set, and test set is 0.9:0.5:0.5.

[0019] Furthermore, in step 4-1, the overlap between black smoke and fire... The calculation formula is as follows:

[0020] ;

[0021] in These represent the areas of the black smoke label detection frame and the fire label detection frame, respectively.

[0022] Specifically, in step 4-1, the crossover ratio of black smoke and fire is calculated. If the crossover ratio is greater than 0, the black smoke is saved as smoke. Even if there is a fire, clouds or fog may also exist, and they are mostly white. They cannot be directly judged. White smoke is directly saved for the next step of processing.

[0023] Furthermore, step 4-2 includes the following steps:

[0024] Step 4-2-1: First, calculate the Intersection over Union (IOU) ratio of the smoke and the chimney. The calculation formula is:

[0025] ,

[0026] in, This indicates the area of ​​the detection frame for smoke and chimneys.

[0027] Step 4-2-2: Calculate the set of intersection ordinates of the chimney and smoke intersection ratio regions. ,

[0028] Step 4-2-3: Calculate the binary mask image of the chimney and smoke in coordinates. arrive For each vertical projection within the interval, calculate the sum of the corresponding projections using the following formula:

[0029] ;

[0030] ;

[0031] in, and Let S and S represent the vertical projections of the smoke and the chimney, respectively; Indicates smoke in arrive Projection of the interval Indicates smoke in arrive Projection of the interval;

[0032] Furthermore, the first preset condition in step 4-3 is: .

[0033] Furthermore, the historical negative sample images are pre-collected and stored normal environmental images without smoke from the same shooting location.

[0034] Furthermore, the feature vector extraction of the image region in step 5 is accomplished using a pre-trained ResNet18 feature extractor.

[0035] Furthermore, in step 5, the similarity is calculated using the Pearson correlation coefficient method; the second preset condition is that all calculated Pearson correlation coefficients are greater than a preset threshold. The Pearson correlation coefficient calculation formula is:

[0036] ,

[0037] Where r represents the calculated similarity, express A value in a vector, express The average value of a vector. express A value in a vector, express The average value of a vector.

[0038] Specifically, in step 5, ResNet18 is used to calculate the feature vector of the current image for the smoke region, denoted as... Based on the location name in the filename of the current image, retrieve all historical images of the same location in the configuration directory, and calculate the feature vectors of each using ResNet18, denoted as... ; calculate in sequence and Pearson similarity of all feature vectors in the dataset; if and If the similarity of all feature vectors is greater than a specified threshold, the current image is considered a normal image, and the detected smoke area is a false alarm and should be filtered out.

[0039] This invention, employing the above technical solutions, offers the following advantages over existing technologies: It labels objects related to smoke and the color of the smoke, effectively aiding in the differentiation between smoke and fog caused by fires. For identified bounding boxes, it determines whether the smoke overlaps with the chimney; if the overlap is small and the bottom width of the smoke is close to the width of the chimney, filtering is performed to reduce the likelihood of chimney smoke being reported as fire smoke. Since there is a significant difference between dense smoke and fog, this invention directly outputs smoke if a fire or black smoke is detected in the bounding box, ensuring a high smoke detection rate. If it is white smoke, its difference from fog is assessed for false alarm elimination. Based on historical negative samples and combined with the ResNet18 deep learning feature extractor, features are extracted from the current image and historical images respectively. The similarity between the historical image features and the alarm region of the current image is then calculated; close similarity indicates a false alarm. This method effectively eliminates false alarms related to fog. Attached Figure Description

[0040] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments;

[0041] Figure 1 This is a flowchart illustrating the smoke recognition and false alarm elimination method based on smoke color and negative sample similarity of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0043] like Figure 1 As shown, this invention discloses a smoke recognition and false alarm elimination method based on smoke color and negative sample similarity, comprising the following steps:

[0044] Step 1: Obtain images from cameras patrolling overhead power transmission lines.

[0045] Step 2: After labeling the patrol images, the labeled data is generated. The labels should include at least four categories: black smoke, white smoke, chimney, and fire.

[0046] Step 3: Divide the labeled data into training set, validation set and test set according to the preset distribution ratio, and then use the instance segmentation model to train the model using the training data.

[0047] Step 4: Use the trained instance segmentation model to identify the input image to be processed, and obtain the recognition results including category labels, detection boxes, and segmentation masks. Based on the recognition results, perform the following logical judgment:

[0048] Step 4-1: When the document recognition result contains both black smoke and fire labels, calculate the cross-union ratio (CURBR) between black smoke and fire. If the CURBR is greater than 0, retain the current recognition result and proceed to step 5; otherwise, proceed to step 4-2.

[0049] Step 4-2: When the identification result contains both smoke and chimneys, calculate the cross-union ratio between smoke and chimneys; when the cross-union ratio between black smoke and fire is greater than 0, calculate the sum of the vertical projections (width ratio) of the smoke segmentation mask and the chimney segmentation mask in the intersecting ordinate interval and execute step 4-3; otherwise, execute step 5.

[0050] Step 4-3: Determine whether the ratio of the vertical projection of the smoke to the vertical projection of the chimney meets the first preset condition; if yes, determine that the smoke is non-fire smoke (released from the chimney); otherwise, proceed to step 5.

[0051] Step 5: Extract the feature vector of the image region that has been identified as smoke in the image to be processed; obtain the feature vector of at least one historical negative sample image without smoke that is taken at the same location as the image to be processed; calculate the similarity between the feature vector of the smoke region and the feature vector of each historical negative sample image; if all the calculated similarities meet the second preset condition, then the current smoke is determined to be a false alarm and filtered; otherwise, the current smoke is determined to be a real smoke alarm and output.

[0052] Furthermore, in step 2, labelme is used to label the rotating images.

[0053] Furthermore, the data labeled in step 2 includes the precise contour information of the target object.

[0054] Furthermore, in step 3, the instance segmentation model is the YOLOv12 instance segmentation model.

[0055] Furthermore, in step 3, the preset distribution ratio of the training set, validation set, and test set is 0.9:0.5:0.5.

[0056] Furthermore, in step 4-1, the overlap between black smoke and fire... The calculation formula is as follows:

[0057] ;

[0058] in These represent the areas of the black smoke label detection frame and the fire label detection frame, respectively.

[0059] Specifically, in step 4-1, the crossover ratio of black smoke and fire is calculated. If the crossover ratio is greater than 0, the black smoke is saved as smoke. Even if there is a fire, clouds or fog may also exist, and they are mostly white. They cannot be directly judged. White smoke is directly saved for the next step of processing.

[0060] Furthermore, step 4-2 includes the following steps:

[0061] Step 4-2-1: First, calculate the Intersection over Union (IOU) ratio of the smoke and the chimney. The calculation formula is:

[0062] ,

[0063] in, This indicates the area of ​​the detection frame for smoke and chimneys.

[0064] Step 4-2-2: Calculate the set of intersection ordinates of the chimney and smoke intersection ratio regions. ,

[0065] Step 4-2-3: Calculate the binary mask image of the chimney and smoke in coordinates. arrive For each vertical projection within the interval, calculate the sum of the corresponding projections using the following formula:

[0066] ;

[0067] ;

[0068] in, and Let S and S represent the vertical projections of the smoke and the chimney, respectively; Indicates smoke in arrive Projection of the interval Indicates smoke in arrive The projection of the interval; yw is the general term for black smoke and white smoke, and yc represents the chimney.

[0069] Furthermore, the first preset condition in step 4-3 is: .

[0070] Furthermore, the feature vector extraction of the image region in step 5 is accomplished using a pre-trained ResNet18 feature extractor.

[0071] Furthermore, in step 5, the similarity is calculated using the Pearson correlation coefficient method; the second preset condition is that all calculated Pearson correlation coefficients are greater than a preset threshold. The Pearson correlation coefficient calculation formula is:

[0072] ,

[0073] Where r represents the calculated similarity, express A value in a vector, express The average value of a vector. express A value in a vector, express The average value of a vector.

[0074] Specifically, in step 5, ResNet18 is used to calculate the feature vector of the current image for the smoke region, denoted as... Based on the location name in the filename of the current image, retrieve all historical images of the same location in the configuration directory, and calculate the feature vectors of each using ResNet18, denoted as... ; calculate in sequence and Pearson similarity of all feature vectors in the dataset; if and If the similarity of all feature vectors is greater than a specified threshold, the current image is considered a normal image, and the detected smoke area is a false alarm and should be filtered out.

[0075] This invention, employing the above technical solutions, offers the following advantages over existing technologies: It labels objects related to smoke and the color of the smoke, effectively aiding in the differentiation between smoke and fog caused by fires. For identified bounding boxes, it determines whether the smoke overlaps with the chimney; if the overlap is small and the bottom width of the smoke is close to the width of the chimney, filtering is performed to reduce the likelihood of chimney smoke being reported as fire smoke. Since there is a significant difference between dense smoke and fog, this invention directly outputs smoke if a fire or black smoke is detected in the bounding box, ensuring a high smoke detection rate. If it is white smoke, its difference from fog is assessed for false alarm elimination. Based on historical negative samples and combined with the ResNet18 deep learning feature extractor, features are extracted from the current image and historical images respectively. The similarity between the historical image features and the alarm region of the current image is then calculated; close similarity indicates a false alarm. This method effectively eliminates false alarms related to fog.

[0076] Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Without conflict, the embodiments and features in the embodiments of this application can be combined with each other. The components of the embodiments of this application described and illustrated herein can generally be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of this application is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

Claims

1. A smoke recognition and false alarm elimination method based on smoke color and negative sample similarity, characterized in that: It includes the following steps: Step 1: Obtain images from cameras patrolling overhead power transmission lines. Step 2: After labeling the patrol images, the labeled data is generated. The labels should include at least four categories: black smoke, white smoke, chimney, and fire. Step 3: Divide the labeled data into training set, validation set and test set according to the preset distribution ratio, and then use the instance segmentation model to train the model using the training data. Step 4: Use the trained instance segmentation model to identify the input image to be processed, and obtain the recognition results including category labels, detection boxes, and segmentation masks. Based on the recognition results, perform the following logical judgment: Step 4-1: When the document recognition result contains both black smoke and fire labels, calculate the cross-union ratio (CURRY) between black smoke and fire; when the CURRY between black smoke and fire is greater than 0, retain the current recognition result and proceed to step 5. Otherwise, proceed to step 4-2; Step 4-2: When the identification result contains both smoke and chimney, calculate the cross-union ratio between smoke and chimney; when the cross-union ratio between black smoke and fire is greater than 0, calculate the vertical projection of the smoke segmentation mask and the chimney segmentation mask in the intersecting ordinate interval and execute step 4-3; otherwise, execute step 5. Step 4-3: Determine whether the ratio of the vertical projection of the smoke to the vertical projection of the chimney satisfies the first preset condition. If so, determine that the smoke is not fire smoke; otherwise, proceed to step 5. Step 5: Extract the feature vector of the image region that has been identified as smoke in the image to be processed; obtain the feature vector of at least one historical negative sample image without smoke that is taken at the same location as the image to be processed; calculate the similarity between the feature vector of the smoke region and the feature vector of each historical negative sample image; if all the calculated similarities meet the second preset condition, then the current smoke is determined to be a false alarm and filtered; otherwise, the current smoke is determined to be a real smoke alarm and output.

2. The smoke recognition and false alarm elimination method based on smoke color and negative sample similarity according to claim 1, characterized in that: In step 2, labelme is used to label the round-robin images. The label data includes the precise outline information of the target objects.

3. The smoke recognition and false alarm elimination method based on smoke color and negative sample similarity according to claim 1, characterized in that: In step 3, the instance segmentation model is the YOLOv12 instance segmentation model.

4. The smoke recognition and false alarm elimination method based on smoke color and negative sample similarity according to claim 1, characterized in that: In step 3, the preset distribution ratio of the training set, validation set, and test set is 0.9:0.5:0.

5.

5. The smoke recognition and false alarm elimination method based on smoke color and negative sample similarity according to claim 1, characterized in that: The crossover ratio of black smoke and fire in step 4-1 The calculation formula is as follows: ； in These represent the areas of the black smoke label detection frame and the fire label detection frame, respectively.

6. The smoke recognition and false alarm elimination method based on smoke color and negative sample similarity according to claim 1, characterized in that: Step 4-2 includes the following steps: Step 4-2-1: First, calculate the Intersection over Union (IOU) ratio of the smoke and the chimney. The calculation formula is: ， in, This indicates the area of ​​the detection frame for smoke and chimneys. Step 4-2-2: Calculate the set of intersection ordinates of the chimney and smoke intersection ratio regions. , Step 4-2-3: Calculate the binary mask image of the chimney and smoke in coordinates. arrive For each vertical projection within the interval, calculate the sum of the corresponding projections using the following formula: ； ； in, and Let S and S represent the vertical projections of the smoke and the chimney, respectively; Indicates smoke in arrive Projection of the interval Indicates smoke in arrive Projection of the interval.

7. The smoke recognition and false alarm elimination method based on smoke color and negative sample similarity according to claim 1, characterized in that: The first preset condition in step 4-3 is .

8. The smoke recognition and false alarm elimination method based on smoke color and negative sample similarity according to claim 1, characterized in that: Step 5 extracts the feature vectors of the image region using a pre-trained ResNet18 feature extractor.

9. The smoke recognition and false alarm elimination method based on smoke color and negative sample similarity according to claim 1, characterized in that: In step 5, the similarity is calculated using the Pearson correlation coefficient method; the second preset condition is that the calculated Pearson correlation coefficients are all greater than a preset threshold.

10. The smoke recognition and false alarm elimination method based on smoke color and negative sample similarity according to claim 9, characterized in that: Pearson's formula is: ； Where r represents the calculated similarity, express A value in a vector, express The average value of a vector. express A value in a vector, express The average value of a vector.

Images