A wildfire anomaly detection method based on multi-frame dynamic processing of positive samples

Abstract

The present invention relates to the field of image safety supervision, and provides a wildfire anomaly detection method based on multi-frame dynamic processing of positive samples. The purpose is to improve the accuracy and stability of the wildfire detection algorithm through innovative dynamic processing and positive sample strategies, especially in the application of power transmission scenarios, so as to effectively reduce false alarms and improve detection efficiency. The main scheme includes obtaining a fine data set and training a detection model by annotating wildfire data. The video frame is differentially processed to identify the changed area, and the specific size area is extracted and mapped to N equally divided masks to achieve dynamic target detection. The sky segmentation model and OpenCV process the image, save the outline of the sky area, and filter out false alarms of clouds. The scoring mechanism determines the alarm, and an alarm is triggered when the continuous score exceeds the specified threshold. After the alarm, the fire alarm score is reset, the cooling period is started, and the alarm is suspended.

Description

Technical Field

[0001] The present invention relates to the field of image safety supervision and provides a forest fire anomaly detection method based on multi-frame dynamic processing of positive samples. Background Art

[0002] In the field of wildfire detection, the traditional method is to manually review the data sent back by the camera to determine whether there is a wildfire incident. In recent years, with the development of deep learning, target detection technology and large model technology have emerged in an endless stream. The schemes for wildfire detection using target detection technology and large model technology are gradually increasing, and at the same time, there are also tasks of inspection by drones equipped with deep learning algorithms. The above technologies face the following problems:

[0003] The traditional method of manually reviewing fire conditions is time-consuming and labor-intensive. It is impossible to take into account thousands of data at the same time every day, which is bound to result in omissions and is very costly.

[0004] Traditional deep learning models cannot guarantee on-site operation when they require a large amount of data annotation for training. The algorithm is unstable and the false alarms of objects similar to wildfires are abnormally high (such as clouds, sun, yellow soil, reflective objects).

[0005] The technical threshold for large models is high, and extremely expensive graphics cards are required to support training and reasoning.

[0006] Drone technology equipped with deep learning algorithms has certain limitations. The drone platform itself has many limitations and does not have many analytical functions. At the same time, drones need to recruit corresponding pilots, and drones also have problems with endurance, resulting in high costs. Summary of the invention

[0007] The purpose of the present invention is to improve the accuracy and stability of wildfire detection algorithms through innovative dynamic processing and positive sample strategies, especially in power transmission scenarios, thereby effectively reducing false alarms and improving detection efficiency.

[0008] In order to achieve the above purpose, the present invention adopts the following technical means:

[0009] The present invention provides a method for detecting abnormal mountain fires based on multi-frame dynamic processing of positive samples, comprising the following steps:

[0010] Step 1: Perform multi-level annotation on wildfire data to obtain a finely annotated data set;

[0011] Step 2: Use the finely labeled data set to train the model and obtain the detection model, which is used to directly detect the target in the image and obtain the target detection frame;

[0012] Step 3: Use image processing technology to perform differential processing on the video frame to identify the changed area, then extract the contour and filter out the area of ​​size, and finally map these areas to an N-equally divided mask for processing to achieve dynamic target detection, determine whether the target detection box is located in the changed area, and filter static false alarms;

[0013] Step 4: Use the sky segmentation model and OpenCV to process the image, save the sky area contour information to a json file, load the sky area contour by reading the specified JSON file, and use it to filter out cloud false alarms in wildfire detection. After filtering static false alarms through step 3, step 2 uses sky segmentation and OpenCV processing again to filter out cloud false alarms.

[0014] Step 5: record the judgment results of step 3 and step 4 on the target frame in step 2, analyze the judgment results, calculate the score through the scoring mechanism, and issue a wildfire alarm if the score reaches the specified threshold;

[0015] Step 6: After the algorithm successfully issues an alarm, the fire alarm score is cleared and a 5-minute cooling-off period is started, during which the alarm push is suspended. The cooling-off time can be adjusted. During the cooling-off period, the scoring mechanism in step 5 runs simultaneously. If the score alarm threshold is reached during the cooling-off period, no report is made.

[0016] In the above technical solution, step 1 includes the following steps:

[0017] Step 1.1, classify and label wildfires into “obvious flames”, “possible flames” and “flame-like background”;

[0018] Step 1.2: Classify and label the smoke into “thick smoke” and “thin smoke”;

[0019] Step 1.3: Contextually label the environmental information around the wildfire, including clouds, trees, and buildings;

[0020] Step 1.4: Integrate the above annotation information to form a finely annotated dataset.

[0021] In the above technical solution, step 2 includes the following steps:

[0022] Step 2.1, use the yolov5l model to train flame and smoke separately;

[0023] Step 2.2, integrating the trained flame and smoke detection models to form a complete wildfire detection model;

[0024] Step 2.3: Use the complete wildfire detection model to detect the original image and obtain the target detection frame.

[0025] In the above technical solution, step 3 includes the following steps:

[0026] Step 3.1, use the python queue to store the current frame and the previous frame, perform frame difference processing, and obtain the changed area img_poor_thres1;

[0027] Step 3.2, use the binarization function to binarize img_poor_thre1, and then use the 7x7 rectangular structure element to perform a dilation and erosion operation on img_poor_thre1 to obtain img_poor_dilate;

[0028] Step 3.3, use the opencv findContours function to extract the contours of the eroded image img_poor_dilate, and filter out the areas with contour areas greater than 800, recorded as contours_800;

[0029] Step 3.4, divide the image into 6 regions, process contours_800 in each region, find the minimum circumscribed rectangle of contours_800 in each region and fill it, and construct mask_motion for motion detection;

[0030] Step 3.5: Calculate the target detection box and mask_motion in step 2.2 Value, to determine whether the target box is in the change area.

[0031] In the above technical solution, step 4 includes the following steps:

[0032] Step 4.1: Use the sky segmentation model to segment the sky in the picture with sky and obtain the outline of the sky area;

[0033] Step 4.2: Save the sky area contour information to a json file that matches the image as a segmentation positive sample, and store it in a specified location where the algorithm can read it;

[0034] Step 4.3, during the detection, read the json file matching the image to be detected from the specified location; load the roi information in the json file, and determine the relative position of the detection box and the sky area roi;

[0035] Step 4.4, calculate the minimum distance between the lower boundary of the target detection box and the sky contour area in step 2.2;

[0036] Step 4.5: Compare the calculated distance with the safety threshold to determine whether it is a false alarm of cloud cover.

[0037] In the above technical solution, step 5 includes the following steps:

[0038] Step 5.1, use score_queue to record each detection result and score it according to different situations. The final score is score_mountain_fire;

[0039] Step 5.2: Determine whether the score score_mountain_fire exceeds the alarm threshold and decide whether to issue an alarm.

[0040] In the above technical solution, step 3.5 includes the following steps:

[0041] Step 3.5.1, obtain the coordinates of the detection box in the current frame image;

[0042] Step 3.5.2, obtain the mask_motion area in the current frame image;

[0043] Step 3.5.3: Calculate the detection box and mask_motion area value;

[0044] Step 3.5.4: The calculated The value is compared with a preset threshold;

[0045] Step 3.5.5: If If the value is greater than or equal to the threshold, the target area is judged to be in the change area.

[0046] In the above technical solution, step 5.1 includes the following steps:

[0047] Step 5.1.1, initialize the python queue score_queue;

[0048] Step 5.1.2, add the test results after step 4 to the score_queue queue;

[0049] Step 5.1.3: If there are two test results in the queue, take out these two results for analysis;

[0050] Step 5.1.4: Scoring is based on the results of the two tests: 5 points are added if two fires are detected consecutively; 1 point is deducted if the fire is detected once but not the second time; 2 points are added if the fire is detected once but not the first time; and 2 points are deducted if the fire is not detected twice;

[0051] Step 5.1.5, update the score score_mountain_fire.

[0052] In the above technical solution, step 5.2 includes the following steps:

[0053] Step 5.2.1, get the current score_mountain_fire score;

[0054] Step 5.2.2, compare the score_mountain_fire score with the preset alarm threshold;

[0055] Step 5.2.3: If the score_mountain_fire score is greater than or equal to the alarm threshold, an alarm is triggered.

[0056] In the above technical solution, in step 3.5 Calculation formula:

[0057] )

[0058]

[0059]

[0060] in is the intersection-union ratio, which is the ratio of the intersection and union of the model prediction box and the sample real box. b is the center coordinate of the prediction box, is the center coordinate of the real frame; for b and The Euclidean distance between c is the Euclidean distance between two diagonal vertices of the minimum rectangular box; is the width of the real frame, is the height of the real frame; w is the width of the prediction box, h is the height of the prediction box.

[0061] Because the present invention adopts the above technical means, it has the following beneficial effects:

[0062] Multi-level labeling technology: Through refined multi-level labeling, such as dividing flames into different levels of certainty, the model can learn more accurate flame characteristics. This solves the problem of insufficient description of wildfire characteristics in traditional methods and improves the model's ability to recognize wildfire characteristics.

[0063] Contextual annotation technology: While annotating the wildfire itself, it also annotates its surrounding environment information, such as clouds, fog, buildings, etc., to help the model learn the relationship between the wildfire and the surrounding environment. This solves the problem of insufficient use of environmental information in traditional methods and enhances the model's ability to understand complex scenes.

[0064] Dynamic detection mechanism: Utilizes image frame difference processing and contour extraction technology to effectively identify dynamically changing areas in the image, thereby detecting wildfires in real time. This technology solves the problem of insufficient dynamic information capture in static image analysis and improves the real-time and accuracy of wildfire detection.

[0065] Cloud filtering technology: Sky segmentation model and contour analysis are used to filter out false alarm factors such as clouds. This solves the problem of high false alarm rate of clouds in traditional wildfire detection and reduces the false alarm rate.

[0066] Scoring and cooling mechanisms: Scoring is performed based on continuous detection results, and a cooling time is set to reduce repeated alarms. This solves the problem of frequent false alarms and repeated alarms in traditional methods, and improves the stability and reliability of the system.

[0067] Positive sample strategy: The positive samples generated by the sky segmentation model effectively avoid false positives of non-wildfire objects such as sky clouds. This solves the false positive problem caused by the imbalance of positive and negative samples in traditional methods and improves the accuracy and efficiency of detection.

[0068] Compared with existing technologies, this solution has significantly improved the accuracy, real-time performance and stability of wildfire detection, while reducing the false alarm rate and improving environmental adaptability and detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 is a flow chart;

[0070] Figure 2 is the image after frame difference;

[0071] Figure 3 is the image after area threshold;

[0072] Figure 4 It is the mask after dynamic detection. DETAILED DESCRIPTION

[0073] The following is a detailed description of the embodiments of the present invention. Although the present invention will be described and illustrated in conjunction with some specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, modifications or equivalent substitutions made to the present invention should all be included in the scope of the claims of the present invention.

[0074] In addition, in order to better illustrate the present invention, numerous specific details are given in the following specific embodiments. It will be understood by those skilled in the art that the present invention can also be implemented without these specific details.

[0075] The present invention provides a method for detecting mountain fire anomalies based on multi-frame dynamic processing of positive samples, comprising the following steps:

[0076] Data preparation:

[0077] Detailed marking:

[0078] Multi-level labeling: Wildfires are labeled at multiple levels, classifying flames into "obvious flames", "possible flames (such as yellow lights)" and "flame-like backgrounds"; smoke is classified into "thick smoke" and "thin smoke"; this helps the model learn flame features at different levels of certainty.

[0079] Contextual annotation: In addition to annotating the wildfire itself, it is also necessary to annotate the environmental information around the wildfire, such as clouds, trees, buildings (transmission towers, houses, factory chimneys), etc., to help the model learn the relationship between the wildfire and its surrounding environment.

[0080] Model preparation:

[0081] The prepared training data is used to train the model and obtain the detection model for standby use. Here, the yolov5l model is used, and since the amount of data for smoke and fire is quite different, the model training is performed separately.

[0082] Dynamic detection process:

[0083] Image frame difference processing:

[0084] Use Python queue to store the current frame image and the previous frame image. When the queue length reaches 2, use the difference between the current frame image and the previous frame image to get the changed area of ​​the image, recorded as img_poor_thres1.

[0085] Image contour extraction:

[0086] Use the binarization function to binarize img_poor_thre1, and then use the 7x7 rectangular structure element to perform a dilation and erosion operation on img_poor_thre1 to obtain img_poor_dilate.

[0087] Use the opencv findContours function to extract the contours of the eroded image img_poor_dilate, and filter out the areas with contour areas greater than 800, recorded as contours_800.

[0088] The picture is divided into 6 parts:

[0089] Construct an all-0 mask array of the same length and width as the current frame image, divide the mask into 6 equal parts at 1 / 3, 2 / 3, and 1 / 2 of the image length, and process each area one by one.

[0090] In each area, get the xmin, ymin, xmax, ymax of the current contour area (contours_800). Fill the rectangle formed by the coordinate area in mask with 255.

[0091] After processing the 6 regions in sequence, the final dynamic detection mask_motion is obtained.

[0092] Cloud Filter:

[0093] Use the sky segmentation model (Grounded-Segment-Anything) to segment the sky, use OpenCV to calculate the segmentation area outline, and save the sky area outline as the segmentation positive sample (considering factors such as camera shake, a safe distance needs to be left between the sky area roi and the ground segmentation line to prevent fireworks at the boundary from being filtered. The safe distance is determined based on actual conditions). Here, a json file with the same name as the image (the unique id or code of the image is required here, and the source of the image can be determined, and the roi file must correspond to the image) is used to save the roi and store it in the specified location.

[0094] During the detection, read the json file with the same name as the image to be detected from the specified location. Load the roi information in the json file and convert it into an array to determine the relative position of the detection box and the sky area roi. Here, the minimum distance between the lower boundary of the detection box and the sky contour area is mainly determined. If it is greater than the safety threshold, it is considered that the wildfire alarm is a false alarm of the cloud layer.

[0095] Calculation method: The coordinates of the lower left corner and the lower right corner of the detection box are bottom_left and bottom_right respectively. The coordinates and the contour area position are judged by the cv2.pointPolygonTest() function in OpenCV, and finally the calculated distance is compared with the safety threshold.

[0096] Alarm judgment-scoring mechanism:

[0097] Calculate the detection box and mask_motion If the value exceeds the set threshold, it is recorded as a successful detection.

[0098] Use the python queue (score_queue) to continuously record two analysis results. The first test has only one result, so no alarm is needed.

[0099] The results in the score_queue queue are taken out for analysis. If two mountain fires are detected consecutively, 5 points are added; if the first fire is detected but not the second time, 1 point is deducted; if the first fire is not detected but the second fire is detected, 2 points are added; if the fire is not detected twice, 2 points are deducted. The score is calculated as score_mountain_fire, and the specific score is adjusted according to the model analysis.

[0100] If score_mountain_fire is greater than or equal to the score alarm threshold (the score is set according to the actual situation, here it is 10 points), the algorithm will issue an alarm through the mountain fire anomaly detection.

[0101] Cooling mechanism:

[0102] After the algorithm successfully issues an alarm, score_mountain_fire is cleared and the cooling switch cooling=True. At the same time, a 5-minute timer starts, during which the analysis results will no longer be pushed, and no alarm processing will be performed. The cooling time can be adjusted according to the actual situation.

[0103] Calculation formula:

[0104]

[0105]

[0106]

[0107] in It is the intersection over union, which is the ratio of the intersection and union of the model prediction box (bounding box) and the sample true box (ground truth). b is the center coordinate of the prediction box, is the center coordinate of the real frame; for b and The Euclidean distance between c is the Euclidean distance between two diagonal vertices of the minimum rectangular box; is the width of the real frame, is the height of the real frame; w is the width of the prediction box, h is the height of the prediction box.

Claims

1. A method for detecting abnormal mountain fires based on multi-frame dynamic processing of positive samples, characterized in that: The following steps are involved: Step 1: Perform multi-level annotation on wildfire data to obtain a finely annotated data set; Step 2: Use the finely labeled data set to train the model and obtain the detection model, which is used to directly detect the target in the image and obtain the target detection frame; Step 3: Use image processing technology to perform differential processing on the video frame to identify the changed area, then extract the contour and filter out the area of ​​preset size, and finally map these areas to an N-equally divided mask for processing to achieve dynamic target detection, determine whether the target detection box is located in the changed area, and filter static false alarms; Step 3.1, use the python queue to store the current frame and the previous frame, perform frame difference processing, and obtain the changed area img_poor_thres1; Step 3.2, use the binarization function to binarize img_poor_thre1, and then use the 7x7 rectangular structure element to perform a dilation and erosion operation on img_poor_thre1 to obtain img_poor_dilate; Step 3.3, use the opencv findContours function to extract the contours of the eroded image img_poor_dilate, and filter out the areas with contour areas greater than 800, recorded as contours_800; Step 3.4, divide the image into 6 regions, process contours_800 in each region, find the minimum circumscribed rectangle of contours_800 in each region and fill it, and construct mask_motion for motion detection; Step 3.5: Calculate the target detection frame and mask_motion Value, to determine whether the target detection box is in the changed area; Step 4: Use the sky segmentation model and OpenCV to process the image, save the sky area contour information to a json file, load the sky area contour by reading the specified JSON file, and use it to filter out cloud false alarms in wildfire detection. After filtering static false alarms through step 3, step 2 uses sky segmentation and OpenCV processing again to filter out cloud false alarms. Step 4.1: Use the sky segmentation model to segment the sky in the picture with sky and obtain the outline of the sky area; Step 4.2: Save the sky area contour information to a json file that matches the image as a segmentation positive sample, and store it in a specified location where the algorithm can read it; Step 4.3, during the detection, read the json file matching the image to be detected from the specified location; Load the roi information in the json file and determine the relative position of the target detection box and the sky area roi; Step 4.4, calculate the minimum distance between the lower boundary of the target detection frame and the sky contour area; Step 4.5, compare the calculated distance with the safety threshold to determine whether it is a false alarm of cloud cover; Step 5: record the judgment results of step 3 and step 4 on the target detection frame in step 2, analyze the judgment results, calculate the score through the scoring mechanism, and issue a wildfire alarm if the score reaches the specified threshold; Step 6: After the algorithm successfully issues an alarm, the fire alarm score is cleared and a 5-minute cooling-off period is started, during which the alarm push is suspended. The cooling-off time can be adjusted. During the cooling-off period, the scoring mechanism in step 5 runs simultaneously. If the score alarm threshold is reached during the cooling-off period, no report is made.

2. The method for detecting abnormal wildfires based on multi-frame dynamic processing of positive samples according to claim 1, characterized in that: Step 1 includes the following steps: Step 1.1: Classify and label wildfires into "obvious flames", "possible flames" and "flame-like background"; Step 1.2: Classify and label the smoke into "thick smoke" and "thin smoke"; Step 1.3: Contextually label the environmental information around the wildfire, including clouds, trees, and buildings; Step 1.4: Integrate the above annotation information to form a finely annotated dataset.

3. The method for detecting abnormal wildfires based on multi-frame dynamic processing of positive samples according to claim 1, characterized in that: Step 2 includes the following steps: Step 2.1, use the yolov5l model to train flame and smoke separately; Step 2.2, integrating the trained flame and smoke detection models to form a complete wildfire detection model; Step 2.3: Use the complete wildfire detection model to detect the original image and obtain the target detection frame.

4. The method for detecting abnormal mountain fires based on multi-frame dynamic processing of positive samples according to claim 1, characterized in that: Step 5 includes the following steps: Step 5.1, use score_queue to record each detection result and score it according to different situations. The final score is score_mountain_fire; Step 5.2: Determine whether the score score_mountain_fire exceeds the alarm threshold and decide whether to issue an alarm.

5. The method for detecting abnormal wildfires based on multi-frame dynamic processing of positive samples according to claim 4 is characterized in that: Step 3.5 includes the following steps: Step 3.5.1, obtain the coordinates of the target detection frame in the current frame image; Step 3.5.2, obtain the mask_motion area in the current frame image; Step 3.5.3: Calculate the target detection box and mask_motion area value; Step 3.5.4: The calculated The value is compared with a preset threshold; Step 3.5.5: If If the value is greater than or equal to the threshold, the target area is judged to be in the change area.

6. The method for detecting abnormal wildfires based on multi-frame dynamic processing of positive samples according to claim 4, characterized in that: Step 5.1 includes the following steps: Step 5.1.1, initialize the python queue score_queue; Step 5.1.2, add the test results after step 4 to the score_queue queue; Step 5.1.3: If there are two test results in the queue, take out these two results for analysis; Step 5.1.4: Scoring is based on the results of the two tests: 5 points are added if two fires are detected consecutively; 1 point is deducted if the fire is detected once but not the second time; 2 points are added if the fire is detected once but not the first time; and 2 points are deducted if the fire is not detected twice; Step 5.1.5, update the score score_mountain_fire.

7. The method for detecting abnormal wildfires based on multi-frame dynamic processing of positive samples according to claim 4, characterized in that: Step 5.2 includes the following steps: Step 5.2.1, get the current score_mountain_fire score; Step 5.2.2, compare the score_mountain_fire score with the preset alarm threshold; Step 5.2.3: If the score_mountain_fire score is greater than or equal to the alarm threshold, an alarm is triggered.

8. The method for detecting abnormal wildfires based on multi-frame dynamic processing of positive samples according to claim 1, characterized in that: In step 3.5 Calculation formula: ) in Iou is the intersection-union ratio, which is the ratio of the intersection and union of the model prediction box and the sample real box. b is the center coordinate of the prediction box, is the center coordinate of the real frame; for b and The Euclidean distance between c is the Euclidean distance between two diagonal vertices of the minimum rectangular box; is the width of the real frame, is the height of the real frame; w is the width of the prediction box, h is the height of the prediction box.

Images