Forest fire detection method fusing multiple features

Abstract

The invention discloses a forest fire detection method fusing multiple features, and belongs to the field of image processing. Firstly, a data image is preprocessed; and then smoke and fire feature extraction is carried out from three aspects of color, shape and dynamic. Performing smoke and fire color feature identification based on multiple spaces, starting from an RGB color space, performing preliminary identification on flame features, then eliminating interference in combination with an HSV color space, and performing identification on smoke and fire shape features based on contours; based on a ViBe algorithm, dynamic characteristics of the smoke and fire are extracted; and finally, performing multi-feature fusion, and performing AND operation based on the multiple feature recognition results to obtain a fused multi-feature result. According to the method, the quality of monitoring video data is improved, various characteristics of smoke and fire are fused for recognition, the recognition accuracy of forest fire is effectively guaranteed, and compared with other traditional methods, the method has the advantages of being high in precision and capable of achieving real-time monitoring.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing, and relates to a forest fire detection method considering various fire characteristics. Specifically, it is a method for detecting forest fires by extracting multiple characteristics of smoke and fire (hereinafter referred to as smoke-fire), such as color, shape, and dynamics, and then fusing them. This method first selects a suitable image preprocessing method to optimize the image quality and improve the effectiveness of smoke-fire feature extraction. Subsequently, the color features of smoke-fire are extracted in multiple color spaces such as RGB and HSV, and the dynamic features of smoke-fire are extracted using the ViBe algorithm. Forest fire detection is achieved through the form of multi-feature fusion. Background Art

[0002] Forest fires are one of the most frequent natural disasters globally, which not only burn large areas of forests but also cause great damage to the global ecosystem. If forest fires are not detected in time in the initial stage, it is easy to form an uncontrollable disaster, resulting in more casualties and economic losses. Therefore, accurate, reliable, and timely forest fire detection is a necessary condition for preventing forest fires and an important guarantee for protecting people's lives and property safety.

[0003] Currently, there are various forest fire monitoring methods that have been practically applied, such as manual patrol, optical satellite remote sensing, and video monitoring. Among them, manual patrol requires forest rangers to patrol the forest for a long time to discover and report fires in a timely manner. However, manual patrol has the disadvantages of limited patrol areas and difficulty in achieving all-weather monitoring. Optical satellite remote sensing can monitor forest fires within a large spatial range. However, due to the conflict between the temporal resolution and spatial resolution of the satellite remote sensing system, it is difficult to achieve real-time monitoring of forest fires with high spatial resolution.

[0004] Video monitoring is a technology based on computer vision, with characteristics such as real-time, high adaptability, and high precision. Compared with manual patrol, video monitoring can achieve real-time, large-area, and all-weather forest fire monitoring. Compared with optical satellite remote sensing monitoring, video monitoring can accurately detect forest fires in the initial stage. Therefore, video monitoring is more effective for forest fire monitoring. Summary of the Invention

[0005] The present invention aims to provide a forest fire detection method based on multiple forest fire characteristics and applicable to video monitoring data for near-real-time detection of forest fire points in the southwestern region of China. This method includes a series of preprocessing processes to eliminate various noises in forest fire images and improve image saturation. Subsequently, through various feature extraction methods, multiple smoke-fire characteristics in the image are analyzed. These features are fused and combined with subsequent target bounding operations to obtain the final forest fire point detection result.

[0006] The technical solution of the present invention is: a forest fire detection method integrating multiple features, and the method includes:

[0007] Step 1: Collect video images of the forest farm and preprocess the images;

[0008] Further, the specific method of the overall preprocessing in Step 1 is: first perform image denoising on the video image using Gaussian filtering, then perform image denoising using mean filtering, and finally enhance the image by adjusting the saturation.

[0009] Step 2: Extract flame pixel points according to color features;

[0010] Step 2.1: Extract flame features using the following formula and extract pixel points that meet the conditions of the following formula;

[0011] R>G>B

[0012] R>R t

[0013] wherein, R, G, and B respectively represent the three components of color, and R t is a set threshold;

[0014] Further, R t takes the value of 180.

[0015] Step 2.2: Calculate the numerical relationship between the differences T1, T2, and T3 among the three components of R, G, and B, and their definitions are as described in the following formula:

[0016] T1 = |R - G|, T2 = |G - B|, T3 = |R - B|

[0017] Step 2.3: Use the following formula to supplement the extraction of flame features and extract pixel points that meet the conditions of the following formula;

[0018] T3 > (T1, T2) max

[0019] T1 < 100

[0020] T3 > 130

[0021] Step 2.4: Use the following step1 to judge the pixel points;

[0022]

[0023] If the conditions of step1 are met, the detected pixel points are considered flame pixel points, otherwise they are defined as suspected flame pixel points; || represents the logical or relationship, and in this step, it means that one of the judgment conditions is that B is less than 80 or T3 is greater than 130;

[0024] Use the following step 2 to judge the suspected flame pixel points;

[0025]

[0026] If the conditions of step 2 are met, the detected pixel points are defined as flame pixel points; otherwise, they are defined as non-flame pixel points;

[0027] Among them, R t1 、R t2 are set thresholds, H represents hue, S represents saturation, and (R, G, B) average represents the average value of the three components of R, G, and B;

[0028] Step 3: According to the result of step 2, the flame area is obtained. Set this flame area as the target area to be recognized, and judge whether the target area to be recognized is a flame based on the flame area contour;

[0029] Step 3.1: Exclude the target areas to be recognized that do not meet , where L represents the perimeter of the area where the object is located, and S represents the area of the area where the object is located;

[0030] Step 3.2: Judge among the remaining target areas to be recognized in step 3.1. The target area to be recognized that meets is the flame area, S O represents the object area, and S R represents the area of the smallest rectangle containing the area where the object is located;

[0031] Step 4: Extract the dynamic features of fireworks;

[0032] Step 4.1: Initialize the background model, and its definition is as follows:

[0033] M(x, y) = r 1 (x, y), r 2 (x, y), …, r n (x, y)

[0034] Among them, M(x, y) represents the sample set storing the position of each pixel point, and r n (x, y) represents the pixel value of the target object; after initializing the background model for the first video frame, perform moving object detection on each subsequent frame in turn, and define the Euclidean distance F n between the detection point and the nth sample as:

[0035]

[0036] Among them, TH r is the set threshold for judging the distance between the pixel value r(x, y) of the target object and the background model M(x, y);

[0037] Further, TH r takes the value of 20.

[0038] Step 4.2: Determine whether the pixel value at this position is the foreground f(x, y):

[0039]

[0040] where N is the number of samples in the background model, f(x, y) being 1 indicates it is the foreground, and being 0 indicates it is the background, and TH min is the set threshold;

[0041] Further, N takes the value of 20, and TH min takes the value of 2.

[0042] Step 4.3: Update the background model; if the point P(x, y) is determined to be a background point, then replace one sample in the M(x, y) sample set with the point P(x, y) with a probability of 1 / w, where w is the set update frequency; at the same time, count the points determined to be the foreground in consecutive detections. If there is a certain pixel point that has been continuously detected and determined to be the foreground k times, then its background will be updated once.

[0043] Further, the sample update frequency w is 50, and the threshold k for the number of consecutive foreground detections is 30.

[0044] Step 5: Multi-feature fusion;

[0045] Step 5.1: Generate binary images from the results obtained in Step 2, Step 3, and Step 4 respectively;

[0046] Step 5.2: Perform an AND judgment on the pixel values of the overlapping pixels of each binary image. If the pixel values are the same, then keep the pixel value of this point at the same position on the new binary image. Otherwise, set a point with a pixel value of 0 at the same position on the new binary image; obtain a new binary image, and the new binary image is the flame detection result.

[0047] Advantages of the present invention: The present invention proposes a forest fire recognition method that fuses multiple features. Aiming at the image quality problems existing in existing forest fire video monitoring devices, Gaussian filtering, mean filtering, and saturation adjustment are selected as preprocessing algorithms. The color, shape, and dynamics are designed and extracted as the fireworks features. The fireworks color features in multiple color spaces such as RGB and HSV are extracted, and the efficient ViBe algorithm is used to extract the fireworks dynamic features. The present invention enhances the quality of video frames, uses the method of multi-feature fusion to achieve forest fire detection, and improves the accuracy and precision of forest fire point detection. Description of the Drawings

[0048] Figure 1 It is the effect diagram of Gaussian noise reduction in the image preprocessing stage. The left figure is the original image, and the right figure is the output image after Gaussian noise reduction.

[0049] Figure 2 It is the effect diagram of mean filtering in the image preprocessing stage. The left figure is the original image, and the right figure is the output image after mean filtering.

[0050] Figure 3 It is the effect diagram of saturation adjustment in the image preprocessing stage. The left figure is the original image, and the right figure is the output image after saturation adjustment.

[0051] Figure 4 The left figure is a yellow flame, and the right figure is a red flame.

[0052] Figure 5 It is the effect diagram of detection after adding the contour feature C. From left to right, they are the original image, the binary image with the contour feature C added, and the binary image without the contour feature C added.

[0053] Figure 6 It is the effect diagram of detection after adding the contour feature R. From left to right, they are the original image, the binary image with the contour feature R added, and the binary image without the contour feature R added.

[0054] Figure 7 It is the result diagram of ViBe fireworks dynamic feature detection. The left figure is the original image, and the right figure is the binary image of the smoke dynamic feature.

[0055] Figure 8 It is the schematic diagram of the AND operation.

[0056] Figure 9 It is the flow chart of the forest fire detection algorithm based on multi-feature fusion. Specific implementation manners

[0057] The present invention will be further described below with reference to the accompanying drawings.

[0058] (1) Image preprocessing

[0059] To ensure that during feature recognition, the fireworks feature extraction algorithm of the present invention is not interfered by Gaussian noise and salt-and-pepper noise generated during image shooting and transmission, it is necessary to perform filtering on these two types of noise during the preprocessing stage. Forest fire video surveillance equipment requires a wide field of view, no obstacles, and a large monitoring angle. Therefore, it is generally installed in an open area at the highest point or halfway up the mountain in the forest to achieve the largest forest area coverage for monitoring. However, this installation method will cause the monitoring equipment to be exposed to the sun for a long time, and the high temperature will cause Gaussian noise in the image sensor. The existence of this noise will greatly reduce the image clarity and interfere with subsequent feature extraction. To effectively eliminate Gaussian noise, the present invention uses Gaussian filtering for image denoising, and sets the convolution kernel size to 3*3. Subsequently, since the forest fire video images collected by the monitoring equipment are generally transmitted back to the monitoring center by wireless transmission or fiber optic transmission, during the transmission process and video decoding process, the image is prone to salt-and-pepper noise. Because this noise appears as flickering black and white pixel points, it not only reduces the image clarity, but the white pixel points will also be confused with white smoke, thus interfering with feature extraction. To effectively eliminate salt-and-pepper noise, the present invention uses mean filtering for image denoising. Set the convolution kernel size to 3*3. In addition, to improve the accuracy of forest fire feature extraction, it is necessary to enhance the color of the forest fire image. The present invention uses saturation adjustment to enhance the image.

[0060] To avoid fireworks recognition errors and improve recognition accuracy, it is necessary to preprocess the forest fire video images collected by the monitoring equipment. The image preprocessing process includes three steps: Gaussian denoising, mean denoising, and saturation adjustment.

[0061] To effectively eliminate Gaussian noise, the present invention uses Gaussian filtering for image denoising, and sets the convolution kernel size to 3*3. The specific denoising effect is as Figure 1 shown.

[0062] To effectively eliminate salt-and-pepper noise, the present invention uses mean filtering for image denoising. Set the convolution kernel size to 3*3. The specific denoising effect is as Figure 2 shown.

[0063] To enhance the color of the forest fire image, the present invention uses saturation adjustment to enhance the image. The specific enhancement effect is as Figure 3 .

[0064] (2) Fireworks color feature extraction based on multiple spaces

[0065] To effectively verify this RGB criterion, the present invention conducted a statistical experiment on the flames in the dataset and used Matlab software to analyze the RGB components of the flames. According to statistics, the flames can be divided into yellow flames and red flames by color. The flame screenshots are as Figure 4 shown. For Figure 4Analyze the color components of the pixels in the middle flame area, mainly analyze its R, G, and B components, and extract the flame features with the following formula:

[0066] R > G > B

[0067] R > R t

[0068] In the present invention, the value of R t is 180. In order to make full use of the RGB color space, this article also analyzes the numerical relationship between the differences T1, T2, and T3 among the three components of R, G, and B. Their definitions are as described in the following formula:

[0069] T1 = |R - G|, T2 = |G - B|, T3 = |R - B|

[0070] Similarly, for Figure 4 the pixels in the middle flame area, analyze the color components, mainly analyze its T1, T2, and T3 components, and perform supplementary extraction of flame features through the following formula:

[0071] T3 > (T1, T2) max

[0072] T1 < 100

[0073] T3 > 130

[0074] After image preprocessing, the present invention extracts fireworks features from three aspects: color, shape, and dynamics. First, extract the color features of the flame and smoke. The HSV color space is a representation method of points in the RGB color space in an inverted cone. In order to reduce interference and accurately extract the color features of fireworks, the present invention adds the HSV color space on the basis of the RGB color space to exclude some interfering substances and improve the accuracy, and reduces the computational complexity of the algorithm in pixel extraction. Considering that common flame detection interfering substances include forest rangers, red leaves, the setting sun, car lights, etc., which are mainly similar to the flame in color. Through the component analysis of the interfering substances in the RGB and HSV color spaces and combined with a large number of experimental statistics, the flame judgment basis in the RGB combined with the HSV color space is as shown in the following two formulas:

[0075]

[0076] After reading the video frame, first go through step1, which is used for the initial extraction of flame features. If step1 judges positively, the detected pixel points are defined as flame features; otherwise, they are defined as suspected flame pixel points and passed to step2. Step2 is used for the secondary extraction of flame features. If step2 judges positively, the detected pixel points are defined as flame features; otherwise, they are defined as non-flames.

[0077] (3) Extraction of Firework Shape Features Based on Contour

[0078] According to the definition of the contour shape feature C:

[0079]

[0080] Therefore, a threshold of C > 2 is selected for discrimination. The detection effect Figure 5 is shown as follows.

[0081] According to the definition of the contour shape feature R:

[0082]

[0083] A range of 0.1 ≤ R ≤ 0.9 is taken to distinguish fireworks from interfering objects. The detection effect is as Figure 6 shown as follows.

[0084] (4) Extraction of Firework Dynamic Features Based on ViBe

[0085] First, initialize the background model, and its definition is shown as the following formula:

[0086] M(x, y) = r 1 (x, y), r 2 (x, y), …, r n (x, y)

[0087] After initializing the background model for the first video frame, perform moving object detection on subsequent frames in sequence. Define the Euclidean distance F between the detection point and the nth sample n as:

[0088]

[0089] where the threshold TH r is taken as 20 in the present invention to determine the distance between the pixel value r(x, y) of the target object and the background model M(x, y). Subsequently, determine whether the pixel value at this position is the foreground f(x, y):

[0090]

[0091] where N is the number of samples in the background model, and f(x, y) being 1 indicates the foreground, and being 0 indicates the background. In the present invention, the neighborhood is selected as 8, the number of samples N of the background model is taken as 20, and TH min is taken as 2.

[0092] Subsequently, background model update is performed. If the point P(x, y) is determined to be a background point, then the point P(x, y) replaces a sample in the M(x, y) sample set with a probability of 1 / w. At the same time, the points determined to be foreground in consecutive detections are counted. If there is a certain pixel point that is determined to be foreground in consecutive detections for k times, its background will be updated once. The sample update frequency w is selected as 50, the radius R is 20, and the foreground consecutive detection times threshold k is taken as 30. The detection effect is as Figure 7 shown.

[0093] (5) Multi-feature fusion and algorithm implementation

[0094] Multi-feature fusion performs an AND operation on the feature binary maps obtained by multiple feature extraction methods and generates a new binary map to maintain pixel points: overlapping pixels will perform a pixel value AND judgment. If the pixel values are the same, the pixel value of the point is maintained at the same position in the new binary map; otherwise, a point with a pixel value of 0 is set at the same position in the new binary map, as Figure 8 shown. The overall flowchart of the present invention is as Figure 9 shown.

Claims

1. A forest fire detection method integrating multiple features, the method comprising: Step 1: Collect forest farm video images and pre-process the images; Step 2: Extract flame pixels based on color features; Step 2.1: Use the following formula to extract flame features and extract pixel points that meet the following conditions; R>G>B R>R t Among them, R, G, and B represent the three components of color respectively. t To set the threshold; Step 2.2: Calculate the numerical relationship between the differences T1, T2, and T3 between the three components of R, G, and B, which are defined as follows: T1=|RG|, T2=|GB|, T3=|RB| Step 2.3: Use the following formula to perform supplementary extraction of flame features and extract pixel points that meet the following conditions; <h2 style=";text-align:left;direction:ltr">T3>(T1,T2)<h2 style=";text-align:left;direction:ltr"> max T1<100 T3>130 Step 2.4: Use the following formula step 1 to judge the pixel point; If the condition of step 1 is met, the detected pixel is considered to be a flame pixel, otherwise it is defined as a suspected flame pixel; || represents a logical or relationship. In this step, it means that the judgment condition is that B is less than 80 or T3 is greater than 130; The following formula step 2 is used to judge the suspected flame pixels; If the condition of step 2 is met, the detected pixel is defined as a flame pixel, otherwise it is defined as a non-flame pixel; Among them, R t1 , R t2 is the threshold value to be set, H represents hue, S represents saturation, (R, G, B) average Represents the average value of the three components R, G, and B; Step 3: According to the result of step 2, the flame area is obtained, and the flame area is set as the target area to be identified, and whether the target area to be identified is a flame is determined based on the contour of the flame area; Step 4: Based on dynamic feature recognition of fireworks; Step 4.1: Initialize the background model, which is defined as follows: M(x,y)=r1(x,y),r2(x,y),…,r n (x,y) Among them, M(x,y) represents the sample set that saves the position of each pixel, r n (x, y) represents the pixel value of the target object; after completing the background model initialization for the first video frame, the moving object detection is performed on the subsequent frames in turn, and the Euclidean distance F between the detection point and the nth sample is defined n for: Among them, TH r is the set threshold used to determine the distance between the pixel value r(x,y) of the target object and the background model M(x,y); Step 4.2: Determine whether the pixel value at this position is foreground f(x, y): Where N is the number of samples in the background model, f(x,y) is 1 for foreground and 0 for background, TH min is the set threshold; Step 4.3: Update the background model; if point P(x,y) is judged as a background point, then point P(x,y) will replace a sample in the M(x,y) sample set with a probability of 1 / w, where w is the set update frequency; at the same time, count the points judged as foreground in continuous detection. If a pixel point is judged as foreground k times in continuous detection, the background will be updated once. Step 5: Multi-feature fusion; Step 5.1: Generate binary images from the results of step 2, step 3, and step 4 respectively; Step 5.2: Perform AND judgment on the pixel values ​​of the overlapping pixels of each binary image. If the pixel values ​​are the same, keep the pixel value of the point at the same position on the new binary image. Otherwise, set a point with a pixel value of 0 at the same position on the new binary image. Get a new binary image, which is the flame detection result.

2. A forest fire detection method integrating multiple features as claimed in claim 1, characterized in that: The preprocessing method in step 1 is: firstly, Gaussian filtering is used to reduce image noise on the video image, then mean filtering is used to reduce image noise, and finally saturation adjustment is used to enhance the image.

3. A forest fire detection method integrating multiple features as claimed in claim 1, characterized in that: In step 2.1, R t The value is 180.

4. A forest fire detection method integrating multiple features as claimed in claim 1, characterized in that: The specific method of step 3 is: Step 3.1: Eliminate unsatisfactory The target area to be identified, where L represents the perimeter of the area where the object is located, and S represents the area of ​​the area where the object is located; Step 3.2: Determine in the remaining target area to be identified in step 3.1 whether The target area to be identified is the flame area, S O Represents the area of ​​the object, S R Indicates the area of ​​the smallest rectangle that contains the object.

5. A forest fire detection method integrating multiple features as claimed in claim 1, characterized in that: In step 4.1, r The value is 20.

6. A forest fire detection method integrating multiple features as claimed in claim 1, characterized in that: In step 4.2, N is set to 20, TH min The value is 2.

7. A forest fire detection method integrating multiple features as claimed in claim 1, characterized in that: In step 4.3, the sample update frequency w is 50, and the threshold value k of the number of consecutive foreground detections is 30.

Images