A fire detection method based on BES-BP classifier and space-time visual features

Abstract

The application discloses a fire detection method based on a BES-BP classifier and space-time visual features. The method comprises the following steps: using a first frame and a second frame of fire images, determining a candidate fire area through red and yellow flame colors, and completing preprocessing to obtain a binary image by applying a maximum inter-class variance threshold algorithm Otsu; based on a flame motion detection technology of motion intensity perception, extracting motion intensity information and a motion area containing motion pixels; extracting four space-time features and two space features. The weights of a BP neural network are optimized by using a vulture algorithm to construct a BES-BP classifier for distinguishing between a real fire area and a non-real fire area. Compared with the prior art, the application realizes monitoring of a fire by automatically detecting flame video information shot by a camera, uses efficient space-time visual features to reliably identify fire flames in different situations, and ensures that the detection of the fire is accurately and quickly completed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fire detection, and particularly relates to a fire detection method based on a BES-BP classifier and a space-time visual feature. BACKGROUND

[0002] Fire is a kind of natural or man-made disaster with strong destructive power. Rapid and accurate detection of fire can minimize personnel casualties and property losses, and plays a vital role. At present, various existing fire detection methods generally use temperature, smoke or photoelectric sensors to complete the detection. However, these methods generally require sensors to be close to the fire source, and have high requirements for the heat resistance of sensors. Therefore, these methods cannot be used for detection in outdoor environments or large spaces. Video monitoring based on computer vision is a method that can solve the above problems. By using a single monitoring camera and advanced image and video processing technology, we can detect fire more quickly without strict requirements for sensors.

[0003] Fire detection methods based on computer vision mainly include color-based and motion-based detection methods. The color-based detection method mainly considers the obvious color features of the fire area. For example, the flame is generally in the red-yellow color range. The disadvantage of these methods is that they are very sensitive to light changes and similar fire objects in the fire scene, which leads to detection errors caused by similar colors due to different shades of red and yellow. On the other hand, the motion-based detection method uses the chaotic motion of the flame to effectively distinguish from other moving objects in the scene. Therefore, space-time motion-based features such as motion direction, dynamic texture and optical flow can be used to detect the flame. However, in some cases, for example, when there are other moving objects in the scene, methods that only use motion information have limitations in application. In order to improve the accuracy of fire detection, color and motion features are generally combined for detection. Generally, these methods need to identify the color features of the candidate fire area first, and then evaluate the candidate area to classify it into real fire area and non-real area. Although these methods that use both color and motion have achieved certain detection effect, they generally lack robustness.

[0004] These methods have two limitations. First, the traditional color thresholding technique or color segmentation method for detecting candidate fire regions is not robust enough, so it may inject a large amount of false data into the classifier, thereby reducing the effect of the classification stage; second, a large amount of irregular motion information poses a classification challenge to the classification algorithm in machine learning. In recent years, fire and flame detection methods based on convolutional neural networks have been proposed, however, these conventional deep learning-based methods usually require more computing time and high memory, and the limitations of traditional hardware limit the application of the method. Therefore, it is a challenging task to develop a reliable and efficient fire detection system. SUMMARY

[0005] The purpose of the application is to solve the problems pointed out in the background art, and the application proposes a fire detection method based on BES-BP classifier and spatiotemporal visual features, which realizes the monitoring of fire by automatically detecting flame video information shot by a camera, uses efficient spatiotemporal visual features to reliably identify fire flames in different situations, and ensures accurate and rapid detection of fire.

[0006] Technical scheme: The application proposes a fire detection method based on BES-BP classifier and spatiotemporal visual features, comprising the following steps:

[0007] Step (1) determines the candidate fire region and completes the preprocessing;

[0008] Step (2) uses a flame motion detection technology based on motion intensity perception to extract motion intensity information and motion regions containing motion pixels;

[0009] Step (3) for each candidate fire region, a set of complementary features are extracted, including four spatiotemporal features and two spatial features, the spatiotemporal features include two surface spatiotemporal features and two boundary spatiotemporal features, the surface spatiotemporal features are surface motion rate SMR and motion intensity rate MIR; the boundary spatiotemporal features are random motion value feature SMV and random motion direction feature SMD; the two spatiotemporal features are color change rate CVR and boundary roughness rate BRR;

[0010] Step (4) uses the BES algorithm to optimize the weights of the BP neural network to construct the BES-BP classifier, creates a six-dimensional feature vector using the six features extracted from the candidate fire region in step (3), and effectively distinguishes the fire region using the BES-BP classifier.

[0011] Further, the step (1) is specifically as follows:

[0012] First, the flame region in the input video frame picture is identified by flame color, the color of the flame varies from red to yellow, the pixels containing the color range are selected as the candidate fire region, then pre-processing is performed, and the maximum class variance threshold algorithm Otsu is applied to obtain a binary image.

[0013] Further, the flame motion detection technology based on motion intensity perception in step (2) extracts the motion intensity information and motion region containing the motion pixels, and the specific steps are as follows:

[0014] Step (2.1): Calculate the difference image between two frames, calculate the corresponding pixels on R, G, and B channels respectively, and finally obtain the difference image S on R, G, and B channels R (x,y) and S G (x,y) and S B (x,y);

[0015] Step (2.2): Calculate the pixel mean value μ of the difference image on R, G, and B channels of the difference image R , μ G , μ B ; for a difference image with size m x n, calculate the average value on RGB channels as follows:

[0016]

[0017]

[0018]

[0019] Step (2.3): Calculate the motion intensity map M(x,y):

[0020] M(x,y)=((S R (x,y)-μ R )+(S G (x,y)-μ G )+(S B (x,y)-μ B )) / 3

[0021] Step (2.4): Calculate the expected motion intensity map I(x,y) after denoising by thresholding algorithm;

[0022]

[0023] Where I(x,y) is the expected motion intensity map, and α is the threshold value, α is in the range of [3, 8];

[0024] Step (2.5): Calculate the binary motion image B(x, y), which is obtained by performing hard threshold processing on the image I(x, y) to obtain the corresponding binary image B(x, y).

[0025] Further, in step (3), for each candidate fire area, a set of mutually complementary features are extracted, and the specific steps are as follows:

[0026] Step (3.1): Extract the surface spatiotemporal feature, i.e., the surface motion rate SMR and the motion intensity rate MIR;

[0027] Step 1: Extract the SMR feature:

[0028]

[0029] where A is the area of the region being processed, i.e., the number of pixels within the region, and B is the number of moving pixels within the motion image, considering the moving surface feature of the fire, the SMR of the fire area is greater than that of the related area of the non-fire moving object;

[0030] Step 2: Extract the MIR feature

[0031]

[0032] where A is the area of the region being processed, and I is the sum of the motion intensity values of the pixels within the region being processed; the MIR of the fire area is greater than that of the related area of the non-fire moving object;

[0033] Step (3.2): Extract the boundary spatiotemporal feature, i.e., the random motion value feature SMV and the random motion direction feature SMD;

[0034] Step 1: Select the study point, the boundary spatiotemporal feature is calculated by analyzing the position and direction of the boundary of the candidate fire area over time, and eight points are selected on each corner and the center of the four sides of the boundary of the region;

[0035] Step 2: Construct the set of motion vectors V

[0036] In order to analyze the motion behavior of the above selected points, the positions of the two corresponding boundary study points are specified and the motion vectors are measured, and the following mathematical formula is defined to describe them:

[0037] For a given region, define eight points in two frames as follows:

[0038]

[0039] where pf t and pf t1 are the f tFrame and f t1 The set of points studied on the frame boundary, for pf t Each point in Calculate the corresponding pf t1 The closest point The specific calculation formula is as follows:

[0040]

[0041] Where |·| is the Euclidean distance. It is the fth t The m-th point in the frame, It is the fth t1 The i-th point in the frame, It is the fth t1 Mid-frame distance The nearest point is used to determine eight corresponding points in the two frames, and these are used to construct a set V of eight motion vectors, as shown below:

[0042] V = {v1, v2, ..., v8}

[0043] in,

[0044] Step 3: Extract SMV features

[0045] Let X = {x1, x2, x3, ..., x} n} represents the dataset, n is the number of data points, and M is the average value of the dataset. X and standard deviation SD X The details are as follows:

[0046]

[0047]

[0048] Calculate the motion value for each motion vector and input it into the set MV:

[0049] MV = {mv1, mv2, ..., mv8}

[0050] in, respectively at the f t Frame and f t1 The x and y coordinates of the i-th point in the frame; this set represents the amount of movement through each part of the region boundary, and the standard deviation SD of the MV set is calculated. MV This is used as an SMV feature; the SMV value of the fire zone is greater than the corresponding value of the non-fire zone;

[0051] Step 4: Extract SMD features

[0052] The motion direction of each motion vector in V is calculated and input into a set MD, which is defined as follows:

[0053] MD = {md1, md2, …, md8}

[0054] wherein, are the x and y coordinates of the i-th point in the f t frame and the f t1 frame, respectively; the standard deviation SD of the set MD is calculated and taken as the SMD feature; the SMD of the fire region is greater than the corresponding value of the non-fire region; MD

[0055] Step (3.3): Extracting spatial features, i.e. color variation rate CVR and boundary roughness rate BRR;

[0056] Step 1: Extracting CVR feature

[0057] The color variation amount is used as a feature to distinguish fire and non-fire regions; for each pixel value in the fire region, the standard deviation of each pixel value in the three color channels RGB is calculated, and then the corresponding average value is calculated and taken as the color variation rate feature CVR, which is higher in the fire region than in the non-fire region;

[0058] Step 2: Extracting BRR feature

[0059] For a given region, the number of convex hull pixels P CH of a set of pixels is calculated, which is the smallest convex set containing the set of pixels, and then the BRR feature of the region is calculated as follows:

[0060]

[0061] wherein, P is the number of peripheral pixels of the region, and P CH is the corresponding number of convex hull pixels.

[0062] Further, in the step (4), the bald eagle algorithm BES is used to optimize the weight of the BP neural network to construct a BES-BP classifier, and the fire features extracted from the candidate fire region are input into the BES-BP classifier to effectively distinguish the real fire region and the non-real fire region, and the specific steps are as follows:

[0063] Step (4.1): Implementing the bald eagle search algorithm BES, which includes the selection of the search space stage, the search of the prey stage and the capture of the prey stage;

[0064] ​Step (4.2): Constructing BES-optimized BP neural network, introducing BES algorithm to optimize the weights of BP neural network, the specific steps are as follows:

[0065] Step 1: Construct and initialize a three-layer BP neural network

[0066] The number of layers of BP neural network is 3, i.e. input layer, output layer and hidden layer, full connection is adopted between each layer; Sigmoidal function is adopted as the activation function of each hidden node, and hard limit function is adopted as the activation function of output node; the number of input layer nodes is designed to be 6, representing 6 fire characteristics, the number of hidden layer nodes is 15, and the number of iterations is 100 times;

[0067] Step 2: Initial BES algorithm, and map it to the weights of BP network; first, initialize the individual position of each hawk to a random number, and map it to the weights of BP network;

[0068] Step 3: Take the optimal position of the hawk as the weights of BP network, and update the output of BP network;

[0069] The fitness function E defined in (1) is used to calculate the individual optimal value of each hawk, which is the mean square error between the actual output and the target output of the network, and is taken as the connection weight into the calculation formula of BP neural network defined in (2)-(3) to calculate the output of the hidden layer nodes and the output layer nodes of the network respectively;

[0070] (1) Define the fitness function, i.e. error E

[0071]

[0072] Where y is the actual output, and d is the target value;

[0073] (2) Hidden node

[0074]

[0075] Where x j is the output of the jth hidden node, p i is the ith input, v ij is the connection weight from the ith input to the jth hidden node, r is the number of input nodes, k is the number of hidden nodes, θ j is the threshold value of the jth hidden node, and f(x) is the activation function of the hidden node;

[0076] (3) Output node

[0077]

[0078] Where y lw is the weight of the connection from the jth hidden node to the lth output node, n is the number of output nodes, k is the number of hidden nodes, and θ jl w is the weight of the connection from the jth hidden node to the lth output node, n is the number of output nodes, k is the number of hidden nodes, and θ l is the threshold value of the lth output node.

[0079] Step 4: Update the optimal value of the vulture, and compare the current optimal value of the vulture individual with the global optimal value according to the fitness function;

[0080] Step 5: Check the end condition, if the number of stacking times is greater than the maximum number or the evaluation error value is greater than a given value, stop iteration and go to Step 6, otherwise, go to Step 3 to update the state of the vulture for a new round;

[0081] Step 6: Save the global optimal value.

[0082] Beneficial effects:

[0083] The application utilizes a flame motion detection technology based on motion intensity perception, simplifies the extraction process of the fire motion region, obtains the motion intensity rate of each motion pixel and the motion region, and uses the same in the extraction of the features of the fire. In addition, four space-time features and two space features are extracted from the fire video frame data, the space-time features include the surface space-time features obtained by using the motion pixel information and the boundary space-time features extracted by using the boundary change of the flame region. Finally, the weight value of the BP neural network is optimized by using the vulture algorithm BES, the design idea of the BES-BP classifier is proposed, and the fire features extracted from the candidate fire region are input into the BES-BP classifier, so that the real fire region and the non-real fire region can be effectively distinguished. The application carries out experimental verification on six video fire data sets in the VisiFire and AzarFire data sets, and the experimental results show that the proposed method can accurately classify the fire picture state in most data sets, and the highest classification rate reaches 99.12%. Therefore, the method is a fire detection method with high detection accuracy, and can provide high reliability and low error rate detection results for different detection environments. BRIEF DESCRIPTION OF DRAWINGS

[0084] Figure 1 is the implementation process of the fire detection method based on the BES-BP classifier and the space-time visual features;

[0085] Figure 2 is an example of the positions of the selected eight research points in the fire sample region;

[0086] Figure 3 is the search principle of the vulture search algorithm BES;

[0087] Figure 4is the implementation flow of the bald eagle search algorithm BES;

[0088] Figure 5 is the topological structure diagram of the constructed three-layer BP network;

[0089] Figure 6 is the flow chart of the BES-optimized BP neural network training algorithm;

[0090] Figure 7 is a part of fire video frame sample pictures. DETAILED DESCRIPTION

[0091] In order to better explain the present application, so as to be understood, the technical solutions of the present application are described in detail below. The following examples are an explanation of the present application, and the present application is not limited to the following examples.

[0092] In view of the shortcoming that the conventional fire detection method has high requirements for sensor heat resistance, the patent proposes a fire detection method based on BES-BP classifier and spatiotemporal visual features, uses a flame movement detection technology based on movement intensity perception to extract movement intensity information and movement regions containing movement pixels, extracts four spatiotemporal features and two spatial features, wherein the spatiotemporal features include surface spatiotemporal features and boundary spatiotemporal features, finally, the weights of the BP neural network are optimized by using the bald eagle algorithm (BES), the BES-BP classifier is constructed, and is used to effectively distinguish fire regions.

[0093] Figure 1 The implementation process of the fire detection method based on the BES-BP classifier and the spatiotemporal visual features in the embodiment of the present application is shown.

[0094] The purpose of the method proposed in the present application is to detect each fire region in the given frame of the input video by analyzing the spatiotemporal features of the fire, the spatial features can be extracted from the processing frame, but to analyze the temporal features, other frames in the input fire video should be used. In order to make the proposed system as simple as possible and reduce the computational complexity of the whole algorithm, the patent only uses one frame with a reasonable interval from the processing frame. Therefore, in order to identify the fire region in the t th frame, the t frame is also used, which is called f t and f t1 , and experiments are conducted to verify the state of fire and non-fire using video frames with different time intervals.

[0095] In order to detect the fire region in the t th frame, the f t and f t1 frames in the fire video are extracted, and the following operations are performed: (1) the f t frame and the ft1 (2) Fire images of frames are used to determine candidate fire areas by the flame color from red to yellow, and preprocessing is performed. The Otsu maximum class variance thresholding algorithm is applied to obtain binary images. (3) Motion detection technology based on motion intensity perception is used to extract f. t In addition to detecting moving pixels, this method can also extract the motion intensity of each pixel in the frame. (3) Using the f-th... t Frame and f t1 Candidate fire areas in the frame, and the fth t Motion information of moving pixels in the frame is used to extract the f-th pixel. t The fire spatiotemporal features in the frame include surface spatiotemporal features and boundary spatiotemporal features. (4) The extracted features are fed into the BES-BP classifier to effectively distinguish between real fire areas and non-real fire areas. Finally, the training set of video frames of fire and non-fire samples is used to complete the training and testing of the fire. The specific steps are as follows:

[0096] Combination Figure 1 A fire detection method based on BES-BP classifier and spatiotemporal visual features includes the following steps:

[0097] Step 1: Determine the candidate fire areas and complete the preprocessing.

[0098] First, flame regions in the input frame are identified, and candidate fire regions are determined by the color of the flames. Generally, the color of flames ranges from red to yellow; therefore, pixels in the image containing colors within this range are selected as candidate fire regions. Then, preprocessing is performed, and the Otsu's maximum class variance thresholding algorithm is applied to obtain a binary image.

[0099] Step 2: Propose a flame motion detection technology based on motion intensity sensing.

[0100] Step (2)a: Calculate the difference image S between the two frames. R (x,y),S G (x,y),S B (x,y)

[0101] To extract motion-based fire features, it is necessary to detect a given frame f t For moving pixels in a dataset, common motion detection methods, such as background subtraction, can be used to identify these moving pixels. However, these methods generally require a learning phase and involve extracting background information, and they are highly sensitive to noise and lighting changes. The proposed method introduces a simple and efficient motion intensity-aware motion pixel detection algorithm, described as follows:

[0102] f t Frame and f t1The frame is used as the input frame in order to detect the f-th frame. t For moving pixels in a frame, first, calculate the distance S(x,y) between two frames, i.e., the difference image, as follows:

[0103] S(x,y)=|f t1 (x,y)-f t (x,y)| (1)

[0104] In this process, the corresponding pixels are calculated on the R, G, and B channels respectively, and the final difference image S on the R, G, and B channels is obtained. R (x,y),S G (x,y),S B (x,y).

[0105] Step (2)b: Calculate the average value μ of the difference image. R μ G μ B

[0106] For a difference image of size m×n, calculate its average value across the RGB channels as follows:

[0107]

[0108]

[0109]

[0110] Where, μ R μ G μ B It is the pixel mean of the difference image on the R, G, and B channels.

[0111] Step (2)c: Calculate the motion intensity map M(x,y)

[0112] Next, the mean image M(x,y) is calculated, as follows:

[0113] M(x,y)=((S R (x,y)-μ R )+(S G (x,y)-μ G )+(S B (x,y)-μ B )) / 3 (3)

[0114] The resulting image M(x,y) represents the changes in motion pixels and their corresponding intensities over time. Larger pixel values ​​indicate more color changes during motion. Therefore, this image is designated as a motion intensity map.

[0115] Step (2)d: Calculate the denoised expected motion intensity map I(x,y)

[0116] In order to eliminate the noise in the image M(x,y) caused by small camera motion or lighting, the following thresholding algorithm is used to filter out the noise:

[0117]

[0118] Where I(x,y) is the expected motion intensity map, and a is the threshold. A smaller a value will make the algorithm very sensitive to lighting and camera vibration changes, and a larger a value will cause some valuable real motion data to be lost. Through experimental analysis of various fire videos during implementation, it is found that a value in the range of [3, 8] has better results, so a = 5 is set.

[0119] Step (2)e: Calculate the binary motion image B(x,y)

[0120] Finally, the motion pixels I(x,y) are represented as a binary image, and the corresponding binary image B(x,y) is obtained by performing hard thresholding on the image I(x,y). The specific algorithm is as follows:

[0121]

[0122] Using the above method, motion pixels can be extracted, and information about the motion intensity of the pixels (motion intensity map) can also be provided, which is helpful for the extraction of fire spatiotemporal features in the next stage.

[0123] Step 3, a set of complementary features are extracted, including four spatiotemporal features and two spatial features. Among them, the spatiotemporal features include surface spatiotemporal features and boundary spatiotemporal features. The calculation steps of the six features are as follows:

[0124] Step (3)a: Extract surface spatiotemporal features

[0125] When using the motion intensity perception strategy to detect moving motion pixels, in addition to obtaining motion pixels, information about the change in pixel intensity caused by motion can also be provided, and using this information, surface spatiotemporal features can be calculated. Using the motion intensity map I(x,y) and binary image B(x,y) calculated by the above equations (4) and (5), two surface spatiotemporal features, namely surface motion rate SMR and motion intensity rate MIR, are calculated. The specific calculation steps are as follows:

[0126] Step 1: Extract SMR feature

[0127] Based on the characteristics of the chaotic movement of the surface of the fire, the surface motion rate SMR feature is introduced, which only uses the motion pixels detected in the candidate fire area. The SMR feature is defined as follows:

[0128]

[0129] where A is the area of the region, i.e. the number of pixels in the region, and B is the number of moving pixels in the region obtained using equation (5). Considering the moving surface feature of fire, the SMR of a fire region is larger than that of a non-fire moving object.

[0130] Step 2: Extracting MIR feature

[0131] In addition to the moving surface of a fire region, the surface of a fire region usually has many intensity changes. Inspired by this fire visual feature, the motion intensity rate (MIR) feature is defined, which is calculated as follows:

[0132]

[0133] where A is the area of the region, and I is the sum of the motion intensity values of the pixels in the region, which can be calculated from the motion intensity map in equation (6). The MIR of a fire region is larger than that of a non-fire moving object.

[0134] Step (3)b: Extracting boundary spatiotemporal feature

[0135] Step 1: Selecting study points

[0136] The boundary spatiotemporal feature is calculated by analyzing the position and direction of the boundary of a candidate fire region changing over time (between the f t frame and the f t1 frame), because the boundary of a fire is usually rough, and different parts of a fire generally move in different directions and with different amounts of movement, while most non-fire moving objects generally move in a regular pattern and at a constant speed, and their movement direction is limited, especially rigid objects such as vehicles, which usually move in the same direction at the same speed. Therefore, in order to analyze the movement behavior feature of each region, some points on the boundary of each region are selected for study, specifically, eight points are selected from the four corners and the centers of the four sides of the region.

[0137] Figure 2 An example of the positions of the eight selected study points in a fire sample region.

[0138] Figure 2 In the figure, the blue dots are the positions of the selected study points on the boundary of the f t frame. In the specific implementation, the same selection operation is performed on the f t frame and the f t1 frame to obtain the desired points on the boundaries of the two candidate regions, and the corresponding points of the same region in the two frames are used to analyze the changes in the boundary of the region.

[0139] Step 2: Construct the set V of motion vectors

[0140] To analyze the motion behavior of the selected points, specify the positions of the study points on the two corresponding boundaries, and measure their motion vectors, the following mathematical formula is defined to describe it.

[0141] For a given region, define eight points in the two frames as follows:

[0142] pf t ={p1 t p2 t ...,p8 t},pf t1 ={p1 t1 p2 t1 ...,p8 t1} (8)

[0143] Among them, pf t and pf t1 They are respectively the fth t Frame and f t1 The set of points studied on the frame boundary. For pf t Each point in It is possible to calculate the corresponding value in pf t1 The closest point The specific calculation formula is as follows:

[0144]

[0145] Where |·| is the Euclidean distance. It is the fth t The m-th point in the frame, It is the fth t1 The i-th point in the frame, It is the fth t1 Mid-frame distance The nearest point is used to determine eight corresponding points in the two frames, and these are used to construct a set V of eight motion vectors, as shown below:

[0146] V={v1,v2,…,v8} (10)

[0147] in,

[0148] Step 3: Extract SMV features

[0149] Let X = {x1, x2, x3, ..., x} n} represents the dataset, n is the number of data points, and M is the average value of the dataset. X and standard deviation SD X, in particular as follows:

[0150]

[0151]

[0152] To extract the random motion value feature SMV, first, the motion value of each motion vector is calculated and input into the set MV:

[0153] MV = {mv1, mv2,..., mv8} (13)

[0154] where, are the x and y coordinates of the i-th point position in the f t frame and the f t1 frame, respectively. As can be seen, this set can represent the amount of movement through each part of the region boundary. Therefore, the standard deviation SD MV of the MV set is calculated and used as the SMV feature, in particular as follows:

[0155] SMV = SD MV (14)

[0156] As mentioned before, due to the random movement of the boundary edges, the SMV value of the fire region is greater than the corresponding value of the non-fire region.

[0157] Step 4: Extracting the SMD feature

[0158] To extract the random motion direction feature SMD, first, the motion direction of each motion vector in V is calculated and input into the set MD, in particular defined as follows:

[0159] MD = {md1, md2,..., md8} (15)

[0160] where, are the x and y coordinates of the i-th point position in the f t frame and the f t1 frame, respectively.

[0161] Then, similarly to the idea of SMV, the standard deviation SD MD of the set MD is calculated and used as the SMD feature, in particular defined as follows:

[0162] SMD = SD MD (16)

[0163] Due to the random way in which the boundary edges of the flame move in different directions, the SMD of the fire region is greater than the corresponding value of the non-fire region.

[0164] Step (3)c: Extracting spatial features

[0165] This method uses two spatial features, namely color variation rate (CVR) and boundary roughness rate (BRR). The extraction process of the two features is as follows:

[0166] Step 1: Extracting CVR feature

[0167] Compared with non-flame or flame-like regions, flame regions usually have very large color variation, while in flame-like regions, there is almost no color variation. Based on this assumption, the color variation is used as a feature to distinguish fire and non-fire regions. First, for each pixel value in the fire region, the standard deviation of its value in the three color channels (RGB) is calculated respectively, then the corresponding average value is calculated and taken as the color variation rate feature (CVR), thus it can be seen that the CVR of the fire region is higher than that of the non-fire region.

[0168] Step 2: Extracting BRR feature

[0169] One of the significant features of the fire region is that its boundary edge is usually rough, therefore, the boundary roughness rate feature (BRR) of the candidate flame region boundary is extracted, which is calculated as follows:

[0170] For a given region, first, the number of convex hull pixels P CH of a set of pixels is calculated, which is the smallest convex set containing the set of pixels, then the BRR feature of the region is calculated, which is as follows:

[0171]

[0172] where P is the number of perimeter pixels of the region, and P CH is the corresponding number of convex hull pixels. It is obvious that when the region boundary is more consistent with its convex hull, the BRR is closer to 1, the edge of the region is less, and it is more likely to be a non-fire region, on the contrary, the higher the ratio, the rougher the boundary, and the more likely it is a fire region.

[0173] Step 4, using the bald eagle algorithm BES to optimize the weight of BP neural network, and constructing BES-BP classifier, using the six features extracted from the candidate fire region to create a six-dimensional feature vector, denoted as: sv i ={CVR, BRR, SMR, MIR, SMV, SMD}, the feature vector is used as the input of the BES-BP classifier to effectively distinguish the true fire region and the non-true fire region. The specific steps are as follows:

[0174] Step (4)a: Implementing bald eagle search algorithm BES

[0175] Bald eagle population is distributed in North America, with excellent vision, strong ability to observe prey in flight, and often preys on salmon. In the search process, first, the bald eagle will check the number of salmon population, and then determine the search space. Next, within the determined search space, the bald eagle will fly in a direction and search for salmon until the location of the prey is found. Finally, the bald eagle will quickly dive towards the prey and catch the salmon. The bald eagle search algorithm BES simulates the behavior of the bald eagle in the hunting process, with the characteristics of fast speed and accurate optimization.

[0176] The search principle of bald eagle search algorithm (BES) is shown in Figure 3. The whole algorithm includes the selection of search space stage, the search of prey stage and the capture of prey stage. The specific steps are as follows:

[0177] Step 1: Selection of search space stage

[0178] The bald eagle randomly selects a search area and determines the best search position by judging the number of prey. In this stage, the position of the bald eagle is determined by multiplying the prior information by β, which is used to randomly change the search point. The mathematical model of this behavior is described as follows:

[0179] P i,new =P best +β*r(P mean -P i ) (18)

[0180] Where β is the position control parameter in the range of [1.5, 2], r is a random number between 0 and 1, P best is the best search position of the current bald eagle, P mean is the average position, and P i is the position of the ith bald eagle.

[0181] Step 2: Search for prey stage

[0182] The bald eagle flies in a spiral shape within the selected space to find prey, which can ensure to speed up the search process and find the best capture position. In this stage, all search points move towards the center point, and the average solution is used to calculate and update the individual position. By multiplying the distance between the current position and the next point by y, and multiplying the distance between the current position and the center point by x, a new search space is found, thereby increasing the diversity of the exploration process. Specifically, the polar coordinate position update equation of the spiral flight mathematical model is as follows:

[0183] θ(i)=α*π*rand (19)

[0184] r(i)=θ(i)+R*rand (20)

[0185] xr(i) = r(i) * sin(0(i)) (21)

[0186] yr(i) = r(i) * cos(0(i)) (22)

[0187] x(i) = xr(i) / max(|xr|) (23)

[0188] y(i) = yr(i) / max(|yr|) (24)

[0189] where 0(i) and r(i) are the polar angle and polar radius of the spiral equation, respectively, a and R are parameters that control the spiral trajectory, a is used to determine the angle between the center point and the search point, R is used to determine the search period, which ranges from [5, 10] and [0.5, 2], respectively, rand is a random number in [0, 1], and x(i) and y(i) represent the position of the vulture in the polar coordinate system, with a value range of [-1, 1].

[0190] According to the above formula, the position update is shown in equation (25):

[0191] P i.new = P i +x(i)*(P i -P mean )+y(i)*(P i -P i+1 ) (25)

[0192] P i+1 is the position of the next vulture.

[0193] Step 3: Capture prey stage

[0194] In this stage, the vulture quickly dives to the best position in the search space and flies towards the target prey, while other individuals in the population also move towards the best position and attack the prey. The polar coordinate equation describing the motion state is as follows:

[0195] 0(i) = a * pi * rand (26)

[0196] r(i) = 0(i) (27)

[0197] xr(i) = r(i) * sinh[0(i)] (28)

[0198] yr(i) = r(i) * cosh[0(i)] (29)

[0199] x1(i) = xr(i) / max(|xr|) (30)

[0200] y1(i) = yr(i) / max(|yr|) (31)

[0201] Thus, the position update equation of the BES is obtained as shown in equation (32):

[0202] P i.new = rand*P best +x1(i)*(P i -c1*P mean )+y1(i)*(P i -c2*P best ) (32)

[0203] where c1 and c2 represent the movement strength of the BES to the optimal position and the center position in the range of [1, 2].

[0204] The implementation process of the BES is shown in Figure 4 .

[0205] Step (4) b: Construct the BES-optimized BP neural network, and introduce the BES to optimize the weights of the BP neural network. The specific steps are as follows:

[0206] Step 1: Construct and initialize a three-layer BP neural network

[0207] In the implementation, the number of layers of the BP neural network is set to 3, i.e., the input layer, the output layer, and the hidden layer, and full connection is used between each layer. The activation function of each hidden node is a Sigmoidal function, and the activation function of the output node is a hard limit function. Since the extracted features of the fire are 6, the number of input layer nodes is designed to be 6, representing 6 fire features, the number of hidden layer nodes is 15, and the number of iterations is 100.

[0208] The topology structure of the constructed three-layer BP network is shown in Figure 5 .

[0209] Step 2: Initialize the BES algorithm and map it to the weights of the BP network

[0210] In the design, the initial positions of each BES are set to random numbers, and they are mapped to the weights of the BP network. The initialization parameters of the BES algorithm are shown in Table 1:

[0211] Table 1 Initialization parameters of the BES algorithm

[0212] Parameter Value Parameter Value Position control parameter β＝2 Random parameter r＝[0,1] Search period parameter R＝1.5 Motion intensity control parameter <![CDATA[c1 = c2 = 1.5]]> Angle control parameter α＝10

[0213] Step 3: Take the optimal position of the BES as the weight of the BP network, and update the output of the BP network

[0214] The individual optimal value of each vulture is calculated by using the fitness function defined by formula (33), i.e. the mean square error between the actual output and the target output of the network, and is taken as the connection weight into the calculation formula (34)-(35) of BP neural network to calculate the output of the hidden layer node and the output layer node of the network respectively.

[0215] (1) Define the fitness function (error E)

[0216]

[0217] Where y is the actual output and d is the target value.

[0218] (2) Hidden node

[0219]

[0220] Where x j is the output of the jth hidden node, p i is the ith input, v ij is the connection weight from the ith input to the jth hidden node, r is the number of input nodes, k is the number of hidden nodes, θ j is the threshold value of the jth hidden node, and f(x) is the activation function of the hidden node.

[0221] (3) Output node

[0222]

[0223] Where y l is the output of the lth output node, x j is the output of the jth hidden node, w jl is the connection weight from the jth hidden node to the lth output node, n is the number of output nodes, k is the number of hidden nodes, θ l is the threshold value of the lth output node, and f(x) is the activation function of the output node.

[0224] Step 4: Update the optimal value of the vulture

[0225] According to the fitness function, the current optimal value of the vulture individual is compared with the global optimal value, and the optimal value of the vulture is updated according to the following steps:

[0226] (1) The vulture selects the search space and updates the position according to formula (18).

[0227] (2) The vulture uses spiral search to search for prey and select the area, and updates the position by using formula (25).

[0228] (3) Use the new position in the search space to dive towards the prey, and update the position by using (32).

[0229] Step 5: Check the end condition

[0230] If the number of iterations is greater than the maximum number of iterations or the evaluation error value is greater than the given value, stop the iteration and go to Step 6, otherwise, go to the next round of hawk state update (Step 3).

[0231] Step 6: Save the set of global optimal values.

[0232] The flow of the BES-optimized BP neural network training algorithm is as follows Figure 6 .

[0233] Step 5, based on the above technology, use MATLAB software to build an experimental platform, use different video data sets to evaluate the performance of the proposed method, and give the detection results on each video data set. The following test experiments are mainly completed:

[0234] Step (5) a: Select the data set

[0235] Two video data sets are used in this experiment, namely the data set VisiFire and AzarFire, and Table 2 introduces the characteristics of the VisiFire and AzarFire data sets.

[0236] Table 2 Characteristics of VisiFire and AzarFire data sets

[0237]

[0238]

[0239] Some sample pictures of the fire video frames used are as shown in Figure 7 .

[0240] Step (5) b: Design evaluation index

[0241] In the process of using BES-BP classifier for fire identification, it is necessary to use the training data set to train the classifier, and use the test set to evaluate the performance of the proposed method. Therefore, the following evaluation indexes are defined:

[0242] The first evaluation index is the system sensitivity or true positive rate TPR, which is used to represent the ability of the test system to identify fire areas, and is calculated as follows:

[0243]

[0244] The second evaluation index is the system specificity or true negative rate (TNR), which is used to represent the ability of the system to identify non-fire areas, and is calculated as follows:

[0245]

[0246] The third evaluation index is accuracy, which is used to represent the classification accuracy of fire and non-fire areas, and is calculated as follows:

[0247]

[0248] Step (5) c: Obtain experimental results

[0249] According to the above defined evaluation indexes, the performance of the proposed method is evaluated. In the 6 video fire data sets of VisiFire and AzarFire data sets, 500 video fire frame images are selected, and some experimental pictures are shown in FIG. 6. Figure 7 During the implementation, for the video fire frame images VC1, VC2, VC3, VC6, TPR is used to evaluate the accuracy of the detection classification, and for the video fire frame images VC4 and VC5, TNR is used to evaluate the accuracy of the detection classification. The specific experimental results are shown in Table 3.

[0250] Table 3 Classification results for VisiFire and AzarFire data sets

[0251]

[0252] In Table 3, "NaN" represents the case where the denominator and numerator in the calculation are both zero. From the experimental results, it can be seen that the proposed method can complete the accurate classification of the fire state of most data sets, and the highest classification rate reaches 99.12%.

[0253] The above embodiments are only for illustrating the technical concept and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the content of the present application and implement it, and cannot limit the protection scope of the present application. Any equivalent transformation or modification made according to the spirit and essence of the present application shall be covered within the protection scope of the present application.

Claims

1. A fire detection method based on BES-BP classifier and spatio-temporal visual features, characterized in that, The method comprises the following steps: Step (1) determines candidate fire areas and completes preprocessing; Step (2) uses a flame motion detection technology based on motion intensity perception to extract motion intensity information and a motion area containing moving pixels; Step (3) for each candidate fire area, a set of complementary features are extracted, including four spatio-temporal features and two spatial features, the spatio-temporal features include two surface spatio-temporal features and two boundary spatio-temporal features, the surface spatio-temporal features are surface motion rate SMR and motion intensity rate MIR; the boundary spatio-temporal features are random motion value feature SMV and random motion direction feature SMD; the two spatio-temporal features are color variation rate CVR and boundary roughness rate BRR; Step (4) the weights of the BP neural network are optimized by using the bald eagle algorithm BES to construct a BES-BP classifier, a six-dimensional feature vector is created by using the six features extracted from the candidate fire area in step (3), and the BES-BP classifier is used to effectively distinguish fire areas. 2.The fire detection method based on BES-BP classifier and spatio-temporal visual features according to claim 1, wherein, The step (1) is specifically as follows: Firstly, the flame area in the input video frame picture is recognized by flame color, the color of the flame varies from red to yellow, the pixels containing the color range are selected as candidate fire areas, and then preprocessing is performed, and an Otsu maximum inter-class variance threshold algorithm is applied to obtain a binary image. 3.The fire detection method based on BES-BP classifier and spatio-temporal visual features according to claim 1, wherein, In the step (2), the motion intensity information and a motion area containing moving pixels are extracted by using the flame motion detection technology based on motion intensity perception, and the specific steps are as follows: Step (2.1): Take the f t frame and the f t1 frame as input frames, calculate the difference image between the two frames, calculate the corresponding pixels on R, G, B three channels respectively, and finally obtain the difference image S R (x, y), S G (x, y), S B (x, y) on R, G, B channels; Step (2.2): Compute the pixel mean μ of the difference image on the R, G and B channels of the difference image R , μ G , μ B ; for a difference image of size m x n, compute its mean value on the RGB channels as follows: Step (2.3): calculate the motion intensity map M(x, y): M(x, y) = ((S R (x, y) - μ R ) + (S G (x, y) - μ G ) + (S B (x, y) - μ B )) / 3 Step (2.4): calculate the expected motion intensity map I(x, y) after denoising by using a thresholding algorithm; Wherein, I(x, y) is the expected motion intensity map, and alpha is a threshold value, alpha is in the range of [3, 8]; Step (2.5): calculate the binary motion image B(x, y), and obtain the corresponding binary image B(x, y) by performing hard thresholding on the image I(x, y).

4. The fire detection method based on BES-BP classifier and spatio-temporal visual features according to claim 3, characterized in that, Detecting each fire region within a given frame of the input video, spatial features are extracted from the processing frame, temporal features use other frames in the input fire video, using one frame at a reasonable interval from the processing frame, i.e. to identify fire regions in the t th frame, the t frame is used, respectively referred to as f t and f t1 .

5. The fire detection method based on BES-BP classifier and spatio-temporal visual features according to claim 1, characterized in that, In the step (3), for each candidate fire area, a set of complementary features are extracted, and the specific steps are as follows: Step (3.1): surface spatio-temporal features, i.e. surface motion rate SMR and motion intensity rate MIR, are extracted; Step 1: extract SMR features: Wherein, A is the area of the region being processed, i.e. the number of pixels in the region, and B is the number of moving pixels in the motion image; considering the moving surface feature of the fire, the SMR of the fire area is greater than that of the related area of the non-fire moving object; Step 2: extract MIR features Wherein, A is the area of the region being processed, and I is the sum of the motion intensity values of the pixels in the region being processed; the MIR of the fire area is greater than that of the related area of the non-fire moving object; Step (3.2): boundary spatio-temporal features, i.e. random motion value feature SMV and random motion direction feature SMD, are extracted; Step 1: select the research point; the boundary spatio-temporal features are calculated by analyzing the position and direction of the boundary of the candidate fire area changing with time, and eight points of four corners and four edges of each region boundary are selected; Step 2: construct a set of motion vectors V To analyze the motion behavior of the selected points, the positions of the study points on the two corresponding boundaries are specified and the motion vectors are measured, and the following mathematical formula is defined to describe them: For a given region, eight points in two frames are defined as follows: pf t = {p1 t , p2 t , …, p8 t}, pf t1 = {p1 t1 , p2 t1 , …, p8 t1} Among them, pf t and pf t1 They are respectively the fth t Frame and f t1 The set of points studied on the frame boundary, for pf t Each point in Calculate the corresponding pf t1 The closest point The specific calculation formula is as follows: Where |·| is the Euclidean distance. It is the fth t The m-th point in the frame, It is the fth t1 The i-th point in the frame, It is the fth t1 Mid-frame distance The nearest point is used to determine eight corresponding points in the two frames, and these are used to construct a set V of eight motion vectors, as shown below: V = {v1, v2, …, v8} wherein Step 3: Extract SMV features Let X = {x1, x2, x3,..., xn} represent a data set, n the number of data points, define the mean M n and standard deviation SD X of the data set as follows: X M = 1 / n Σxi SD = sqrt(1 / n Σ (xi - M)2) Calculate the motion value of each motion vector and input it into the set MV: MV = {mv1, mv2, …, mv8} wherein, respectively the x and y coordinates of the i-th point position in the f t frame and the f t1 frame; this set is able to characterize the amount of movement of each portion of the region boundary, the standard deviation SD MV of the set of MVs is calculated to be a SMV feature; the SMV value of a fire region is greater than the corresponding value of a non-fire region; Step 4: Extract SMD features Calculate the motion direction of each motion vector in V and input it into the set MD, which is defined as follows: MD = {md1, md2, …, md8} wherein, respectively, the x and y coordinates of the i-th point position in the f t frame and the f t1 frame; calculating the standard deviation SD MD of the set MD and taking it as the SMD feature; the SMD of the fire region is greater than the corresponding value of the non-fire region; Step (3.3): Extract spatial features, i.e. color variation rate CVR and boundary roughness rate BRR; Step 1: Extract CVR features Use the color variation as a feature to distinguish between fire and non-fire areas; for each pixel value in the fire area, calculate the standard deviation in the three color channels RGB respectively, then calculate the corresponding average value, and take it as the color variation rate feature CVR, the CVR of the fire area is higher than that of the non-fire area; Step 2: Extract BRR features For a given region, calculate the number of its convex hull pixels P CH The convex hull of a set of pixels is the smallest convex set containing the set of pixels, then calculate the BRR feature of the region as follows: where P is the number of perimeter pixels of the region, P CH is the number of corresponding convex hull pixels.

6. The fire detection method based on BES-BP classifier and spatio-temporal visual features according to claim 1, characterized in that, In step (4), the BES algorithm is used to optimize the weights of the BP neural network to build a BES-BP classifier, and the fire features extracted from the candidate fire area are input into the BES-BP classifier to effectively distinguish between real fire areas and non-real fire areas, the specific steps are as follows: Step (4.1): Implement the BES algorithm, which includes the selection of the search space stage, the search of the prey stage and the capture of the prey stage; Step (4.2): Build a BES-optimized BP neural network, introduce the BES algorithm to optimize the weights of the BP neural network, the specific steps are as follows: Step 1: Build and initialize a three-layer BP neural network The number of layers of the BP neural network is 3, i.e. input layer, output layer and hidden layer, and full connection is used between each layer; the activation function of each hidden node is Sigmoidal type function, and the activation function of the output node is hard limit function; the number of input layer nodes is designed to be 6, representing 6 fire features, the number of hidden layer nodes is 15, and the number of iterations is 100 times; Step 2: Initialize the BES algorithm and map it to the weights of the BP network; first initialize the individual positions of each vulture to random numbers and map them to the weights of the BP network; Step 3: Take the optimal position of the vulture as the weight of the BP network and update the output of the BP network; Use the fitness function E defined in (1), i.e. the mean square error of the actual output and the target output, to calculate the individual optimal value of each vulture, and take it as the connection weight to bring it into the calculation formula of the BP neural network defined in (2)-(3) to calculate the output of the hidden layer nodes and the output layer nodes of the network respectively; (1) Define the fitness function, i.e. error E Where y is the actual output and d is the target value; (2) Hidden node where x j is the output of the jth hidden node, p i is the ith input, v ij is the connection weight from the ith input to the jth hidden node, r is the number of input nodes, k is the number of hidden nodes, θ j is the threshold of the jth hidden node, and f(x) is the activation function of the hidden node. (3) Output node where y l is the output of the lth output node, w jl is the connection weight from the jth hidden node to the lth output node, n is the number of output nodes, k is the number of hidden nodes, and θ l is the threshold value of the lth output node. Step 4: Update the optimal value of the bald eagle, compare the current optimal value of the bald eagle individual with the global optimal value according to the fitness function; Step 5: Check the end condition, if the number of superimposed bands is greater than the maximum number or the evaluation error value is greater than the given value, stop iteration and turn to Step 6, otherwise, turn to Step 3 to update the state of the new round of bald eagle; Step 6: Save the global optimal value of the group.

Images