Rapid Flame Detection Method for Forest Fire Identification

Abstract

Flame detection, comprising a multi-threshold segmentation method based on spline regression, comprises four steps: (1) multi-threshold segmentation is carried out on each colour channel using a HistS

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of machine learning, and in particular, to a rapid flame detection method for forest fire identification. BACKGROUND

[0002] The forest resources in China are quite limited and unevenly distributed. The forest stock is only 9.14 billion m3 and about 8m3 per capita, which is far from the world standard of 72 m3 per capita. The disaster resulted from forest fires to forests, forest ecosystems and human life and property is devastating. In recent years, forest fires have occurred more frequently, showing new characteristics such as increased difficulty in control, blurred seasonal boundaries of forest fire prevention, and increased difficulty in predicting fires. Countries all over the world spend huge amounts of money to develop a forest fire early warning system. The 13th Five-Year Development Plan for Forestry Informatization places special emphasis on the importance of intelligent "Intemet+" technologies, led by machine intelligence and computer vision, in the construction of informatisation. There are mainly three types of forest fire video monitoring systems at present: visible light monitoring system, thermal imaging monitoring system, and dual-monitoring system of visible light and thermal imaging, in which the visible light monitoring system is the most popular, which has the characteristics of being all-weather, long in range and low in cost. It is one of the research hotspots in recent years to find a fire as early as possible from the images or videos of the monitoring system and give early warning.

[0003] Image segmentation is the most important preprocessing step in image processing. It is expected that an area of interest can be correctly distinguished from other areas. The quality of segmentation directly influences the subsequent target identification. Commonly used image segmentation methods include: a threshold segmentation method, an edge detection method, an area growth method and a segmentation method based on other theories (such as a morphology method). Due to the particularity of the research object "fire" and the previous work of the inventors, the present disclosure only focuses on to the threshold segmentation method. Such methods mainly use a histogram to solve the thresholds, such as the Otsu and the maximum entropy method. The Otsu-based segmentation method works better when the area occupied by the target and that occupied by the background are close, otherwise it is easy to fail. At the same time, there is also the problem of insufficient segmentation in the Otsu-based segmentation method. The maximum entropy method determines the thresholds by maximizing the sum of the target entropy and the background entropy. The richer the details of the image, the better the segmentation effect. A color image thresholding method mainly includes two steps: step 1, acquiring a smooth spline function by fitting a histogram; and step 2, determining the thresholds and the number of thresholds according to the extreme value of the spline. The more the thresholds, the more detailed the image segmentation, and the easier it is to separate small image targets from the background. In the case of multiple thresholds, the calculation amount of the two types of methods increases exponentially, so that it is difficult to satisfy the real-time requirements. It is another research branch of image segmentation to use a curve fitting histogram and then adaptively determine the thresholds and the number of thresholds, such as a Support Vector Regression (SVR) fitting histogram, regression methods using a Gaussian filter or a mixed Gaussian function, the quasi-linear regression of a higher-order polynomial, HistSplineReg based on a smooth spline regression histogram (Yang Xubing et al), etc. SVR obtains the Boundary Support Vector (BSV) when fitting the histogram, and determines the threshold based on the boundary support vector. Although the method avoids solving the extreme value and the threshold selection space can be limited to a boundary support vector set, the method faces the problem of selecting the kernel and kernel parameters. The Gaussian function method transforms the problem of threshold selection into a mixed problem of multiple Gaussian functions, and the method also faces the problem of selecting multiple parameters. Although the form of the high-order polynomial fitting method is simple, the method faces the problems of large fitting empirical errors and selecting the polynomial order, which cannot satisfy the requirements of automation. Because HistSplineReg uses a cubic natural spline function and only needs to adjust one parameter (a smoothness factor, which is used to adjust the function smoothness and the empirical regression errors), the method has the characteristics of prominent details and easy calculation in terms of the forest fire image segmentation effect. Taking HistSplineReg as an example, the present disclosure researches the problem of extracting flame pixels after segmentation and proposes a rapid flame detection algorithm for forest fire identification.

[0004] The current research work of the flame identification method is mainly carried out based on the color space, that is, the rules suitable for distinguishing flame pixels are extracted from the researched color space. During identification, pixels that can satisfy all the rules are determined as "fire", otherwise, the pixels are determined as "non-fire". Because different color spaces have different primary colors, the combinations of different primary colors highlight different color characteristics, such as brightness, color difference, hue and color saturation. Therefore, a series of forest fire identification rules also come into being, mainly including the multi-space combination rule methods of RGB, Luminance, Rhrominance-Blue and Chrominance-Red (YCbCr) , Hue, Intensity, Saturation (HIS) and RGB+HSI (Hue, Saturation, Intensity). The common problems of these methods are as follows: first, these methods need to transform the image into the color space where the rules are located, and the combination rules need to be transformed many times; thereafter, the transformed image is subject to rule verification pixel by pixel, so that the identification speed is low and it is difficult to satisfy the real-time requirements. For the threshold segmentation method, the existing flame detection methods are all completed in at least three or more color channels, which also pose a new challenge to the detection method based on threshold segmentation. The number of areas formed by the multi-channel threshold combination increases exponentially, which undoubtedly increases the difficulty of the subsequent identification work. The present disclosure proposes a rapid flame scanning method for the problem of forest fire identification, which constructs a Cartesian set on the multi-color channel threshold combination area, so that the elements of the Cartesian set correspond to the segmented threshold area one by one. As the number of elements of the constructed Cartesian set is far less than the number of pixels in the image, these elements only need to be detected during identification, so as to achieve the purpose of rapid identification. SUMMARY

[0005] At present, a great importance is particularly given to the multi-threshold color image segmentation method because of a high segmentation quality and a strong visual hierarchy. However, because the method first completes the segmentation on each color channel independently, and then superimposes the segmentation results on multiple channels, so that the number of dividing areas increases exponentially, thereby greatly increasing the difficulty in detecting the area of interest subsequently. In order to solve this problem, taking forest fire identification as an example, a rapid flame detection method for forest fire identification is proposed. The method can ensure a high flame detection rate and a low false alarm rate while ensuring real-time detection. Second, the two detection indexes of forest fire are considered comprehensively, so that a flame criterion based on YCbCr is more suitable for forest fire detection in a real scene.

[0006] The present disclosure provides a rapid flame detection method for forest fire identification, including:

[0007] a multi-threshold segmentation method based on spline regression, which mainly includes four steps:

[0008] step (1), carrying out multi-threshold segmentation on each color channel by using a HistSplineReg segmentation method firstly, to ensure that image segmentation details are clear, and calculating and recording segmentation areas on each channel as SR , SG and SB \

[0009] step (2), selecting a representative point in each segmentation area, and recombining representative points according to a sorting order in a color space to form a pixel representation of an area;

[0010] step (3), finally, comparing a pixel representative point with provided supervision information to retrieve a target area where the pixel representative point is located; and

[0011] step (4) acquiring a dividing subset where a "fire" representative point is located and a corresponding flame pixel interval thereof.

[0012] In an embodiment, in step (1), steps of the HistSplineReg segmentation method are as follows:

[0013] using a local fitting idea in the spline regression, where a regression curve is second-order continuous differentiable over entire interval to ensure smoothness of a regression function g(-); and in the case of an observation set of {(x^yj}, i =1,2,L , an optimization goal is:

[0014] min                                           (1) s i

[0015] where is a weight parameter, g" denotes a second derivative of the function g(-) , and a smoothness parameter q satisfies 7 e[0,1];

[0016] for an image, assuming that a pixel level L is 256; under an assumption that an independent variable is y.= i , which denotes an Z -th gray level, yt = p(i) denotes a histogram distribution probability, simplifying Equation (1) as:

[0017] min -^(012 [g"(012^ (2) g i

[0018] where a =1 / / 7—1(>0) is a smoothness penalty factor;

[0019] introducing two constant band matrices q and 7? for solving the regression function g(-) , where:

[0020] g = {I + aK) xY (3)

[0021] where a vector g is a discrete representation of the function g(-) , I is an identity matrix, Y = [y17y2,L ,yL 2]r , K = QRQT,

[0022] in terms of calculation, a minimum point of the regression curve is a threshold, and then rapid multi-threshold detection is carried out.

[0023] In an embodiment, in step (2), a rapid multi-threshold detection method includes:

[0024] defining symbols as follows: k denoting the number of thresholds, and the number of thresholds on respective color channel being denoted as Rk, Gk and Bk \ assuming that the thresholds have been sorted in an ascending order, the corresponding dividing subset being denoted as ^k, in which and 5)1 ^=00*7); assuming that the pixel level is 256, which being denoted as S = 1 =[0,255] ; and taking a RGB image as an example, and above equation being expanded on each color channel to obtain:

[0025] SR ={SRi,SR2,-,SRk},

[0026] SG ={SGi,SG2,-,SGk},

[0027] Sb = {5Bi,5B2,-,5sJ; (4)

[0028] in order to highlight a multi-threshold segmentation effect of a color images, calling a pixel interval between adjacent thresholds as a dividing subset. In the dividing subset, if a pixel is used to represent the interval, and then a hierarchical sense of image segmentation is generated visually by superimposing the colors on each channel, and each of superimposed areas is:

[0029] S^®SG®SB (5)

[0030] where ® denotes a Cartesian product; then the above color image threshold segmentation problem is transformed into a set operation of Equation (6):

[0031] Q = {SR®SG®SBi\SReSR,SG eSG,SBeSB} (6) 7 J 1 7 J 1

[0032] on a segmented sub-area ®SG ®SBi , selecting a set of suitable representative points, which is denoted as ( ri,gJ,bl ), replacing pixel values in range of each of the sub-intervals S^, SG. and SBi by r^g^^ , respectively, and completing target segmentation on the area ®SG ®SBi .

[0033] In an embodiment, in Step (3), a supervision information method is introduced as follows:

[0034] according to a lot of statistical analysis, flame pixel criteria are summarized four criteria according to different color spaces which are used, and which are applied to the flame determination of the dividing subset:

[0035] criterion 1: a flame pixel criterion of an RGB space

[0036] it is assumed that the RGB value of current pixel is (R,G,B\ and if the current pixel satisfies all rules in Equation (7), the current pixel is determined as a flame pixel; otherwise, the current pixel is determined as a non-flame;

[0037] R >^mean ’ (7) G >Crmean, R >G >B

[0038] where I,iean and mean denote mean values of pixels on red and green channels, respectively;

[0039] criterion 2: a flame pixel criterion of a YCbCr space

[0040] since information such as hue, brightness and color saturation of a color is hidden in three components of RGB and is indistinguishable, a rule of the YCbCr space is:

[0041] Y >Cb,

[0042] Cr >Cb,

[0043] Y >^mean’ Cb <Cbmean , Ct >CTmean ,

[0044] \Cb - Cr\ >t; (8)

[0045] criterion 3: a flame pixel criterion of an HSI space

[0046] in a color model HSI, H,S,I are corresponding to hue, color saturation and light intensity of pixels, respectively; the flame pixel criterion of the space is: a a • H e (0,60) Se (20,100) , Ze (100,255)

[0048] a recommended range is 1 G v ’ and v ’ 7;

[0049] criterion 4: a mixed flame pixel criterion of RGB+HSI

[0050] combining respective characteristics of color spaces of RGB and HSI, the mixed flame pixel criterion is described as follows: R>R^

[0051] R>G>B, (10) S>(255-R)-S7lm / Rnm

[0052] where and 7?Thres denote thresholds of color components S and R , respectively, and corresponding reference ranges are 55 to 65 and 115 to 135, respectively;

[0053] according to the above information, an appropriate representative point is selected from each element AS,. of Q , and if the representative point passes the rule verification, it indicates that all pixels in the segmentation area corresponding to the representative point are determined as "fire".

[0054] In Step (4), a rapid detection algorithm includes following five steps: acquiring a dividing subset where the "fire" representative point are located and a corresponding flame pixel interval thereof;

[0055] arranging SR , SG and SB according to magnitude of thresholds, and then merging the representative points of adjacent subsets; assuming that elements in the set shown in Equation (4) are arranged in an ascending order of the thresholds; and describing adjacent subsets and their definitions on Cartesian product as follows:

[0056] theorem 1: sets B and C denote intervals in a one-dimensional space; assuming that Cartesian sets Ai®B®C and A^ ® B ® C are adj acent, in which

[0057] Uz(4®B®C)=D®B®C, i = \2 (12)

[0058] holds true , in which D = A1UA2 and it is a convex set;

[0059] theorem 2 (adjacent mean values are merged): assuming that mA and mB are mean values on adjacent sets A and B, so that the mean value of sets are merged: nAmA + nBmB

[0060] m = —— ----—£. (13) + nB

[0061] where nA and denote the number of elements of the sets, respectively; 21 D

[0062] assuming that a sub-area containing detection target "flame" is:

[0063] e' = U(5R.®SG. ®S.), (11) s.zSR,sG.zSG,s.zSB

[0064] if adjacent Cartesian sets are contained in Q', merging intervals according to the theorem 1, so as to further reduce items in 2', and achieve a purpose of reducing the number of traversals and improving detection speed; merging the representative points according to the theorem 2, replacing nA and by length of the intervals for continuous intervals, and merging the representative points to improve the detection speed.

[0065] The present disclosure has the following beneficial effects.

[0066] Compared with the area detection method, the method of the present disclosure has the following advantages: 1) the introduced multi-threshold method is high in precision and more obvious in image details; 2) the target area of interest can be acquired only by detecting a small number of representative points without pixel-by-pixel scanning; 3) appropriate representative points are selected in fused area, and then the representative points are compared with the introduced supervision information to achieve the purpose of detecting the areas rapidly; 4) the target detection can be carried out at the pixel level without losing the image quality; and 5) the problem of area detection can be transformed into the Cartesian product operation finally, so that the algorithm is guaranteed theoretically. In addition, the present disclosure further provides two acceleration strategies: method for merging areas of interest and method for merging representative points. Experiments have been carried out on public forest fire images and videos. The detection result is reported using a visualization method and a numerical method. In terms of the video detection time of a forest fire, the method of the present disclosure is 4.9 times faster than the YCbCr method and 6.7 times faster than the RGB+HIS method, which achieves the goal of real-time detection. The method can ensure a high flame detection rate and a low false alarm rate while ensuring real-time detection. Second, considering the two detection indexes of forest fire comprehensively, a flame criterion based on YCbCr is more suitable for forest fire detection in a real scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to explain the technical scheme of the present disclosure more clearly, the drawings needed in the embodiment will be briefly introduced hereinafter. Obviously, other drawings can be obtained by those skilled in the art according to these drawings without creative labor.

[0068] FIG. 1 is pictures showing flame detection results of four criteria on a forest fire image (e); in which (a) is an RGB rule; (b) is a YCbCr rule; (c) is an HSI rule; (d) is an RGB+HSI rule; (e) is an original image;

[0069] FIG. 2 is pictures showing rapid flame detection results based on multi-threshold segmentation; in which: (a) is an original image; (b) is a spline-based multi-threshold segmentation result; (c) and (d) are detection results when directly using an RGB+HSI rule and a YCbCr rule; (e) and (f) are methods of the present disclosure, whose supervision information is from (c) and (d), respectively. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0070] Reference will now be made in detail to embodiments, examples of which are illustrated in the drawings. When the following description refers to the drawings, the same numbers in different drawings indicate the same or similar elements unless otherwise indicated. The implementations described in the following embodiments do not represent all the implementations consistent with the present disclosure, but are merely examples of systems and methods consistent with some aspects of the present disclosure as detailed in the claims.

[0071] 1. Research method

[0072] In order to segment small targets accurately, a multi-threshold segmentation method is often used in image processing to complete image preprocessing. A spline function is one of the fitting histogram methods of smooth functions. Since this work is aimed at forest fire problems, the present disclosure is a follow-up to the work. HistSplineReg segmentation is introduced briefly hereinafter.

[0073] 1.1. HistSplineReg segmentation

[0074] Spline regression uses a local fitting idea, and a regression curve is required to be second-order continuous differentiable over the entire interval to ensure the smoothness of a regression function . In the case of an observation set of {(x^y^} , z=l,2,L , an optimization goal is:

[0075] min (1) g i

[0076] where is a weight parameter, g ” denotes a second derivative of the function g(-) , and a smooth parameter is 7 e [0,1] •

[0077] For an image, it is assumed that a pixel level L is 256. It is assumed that an independent variable is xz = i , which denotes an i -th gray level; yt = p(i) denotes a histogram distribution probability, and Equation (1) is simplified as:

[0078] mgin S^0’)-g(0]2+«J[g "(0]2^V (2) i

[0079] where a =1 / / 7—1(>0) is a smoothness penalty factor.

[0080] In order to solve the smooth function g(-) , two constant band matrices q and 7? are introduced, in which:

[0081]    =   +           (3)

[0082] where a vector § is a discrete representation of the function g(-) , I is an identity matrix, Y = , y^ ,L , yl2 f K = QRQ ■

[0083] In terms of calculation, a minimum point of the regression curve is a threshold.

[0084] 1.2. Rapid multi-threshold detection method

[0085] For the convenience of description, symbols are defined as follows: k denotes the number of thresholds, and the number of thresholds on each color channel is denoted as Rk, Gk and Bk , respectively. Without losing generality, it is assumed that the thresholds have been sorted in an ascending order, and the corresponding dividing subset is denoted as S2,L ,Sk, in which £ = and = 0(1^ / ) . It is assumed that the pixel level is 256, which is denoted as S = / =[0,255] . Taking RGB images as an example, the above equation is expanded on each color channel to obtain:

[0086] ^R “     ’ ^2 ’''' ’      ’ (4) SC={SC,SC ,--,Sc }, (_r t (jj 5 (_r2 5 5 / 5 SB={SB,SB ,---,¾}.

[0087] In order to highlight a multi-threshold segmentation effect of color images, a pixel interval between adjacent thresholds is called as a dividing subset. For example, the dividing subset of the red channel can be expressed as Sa,S„,L ,SR Jn I SR =0 and SR UL = 1 Ki Kj Kk hold true. In a dividing subset, if a pixel is used to represent the interval, and then the colors on each channel are superimposed, a hierarchical sense of image segmentation is generated visually, and each of the superimposed areas is:

[0088] ®SG ®SB (5) ‘JI

[0089] where ® denotes a Cartesian product; where the above color image threshold segmentation problem is transformed into a set operation of Equation (6):

[0090] Q = ® SGj ® SBi I e , SG. e SG, SBi e SB } (6)

[0091] As mentioned above, representative points are often used in image segmentation to represent the dividing subset, that is, all pixels within the interval can be approximated by representative points. Since the representative points corresponding to different intervals are different, and the distance between the representative points is larger than distance between the original sub-intervals, the visual effects that the same target is more similar and the contrast between different targets is more obvious can be achieved on the image. In order to highlight the target in the image, in a segmented sub-area ® SG. ®SBi , a set of suitable representative points is selected, which is denoted as ( ). Pixel values in the range of each of the sub-intervals and are replaced by , respectively, and target segmentation is completed in the area ®SG ®SBi . Obviously, the more detailed such segmentation is, the more obvious the target of the image is.

[0092] Inspired by supervised learning, it is proposed to introduce "supervision information" into the segmented target, so as to check all representative points. If the representative point is consistent with the target provided by the supervision information, the corresponding segmented area is labelled as the target area. Compared with pixel-by-pixel detection, the advantage of the method is that the calculation amount can be greatly reduced due to the fact that only Q is detected.

[0093] In order to achieve rapid flame detection, the following tasks need to be completed: 1) how to introduce supervision information; 2) how to combine segmentation and detection automatically and provide a detection algorithm.

[0094] 1.3. Introduction of supervision information

[0095] According to a lot of statistical analysis and flame pixel criteria, based on different color spaces which are used, a summary is given as follows.

[0096] Criterion 1: a flame pixel criterion of an RGB space

[0097] It is assumed that the RGB value of the current pixel is (R,G,B\ and if the pixel satisfies all the rules in Equation (7), the pixel is determined as a flame pixel; otherwise, the pixel is determined as a non-flame;

[0098] mean ’ mean,

[0099] where Rmean and Gmean denote mean values of pixels on red and green channels, respectively.

[0100] criterion 2: a flame pixel criterion of a YCbCr space

[0101] Since information such as hue, brightness and color saturation of a color is hidden in three components of RGB and is indistinguishable, Celik et al provide a set of rules in the YCbCr space: Y>Cb, Cr >Cb,

[0102] (8) Y>Y Cb<Cb Cr>Cr | Cb - Cr |> T.

[0103] Criterion 3: a flame pixel criterion of an HSI space

[0104] In a color model HSI, H. S, I are corresponding to the hue, the color saturation and the light intensity of pixels, respectively. The flame pixel criterion of the space is: [01051 S^<S<S^, (9) ^min <I <■

[0106] The recommended range is 77 G (0,60), G (20,100) and / g (100,255).

[0107] Criterion 4: a mixed flame pixel criterion of RGB+HSI

[0108] Combining the respective characteristics of the color spaces of RGB and HSI, Bian Yongming et al considered a mixed criterion, which is described as follows:

[0109] R>G>B, (10) S > (255 - R) ■ STtes /

[0110] where and denote thresholds of color components S and R, respectively, and respective reference ranges are 55 to 65 and 115 to 135.

[0111] 1.4. Rapid detection algorithm

[0112] When the above four criteria are applied to the flame determination of the dividing subset, it is expected to overcome the inefficient detection problem of pixel-by-pixel scanning. According to the above information, an appropriate representative point is selected from each element So ®SG ®SB of Q . If the representative point passes the rule verification, it indicates that all pixels in the segmentation area corresponding to the representative point are determined as "fire". Intuitively, since only a small number of representative points need to be verified, the detection speed is expected to be greatly improved. The rapid detection algorithm mainly includes the following five steps:

[0113] Step 1, calling HistSplineReg to carry out multi-threshold segmentation on each channel of the RGB color space, in which the number of thresholds of respective channels are denoted as Rk, Gk and Bk \

[0114] Step 2, calculating and recording the segmentation areas as SB , SG and SB on each channel according to Equation (4), and

[0115] Step 3, calculating the set Q and selecting the representative points of the dividing subset according to Equation (6);

[0116] Step 4, carrying out flame determination on the representative points according to the rules provided in 1.3, and recording all the representative points determined as "fire";

[0117] Step 5, acquiring a dividing subset where "fire" representative point is located and a corresponding flame pixel interval thereof:

[0118] Compared with pixel-by-pixel detection, the detection speed of the method is obviously improved, but the detection speed can still be further accelerated. The acceleration problem can be considered from two aspects, that is, detection area merging and representative point merging. The specific strategy is as follows: arranging SR,SG and SB according to magnitude of thresholds, and then merging the representative points of adjacent subsets. Without losing generality, it is assumed that the elements in the set shown in Equation (4) are arranged in an ascending order of thresholds. Adjacent subsets and their definitions on the Cartesian product are described as follows.

[0119] Definition 1 (adjacent sets): the sets A and B denote one-dimensional intervals. If A<B and sup(A) = inf(B), the sets A and B are called to be adjacent.

[0120] sup() and infQ in definition 1 denote a supremum and an infimum, respectively, that is, the right end point of the set a coincides with the left end point of the set b • The partial order relation a<B indicates that x<y holds true for any x e and y^B ■

[0121] Definition 2 (adjacent Cartesian sets): sets a, B and C denote the intervals defined in a one-dimensional space. For the Cartesian sets A®C and B®C defined on the interval, if the sets a and b are adjacent, A®C and B®C are adjacent Cartesian sets.

[0122] It is assumed that the sub-area containing the detection target "flame" is:

[0123] Q' = 0¾ ® SG.®SB.),   (11) eSG,S eSB

[0124] if adjacent Cartesian sets are contained in Q', intervals can be merged according to theorem 1, so as to further reduce items in Q', and achieve the purpose of reducing the number of traversals and improving the detection speed.

[0125] Theorem 1: sets , A2 , B and C denote intervals in a one-dimensional space. It is assumed that Cartesian sets Ay ®B®C and A^® B®C are adjacent, in which U,(4 ®B®C)=D®B®C, / = 1,2 (12) holds true , in which D = Ay and it is a convex set.

[0126] It is proved that the Cartesian set satisfy the distribution law on intersection of the sets or union of the sets, i.e., U^4 ®B®C) = (U,4)®B®C ’ / = 1,2 holds ture- Since A®B®C and A1®B®C are adjacent, that is, and are adjacent. Therefore, A and A can be merged into a larger interval, which is denoted as D = a, • Since the merged interval still has only two endpoints, the interval is a convex set.

[0127] Although theorem 1 is expressed for two adjacent sets, theorem 1 can be generalized to the problem of a finite number of adjacent sets. In the same way, according to the associative law, U,(5®4®C) = B® (U,4) ®C and U,(5®C®4) = B®C ® (U;4) hold true- The detection algorithm is handled according to theorem 1, so that it is expected to accelerate the flame detection.

[0128] Finally, the problem of merging representative points is described. It can be seen from the above that the representative points that need to be merged must exist in two adjacent sets. There are two calculation methods: one method is to carry out calculation on the merged set, and the other method is to carry out generation directly from two adjacent representative points. In the present disclosure, the mode and the mean value of the dividing subset of the histogram are used as representative points. The mode refers to the pixel value with the highest number of occurrences (frequency) in the interval, which can be solved according to the histogram on the channel. Additionally, the representative points using the mean value can be acquired directly from the adjacent representative points, which is described by theorem 2.

[0129] Theorem 2 (adjacent mean values are merged): it is assumed that mA and mB are mean values on the adjacent sets A and B , so that the mean value of sets are merged n.mA + nRmR

[0130] m = (13) nA+nB

[0131] where nA and nB denote the number of elements of the sets, respectively;

[0132] It is proved that: without losing generality, the partial ordering relation is assumed to be A<B, and the merged set is denoted as I v T v v T v 1 , in which the first n elements belong to the set J , and the last M elements belong to the set R . hA riB According to the definition of the mean value, the following equation: [0B3] + )zn = (x, + ■ • ■ + ) + (X^+1... + ) = + holds true. The conclusion (13) can be obtained by dividing both sides of the above equation by the coefficient M । M . rlA -1- rlB

[0134] As a matter of fact, theorem 2 also holds true when there is no partial order relation, and the order relation is given only for describing conveniently. Similarly, this conclusion also holds true for continuous intervals, and nA and nB are replaced by the length of the intervals. The contents described in theorems 1 and 2 can be both used to accelerate the flame detection.

[0135] 2. Results and analysis

[0136] This section will verify the above conclusion. The experiment includes two parts: the flame pixel criterion and the verification of the flame detection ability of the method of the present disclosure.

[0137] 2.1. Flame criterion experiment

[0138] In order to verify the detection effect of flame criterion on forest fires, the above four criteria are experimented on forest fire images for many times. A forest fire image is illustrated as an example hereinafter, as shown in FIG. 1. On the forest fire image shown in FIG. Ie, a flame detection experiment is carried out according to four criteria. The detection results are shown in FIGS, la-Id, in which the area indicated by black has no flame detected therein, and the area indicated by other colors has flame detected therein. In the experiment, the flame pixel is determined in the color space corresponding to each rule. If a pixel is determined as "fire", the pixel is filled with the pixel in the corresponding position in the original image, otherwise the pixel in the detection result is filled with black. In the detection results, although the RGB rule of criterion 1 can detect some forest fires, there is a problem of missed flame detection, and large areas of "smoke" in the original image are mistakenly detected as "flames", which has a high false alarm rate (as shown in FIG. la). FIG. lb shows the flame pixel detection in the YCbCr space. Most of the flames can be detected correctly. Compared with the other three methods, the method has a higher flame detection ability and a lower false alarm rate. FIG. 1c shows the detection results in the HSI space. Although the false alarm rate is low, most of the flames cannot be detected correctly. Therefore, the criterion is invalid in the experiment. Criterion 4 makes use of the color information of the RGB and HSI spaces at the same time. Although criterion 4 suppresses the high false alarm rate of criterion 1, the flame detection ability is still quite limited. In the result shown in FIG. 2d, many flame pixels are missed. For forest fire images with a small area and rich details, there is a problem of inadequate detection. In addition, a large number of experiments on forest fire images have similar conclusions. Therefore, in the forest fire detection experiment hereinafter, the supervision information uses criterion 2. The detection results are still displayed in the RGB space for the convenience of comparison with the RGB original image.

[0139] 2.2. Forest fire detection experiment

[0140] 2.2.1. The forest fire detection ability is to verify the forest fire detection effect of the algorithm in the real scene. The experimental data consists of two parts. One part of the experimental data comes from the Internet, RGB forest fire images of the real forest scene with different light, different fire periods and different backgrounds are selected, and the basic information is shown in Table 1. The other part of the experimental data is taken from VisiFire, a fire video library published by Bilkent University in Turkey. Two of the forest fire videos are selected in Table 1.

[0141] The experimental environment uses the window7SP164-bit operating system, InterCorei7-3632QMCPU, 2.2GHz, 8GRAM, MatlabR2015b. Since the acquired data is in lack of pixel category labels, the detection results are still displayed by using a visualization method. The visualization method is the same as that in Section 2.1.

[0142] Table 1: Description of image information Data name Image resolution Data source Imagel 300x450 Fire in Jinzhong, Shanxi, 2011 Image2 300x400 Fire in Liangshan, Sichuan , 2014 Image3 575x950 Fire in Colorado, USA, 2013 Image4 542x900 Fire in Qitai Mountain, Yunnan, 2014 ImageS 282x472 Fire in Yuxi, Yunnan, 2012 Vedio4 256x400 VisiFire video library from Turkey VedioS 256x400 VisiFire video library from Turkey

[0143] According to the order of data names in Table 1, the corresponding forest fire detection results are shown in FIG. 2. Because the data in the VisiFire video library has been normalized, and has the characteristics of being low in resolution and not rich in details, only two videos related to forest scenes are selected in the present disclosure, which are named Vedio4 and Vedio5 according to their numbers in the video library, respectively. The original image shown in FIG. 2 is a first frame of each video. The selected forest fire images are named Imagel to Image5. In particular, the targets in the non-fire areas are complex, including targets such as the sky, trees, grassland, smoke, etc. The flame area also varies in size. The purpose of selecting images with large differences for the experiment is to test the forest fire detection ability of the algorithm in the real scene. The detection results are shown in FIG. 2. In order to explain the problem about pixel distortion in the detection results, the multi-threshold segmentation results based on spline regression are also listed in FIG. 2.

[0144] The segmented image shown in a second column of FIG. 2 has an obvious hierarchical sense. The green feature is more obvious visually, such as the forest or grassland shown in Image2, Image3 and Vedio4. Since the separability between the colors of flame pixels and green plants increases, this will facilitate subsequent target detection. In the segmentation results on Vedio5, the target near the flame tends to be red after being segmented when compared with the original image, so that similar red appears in the subsequent flame detection results. A similar phenomenon is also reflected in Image5. The phenomenon is resulted from the fact that the threshold interval where flame pixels are located is small, so that it is difficult to distinguish the flame from the reflective area of the trunk resulted from fire light irradiation. It is very likely for the visual flame area to contain non-flame pixels. For example, in the YCbCr detection results on Vedio5, non-fire targets are mistakenly determined as "flames" obviously. In addition, the information loss in calculation precision (unsigned integer 8-bit uint8 is used for displaying images, while double precisions are used for calculation) and the slight difference in a single pixel may produce such visual difference together after superposition.

[0145] In terms of detection effects, the two flame criteria are compared, as described in 1.3. The YCbCr rule of criterion 2 has a higher flame detection ability than the RGB+HSI rule of criterion 4, so that the flame detection effect achieved upon using criterion 2 as supervision information is obviously better than the flame detection effect achieved upon using criterion 4 as supervision information. This advantage is due to the combination of the two factors, that is, the flame detection ability and the misjudgment ability of non-fire targets. Additionally, it also indicates that the accuracy of the supervision information directly affects the subsequent target identification. The two criteria make misjudgment in the highlight areas of the flame, such as the flame center position, so that "black holes" appear in the flame center position of the detection results, such as those on Image2, Image3 and Image4 (FIGS. 2c-2d). Furthermore, the inaccuracy of the supervision information is further amplified by the subsequent flame detection work, so that the "black hole" area in the image is further expanded , which is more obvious on Image2 (FIGS. 2e-2f). The experimental results show that the accuracy of the supervision information will affect the subsequent target detection, that is, the criterion in the YCbCr space is more beneficial to flame detection. It is expected to achieve the goal of a high flame detection rate and a low false alarm rate at the same time.

[0146] 2.2.2. Forest fire detection time.

[0147] In the VisiFire video library, the frame rates of Vedio4 and Vedio5 are both 15FPS (frames per second). The corresponding number of frames is 246 and 208, respectively. Flame detection experiments are carried out on all frames of the video, and the required time is recorded in Table 2. For a single image, in order to prevent the random value problem resulted from too short detection time, the total time of 10 detections is recorded on Imagel to Image5. A stop watch timer of Matlab is used for timing in unit of second (s).

[0148] Table 2: Comparison of detection time Dataset Criterion 2 (second) YCbCr(s) Criterion 4 (second) RGB+HSI(s) Method of the present disclosure + criterion 2 (second) Method of the present disclosure + YCbCr(s) Method of the present disclosure + criterion 4 (second) Method of the present disclosure + RGB+HSI(s) Imagel 3.3059 4.3354 0.4273# 1.3208 Image2 5.0087 6.7590 0.9807 1.3323 Image3 5.4992 6.7706 0.4893 0.4137 Image4 2.6093 3.0045 0.9258 1.6103 Image5 3.1933 1.9689 0.5456 2.9625 Vedio4 81.5687 98.5501 14.9716 36.3637 Vedio5 49.6748 53.6009 12.6223 37.5421 Average 21.5514 24.9985 4.4232 11.6493

[0149] # indicates that the detection time of forest fires on the image is the shortest.

[0150] "RGB+HSI" in Table 2 refers to the pixel detection rule of criterion 4, which needs to combine the information of RGB and HSI color spaces. YCbCr refers to criterion 2, in which pixel detection only needs to be completed in the YCbCr space. Similarly, the method of the present disclosure + criterion 2 (criterion 2) indicates that the supervision information of the method of the present disclosure comes from the YCbCr (RGB+HSI) rule, which is used to detect whether the representative point is a "flame" area. The detection results show that the detection speed of the method of the present disclosure is rapid, and in particularly, the speed of the detection method based on the YCbCr rule is the rapidest. According to the average time, the method of the present disclosure is 4.9 times rapider than the YCbCr judging rule and 6.7 times rapider than RGB+HSI. According to the acquisition speed of 15 frames per second, it takes 16.4 seconds and 13.9 seconds to acquire videos Vedio4 and Vedio5, while the detection time of the method of the present disclosure is 15.0 seconds and 12.6 seconds, respectively, which are both shorter than the acquisition time. Therefore, it is expected that the method is applied to real-time detection of forest fires.

[0151] 3. Discussion

[0152] The rule-based flame detection method has the characteristics of being simple, intuitive and easy to visualize. However, the particularity of the flame itself, such as no shape and color uncertainty, further increases the difficulty of forest fire identification. So far, no universal flame detection method has been found. Due to the lack of a forest fire image or a video database which are labeled at a high precision (such as, at a pixel level), it is difficult to use numerical measurement for evaluation indexes such as a flame detection rate and a false alarm rate. At present, disclosed methods are mainly achieved through subjective evaluation, such as image visualization. How to use more objective standards to measure learning algorithms is also an issue that forest fire identification workers should pay attention to. Compared with the human intelligence, the human vision has a comprehensive understanding of natural images. This not only can grasp the target outline in the image from a macro perspective, but also pay attention to details from a micro perspective, which are completed in parallel. Although there are differences between the human vision and the machine vision, the research on pixel-level labeling and identification problems will inevitably help target detection at a higher level, such as labeling and identification based on image blocking (Yang Xubing et al., 2019) and real-time processing of high-definition images. In addition, for the color space problem of forest fires, the existing research is mainly based on the color space or the multi-space combination, including commonly used RGB, YCbCr, HSI, Hue Saturation Value (HSV), etc. Whether there is a certain space which is more conducive to the target detection of forest fires requires further research.

[0153] The above is for the color rule method. Another branch is based on machine learning models (Model-base detection), including a forest fire identification method of a deep network. Different from the rule method, such model can be regarded as a two-step method. That is, training samples are selected from the researched scene, the model is trained according to the selected samples, and then the trained model is used for flame detection. Therefore, the performance of such method replies on the sample selection of the scene and the training degree of the model. However, how to construct the features that are beneficial to flame detection is still an unsolved problem. Although the depth-based method does not need to take into account the problem of feature selection, the method integrates parameter selection, feature extraction and target identification. However, more and more scholars think that the deep network is in lack of interpretability, robustness and generalization, and even outputs some unexpected identification errors. Some scholars have pointed out that modifying a small number of pixels or even a single pixel in an image can cause a deep network to completely identify errors (Suetal.,2019). According to the view of machine learning, these model-based methods fail to fully incorporate the background knowledge of forest fires and are simply the application of machine learning methods. In addition, from the practical point of view, there may be no fire in the monitored target scene for a long time, or even no fire has ever happened. That is, the "flame" samples in the area are actually missing, resulting in the problem that the model cannot be trained. How to train a model suitable for the scene in the case of "no fire" is also a topic worthy of research.

[0154] 4. Conclusion

[0155] On the basis of multi-threshold image segmentation, the present disclosure proposes a rapid flame pixel detection method. By performing smooth spline regression on the color histogram, multiple thresholds are obtained and image segmentation is completed accordingly. In order to overcome the difficulty in selecting the target area resulted from the large number of combined areas in color images, a Cartesian set is introduced to describe the area and is verified using four different flame pixel criteria. The experimental results show that the Cartesian set method based on the YCbCr criterion has the characteristics of a high flame detection rate and a low false alarm rate. In addition, flame pixel detection is only completed in the same color space, so that the detection speed is the rapidest when compared with the other three methods.

[0156] The similar parts between the embodiments provided in the present disclosure can only be referred to each other. The specific implementations provided above are only a few examples under the general concept of the present disclosure, and do not constitute a limitation on the scope of protection of the present disclosure. For those skilled in the art, any other implementations expanded according to the scheme of the present disclosure without creative labor fall within the scope of protection of the present disclosure.

Claims

1. A rapid flame detection method for forest fire identification, comprising a multi-threshold segmentation method based on spline regression, which comprises four steps:step (1), carrying out multi-threshold segmentation on each color channel by using a HistSplineReg segmentation method firstly, to ensure that image segmentation details are clear, and calculating and recording segmentation areas on each channel as , SG and SB \step (2), selecting a representative point in each segmentation area, and recombining representative points according to a sorting order in a color space to form a pixel representation of an area;step (3), finally, comparing a pixel representative point with provided supervision information to retrieve a target area where the pixel representative point is located; andstep (4), acquiring a dividing subset where a "fire" representative point is located and a corresponding flame pixel interval thereof.

2. The method according to claim 1, wherein in step (1), steps of the HistSplineReg segmentation method are as follows:using a local fitting idea in the spline regression, wherein a regression curve is second-order continuous differentiable over entire interval to ensure smoothness of a regression function g(-) ;and in the case of an observation set of { (Xz,y,)}. ' = , an optimization goal is:g\x)]2dx (1)where w. is a weight parameter, g” denotes a second derivative of the function g(-) , and a smoothness parameter q satisfies 7? e [0,1];for an image, assuming that a pixel level L is 256; under an assumption that an independent variable is x* = i, which denotes an i -th gray level, yt = p(l) denotes a histogram distribution probability, simplifying Equation (1) as:nun ^[p(0-g(0]2+«f[g"(0]2^ (2)where a = 1 / r / —1(> 0) is a smoothness penalty factor;introducing two constant banded matrices q and 7? for solving the regression functiong(-) , wherein:g = (I + aK)~1Y (3)where a vector "is a discrete representation of the function g(-) , / is an identity matrix, F = , y2 ,L , )'l2 X K = ’in terms of calculation, a minimum point of the regression curve is a threshold, and then multi-threshold detection is carried out.

3. The method according to claim 2, wherein in step (2), a multi-threshold detection method comprises:defining symbols as follows: k denoting the number of thresholds, and a number of thresholds on respective color channel being denoted as Rk , Gk and Bk \ assuming that the thresholds have been sorted in an ascending order, the corresponding dividing subset being denoted as ,Sk, in which S = and S.I S.=0(iiij); assuming thatthe pixel level is 256, which being denoted as S = 1 =[0,255]; and taking a RGB image as an example, and above equation being expanded on each color channel to obtain:^R = '">SRk}>$G = {SG1>SG2> "'^Gk}’^B = {^B1’^B2’"' ’^BjJ’ (4)in order to highlight a multi-threshold segmentation effect of a color image, calling a pixel interval between adjacent thresholds as a dividing subset, wherein in the dividing subset, if a pixel is used to represent the interval, and then colors on each channel are superimposed, a hierarchical sense of image segmentation is generated visually, and each of superimposed areas is:Sk®SGj®SBi (5)wherein ® denotes a Cartesian product; wherein above color image threshold segmentation problem is transformed into a set operation of Equation (6):Q = {Sr ®Sc ®Sr 1¾ eSR,Sc ^SC,SR ^SR} (6)V kJ Di \ Kj K" kJ kJ IJi D J \ / on a segmented sub-area selecting a set of suitable representativepoints, which is denoted as ( 7*, gj, b / ), replacing pixel values in range of each of thesub-intervals Sp,SG and SB by , respectively, and completing target* J Jsegmentation on the area ®SG ®SBi .

4. The method according to claim 3, wherein in step (3), a supervision information method is introduced as follows:according to statistical analysis, and flame pixel criteria are summarized four criteria according to different color spaces which are used, and which are applied to the flame determination of the dividing subset:criterion 1: a flame pixel criterion of an RGB spaceit is assumed that the RGB value of current pixel is and if the current pixelsatisfies all rules in Equation (7), the current pixel is determined as a flame pixel; otherwise, the current pixel is determined as a non-flame;> ^mean ’ (7)G >Gmean,where and G^^ denote mean values of pixels on red and green channels,respectively;criterion 2: a flame pixel criterion of a luminance, rhrominance-blue and chrominance-red (YCbCr) spacesince information such as hue, brightness and color saturation of a color is hidden in three components of RGB and is indistinguishable, a rule of the YCbCr space is:Y >Cb,Cr >Cb, Y >Y Cb <Cb Cr >Cr \Cb — Cr\ >t; (8)criterion 3: a flame pixel criterion of a hue, saturation, intensity (HSI) spacein a color model HSI, H,S,I are corresponding to the hue, the color saturation and light intensity of pixels, respectively; the flame pixel criterion of the space is:minmax 9(9)^min < I < ^max *a recommended range is H e (0,60), Se (20,100) and / e (100,255);criterion 4: a mixed flame pixel criterion of RGB+HSIcombining respective characteristics of color spaces of RGB and HSI, the mixed flame pixel criterion is described as follows:mean ’R>G>B,            <10>S > (255-R\ S^ / R^swherein and ZVyj denote thresholds of color components S and R, andcorresponding reference ranges are 55 to 65 and 115 to 135, respectively;according to above information, an appropriate representative point is selected from each element 0 SG 0 SB of Q , and if the representative point passes rule verification, it indicates that all pixels in the segmentation area corresponding to the representative point are determined as "fire".

5. The method according to claim 4, wherein in step (4), a detection algorithm comprises following five steps:acquiring the dividing subset where "fire" representative point is located and a corresponding flame pixel interval thereof:arranging SR,SG and SB according to magnitude of thresholds, and then merging the representative points of adjacent subsets; assuming that elements in set shown in Equation (4) are arranged in an ascending order of the thresholds; and describing adjacent subsets and their definitions on Cartesian product as follows:theorem 1: sets X, A2 , b and C denote intervals in a one-dimensional space; assuming that Cartesian sets A^® B®C and A2®B®C are adj acent, in which\Ji(Ai®B®C)=D®B®C, / = 1,2 (12)holds true , in which D = and it is a convex set;theorem 2 (adjacent mean values are merged): assuming that mA and mB are mean values on adjacent sets A and B, so that the mean value of sets are mergedm nAmA + nBmB nA+nB(13)where n , and nR denote a number of elements of the sets, respectively;assuming that a sub-area containing detection target "flame" is:(11)S.SSr,Sg.SSg,S.sSbif adjacent Cartesian sets are contained in Q', merging intervals according to the theorem 1, so as to further reduce items in Q' , and achieve a purpose of reducing a number of traversals and improving detection speed; merging the representative points according to the theorem 2, replacing nA and nB by length of intervals for continuous intervals, and merging the representative points to improve the detection speed.

Images