A flame recognition method based on deep learning

Abstract

A flame recognition method based on deep learning, which relates to the field of image recognition technology. Aiming at the problems in the prior art that it is extremely easy to misdetect a slightly moving background as a moving target and misdetect a stationary background as a moving target in some cases, resulting in low accuracy of flame recognition, this application combines a moving target detection and an image classification network. The moving target detection is used to screen out the moving targets, and then the image classification network is used to determine whether the moving target is a flame, which can accurately and efficiently detect the flame in the video and improve the accuracy of flame recognition. Moreover, this application improves the ViBe algorithm, making the improved ViBe algorithm more accurate in screening moving targets.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and specifically to a flame recognition method based on deep learning. Background Art

[0002] In recent years, with the wide application of edge computing technology, digital communication technology, computer technology, and digital image processing technology. How to automatically, quickly, and efficiently discover fires through video images and prevent fires has become a key research issue.

[0003] In the prior art, a method for recognizing flames by recognizing moving targets in an image sequence is disclosed. However, this method is prone to misdetecting slightly moving backgrounds as moving targets and in some cases misdetecting stationary backgrounds as moving targets, resulting in low accuracy of flame recognition. Summary of the Invention

[0004] The object of the present invention is to propose a flame recognition method based on deep learning for the problem that the prior art is prone to misdetecting slightly moving backgrounds as moving targets and in some cases misdetecting stationary backgrounds as moving targets, resulting in low accuracy of flame recognition.

[0005] The technical solution adopted by the present invention to solve the above technical problems is:

[0006] A flame recognition method based on deep learning includes the following steps:

[0007] Step 1: Obtain an image sequence;

[0008] Step 2: Process the image sequence using the ViBe algorithm. The specific steps are as follows:

[0009] Step 2-1: Initialize the background model using the first frame image of the image sequence, and perform foreground object segmentation on the subsequent image sequence using the initialized background model;

[0010] Step 2-2: Calculate the number of pixels included in the foreground object, and sort them in descending order according to the number of pixels;

[0011] Step 2-3: Select the foreground object corresponding to the largest number of pixels count max as the moving target Tatget, and obtain the maximum width T D of this foreground object and the distance D i from it to other foreground objects, that is, the distance from it to other connected domains;

[0012] Step 2-4: Set the threshold T count to 1 / 10 count max, screen all foreground targets according to the sorting. The specific process is as follows:

[0013] If the foreground target contains a number of pixels less than the threshold T count , and the distance between this foreground target and the moving target Tatget is greater than the maximum foreground target width T D , then it is marked as a background target and filtered, otherwise the target is marked as a moving target;

[0014] Step 25: Update the background model according to the time sampling factor;

[0015] Step 3: Use the classification network to identify flames for the moving targets.

[0016] Furthermore, the classification network is a GoogleNet network.

[0017] Furthermore, the GoogleNet network is an improved GoogleNet network. The improvement steps of the GoogleNet network are specifically as follows:

[0018] First, replace the ReLU activation function with the PReLU activation function, and finally introduce the attention mechanism into the Inception module of the GoogleNet network.

[0019] Furthermore, before step 25, there is also a ghosting detection step. The ghosting detection is specifically as follows:

[0020] Perform ghosting detection on the moving targets obtained in step 24. If ghosting occurs, eliminate the ghosting by adjusting the time sampling factor.

[0021] Furthermore, the elimination of ghosting by adjusting the time sampling factor is specifically as follows:

[0022] Set a dynamic time sampling factor λ for the background model and a counter pghost for measuring the possibility of ghosting. pghost is initially 0. When the current pixel is judged to be a foreground pixel, compare the value of this pixel with the value of the pixel at the same position in the previous frame. If the Euclidean distance between the two is not less than the preset threshold E th , then increment the value of pghost by 1, otherwise set pghost to 0;

[0023] When p ghost is greater than the preset value T ghost , it is considered that the current pixel is a ghost pixel and update it to the background model;

[0024] ] After obtaining all the ghost pixels, calculate the minimum circumscribed rectangle of the connected domain of the ghost pixels, that is, the ghosting area, and then set the time sampling factor of the pixels within the ghosting area to half of the original time sampling factor.

[0025] Furthermore, the specific steps for initializing the background model are as follows:

[0026] For each pixel point p(x) in the first frame image, randomly select pixel points from the 8-neighborhood of p(x) as samples and put them into the sample model M0(x) corresponding to p(x) to complete the initialization of p(x);

[0027] After the initialization of the sample models corresponding to all pixel points in the first frame image, the initialization of the background model is completed.

[0028] Furthermore, when the threshold T count is less than 10 pixels, set the threshold to 10.

[0029] Furthermore, the screening in step 24 is expressed as:

[0030]

[0031] where Ai represents the current foreground target, count A represents the number of pixels of the current foreground target, A Tatget represents that the current foreground target is a moving target, A noise represents that the current foreground target is a background target to be filtered out.

[0032] Furthermore, the elimination of ghosting by adjusting the time sampling factor is expressed as:

[0033]

[0034] where p ghost represents the possibility that the current pixel is a ghost pixel, p fore represents whether the current pixel is a foreground pixel, p fore =1 indicates that the current pixel is judged as a foreground pixel, p fore =0 indicates that the current pixel is judged as a background pixel; E(x) represents the result of comparing the Euclidean distance between the current pixel and the previous frame pixel with the threshold E th When the Euclidean distance is greater than or equal to E th then E(x)=1, otherwise E(x)=0.

[0035] The beneficial effects of the present invention are:

[0036] This application combines moving target detection and an image classification network. It uses moving target detection to screen out moving targets, and then uses the image classification network to determine whether the moving target is a flame, which can accurately and efficiently detect flames in videos and improve the accuracy of flame recognition. And this application improves the ViBe algorithm, making the improved ViBe algorithm more accurate in screening moving targets. Description of the Drawings

[0037] Figure 1 Flow chart for improving the ViBe algorithm;

[0038] Figure 2 Schematic diagram of the Inception module with an attention mechanism added in GoogleNet for improvement;

[0039] Figure 3 Schematic diagram of the overall structure of this application. Detailed Implementation Modes

[0040] It should be specifically noted that, without conflict, the various implementation modes disclosed in this application can be combined with each other.

[0041] Detailed Implementation Mode 1: Refer to Figure 1 This detailed implementation mode will be specifically described. A flame recognition method based on deep learning described in this detailed implementation mode includes the following steps:

[0042] Step 1: Obtain an image sequence;

[0043] Step 2: Process the image sequence using the ViBe algorithm. The specific steps are as follows:

[0044] Step 2-1: Initialize the background model using the first frame image of the image sequence, and perform foreground object segmentation on the subsequent image sequence using the initialized background model;

[0045] Step 2-2: Calculate the number of pixels included in the foreground object, and sort them in descending order according to the number of pixels;

[0046] Step 2-3: Select the foreground object corresponding to the number of pixels count max with the largest number of pixels as the moving target Tatget, and obtain the maximum width T D of this foreground object and its distance D i from other foreground objects, that is, its distance from other connected domains;

[0047] Step 2-4: Set the threshold T count to 1 / 10 count max , and screen all foreground objects according to the sorting. The specific process is as follows:

[0048] If the number of pixels included in the foreground object is less than the threshold T count , and the distance between this foreground object and the moving target Tatget is greater than the maximum foreground object width T D , then it is marked as a background object and filtered, otherwise this object is marked as a moving target;

[0049] Step 25: Update the background model according to the time sampling factor;

[0050] Step 3: Use the classification network to perform flame recognition on the moving target.

[0051] Extract the moving target in the video through the improved ViBe algorithm, and then send the image of the area where the moving target is located into the trained image classification model. If the image classification model classifies the image as an image with fire, it indicates that there is a flame in the image and an alarm is issued; otherwise, continue to detect the next moving target.

[0052] Specific Embodiment 2: This embodiment is a further description of Specific Embodiment 1. The difference between this embodiment and Specific Embodiment 1 is that the classification network is a GoogleNet network.

[0053] Specific Embodiment 3: This embodiment is a further description of Specific Embodiment 2. The difference between this embodiment and Specific Embodiment 2 is that the GoogleNet network is an improved GoogleNet network. The improvement steps of the GoogleNet network are specifically as follows:

[0054] First, replace the ReLU activation function with the PReLU activation function, and finally introduce the attention mechanism into the Inception module of the GoogleNet network.

[0055] Specific Embodiment 4: This embodiment is a further description of Specific Embodiment 3. The difference between this embodiment and Specific Embodiment 3 is that a ghost detection step is further included before Step 25. The ghost detection is specifically as follows:

[0056] Perform ghost detection (ghost detection) on the moving target obtained in Step 24. If a ghost appears, eliminate the ghost by adjusting the time sampling factor.

[0057] Specific Embodiment 5: This embodiment is a further description of Specific Embodiment 4. The difference between this embodiment and Specific Embodiment 4 is that the method of eliminating ghosts by adjusting the time sampling factor is specifically as follows:

[0058] Set a dynamic time sampling factor λ for the background model and a counter pghost for measuring the possibility of ghosts. The initial value of pghost is 0. When the current pixel is judged to be a foreground pixel, compare the value of this pixel with the value of the pixel at the same position in the previous frame. If the Euclidean distance between the two is not less than the preset threshold E th , then increment the value of pghost by 1; otherwise, set pghost to 0;

[0059] When p ghost is greater than the preset value T ghostWhen it is, the current pixel is considered a ghost pixel and updated into the background model;

[0060] After obtaining all ghost pixels, calculate the minimum bounding rectangle of the connected components of the ghost pixels, that is, the ghost region, and then set the temporal sampling factor of the pixels within the ghost region to half of the original temporal sampling factor.

[0061] Specific Embodiment Six: This embodiment is a further description of Specific Embodiment One. The difference between this embodiment and Specific Embodiment One is that the specific steps for initializing the background model are as follows:

[0062] For each pixel point p(x) of the first frame image, randomly select a pixel point from the 8-neighborhood of p(x) as a sample and put it into the corresponding sample model M0(x) of p(x) to complete the initialization of p(x);

[0063] After the initialization of the sample models corresponding to all pixel points in the first frame image, the initialization of the background model is completed.

[0064] Specific Embodiment Seven: This embodiment is a further description of Specific Embodiment Five. The difference between this embodiment and Specific Embodiment Five is that when the value of the threshold Tcount is less than 10 pixels, the threshold is set to 10.

[0065] Specific Embodiment Eight: This embodiment is a further description of Specific Embodiment One. The difference between this embodiment and Specific Embodiment One is that the screening in Step 24 is expressed as:

[0066]

[0067] Among them, Ai represents the current foreground object, count A represents the number of pixels of the current foreground object, A Tatget represents that the current foreground object is a moving object, A noise represents that the current foreground object is a background object to be filtered out.

[0068] Specific Embodiment Nine: This embodiment is a further description of Specific Embodiment Five. The difference between this embodiment and Specific Embodiment Five is that the elimination of ghosts by adjusting the temporal sampling factor is expressed as:

[0069]

[0070] Among them, p ghost represents the possibility that the current pixel is a ghost pixel, p fore represents whether the current pixel is a foreground pixel, p fore =1 represents that the current pixel is judged as a foreground pixel, p fore= 0 indicates that the current pixel is determined to be a background pixel; E(x) represents the Euclidean distance between the current pixel and the pixel in the previous frame and the threshold E th The result of the comparison, when the Euclidean distance is greater than or equal to E th then E(x) = 1, otherwise E(x) = 0.

[0071] ViBe algorithm (moving object detection), GoogleNet image classification model (deep learning)

[0072] Principle and deficiencies of the ViBe algorithm:

[0073] The ViBe algorithm is a visual background extractor algorithm proposed by Olivier Barnich and Marc Van Droogenbroeck. Facts have shown that the ViBe algorithm is an excellent algorithm that can quickly establish a background model to extract foreground moving objects. It can adapt to the influence brought by slow changes in illumination to a certain extent by adjusting its own parameters. Compared with other similar algorithms, the ViBe algorithm has the advantages of simple principle, good segmentation effect, high detection efficiency, etc., and has achieved good results in practice and is widely used in the field of target detection. ViBe is mainly divided into four parts: background pixel point modeling, single-frame initialization of the background model, pixel point classification process, and background model update strategy.

[0074] (1) Background pixel point modeling

[0075] The ViBe algorithm establishes the same number of sample models according to the number of pixels in the input image. Each sample model is a set of pixel values, corresponding to the position of the corresponding pixel. The set contains N pixel value samples associated with this position.

[0076] (2) Single-frame initialization of the background model

[0077] Before classifying pixel points, ViBe needs to initialize the model first, usually using the first frame of the image to complete the initialization of the model. For each pixel point p(x) in the first frame of the image, randomly select pixel points from the 8-neighborhood of p(x) as samples and put them into the sample model M0(x) corresponding to p(x) to complete the initialization of p(x).

[0078] (3) Pixel point classification

[0079] The ViBe algorithm classifies pixel points into two categories, one is the background and the other is the foreground, based on the number of Euclidean distances between the pixel in the image and all sample pixels in its sample model within a set range, so as to distinguish moving objects and the background.

[0080] (4) Model update mechanism

[0081] To reduce the impact of light changes on image pixels, the ViBe algorithm uses the method of dynamically updating samples in the model to improve its own accuracy. The update of the model is mainly divided into three parts: memoryless update strategy, spatial neighborhood diffusion mechanism, and temporal resampling diffusion of background pixels. The specific steps are as follows: If a pixel is classified as a background pixel, there is a probability of 1 / φ (where φ is the temporal sampling factor) to overwrite a sample value in the background model of this pixel and become a new background model sample, thus completing the update of the current pixel sample model. After the update of the current pixel sample model is completed, a neighborhood pixel of the current pixel is randomly selected, and the current pixel value is used as a model sample to update the sample model of this neighborhood pixel. This update mechanism that can spread the current pixel value to the neighborhood pixel sample model is called the spatial neighborhood diffusion mechanism.

[0082] Compared with the frame difference method, the ViBe algorithm has higher accuracy. Compared with the Gaussian mixture model, it has a simpler principle and smaller computational complexity. Therefore, it is widely used in the field of moving target detection. However, the ViBe algorithm itself also has deficiencies. It is very easy to misdetect slightly moving backgrounds as moving targets and in some cases misdetect stationary backgrounds as moving targets, resulting in ghosting. This algorithm analyzes the causes of the problems in the ViBe algorithm and improves it to enable more accurate detection of moving targets.

[0083] Introduction to the GoogleNet model and its deficiencies:

[0084] GoogLeNet is a network model designed by the Google team in the 2014 ILSVRC competition. It won the championship of that competition with a top-5 error rate of 6.67% on ImageNet. GoogLeNet not only increased the network depth, improved the network accuracy, but also reduced the number of parameters. GoogLeNet innovatively proposed the Inception module and used the NIN (NetInNet) structure to replace the ordinary convolutional layer to improve the network efficiency. The Inception module uses convolutions of different sizes to extract different features, which can better extract the feature information of the image and at the same time allows combinations between the outputs of each branch, improving the expression ability of the model without increasing the number of output channels.

[0085] GoogleNet improves the network accuracy while reducing the number of parameters. However, the ReLU activation function it uses may cause neuron necrosis when the input is less than 0. Therefore, there are still deficiencies. This algorithm replaces the ReLU activation function of GoogleNet with PReLU and sets the part where the input is less than 0 to a negative value to solve the phenomenon of neuron necrosis. At the same time, this algorithm introduces the attention mechanism into the Inception module of GoogleNet (as shown in the appendix Figure 2as shown in the figure), thus increasing the accuracy of the model.

[0086] (1) ViBe algorithm:

[0087] Improve the ViBe algorithm in the classical moving object detection algorithm. Analyze the reasons for the misdetection of background objects and the generation of ghosts in the classical ViBe algorithm and improve the algorithm accordingly to achieve the goal of filtering out misdetected background objects and accelerating the elimination of ghosts.

[0088] Filter out background objects:

[0089] ① Calculate the number of pixels count i contained in each foreground object A i , and sort them from largest to smallest according to the number of pixels;

[0090] ② Find out the number of pixels count max contained in the largest foreground object among all foreground objects, its maximum width T D and its distance D i from other connected components;

[0091] ③ Set the threshold T count to 1 / 10 count max , default the largest foreground object as the moving object, and screen all foreground objects in descending order. If the foreground object contains fewer pixels than the threshold and its distance from the moving object is greater than the maximum width T D of the largest foreground object, then mark it as a background object and perform a filtering operation on it. Otherwise, mark the object as a moving object. The screening process is shown in formula (3-1):

[0092]

[0093] It should be noted that when the value of the threshold T count is less than 10 pixels, set the threshold to 10;

[0094] Accelerate the elimination of ghosts:

[0095] The overall idea of accelerating ghost elimination is that if a pixel is continuously judged as a foreground target and its pixel value does not change significantly, then this pixel is considered a ghost pixel. Incorporate this pixel into the background model of this point and adjust the time sampling factor of the ghost region to accelerate the update speed of the model in this region, thereby accelerating the speed of ghost elimination. The specific approach is as follows: Set a dynamic time sampling factor λ and a counter pghost for measuring the likelihood of ghosting for the background model M(x). λ determines the update rate of this pixel model, and pghost represents the likelihood that the current pixel is a ghost pixel, initialized to 0. When the current pixel is judged as a foreground pixel, compare the value of this pixel with the pixel value at the same position in the previous frame. If the Euclidean distance between the two is less than the preset threshold Eth, then increment the value of pghost by 1; if the current pixel is detected as a background pixel or the Euclidean distance between the pixel value at the same position in the current pixel and the previous frame is greater than the threshold Eth, then set pghost to 0. This method can be expressed by formula (3-2):

[0096]

[0097] where p ghost represents the likelihood that the current pixel is a ghost pixel, p fore represents whether the current pixel is a foreground pixel, p fore = 1 indicates that the current pixel is judged as a foreground pixel, p fore = 0 indicates that the current pixel is judged as a background pixel; E(x) represents the result of comparing the Euclidean distance between the current pixel and the pixel in the previous frame with the threshold E th When the Euclidean distance is greater than or equal to E th , E(x) = 1, otherwise E(x) = 0. When p ghost is greater than the preset value T ghost , the current pixel is considered a ghost pixel and updated into the background model.

[0098] The algorithm marks pixels that are continuously judged as foreground pixels within a period of time but whose own pixel values do not change significantly as ghost pixels, and adds them as background samples to the background model; after obtaining all ghost pixels, calculate the minimum bounding rectangle of the connected domain of the ghost pixels, and call this rectangle the ghost region. Then, combine it with the spatial information neighborhood diffusion mechanism in the classical ViBe algorithm. By adjusting the time sampling factor of the ghost region, accelerate the model update speed of the ghost region, make the spatial information neighborhood diffusion mechanism more efficient, and can eliminate the ghost region faster, reducing the impact of the ghost region on the target detection effect.

[0099] (2) Image classification network:

[0100] Improve GoogLeNet in the convolutional neural network. On the basis of the traditional GoogLeNet, modify its activation function and add a channel attention mechanism to the Inception module, so that the network can have a more accurate classification effect.

[0101] Replace the activation function: Replace the ReLU function with the PReLU to solve the problem of neuron necrosis, and the gradient of the negative part can be continuously adjusted during the training of the network to achieve the best effect.

[0102] Add an attention mechanism: Use the channel attention mechanism to improve GoogLeNet and enhance the performance of the model.

[0103] It should be noted that the specific implementation manners are only explanations and illustrations of the technical solutions of the present invention, and the scope of the right protection cannot be limited thereby. Any changes that are merely partial based on the claims and the description of the present invention should still fall within the protection scope of the present invention.

Claims

1. A flame recognition method based on deep learning, characterized in that It includes the following steps: Step 1: Obtain an image sequence; Step 2: Process the image sequence using the ViBe algorithm. The specific steps are as follows: Step 2-1: Initialize the background model using the first frame image of the image sequence, and perform foreground object segmentation on the subsequent image sequence using the initialized background model; Step 2-2: Calculate the number of pixels contained in the foreground object, and sort them in descending order according to the number of pixels; Step 2-3: Select the pixel number count with the first ranking max The corresponding foreground target is used as the moving target Tatget, and the maximum width T of the foreground target is obtained D And the distance D between it and other foreground targets i , that is, the distance between it and other connected components; Step 24: Set the threshold T count to be 1 / 10 count max , and screen all foreground targets according to the sorting. The specific process is as follows: If the number of pixels contained in the foreground target is less than the threshold T count , and the distance between the foreground target and the moving target Tatget is greater than the maximum foreground target width T D , then it is recorded as a background target and filtered, otherwise the target is recorded as a moving target; Step 2-5: Update the background model according to the time sampling factor; Step 3: Use a classification network to perform flame recognition on the moving object.

2. The method for flame recognition based on deep learning according to claim 1, wherein The classification network is the GoogleNet network.

3. The method for flame recognition based on deep learning according to claim 2, wherein The GoogleNet network is an improved GoogleNet network. The improvement steps of the GoogleNet network are specifically as follows: First, replace the ReLU activation function with the PReLU activation function, and finally introduce the attention mechanism into the Inception module of the GoogleNet network.

4. The method for flame recognition based on deep learning according to claim 3, wherein Before Step 2-5, there is also a ghosting detection step. The ghosting detection is specifically as follows: Perform ghosting detection on the moving object obtained in Step 2-4. If ghosting occurs, eliminate the ghosting by adjusting the time sampling factor.

5. The method for flame recognition based on deep learning according to claim 4, characterized in that The elimination of ghosting by adjusting the time sampling factor is specifically as follows: Set a dynamic time sampling factor λ and a counter pghost for measuring the likelihood of ghosting for the background model. pghost is initially 0. When the current pixel is determined to be a foreground pixel, compare the value of this pixel with the pixel at the same position in the previous frame. If the Euclidean distance between the two is not less than the preset threshold E th , then increment the value of pghost by 1; otherwise, set pghost to 0. When p ghost is greater than a preset value T ghost it is considered that the current pixel is a ghost pixel and is updated into the background model; After obtaining all the ghost pixels, calculate the minimum bounding rectangle of the connected component of the ghost pixels, that is, the ghosting area, and then set the time sampling factor of the pixels in the ghosting area to half of the original time sampling factor.

6. The method for flame recognition based on deep learning according to claim 1, wherein The specific steps for initializing the background model are as follows: For each pixel point p(x) in the first frame image, randomly select a pixel point from the 8-neighborhood of p(x) as a sample and put it into the sample model M0(x) corresponding to p(x) to complete the initialization of p(x); After the sample models corresponding to all pixel points in the first frame image are initialized, the initialization of the background model is completed.

7. The method for flame recognition based on deep learning according to claim 5, characterized in that: When the threshold value T count is less than 10 pixels, set the threshold to 10.

8. A flame recognition method based on deep learning according to claim 1, characterized in that The screening in Step 2-4 is expressed as: Among them, Ai represents the current foreground target, and count A represents the number of pixels of the current foreground target, and A Tatget represents that the current foreground target is a moving target, and A noise represents that the current foreground target is a background target to be filtered out.

9. A flame recognition method based on deep learning according to claim 5, characterized in that The elimination of ghosting by adjusting the time sampling factor is expressed as: Among them, p ghost represents the possibility that the current pixel is a ghost pixel, and p fore represents whether the current pixel is a foreground pixel. p fore = 1 indicates that the current pixel is determined to be a foreground pixel, and p fore = 0 indicates that the current pixel is determined to be a background pixel; E(x) represents the result of comparing the Euclidean distance between the current pixel and the pixel in the previous frame with the threshold E th . When the Euclidean distance is greater than or equal to E th , E(x) = 1; otherwise, E(x) = 0.

Images