A smoke determination method based on video analysis

Abstract

The present invention relates to the field of smoke detection, and discloses a smoke determination method based on video analysis, specifically a method for performing HSV threshold filtering on a video image and analyzing the change rate of the video image to determine whether there is smoke. The present invention first uses the HSV color space to filter some non-smoky areas in the video image, continuously counts the pixel changes of adjacent frames of the video image after filtering, and then uses the YOLOv5 neural network to detect the image and find the suspected smoke area, and then accumulates the image pixel changes for a period of time before and after the suspected smoke area is found, and finally compares the total amount of image pixel changes for a period of time before and after the suspected smoke area to determine whether there is smoke.

Description

Technical Field

[0001] The present invention relates to the field of smoke detection, and in particular to a method for performing HSV threshold filtering on a video image and analyzing the video image change rate to thereby determine whether smoke exists. Background Art

[0002] Video smoke detection is an important way to detect and find early fires. Object detection networks, such as YOLO and Faster-RCNN, can achieve higher accuracy and faster detection speed after being trained on smoke datasets. However, since smoke datasets are very small and smoke itself has no fixed form, object detection networks often have a certain missed detection rate and false detection rate, which is unacceptable for fire safety.

[0003] Before the target detection network was applied to smoke detection, smoke detection systems based on hand-crafted features were widely used. Filtering and screening images according to the texture, color and dynamic characteristics of the smoke can effectively reduce the false detection rate. Although the target detection network occupies the mainstream in the current image detection field, the small number of smoke data sets and the irregular morphological characteristics of smoke make it impossible for a single target detection network to achieve satisfactory results. It is also difficult to effectively utilize continuous video image information.

[0004] Therefore, a smoke determination method based on video analysis is proposed by combining neural network with traditional image processing methods. Summary of the invention

[0005] The purpose of the present invention is to use the HSV color space to filter some non-smoky areas in the video image, continuously count the pixel changes of adjacent frames of the video image after filtering, and then use the YOLOv5 neural network to detect the image and find the suspected smoke area. The image pixel changes before and after the suspected smoke area are accumulated respectively, and finally the total amount of image pixel changes before and after the suspected smoke area is compared to determine whether there is smoke, so as to further improve the detection accuracy and reduce the false alarm rate.

[0006] The technical solution of the present invention is as follows:

[0007] A smoke determination method based on video analysis comprises the following steps:

[0008] Step 1: Collect n frames of images from the surveillance video, denoted as I1, I2, …, I n ; Use YOLOv5 to analyze any i-th frame image I i Perform smoke detection and obtain K i candidate smoke regions, and the kth candidate smoke region is recorded as in, represents the coordinates of the upper left corner of the kth candidate smoke region, represents the coordinates of the lower right corner of the kth candidate smoke area;

[0009] Step 2: Calculate the changes of adjacent frames of n frames of image and obtain the image set S = {I 1a ,I 2a ,…,I n-1a}, the specific steps are as follows:

[0010] Step 2.1: For any I i and I i+1 Perform HSV threshold filtering, i = 1, 2, ..., n-1, and convert the image from RGB color space to HSV color space to obtain image G i and G i+1 , set the filter condition Q = {V∈[150,255],S∈[0,25]}, for G i and G i+1 The HSV value of each pixel is determined by using equations (1), (2) and (3). i and I i+1 Filter and get I i and I i+1 Image I after filtering under filtering condition Q iq and I i+1q , I i and I i+1 The RGB values ​​of pixels that do not meet the filtering conditions are set to zero. The HSV threshold filtering formula is as follows:

[0011]

[0012]

[0013]

[0014] Among them, I iR (x,y),I iG (x,y) and I iB (x,y) are I i The RGB value of the pixel at the coordinate (x, y) in G iS (x,y) and G iV (x,y) are I i The SV value of the pixel at the coordinate (x, y) in iqR ,I iqG and I iqB I i The RGB value of the pixel at coordinate (x, y) after threshold filtering is stored in the filtered image I iq For I i+1Perform the same operation to obtain the filtered image I i+1q ;

[0015] Step 2.2: Calculate I according to formula (4) iq The sum of the RGB values ​​of the pixel at any coordinate (x, y) in iq (x,y), and the same method is used to calculate I i+1q The sum of the RGB values ​​of the pixel at any coordinate (x, y) in i+1q (x, y); Calculate the adjacent frame I according to formula (5) iq and I i+1q Change image I ia ;

[0016] C iq (x,y)=I iqR (x,y)+I iqG (x,y)+I iqB (x,y) (4)

[0017]

[0018] Among them, I ia (x,y) represents I ia The value of the pixel at the coordinate (x, y) in ;

[0019] Step 2.3: For image I ia Carry out corrosion and expansion;

[0020] Step 3: Take S1 = {I i-t-1a ,I i-ta ,…,I i-1a},S2={I ia ,I i+1a ,…,I i+t-1a}, S1 is the adjacent frame change image of t frames before the i-th frame image, S2 is the adjacent frame change image of t frames after the i-th frame image, 1≤it≤n, according to formula (6), the cumulative change of image pixels in S1 and S2 is calculated to obtain images R1 and R2;

[0021]

[0022] Among them, | represents the bitwise OR operation;

[0023] Step 4: Select candidate smoke regions from R1 and R2 Overlapping images, we get R 1ik and R 2ik , Statistics R 1ik and R 2ik The number of pixels in the three channels of the image is 255, denoted as U 1ik and U2ik , if U 1ik / U 2ik <0.5, then it is considered There is smoke.

[0024] The beneficial effects of the present invention are as follows: the convolutional neural network model can be effectively used to detect the video images before and after the smoke, and the total amount of image pixel changes between adjacent frames of the monitoring video over a period of time can be analyzed. The influence of video noise and jitter on the image pixel change statistics is reduced through HSV color space filtering and corrosion and expansion operations, and some false detection phenomena caused by insufficient accuracy of the neural network are eliminated. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is an unprocessed video image of the present invention;

[0026] Figure 2 The image is detected by the convolutional neural network model of the present invention;

[0027] Figure 3 The image is an image that has been filtered by HSV and the filtering condition is Q according to the present invention;

[0028] Figure 4 It is the change image of adjacent frames of the present invention;

[0029] Figure 5 The image after corrosion and expansion of the present invention;

[0030] Figure 6 The present invention accumulates images of pixel changes of 100 frames of adjacent frames. DETAILED DESCRIPTION

[0031] The present invention is described in detail below with reference to examples and images.

[0032] A smoke determination method based on video analysis, the specific steps are as follows:

[0033] Step 1: Collect n frames of images from the surveillance video, denoted as I1, I2, …, I n , Figure 1 is one of the unprocessed video images; the convolutional neural network model is used to process any i-th frame image I i Perform smoke detection and obtain K i candidate smoke regions, and the kth candidate smoke region is recorded as in, represents the coordinates of the upper left corner of the kth candidate smoke region, represents the coordinates of the lower right corner of the kth candidate smoke region, Figure 2 for Figure 1The image after detection using the convolutional neural network model, where smoke represents the detected object category, 0.39 represents the confidence level, and the box represents a candidate smoke area;

[0034] Step 2: Calculate the changes of adjacent frames of n frames of image and obtain the image set S = {I 1a ,I 2a ,…,I n-1a}, the specific steps are as follows:

[0035] Step 2.1: For any I i and I i+1 Perform HSV threshold filtering, i = 1, 2, ..., n-1, and convert the image from RGB color space to HSV color space to obtain image G i and G i+1 , set the filter condition Q = {V∈[150,255],S∈[0,25]}, for G i and G i+1 The HSV value of each pixel is determined by using equations (1), (2) and (3). i and I i+1 Filter and get I i and I i+1 Image I after filtering under filtering condition Q iq and I i+1q , I i and I i+1 The RGB values ​​of pixels that do not meet the filtering conditions are set to zero. The HSV threshold filtering formula is as follows:

[0036]

[0037]

[0038]

[0039] Among them, I iR (x,y),I iG (x,y) and I iB (x,y) are I i The RGB value of the pixel at the coordinate (x, y) in G iS (x,y) and G iV (x,y) are I i The SV value of the pixel at the coordinate (x, y) in I iqR ,I iqG and I iqB I i The RGB value of the pixel at coordinate (x, y) after threshold filtering is stored in the filtered image I iqFor I i+1 Perform the same operation to obtain the filtered image I i+1q ; Figure 3 for Figure 1 After the HSV filter and the filter condition is Q, as shown in the figure, there are a few pixels that meet the conditions left after filtering;

[0040] Step 2.2: Calculate I according to formula (4) iq The sum of the RGB values ​​of the pixel at any coordinate (x, y) in iq (x,y), and the same method is used to calculate I i+1q The sum of the RGB values ​​of the pixel at any coordinate (x, y) in i+1q (x, y); Calculate the adjacent frame I according to formula (5) iq and I i+1q Change image I ia , Figure 4 for Figure 3 and the change images of its adjacent frames;

[0041] C iq (x,y)=I iqR (x,y)+I iqG (x,y)+I iqB (x,y) (4)

[0042]

[0043] Among them, I ia (x,y) represents I ia The value of the pixel at the coordinate (x, y) in ;

[0044] Step 2.3: For image I ia Corrosion and expansion, Figure 5 for Figure 4 After the image is eroded and expanded, small interfering pixels are filtered out;

[0045] Step 3: Take S1 = {I i-t-1a ,I i-ta ,…,I i-1a},S2={I ia ,I i+1a ,…,I i+t-1a}, S1 is the adjacent frame change image of t frames before the i-th frame image, S2 is the adjacent frame change image of t frames after the i-th frame image, 1≤it≤n, according to formula (6), the cumulative change of image pixels in S1 and S2 is calculated to obtain images R1 and R2, Figure 6 This is an image that accumulates pixel changes in 100 adjacent frames. As can be seen in the figure, the candidate smoke area has more accumulated pixels than other non-candidate smoke areas;

[0046]

[0047] Among them, | represents the bitwise OR operation;

[0048] Step 4: Select candidate smoke regions from R1 and R2 Overlapping images, we get R 1ik and R 2ik , Statistics R 1ik and R 2ik The number of pixels in the three channels of the image is 255, denoted as U 1ik and U 2ik , if U 1ik / U 2ik <0.5, then it is considered There is smoke.

[0049] The contents described in the embodiments of this specification are merely an enumeration of implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as limited to the specific forms described in the embodiments. The protection scope of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A smoke determination method based on video analysis, characterized in that: The steps include: Step 1: Collect n frames of images from the surveillance video, denoted as I1, I2, …, I n ; Use YOLOv5 to analyze any i-th frame image I i Perform smoke detection and obtain K i candidate smoke regions, and the kth candidate smoke region is recorded as in, represents the coordinates of the upper left corner of the kth candidate smoke region, represents the coordinates of the lower right corner of the kth candidate smoke area; Step 2: Calculate the changes of adjacent frames of n frames of image and obtain the image set S = {I 1a ,I 2a ,…,I n-1a }; Step 2.1: For any I i and I i+1 Perform HSV threshold filtering, i = 1, 2, ..., n-1, and convert the image from RGB color space to HSV color space to obtain image G i and G i+1 , set the filter condition Q = {V∈[150,255],S∈[0,25]}, for G i and G i+1 The HSV value of each pixel is determined by using equations (1), (2) and (3). i and I i+1 Filter and get I i and I i+1 Image I after filtering under filtering condition Q iq and I i+1q , I i and I i+1 The RGB values ​​of pixels that do not meet the filtering conditions are set to zero. The HSV threshold filtering formula is as follows: Among them, I iR (x,y),I iG (x,y) and I iB (x,y) are I i The RGB value of the pixel at the coordinate (x, y) in G iS (x,y) and G iV (x,y) are I i The SV value of the pixel at the coordinate (x, y) in I iqR ,I iqG and I iqB I i The RGB value of the pixel at coordinate (x, y) after threshold filtering is stored in the filtered image I iq For I i+1 Perform the same operation to obtain the filtered image I i+1q ; Step 2.2: Calculate I according to formula (4) iq The sum of the RGB values ​​of the pixel at any coordinate (x, y) in iq (x,y), and the same method is used to calculate I i+1q The sum of the RGB values ​​of the pixel at any coordinate (x, y) in i+1q (x, y); Calculate the adjacent frame I according to formula (5) iq and I i+1q Change image I ia ; C iq (x,y)=I iqR (x,y)+I iqG (x,y)+I iqB (x,y) (4) Among them, I ia (x,y) represents I ia The value of the pixel at the coordinate (x, y) in ; Step 2.3: For image I ia Carry out corrosion and expansion; Step 3: Take S1 = {I i-t-1a ,I i-ta ,…,I i-1a },S2={I ia ,I i+1a ,…,I i+t-1a }, S1 is the adjacent frame change image of t frames before the i-th frame image, S2 is the adjacent frame change image of t frames after the i-th frame image, 1≤it≤n, according to formula (6), the cumulative change of image pixels in S1 and S2 is calculated to obtain images R1 and R2; Among them, | represents the bitwise OR operation; Step 4: Select candidate smoke regions from R1 and R2 Overlapping images, we get R 1ik and R 2ik , Statistics R 1ik and R 2ik The number of pixels in the three channels of the image is 255, denoted as U 1ik and U 2ik , if U 1ik / U 2ik <0.5, then it is considered There is smoke.

Images