Smoke and fire identification method and system based on edge depth feature segmentation

Abstract

The invention discloses a smoke and fire identification method and system based on edge depth feature segmentation, and the method comprises the steps: collecting a historical infrared image of a to-be-detected region within a period of time, and constructing an image set; calculating an average image of the image set; calculating a variance map of the image set; and determining a detection threshold value based on the gray values of the average image and the variance image so as to judge a smoke and fire abnormal area in the infrared image of the to-be-detected area acquired in real time, and giving an alarm for the smoke and fire abnormal area. The method has high detection sensitivity and adaptability to the gray scale of the infrared image, meanwhile, gray scale fluctuation within a certain range is allowed, and the method is suitable for temperature abnormal area identification and alarm on an infrared imaging system.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and image processing, and in particular to a method and system for fireworks recognition based on edge depth feature segmentation. Background Art

[0002] Fire is one of the major safety hazards facing human society, and the loss of life and property it causes is difficult to estimate. Traditional fire alarm systems, such as smoke detectors, temperature sensors, etc., are widely used in buildings, but they usually rely on physical environmental changes, such as rising smoke concentrations or rising temperatures. This method works well in closed spaces, but its sensitivity and reliability may be unsatisfactory in open areas, well-ventilated environments, or in the early stages of a fire. In addition, sensor detection also faces the problems of false alarms and missed alarms. For example, smoke detectors may be mistakenly triggered by cigarette smoke, cooking smoke, etc., and miss real fire events. Therefore, smoke and fire recognition technology based on visual analysis has gradually become a research hotspot because it can provide earlier warning signals when smoke and flames first appear. It mainly includes the following methods:

[0003] 1. Recognition method based on color features

[0004] Color features are one of the earliest and most widely used methods for firework recognition. Flames usually appear bright yellow, orange, and red, while smoke appears light gray or white. Therefore, many research works detect flames and smoke based on color models [GaurA, SinghA, KumarA, et al. Video flame and smoke based fire detection algorithms: A literature review [J]. Fire technology, 2020, 56: 1943-1980]. Commonly used color spaces include RGB, YUV, HSV, etc. Typical methods convert the input image into these color spaces and use preset color thresholds to segment the flame or smoke area. For example, in the HSV space, flames usually have higher brightness values ​​and higher saturation, while smoke appears low saturation and high brightness.

[0005] 2. Recognition method based on motion features

[0006] Smoke and flames usually have unique motion features, such as floating, diffusion, and jumping. Therefore, recognition methods based on motion features are widely used. Optical flow is one of the commonly used motion analysis methods. It detects the dynamic changes of smoke and flames by tracking the motion trajectory of pixels in the image. The optical flow method can capture the slow drift of smoke and the jumping changes of flames, thereby achieving detection.

[0007] In addition, there are some methods that use background modeling technology to identify fireworks. Background modeling is usually based on image sequences, and detects anomalies by building a background model and analyzing changes in the foreground area. Typical methods include spatio-temporal image difference method (Lee CY, Lin CT, Hong CT. Spatio-temporal analysis in smoke detection [C] / / 2009 IEEE International conference on signal and Image processing applications. IEEE, 2009: 80-83) and Kalman filtering (Ma L, Wu K, Zhu L. Fire smoke detection in video images using Kalman filter and Gaussian mixture color model [C] / / 2010 International Conference on Artificial Intelligence and Computational Intelligence. IEEE, 2010, 1: 484-487). These methods can effectively detect abnormal motion changes in images, especially when the background changes are not obvious.

[0008] 3. Recognition method based on edge and texture features

[0009] Edge and texture features are important tools in image analysis, especially for identifying smoke and flames with complex structures. Flames usually have irregular shapes and strong edge features, while smoke has soft, diffuse texture structures. The edge detection-based firework recognition method uses classic algorithms such as the Canny edge detector and the Sobel operator to identify flames by extracting edge features from the image. In texture analysis, commonly used methods include Local Binary Pattern (LBP), Gray Level Co-occurrence Matrix (GLCM), etc., which identify the characteristics of smoke by analyzing the texture distribution of the image.

[0010] 4. Application of fusion detection technology

[0011] In order to improve the accuracy and robustness of smoke and fire recognition, a variety of fusion detection techniques have been proposed in recent years. These methods combine the advantages of color features, motion features, edge features and deep learning models to form a multi-level, multi-feature comprehensive detection method. For example, by combining color and edge features (Xiao S, Wang S, Ge L, et al. Hybrid Feature Fusion-Based High-Sensitivity Fire Detection and Early Warning for Intelligent Building Systems [J]. Sensors, 2023, 23 (2): 859), the brightness of the flame can be detected by color and the shape of the flame can be detected by edge; by combining deep learning and traditional feature extraction (Guo SJ, Li BH, Zhang JJ, et al. Flame and smoke detection based on channel shuffling and adaptive spatial feature fusion [J]. Journal of Electronic Imaging, 2024, 33 (5): 053036-053036), CNN can be used to extract deep features, and the optical flow method can be used to detect the movement of smoke.

[0012] In summary, current fireworks recognition technology covers methods based on color, motion, edge texture analysis and deep learning, and has made significant progress. However, various methods still have shortcomings: color detection is easily affected by lighting changes and interference, motion feature analysis performs poorly in dynamic backgrounds and static environments, edge and texture detection is not effective in low contrast or complex backgrounds, and deep learning methods, although with high accuracy, rely on a large amount of labeled data and have high computational costs. In addition, although the multi-feature fusion method improves detection accuracy, the model complexity increases and it is difficult to adapt to environmental changes. Future research should focus on lightweight models, multimodal fusion, adaptive algorithms and enhanced interpretability to improve robustness and adaptability in practical applications. Summary of the invention

[0013] The purpose of the present invention is to provide a method and system for fireworks recognition based on edge depth feature segmentation in order to solve the problems existing in the above-mentioned prior art.

[0014] The technical solution to achieve the purpose of the present invention is: on the one hand, a method for fireworks recognition based on edge depth feature segmentation is provided, the method comprising:

[0015] Step 1: Collect historical infrared images of the area to be detected within a period of time and construct an image set;

[0016] Step 2, calculating the average image of the image set;

[0017] Step 3, calculating the variance map of the image set;

[0018] Step 4: Determine the detection threshold based on the grayscale values ​​of the average map and the variance map, so as to judge the abnormal fireworks area in the infrared image of the area to be detected collected in real time, and alarm the abnormal fireworks area.

[0019] Furthermore, step 2 specifically includes:

[0020] Suppose a set of historical images I1,I2,…,I n , each image size is H×W, I n represents the nth historical image, n is the total number of historical images, H and W are the width and height of the image respectively;

[0021] For each pixel position (x, y), the average grayscale value of all historical images at that pixel position is calculated using the following formula:

[0022]

[0023] Where M(x,y) is the grayscale value of the average image at position (x,y), I i (x,y) is the grayscale value of the i-th historical image at position (x,y).

[0024] Furthermore, in step 3, the edge region of the original image is deeply segmented by edge search and mean fuzzy algorithm to obtain a variance map.

[0025] Furthermore, step 3 specifically includes:

[0026] Step 3-1, construct two convolution units:

[0027]

[0028] Step 3-2, convolve the two convolution units A and B with the pixels of a unit in the image respectively, and obtain:

[0029]

[0030] In the formula, I(x+i,y+j) represents the gray value of the original image at the position (x+i,y+j) in the image set;

[0031] Step 3-3, calculate the edge image based on the result of step 3-2:

[0032]

[0033] Where p(x,y) represents the edge image;

[0034] Step 3-4, perform image enhancement on the edge image to obtain an enhanced grayscale image

[0035] Step 3-5, for the grayscale image Perform threshold segmentation and generate a variance map;

[0036] Step 3-6, repeat steps 3-1 to 3-5, compare the grayscale values ​​of all variance maps at the position (x, y), take the maximum grayscale value as the grayscale value of the final variance map at the position (x, y), and thus obtain the final variance map.

[0037] Furthermore, the formula for image enhancement in step 3-4 is:

[0038]

[0039] Where round is the rounding function, f is the multiplication factor, and M(x,y) is the grayscale value of the average image at position (x,y).

[0040] Furthermore, the specific formula for threshold segmentation in steps 3-5 is:

[0041]

[0042] In the formula, m is calculated based on the grayscale histogram of the image or the mean value of the pixel grayscale. And the standard deviation σ is calculated, and the calculation formula is:

[0043]

[0044] Furthermore, step 4 specifically includes:

[0045] Let the gray value at a point (x, y) in the mean map be I(x, y), and the corresponding gray value at the point (x, y) in the variance map be p′(x, y). Then the lower and upper limits of the gray value allowed at the point (x, y) are:

[0046] t l (x,y)=I(x,y)+max{abs,var×p′(x,y)}

[0047] t u (x,y)=I(x,y)-max{abs,var×p′(x,y)}

[0048] Where, t l (x,y),t u(x, y) are the upper and lower limits of the grayscale, respectively. abs and var are two adjustment parameters, where abs is used to control the grayscale change amplitude based on the average image, and var is used to control the change ratio of the detection range based on the variance image.

[0049] For the infrared image of the area to be detected collected in real time, if [t u (x,y),t l The gray area within the range of [x, y)] is the abnormal fireworks area, and an alarm is issued for the abnormal fireworks area.

[0050] On the other hand, a fireworks recognition system based on edge depth feature segmentation is provided, the system comprising:

[0051] The first module is used to collect historical infrared images of the area to be detected within a period of time and construct an image set;

[0052] A second module is used to calculate an average map of the image set;

[0053] A third module is used to calculate the variance map of the image set;

[0054] The fourth module determines the detection threshold based on the grayscale values ​​of the average map and the variance map, so as to judge the abnormal fireworks area in the infrared image of the area to be detected collected in real time, and alarm the abnormal fireworks area.

[0055] On the other hand, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for identifying fireworks based on edge depth feature segmentation when executing the computer program.

[0056] On the other hand, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for identifying fireworks based on edge depth feature segmentation is implemented.

[0057] Compared with the prior art, the present invention has the following significant advantages: the method is based on the average map and variance map of the image, uses historical images to calculate the background model, and extracts the grayscale change area through edge detection and mean fuzzy processing. Two adjustable parameters are designed: abs controls the grayscale change amplitude based on the average map, and var controls the change detection range of the variance map. The abnormal area is judged according to the grayscale lower and upper limits, and pixels exceeding the upper limit are detected as suspected fireworks areas. The method has high detection sensitivity and adaptability to the grayscale of infrared images, and allows a certain range of grayscale fluctuations, which is suitable for the identification and alarm of abnormal temperature areas on infrared imaging systems.

[0058] The present invention is further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a flow chart of the fireworks recognition method based on edge depth feature segmentation of the present invention.

[0060] Figure 2 It is an average graph obtained according to the collected historical graphs in one embodiment.

[0061] Figure 3 In one embodiment, based on Figure 2 Obtained edge map.

[0062] Figure 4 In one embodiment, Figure 3 The result of enhancing the edge map.

[0063] Figure 5 In one embodiment, Figure 4 Variance map obtained by thresholding.

[0064] Figure 6 Schematic diagram of grayscale band detection algorithm when abs is 60 and var is 2.0 in one embodiment, Figure 6 (a) is the extremely dark grayscale distribution allowed. Figure 6 (b) in the figure is the extremely bright grayscale distribution allowed. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0066] It should be noted that if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0067] In one embodiment, a method for fireworks recognition based on edge depth feature segmentation is provided, the method comprising:

[0068] Step 1: Collect historical infrared images of the area to be detected within a period of time and construct an image set;

[0069] Step 2, calculating the average image of the image set;

[0070] Step 3, calculating the variance map of the image set;

[0071] Step 4: Determine the detection threshold based on the grayscale values ​​of the average map and the variance map, so as to judge the abnormal fireworks area in the infrared image of the area to be detected collected in real time, and alarm the abnormal fireworks area.

[0072] Further, in one embodiment, step 2 specifically includes:

[0073] Suppose a set of historical images I1,I2,...,I n , each image size is H×W, I n represents the nth historical image, n is the total number of historical images, H and W are the width and height of the image respectively;

[0074] For each pixel position (x, y), the average grayscale value of all historical images at that pixel position is calculated using the following formula:

[0075]

[0076] Where M(x,y) is the grayscale value of the average image at position (x,y), I i (x,y) is the grayscale value of the i-th historical image at position (x,y).

[0077] Furthermore, in one of the embodiments, in step 3, the edge region of the original image is deeply segmented by edge search and mean fuzzy algorithm to obtain a variance map.

[0078] Further, in one embodiment, step 3 specifically includes:

[0079] Step 3-1, construct two convolution units:

[0080]

[0081] Step 3-2, convolve the two convolution units A and B with the pixels of a unit in the image respectively, and obtain:

[0082]

[0083] In the formula, I(x+i,y+j) represents the gray value of the original image at the position (x+i,y+j) in the image set;

[0084] Step 3-3, calculate the edge image based on the result of step 3-2:

[0085]

[0086] Where p(x,y) represents the edge image;

[0087] Step 3-4, perform image enhancement on the edge image to obtain an enhanced grayscale image

[0088] Step 3-5, for the grayscale image Perform threshold segmentation and generate a variance map;

[0089] Step 3-6, repeat steps 3-1 to 3-5 several times to obtain several variance maps, compare the grayscale values ​​of all variance maps at the position (x, y), take the maximum grayscale value as the grayscale value of the final variance map at the position (x, y), and thus obtain the final variance map.

[0090] Furthermore, in one embodiment, the formula for image enhancement in step 3-4 is:

[0091]

[0092] Where round is the rounding function, f is the multiplication factor, and M(x,y) is the grayscale value of the average image at position (x,y).

[0093] Furthermore, in one embodiment, the specific formula for threshold segmentation in step 3-5 is:

[0094]

[0095] In the formula, m is calculated based on the grayscale histogram of the image or the mean value of the pixel grayscale. And the standard deviation σ is calculated, and the calculation formula is:

[0096]

[0097] Further, in one embodiment, step 4 specifically includes:

[0098] Let the gray value at a point (x, y) in the mean map be I(x, y), and the corresponding gray value at the point (x, y) in the variance map be p′(x, y). Then the lower and upper limits of the gray value allowed at the point (x, y) are:

[0099] t l (x,y)=I(x,y)+max{abs,var×p′(x,y)}

[0100] t u (x,y)=I(x,y)-max{abs,var×p′(x,y)}

[0101] Where, t l (x,y),t u (x, y) are the upper and lower limits of the grayscale, respectively. abs and var are two adjustment parameters, where abs is used to control the grayscale change amplitude based on the average image, and var is used to control the change ratio of the detection range based on the variance image.

[0102] For the infrared image of the area to be detected collected in real time, if [t u (x,y),t l The gray area within the range of [x, y)] is the abnormal fireworks area, and an alarm is issued for the abnormal fireworks area.

[0103] In one embodiment, a fireworks recognition system based on edge depth feature segmentation is provided, the system comprising:

[0104] The first module is used to collect historical infrared images of the area to be detected within a period of time and construct an image set;

[0105] A second module is used to calculate an average map of the image set;

[0106] A third module is used to calculate the variance map of the image set;

[0107] The fourth module is used to determine the detection threshold based on the grayscale values ​​of the average map and the variance map, so as to judge the abnormal fireworks area in the infrared image of the area to be detected collected in real time, and to alarm the abnormal fireworks area.

[0108] For the specific limitations of the fireworks recognition system based on edge depth feature segmentation, please refer to the limitations of the fireworks recognition method based on edge depth feature segmentation above, which will not be repeated here. Each module in the above-mentioned fireworks recognition system based on edge depth feature segmentation can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0109] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following is achieved:

[0110] Step 1: Collect historical infrared images of the area to be detected within a period of time and construct an image set;

[0111] Step 2, calculating the average image of the image set;

[0112] Step 3, calculating the variance map of the image set;

[0113] Step 4: Determine the detection range based on the grayscale values ​​of the average map and the variance map, so as to judge the abnormal fireworks area in the infrared image of the area to be detected collected in real time, and alarm the abnormal fireworks area.

[0114] For the specific limitations of each step, please refer to the limitations of the fireworks recognition method based on edge depth feature segmentation mentioned above, which will not be repeated here.

[0115] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the computer program implements:

[0116] Step 1: Collect historical infrared images of the area to be detected within a period of time and construct an image set;

[0117] Step 2, calculating the average image of the image set;

[0118] Step 3, calculating the variance map of the image set;

[0119] Step 4: Determine the detection range based on the grayscale values ​​of the average map and the variance map, so as to judge the abnormal fireworks area in the infrared image of the area to be detected collected in real time, and alarm the abnormal fireworks area.

[0120] For the specific limitations of each step, please refer to the limitations of the fireworks recognition method based on edge depth feature segmentation mentioned above, which will not be repeated here.

[0121] As a specific example, in one of the embodiments, the present invention is further verified and explained in detail.

[0122] 1. Collect historical images over a period of time through on-site equipment and build an image collection.

[0123] 2. Calculate the average image of the image set. The average image is the grayscale average value of the same points on the historical images to form a new image. Figure 2 It is an average graph obtained based on the collected historical graphs.

[0124] 3. Calculate the variance map of the image set. The variance map is obtained by deeply segmenting the edge area of ​​the original image through edge search, mean fuzzy and other algorithms to obtain a variance map that can reflect the grayscale changes in the image set. The first step is to obtain the edge image. The edge image is as follows Figure 3 As shown. Image enhancement is performed on the edge image, such as Figure 4 As shown. The enhanced image is thresholded, the pixels within [0, m] retain the original grayscale, and the rest are assigned 0, so a variance image is obtained, as shown Figure 5 shown.

[0125] 4. Repeat the above process several times to obtain several variance maps, compare the grayscale values ​​of each generated variance map at (x, y), take the largest grayscale value as the grayscale of the final variance image at (x, y), and obtain the final grayscale variance map.

[0126] 5. According to the core judgment relationship, not in [t u (x,y),t l The grayscale area within the range of [x, y] is the abnormal area. Correspondingly, for infrared images, the area exceeding the upper and lower limits of the grayscale is the suspected fireworks abnormal area.

[0127] When abs is 60 and var is 2.0, the algorithm detects grayscale bands as follows Figure 6 As shown, for smoke and fire detection, only the extremely bright range detection is needed, that is, if the grayscale distribution of a certain area is detected to be greater than the extremely bright threshold map, an alarm is issued in that area.

[0128] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for fireworks recognition based on edge depth feature segmentation, characterized in that: The method comprises: Step 1: Collect historical infrared images of the area to be detected within a period of time and construct an image set; Step 2, calculating the average image of the image set; Step 3, calculating the variance map of the image set; Step 4: Determine the detection threshold based on the grayscale values ​​of the average map and the variance map, so as to judge the abnormal fireworks area in the infrared image of the area to be detected collected in real time, and alarm the abnormal fireworks area.

2. The method for fireworks recognition based on edge depth feature segmentation according to claim 1, characterized in that: Step 2 specifically includes: Suppose a set of historical images I1,I2,...,I n , each image size is H×W, I n represents the nth historical image, n is the total number of historical images, H and W are the width and height of the image respectively; For each pixel position (x, y), the average grayscale value of all historical images at that pixel position is calculated using the following formula: Where M(x,y) is the grayscale value of the average image at position (x,y), I i (x,y) is the grayscale value of the i-th historical image at position (x,y).

3. The method for fireworks recognition based on edge depth feature segmentation according to claim 1, characterized in that: In step 3, the edge area of ​​the original image is deeply segmented by edge search and mean fuzzy algorithm to obtain a variance map.

4. The method for fireworks recognition based on edge depth feature segmentation according to claim 3, characterized in that: Step 3 specifically includes: Step 3-1, construct two convolution units: Step 3-2, convolve the two convolution units A and B with the pixels of a unit in the image respectively, and obtain: In the formula, I(x+i,y+j) represents the gray value of the original image at the position (x+i,y+j) in the image set; Step 3-3, calculate the edge image based on the result of step 3-2: Where p(x,y) represents the edge image; Step 3-4, perform image enhancement on the edge image to obtain an enhanced grayscale image Step 3-5, for the grayscale image Perform threshold segmentation and generate a variance map; Step 3-6, repeat steps 3-1 to 3-5 several times to obtain several variance maps, compare the grayscale values ​​of all variance maps at the position (x, y), take the maximum grayscale value as the grayscale value of the final variance map at the position (x, y), and thus obtain the final variance map.

5. The method for fireworks recognition based on edge depth feature segmentation according to claim 4, characterized in that: The formula for image enhancement in steps 3-4 is: Where round is the rounding function, f is the multiplication factor, and M(x,y) is the grayscale value of the average image at position (x,y).

6. The method for fireworks recognition based on edge depth feature segmentation according to claim 4, characterized in that: The specific formula for threshold segmentation in steps 3-5 is: In the formula, m is calculated based on the grayscale histogram of the image or the mean value of the pixel grayscale. And the standard deviation σ is calculated, and the calculation formula is:

7. The method for fireworks recognition based on edge depth feature segmentation according to claim 1, characterized in that: Step 4 specifically includes: Let the gray value at a point (x, y) in the mean map be I(x, y), and the corresponding gray value at the point (x, y) in the variance map be p′(x, y). Then the lower and upper limits of the gray value allowed at the point (x, y) are: t l (x,y)=I(x,y)+max{abs,var×p′(x,y)} t u (x,y)=I(x,y)-max{abs,var×p′(x,y)} Where, t l (x,y),t u (x, y) are the upper and lower limits of the grayscale, respectively. abs and var are two adjustment parameters, where abs is used to control the grayscale change amplitude based on the average image, and var is used to control the change ratio of the detection range based on the variance image. For the infrared image of the area to be detected collected in real time, if [t u (x,y),t l The gray area within the range of [x, y)] is the abnormal fireworks area, and an alarm is issued for the abnormal fireworks area.

8. A fireworks recognition system based on edge depth feature segmentation based on the method according to any one of claims 1 to 7, characterized in that: The system comprises: The first module is used to collect historical infrared images of the area to be detected within a period of time and construct an image set; A second module is used to calculate an average map of the image set; A third module is used to calculate the variance map of the image set; The fourth module is used to determine the detection threshold based on the grayscale values ​​of the average map and the variance map, so as to judge the abnormal fireworks area in the infrared image of the area to be detected collected in real time, and to alarm the abnormal fireworks area.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images