Fire source detection and positioning system, method and device and storage medium

Abstract

The invention discloses a fire source detection and positioning method. The method comprises the following steps: carrying out smoke temperature and terahertz monitoring on a current environment; judging whether fire source information is met or not, and if not, returning to the previous step to continue to monitor the current environment; if yes, triggering a camera to capture the fire source information; performing motion detection on the snapshot image; carrying out region segmentation on the snapshot image; performing feature detection on the image; further judging whether fire source characteristics are met or not, and if not, returning to the initial step to continuously monitor the current environment; if the fire source characteristics are met, the fire hazard grade is further judged, and corresponding processing is executed. The terahertz imaging detection technology is applied to the low-visibility fire environment, compared with a traditional fire detector, the probability of false detection and missing detection is reduced, and the defect that traditional imaging detection equipment is greatly influenced by smoke particle morphology and particle size distribution changes is overcome.

Description

Technical Field

[0001] The present invention relates to the field of fire rescue, and in particular to a fire source detection and positioning system, method, device and storage medium. Background Art

[0002] In real-world scenarios, the generation and development of various types of fires are usually accompanied by open flames and smoke. Single types of detectors and detection methods have low sensitivity, low reliability, and one-sided data information, which easily cause problems such as false alarms and missed alarms. Moreover, fires usually occur with a large amount of smoke, and during the process of smoke diffusion, the environmental visibility continuously decreases, further increasing the difficulty of confirming the location of the fire source, thus increasing the difficulty of fire fighting and rescue.

[0003] Utilizing the high penetrability of terahertz waves, the environment can be monitored. The wavelength of terahertz waves is longer than that of infrared or visible light radiation. Therefore, terahertz waves will not be attenuated due to scattering by passing through dust, smoke, or similar media. Since the wavelength of terahertz waves is much larger than the scale of dust or soot particles suspended in the air, terahertz waves can be transmitted with low loss in a thick smoke or sandy environment. When a fire occurs, even if there are suspended soot particles in the air, terahertz waves can still penetrate. By the change of terahertz radiation received by a terahertz detector, the corresponding environmental information can be reflected, thereby realizing the monitoring of fires. Utilizing the fast propagation and positioning of terahertz waves helps to reduce property losses and personal injuries and minimize the damage by reducing secondary disasters.

[0004] Chinese Patent Application No. 2021103500697 discloses a fire smoke detector combined with terahertz wave detection. Although the technical solution of this patent also partially uses terahertz waves for smoke detection, its sensitivity and accuracy still need to be improved, and it cannot perform deep learning in combination with ever-changing fire phenomena. Therefore, there are still cases of missed alarms and false alarms in the alarm and processing methods.

[0005] Based on the deficiencies of the existing technology, it is necessary to develop a terahertz wave-based fire source detection and positioning device and method that can perform deep learning, effectively improve the accuracy of fire alarms and processing, avoid situations such as occlusion, false detection, and missed detection caused by traditional cameras, image sensors, temperature sensors, infrared sensors, smoke sensors, etc. in large spaces and complex environments. And for the situation where current traditional fire detectors cannot detect the size of a fire, the present invention adds a function of measuring information such as the size of a fire, distance, and fire source location on the basis of fire detection, enabling firefighters to master more detailed fire information. The present invention uses a method of combining a terahertz wave fire locator with computer vision technology to detect fires. Compared with traditional fire detectors, it has strong robustness in large spaces and complex environments and reduces the probability of false detection and missed detection. Summary of the Invention

[0006] To solve the deficiencies of the prior art, the present invention proposes a fire source detection and positioning system, method, device, and storage medium: In a first aspect, the present invention provides a fire source detection and positioning method, including the following steps: Monitor the smoke temperature and terahertz of the current environment; Determine whether it conforms to the fire source information. If not, return to the previous step to continue monitoring the current environment; If it is determined that the fire source information is met, trigger the camera to capture the fire source information; Perform motion detection on the captured image; Perform region segmentation on the captured image; Perform feature detection on the image; Further determine whether it conforms to the fire source characteristics. If not, return to the initial step to continue monitoring the current environment; If it conforms to the fire source characteristics, further determine the fire level and perform corresponding processing.

[0007] Furthermore, the motion detection uses the MOG2 algorithm to establish a background model, performs a difference operation between the current frame and the background image, segments the moving area of the image, and obtains a segmented image. The formula is as follows: I k (i, j) = B k (i, j) + M k (i, j) + N k (i, j); D k (i, j) = I k (i, j) - B K (i, j); Where I k (i, j) is the pixel information of the current image at the position (i, j) in the two-dimensional coordinate plane, N k (i, j) is the pixel information of the current background image at the position (i, j), M k (i, j) represents the pixel information of the moving object at the position (i, j), N k (i, j) represents the noise information in the image, D k (i, j) is the pixel information of the foreground image.

[0008] Furthermore, the region segmentation specifically includes the following steps: Judge the current image according to the HSV color model, screen the pixel points that meet the flame color, use the screened pixel points as the seed points for region growing, use M(x, y) to represent the fire seeds, and set V i(x, y) is a judgment flag for whether it has been traversed. If the pixel point P i (x, y) has been traversed, then V i (x, y) has a value of 1; otherwise, it is marked as 0. Mark the fire seed M i The point P corresponding to (x, y) in the original image F(x, y) i Take (x, y) as the starting point of region growing, and at the same time mark V i (x, y) as 1, and traverse the point P i The eight neighboring points around (x, y). Use the HSV model to judge whether it is a flame pixel point. If the traversed pixel point is a flame pixel point, then mark its M(x, y) value as 1, and at the same time mark the V(x, y) value as 1; otherwise, the M(x, y) value remains unchanged and the V(x, y) value is 0. If the traversed pixel point P i+1 (x, y) is determined to be a flame pixel point after judgment, take it as a new fire seed, and repeat the traversal until the V values of the eight surrounding pixel points of the pixel point are all 1, then return to the previous starting point to continue the eight-neighborhood traversal until returning to the initial point P(x, y).

[0009] Furthermore, the feature detection further includes removing interference sources, and the following formula is used to remove the interference sources: C = P 2 / A t ; Where P represents the perimeter of the connected domain boundary, and A t represents the area of the connected domain; By taking the reciprocal of C = P 2 / A t and multiplying it by 4π to normalize it to a value between 0 and 1, the formula is as follows: C = 4πA t / P 2 ; Where 4π is the minimum value of circularity, and the shape complexity is proportional to the C value.

[0010] In a second aspect, the present invention provides a fire source detection and positioning system, including: A toxic and harmful gas and smoke temperature sensing module, an infrared sensing module, a terahertz sensing module, a visible light sensing module, a deep learning module, and a control module; The toxic and harmful gas and smoke temperature sensing module detects and processes smoke signals and toxic and harmful gas signals and transmits them to the deep learning module; The infrared sensing module collects and generates a thermal infrared map and transmits it to the deep learning module; The terahertz sensing module detects the change of terahertz radiation in space and generates a second visual depth map, which is further transmitted to the deep learning module; The visible light sensing module detects and acquires visible light images, and at the same time outputs a first visual depth map and transmits it to the deep learning module. The first visual depth map and the second visual depth map are further subjected to depth difference to output a distance difference map to the deep learning module; The deep learning module makes a comprehensive judgment through a collaborative multi-signal system, and outputs the judgment result to the control module for fire extinguishing or alarm processing.

[0011] Further, the terahertz sensing module detects the spectrum in the 30μm - 3000μm band.

[0012] In a third aspect, the present invention provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, it implements the steps of the fire source detection and positioning method as described in any one of the first aspects.

[0013] In a fourth aspect, the present invention provides a fire source detection and positioning device, including: One or more processors; A memory; and One or more computer programs, wherein the one or more computer programs are stored in the memory and are configured to be executed by the one or more processors, and when the processor executes the computer program, it implements the steps of the fire source detection and positioning method as described in any one of the first aspects.

[0014] The present invention uses a method combining a terahertz wave fire source locator and computer vision technology to detect fires. Compared with traditional fire detectors, it has stronger robustness in large spaces and complex environments, reduces the probability of false detection and missed detection, and applies terahertz imaging detection technology to low-visibility fire environments, improving the deficiency that traditional imaging detection equipment is greatly affected by the change of smoke particle morphology and particle size distribution. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 : Schematic diagram of the composition of the fire source detection and positioning device of the present invention.

[0016] Figure 2 : Schematic diagram of the fire source detection and positioning method of the present invention.

[0017] Figure 3 : Schematic diagram of the motion detection and judgment process of the present invention.

[0018] Figure 4 : Schematic diagram of the original flame frame of the first embodiment of the present invention.

[0019] Figure 5 : Schematic diagram of the original flame frame of the second embodiment of the present invention.

[0020] Figure 6 : Schematic diagram of the foreground image of the background difference method of the embodiment of the present invention.

[0021] Figure 7 : The first effect diagram of the image segmentation based on region growing combined with HSV color model of the present invention.

[0022] Figure 8 : The second effect diagram of the image segmentation based on region growing combined with HSV color model of the present invention.

[0023] Figure 9 : Schematic diagram of the composition of the fire source detection and positioning device of the present invention. Detailed implementation manners

[0024] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0025] Refer to Figure 1 Schematic diagram of the composition of the fire source detection and positioning device of the present invention. The fire source detection and positioning system of the present invention includes a toxic and harmful gas and smoke temperature sensing module 101, an infrared sensing module 102, a terahertz sensing module 103, a visible light sensing module 104, a deep learning module 105 and a control module 106.

[0026] The toxic and harmful gas and smoke temperature sensing module 101 detects and processes smoke signals and toxic and harmful gas signals and transmits them to the deep learning module 105; the infrared sensing module 102 collects and generates a thermal infrared map and transmits it to the deep learning module 105; the terahertz sensing module 103 detects changes in terahertz radiation in space and generates a second visual depth map, which is further transmitted to the deep learning module 105; the visible light sensing module 104 detects and collects visible light images, and at the same time outputs a first visual depth map and transmits it to the deep learning module 105; the first visual depth map and the second visual depth map are further subjected to depth difference to output a distance difference map to the deep learning module 105; the deep learning module 105 makes a comprehensive determination through a collaborative multi-signal system and outputs the determination result to the control module 106 for fire extinguishing or alarm processing.

[0027] Furthermore, the terahertz sensing module of the present invention detects the spectrum in the 30μm - 3000μm band.

[0028] Refer to Figure 2Schematic diagram of the method for detecting and locating the fire source of the present invention, including the following steps: Step 201: Monitor the smoke temperature and terahertz of the current environment; Step 202: Determine whether it conforms to the fire source information. If not, return to Step 201 to continue monitoring; Step 203: If it is determined to belong to the fire source information, trigger the camera to capture pictures until the flame ends, and completely record the video images of the whole process from the occurrence to the end of the flame; specifically: set the detection data transmitted back by the terahertz sensing module to 00 for normal and 01 for alarm data. When the terahertz wave detector transmits back data of 01, start the camera to capture pictures; Step 204: If it conforms to the fire source information, further perform motion detection, and further refer to Figure 4 Schematic diagram of the original flame frame of the first embodiment of the present invention and Figure 5 Schematic diagram of the original flame frame of the second embodiment of the present invention. Specifically, perform image analysis on the obtained environmental image data, and use background difference operation for motion detection to remove static interference sources in the image; Step 205: Perform region segmentation on the image. Specifically, based on region growing combined with the HSV color model, use the background difference method to separate the foreground image, refer to Figure 6 Schematic diagram of the foreground image of the background difference method in the embodiment of the present invention. For the static and dynamic characteristics of the flame, segment the flame region in the foreground image, segment the image of the suspected flame region, and segment the image by combining region growing with the HSV color space for the suspected flame region. Further refer to Figure 7 The first effect diagram of image segmentation based on region growing combined with the HSV color model of the present invention and Figure 8 The second effect diagram of image segmentation based on region growing combined with the HSV color model of the present invention. The specific segmentation algorithm includes: Step 2051: Judge the current image according to the HSV color model, screen the pixel points that meet the flame color, and use the screened pixel points as the seed points for region growing. Use M(x, y) to represent the fire seeds. Let V i (x, y) be the judgment flag for whether it has been traversed. If the pixel point P i (x, y) has been traversed, then the value of V i (x, y) is 1, otherwise it is recorded as 0; Step 2052: Map the point P i (x, y) where the fire seed label M i (x, y) is mapped in the original image F(x, y) as the starting point of region growing, and at the same time mark V i (x, y) as 1, and traverse the point P iThe eight neighboring points around (x, y) are used to determine whether they are flame pixel points using the HSV model. If a traversed pixel point is a flame pixel point, then mark its M(x, y) value as 1 and at the same time mark its V(x, y) value as 1; otherwise, the M(x, y) value remains unchanged and the V(x, y) value is 0; Step 2053: If a certain point P traversed i+1 (x, y) is determined to be a flame pixel point through judgment, it can be used as a new flame seed, and step 2 is repeated until the V values of the eight surrounding pixel points of a certain pixel point are all 1, then return to the previous starting point to continue the eight-neighborhood traversal until returning to the initial point P(x, y).

[0029] Region growing method for image segmentation clusters images based on pixel similarity and connectivity. By setting several seed points, it is successively determined whether the pixel points in the four-neighborhood or eight-neighborhood of the seed points are similar to the seed points. If they are similar, they are added to the seed point set until the seed point set is empty. Region growing combines with the HSV color space, judges the suspected flame region through the HSV color model, uses the pixel points in this region as seed points, and then completes the segmentation of the suspected flame region through the region growing algorithm; At the beginning of the design of the present invention, the color characteristics of dozens of flame pictures at different times and in different environments were collected, as well as the color characteristics of pictures similar to the flame color, such as the color characteristics of electric lights and sunlight. The HSV values of dozens of flame pictures and non-flame pictures were saved in the database, and the distribution of each component was analyzed to obtain the threshold values of the flame pictures under each component, as shown in Table 1 below: .

[0030] Furthermore, using the flickering characteristics of the fire source and the area transformation of the approximately flame color region, the suspected flame is identified and determined to obtain the suspected flame region in the image. Specifically, the change in the flame area at adjacent moments is calculated to determine the flickering frequency of the flame; Furthermore, using the non-circular characteristics of the fire source and calculating the roundness of the region, circular dynamic interference sources similar to car lights in the image are removed to obtain the final fire source region; Step 206: Perform feature detection on the image. The flickering frequency of the flame is distributed between 3 and 25 Hz, and the main frequency is between 7 and 12 Hz. Due to the continuous change of the air flow, the area of the flame keeps changing, thus showing a flickering state. The present invention calculates the change in the flame area at adjacent moments to determine the flickering frequency of the flame. The algorithm formula is as follows: DP t =(A t+1 -A t )×(A t -A t-1)(t ∈ [1, N - 2]), Formula 1; where DP t is the product of the area differences between adjacent frames, and A t-1 , A t , A t+1 are the connected component areas of three adjacent frames. N is the total number of video frames in 1 second. By the value of DP t , it can be judged whether the area of the detection region changes during this time period. When the value of DP t meets the set threshold, that is, the area changes, the flame flicker frequency is incremented by 1. The statistical results of the circularity features of the fire and interference objects after normalization are shown in Table 2: .

[0031] Step 207: Further determine whether it meets the fire source characteristics. Specifically, binocular vision reconstruction is performed according to the camera parameters to obtain the actual area and distance of the flame, which specifically includes: Let the circularity C be used to describe the complexity of the object shape. The interference sources of the fire include objects with relatively smooth and regular shapes such as street lights and car lights. The image shape of the flame is relatively complex, and this feature can be well distinguished from the interference sources and can be used as one of the criteria for detecting the flame; its formula is: C = P 2 / A t , Formula 2; where P represents the perimeter of the connected component boundary, and A t represents the connected component area; By taking the reciprocal of C = P 2 / A t and multiplying it by 4π, it is normalized to a value between 0 and 1. The formula is as follows: C = 4πA t / P 2 , Formula 3; where 4π is the minimum value of the circularity. The shape complexity is proportional to the value of C. On the basis of segmenting and extracting the flame region, binocular vision is used to measure the area and distance of the flame in the actual environment. By binocular vision reconstruction of the flame, the true area of the flame is obtained and compared with the current environmental space plane. The same scene is photographed by two cameras at different positions, and the three-dimensional coordinates of the point are obtained by calculating the parallax of the spatial point in the two images. The specific principle is as follows: Let a certain feature point in space be P(x, y, z). In the images formed by the binocular cameras, the imaging point of the left camera is P left (x left , y left ), and the imaging point of the right camera is P right (x right , y right) The left and right cameras are installed at the same horizontal position. Therefore, the ordinate in the imaging positions of the feature points in the two cameras is the same, that is, y left = y right , and the geometric coordinates of the feature point in the actual space can be obtained according to the triangular relationship. The algorithm formula is as follows: d = x left - x right , Formula Four.

[0032] Step 208: Determine the fire level, specifically, judge the fire level at the current moment and during the entire combustion process of the flame based on the actual area and distance of the flame.

[0033] Figure 3 is a schematic diagram of the motion detection and judgment process of the present invention, including the following steps: Step 301: Obtain the current image; Step 302: Obtain the background image; Step 303: Difference image. Specifically: Establish a background model using the MOG2 algorithm, perform a difference operation on the current frame and the background image, and segment the moving area of the image to obtain a segmented image. The algorithm is as follows: I k (i, j) = B k (i, j) + M k (i, j) + N k (i, j), Formula Five; D k (i, j) = I k (i, j) - B K (i, j), Formula Six; where I k (i, j) is the pixel information of the current image at the position (i, j) in the two-dimensional coordinate plane, N k (i, j) is the pixel information of the current background image at the position (i, j), M k (i, j) represents the pixel information of the moving object at the position (i, j), N k (i, j) represents the noise information in the image, D k (i, j) is the pixel information of the foreground image.

[0034] Step 304: Threshold processing. Specifically: Let the current frame be k, where k is a positive integer, the area of the flame region is S f , and the cross-sectional area of the current environmental space region is S a , calculate its percentage as T k = S f / S a, the level determination of the entire flame combustion process is based on the level calculation of the entire flame combustion process by the terahertz wave detector and the video, and the percentage T of the pixels in the flame area of each frame in the previous combustion process is obtained. k , and its mean value E and maximum value T are calculated. max , E and T max are used to calculate the weighted average value, and the specific algorithm is as follows: E = ∑1 K T K / K; Formula Seven; Formula Eight; a + b = 1; Formula Nine; where a and b are the proportionality coefficients of the mean value E and the maximum value T respectively. Preferably, a takes the value of 0.8 and b takes the value of 0.2. max Step 305: Perform fire level discrimination, specifically including the flame level determination of the current frame and the level determination of the entire flame combustion process. In different environments, the corresponding values of a and b are set, and the weighted average is calculated. Based on the weighted average value and combined with the current environment, the flame level of the entire combustion process is judged.

[0035] Further referring to Figure 9 the schematic diagram of the composition of the fire source detection and positioning device of the present invention, the fire source detection and positioning device 10 of the present invention further includes one or more memories 20 and one or more processors 30, wherein the one or more computer programs are stored in the memory 20 and are configured to be executed by the one or more processors 30. When the processor 30 executes the computer program, the steps of the fire source detection and positioning method are realized.

[0036] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program. This program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.

[0037] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for fire source detection and location, comprising the following steps: Monitor the smoke temperature and terahertz of the current environment; Judge whether it conforms to the fire source information. If not, return to the previous step to continue monitoring the current environment; If it is judged that the fire source information is met, trigger the camera to capture the fire source information; Perform motion detection on the captured image; Perform region segmentation on the captured image; Perform feature detection on the image; Further judge whether it conforms to the fire source characteristics. If not, return to the initial step to continue monitoring the current environment; If the fire source characteristics are met, further determine the fire level and perform corresponding processing.

2. The method for fire source detection and positioning according to claim 1, characterized in that, The motion detection uses the MOG2 algorithm to establish a background model, performs a difference operation between the current frame and the background image, segments the moving area of the image, and obtains a segmented image. The formula is as follows: I k (i, j) = B k (i, j) + M k (i, j) + N k (i, j); D k (i, j) = I k (i, j) - B K (i, j); where I k (i, j) is the pixel information at the (i, j) position in the two-dimensional coordinate plane of the current image, N k (i, j) is the pixel information at the (i, j) position of the current background image, M k (i, j) represents the pixel information of the moving object at the (i, j) position, N k (i, j) represents the noise information in the image, D k (i, j) is the pixel information of the foreground image.

3. The fire source detection and positioning method according to claim 1, characterized in that, The region segmentation specifically includes the following steps: Judge the current image according to the HSV color model, screen the pixel points that meet the flame color, use the screened pixel points as the seed points for region growing, use M(x, y) to represent the fire seeds, and set V i (x, y) as the judgment flag for whether it has been traversed. If the pixel point P i (x, y) has been traversed, then the value of V i (x, y) is 1, otherwise it is recorded as 0; Mark the fire seed as M i (x, y) is the point P mapped in the original image F(x, y) i (x, y) is the starting point of region growing and mark V i (x, y) is 1, traverse point P i Traverse the eight neighboring points around (x, y), and use the HSV model to determine whether it is a flame pixel point. If the traversed pixel point is a flame pixel point, mark its M(x, y) value as 1 and mark its V(x, y) value as 1 at the same time; otherwise, the M(x, y) value remains unchanged and the V(x, y) value is 0; If the traversed pixel point P i+1 (x, y) is determined to be a flame pixel point through judgment, take it as a new flame seed, and repeat the traversal until the V values of the eight surrounding pixel points of the pixel point are all 1, then return to the previous starting point to continue the eight-neighborhood traversal until returning to the initial point P(x, y).

4. The fire source detection and positioning method according to claim 1, characterized in that The feature detection also includes removing interference sources, and the following formula is used for removing interference sources: C=P 2 / A t ; Among them, P represents the perimeter of the connected domain boundary, and A t represents the area of the connected domain; By taking C = P 2 / A t Taking the reciprocal and multiplying by 4π to normalize it to a value between 0 and 1, the formula is as follows: C = 4πA t / P 2 ; Where 4π is the minimum value of circularity, and the shape complexity is proportional to the C value.

5. A fire source detection and positioning system, characterized in that, Including: A toxic and harmful gas and smoke temperature sensing module, an infrared sensing module, a terahertz sensing module, a visible light sensing module, a deep learning module and a control module; The toxic and harmful gas and smoke temperature sensing module detects and processes the smoke signal and the toxic and harmful gas signal and transmits them to the deep learning module; The infrared sensing module collects and generates a thermal infrared map and transmits it to the deep learning module; The terahertz sensing module detects the change of terahertz radiation in space and generates a second visual depth map, and further transmits it to the deep learning module; The visible light sensing module detects and collects visible light images, and at the same time outputs a first visual depth map and transmits it to the deep learning module. The first visual depth map and the second visual depth map are further subjected to depth difference to output a distance difference map to the deep learning module; The deep learning module makes a comprehensive judgment through a collaborative multi-signal system, and outputs the judgment result to the control module for fire extinguishing or alarm processing.

6. The fire source detection and positioning device according to claim 5, wherein The terahertz sensing module detects the spectrum in the 30μm - 3000μm band.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the fire source detection and location method according to any one of claims 1 to 4.

8. A fire source detection and location device, comprising: One or more processors; A memory; And One or more computer programs, wherein the one or more computer programs are stored in the memory and are configured to be executed by the one or more processors. The feature is that when the processor executes the computer program, it implements the steps of the fire source detection and location method according to any one of claims 1 to 4.

Images