Fire-fighting information management system and method based on Internet of Things

Abstract

The invention is suitable for the field of fire-fighting information management, and provides a fire-fighting information management system and method based on the Internet of Things, and the system comprises a thermal imaging and visible light fusion module, a high-temperature region recognition and analysis module, a fire source information screening and confirmation module, and an evacuation analysis module. The progress of the scheme in the aspect of fire-fighting information management is embodied in multiple levels. The early warning capability is improved, so that the system can detect abnormal conditions in time at the early stage of the fire, the early warning capability of the fire is enhanced, and the loss caused by the fire is effectively reduced. Meanwhile, the system provides a scientific basis for command decision making through visual data and comprehensive analysis, so that a decision maker can make more accurate emergency response according to real-time data, and resources and deployment are effectively coordinated. In addition, risk assessment and dynamic behavior analysis enable an emergency management team to configure resources in a more targeted manner and implement fire extinguishing and evacuation measures, so that the efficiency and effect of emergency response are improved.

Description

Technical Field

[0001] The present invention belongs to the field of fire information management, and in particular relates to a fire information management system and method based on the Internet of Things. Background Art

[0002] The field of fire information management is a professional field that uses a variety of technical means and management methods to improve fire prevention, response and rescue capabilities. Its core goal is to ensure the timely collection, analysis, transmission and application of relevant information before, during and after a fire, thereby ensuring public safety and property protection.

[0003] The field of fire information management faces unique challenges in large venues and events (such as concerts, music festivals, sports competitions, etc.). These venues usually have dense crowds, high-heat equipment and spotlights, which will interfere with the accurate identification of fire sources. In large-scale events, there are a large number of spectators and frequent personnel movement, which makes it difficult for traditional monitoring methods to accurately analyze crowd behavior and promptly determine potential fire risks. In addition, the audio equipment, spotlights, stage equipment, etc. used in the event generate a lot of heat, which may be confused with the real fire source, affecting the thermal imaging equipment's identification of the fire source.

[0004] In the case of a large crowd, the fire source is often located below the crowd or blocked by many people, resulting in the inability of traditional video surveillance to effectively identify the existence of the fire source, especially in the case of smoke or unclear vision. In addition, existing thermal imaging technology often has difficulty effectively distinguishing between human heat sources and fire sources when analyzing heat sources. In a high-density crowd, a large number of confusing signals may appear in the thermal image, making it difficult to identify the real fire source. Summary of the invention

[0005] The purpose of the present invention is to provide a fire information management system based on the Internet of Things, aiming to solve the technical problems existing in the prior art identified in the background technology.

[0006] The present invention is implemented in this way: a fire information management system based on the Internet of Things, the system comprising:

[0007] Thermal imaging and visible light fusion module, which is used to capture thermal imaging data and visible light video streams in the venue in real time through cameras, reflect the temperature distribution of various objects and areas in the form of images, and align the thermal imaging data with the visible light video stream in space and superimpose them to form a composite image that presents both thermal and visual features;

[0008] High temperature area recognition and analysis module, which is used to mark the high temperature area detected by thermal imaging in the composite image, distinguish the temperature levels through different color codes, identify the objects overlapping with the high temperature area through the visible light image, use the clear boundary and shape information of the visible light image, combined with the temperature distribution of the thermal imaging, analyze the shape and edge characteristics of the heat source, and confirm whether it has the typical characteristics of the fire source;

[0009] The fire source information screening and confirmation module is used to compare the dynamic changes of the area with fire source features in the composite image through the dynamic information of people and objects in the visible light video stream, screen out false features affected by other factors, and obtain the area where the fire source is confirmed;

[0010] The evacuation analysis module is used to evaluate the potential fire source risk based on the relative position of the area where the fire source is confirmed and the site environment, analyze crowd behavior from the visible light video stream, identify flow and gathering patterns, and analyze the evacuation area and evacuation path based on the potential fire source risk.

[0011] Another object of the present invention is to provide a fire information management method based on the Internet of Things, the method comprising:

[0012] The camera captures the thermal imaging data and visible light video stream in the venue in real time, reflects the temperature distribution of each object and area in the form of an image, and aligns the thermal imaging data with the visible light video stream in space and superimposes them to form a composite image that presents both thermal and visual features;

[0013] High-temperature areas detected by thermal imaging are marked in the composite image, and different temperature levels are distinguished by different color codes. Objects overlapping with high-temperature areas are identified through visible light images. The clear boundary and shape information of visible light images is combined with the temperature distribution of thermal imaging to analyze the shape and edge features of the heat source to confirm whether it has typical characteristics of a fire source.

[0014] By comparing the dynamic information of people and objects in the visible light video stream with the dynamic changes of the area with fire source features in the composite image, false features affected by other factors are screened out to obtain the area where the fire source is confirmed;

[0015] Based on the relative position of the area where the fire source is confirmed and the site environment, the potential fire source risk is assessed, crowd behavior is analyzed from the visible light video stream, flow and gathering patterns are identified, and the evacuation area and evacuation route are analyzed in combination with the potential fire source risk.

[0016] As a further solution of the present invention, the method of capturing the thermal imaging data and visible light video stream in the venue in real time by a camera, reflecting the temperature distribution of each object and area in the form of an image, and aligning the thermal imaging data and the visible light video stream in space and superimposing them specifically includes:

[0017] Acquire data from thermal imaging cameras in real time and record the temperature distribution of various objects and areas in the venue;

[0018] Synchronously acquire the video stream of the visible light camera to capture the visual features of objects and environment in the venue;

[0019] spatially aligning the thermal imaging data with the visible light video image using an image registration algorithm, and combining the aligned thermal imaging data and visible light data to form a composite image;

[0020] As a further solution of the present invention, in the composite image, the thermal imaging information is superimposed on the visible light image in the form of a semi-transparent layer.

[0021] As a further solution of the present invention, the high temperature area detected by thermal imaging is marked in the composite image, and the temperature levels are distinguished by different color codes, and the objects overlapping with the high temperature area are identified by visible light images, and the clear boundary and shape information of the visible light image is used, combined with the temperature distribution of the thermal imaging, to analyze the shape and edge features of the heat source, and confirm whether it has typical characteristics of the fire source, specifically including:

[0022] Set temperature thresholds, use thermal imaging data to identify areas that exceed the predetermined temperature thresholds, and mark these areas as high temperature areas, using different color codes to represent different temperature levels when marking;

[0023] Convert the graded image into a temperature matrix T(x,y), where each element in the matrix represents the temperature value of a pixel in the image;

[0024] Calculate the temperature change rate of each pixel in the matrix, set the change rate threshold, and identify the areas where the temperature change exceeds the change rate threshold:

[0025]

[0026] Where T is the temperature matrix, and are the temperature change rates along and directions respectively, is the temperature gradient vector, indicating the direction and speed of temperature change;

[0027] Define the temperature threshold T threshold , will be higher than the temperature threshold T threshold The area is marked as high temperature area;

[0028] Calculate the shape moment of the high temperature area, use the shape moment to extract the shape features of the area, and compare the shape features with the features of known fire sources to determine the similarity;

[0029] Calculate the area and perimeter of high temperature areas, monitor changes in area and perimeter over time, and identify rapid expansion features;

[0030] Calculate the temperature mean and variance in the high temperature area, evaluate the temperature concentration and distribution characteristics, and determine whether there are statistical characteristics of the fire source:

[0031] The extracted features, including temperature gradient, shape features and statistical features, are matched with predefined fire source features to verify whether the high-temperature area meets the known fire source features, and the areas with fire source features are marked in the composite image.

[0032] As a further solution of the present invention, the step of calculating the shape moment of the high temperature area, extracting the shape features of the area using the shape moment, and comparing the shape features with the features of the known fire source to determine the similarity specifically includes:

[0033] Compute shape moments of arbitrary order in an image:

[0034] M pq =∑ x ∑ y x p y q f(x,y);

[0035] Among them, M pq is the shape moment of any order in the image, f(x,y) is the temperature value in the temperature matrix, and x,y are the coordinates of the image;

[0036] Based on the shape moments of any order obtained, all Hu moments are calculated, and the shape features of the region are extracted using the shape moments:

[0037] φ1=M 20 +M 02 ;

[0038]

[0039] φ3=(M 30 -3M 12 ) 2 +(3M 21 -M 03 ) 2 ;

[0040] φ4=(M 30 +M 12 ) 2 +(M 21 +M 03 ) 2 ;

[0041] φ5=(M 30 -3M 12 )(M 30 +M 12 )[(M 30 +M12 ) 2 -3(M 21 -M 03 ) 2 ];

[0042] φ6=(M 20 -M 02 )[(M 30 +M 12 ) 2 -(M 21 +M 03 ) 2 ];

[0043] φ7=3(M 21 -M 03 )(M 30 +M 12 ) 2 [(M 30 +M 12 ) 2 -3(M 21 -M 03 ) 2 ];

[0044] Among them, φ1 to φ7 are Hu invariant moments of different orders, which are used for describing and matching shape features.

[0045] As a further solution of the present invention, the calculating the area and perimeter of the high temperature area specifically includes:

[0046] A=∑ x,y δ(T(x,y)>T threshold );

[0047] P=∑ boundary ;

[0048] Where A is the area of ​​the high temperature region, δ is the indicator function, if the temperature T(x,y) exceeds the threshold T threshold , then δ is 1, otherwise δ is 0, and P is the perimeter of the region boundary;

[0049] As a further solution of the present invention, the temperature mean and variance in the high temperature area are calculated, specifically:

[0050]

[0051] Among them, μ is the average temperature of the high temperature area, σ 2 is the variance of the temperature in the high temperature area, and N is the total number of pixels in the high temperature area.

[0052] As a further solution of the present invention, the dynamic information of people and objects in the visible light video stream is compared with the dynamic changes of the area with fire source features in the composite image, false features affected by other factors are screened out, and the area where the fire source is confirmed is obtained, which specifically includes:

[0053] In areas with fire source characteristics, use visible light video streams to analyze the surrounding environment, identify strong light sources, and analyze the background temperature of the surrounding environment to determine whether there are high-temperature objects;

[0054] Compare the area with fire source characteristics in the thermal imaging data with the light source position in the visible light image to determine whether the high temperature is associated with the light source, monitor the temperature change trend in the high temperature area, and determine whether the temperature fluctuation is related to the activity of the high temperature object;

[0055] Record the duration of high temperature in areas with fire source characteristics, combine crowd flow and environmental changes, compare data at different time points, judge the true performance of the area, and identify areas where fire sources are confirmed.

[0056] As a further solution of the present invention, the relative position of the area where the fire source is confirmed and the site environment is used to evaluate the potential fire source risk, analyze crowd behavior from the visible light video stream, identify flow and gathering patterns, and analyze the evacuation area and evacuation path in combination with the potential fire source risk.

[0057] Use composite images to generate a spatial layout of the site environment, mark and confirm the fire source area, escape routes and safety exits, and confirm the relative position of the fire source area with surrounding facilities and crowded areas;

[0058] Based on the distance between the fire source and the surrounding environment, the distribution of flammable items, and the dynamic information of the crowd in the visible light video stream, risk levels are divided to identify environmental factors that may affect the spread of the fire source;

[0059] Analyze the dynamic behavior of the crowd in the visible light video stream, including flow patterns, aggregation, and personnel flow. Based on the dynamic behavior of the crowd, the risk level of different areas, and site layout data, analyze the evacuation path from the fire source area to the evacuation area.

[0060] The beneficial effects of the present invention are:

[0061] This method can ensure accurate grasp of the temperature distribution in the venue. Thermal imaging data promptly reflects the temperature abnormality area, while visible light images provide clear visual information. The spatial alignment and superposition of this data allows decision makers to more intuitively identify potential fire sources, greatly improving the ability of early warning of fires. Marking high-temperature areas in composite images and using color coding to distinguish temperature levels not only improves recognition efficiency, but also analyzes the shape and edge features of heat sources through the boundary information of visible light images, thereby enhancing the accuracy of fire source identification. This precise analysis capability enables the system to quickly distinguish between real fire sources and other high-temperature objects, reducing the possibility of false alarms.

[0062] By monitoring the dynamic changes of people and objects through visible light video streams, the fire source features in the composite image are compared and analyzed in real time, effectively filtering out false features caused by external factors. This mechanism ensures that the confirmation of the existence of the fire source is based on a comprehensive judgment of dynamic data, providing a more reliable information basis for subsequent emergency response.

[0063] The program's progress in fire information management is reflected in multiple aspects. The improved early warning capability enables the system to detect abnormal conditions in a timely manner at the early stage of a fire, enhances the early warning capability of a fire, and effectively reduces the losses caused by the fire. At the same time, the system provides a scientific basis for command decisions through visual data and comprehensive analysis, allowing decision makers to make more accurate emergency responses based on real-time data and effectively coordinate resources and deployment. In addition, risk assessment and dynamic behavior analysis allow the emergency management team to allocate resources more specifically, implement fire extinguishing and evacuation measures, and thus improve the efficiency and effectiveness of emergency response. By monitoring the dynamic behavior of the crowd in real time, the system can also identify potential risks of crowd gathering and congestion, optimize evacuation routes, and ensure the safe and efficient evacuation of the crowd, which is especially important in emergency situations such as fires. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A structural block diagram of a fire information management system based on the Internet of Things provided by an embodiment of the present invention;

[0065] Figure 2 A flowchart of a fire information management method based on the Internet of Things provided by an embodiment of the present invention;

[0066] Figure 3 A flow chart for spatially aligning thermal imaging data with a visible light video stream and superimposing them provided in an embodiment of the present invention;

[0067] Figure 4 A flow chart for analyzing the shape and edge features of a heat source and determining whether it has typical features of a fire source provided by an embodiment of the present invention;

[0068] Figure 5 A flowchart for filtering out false features affected by other factors and obtaining an area where a fire source is confirmed to exist provided in an embodiment of the present invention;

[0069] Figure 6 A flowchart for analyzing crowd behavior, identifying flow and gathering patterns, and analyzing evacuation areas and evacuation routes in combination with potential fire source risks provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0071] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.

[0072] Figure 1 The structural block diagram of the fire information management system based on the Internet of Things provided by the embodiment of the present invention is as follows: Figure 1 As shown, the system comprises:

[0073] The thermal imaging and visible light fusion module 100 is used to capture the thermal imaging data and visible light video stream in the venue in real time through the camera, reflect the temperature distribution of each object and area in the form of an image, and align the thermal imaging data with the visible light video stream in space and superimpose them to form a composite image that simultaneously presents thermal and visual features;

[0074] The high temperature area recognition and analysis module 200 is used to mark the high temperature area detected by the thermal imaging in the composite image, distinguish the temperature levels by different color codes, recognize the objects overlapping with the high temperature area by the visible light image, use the clear boundary and shape information of the visible light image, combine with the temperature distribution of the thermal imaging, analyze the shape and edge features of the heat source, and confirm whether it has the typical characteristics of the fire source;

[0075] The fire source information screening and confirmation module 300 is used to compare the dynamic changes of the area with fire source features in the composite image through the dynamic information of people and objects in the visible light video stream, screen out false features affected by other factors, and obtain the area where the fire source is confirmed;

[0076] The evacuation analysis module 400 is used to evaluate the potential fire source risk based on the relative position of the area where the fire source is confirmed and the site environment, analyze crowd behavior from the visible light video stream, identify flow and gathering patterns, and analyze the evacuation area and evacuation path in combination with the potential fire source risk.

[0077] Figure 2 A flowchart of a fire information management method based on the Internet of Things provided by an embodiment of the present invention, such as Figure 2 As shown, the method includes:

[0078] S100, which uses a camera to capture thermal imaging data and visible light video streams in the venue in real time, reflects the temperature distribution of various objects and areas in the form of images, and spatially aligns the thermal imaging data with the visible light video stream and superimposes them to form a composite image that simultaneously presents thermal and visual features;

[0079] In this step, the temperature distribution data of each object and area in the venue is first captured in real time by a highly sensitive thermal imaging camera. These data are presented in the form of a heat map, with each pixel representing a temperature value. At the same time, a high-definition visible light camera is used to capture the real-time video stream of objects and environments in the venue to ensure that the video stream has high resolution and clarity. Next, advanced image registration algorithms (for example, feature point matching, optical flow method, etc.) are used to spatially align the thermal imaging data with the visible light video stream to ensure that the two data accurately correspond in the same coordinate system. Optimization is performed through a variety of algorithms (such as multi-scale pyramid algorithm) to improve the registration accuracy and reduce errors caused by motion blur or environmental changes. In the generated composite image, the thermal imaging information is superimposed on the visible light image in the form of a semi-transparent layer, allowing users to observe the temperature distribution and visual features at a glance. This processing method that combines visual features with thermal signals further enhances the display effect of the image and ensures that users can intuitively understand the potential fire risk. The generated composite image is stored in real time on a local server or cloud for subsequent analysis, backtracking and recording, and through an encrypted network channel to ensure data security and prevent information leakage.

[0080] This step ensures that the fire monitoring system can respond quickly to potential fire risks and reduce the response time of fires by acquiring thermal imaging and visible light data in real time. At the same time, the precise image registration algorithm improves the correspondence accuracy of thermal imaging and visible light images, ensuring a high degree of match between high-temperature areas and objects during the analysis process and reducing the false alarm rate. The composite image presents a combination of thermal imaging and visible light information, allowing users to obtain temperature distribution and visual features at the same time, thereby comprehensively assessing fire risks. Different color-coded temperature levels allow users to intuitively identify high-temperature areas, quickly determine the degree of risk, and improve decision-making efficiency.

[0081] like Figure 3As shown, the thermal imaging data and visible light video stream in the venue are captured in real time by the camera, the temperature distribution of each object and area is reflected in the form of an image, and the thermal imaging data and the visible light video stream are spatially aligned and superimposed, specifically including:

[0082] S110, real-time acquisition of data from thermal imaging cameras to record the temperature distribution of various objects and areas in the venue;

[0083] S120, synchronously acquiring a video stream of a visible light camera to capture visual features of objects and the environment in the venue;

[0084] S130, using an image registration algorithm to spatially align the thermal imaging data with the visible light video image, and combining the aligned thermal imaging data and visible light data to form a composite image;

[0085] In the embodiment of the present invention, in the composite image, the thermal imaging information is superimposed on the visible light image in the form of a semi-transparent layer.

[0086] S200, marks the high temperature area detected by thermal imaging in the composite image and distinguishes the temperature level through different color codes, identifies objects overlapping with the high temperature area through visible light images, uses the clear boundary and shape information of visible light images, combined with the temperature distribution of thermal imaging, analyzes the shape and edge features of the heat source, and confirms whether it has typical characteristics of a fire source;

[0087] This step sets a temperature threshold, uses thermal imaging data to identify areas that exceed the threshold, and marks these areas as high temperature areas. This process is not only the basis for identifying heat sources, but also avoids false alarms by setting critical values. Through different color coding methods, users can quickly determine which areas have potential fire risks. For example, red may represent extremely high temperature areas, while yellow represents medium temperature areas. This intuitive visual presentation helps users respond quickly.

[0088] In the process of converting the graded image into a temperature matrix, each matrix element corresponds to a pixel temperature value in the image. This structured data processing method makes subsequent analysis more systematic and helps to conduct a more in-depth analysis of temperature changes. By calculating the temperature change rate of each pixel in the matrix and setting the change rate threshold, the system can quickly identify areas with large temperature fluctuations, which is crucial for early detection of fire sources.

[0089] Furthermore, shape moments are calculated for high-temperature areas. This process is to extract the geometric features of the area to determine whether it meets the characteristics of a known fire source. Shape moments can not only describe the shape and size of the area, but also reveal its complexity and irregularity. For example, fire sources often have certain shape characteristics, such as round or irregular shapes, which can be effectively captured and compared through shape moments.

[0090] At the same time, monitoring the changes in the area and perimeter of the high-temperature area over time can help identify the characteristics of rapidly expanding fire sources. By tracking the changes in these indicators, the system can dynamically evaluate the spread trend of the fire source, which provides important information in emergency response. For example, if the area of ​​a high-temperature area increases sharply in a short period of time, it may indicate that the fire is spreading rapidly and priority measures need to be taken.

[0091] When calculating the mean and variance of the temperature within a high-temperature area, the system is able to evaluate the temperature concentration and distribution characteristics within the area. This statistical analysis helps determine whether a high-temperature area has the characteristics of a fire source. For example, if the mean temperature of an area is much higher than the surrounding environment, but the variance is small, this may indicate that the area is a stable fire source rather than a momentary temperature increase caused by external factors.

[0092] Finally, all extracted features, including temperature gradients, shape features, and statistical features, are matched with predefined fire source features. The accuracy of this matching process directly affects the reliability of fire source identification. The areas with fire source features in the composite image are marked, which not only provides a direct basis for subsequent firefighting decisions, but also provides clear visual guidance for on-site personnel.

[0093] By setting the temperature threshold and the change rate threshold in this step, the system can quickly locate the high-temperature area, improve the speed of fire source detection, and reduce the possibility of false alarms. At the same time, the color coding method allows users to intuitively identify the temperature level in complex scenarios, helping firefighters to respond in the first time. The introduction of the temperature matrix greatly enriches the depth of data analysis and can effectively capture the dynamic characteristics of temperature changes, which is very important for judging the potential expansion of the fire source. In addition, the extraction and comparison of shape features help to accurately identify the characteristics of the fire source, further improving the intelligence of the system. Through the monitoring of multi-dimensional data such as area and perimeter, the system can dynamically track the changes in high-temperature areas, providing strong data support for subsequent risk assessments, thereby providing a scientific basis for firefighting decisions.

[0094] like Figure 4As shown, the high temperature area detected by thermal imaging is marked in the composite image, and different color codes are used to distinguish the temperature levels. The objects overlapping with the high temperature area are identified through the visible light image. The clear boundary and shape information of the visible light image is used in combination with the temperature distribution of the thermal imaging to analyze the shape and edge features of the heat source to confirm whether it has the typical characteristics of the fire source, including:

[0095] S210, setting a temperature threshold, using thermal imaging data, identifying areas exceeding a predetermined temperature threshold, marking these areas as high temperature areas, and using different color codes to represent different temperature levels when marking;

[0096] S220, converting the graded image into a temperature matrix T(x,y), wherein each element in the matrix represents a temperature value of a pixel in the image;

[0097] S230, calculating the temperature change rate of each pixel in the matrix, setting a change rate threshold, and identifying areas where the temperature change exceeds the change rate threshold:

[0098]

[0099] Where T is the temperature matrix, and are the temperature change rates along and directions respectively, is the temperature gradient vector, indicating the direction and speed of temperature change;

[0100] S240, define temperature threshold T threshold , will be higher than the temperature threshold T threshold The area is marked as high temperature area;

[0101] S250, calculating the shape moment of the high temperature area, extracting the shape feature of the area using the shape moment, and comparing the shape feature with the feature of the known fire source to determine the similarity;

[0102] S260, calculating the area and perimeter of the high temperature region, monitoring the changes in the area and perimeter of the region over time, and identifying rapid expansion features;

[0103] S270, calculate the temperature mean and variance in the high temperature area, evaluate the temperature concentration and distribution characteristics, and determine whether there are statistical characteristics of the fire source:

[0104] S280, matches the extracted features, including temperature gradient, shape features and statistical features, with predefined fire source features, verifies whether the high temperature area meets the known fire source features, and marks the area with the fire source features in the composite image.

[0105] In the embodiment of the present invention, the shape moment is calculated for the high temperature area, the shape feature of the area is extracted using the shape moment, and the shape feature is compared with the feature of the known fire source to determine the similarity, specifically including:

[0106] Compute shape moments of arbitrary order in an image:

[0107] M pq =∑ x ∑ y x p y q f(x,y);

[0108] Among them, M pq is the shape moment of any order in the image, f(x,y) is the temperature value in the temperature matrix, and x,y are the coordinates of the image;

[0109] Based on the shape moments of any order obtained, all Hu moments are calculated, and the shape features of the region are extracted using the shape moments:

[0110] φ1=M 20 +M 02 ;

[0111]

[0112] φ3=(M 30 -3M 12 ) 2 +(3M 21 -M 03 ) 2 ;

[0113] φ4=(M 30 +M 12 ) 2 +(M 21 +M 03 ) 2 ;

[0114] φ5=(M 30 -3M 12 )(M 30 +M 12 )[(M 30 +M 12 ) 2 -3(M 21 -M 03 ) 2 ];

[0115] φ6=(M 20 -M 02 )[(M 30 +M 12 ) 2 -(M 21 +M03 ) 2 ];

[0116] φ7=3(M 21 -M 03 )(M 30 +M 12 ) 2 [(M 30 +M 12 ) 2 -3(M 21 -M 03 ) 2 ];

[0117] Among them, φ1 to φ7 are Hu invariant moments of different orders, which are used for describing and matching shape features.

[0118] In the embodiment of the present invention, the calculating the area and perimeter of the high temperature region specifically includes:

[0119] A=∑ x,y δ(T(x,y)>T threshold );

[0120] P=∑ boundary ;

[0121] Where A is the area of ​​the high temperature region, δ is the indicator function, if the temperature T(x,y) exceeds the threshold T threshold , then δ is 1, otherwise δ is 0, and P is the perimeter of the region boundary;

[0122] In the embodiment of the present invention, the temperature mean and variance in the high temperature region are calculated, specifically:

[0123]

[0124] Among them, μ is the average temperature of the high temperature area, σ 2 is the variance of the temperature in the high temperature area, and N is the total number of pixels in the high temperature area.

[0125] S300, comparing the dynamic changes of the area with the fire source feature in the composite image through the dynamic information of the crowd and objects in the visible light video stream, filtering out the false features affected by other factors, and obtaining the area where the fire source is confirmed;

[0126] This step first uses the visible light video stream to analyze the surrounding environment and identify possible strong light sources, such as flickering lights and direct sunlight. These strong light sources may interfere with the interpretation of thermal imaging data, so they must be screened and analyzed first. At the same time, the background temperature of the environment is recorded to set a baseline to help identify high-temperature objects. By comparing the background temperature and thermal imaging data, it is possible to more accurately determine whether a high-temperature object exists. Next, the area with fire source characteristics in the thermal imaging data is compared with the position of the light source in the visible light image to determine whether the temperature in the high-temperature area is associated with the light source, which helps to confirm whether the high temperature is caused by the light source rather than a potential fire source. In addition, the temperature change trend of the high-temperature area is monitored to analyze whether the temperature fluctuation is associated with the activity of the surrounding high-temperature objects. For example, if the temperature in the high-temperature area changes with the flickering of the light source, it may indicate that the area is not a real fire source. This step also records the duration of the fire source characteristics in the high-temperature area, which is crucial for judging the authenticity of the fire source. If the high temperature is short-lived and consistent with the change of the light source, it may only be a false feature. Therefore, it is particularly important to analyze the dynamic changes in the area by comparing data at different time points in combination with crowd flow and environmental changes. If the high temperature area persists at multiple time points and there are no other factors interfering, it can be confirmed that the area is the real fire source. Finally, compare the areas with fire source characteristics in thermal imaging and visible light data, and analyze their changes in different time periods to further confirm the existence of the fire source. In the area where the fire source is confirmed, thermal imaging and visible light images are further integrated to generate a composite image, which is provided to the decision support system to help assess risks.

[0127] This step can effectively eliminate false alarms and ensure the high accuracy of the system by comparing different data sources (thermal imaging and visible light) and combining environmental background analysis. This multi-dimensional data fusion makes the judgment of the fire source more reliable. At the same time, through real-time video stream analysis, it can quickly respond to the dynamic changes of the fire source and ensure a rapid response to sudden fires. This real-time nature is crucial in fire emergency management and helps reduce potential losses. By recording the duration and dynamic changes of high-temperature areas, this step can dynamically evaluate the authenticity of the fire source. This dynamic analysis enables the system to adapt to different environmental changes and improve monitoring capabilities. In addition, combined with crowd flow and environmental changes, the system can intelligently analyze potential fire source risks, and then provide data support for the formulation of evacuation routes and safety measures. This intelligent management improves the decision-making efficiency and accuracy of firefighters. Finally, step S300 not only considers thermal imaging data, but also analyzes the surrounding environment through visible light video streams, making the judgment more comprehensive and reducing the deviation that may be caused by a single data source.

[0128] like Figure 5As shown, the dynamic information of people and objects in the visible light video stream is compared with the dynamic changes of the area with fire source features in the composite image, false features affected by other factors are screened out, and the area where the fire source is confirmed is obtained, which specifically includes:

[0129] S310, in the area where there are fire source characteristics, using the visible light video stream to analyze the surrounding environment, identify the strong light source, and analyze the background temperature of the surrounding environment to determine whether there is a high-temperature object;

[0130] S320, comparing the area with fire source characteristics in the thermal imaging data with the light source position in the visible light image, determining whether the high temperature is associated with the light source, monitoring the temperature change trend of the high temperature area, and determining whether the temperature fluctuation is related to the activity of the high temperature object;

[0131] S330, record the high temperature duration of the area with fire source characteristics, combine crowd flow and environmental changes, compare data at different time points, judge the actual performance of the area, and identify the area where the fire source is confirmed.

[0132] S400, assesses the potential fire source risk based on the relative position of the area where the fire source is confirmed and the site environment, analyzes crowd behavior from the visible light video stream, identifies flow and gathering patterns, and analyzes evacuation areas and evacuation routes based on the potential fire source risk.

[0133] This step uses composite images to generate a spatial layout of the site environment, marking the confirmed fire source area, escape routes and safe exits. This layout will intuitively show the relative position of the fire source area with surrounding facilities and crowded areas, helping decision makers quickly identify potential risk areas. In this process, it is important to clearly understand the distance between the fire source and surrounding flammable items in order to assess the possibility of the fire source spreading.

[0134] Next, the risk level is divided according to the distance between the fire source and the surrounding environment, the distribution of flammable materials, and the dynamic information of the crowd in the visible light video stream. This process involves analyzing each area in the venue and identifying environmental factors that may affect the spread of the fire source, such as the type of flammable materials, their storage location and their relative position to the fire source. With this data, a risk level can be assigned to each area in the venue, and high-risk areas can be identified so that corresponding preventive measures can be taken.

[0135] In addition, analyzing the dynamic behavior of the crowd in the visible light video stream is also extremely critical. This step involves in-depth analysis of flow patterns, aggregation, and flow of people. By monitoring the behavior of the crowd in real time, it is possible to identify which areas have a large flow of people and which areas may be at risk of congestion. This information helps to determine the efficiency of the crowd's escape in the event of a fire, thereby optimizing the evacuation plan.

[0136] Based on the dynamic behavior of the crowd, the risk level of different areas and the site layout data, the evacuation path from the fire source area to the evacuation area will be comprehensively analyzed. This analysis will take into account the flow of people and the complexity of the environment to ensure that the evacuation path is safe and efficient. At the same time, the path analysis will also provide solutions for possible obstacles and bottlenecks in the evacuation process, ensuring that people can evacuate quickly and safely in the event of a fire.

[0137] By generating a spatial layout diagram and marking key areas, decision makers can quickly understand the overall situation of the site, providing an intuitive basis for fire emergency response. This visualization method helps to improve the efficiency and accuracy of on-site command. Secondly, through risk level classification, potential dangerous areas can be effectively identified, so that more targeted emergency measures can be formulated to reduce the risks caused by fire. In addition, real-time analysis of crowd dynamic behavior enables the system to flexibly respond to emergencies, adjust evacuation strategies in a timely manner, and ensure the safety of the crowd. Finally, the evacuation path analysis that comprehensively considers multiple factors can optimize the efficiency of crowd evacuation, reduce confusion and congestion, and improve the success rate of escape.

[0138] like Figure 6 As shown, the potential fire source risk is assessed based on the relative position of the area where the fire source is confirmed and the site environment, crowd behavior is analyzed from the visible light video stream, flow and gathering patterns are identified, and evacuation areas and evacuation paths are analyzed in combination with the potential fire source risk.

[0139] S410, using the composite image to generate a spatial layout diagram of the site environment, marking and confirming the fire source area, escape routes and safety exits, and confirming the relative position of the fire source area and surrounding facilities and crowded areas;

[0140] S420, based on the distance between the fire source and the surrounding environment, the distribution of flammable items, and the dynamic information of the crowd in the visible light video stream, risk level classification is performed to identify environmental factors that may affect the spread of the fire source;

[0141] S430, analyzing the dynamic behavior of the crowd in the visible light video stream, including flow patterns, aggregation conditions, and personnel flow directions, and analyzing the evacuation path from the fire source area to the evacuation area based on the dynamic behavior of the crowd, the risk levels of different areas, and the site layout data.

[0142] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0143] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0144] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0145] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

[0146] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A fire information management system based on the Internet of Things, characterized in that: The system comprises: Thermal imaging and visible light fusion module, which is used to capture thermal imaging data and visible light video streams in the venue in real time through cameras, reflect the temperature distribution of various objects and areas in the form of images, and align the thermal imaging data with the visible light video stream in space and superimpose them to form a composite image that presents both thermal and visual features; High temperature area recognition and analysis module, which is used to mark the high temperature area detected by thermal imaging in the composite image, distinguish the temperature levels through different color codes, identify the objects overlapping with the high temperature area through the visible light image, use the clear boundary and shape information of the visible light image, combined with the temperature distribution of the thermal imaging, analyze the shape and edge characteristics of the heat source, and confirm whether it has the typical characteristics of the fire source; The fire source information screening and confirmation module is used to compare the dynamic changes of the area with fire source features in the composite image through the dynamic information of people and objects in the visible light video stream, screen out false features affected by other factors, and obtain the area where the fire source is confirmed; The evacuation analysis module is used to evaluate the potential fire source risk based on the relative position of the area where the fire source is confirmed and the site environment, analyze crowd behavior from the visible light video stream, identify flow and gathering patterns, and analyze the evacuation area and evacuation path based on the potential fire source risk.

2. A fire information management method based on the Internet of Things, characterized in that: The method comprises: The camera captures the thermal imaging data and visible light video stream in the venue in real time, reflects the temperature distribution of each object and area in the form of an image, and aligns the thermal imaging data with the visible light video stream in space and superimposes them to form a composite image that presents both thermal and visual features; High-temperature areas detected by thermal imaging are marked in the composite image, and different temperature levels are distinguished by different color codes. Objects overlapping with high-temperature areas are identified through visible light images. The clear boundary and shape information of visible light images is combined with the temperature distribution of thermal imaging to analyze the shape and edge features of the heat source to confirm whether it has typical characteristics of a fire source. By comparing the dynamic information of people and objects in the visible light video stream with the dynamic changes of the area with fire source features in the composite image, false features affected by other factors are screened out to obtain the area where the fire source is confirmed; Based on the relative position of the area where the fire source is confirmed and the site environment, the potential fire source risk is assessed, crowd behavior is analyzed from the visible light video stream, flow and gathering patterns are identified, and the evacuation area and evacuation route are analyzed in combination with the potential fire source risk.

3. The method according to claim 2, characterized in that The method of capturing the thermal imaging data and visible light video stream in the venue in real time through the camera, reflecting the temperature distribution of each object and area in the form of an image, and spatially aligning the thermal imaging data and the visible light video stream and superimposing them specifically includes: Acquire data from thermal imaging cameras in real time and record the temperature distribution of various objects and areas in the venue; Synchronously acquire the video stream of the visible light camera to capture the visual features of objects and environment in the venue; The thermal imaging data is spatially aligned with the visible light video image using an image registration algorithm, and the aligned thermal imaging data and visible light data are combined to form a composite image.

4. The method according to claim 3, characterized in that In the composite image, the thermal imaging information is superimposed on the visible light image in the form of a semi-transparent layer.

5. The method according to claim 3, characterized in that: The high temperature area detected by thermal imaging is marked in the composite image, and the temperature levels are distinguished by different color codes. The objects overlapping with the high temperature area are identified by visible light images. The clear boundary and shape information of the visible light image is combined with the temperature distribution of the thermal imaging to analyze the shape and edge features of the heat source to confirm whether it has the typical characteristics of the fire source, including: Set temperature thresholds, use thermal imaging data to identify areas that exceed the predetermined temperature thresholds, and mark these areas as high temperature areas, using different color codes to represent different temperature levels when marking; Convert the graded image into a temperature matrix T(x,y), where each element in the matrix represents the temperature value of a pixel in the image; Calculate the temperature change rate of each pixel in the matrix, set the change rate threshold, and identify the areas where the temperature change exceeds the change rate threshold: Where T is the temperature matrix, and are the temperature change rates along and directions respectively, is the temperature gradient vector, indicating the direction and speed of temperature change; Define the temperature threshold T threshold , will be higher than the temperature threshold T threshold The area is marked as high temperature area; Calculate the shape moment of the high temperature area, use the shape moment to extract the shape features of the area, and compare the shape features with the features of known fire sources to determine the similarity; Calculate the area and perimeter of high temperature areas, monitor changes in area and perimeter over time, and identify rapid expansion features; Calculate the temperature mean and variance in the high temperature area, evaluate the temperature concentration and distribution characteristics, and determine whether there are statistical characteristics of the fire source; The extracted features, including temperature gradient, shape features and statistical features, are matched with predefined fire source features to verify whether the high-temperature area meets the known fire source features, and the areas with fire source features are marked in the composite image.

6. The method according to claim 5, characterized in that The step of calculating the shape moment of the high temperature area, extracting the shape features of the area using the shape moment, and comparing the shape features with the features of known fire sources to determine the similarity specifically includes: Compute shape moments of arbitrary order in an image: M pq =∑ x ∑ y x p y q f(x,y); Among them, M pq is the shape moment of any order in the image, f(x,y) is the temperature value in the temperature matrix, and x,y are the coordinates of the image; Based on the shape moments of any order obtained, all Hu moments are calculated, and the shape features of the region are extracted using the shape moments: φ1=M 20 +M 02 ; <h2 style=";text-align:left;direction:ltr">φ3=(M<h2 style=";text-align:left;direction:ltr"> 30 <h2 style=";text-align:left;direction:ltr"> -3M<h2 style=";text-align:left;direction:ltr"> 12 <h2 style=";text-align:left;direction:ltr"> )<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +(3M<h2 style=";text-align:left;direction:ltr"> 21 <h2 style=";text-align:left;direction:ltr"> -M<h2 style=";text-align:left;direction:ltr"> 03 <h2 style=";text-align:left;direction:ltr"> )<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> ; φ4=(M 30 +M 12 ) 2 +(M 21 +M 03 ) 2 ; φ5=(M 30 -3M 12 )(I 30 +M 12 )[(I 30 +M 12 ) 2 -3(M 21 -M 03 ) 2 ]; φ6=(M 20 -M 02 )[(M 30 +M 12 ) 2 -(M 21 +M 03 ) 2 ]; φ7=3(M 21 -M 03 )(M 30 +M 12 ) 2 [(M 30 +M 12 ) 2 -3(M 21 -M 03 ) 2 ]; Among them, φ1 to φ7 are Hu invariant moments of different orders, which are used for describing and matching shape features.

7. The method according to claim 5, characterized in that The calculating of the area and perimeter of the high temperature region specifically includes: A=∑ x,y δ(T(x,y)>T threshold ); P＝∑ boundary ； Where A is the area of ​​the high temperature region, δ is the indicator function, if the temperature T(x,y) exceeds the threshold T threshold , then φ is 1, otherwise δ is 0, and P is the perimeter of the region boundary.

8. The method according to claim 5, characterized in that The temperature mean and variance in the high temperature area are calculated as follows: Among them, μ is the average temperature of the high temperature area, σ 2 is the variance of the temperature in the high temperature area, and N is the total number of pixels in the high temperature area.

9. The method according to claim 5, characterized in that The method uses the dynamic information of people and objects in the visible light video stream to compare the dynamic changes of the area with the fire source feature in the composite image, screens out false features affected by other factors, and obtains the area where the fire source is confirmed, specifically including: In areas with fire source characteristics, use visible light video streams to analyze the surrounding environment, identify strong light sources, and analyze the background temperature of the surrounding environment to determine whether there are high-temperature objects; Compare the area with fire source characteristics in the thermal imaging data with the light source position in the visible light image to determine whether the high temperature is associated with the light source, monitor the temperature change trend in the high temperature area, and determine whether the temperature fluctuation is related to the activity of the high temperature object; Record the duration of high temperature in areas with fire source characteristics, combine crowd flow and environmental changes, compare data at different time points, judge the true performance of the area, and identify areas where fire sources are confirmed.

10. The method according to claim 9, characterized in that According to the relative position of the confirmed fire source area and the site environment, the potential fire source risk is assessed, crowd behavior is analyzed from the visible light video stream, flow and gathering patterns are identified, and the evacuation area and evacuation path are analyzed in combination with the potential fire source risk. Use composite images to generate a spatial layout of the site environment, mark and confirm the fire source area, escape routes and safety exits, and confirm the relative position of the fire source area with surrounding facilities and crowded areas; Based on the distance between the fire source and the surrounding environment, the distribution of flammable items, and the dynamic information of the crowd in the visible light video stream, risk levels are divided to identify environmental factors that may affect the spread of the fire source; Analyze the dynamic behavior of the crowd in the visible light video stream, including flow patterns, aggregation, and personnel flow. Based on the dynamic behavior of the crowd, the risk level of different areas, and site layout data, analyze the evacuation path from the fire source area to the evacuation area.

Images