A Fire Safety Monitoring Device and Monitoring Method for Cable Shafts in High-Rise Buildings Based on Multimodal Fusion AI Technology

Abstract

A fire safety monitoring device for high-rise building cable shafts based on multi-modal fusion AI technology, comprising a plurality of data acquisition units, a plurality of communication units, a plurality of routing communication calculation units, a safety alarm center platform and a plurality of fire extinguishing units. The data acquisition units are connected to the routing communication calculation units through the sensing node communication units. The plurality of routing communication calculation units are connected to the safety alarm center platform. The fire extinguishing units are connected to the routing communication calculation units. The data acquisition units and the sensing node communication units are both arranged inside the high-rise building cable shafts. By effectively fusing multi-sensor timing parameters and infrared image parameters, the perception fault tolerance can be effectively improved. When using the fusion of multi-sensor timing parameters and image parameters for perception, various types of data can complement each other, effectively improving the fault tolerance of perception for single-type data, thereby effectively enhancing the accuracy of fire safety monitoring of high-rise building cable shafts.

Description

Technical Field

[0001] The present invention relates to the field of fire safety monitoring devices, and in particular to a high-rise building cable shaft fire safety monitoring device and method based on multi-modal fusion AI technology. Background Art

[0002] In the fire safety monitoring of high-rise building cable shafts, the data obtained by single-type sensors may have large deviations due to reasons such as equipment failures and insufficient sensitivity. That is, when single-type parameters have data errors due to electromagnetic, communication, and measurement reasons, when only relying on single-type error data for perception at this time, the perception accuracy will decrease or even become unable to perceive. For example, since there is no fire separation layer set between the cable shaft floors, the smoke will rise along the cable shaft all the way to the top floor without reaching the concentration that triggers the alarm. At this time, relying solely on the smoke alarm for fire safety monitoring cannot detect fire hazards in time, which may cause serious losses of life and property.

[0003] Therefore, the research focuses on the cause monitoring, prediction, isolation, traceability, and responsibility identification of electrical fires in high-rise buildings, masters the key technologies of sensing, monitoring, and boundary traceability of typical electrical fire characteristic quantities, and is an urgent common need to achieve the goals of effective monitoring, timely warning, scientific isolation, and responsibility traceability, and enhance the ability to obtain, implement, feedback, and control fire hazards at any time, so as to effectively improve the fire safety index of the high-rise living environment and ensure the safe and reliable operation of the power system. Summary of the Invention

[0004] The purpose of the present invention is to solve the problem that when single-type parameters in the fire safety monitoring of high-rise building cable shafts in the prior art have data errors due to electromagnetic, communication, and measurement reasons, when only relying on single-type error data for perception at this time, the perception accuracy will decrease or even become unable to perceive.

[0005] The technical solution adopted by the present invention is: a high-rise building cable shaft fire safety monitoring device based on multi-modal fusion AI technology, which is characterized in that: it includes a plurality of data acquisition units, a plurality of communication units, a plurality of routing communication calculation units, a safety alarm center platform, and a plurality of fire extinguishing units. The data acquisition units are connected to the routing communication calculation units through the sensing node communication units. A plurality of routing communication calculation units are connected to the safety alarm center platform, and the fire extinguishing units are connected to the routing communication calculation units. The data acquisition units and the sensing node communication units are both arranged inside the high-rise building cable shaft;

[0006] The data acquisition unit includes an ambient temperature collector, a cable temperature collector, a smoke concentration collector, and an infrared image data collector;

[0007] The communication unit includes a broadband power line carrier communication module, a micro-power wireless communication module, and a WiFi communication module;

[0008] The routing communication computing unit includes an AI chip module and a Beidou positioning / time service module;

[0009] The AI chip module is responsible for aggregating the data of the communication units of the sensing nodes and performing real-time calculations; it is equipped with a USB interface and a wifi processing module, receives the data transmitted by the data acquisition unit, and its inputs are structured multi-sensor time series parameters and unstructured image parameters; the outputs are the categories of fire safety hazards and the positions of the safety hazards in the infrared images; the multi-sensor time series parameters include voltage, current, ambient temperature, cable temperature, and smoke concentration, and the image parameter is infrared image data; first, feature extraction is performed on various parameters according to the data form and characteristics, then the features are effectively fused, and finally, fire monitoring and positioning of the cable shaft are carried out based on the fused features;

[0010] For the sensor time series parameters, arrange them into a matrix of size k X n, where k and n are the number of categories and the length of the time series respectively; then convert it into a multi-parameter recurrence plot applicable to a non-linear chaotic system to make it have the same representation form as the image data, that is, perform feature assimilation, and finally use a shallow convolutional neural network for feature extraction;

[0011] For the image parameters, since the number of pixels is generally large, directly use a mature convolutional neural network for feature extraction (such as using Faster R-CNN), and use the output of the first fully connected layer after ROI Pooling as the image extraction feature; for the fusion of the two types of features, adopt a splicing fusion method based on a weight factor. To avoid interference from human factors, the weight factor is used as a network parameter and obtained by training;

[0012] The AI chip module can communicate with the Beidou positioning / time service module, notify the Beidou positioning / time service module to send down data with spatio-temporal attributes, provide accurate position guidance for fire rescue, and provide a basis for time series tags for electrical accident analysis and traceability;

[0013] The safety alarm center platform is a visual display screen for displaying the monitoring situation;

[0014] The fire extinguishing unit is a fire extinguishing execution device that can receive execution instructions from the routing communication computing unit and perform fire extinguishing tasks.

[0015] Further, the cable temperature collector uses an SPS076 sensor.

[0016] Further, the smoke concentration collector uses a BYC100-YW sensor.

[0017] Further, the Wi-Fi communication module selects a commercially mature Wi-Fi module for industrial wireless control on the market, with the model number ESP8266-12F, using SMD packaging, serial communication, and data can be transparently transmitted; a GPIO is allocated to control its reset.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: effectively fusing the multi-sensor timing parameters and infrared image parameters can effectively improve the perception fault tolerance. When using the fusion of multi-sensor timing parameters and image parameters for perception, various types of data can complement each other, effectively improving the fault tolerance of perception for single-type data, thereby effectively enhancing the accuracy of fire safety monitoring in high-rise building cable shafts. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a schematic diagram of the overall structure of a preferred embodiment of the present invention;

[0020] Figure 2 is a schematic diagram of the overall architecture of a preferred embodiment of the present invention;

[0021] Figure 3 is an arrangement diagram of a preferred embodiment of the present invention on a high-rise building cable shaft;

[0022] Figure 4 is a circuit diagram of the WIFI module of a preferred embodiment of the present invention;

[0023] Figure 5 is a circuit diagram of the debugging serial port of a preferred embodiment of the present invention;

[0024] Figure 6 is a circuit diagram of the current acquisition of a preferred embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0025] The present invention will be further described below with reference to the accompanying drawings:

[0026] Please refer to Figure 1 、 Figure 2 and Figure 3 , a fire safety monitoring device for high-rise building cable shafts based on multi-modal fusion AI technology, characterized in that: it includes a plurality of data acquisition units 1, a plurality of communication units 2, a plurality of routing communication calculation units 3, a safety alarm center platform 4 and a plurality of fire extinguishing units 5. The data acquisition unit 1 is connected to the routing communication calculation unit 3 through the sensor node communication unit 2. A plurality of routing communication calculation units 3 are connected to the safety alarm center platform 4, and the fire extinguishing unit 5 is connected to the routing communication calculation unit 3. The data acquisition unit 1 and the sensor node communication unit 2 are both arranged inside the high-rise building cable shaft;

[0027] The data acquisition unit 1 includes an environmental temperature collector, a cable temperature collector, a smoke concentration collector, and an infrared image data collector;

[0028] The communication unit 2 includes a broadband power line carrier communication module, a micro-power wireless communication module, and a Wi-Fi communication module;

[0029] The routing communication calculation unit 3 includes an AI chip module and a Beidou positioning / time service module;

[0030] The AI chip module is responsible for aggregating the data of the communication units of the sensing nodes and performing real-time calculations; it is equipped with a USB interface and a wifi processing module, receives the data transmitted by the data acquisition unit 1, and its inputs are structured multi-sensor time series parameters and unstructured image parameters; the outputs are the categories of fire safety hazards and the positions of the safety hazards in the infrared images; the multi-sensor time series parameters include voltage, current, environmental temperature, cable temperature, and smoke concentration, and the image parameter is infrared image data; first, feature extraction is performed on various parameters according to the data form and characteristics, then the features are effectively fused, and finally, fire monitoring and positioning of the cable shaft are performed based on the fused features;

[0031] For the sensor time series parameters, arrange them into a matrix of size k X n, where k and n are the number of categories and the length of the time series respectively; then convert it into a multi-parameter recurrence plot applicable to a non-linear chaotic system to make it have the same representation form as the image data, that is, perform feature assimilation, and finally use a shallow convolutional neural network for feature extraction;

[0032] For the image parameters, since the number of pixels is generally large, directly use a mature convolutional neural network for feature extraction (such as using Faster R-CNN), and use the output of the first fully connected layer after ROI Pooling as the image extraction feature; for the fusion of the two types of features, adopt a splicing fusion method based on weight factors. To avoid interference from human factors, the weight factors are used as network parameters and obtained through training;

[0033] When an electrical accident occurs, the routing communication calculation unit 3 is triggered by processing multi-source data, and can issue action instructions to the hierarchical power disconnectors in the cable shaft to achieve rapid isolation of electrical fault cable equipment.

[0034] When an electrical fire occurs, the routing communication calculation unit 3 is triggered by processing multi-source data, and can issue action instructions to the intelligent sensing fire extinguishing device 5 in the cable shaft to achieve rapid extinguishing of the initial open fire ignition point.

[0035] The AI chip module can communicate with the Beidou positioning / time service module, notify the Beidou positioning / time service module to send data with spatio-temporal attributes, provide accurate location guidance for fire rescue, and provide a basis for time series tags for the analysis and traceability of electrical accident sources;

[0036] The safety alarm center platform 4 is a visual display screen for displaying the monitoring situation;

[0037] The fire extinguishing unit 5 is a fire extinguishing execution device that can receive execution instructions from the routing communication calculation unit 3 and execute fire extinguishing tasks.

[0038] The ambient temperature collector uses a MAX31820MCR+ sensor; the cable temperature collector uses an SPS076 sensor; the smoke concentration collector uses a BYC100-YW sensor.

[0039] Please refer to Figure 4 , the Wi-Fi communication module selects a commercially available Wi-Fi module for industrial wireless control, model ESP8266-12F, uses SMD packaging, uses serial communication, and data can be transparently transmitted; a GPIO is allocated to control its reset.

[0040] The monitoring method of the high-rise building cable shaft fire safety monitoring device based on multi-modal fusion AI technology includes the following steps:

[0041] Step 1: Data collection. Sensors are arranged in the cable shaft to collect ambient temperature, cable temperature, and smoke concentration. An infrared camera is arranged to collect infrared image data of the cable, and an intelligent electricity meter is used to collect voltage and current data;

[0042] Step 2: Align the sensor data, intelligent electricity meter data, and infrared image data collected in Step 1 in time;

[0043] Specifically, it includes:

[0044] Step 2.1 Construct a time series matrix of size k×n for ambient temperature, cable temperature, smoke concentration, voltage, and current data, as shown in Equation (1). Where k is the number of categories of time series, here k = 5, and n is the length of the time series, which can be selected according to the sampling frequency of each time series.

[0045]

[0046] Step 2.2 Align the time series matrix and the infrared image data, that is, make the sampling time of the last element in the time series matrix close to or the same as the sampling time of the infrared image data.

[0047] Step 3: Extract features from the sensor data and smart meter data collected in Step 1 to obtain corresponding feature vectors;

[0048] Specifically, it includes:

[0049] Step 3.1 Generation of chaos graph R

[0050]

[0051]

[0052] Among them, R ij is the element in the i-th row and j-th column of the chaos graph, and t i is the time value,

[0053] x1(t i ) is the sampling value of the first type of sensor at time t i ,

[0054] x j (t i ) is the sampling value of the j-th type of sensor at time t i ;

[0055] Step 3.2 Feature extraction

[0056] Take the chaos graph R as the input, use a convolutional neural network for feature extraction, and use the output W1 after the fully connected layer as the feature vector.

[0057] Step 4: Send the infrared image data collected in Step 1 into the feature extraction network for feature extraction to obtain corresponding feature vectors; specifically, take the infrared image as the input, use a convolutional neural network for feature extraction, and use the output W2 after the fully connected layer as the feature vector.

[0058] Step 5: Fuse the features obtained in Step 3 and Step 4; specifically, take the feature vectors obtained in Step 3 and Step 4 as the input, and fuse them according to weights m1 and m2 respectively to obtain the fused feature vector W, as shown in Equation (4):

[0059] W = m1W1 + m2W2 (4)

[0060] Step 6: Based on the fused features obtained in Step 5, conduct fire monitoring and hazard location for the cable shaft, and display the type of fire hazard and the location of the hazard in the infrared image;

[0061] Specifically, it includes

[0062] The fully-connected layer is used to further extract features from the fused feature vector W obtained in step 5, and perform fire hazard classification and location regression of the fire hazard in the infrared image; among them, the fire hazards are classified into three categories according to the type: no hazard, fire hazard caused by line fault / overload, and fire hazard caused by the cable itself.

[0063] Please refer to Figure 5 , Figure 5 Figure [X] shows the debug serial port circuit diagram of this solution. UARTI is used as the maintenance serial port of the terminal device. The reason for choosing UARTI is that UART1 is one of the serial peripherals supported by the STM32F407 embedded bootstrap program, which can facilitate future program upgrades and other operations through the serial port. The RS232 interface chip is selected as MAX3232CSE, and the external interface is protected bidirectionally with a TVS tube. The TVS tube is selected as ESDA14V2L, its VRM is 12V, and the minimum value of VBR is 14.2V. In order to match the interface of the industrial model, the maintenance serial port uses a USBA interface.

[0064] This solution effectively fuses infrared images and various sensor data at the same time, supplements and enhances different types of data, thereby effectively enhancing the accuracy of monitoring.

[0065] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and the descriptions in the specification only illustrate the structural relationship and principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A fire safety monitoring device for cable shafts in high-rise buildings based on multi-modal fusion AI technology, characterized in that: It includes multiple data acquisition units (1), multiple communication units (2), multiple routing communication calculation units (3), a security alarm center platform (4), and multiple fire extinguishing units (5). The data acquisition units (1) are connected to the routing communication calculation units (3) through the sensor node communication units (2). Multiple routing communication calculation units (3) are connected to the security alarm center platform (4). The fire extinguishing units (5) are connected to the routing communication calculation units (3). The data acquisition units (1) and the sensor node communication units (2) are both arranged inside the cable shaft of high-rise buildings; The data acquisition unit (1) includes an environmental temperature collector, a cable temperature collector, a smoke concentration collector, and an infrared image data collector; The communication unit (2) includes a broadband power line carrier communication module, a micro-power wireless communication module, and a Wi-Fi communication module; The routing communication calculation unit (3) includes an AI chip module and a Beidou positioning / time service module; The AI chip module is responsible for aggregating the data of the sensor node communication unit and performing real-time calculations. The AI chip module is equipped with a USB interface and a wifi processing module to receive the data transmitted by the data acquisition unit (1). The input of the AI chip module is structured multi-sensor time series parameters and unstructured image parameters; the output is the category of fire safety hazards and the location of the safety hazards in the infrared image. The multi-sensor time series parameters include voltage, current, environmental temperature, cable temperature, and smoke concentration, and the image parameter is infrared image data. First, feature extraction is performed on various parameters according to the data form and characteristics, then the features are effectively fused, and finally, fire monitoring and positioning of the cable shaft are carried out based on the fused features; For the sensor time series parameters, arrange them into a matrix of size kxn, where k and n are the number of categories and the length of the time series respectively; then convert it into a multi-parameter recurrence plot applicable to the non-linear chaotic system to make it have the same representation form as the image data, that is, perform feature assimilation, and finally use a shallow convolutional neural network for feature extraction; For the image parameters, use a mature convolutional neural network for feature extraction, and use the output of the first fully connected layer after ROI Pooling as the image extraction feature; for the fusion of the two types of features, adopt a splicing fusion method based on the weight factor. To avoid the interference of human factors, the weight factor is used as a network parameter and obtained by training; When an electrical accident occurs, the routing communication calculation unit (3) is triggered by processing multi-source data and can issue an action instruction to the hierarchical power isolation switch in the cable shaft to achieve rapid isolation of the electrical fault cable equipment; The security alarm center platform (4) is a visual display screen for displaying the monitoring situation; The fire extinguishing unit (5) is a fire extinguishing execution device that can receive the execution instruction from the routing communication calculation unit (3) and execute the fire extinguishing task; The environmental temperature collector uses a MAX31820MCR+ sensor; the cable temperature collector uses an SPS076 sensor; The smoke concentration collector uses a BYC100-YW sensor.

2. The fire safety monitoring device for high-rise building cable shafts based on multi-modal fusion AI technology according to claim 1, wherein: The Wi-Fi communication module selected is a Wi-Fi module for industrial wireless control, with the model number ESP8266-12F. It uses SMD packaging, serial communication, and assigns a GPIO for its reset control.

3. The monitoring method of the fire safety monitoring device for the cable shaft of high-rise buildings based on multi-modal fusion AI technology according to claim 1, characterized in that, It includes the following steps: Step 1: Data acquisition. Sensors are arranged in the cable shaft to collect ambient temperature, cable temperature, and smoke concentration. An infrared camera is arranged to collect infrared image data of the cable, and an intelligent electricity meter is used to collect voltage and current data. Step 2: Align the sensor data, intelligent electricity meter data, and infrared image data collected in Step 1 in terms of time. Step 3: Extract features from the sensor data and intelligent electricity meter data collected in Step 1 to obtain corresponding feature vectors. Step 4: Send the infrared image data collected in Step 1 into a feature extraction network for feature extraction to obtain corresponding feature vectors. Step 5: Fuse the features obtained in Step 3 and Step 4. Step 6: Based on the fused features obtained in Step 5, conduct fire monitoring and hazard location for the cable shaft, and display the type of fire hazard and the location of the hazard in the infrared image.

4. The monitoring method of the high-rise building cable shaft fire safety monitoring device based on multi-modal fusion AI technology according to claim 3, characterized in that, The specific content of Step 2 includes: Step 2.1 Construct a time series matrix of size k×n for the environmental temperature, cable temperature, smoke concentration, voltage, and current data, as shown in Equation (1), where k is the number of categories of time series, and n is the length of the time series, which is selected according to the sampling frequency of each time series; Step 2.2 Align the time series matrix and the infrared image data, that is, make the sampling moment of the last element in the time series matrix close to or the same as the sampling moment of the infrared image data.

5. The monitoring method of the fire safety monitoring device for high-rise building cable shafts based on multi-modal fusion AI technology according to claim 3, characterized in that, The specific content of Step 3 includes: Step 3.1 Generation of the chaos graph R Among them, R ij is the element in the i-th row and j-th column of the chaos diagram, and t i is the time value. x1(t i ) is the sampled value of the first type of sensor at t i moment, x j (t i ) is the sampled value of the j-th type of sensor at time t i . Step 3.2 Feature extraction Use the chaos graph R as the input, perform feature extraction with a convolutional neural network, and use the output W1 after the fully connected layer as the feature vector.

6. The monitoring method of the fire safety monitoring device for high-rise building cable shafts based on multi-modal fusion AI technology according to claim 3, characterized in that, The specific content of Step 4 includes: Use the infrared image as the input, perform feature extraction with a convolutional neural network, and use the output W2 after full connection as the feature vector.

7. The monitoring method of the fire safety monitoring device for high-rise building cable shafts based on multi-modal fusion AI technology according to claim 3, characterized in that, The specific content of Step 5 includes: Use the feature vectors obtained in Step 3 and Step 4 as the input, and fuse them according to the weights m1 and m2 respectively to obtain the fused feature vector W, as shown in Equation (4): W = m1W1 + m2W2 (4).

8. The monitoring method of the fire safety monitoring device for high-rise building cable shafts based on multi-modal fusion AI technology according to claim 3, characterized in that, The specific content of Step 6 includes: Use a fully connected layer to further extract features from the fused feature vector W obtained in Step 5, and conduct fire hazard classification and location regression of the hazard in the infrared image; among them, the fire hazards are classified into three categories: no hazard, fire hazard caused by line fault / overload, and fire hazard caused by the cable itself.

Images