Building talkback intelligent emergency safety center system based on multi-mode perception

Abstract

The invention discloses a building talkback intelligent emergency safety center system based on multi-mode perception. The building talkback intelligent emergency safety center system comprises a perception layer, a data processing layer, a decision response layer and a collaboration layer. According to the invention, an active security and protection mechanism integrating real-time monitoring, second-level data processing, dynamic decision making and cross-domain linkage is constructed, so that rapid emergency response within one minute is realized, access control locking can be triggered within 10 seconds in a high-risk scene, alarm pushing and linkage of 110 and medical resources are realized, and the emergency handling efficiency is effectively improved; through multi-mode perception fusion and AI cross validation, environment interference and real risks are effectively distinguished, the false alarm rate is effectively reduced, and the recognition accuracy of events such as falling down and invasion of the old is remarkably improved; and the intercom system is used as a core to seamlessly integrate the elevator, the gas valve and other terminals, data intercommunication and co-processing with community medical treatment, security and other resources are realized, information islands are broken, and the comprehensive security cost of the community is reduced.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of building intercoms, and in particular to a building intercom intelligent emergency safety hub system based on multi-modal sensing. BACKGROUND

[0002] With the development of economy and the gradual improvement of the national living standard, people have a high pursuit of the quality of life. Nowadays, building intercoms are developing in the direction of integration, intelligence and informatization. However, the main function of the traditional building intercom system is limited to audio and video communication and access control between visitors and residents, and in the aspect of safety protection, the system is in a passive response state, cannot actively identify potential risks, and has weak emergency handling capability. When an emergency occurs, such as an old person falling down, gas leakage, violent intrusion, etc., the system often cannot respond in time and effectively, and it is difficult to meet the demand for rapid response in an emergency. In addition, the existing system has obvious deficiencies in the accuracy of risk identification. A single sensor is easily disturbed by the environment and produces false alarms. For example, an infrared sensor may misjudge as personnel activity due to the passing of a pet. Moreover, the existing system has poor synergy with other community resources, cannot realize effective data intercommunication and linkage disposal with community medical stations, fire control rooms and the like, forms an information island, and is difficult to form a joint force to deal with various safety events, and is difficult to meet the high requirements of intelligent communities on safety resilience. In view of the above, the application provides a building intercom intelligent emergency safety hub system based on multi-modal sensing. SUMMARY

[0003] Based on the technical problems existing in the background art, the application provides a building intercom intelligent emergency safety hub system based on multi-modal sensing.

[0004] The building intercom intelligent emergency safety hub system based on multi-modal sensing provided by the application comprises a sensing layer, a data processing layer, a decision response layer and a synergy layer. The sensing layer comprises an indoor sensing unit, a doorway sensing unit and an associated device interface. The indoor sensing unit comprises a six-axis attitude sensor, a wideband voiceprint collector and an infrared thermal imaging module. The doorway sensing unit comprises a vibration fiber sensor and a 77GHz millimeter wave radar.

[0005] The data processing layer comprises an embedded processor.

[0006] The decision response layer comprises an edge computing server.

[0007] Preferably, the six-axis attitude sensor is selected from MPU-9250 models. Based on micro-electromechanical system technology, the six-axis attitude sensor measures the acceleration and angular velocity of an object through an internal 3-axis accelerometer and 3-axis gyroscope, and then calculates the attitude angle of the object through a Kalman filtering algorithm, so as to distinguish between the action of an old person falling down and sitting down.

[0008] The specific principle of the six-axis attitude sensor is that a specific negative acceleration and a rapid attitude change curve are generated when the old person falls, and the sensor compares the collected data with the preset characteristic curve to realize identification.

[0009] The wideband voiceprint collector adopts a 16 kHz sampling rate microphone array composed of four omnidirectional microphones, which are distributed in a square shape with a spacing of 10 cm. The technical principle is that the sound is collected by the microphone array, and after pre-amplification and filtering processing, the characteristic parameters of the sound are extracted, including the frequency spectrum and the mel frequency cepstral coefficient. The extracted sound characteristic parameters are matched with the known characteristic models of 12 types of abnormal sounds. The filter processing uses a Butterworth filter with a cutoff frequency of 20 Hz-16 kHz.

[0010] The infrared thermal imaging module selects a non-cooled infrared focal plane detector with a resolution of 320x240. The infrared detector detects the infrared radiation emitted by the object, converts it into an electrical signal, and then forms a thermal image after A / D conversion and image enhancement processing. By analyzing the temperature distribution and changes of the thermal image, indoor temperature anomalies or personnel retention for more than 8 hours within 0-60℃ can be monitored. When the temperature exceeds the normal range or there is a long-term stable heat source, it is determined as an abnormal situation.

[0011] Preferably, the vibration optical fiber sensor uses distributed optical fiber sensing technology based on the principle of light interference. When the optical fiber is vibrated, it will cause a change in the phase of light. The vibration is sensed by detecting the phase change through a Michelson interferometer. Its sensitivity reaches 0.1g acceleration, which can identify the small vibration frequency of violent lock picking. By analyzing the frequency, amplitude and duration characteristics of the vibration, normal vibration and abnormal vibration can be distinguished.

[0012] The 77GHz millimeter wave radar selects the IWR6843 model of Texas Instruments. It transmits and receives 77GHz millimeter wave signals, uses the Doppler effect and target reflection characteristics to detect targets, and has a detection distance of 5-10 meters. It can penetrate bad weather such as rain and monitor abnormal gatherings of more than 3 people outside the unit door. Through fast Fourier transform processing and analysis of the echo signal, the number, position and motion state of the target are determined.

[0013] Preferably, the associated device interface uses an RS485 interface to interact with the elevator control system, follows the Modbus protocol, and obtains elevator operation data including operation status, floor, and number of people in the car to facilitate combined risk judgment.

[0014] Preferably, the data processing layer adopts an embedded processor as the core processing unit, responsible for pre-processing, fusion and analysis of the data collected by the perception layer, cleaning the data during pre-processing, removing noise and outliers based on the 3σ criterion method, and for continuously collected data, calculating the mean and standard deviation, and determining outliers and removing them if the data exceeds the mean ± 3 times the standard deviation.

[0015] The specific steps of the 3σ criterion method are as follows:

[0016] S2011: Statistical analysis of continuous data collected by the perception layer, calculating the mean μ and standard deviation σ of the data set;

[0017] S2012: Determine the outlier judgment threshold: the lower limit is μ-3σ, and the upper limit is μ+3σ;

[0018] S2013: Check each data point in the data set one by one, if the data value exceeds the range of [μ-3σ, μ+3σ], it is determined as an outlier and removed;

[0019] When fusing data, a weighted average fusion algorithm is used to integrate data collected by different sensors. For temperature data, the weight of the infrared thermal imaging module is higher, 0.7, and the weight of other auxiliary temperature sensors is lower, 0.3. For sound data, the weight of the wideband voice collector is 0.8, and the weight of other auxiliary sound sensors is 0.2. The determination of the weight is based on the performance index of the sensor and the error analysis of historical data, and the calculation formula is: Where F is the fused data, w i is the weight of the i-th sensor, and d i is the data collected by the i-th sensor, and n is the number of sensors;

[0020] When analyzing the data, wavelet transform method is used to extract risk-related feature information from the fused data, including temperature change rate and sound energy spectrum, which provides basis for decision response. Wavelet transform method realizes the localization analysis of different frequency components of the signal by "stretching" and "shifting" the wavelet basis function, and the calculation formula is: Where f(t) is the original signal, ψ(t) is the wavelet basis function, a is the stretching factor, which controls the frequency resolution, a smaller a corresponds to high frequency signal, and a larger a corresponds to low frequency signal, b is the shift factor, which controls the time resolution, ψ* is the conjugate function of ψ, W f (a,b) is the wavelet transform coefficient;

[0021] The specific steps of the wavelet transform method are as follows:

[0022] S3011: decompose the fused multi-modal signal to obtain wavelet coefficients of different frequency bands;

[0023] S3012: extract instantaneous features from high-frequency coefficients and trend features from low-frequency coefficients;

[0024] S3013: input these feature parameters into an AI model for accurate identification of risk types.

[0025] Preferably, the decision response layer constructs a dynamic response decision engine based on an AI model trained by the safety big data of the Hongmeng community, the AI model is deployed on an edge computing server, according to the risk feature information provided by the data processing layer, an algorithm combining convolutional neural network (CNN) and recurrent neural network (RNN) in deep learning is adopted, and a corresponding disposal scheme is automatically generated in combination with the risk type, occurrence time and household characteristic factors.

[0026] Preferably, the coordination layer adopts a standardized interface to dock with community medical stations, fire control rooms and emergency resource storage points, realizes data interconnection and resource coordination, the API interface adopts RESTful style, supports HTTP / HTTPS protocol, and the MQTT protocol adopts QoS level 2 to ensure reliable transmission of messages. Through the coordination layer, the warning information and control instructions of the system can be sent to the related external resources, and the feedback information of the external resources can be received.

[0027] Compared with the existing technology, the beneficial effects of the present application are:

[0028] The present application realizes rapid emergency response within 1 minute by constructing an active security mechanism integrating real-time monitoring, second-level data processing, dynamic decision and cross-domain linkage, triggers access control locking, alarm pushing and linkage with 110 and medical resources within 10 seconds in high-risk scenarios, effectively improves the efficiency of emergency event disposal, effectively distinguishes environmental interference and real risks through multi-modal perception fusion and AI cross-validation, effectively reduces the false alarm rate, significantly improves the identification accuracy of events such as old people falling and intrusion, seamlessly integrates terminals such as elevators and gas valves with the intercom system as the core, realizes data interconnection and collaborative disposal with community medical resources, security resources and other resources, breaks the information island, reduces the cost of community comprehensive security, and improves the adaptability of the system to the safety and toughness requirements of smart communities. BRIEF DESCRIPTION OF DRAWINGS

[0029] Fig. 1 A block diagram of a building intercom intelligent emergency safety hub system based on multi-modal perception is provided for the present application;

[0030] Fig. 2 A flowchart of a building intercom intelligent emergency safety hub system based on multi-modal perception is provided for the present application. DETAILED DESCRIPTION

[0031] The application will be further described below with reference to specific examples.

[0032] Embodiment

[0033] With reference to Figs. 1-2 The embodiment proposes an intelligent emergency safety hub system based on multi-modal perception, which comprises a perception layer, a data processing layer, a decision response layer and a coordination layer. The perception layer comprises an indoor perception unit, a doorway perception unit and an associated device interface. The indoor perception unit comprises a six-axis attitude sensor, a wideband voiceprint collector and an infrared thermal imaging module. The doorway perception unit comprises a vibration fiber sensor and a 77GHz millimeter wave radar.

[0034] The six-axis attitude sensor is selected from MPU-9250. It is based on micro-electromechanical system technology and measures the acceleration and angular velocity of an object through an internal 3-axis accelerometer and 3-axis gyroscope. Then, the attitude angle of the object is calculated through a Kalman filtering algorithm, which can distinguish between the falling and sitting actions of the elderly.

[0035] The specific principle of the six-axis attitude sensor is that when the elderly fall, a specific negative acceleration and a rapid attitude change curve will be generated. The sensor compares the collected data with the preset characteristic curve to achieve recognition.

[0036] The wideband voiceprint collector adopts a 16kHz sampling rate microphone array composed of four omnidirectional microphones arranged in a square distribution with a spacing of 10cm. Its technical principle is that the sound is collected by the microphone array, and after pre-amplification and filtering processing, the characteristic parameters of the sound are extracted. The characteristic parameters include the frequency spectrum and the mel frequency cepstral coefficient. The extracted sound characteristic parameters are matched with the known characteristic models of 12 types of abnormal sounds. The filter processing adopts a Butterworth filter with a cutoff frequency of 20Hz-16kHz.

[0037] The infrared thermal imaging module selects a non-cooled infrared focal plane detector with a resolution of 320x240. It uses an infrared detector to detect the infrared radiation emitted by an object, converts it into an electrical signal, and then forms a thermal image after A / D conversion and image enhancement processing. By analyzing the temperature distribution and changes of the thermal image, indoor temperature anomalies or personnel retention for more than 8 hours can be monitored. When the temperature exceeds the normal range or there is a long-term stable heat source, it is determined as an abnormal situation.

[0038] The vibration optical fiber sensor adopts a distributed optical fiber sensing technology and is based on an interference principle of light. When the optical fiber is subjected to vibration, the phase of light changes. The vibration is sensed by detecting the phase change through a Michelson interferometer. The sensitivity is 0.1 g acceleration. The vibration frequency of a small vibration caused by violent lock picking can be identified. Normal vibration and abnormal vibration are distinguished by analyzing the frequency, amplitude and duration characteristics of the vibration.

[0039] The 77GHz millimeter wave radar selects a model IWR6843 of Texas Instruments. The 77GHz millimeter wave signals are transmitted and received. The target is detected by using the Doppler effect and the reflection characteristics of the target. The detection distance is 5-10 meters. The radar can penetrate bad weather such as rain. The monitoring unit monitors the abnormal gathering of more than 3 people outside the unit door. The number, position and motion state of the target are determined by performing fast Fourier transform processing and analysis on the echo signals.

[0040] The RS485 interface is used to interact with the elevator control system. The Modbus protocol is followed. The elevator operation data are acquired, including the operation state, the floor, and the number of people in the car. The data are combined to determine the risk, for example, when it is detected that the elevator stops at a floor without an instruction at night and the unit door is abnormally opened, the situation is determined to be high risk.

[0041] The data processing layer includes an embedded processor.

[0042] The embedded processor is used as the core processing unit in the data processing layer. The data collected by the perception layer are preprocessed, fused and analyzed. The data are cleaned during the preprocessing. The 3σ criterion-based method is used to remove noise and outliers. For the continuously collected data, the mean and standard deviation are calculated. The data that are more than ±3 times the standard deviation of the mean are determined to be outliers and are removed.

[0043] The specific steps of the 3σ criterion-based method are as follows:

[0044] S2011: The continuously collected data of the perception layer are statistically analyzed. The mean μ and the standard deviation σ of the data set are calculated.

[0045] S2012: The threshold for determining outliers is determined. The lower limit is μ-3σ, and the upper limit is μ+3σ.

[0046] S2013: Each data point in the data set is checked one by one. If the data value is outside the range [μ-3σ, μ+3σ], the data value is determined to be an outlier and is removed. For example, in the indoor temperature data collected by the infrared thermal imaging module, if the mean of a group of data is 25℃ and the standard deviation is 2℃, the 3σ range is 19℃-31℃. The temperature values (such as 35℃ or 15℃) that are outside the range are determined to be outliers. The outliers may be caused by temporary sensor failure or external interference. The outliers are removed to avoid affecting the accuracy of subsequent data analysis.

[0047] The data fusion adopts a weighted average fusion algorithm to integrate the data collected by different sensors. For temperature data, the weight of the infrared thermal imaging module is higher, 0.7, and the weight of other auxiliary temperature sensors is lower, 0.3. For sound data, the weight of the wideband voice collector is 0.8, and the weight of other auxiliary sound sensors is 0.2. The determination of the weight is based on the performance index of the sensor and the error analysis of historical data, and the calculation formula is: Where F is the fused data, w i is the weight of the i-th sensor, and d i is the data collected by the i-th sensor, and n is the number of sensors. The principle of the formula is to consider the information of multiple sensors through weighted average, so that the data of sensors with high precision and strong reliability occupies a larger proportion in the fusion result, reduces the influence of single sensor error on the result, and improves the accuracy of data;

[0048] When analyzing the data, wavelet transform method is used to extract risk-related feature information from the fused data, including temperature change rate and sound energy spectrum, which provides basis for decision response. The wavelet transform method realizes the localization analysis of different frequency components of the signal by "stretching" and "shifting" the wavelet basis function, and the calculation formula is: Where f(t) is the original signal, ψ(t) is the wavelet basis function, a is the stretching factor for controlling the frequency resolution, the smaller a corresponds to high frequency signal, and the larger a corresponds to low frequency signal, b is the shift factor for controlling the time resolution, ψ* is the conjugate function of ψ, and W f (a, b) is the wavelet transform coefficient;

[0049] The specific steps of the wavelet transform method are as follows:

[0050] S3011: Decompose the fused multi-modal signal to obtain wavelet coefficients of different frequency bands;

[0051] S3012: Extract instantaneous features from high frequency coefficients and trend features from low frequency coefficients;

[0052] S3013: Input these feature parameters into the AI model for accurate identification of risk type. For example, when processing the sound signal of the wideband voice collector, the wavelet transform can separate the low frequency fundamental component and the high frequency harmonic component of the distress sound, while retaining the time sequence characteristics of the sound. Compared with the traditional Fourier transform, it can more accurately capture the transient change of the sound and improve the accuracy of abnormal sound recognition;

[0053] The decision response layer includes an edge computing server;

[0054] The decision response layer constructs a dynamic response decision engine based on an AI model trained by the safety big data of the Hongmeng community. The AI model is deployed on an edge computing server. According to the risk feature information provided by the data processing layer, the algorithm combining a convolutional neural network (CNN) and a recurrent neural network (RNN) in deep learning is used to automatically generate a corresponding disposal scheme in combination with the risk type, occurrence time and household characteristic factors.

[0055] The convolutional neural network part adopts a ResNet-18 architecture to extract spatial features in the data, such as the temperature distribution features (shape, size, etc. of different temperature regions) of the infrared thermal image and the frequency spectrum features (energy distribution of different frequency components) of the sound signal. The recurrent neural network part adopts LSTM to process time series data, such as the change of sensor data over time. The model is trained by a large amount of safety data of the Hongmeng community, uses an Adam optimizer, sets the learning rate to 0.001, and iterates 1000 times to continuously adjust the model parameters and improve the risk identification and decision accuracy of the model.

[0056] The coordination layer adopts a standardized interface to dock with the community medical station, the fire control room and the emergency resource reserve point to realize data interconnection and resource coordination. The API interface adopts the RESTful style and supports the HTTP / HTTPS protocol. The MQTT protocol adopts QoS level 2 to ensure reliable transmission of messages. Through the coordination layer, the system can send early warning information and control instructions to related external resources and receive feedback information from the external resources at the same time.

[0057] The embodiment realizes rapid emergency response within 1 minute, triggers access control locking, alarm pushing and linkage with 110 and medical resources within 10 seconds in a high-risk scenario, effectively improves the efficiency of emergency disposal, effectively distinguishes environmental interference and real risks through multi-modal perception fusion and AI cross-validation, effectively reduces the false alarm rate, significantly improves the identification accuracy of events such as falls of the elderly and intrusion, seamlessly integrates terminals such as elevators and gas valves with the intercom system as the core, realizes data interconnection and collaborative disposal with community medical and security resources, breaks the information silos, reduces the cost of community comprehensive security, and improves the adaptability of the system to the safety resilience requirements of smart communities.

[0058] In the embodiment, the sampling frequencies of the sensors in the perception layer are set in advance. The six-axis attitude sensor is set to 100 Hz, the wideband voiceprint collector is set to 16 kHz, the infrared thermal imaging module is set to 30 Hz, the vibration fiber sensor is set to 500 Hz, and the 77GHz millimeter wave radar is set to 10 Hz.

[0059] The specific operation logic steps of the building intercom intelligent emergency safety hub system based on multi-modal perception are as follows:

[0060] S1: Data acquisition: the indoor perception unit and the entrance perception unit of the perception layer continuously collect various types of data at a set sampling frequency, including indoor posture, sound, temperature information, entrance vibration, personnel gathering information, and elevator operation data, and transmit the collected data to the data processing layer in real time through Wi-Fi, with a transmission delay controlled within 100ms;

[0061] S2: Data preprocessing: the data processing layer cleans the received data, removes invalid and erroneous data using the 3σ criterion; then synchronizes the time and calibrates the space of different types and sources of data, and then uses a weighted average fusion algorithm for fusion, considering various information to improve the quality and availability of data, for example, fusing the temperature anomaly information monitored by the infrared thermal imaging module with the sound information collected by the wideband voiceprint collector, when the temperature exceeds 50℃ and is accompanied by a smoke alarm sound, more accurately judge that a fire may occur indoors;

[0062] S3: Risk identification and assessment: the data processing layer extracts features from the fused data, extracts temperature change rate, sound energy spectrum and other features using wavelet transform, and inputs the extracted features into the AI model in the decision response layer. The AI model identifies and assesses risks according to pre-set rules and algorithms, determines the risk level, and the risk level is divided into low risk, medium risk and high risk. The low risk criterion is that the impact on personnel safety and property loss is small, and the probability of occurrence is low; the medium risk is that it may cause certain harm to personnel or property loss, and the probability of occurrence is medium; the high risk is that it may cause serious personnel casualties or major property loss, and the probability of occurrence is high;

[0063] S4: Generate disposal scheme: the decision response layer automatically generates a disposal scheme according to the risk level and related information, as follows:

[0064] Low risk: only push a prompt to the resident's mobile phone APP, the prompt content includes trigger time, trigger location and other information, do not trigger sound and light alarm, and record the trigger time to form a behavior habit analysis library for analyzing the activity rules of pets;

[0065] Medium risk: first send a voice inquiry through the indoor unit, the voice content is "Your kitchen temperature is abnormal, please confirm whether it is safe", if there is no response within 30 seconds, automatically link the intelligent home to cut off the gas valve, and push the warning to the property and the resident's emergency contact person. The property personnel need to contact the resident to confirm the situation within 5 minutes after receiving the warning;

[0066] High risk: start "three-level response" within 10 seconds - synchronize locking unit door and electromagnetic lock of adjacent unit door in the same building, response delay ≤ 2 seconds; trigger sound and light alarm in public area of the building, alarm volume ≥ 110 decibels, flashing frequency 2Hz; push real-time video with AR mark to property security terminal, mark the location of suspicious personnel in the video, positioning error ≤ 0.5 meters; automatically dial 110 and upload positioning information accurate to the house number, the positioning information is calibrated by GPS+Beidou dual-mode positioning; at the same time, broadcast calming voice through the indoor unit, which includes "security has been notified, please do not open the door, remain calm";

[0067] S5: Cross-domain resource coordination: the coordination layer sends relevant instructions and information to corresponding external resources according to the treatment scheme, realizes cross-domain resource coordination, including when detecting "heart arrest call for help sound", automatically pushing the resident's past medical history and real-time positioning to community doctors, at the same time, linking elevator priority scheduling to medical station floor, elevator response time ≤ 30 seconds, the display screen in the elevator will display "emergency rescue, please do not occupy"; when a fire occurs, the indoor unit AR projection function is used to project the escape route on the ground, avoiding the smoke area, the route update frequency is 1 time / second, and the emergency supplies cabinet at the unit door is automatically opened to pop out the fire extinguisher, the cabinet door opening time ≤ 1 second, at the same time, the indicator light on the emergency supplies cabinet will light up to indicate the position;

[0068] S6: Feedback and record: the system receives feedback information from external resources, including property security's out-of-police situation, departure time, estimated arrival time; rescue progress of medical station, has departed, is in rescue, rescue is completed; and stores the whole event processing process and related data in local MySQL database and cloud server Aliyun ECS for subsequent analysis and optimization, and the data saving time is 3 years.

[0069] The above is only the preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any skilled person in the art can make equivalent replacement or change according to the technical scheme and the inventive concept of the present application within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A multi-modal perception based intelligent emergency security hub system for building intercoms, characterized in that, It comprises a perception layer, a data processing layer, a decision response layer and a coordination layer, the perception layer comprises an indoor perception unit, a doorway perception unit and an associated device interface, the indoor perception unit comprises a six-axis attitude sensor, a wideband voiceprint collector and an infrared thermal imaging module, the doorway perception unit comprises a vibration optical fiber sensor and a 77GHz millimeter wave radar; The data processing layer comprises an embedded processor; The decision response layer comprises an edge computing server.

2. The multi-modal perception based intelligent emergency security hub system for building intercoms as claimed in claim 1 wherein, The six-axis attitude sensor selects MPU-9250 model, which is based on micro-electromechanical system technology, measures the acceleration and angular velocity of the object through the internal 3-axis accelerometer and 3-axis gyroscope, and then calculates the attitude angle of the object through Kalman filtering algorithm, which can distinguish between the falling and sitting actions of the old people; The wideband voiceprint collector adopts a microphone array with a sampling rate of 16kHz, which is composed of 4 omnidirectional microphones and is distributed in a square shape with a spacing of 10cm, its technical principle is: through the microphone array to collect sound, after pre-amplification and filtering processing, the characteristic parameters of the sound are extracted, Its characteristic parameters include frequency spectrum and mel frequency cepstral coefficient, the extracted sound characteristic parameters are matched with the known characteristic models of 12 types of abnormal sounds, and the Butterworth filter is used for filtering processing with a cutoff frequency of 20Hz-16kHz; The infrared thermal imaging module selects a non-cooled infrared focal plane detector with a resolution of 320x240, uses the infrared detector to detect the infrared radiation emitted by the object, converts it into an electrical signal, and then forms a thermal image through A / D conversion and image enhancement processing, by analyzing the temperature distribution and change of the thermal image, the indoor temperature anomaly of 0-60℃ or the personnel retention of more than 8 hours can be monitored, when the temperature exceeds the normal range or there is a long-term stable heat source, it is determined as an abnormal situation.

3. The multi-modal perception based intelligent emergency security hub system for building intercoms of claim 1, wherein, The vibration optical fiber sensor adopts distributed optical fiber sensing technology, based on the interference principle of light, when the optical fiber is vibrated, the phase of light will change, the vibration is sensed by detecting the phase change through the Michelson interferometer, the sensitivity reaches 0.1g acceleration, can identify the small vibration frequency of violent lock picking, and distinguish normal vibration and abnormal vibration by analyzing the frequency, amplitude and duration characteristics of vibration; The 77GHz millimeter wave radar selects IWR6843 model of Texas Instruments, transmits and receives 77GHz millimeter wave signals, uses Doppler effect and target reflection characteristics to detect targets, the detection distance is 5-10 meters, can penetrate bad weather such as rain, and monitor the abnormal gathering of more than 3 people outside the unit door, through fast Fourier transform processing and analysis of the echo signal, the number, position and motion state of the target are determined.

4. The multi-modal perception based intelligent emergency security hub system for building intercoms of claim 1, wherein, The associated device interface adopts RS485 interface to interact with the elevator control system, follows Modbus protocol, obtains elevator operation data including operation state, floor, number of people in the car, so as to make combined risk judgment.

5. The multi-modal perception based intelligent emergency security hub system for building intercoms of claim 1, wherein, The data processing layer adopts an embedded processor as a core processing unit, is responsible for pre-processing, fusion and analysis of data collected by the perception layer, cleans the data during data preprocessing, removes noise and outliers based on the 3σ criterion method, and calculates the mean and standard deviation of continuously collected data, and determines and removes data exceeding the mean ± 3 times the standard deviation as outliers; The specific steps of the 3σ criterion method are as follows: S2011: statistically analyze the continuous data collected by the perception layer, and calculate the mean μ and standard deviation σ of the data set; S2012: determine the abnormal value judgment threshold: the lower limit is μ-3σ, and the upper limit is μ+3σ; S2013: check each data point in the data set one by one, if the data value exceeds the range of [μ-3σ, μ+3σ], it is determined as an abnormal value and removed; The data collected by different sensors are integrated by using a weighted average fusion algorithm. For temperature data, the weight of the infrared thermal imaging module is higher, i.e. 0.7, and the weight of other auxiliary temperature sensors is lower, i.e. 0.

3. For sound data, the weight of the wideband voiceprint collector is 0.8, and the weight of other auxiliary sound sensors is 0.

2. The weight is determined based on the performance index of the sensor and error analysis of historical data, and the calculation formula is: wherein F is the fused data, w i is the weight of the i th sensor, and d i is the data collected by the i th sensor, and n is the number of sensors. When analyzing the data, wavelet transform method is used to extract the risk-related characteristic information from the fused data, including temperature change rate and sound energy spectrum, which provides the basis for decision response. Wavelet transform method realizes the localization analysis of different frequency components of the signal by "stretching" and "shifting" the wavelet basis function, and the calculation formula is: Wherein, f(t) is the original signal, ψ(t) is the wavelet basis function, a is the stretching factor, which is used to control the frequency resolution, the smaller a corresponds to the high frequency signal, the larger a corresponds to the low frequency signal, b is the shift factor, which is used to control the time resolution, ψ* is the conjugate function of ψ, W f (a, b) is the wavelet transform coefficient; The specific steps of the wavelet transform method are as follows: S3011: decompose the fused multi-modal signal to obtain wavelet coefficients of different frequency bands; S3012: extract instantaneous features from high-frequency coefficients and trend features from low-frequency coefficients; S3013: input these feature parameters into the AI model for accurate identification of risk types.

6. The multi-modal perception based intelligent emergency security hub system for building intercoms of claim 1, wherein, The decision response layer constructs a dynamic response decision engine based on the AI model trained by the safety big data of the Hongmeng community, the AI model is deployed on the edge computing server, according to the risk feature information provided by the data processing layer, using the algorithm combining convolutional neural network and recurrent neural network in deep learning, combining risk type, occurrence time, and household characteristic factors, automatically generating the corresponding disposal scheme.

7. The multi-modal perception based intelligent emergency security hub system for intercoms in a building as claimed in claim 1, wherein, The coordination layer adopts a standardized interface to dock with community medical stations, fire control rooms and emergency material storage points, realizes data interconnection and resource coordination, the API interface adopts RESTful style, supports HTTP / HTTPS protocol; The MQTT protocol adopts QoS level 2 to ensure reliable transmission of messages, through the coordination layer, the system can send early warning information and control instructions to related external resources, and receive feedback information from external resources.

Images