Intelligent building intercom system based on multi-modal interaction and adaptive scene and control method

Abstract

The invention discloses an intelligent building intercom system based on multi-modal interaction and an adaptive scene and a control method. The system comprises a full-scene interaction terminal, an edge sensing node, an edge server and a cloud platform. According to the invention, the multi-mode interaction technology integrating voice, touch and vision is adopted, and dialects and custom instructions are supported, so that the operation threshold of old and disabled users is greatly reduced, and the operation success rate of the old and disabled users is increased to 98% from 62% of a traditional scheme; through environment self-adaptive technologies such as double cameras, self-adaptive light supplement and panel heating in rainy days, the problem of recognition and touch failure in backlighting, night and humid environments is effectively solved, the face recognition accuracy is improved to 99.5%, and the touch failure rate is reduced to 3% or below; in the aspect of safety response, the system has voice keyword triggering and millimeter wave radar physiological monitoring capabilities, and the average response time of emergency events is effectively shortened in combination with precise positioning and intelligent verification of an edge server.

Description

Technical Field

[0001] This invention relates to the field of intelligent building intercom technology, and in particular to an intelligent building intercom system and control method based on multimodal interaction and adaptive scenarios. Background Technology

[0002] Existing building intercom systems suffer from the following technical defects in practical applications: 1. High interaction threshold: Relying on touchscreen or physical button operation, they are difficult for elderly users and people with disabilities to use. A survey in a community showed that the success rate of operation for users over 65 years old was only 62%; 2. Poor scene adaptability: The accuracy of facial recognition drops sharply at night or in backlight conditions, and the failure rate of touch control on glass panels in rainy weather reaches 27%; 3. Delayed emergency response: Traditional systems only support one-way calls and cannot automatically locate the location of the person seeking help. The average response time for emergency events exceeds 8 minutes; 4. Data silo problem: There is a lack of linkage with community medical, fire protection and other systems. When an elderly person living alone suffers a sudden illness, the rescue process cannot be automatically triggered. In view of the above, this application proposes an intelligent building intercom system and control method based on multimodal interaction and adaptive scenarios. Summary of the Invention

[0003] Based on the technical problems existing in the background technology, this invention proposes an intelligent building intercom system and control method based on multimodal interaction and adaptive scenarios.

[0004] The present invention proposes an intelligent building intercom system based on multimodal interaction and adaptive scenarios, including a full-scenario interactive terminal, edge sensing nodes, edge servers and cloud platforms; The full-scene interactive terminal includes a far-field voice module, dual cameras, an adaptive fill light unit, and a pressure-sensing panel; The edge sensing node includes a millimeter-wave radar sensor and an environmental sensor; The edge server has a built-in speech recognition model and face comparison algorithm, supporting local data processing and emergency event location. The cloud platform is used to store user permission data and operation logs, and to connect with third-party service systems.

[0005] Preferably, the millimeter-wave radar sensor is used for human presence detection. The edge sensing node detects human micro-movement signals through millimeter-wave radar. When no signal is detected for 24 consecutive hours, the emergency assistance process for elderly people living alone is automatically triggered. The environmental sensors include a temperature sensor, a humidity sensor, a smoke sensor, a light intensity sensor, and a carbon monoxide sensor; The far-field voice module supports dialect recognition, and the dual cameras are a combination of infrared and RGB. The third-party service systems mentioned are property management, medical, and fire protection systems.

[0006] Preferably, the all-scenario interactive terminal supports multimodal authentication to identify user types. Its multimodal authentication includes a combination of voiceprint, face, and IC card authentication, and can automatically switch authentication methods according to user types, including the elderly and disabled.

[0007] Preferably, the adaptive supplementary lighting unit can adjust the supplementary lighting intensity by 10-1000 lumens according to the ambient light intensity, and activate the infrared camera for face recognition in backlight conditions.

[0008] This invention also proposes a control method for an intelligent building intercom system based on multimodal interaction and adaptive scenarios, comprising the following steps: S1: The full-scenario interactive terminal detects emergency events through voice recognition or sensors; S2: The edge server obtains the location information of the emergency event detected in S1 and verifies whether it is a real event through millimeter-wave radar. S3: If the event is real, trigger a multi-level response and push alarms to the property terminal, emergency contacts and third-party service platforms according to the preset priority. S4: The property terminal, emergency contacts, and third-party service platforms handle the incident. The cloud platform receives the incident handling feedback, generates a closed-loop log, and synchronizes it to the cloud storage.

[0009] Preferably, in step S1, the specific steps for the sensor to detect an emergency event are as follows: S101: Continuous data acquisition and preprocessing: The millimeter-wave radar sensor continuously emits low-frequency electromagnetic waves and receives echoes. Through algorithm analysis of the micro-Doppler effect, it extracts vital signs signals such as respiratory rate, heart rate, and body micro-movements, and determines whether the human body exists and its static / dynamic state. At the same time, the temperature and humidity sensor collects environmental temperature and humidity data at a frequency of 1 time / second, the smoke sensor continuously monitors the concentration of smoke particles in the air, and the carbon monoxide sensor continuously monitors the CO gas concentration value in the environment. S102: Local Abnormal State Identification: The system compares the data collected in real time in S1 with the preset safety threshold, and uses a lightweight algorithm model to judge physiological abnormalities and environmental disasters.

[0010] Preferably, in S102, the algorithm logic for judging physiological abnormalities is as follows: if the millimeter-wave radar sensor does not detect any human micro-movement for 24 hours, but continuously detects weak breathing and heartbeat signals in a static state, it is initially judged that "the elderly may have fallen or lost the ability to move". The logic for environmental disaster judgment is as follows: (1) Fire judgment: The smoke sensor concentration value is greater than the preset threshold and the temperature and humidity sensor temperature value rises sharply in a short period of time to the preset threshold. Only when both conditions are met can it be initially judged as a "fire alarm" to avoid false alarms caused by smoking or dust; (2) Gas leak judgment: The carbon monoxide sensor concentration value is greater than the safety threshold, and it is initially judged as a "gas leak".

[0011] Preferably, the specific logical steps of S2 are as follows: S201: Receiving Alarms and Context Extraction: The edge server receives an emergency event alarm from the terminal. The alarm includes: event unique identifier event_id, device ID that triggered the event terminal_id, event type event_type, timestamp, and confidence_level locally identified by the terminal. S202: Calling the radar data stream: The edge server accurately locates the physical location of the event based on the terminal_id and sends a command to the edge sensing node that manages the area to request the raw data stream and historical analysis results of the millimeter-wave radar at that location over the most recent period. S203: Event Type-Based Specific Algorithm Verification: The edge server calls different pre-built algorithm models to analyze and verify radar data based on different event types (event_type). The specific verification algorithm is as follows: Verification Algorithm 1: Physiological Abnormality Verification: The specific steps are as follows: S20311: Based on the constant false alarm rate detection algorithm, it distinguishes between human targets and static clutter, and confirms whether there are stationary human targets within the radar field of view; S20312: Perform Fourier Transform (FFT) or Continuous Wavelet Transform (CWT) on radar echo signals to separate the spectral characteristics of respiration and heartbeat, and analyze the peak values ​​in the spectrum; S20313: The radar detected the presence of a human body and was able to extract valid, but possibly abnormal, vital signs. This verified the hypothesis that "the person is still here, but not moving," confirming it as a real event. The radar did not detect any human targets, indicating that the user may have gone out, confirming it as a false alarm.

[0012] Verification Algorithm 2: Violent Action Verification: The specific steps are as follows: S20321: Micro-Doppler feature extraction of Doppler frequency shift in radar echoes, motion is sensed by Doppler frequency shift in radar echoes, and a "micro-Doppler spectrum map" is generated by time-frequency analysis, which can depict the details of human movement; S20322: Real-time analysis of micro-Doppler spectrograms using a pre-trained deep learning model; S20323: In the spectrogram near the alarm time point, the model identified high-confidence features of "falling down" or "violent movement", which corroborated the voice keywords and confirmed that it was a real event. The model identified only features of "normal walking", "sitting down" or "no activity" during this time period, indicating that the user may have misspoke the keywords while watching TV, and confirmed it as a false alarm. Verification Algorithm 3: Spatial Situation Verification: The specific steps are as follows: S20331: Use Kalman filtering or particle filtering to track multiple targets detected by radar and generate motion trajectories; S20332: Analyzes the real-time distribution and movement vectors of personnel in space through situational analysis; S20333: For high-risk real events, after the smoke alarm is triggered, the radar detects a human target that remains stationary in the danger zone for a long time. This verifies that "someone is trapped or unconscious" and is confirmed as a real event with extremely high priority. For low-risk / false alarm situations, after the smoke alarm is triggered, the radar shows that people are moving quickly toward the exit. The system can determine that it is a real fire alarm but people have already responded. The alarm can still be triggered, but the priority can be adjusted appropriately. S204: Generate verification results and execute: The edge server, based on the above analysis, generates a final "verification confidence level". If the confidence level exceeds the preset threshold, it is determined to be a real event, and a complete multi-level response process is triggered. If the confidence level is lower than the threshold, it is determined to be a false alarm. The system does not trigger external alarms, but will record this false alarm event in the cloud log for subsequent algorithm optimization.

[0013] Compared with existing technologies, the beneficial effects of this invention are: This invention significantly reduces the operational threshold for elderly and disabled users by employing multimodal interaction technology that integrates voice, touch, and vision, and supports dialects and custom commands, increasing the success rate from 62% in traditional solutions to 98%. Through dual cameras and environmental adaptive technologies such as adaptive lighting and panel heating in rainy conditions, it effectively overcomes the challenges of recognition and touch malfunctions in backlight, nighttime, and humid environments, raising the facial recognition accuracy to 99.5% and reducing the touch malfunction rate to below 3%. In terms of security response, the system features voice keyword triggering and millimeter-wave radar physiological monitoring capabilities. Combined with precise positioning and intelligent verification by edge servers, it effectively shortens the average response time for emergency events and can automatically link with property management, family members, and community service departments to form an efficient closed-loop response. Simultaneously, the system has open ecosystem interfaces that can seamlessly connect with third-party services such as medical, fire protection, and smart home solutions, ultimately constructing a smart building intercom solution that is adaptable to all user groups and scenarios, responds rapidly, and has strong ecosystem scalability. Attached Figure Description

[0014] Figure 1 This is a system block diagram of an intelligent building intercom system based on multimodal interaction and adaptive scenarios proposed in this invention. Figure 2 This is a flowchart illustrating the sensor detection of emergency events in the control method of an intelligent building intercom system based on multimodal interaction and adaptive scenarios proposed in this invention. Figure 3 This is a flowchart of step S2 in the control method of an intelligent building intercom system based on multimodal interaction and adaptive scenarios proposed in this invention. Detailed Implementation

[0015] The present invention will be further explained below with reference to specific embodiments. Example

[0016] Reference Figure 1-3 This embodiment proposes an intelligent building intercom system based on multimodal interaction and adaptive scenarios, including a full-scenario interactive terminal, edge sensing nodes, edge servers and a cloud platform; The all-scenario interactive terminal includes a far-field voice module, dual cameras, an adaptive fill light unit, and a pressure-sensing panel. The all-scenario interactive terminal supports multimodal authentication to identify user types. Its multimodal authentication includes a combination of voiceprint, face, and IC card authentication, and can automatically switch authentication methods according to user types, including the elderly and disabled. The adaptive fill light unit can adjust the fill light intensity from 10 to 1000 lumens according to the ambient light intensity, and activate the infrared camera for face recognition in backlight conditions. Edge sensing nodes include millimeter-wave radar sensors and environmental sensors; The edge server has a built-in speech recognition model and face comparison algorithm, supporting local data processing and emergency event location; The cloud platform is used to store user permission data and operation logs, and to connect with third-party service systems; Among them, the millimeter-wave radar sensor is used to detect the presence of the human body, and the edge perception node detects the micro-movement signals of the human body through the millimeter-wave radar. When no signal is detected for 24 consecutive hours, the emergency help process for elderly people living alone is automatically triggered. Environmental sensors include temperature sensors, humidity sensors, smoke sensors, light intensity sensors, and carbon monoxide sensors; The far-field voice module supports dialect recognition, and the dual cameras are a combination of infrared and RGB. Third-party service systems include property management, medical, and fire protection systems.

[0017] This embodiment also proposes a control method for an intelligent building intercom system based on multimodal interaction and adaptive scenarios, including the following steps: S1: The full-scenario interactive terminal detects emergency events through voice recognition or sensors; The specific steps for the sensor to detect emergency events are as follows: S101: Continuous data acquisition and preprocessing: The millimeter-wave radar sensor continuously emits low-frequency electromagnetic waves and receives echoes. Through algorithm analysis of the micro-Doppler effect, it extracts vital signs signals such as respiratory rate, heart rate, and body micro-movements, and determines whether the human body exists and its static / dynamic state. At the same time, the temperature and humidity sensor collects environmental temperature and humidity data at a frequency of 1 time / second, the smoke sensor continuously monitors the concentration of smoke particles in the air, and the carbon monoxide sensor continuously monitors the CO gas concentration value in the environment. S102: Local Abnormal State Identification: The system compares the data collected in real time in S1 with the preset safety threshold, and uses a lightweight algorithm model to judge physiological abnormalities and environmental disasters. The algorithm logic for judging physiological abnormalities is as follows: if the millimeter-wave radar sensor does not detect any human micro-movement for 24 hours, but continuously detects weak breathing and heartbeat signals in a static state, it is initially judged that "the elderly may have fallen or lost the ability to move". The logic for environmental disaster judgment is as follows: (1) Fire judgment: The smoke sensor concentration value > preset threshold and the temperature and humidity sensor temperature value rises sharply in a short period of time > preset threshold. Only when both conditions are met can it be initially judged as "fire alarm" to avoid false alarms caused by smoking or dust; (2) Gas leak judgment: The carbon monoxide sensor concentration value > safety threshold can be initially judged as "gas leak". S2: The edge server obtains the location information of the emergency event detected in S1 and verifies whether it is a real event through millimeter-wave radar. The specific logical steps are as follows: S201: Receiving Alarms and Context Extraction: The edge server receives an emergency event alarm from the terminal. The alarm includes: event unique identifier event_id, device ID that triggered the event terminal_id, event type event_type, timestamp, and confidence_level locally identified by the terminal. S202: Calling the radar data stream: The edge server accurately locates the physical location of the event based on the terminal_id and sends a command to the edge sensing node that manages the area to request the raw data stream and historical analysis results of the millimeter-wave radar at that location over the most recent period. S203: Event Type-Based Specific Algorithm Verification: The edge server calls different pre-built algorithm models to analyze and verify radar data based on different event types (event_type). The specific verification algorithm is as follows: Verification Algorithm 1: Physiological Abnormality Verification: The specific steps are as follows: S20311: Based on the constant false alarm rate detection algorithm, it distinguishes between human targets and static clutter, and confirms whether there are stationary human targets within the radar field of view; S20312: Perform Fourier Transform (FFT) or Continuous Wavelet Transform (CWT) on radar echo signals to separate the spectral characteristics of respiration and heartbeat, and analyze the peak values ​​in the spectrum; S20313: The radar detected the presence of a human body and was able to extract valid, but possibly abnormal, vital signs. This verified the hypothesis that "the person is still here, but not moving," confirming it as a real event. The radar did not detect any human targets, indicating that the user may have gone out, confirming it as a false alarm.

[0018] Verification Algorithm 2: Violent Action Verification: The specific steps are as follows: S20321: Micro-Doppler feature extraction of Doppler frequency shift in radar echoes, motion is sensed by Doppler frequency shift in radar echoes, and a "micro-Doppler spectrum map" is generated by time-frequency analysis, which can depict the details of human movement; S20322: Real-time analysis of micro-Doppler spectrograms using a pre-trained deep learning model; S20323: In the spectrogram near the alarm time point, the model identified high-confidence features of "falling down" or "violent movement", which corroborated the voice keywords and confirmed that it was a real event. The model identified only features of "normal walking", "sitting down" or "no activity" during this time period, indicating that the user may have misspoke the keywords while watching TV, and confirmed it as a false alarm. Verification Algorithm 3: Spatial Situation Verification: The specific steps are as follows: S20331: Use Kalman filtering or particle filtering to track multiple targets detected by radar and generate motion trajectories; S20332: Analyzes the real-time distribution and movement vectors of personnel in space through situational analysis; S20333: For high-risk real events, after the smoke alarm is triggered, the radar detects a human target that remains stationary in the danger zone for a long time. This verifies that "someone is trapped or unconscious" and is confirmed as a real event with extremely high priority. For low-risk / false alarm situations, after the smoke alarm is triggered, the radar shows that people are moving quickly toward the exit. The system can determine that it is a real fire alarm but people have already responded. The alarm can still be triggered, but the priority can be adjusted appropriately. S204: Generate verification results and execute: The edge server, based on the above analysis, generates a final "verification confidence level". If the confidence level exceeds the preset threshold, it is determined to be a real event and a complete multi-level response process is triggered. If the confidence level is lower than the threshold, it is determined to be a false alarm. The system does not trigger external alarms, but records the false alarm event in the cloud log for subsequent algorithm optimization. S3: If the event is real, trigger a multi-level response and push alarms to property terminals, emergency contacts and third-party platforms according to preset priorities; Its multi-level response protocol is as follows: (1) Property terminal: Immediately pop up a pop-up alarm on the property management system, displaying the event type (e.g., warning of elderly living alone), the exact location (Room 202, Unit 1, Building 8), and trigger the sound and light alarm; (2) Emergency Contacts: Automatically call SMS and telephone interfaces to send alarm information and location to preset family members' mobile phones; (3) Third-party systems (such as community hospitals and fire departments): Through the API interface opened by the cloud platform, structured alarm data is pushed to the corresponding institutions (for example, physiological abnormality alarms are pushed to community hospitals, and fire alarms are pushed to fire departments). S4: The property terminal, emergency contacts and third-party service platforms handle the incident. The cloud platform receives the incident handling feedback, generates a closed-loop log and synchronizes it to the cloud storage. This embodiment, by employing multimodal interaction technology that integrates voice, touch, and vision, and supporting dialects and custom commands, significantly reduces the operational threshold for elderly and disabled users, increasing their success rate from 62% in traditional solutions to 98%. Through dual cameras and environmental adaptive technologies such as adaptive lighting and panel heating in rainy conditions, it effectively overcomes the challenges of recognition and touch malfunctions in backlight, nighttime, and humid environments, raising facial recognition accuracy to 99.5% and reducing touch malfunction rate to below 3%. In terms of security response, the system possesses voice keyword triggering and millimeter-wave radar physiological monitoring capabilities. Combined with precise positioning and intelligent verification by edge servers, it effectively shortens the average response time for emergency events and can automatically link with property management, family members, and community service departments to form an efficient closed-loop response. Simultaneously, the system has open ecosystem interfaces that can seamlessly connect with third-party services such as medical, fire protection, and smart home solutions, ultimately constructing a smart building intercom solution that is adaptable to all user groups and scenarios, responsive, and highly scalable.

[0019] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An intelligent building intercom system based on multimodal interaction and adaptive scenarios, characterized in that, This includes full-scenario interactive terminals, edge sensing nodes, edge servers, and cloud platforms; The full-scene interactive terminal includes a far-field voice module, dual cameras, an adaptive fill light unit, and a pressure-sensing panel; The edge sensing node includes a millimeter-wave radar sensor and an environmental sensor; The edge server has a built-in speech recognition model and face comparison algorithm, supporting local data processing and emergency event location. The cloud platform is used to store user permission data and operation logs, and to connect with third-party service systems.

2. The intelligent building intercom system based on multimodal interaction and adaptive scenarios according to claim 1, characterized in that, The millimeter-wave radar sensor is used for human presence detection. The edge sensing node detects human micro-movement signals through millimeter-wave radar. When no signal is detected for 24 consecutive hours, the emergency assistance process for elderly people living alone is automatically triggered. The environmental sensors include a temperature sensor, a humidity sensor, a smoke sensor, a light intensity sensor, and a carbon monoxide sensor; The far-field voice module supports dialect recognition, and the dual cameras are a combination of infrared and RGB. The third-party service systems mentioned are property management, medical, and fire protection systems.

3. The intelligent building intercom system based on multimodal interaction and adaptive scenarios according to claim 1, characterized in that, The full-scene interactive terminal supports multimodal authentication to identify user types. Its multimodal authentication includes a combination of voiceprint, face, and IC card authentication, and can automatically switch authentication methods according to user types, including the elderly and disabled.

4. The intelligent building intercom system based on multimodal interaction and adaptive scenarios according to claim 1, characterized in that, The adaptive fill light unit can adjust the fill light intensity by 10-1000 lumens according to the ambient light intensity, and activate the infrared camera for face recognition in backlight conditions.

5. A control method for an intelligent building intercom system based on multimodal interaction and adaptive scenarios, used to implement the system described in any one of claims 1-4, characterized in that, Includes the following steps: S1: The full-scenario interactive terminal detects emergency events through voice recognition or sensors; S2: The edge server obtains the location information of the emergency event detected in S1 and verifies whether it is a real event through millimeter-wave radar. S3: If the event is real, trigger a multi-level response and push alarms to the property terminal, emergency contacts and third-party service platforms according to the preset priority. S4: The property terminal, emergency contacts, and third-party service platforms handle the incident. The cloud platform receives the incident handling feedback, generates a closed-loop log, and synchronizes it to the cloud storage.

6. The control method for an intelligent building intercom system based on multimodal interaction and adaptive scenarios according to claim 5, characterized in that, In step S1, the specific steps for the sensor to detect an emergency event are as follows: S101: Continuous data acquisition and preprocessing: The millimeter-wave radar sensor continuously emits low-frequency electromagnetic waves and receives echoes. Through algorithm analysis of the micro-Doppler effect, it extracts vital signs signals such as respiratory rate, heart rate, and body micro-movements, and determines whether the human body exists and its static / dynamic state. At the same time, the temperature and humidity sensor collects environmental temperature and humidity data at a frequency of 1 time / second, the smoke sensor continuously monitors the concentration of smoke particles in the air, and the carbon monoxide sensor continuously monitors the CO gas concentration value in the environment. S102: Local Abnormal State Identification: The system compares the data collected in real time in S1 with the preset safety threshold, and uses a lightweight algorithm model to judge physiological abnormalities and environmental disasters.

7. The control method for an intelligent building intercom system based on multimodal interaction and adaptive scenarios according to claim 6, characterized in that, In S102, the algorithm logic for judging physiological abnormalities is as follows: if the millimeter-wave radar sensor does not detect any human micro-movement for 24 hours, but continuously detects weak breathing and heartbeat signals in a static state, it is initially judged that "the elderly may have fallen or lost the ability to move". The logic for environmental disaster judgment is as follows: (1) Fire judgment: The smoke sensor concentration value is greater than the preset threshold and the temperature and humidity sensor temperature value rises sharply in a short period of time to the preset threshold. Only when both conditions are met can it be initially judged as "fire alarm" to avoid false alarms caused by smoking or dust; (2) Gas leak judgment: The carbon monoxide sensor concentration value is greater than the safety threshold, and it is initially judged as "gas leak".

8. The control method for an intelligent building intercom system based on multimodal interaction and adaptive scenarios according to claim 5, characterized in that, The specific logical steps of S2 are as follows: S201: Receiving Alarms and Context Extraction: The edge server receives an emergency event alarm from the terminal. The alarm includes: event unique identifier event_id, device ID that triggered the event terminal_id, event type event_type, timestamp, and confidence_level locally identified by the terminal. S202: Calling the radar data stream: The edge server accurately locates the physical location of the event based on the terminal_id and sends a command to the edge sensing node that manages the area to request the raw data stream and historical analysis results of the millimeter-wave radar at that location over the most recent period. S203: Event Type-Based Specific Algorithm Verification: The edge server calls different pre-built algorithm models to analyze and verify radar data based on different event types (event_type). The specific verification algorithm is as follows: Verification Algorithm 1: Physiological Abnormality Verification: The specific steps are as follows: S20311: Based on the constant false alarm rate detection algorithm, it distinguishes between human targets and static clutter, and confirms whether there are stationary human targets within the radar field of view; S20312: Perform Fourier transform or continuous wavelet transform on radar echo signals to separate the spectral characteristics of respiration and heartbeat, and analyze the peak values ​​in the spectrum; S20313: The radar detected the presence of a human body and was able to extract valid, but possibly abnormal, vital signs. This verified the hypothesis that "the person is still here, but not moving," confirming it as a real event. The radar did not detect any human targets, indicating that the user may have gone out, confirming it as a false alarm. Verification Algorithm 2: Violent Action Verification: The specific steps are as follows: S20321: Micro-Doppler feature extraction of Doppler frequency shift in radar echoes, motion is perceived by Doppler frequency shift in radar echoes, and a "micro-Doppler spectrum map" is generated by time-frequency analysis. This image can depict the details of human movement. S20322: Real-time analysis of micro-Doppler spectrograms using a pre-trained deep learning model; S20323: In the spectrogram near the alarm time point, the model identified high-confidence features of "falling down" or "violent movement", which corroborated the voice keywords and confirmed them as real events. The model identified only features of "normal walking", "sitting down" or "no activity" during this time period, indicating that the user may have misspoke the keywords while watching TV, and confirmed it as a false alarm. Verification Algorithm 3: Spatial Situation Verification: The specific steps are as follows: S20331: Use Kalman filtering or particle filtering to track multiple targets detected by radar and generate motion trajectories; S20332: Analyzes the real-time distribution and movement vectors of personnel in space through situational analysis; S20333: For high-risk real events, after the smoke alarm is triggered, the radar detects a human target that remains stationary in the danger zone for a long time. This verifies that "someone is trapped or unconscious" and is confirmed as a real event with extremely high priority. For low-risk / false alarm situations, after the smoke alarm is triggered, the radar shows that people are moving quickly toward the exit. The system can determine that it is a real fire alarm but people have already responded. The alarm can still be triggered, but the priority can be adjusted appropriately. S204: Generate verification results and execute: The edge server, based on the above analysis, generates a final "verification confidence level". If the confidence level exceeds the preset threshold, it is determined to be a real event, and a complete multi-level response process is triggered. If the confidence level is lower than the threshold, it is determined to be a false alarm. The system does not trigger external alarms, but will record this false alarm event in the cloud log for subsequent algorithm optimization.

Images