Smart home safety monitoring system and method based on multi-mode perception and intelligent analysis

Abstract

The invention discloses a smart home safety monitoring system and method based on multi-modal sensing and intelligent analysis. The system comprises a multi-modal sensing module, an intelligent analysis module, a communication module, a cloud server, a user terminal and a smart home central control screen module. According to the invention, through multi-mode sensing fusion and intelligent analysis technologies, all-around and multi-level sensing of an intelligent home environment can be realized, compared with a traditional single-sensor monitoring system, various security events can be detected and identified more accurately, the reliability and the security of the system are improved, the identity and behavior actions of personnel can be judged accurately, and the safety of the personnel is improved. The multi-mode sensing module, the intelligent analysis module and the cloud server are integrated, abnormal behaviors are recognized, early warning is given out in time, potential risks are reduced, in addition, real-time data transmission among the multi-mode sensing module, the intelligent analysis module and the cloud server is achieved by adopting multiple wireless communication protocols, and safety and privacy of data transmission are guaranteed in the transmission process in combination with the data encryption unit.

Description

Technical Field

[0001] The present invention relates to the field of smart home security monitoring technology, and in particular to a smart home security monitoring system and method based on multimodal perception and intelligent analysis. Background Art

[0002] With the rapid development of smart homes, people's lives have become more convenient and comfortable. However, the security and monitoring issues in smart home environments have become increasingly prominent. Traditional home security monitoring systems mainly rely on cameras for video monitoring, with relatively simple functions and the following shortcomings: (1) Most existing monitoring systems can only perform simple video recording and real-time viewing. When abnormal situations occur, they cannot be identified and analyzed in a timely and accurate manner, and require manual intervention to respond, which is inefficient. For example, when a stranger breaks in, the system may not be able to quickly determine his behavioral intentions, resulting in security risks that cannot be eliminated in time; (2) Traditional monitoring systems lack the ability to comprehensively perceive multiple environmental information. There are multiple security factors in smart home environments, such as smoke, gas leaks, abnormal sounds, etc., and video monitoring alone cannot fully cover these potential risks; (3) Existing monitoring systems also have security risks in data transmission and storage, which can easily lead to user privacy leaks. In summary, this application proposes a smart home security monitoring system and method based on multimodal perception and intelligent analysis. Summary of the Invention

[0003] Based on the technical problems existing in the background technology, the present invention proposes a smart home security monitoring system and method based on multimodal perception and intelligent analysis.

[0004] The smart home security monitoring system based on multimodal perception and intelligent analysis proposed in the present invention includes a multimodal perception module, an intelligent analysis module, a communication module, a cloud server, a user terminal and a smart home central control screen module;

[0005] The multimodal perception module includes a high-definition camera, an infrared sensor, a smoke sensor, a gas sensor, and a sound sensor;

[0006] The intelligent analysis module has built-in image recognition algorithm, multimodal data fusion algorithm and behavior analysis model;

[0007] The communication module includes a wireless communication unit and a data encryption unit;

[0008] The cloud server includes a data storage unit, an intelligent decision-making unit and a user management unit;

[0009] The user terminal includes a mobile phone APP and a Web-based management interface;

[0010] The smart home central control screen module is built into the external central control screen. The smart home central control screen module includes a centralized control unit, a security monitoring display unit, a scene mode switching unit and a personalized customization unit.

[0011] Preferably, the high-definition camera has high resolution, night vision function and wide-angle shooting capability, can clearly capture indoor and outdoor environment images and videos, supports intelligent image analysis algorithms, and can identify and track people and objects in the picture;

[0012] The infrared sensor is used to detect infrared radiation from the human body and can accurately sense human activities at night or in dimly lit environments. Different sensing sensitivities can be set to suit various scenarios.

[0013] The smoke sensor uses advanced electrochemical or photoelectric sensing technology to quickly and accurately detect changes in smoke concentration and promptly issues an alarm signal when the smoke concentration exceeds a set threshold.

[0014] The gas sensor is specifically used to detect gas leaks, such as natural gas and liquefied petroleum gas. Through the highly sensitive sensor element, it can detect tiny changes in gas concentration in the early stages of gas leaks and issue an alarm.

[0015] The sound sensor is used to monitor the sound intensity and frequency characteristics in the surrounding environment in real time, and identify abnormal sounds such as breaking glass and screaming by analyzing the sound patterns.

[0016] Preferably, the image recognition algorithm uses deep learning technology to analyze images and videos captured by high-definition cameras, and can accurately identify the identity of people, their behavior (such as walking, running, climbing, etc.), and determine whether there is any abnormal behavior;

[0017] The multimodal data fusion algorithm fuses the data collected by infrared sensors, smoke sensors, gas sensors, and sound sensors, and improves the accuracy of security event judgment by comprehensively analyzing multiple sensory information. For example, when the smoke sensor detects smoke and the sound of breaking glass is detected by the sound sensor at the same time, the system can more accurately judge it as a fire or intrusion event.

[0018] The behavioral analysis model is used to establish a personnel behavior analysis model based on a large amount of historical data and machine learning algorithms, to learn and analyze people's daily activities and behavior patterns, and to issue timely warnings when abnormal behavior occurs.

[0019] Preferably, the wireless communication unit supports multiple wireless communication protocols such as Wi-Fi, Bluetooth, and ZigBee, and realizes data transmission between the multimodal perception module, the intelligent analysis module and the cloud server, ensuring the stability and real-time performance of data transmission;

[0020] The data encryption unit uses a combination of symmetric encryption and asymmetric encryption to encrypt the transmitted data, thereby ensuring the security and privacy of the data during transmission.

[0021] Preferably, the data storage unit has a large-capacity storage space and adopts distributed storage technology to store historical data collected by the multimodal perception module, analysis results of the intelligent analysis module and user-related information, thereby improving data reliability and reading and writing efficiency;

[0022] The intelligent decision-making unit makes intelligent decisions based on the data uploaded by the intelligent analysis module and the preset security rules. When it determines that a security incident has occurred, it promptly sends an alarm message to the user and provides corresponding handling suggestions. At the same time, it can dynamically adjust and optimize the security policy.

[0023] The user management unit is responsible for user registration, login and permission management functions. Users can log in to the cloud server through the mobile APP or web page to view security monitoring information, set security rules and receive alarm notifications.

[0024] Preferably, the mobile phone APP provides an intuitive user interface, allowing users to check the security status of smart homes anytime and anywhere. Users can receive alarm notifications, view real-time video, and control device switches. At the same time, it supports users to customize security rules and receive personalized security suggestions;

[0025] The web-based management interface provides users with more comprehensive management functions, such as viewing detailed historical data reports, performing system configuration and maintenance, etc.

[0026] Preferably, the centralized control unit is used for users to centrally control various devices in the smart home system through the central control screen, including lights, air conditioners, curtains, and door locks. The status of the devices can be intuitively viewed on the central control screen, and the devices can be turned on and off and adjusted through touch operations or voice commands;

[0027] The security monitoring display unit is used to display in real time the security monitoring information collected by the multimodal perception module, such as camera images, smoke alarm status, gas leak alarm status, etc. When a security incident occurs, the central control screen will automatically pop up an alarm prompt and display detailed security incident information and handling suggestions;

[0028] The scene mode switching unit supports the setting and switching of multiple scene modes, such as home mode, away mode, sleep mode, etc. Users can switch scene modes with one click on the central control screen, and the system will automatically adjust the status of smart home devices to meet the needs of different scenarios;

[0029] The personalized customization unit is used for users to customize the interface layout, theme color, and voice prompts of the central control screen according to their own preferences and needs, and create their own smart home control center.

[0030] The present invention also proposes a method for smart home security monitoring based on multimodal perception and intelligent analysis, comprising the following steps:

[0031] S1: Data acquisition: Each sensor in the multimodal perception module collects environmental information in real time, including images, videos, infrared signals, smoke concentration, gas concentration, and sound data, and transmits the collected data to the intelligent analysis module;

[0032] S2: Data processing and analysis: After receiving the data, the intelligent analysis module first uses the image recognition algorithm to perform image recognition and analysis on the image and video data to determine whether there are any abnormal people or behaviors. For example, when the camera detects a moving object, it determines whether it is a stranger. If it is, an alarm is triggered; otherwise, continuous monitoring is initiated. At the same time, the multimodal data fusion algorithm is used to fuse and analyze the data collected by other sensors to comprehensively determine whether there is a security incident. For example, when the smoke sensor detects an increase in smoke concentration and the sound sensor detects an abnormal sound, the intelligent analysis module will combine the image recognition results to determine whether a fire has occurred.

[0033] S3: Intelligent Decision-Making and Alarming: The intelligent analysis module transmits the analysis results to the cloud server. The cloud server's intelligent decision-making unit makes a judgment based on preset security rules. If a security incident is detected, the cloud server immediately sends an alarm to the user terminal and displays detailed security incident information and handling suggestions on the user terminal. Users can remotely control smart home devices through the mobile phone app or the smart home central control module, such as turning on lights, closing doors and windows, and activating alarms.

[0034] S4: Data storage and update: The data storage unit of the cloud server stores the data collected in S1 and the analysis results in S2 and S3 for subsequent query and analysis. At the same time, the intelligent analysis module continuously updates the behavior analysis model and security rules based on new data and user feedback to improve the accuracy and reliability of the system.

[0035] S5: User interactive operation: Users can perform various operations on the mobile phone APP or the central control screen, such as setting safety rules, switching scene modes, controlling device switches, etc. The smart home central control screen module will send control signals to the corresponding devices through the communication interface according to the user's operation instructions, and provide real-time feedback on the status information of the device. At the same time, the central control screen will also record the user's operation history for user query and statistics.

[0036] Compared with the existing technology, the beneficial effects of the present invention are:

[0037] 1. By integrating and processing data from multiple sensing devices such as high-definition cameras, infrared sensors, smoke sensors, gas sensors, and sound sensors, it achieves comprehensive and multi-level perception of the smart home environment. Compared with traditional single-sensor monitoring systems, it can more accurately detect and identify various security events, thereby improving system reliability and security.

[0038] 2. Using deep learning and machine learning algorithms, it conducts intelligent analysis and behavior recognition of multimodal perception data, accurately determining the identity and behavior of people, identifying abnormal behavior and issuing timely warnings. By learning and analyzing large amounts of historical data, it continuously optimizes the behavior analysis model and improves the intelligence level of the system.

[0039] 3. Using multiple wireless communication protocols, real-time data transmission between the multimodal perception module, intelligent analysis module and cloud server is realized. Users can check the security status of smart home anytime and anywhere through the mobile phone APP or web management interface, and remotely control the equipment. At the same time, the data encryption unit ensures the security and privacy of data transmission.

[0040] Through multimodal perception fusion and intelligent analysis technology, the present invention can achieve all-round and multi-level perception of the smart home environment. Compared with the traditional single-sensor monitoring system, it can more accurately detect and identify various security events, improve the reliability and security of the system, and at the same time accurately judge the identity and behavior of people, identify abnormal behavior and issue early warnings in time to reduce potential risks. In addition, by adopting multiple wireless communication protocols, real-time data transmission between the multimodal perception module, the intelligent analysis module and the cloud server is realized. The data encryption unit is combined with the transmission process to ensure the security and privacy of data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a block diagram of the smart home security monitoring system based on multimodal perception and intelligent analysis proposed by the present invention;

[0042] Figure 2 This is a flow chart of the smart home security monitoring method based on multimodal perception and intelligent analysis proposed by the present invention;

[0043] Figure 3 This is a logic diagram for smoke detection and judgment in the smart home security monitoring method based on multimodal perception and intelligent analysis proposed by the present invention;

[0044] Figure 4 This is a logic diagram for judging strangers in the smart home security monitoring method based on multimodal perception and intelligent analysis proposed by the present invention;

[0045] Figure 5 This is a flowchart of the cloud server performing intelligent decision-making and alarming in the smart home security monitoring method based on multimodal perception and intelligent analysis proposed by the present invention. DETAILED DESCRIPTION

[0046] The present invention will be further explained below with reference to specific embodiments.

[0047] Example

[0048] Reference Figure 1-5 ,This embodiment proposes a smart home security monitoring system based on multimodal perception and intelligent analysis, including a multimodal perception module, an intelligent analysis module, a communication module, a cloud server, a user terminal and a smart home central control screen module;

[0049] The multimodal perception module includes a high-definition camera, infrared sensor, smoke sensor, gas sensor, and sound sensor;

[0050] The high-definition camera has high resolution, night vision and wide-angle shooting capabilities, which can clearly capture images and videos of indoor and outdoor environments. It supports intelligent image analysis algorithms and can identify and track people and objects in the picture.

[0051] Infrared sensors are used to detect infrared radiation from the human body. They can accurately sense human activity at night or in dimly lit environments. Different sensing sensitivities can be set to suit various scenarios.

[0052] The smoke sensor uses advanced electrochemical or photoelectric sensing technology to quickly and accurately detect changes in smoke concentration and issue an alarm signal in a timely manner when the smoke concentration exceeds the set threshold.

[0053] Gas sensors are specifically used to detect gas leaks, such as natural gas and liquefied petroleum gas. Through highly sensitive sensor elements, they can detect tiny changes in gas concentration in the early stages of gas leaks and issue an alarm.

[0054] Sound sensors are used to monitor the sound intensity and frequency characteristics of the surrounding environment in real time. By analyzing the sound patterns, they can identify abnormal sounds such as breaking glass and screaming.

[0055] The intelligent analysis module has built-in image recognition algorithms, multimodal data fusion algorithms and behavior analysis models;

[0056] The image recognition algorithm uses deep learning technology to analyze images and videos captured by high-definition cameras. It can accurately identify people's identities and actions (such as walking, running, climbing, etc.) and determine whether there are any abnormal behaviors.

[0057] The multimodal data fusion algorithm integrates and processes data collected by infrared sensors, smoke sensors, gas sensors, and sound sensors. By comprehensively analyzing multiple sensory information, it improves the accuracy of security event judgments. For example, when the smoke sensor detects smoke and the sound of breaking glass is detected by the sound sensor, the system can more accurately determine whether it is a fire or intrusion incident.

[0058] The behavioral analysis model is used to establish a personnel behavior analysis model based on a large amount of historical data and machine learning algorithms. It studies and analyzes personnel's daily activities and behavior patterns and issues timely warnings when abnormal behavior occurs.

[0059] The communication module includes a wireless communication unit and a data encryption unit;

[0060] The wireless communication unit supports multiple wireless communication protocols such as Wi-Fi, Bluetooth, and ZigBee, enabling data transmission between the multimodal perception module, intelligent analysis module, and cloud servers, ensuring the stability and real-time performance of data transmission.

[0061] The data encryption unit uses a combination of symmetric and asymmetric encryption to encrypt the transmitted data to ensure the security and privacy of the data during transmission;

[0062] The cloud server includes a data storage unit, an intelligent decision-making unit, and a user management unit;

[0063] The data storage unit has a large storage space and uses distributed storage technology to store historical data collected by the multimodal perception module, analysis results of the intelligent analysis module, and user-related information, thereby improving data reliability and reading and writing efficiency;

[0064] The intelligent decision-making unit makes intelligent decisions based on the data uploaded by the intelligent analysis module and the preset security rules. When it determines that a security incident has occurred, it promptly sends an alarm message to the user and provides corresponding handling suggestions. At the same time, it can dynamically adjust and optimize the security policy.

[0065] The user management unit is responsible for user registration, login, and permission management. Users can log in to the cloud server through the mobile app or web page to view security monitoring information, set security rules, and receive alarm notifications;

[0066] User terminals include mobile APP and Web management interface;

[0067] The mobile app provides an intuitive user interface, allowing users to check the security status of their smart home anytime, anywhere. Users can receive alarm notifications, view real-time video, and control device switches. At the same time, users can customize security rules and receive personalized security recommendations.

[0068] The web-based management interface provides users with more comprehensive management functions, such as viewing detailed historical data reports, performing system configuration and maintenance, etc.

[0069] The smart home central control screen module is built into the external central control screen. The smart home central control screen module includes a centralized control unit, a security monitoring display unit, a scene mode switching unit, and a personalized customization unit;

[0070] The centralized control unit is used to centrally control various devices in the smart home system, including lights, air conditioners, curtains, and door locks, through the central control screen. The user can intuitively view the status of the devices on the central control screen and turn them on and off and adjust them through touch or voice commands.

[0071] The security monitoring display unit is used to display in real time the security monitoring information collected by the multimodal perception module, such as camera images, smoke alarm status, gas leak alarm status, etc. When a security incident occurs, the central control screen will automatically pop up an alarm prompt and display detailed security incident information and handling suggestions;

[0072] The scene mode switching unit supports the setting and switching of multiple scene modes, such as home mode, away mode, sleep mode, etc. Users can switch scene modes with one click on the central control screen, and the system will automatically adjust the status of smart home devices to meet the needs of different scenarios;

[0073] The personalized customization unit allows users to customize the interface layout, theme color, and voice prompts of the central control screen according to their own preferences and needs, creating their own smart home control center;

[0074] This embodiment also proposes a method for smart home security monitoring based on multimodal perception and intelligent analysis, which includes the following steps:

[0075] S1: Data acquisition: Each sensor in the multimodal perception module collects environmental information in real time, including images, videos, infrared signals, smoke concentration, gas concentration, and sound data, and transmits the collected data to the intelligent analysis module;

[0076] S2: Data processing and analysis: After receiving the data, the intelligent analysis module first uses the image recognition algorithm to perform image recognition and analysis on the image and video data to determine whether there are any abnormal people or behaviors. For example, when the camera detects a moving object, it determines whether it is a stranger. If it is, an alarm is triggered; otherwise, continuous monitoring is initiated. At the same time, the multimodal data fusion algorithm is used to fuse and analyze the data collected by other sensors to comprehensively determine whether there is a security incident. For example, when the smoke sensor detects an increase in smoke concentration and the sound sensor detects an abnormal sound, the intelligent analysis module will combine the image recognition results to determine whether a fire has occurred.

[0077] S3: Intelligent Decision-Making and Alarming: The intelligent analysis module transmits the analysis results to the cloud server. The cloud server's intelligent decision-making unit makes a judgment based on preset security rules. If a security incident is detected, the cloud server immediately sends an alarm to the user terminal and displays detailed security incident information and handling suggestions on the user terminal. Users can remotely control smart home devices through the mobile phone app or the smart home central control module, such as turning on lights, closing doors and windows, and activating alarms.

[0078] S4: Data storage and update: The data storage unit of the cloud server stores the data collected in S1 and the analysis results in S2 and S3 for subsequent query and analysis. At the same time, the intelligent analysis module continuously updates the behavior analysis model and security rules based on new data and user feedback to improve the accuracy and reliability of the system.

[0079] S5: User interaction: Users can perform various operations on the mobile phone app or the central control screen, such as setting security rules, switching scene modes, and controlling device switches. The smart home central control screen module will send control signals to the corresponding devices through the communication interface according to the user's operation instructions, and provide real-time feedback on the device status information. At the same time, the central control screen will also record the user's operation history for user query and statistics;

[0080] This embodiment uses multimodal perception fusion and intelligent analysis technology to achieve all-round, multi-level perception of the smart home environment. Compared with traditional single-sensor monitoring systems, it can more accurately detect and identify various security events, improve the reliability and security of the system, and accurately determine the identity and behavior of people, identify abnormal behavior and issue early warnings in a timely manner, thereby reducing potential risks. In addition, by adopting multiple wireless communication protocols, real-time data transmission between the multimodal perception module, the intelligent analysis module and the cloud server is achieved. The data encryption unit is combined during the transmission process to ensure the security and privacy of data transmission.

[0081] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. Smart home security monitoring system based on multimodal perception and intelligent analysis, characterized by: It includes multimodal perception module, intelligent analysis module, communication module, cloud server, user terminal and smart home central control screen module; The multimodal perception module includes a high-definition camera, an infrared sensor, a smoke sensor, a gas sensor, and a sound sensor; The intelligent analysis module has built-in image recognition algorithm, multimodal data fusion algorithm and behavior analysis model; The communication module includes a wireless communication unit and a data encryption unit; The cloud server includes a data storage unit, an intelligent decision-making unit and a user management unit; The user terminal includes a mobile phone APP and a Web-based management interface; The smart home central control screen module is built into the external central control screen. The smart home central control screen module includes a centralized control unit, a security monitoring display unit, a scene mode switching unit and a personalized customization unit.

2. The smart home security monitoring system based on multimodal perception and intelligent analysis according to claim 1 is characterized in that: The HD camera has high resolution, night vision and wide-angle shooting capabilities, can clearly capture images and videos of indoor and outdoor environments, and supports intelligent image analysis algorithms to identify and track people and objects in the picture; The infrared sensor is used to detect infrared radiation from the human body and can accurately sense human activities at night or in dimly lit environments. Different sensing sensitivities can be set to suit various scenarios. The smoke sensor uses advanced electrochemical or photoelectric sensing technology to quickly and accurately detect changes in smoke concentration and promptly issues an alarm signal when the smoke concentration exceeds a set threshold. The gas sensor is specially used to detect gas leaks. Through the highly sensitive sensor element, it can detect tiny changes in gas concentration in the early stages of gas leaks and issue an alarm. The sound sensor is used to monitor the sound intensity and frequency characteristics in the surrounding environment in real time, and identify abnormal sounds by analyzing the sound patterns.

3. The smart home security monitoring system based on multimodal perception and intelligent analysis according to claim 1 is characterized in that: The image recognition algorithm uses deep learning technology to analyze images and videos collected by high-definition cameras, and can accurately identify the identity and behavior of people and determine whether there is any abnormal behavior; The multimodal data fusion algorithm fuses the data collected by infrared sensors, smoke sensors, gas sensors and sound sensors, and improves the accuracy of security event judgment by comprehensively analyzing multiple sensory information; The behavioral analysis model is used to establish a personnel behavior analysis model based on a large amount of historical data and machine learning algorithms, to learn and analyze people's daily activities and behavior patterns, and to issue timely warnings when abnormal behavior occurs.

4. The smart home security monitoring system based on multimodal perception and intelligent analysis according to claim 1 is characterized in that: The wireless communication unit supports multiple wireless communication protocols such as Wi-Fi, Bluetooth, and ZigBee, and realizes data transmission between the multimodal perception module, the intelligent analysis module, and the cloud server, ensuring the stability and real-time performance of data transmission; The data encryption unit uses a combination of symmetric encryption and asymmetric encryption to encrypt the transmitted data, thereby ensuring the security and privacy of the data during transmission.

5. The smart home security monitoring system based on multimodal perception and intelligent analysis according to claim 1 is characterized in that: The data storage unit has a large storage space and adopts distributed storage technology to store historical data collected by the multimodal perception module, analysis results of the intelligent analysis module and user-related information, thereby improving data reliability and reading and writing efficiency; The intelligent decision-making unit makes intelligent decisions based on the data uploaded by the intelligent analysis module and the preset security rules. When it determines that a security incident has occurred, it promptly sends an alarm message to the user and provides corresponding handling suggestions. At the same time, it can dynamically adjust and optimize the security policy. The user management unit is responsible for user registration, login and permission management functions. Users can log in to the cloud server through the mobile APP or web page to view security monitoring information, set security rules and receive alarm notifications.

6. The smart home security monitoring system based on multimodal perception and intelligent analysis according to claim 1 is characterized in that: The mobile app provides an intuitive user interface, allowing users to check the security status of their smart home anytime, anywhere. Users can receive alarm notifications, view real-time video, and control device switches. At the same time, users can customize security rules and receive personalized security recommendations. The web management interface provides users with more comprehensive management functions.

7. The smart home security monitoring system based on multimodal perception and intelligent analysis according to claim 1 is characterized in that: The centralized control unit is used for users to centrally control various devices in the smart home system through the central control screen, including lights, air conditioners, curtains, and door locks. The status of the devices can be intuitively viewed on the central control screen, and the devices can be turned on and off and adjusted through touch operations or voice commands; The security monitoring display unit is used to display the security monitoring information collected by the multimodal perception module in real time. When a security incident occurs, the central control screen will automatically pop up an alarm prompt and display detailed security incident information and handling suggestions; The scene mode switching unit supports the setting and switching of multiple scene modes. Users can switch scene modes with one click on the central control screen, and the system will automatically adjust the status of smart home devices to meet the needs of different scenarios; The personalized customization unit is used for users to customize the interface layout, theme color, and voice prompts of the central control screen according to their own preferences and needs, and create their own smart home control center.

8. The method for smart home security monitoring based on multimodal perception and intelligent analysis according to any of claims 1 to 7, characterized in that: The following steps are involved: S1: Data acquisition: Each sensor in the multimodal perception module collects environmental information in real time, including images, videos, infrared signals, smoke concentration, gas concentration, and sound data, and transmits the collected data to the intelligent analysis module; S2: Data processing and analysis: After receiving the data, the intelligent analysis module first uses the image recognition algorithm to identify and analyze the image and video data to determine whether there are any abnormal people or behaviors. At the same time, it uses the multimodal data fusion algorithm to integrate and analyze the data collected by other sensors to comprehensively determine whether there is a security incident. S3: Intelligent Decision-Making and Alarming: The intelligent analysis module transmits the analysis results to the cloud server. The cloud server's intelligent decision-making unit makes a judgment based on preset security rules. If a security incident is detected, the cloud server immediately sends an alarm to the user terminal and displays detailed security incident information and handling suggestions on the user terminal. Users can remotely control smart home devices through the mobile phone app or the smart home central control module; S4: Data storage and update: The data storage unit of the cloud server stores the data collected in S1 and the analysis results in S2 and S3 for subsequent query and analysis. At the same time, the intelligent analysis module continuously updates the behavior analysis model and security rules based on new data and user feedback to improve the accuracy and reliability of the system. S5: User interactive operation: Users can perform various operations on the mobile phone APP or the central control screen. The smart home central control screen module will send control signals to the corresponding devices through the communication interface according to the user's operation instructions, and provide real-time feedback on the status information of the device. At the same time, the central control screen will also record the user's operation history for user query and statistics.

Images