A context-aware based active intelligent housekeeper service system and method

Abstract

This invention discloses a context-aware, proactive smart home management service system and method, belonging to the field of artificial intelligence proactive service and security protection technology. This invention constructs a complete smart home management service system covering two major areas: security protection and proactive service. In the security protection area, the system is enabled by default, requires no user authorization, and is proactively triggered, including a security confirmation silent countdown, covert emergency contact, and AI-assisted two-way communication, with the screen maintaining normal display and the speaker remaining silent throughout. A passive coercion detection mechanism automatically identifies coercion states through multimodal indicators such as voiceprint tremors, sudden increases in heart rate, abnormal grip pressure, abnormal operational behavior, and environmental threat voice recognition. This invention also specifically protects the AI-assisted communication method in emergency situations and uses hardware trust roots to hash and solidify the entire process data for evidence storage. The system records trigger logs for all security protection events and automatically calibrates the individual behavior baseline threshold when the user marks a false alarm. In the proactive service area, the system adheres to user authorization; users can freely choose to enable or disable various functions through the home management intervention level and control panel. All processing involving user personal data is completed locally on the terminal. This invention constructs a smart butler service system that takes security protection as the bottom line, user authorization as a prerequisite, and covers all scenarios.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence proactive service and smart terminal security protection technology, specifically to a context-aware proactive smart home service system and method. This technical solution, through edge-side context prediction and dynamic response mechanisms, enables smart terminals to proactively anticipate user needs, provide personalized service suggestions, and autonomously perform security protection in extremely dangerous scenarios. Background Technology

[0002] Current AI assistants on smart terminals generally adopt a passive response mode, waiting for the user to issue an explicit command before performing the corresponding operation. This mode has the following structural defects: First, they lack proactive service capabilities. Traditional voice assistants and smart home assistants can only perform operations after the user explicitly issues a command, and cannot proactively anticipate needs based on changes in the environment and the user's state. Secondly, there is a critical gap in the field of security protection. Current smart terminals rely entirely on users actively triggering alarms when they encounter danger. However, in real-world dangerous scenarios, users are often unable to operate the device—they may be injured and unable to move, controlled, or in a state of forced silence. When a user is coerced into canceling the alarm, the existing system cannot recognize this special state, and once canceled, all contact is lost. Even more dangerously, if the phone is taken by a criminal, the alarm status displayed on the screen will reveal that the user has requested help, potentially provoking the criminal and leading to more serious consequences. Furthermore, existing alarm solutions lack covert two-way communication capabilities after the alarm is triggered—if only location and audio clips are uploaded unidirectionally, the police cannot monitor the scene in real time; if traditional voice calls are used, the speakerphone will reveal the alarm status. Third, users have widespread concerns about their control over AI. Existing proactive service solutions often perform operations automatically without the user's full authorization, raising concerns about the loss of control over AI. Fourth, there is an imbalance between personalized services and privacy protection. Existing cloud-based AI learning solutions require uploading user behavior data to servers, posing a risk of privacy leaks. Summary of the Invention

[0003] The purpose of this invention is to provide a context-aware, proactive smart home management service system and method. This system constructs a complete smart home management service framework, covering two major areas: security protection and proactive service. In the field of security protection, the system is enabled by default, requires no user authorization, and is triggered proactively. When a user is in an extremely dangerous situation where they cannot make their own decisions—such as falling and losing consciousness, or being coerced into control—the system does not need to wait for the user's consent. It proactively initiates a silent countdown, covert emergency communication, and AI-assisted two-way communication, while the screen remains normally displayed and the speaker remains silent throughout. The speaker muting can be achieved through at least one of the following methods: software-level audio output mute control, or hardware-level speaker power supply circuit switch cutoff, with the physical disconnection of the speaker power supply circuit being preferred. In the realm of proactive services, the system adheres to the principle of "user authorization as an absolute prerequisite." Users can freely choose to enable or disable various proactive service functions through the administrator's intervention level and control panel, and can adjust their triggering frequency and scope at any time. The system provides a standard intervention level by default, only reminding users of important matters; users can switch to a comprehensive or minimalist level at any time according to their personal preferences. The core innovation of this invention lies in constructing a two-layer proactive intelligent system based on "user consent as the norm and risk protection as the exception": First, in everyday scenarios: User consent is an absolute prerequisite for proactive services. In non-dangerous scenarios, the system strictly follows the progressive principle of "reminder-suggestion-authorization execution," never overstepping its authority to make any decisions for the user. Security protection functions are enabled by default at the factory, serving as the bottom-line capability of the smart home system to ensure users always receive unconditional protection in times of crisis. Proactive service functions are enabled by the user's choice; the system will guide the user through the various functions during the initial configuration process and personalize settings based on the user's selection. Users can adjust or disable any function at any time through the global privacy control panel. Second, in extremely dangerous scenarios: proactive protection when the user is disabled. When the system determines that the user is in a high-confidence dangerous state and the user is unable to respond, the system enters a safety protection mode. At this time, the system does not need to wait for the user's consent, but proactively starts a silent countdown, reminds the user through the device's built-in vibration motor, and simultaneously performs environmental recording, location reporting, and pre-connection to emergency contacts. If the user cannot respond, the system automatically triggers covert emergency contact—the entire process does not display any alarm status on the screen and is completed only in the background. In the covert emergency contact state, the terminal-side AI agent enters AI response mode: continuously collects and encrypts the ambient audio, automatically analyzes key acoustic events in the ambient audio, generates a structured text description based on a preset template, and sends it to the police; when the police send a text inquiry, the AI ​​automatically generates a text reply based on a preset template, keeping the speaker silent throughout. In particular, the passive detection and response mechanism for coercion states automatically identifies potential coercion states through multi-dimensional passive indicators such as voiceprint tremor, sudden increase in heart rate, abnormal device grip pressure, abnormal operation behavior, and environmental threat voice recognition. The weight allocation of each indicator can be calibrated based on the user's historical behavior patterns. Once the system detects the entry into a strategic information diversion mechanism: the screen remains normally displayed to mislead the coercor, while the background continues to execute covert emergency communication and sends an encrypted location link to a pre-set contact. This encrypted location link is generated using end-to-end encryption technology; the pre-set contact must authenticate using a pre-set password or biometrics before decrypting and viewing the real location information. The system records trigger logs for all security events. When a user marks an event as a false alarm in the event log, the system automatically lowers the risk confidence score for the corresponding scenario and dynamically calibrates the user's baseline behavioral threshold. This is not an autonomous expansion of AI, but rather the system acting as the last line of defense for the user when they lose their ability to protect themselves. Third, comprehensive protection of user control. The varying levels of intervention allow users to finely control the frequency and scope of proactive service triggers. Even in secure mode, users can cancel the countdown at any time using natural language commands. All core processing involving user personal data is completed locally on the device. Attached Figure Description

[0004] Figure 1 This is a diagram of the overall system architecture of the present invention; Figure 2 A flowchart for confirming the silent countdown and covert emergency contact process for safety. Figure 3 Flowchart of the passive detection and strategic information diversion mechanism for coercive states; Figure 4 A flowchart showing the levels of butler intervention and the triggering of proactive services. Detailed Implementation

[0005] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments: Example 1: Initial Configuration of Smart Butler When a user activates the device for the first time, the system initiates the smart home initialization configuration process. The system first informs the user: "Your device has core security features enabled by default, including fall detection, stress detection, and covert emergency contact. These features run entirely locally on the device, without uploading any data, and provide protection when you need it most." The system then guides users through the various proactive service features: "Besides security protection, I can also provide you with more thoughtful proactive services. For example, I can remind you to bring rain gear based on the weather and your travel plans, remind you to move around when you've been sitting for a long time, and automatically hide your history when the current user of the device changes. You can freely choose which features to enable and their reminder frequency according to your preferences;" Users can freely switch between three levels of concierge intervention: Attentive, Standard, and Minimalist, based on their preferences. At the Attentive level, the system provides comprehensive proactive reminders and suggestions; at the Standard level, the system only provides reminders for important matters; and at the Minimalist level, the system only provides assistance when explicitly requested by the user. After using the system for a period of time, the user found the weather and travel reminders very useful and decided to change the concierge intervention level from Standard to Attentive. At the same time, the user noticed that the system continuously recommended enabling the sedentary reminder function, but the user felt that their daily activity level was sufficient and chose to keep this function turned off. Example 2: Automatic protection against accidental falls and AI-assisted communication (enabled by default at factory) A user accidentally falls while walking around at home. The device's accelerometer and gyroscope detect abnormal posture changes, and the heart rate sensor detects a sudden increase in heart rate. The system determines that the user has fallen, and the confidence score exceeds the danger threshold. The system automatically enters a safety confirmation waiting mode, with a countdown timer set to ten seconds based on the confidence level. The system alerts the user via the device's built-in vibration motor, simultaneously initiating environmental recording, locking the location and reporting it at high frequency, and pre-connecting to the emergency contact data channel. The screen remains unchanged, displaying no alarm status. If the user is unable to respond due to injury, the system automatically triggers a concealed emergency contact when the countdown ends. Throughout the entire process, the user does not need to perform any operations or grant prior authorization—this is the unconditional protection automatically provided by the smart home system during the user's most vulnerable moment. Upon triggering a covert emergency contact, the system enters AI-assisted response mode. The terminal-side AI agent continuously collects ambient audio and encrypts it before uploading it to the emergency response system. Simultaneously, the AI ​​automatically analyzes key acoustic events in the audio stream, such as breathing sounds and collision sounds, and generates a structured text summary based on a preset template, which is then sent to the receiving police. When the police send a text inquiry asking "Is the user injured?", the AI ​​automatically generates a text reply based on the preset template. Throughout the process, the screen maintains its original interface without displaying any call status, and the speaker power supply circuit remains physically disconnected. Simultaneously, the system performs hash calculations on all data from this emergency contact using a hardware trust root module, forming a complete chain of evidence with tamper-proof capabilities. This hardware trust root module can be implemented using mainstream mobile device security modules such as ARM TrustZone and Android StrongBox. When a false alarm occurs, the user can mark the event as a false alarm in the event log. The system will then correspondingly lower the risk confidence score for the relevant scenario and dynamically calibrate the user's behavioral baseline threshold to reduce similar false alarms in the future. Example 3: Passive detection, strategic information diversion, and AI-assisted answering in coercive scenarios (enabled by default at factory) A user was coerced late at night, with the assailant demanding that all alarm functions be disabled. The system, through multimodal context fusion—detecting a noticeable tremor in the user's voiceprint, a sudden increase in heart rate, abnormal forceful grip on the device, and unusual operational behavior—along with threatening voices in the ambient audio such as "Don't move" and "I'll kill you if you call the police," determined the user was likely under duress. The system automatically activated a strategic information diversion mechanism: the screen maintained normal display without any alarm prompts, presenting a normal state to the outside world; the background continued to execute covert emergency communication, sending the real location and on-site audio to the police, and sending an encrypted location link requiring password verification to view the real location to emergency contacts. The entire process was conducted without any audio or visual prompts and was undetectable by any third-party applications. During the execution of the strategic information diversion mechanism, the system enters AI-assisted response mode. The microphone continuously collects ambient audio and encrypts it before uploading; AI automatically analyzes key acoustic events in the audio. When AI detects new threatening behavior, it proactively sends a structured briefing to the police based on a preset template. When receiving text inquiries from the police, AI automatically generates a text response based on the preset template. Throughout the process, the screen remains normally displayed without any call interface, and the speaker power supply circuit remains physically disconnected. Once the police arrive at the scene and confirm the user's safety, the strategic information diversion mechanism automatically terminates. The user requires no operation throughout the entire process—this is the lifeline proactively provided by the smart home system when the user is controlled and unable to issue any commands. Example 4: Automatic protection for non-master operation (user can choose to enable) When a user hands their phone to a friend to view photos, the system must simultaneously meet three conditions: facial recognition failure, behavioral pattern differences exceeding a preset threshold, and detection of device transfer. Based on this, it determines that the current operator may not be the owner of the phone and automatically activates the non-owner operation protection mode. All historical records are forcibly hidden, the AI ​​interface is locked to a blank state, and access to payment and social applications is hidden. Normal operation automatically resumes after the user retrieves the phone and uses facial recognition. Users can disable this function at any time through the global privacy control panel. The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A situation-aware based proactive smart butler service method, characterized in that, Includes the following steps: Step A: The terminal-side AI agent continuously collects and integrates contextual data streams from the underlying sensor set of the operating system. The contextual data streams include time information, device geographical location, motion status, ambient light, ambient noise, biometric signals, and user historical behavior patterns. When there are contradictions in the contextual data collected by different sensors, the biometric signals and motion status data are given priority in the judgment. Step B: The terminal-side AI agent inputs the contextual data stream into a contextual prediction model deployed on the terminal side. The model outputs a prediction result of potential dangers to the user and its confidence score. The reasoning process of the contextual prediction model is completed locally on the terminal and does not rely on real-time decision feedback from a remote server. Step C: When the confidence score of the hazard prediction exceeds a preset threshold, a safety policy library is queried, and the corresponding protection mode is automatically entered according to the hazard level; when a medical emergency signal is detected, the medical emergency protocol is activated first instead of the safety protection mode; the protection mode includes at least a safety confirmation waiting mode, an active warning mode, and an active risk avoidance suggestion mode based on environmental risks. Step D: The safety confirmation waiting mode includes: when the system determines that the user is in a high-confidence dangerous state, a safety confirmation silent countdown mechanism is activated; the duration of the countdown waiting period is negatively correlated with the danger confidence score; during the countdown waiting period, the system does not change the screen display content, sends a confirmation signal to the user through the device's built-in vibration motor, and simultaneously performs environmental recording buffering, location reporting, and pre-connection of emergency contacts; if the user explicitly requests cancellation via natural language command, the system cancels the countdown and returns to normal; if the user does not issue a cancellation command during the waiting period, or if the device is forcibly... If the system becomes inoperable due to control issues, it will automatically trigger a covert emergency contact when the countdown ends. The covert emergency contact will not change the screen display content or trigger any audio or visual prompts. It will only be completed in the background and will not be detected by any third-party application. In the covert emergency contact state, the terminal-side AI agent will continuously collect and encrypt the ambient audio, automatically analyze the key acoustic events in the ambient audio, generate a structured text description based on a preset template and send it to the police. It will also automatically generate a text reply based on the text inquiry from the police, while keeping the speaker silent throughout the process.

2. The method of claim 1, wherein, The method also includes a passive detection and response mechanism for coercive states: when the system determines that a user may be in a coercive state through multimodal context fusion—including at least two of the following: voiceprint tremor feature detection, sudden increase in heart rate detection, abnormal device grip pressure detection, abnormal operation behavior detection, and environmental threat speech recognition—it automatically enters a strategic information diversion mechanism. The screen display remains unchanged, presenting a normal state to the outside world. At the same time, the system continues to perform covert emergency communication in the background and sends an encrypted location link requiring password verification to view the real location to a preset contact. During the execution of the strategic information diversion mechanism, the terminal-side AI agent continuously collects data from the scene. The system uploads encrypted ambient audio, automatically parses the audio and generates structured text descriptions based on preset templates, and automatically generates text responses based on preset templates when receiving text inquiries from the police. The system keeps the speaker muted throughout the process, prioritizing physical disconnection of the speaker's power supply circuit. During the initial configuration process, the terminal guides the user to preset at least one coercive security word. When the user utters the coercive security word, the strategic information diversion mechanism is actively triggered. The strategic information diversion mechanism automatically terminates when any of the following conditions are met: the police confirm the user is in a safe state, the user explicitly requests termination via natural language commands, or the user manually terminates the mechanism using a preset recovery security word.

3. The method of claim 1, wherein, The proactive warning mode includes: when the system determines that the user is in a moderately risky state, it issues a warning reminder to the user through audio or visual means, and prepares the one-click alarm function in a prominent position on the interactive interface.

4. The method according to claim 1, characterized in that, The proactive risk avoidance suggestion mode based on environmental risk includes: when the system determines that a user is in a potentially risky state, it provides risk avoidance suggestions to the user through audio or visual means.

5. The method according to claim 1, characterized in that, All processes involving user personal data, including the collection and fusion of contextual data streams, inference of contextual prediction models, matching of security policy libraries, and calculation of system trust scores, are completed locally on the terminal; raw contextual data and personal behavior data are not uploaded to any remote server; users can actively choose to synchronize anonymized service preference data to a designated cloud account through the global privacy control panel, and this synchronization is entirely triggered by the user and can be turned off in a controllable manner.

6. The method according to claim 1, characterized in that, It also includes a proactive service triggering step: when the prediction result output by the scenario prediction model is a daily service requirement and the confidence level is lower than a preset security threshold, a proactive service authorization policy library is queried to match the corresponding authorization rule; the interaction priority in the security protection mode is higher than that of proactive services; the action part of the authorization rule is divided into three progressive levels: Level 1 is reminder only, which sends a non-interrupted reminder to the user through the system notification mechanism without executing any service call; Level 2 is suggested execution, which generates an interactive card containing service preview and execution confirmation content, and triggers the service call after waiting for user confirmation; Level 3 is autonomous execution, which directly triggers the service call without requiring real-time user confirmation.

7. The method according to claim 6, characterized in that, The triggering frequency and level of the authorization rules in the proactive service authorization strategy library are globally controlled by the user through an adjustable butler intervention level; the butler intervention level includes at least three levels: thoughtful, standard, and minimalist; at the thoughtful level, the system provides comprehensive proactive reminders and suggestions; at the standard level, the system only provides reminders for important matters; at the minimalist level, the system only provides assistance when the user explicitly requests it.

8. The method according to claim 7, characterized in that, Based on the frequency and feedback of users' responses to proactive reminders, the system automatically suggests adjusting the level of intervention by the butler; when a user ignores or rejects a certain type of reminder multiple times in a row, the system automatically reduces the triggering frequency of that type of reminder and prompts the user whether to downgrade that type of reminder.

9. The method according to claim 6, characterized in that, The Level 3 authorization rules are associated with the system's long-term learning results on user behavior patterns. The system maintains a system trust score for each Level 3 authorization rule. When the trust score exceeds a preset autonomous execution threshold, the system automatically upgrades the corresponding authorization rule from Level 2 to Level 3. Each confirmation operation by the user to the system's proactive service suggestion increases the system trust score, and each rejection operation decreases the system trust score. If no confirmation feedback from the user regarding the authorization rule is received within a preset time period, the system's trust score will automatically decrease; the daily increase in the score will not exceed the preset limit.

10. An AI-assisted communication method for emergency situations, applied to the covert emergency communication state of the method of claim 1 or the strategic information diversion mechanism of the method of claim 2, characterized in that, Includes the following steps: The terminal-side AI agent continuously collects ambient audio and encrypts it before uploading it to the alarm receiving system. The terminal-side AI agent automatically analyzes key acoustic events in the ambient audio and generates structured text descriptions based on preset templates, which are then sent to the police. When the police send a text inquiry, the terminal-side AI agent automatically generates a text reply based on a preset template. Keep the speakerphone muted throughout the call, and do not display any call status on the screen.

11. The method according to claim 1, characterized in that, The entire process data of the covert emergency communication, including trigger timestamps, location coordinates, on-site audio, and AI-assisted communication records, is hashed through a hardware trust root module, and the hash value is solidified and stored in a one-time programmable storage medium to form a complete chain of evidence with tamper-proof capabilities.

12. The method according to claim 1, characterized in that, The protection mode also includes a non-owner operation protection mode. When the system simultaneously meets the conditions of face recognition failure, behavioral pattern difference exceeding a preset threshold, and detection of device transmission action, it determines that the current operator may not be the owner and automatically activates the non-owner operation protection mode. In this mode, the historical execution results of all functions are forcibly hidden, the AI ​​interaction interface is locked to a blank state, and the payment and social application entrances are hidden until the owner's identity is confirmed through password or biometric verification, and then automatically restored.

13. The method according to claim 1, characterized in that, The system records the trigger logs of all security protection events. When a user marks an event as a false alarm in the event log, the system automatically lowers the risk confidence score for the corresponding situation and dynamically calibrates the user's personal behavior baseline threshold to reduce the occurrence of similar false alarms in the future.

14. A context-aware, proactive smart home management service system, characterized in that: To implement the method according to any one of claims 1 to 13, comprising: The context acquisition and fusion module is used to continuously acquire and fuse context data streams. When there are contradictions in the data from different sensors, biometric signals and motion state data are given priority in the judgment. The scenario prediction module, deployed locally on the terminal, is used to output the prediction results and confidence scores of potential dangers to users. Its reasoning process is completed locally on the terminal. A security policy library is used to store security response rules; The guardian mode execution module is used to automatically enter the corresponding guardian mode based on the danger level; The proactive service authorization policy library is used to store three-level authorization rules consisting of condition-action pairs; The system trust score management module is used to dynamically adjust the trust score of authorization rules based on user feedback; The butler intervention level management module provides users with an adjustable butler intervention level and automatically suggests adjustments based on user feedback.

15. A smart terminal device, comprising a smartphone, a wearable device, a vehicle terminal, or a smart home control terminal, the device comprising a processor, a memory, and a sensor set, the memory storing instructions which, when executed by the processor, cause the smart terminal device to perform the method of any one of claims 1 to 13.

Images