Smart home service execution method and system

Abstract

This invention discloses a method and system for executing smart home services, relating to the field of smart home voice interaction and service execution technology. The method includes: acquiring voice information, view description information, and real-time user physiological state data; performing semantic recognition based on the view description information to obtain the operation intent; dynamically adjusting rejection parameters according to the view type and user state; performing asynchronous first semantic rejection and hiding the latency in the backbone link; performing synchronous second semantic rejection; matching the service adapter to generate an instruction sequence; and deciding whether to execute based on the rejection result and user state. This invention achieves scenario-adaptive collaboration between rejection strategies and service execution by integrating user physiological state and interface context, improving user experience and reducing response latency while ensuring security.

Description

Technical Field

[0001] This invention relates to the field of smart home voice interaction and service execution technology, specifically to a smart home service execution method and system. Background Technology

[0002] Existing smart home voice interaction systems face multiple technical bottlenecks when providing services. The high coupling between voice interaction functions and application business logic results in poor system scalability and difficulty in quickly integrating new devices or services. Semantic rejection schemes typically employ fixed strategies, failing to dynamically adjust the rejection severity based on the user's interface or operational context. This leads to overly strict rejection in scenarios requiring rapid response or overly lenient rejection in scenarios demanding high security. The intent recognition process relies solely on a single voice input signal, ignoring operable elements in the terminal's current display view, resulting in low recognition accuracy. The semantic rejection process is typically serial, leading to high end-to-end latency. More critically, existing solutions completely ignore the user's real-time physiological state. For example, a user's response needs and security requirements for voice commands differ drastically depending on whether they are sleeping, exercising, or in an emergency. The system cannot dynamically adjust service execution logic based on the user's state, resulting in false rejection of commands in emergency situations or accidental triggering of high-risk device operations when the user is resting. Summary of the Invention

[0003] This invention proposes a method for executing smart home services, comprising: S1. Obtain the user's voice information, voice command descriptions of voice-operable elements in the current display view of the terminal, and the user's real-time physiological status data. S2. Based on the view description information of the voice-operable element, perform semantic recognition on the voice information to obtain the user's operation intention; S3. Determine the type of the current view based on the view description information, and dynamically adjust the rejection parameters of semantic rejection based on the view type and the real-time physiological state data. S4. Based on the adjusted rejection parameters, perform asynchronous first semantic rejection on the voice request to obtain the first semantic rejection result; wherein, the asynchronous first semantic rejection is executed in parallel with the downstream logic processing of the voice request, and the delay of the first semantic rejection is hidden by the parallel execution. S5. After receiving the downstream logic processing result, perform synchronous second semantic rejection based on the downstream logic processing result and the first semantic rejection result to obtain the second semantic rejection result; S6. Match the target service adapter according to the operation intention and the preset extensible service registry, and convert the operation intention into an executable instruction sequence; S7. Based on the second semantic rejection result and the real-time physiological state data, decide whether to execute the instruction sequence.

[0004] Furthermore, the real-time physiological data includes heart rate, respiratory rate, body movement amplitude, and posture information.

[0005] Furthermore, the step of dynamically adjusting the rejection parameters of semantic rejection based on the view type and the real-time physiological state data includes: increasing the rejection threshold to a first preset value when the real-time physiological state data indicates that the user is in a sleep state; and decreasing the rejection threshold to a second preset value when the real-time physiological state data indicates that the user is in an emergency state.

[0006] Furthermore, the step of dynamically adjusting the rejection parameters of semantic rejection based on the view type and the real-time physiological state data also includes: selecting the corresponding semantic rejection model based on the combination of the view type and the real-time physiological state data, and dynamically loading and switching to the model for rejection judgment.

[0007] Further, the step of determining whether to execute the instruction sequence based on the second semantic rejection result and the real-time physiological state data includes: executing the instruction sequence when the second semantic rejection result indicates valid input and the real-time physiological state data indicates that the user is in a normal activity state; and generating a secondary confirmation request when the second semantic rejection result indicates valid input but the real-time physiological state data indicates that the user is in a high-risk state.

[0008] Furthermore, the step of performing semantic recognition on the speech information based on the view description information of the voice-operable element includes: predicting the acoustic features of the speech information using an acoustic model; and dynamically decoding the acoustic features based on the view description information using a language model to generate query text.

[0009] Further, the step of matching the target service adapter according to the operation intent and the preset extensible service registry includes: receiving a standardized abstract intent object, which contains an intent identifier and a parameter mapping table; matching the target service adapter according to the intent identifier and service priority based on the extensible service registry; and calling the adapter to convert the abstract intent object into an instruction sequence containing deep link template parameters.

[0010] Furthermore, a computer-readable storage medium is proposed, on which a computer program is stored, which, when executed by a processor, implements the aforementioned smart home service execution method.

[0011] Furthermore, a smart home service execution system is proposed, comprising: The multimodal perception module is configured to acquire the user's voice information, the voice command description information of the terminal's current display view, and the user's real-time physiological state data; The intent parsing module is configured to perform semantic recognition on the voice information based on the view description information to obtain the user's operation intent; The view type identification module is configured to determine the type of the current view based on the view description information. The rejection strategy adjustment module is configured to dynamically adjust the rejection parameters of semantic rejection based on the view type and the real-time physiological state data. The asynchronous rejection module is configured to perform asynchronous first semantic rejection of voice requests based on the adjusted rejection parameters. The synchronous rejection module is configured to perform synchronous second semantic rejection after receiving the downstream logic processing result, and obtain the second semantic rejection result. A service orchestration engine is configured to match target service adapters and generate instruction sequences based on the stated operational intent. The execution decision module is configured to determine whether to execute the instruction sequence based on the second semantic rejection result and the real-time physiological state data.

[0012] This invention achieves deep collaboration and scenario adaptation between semantic rejection strategies and service execution logic by integrating real-time user physiological state data and terminal view perception information. View perception provides precise scenario operation context, while user state perception provides physiological evidence of user intent. Both serve as inputs for adjusting the rejection strategy, enabling the strategy to accurately match user interaction needs under different physiological states. Simultaneously, service execution decisions also incorporate user state as a decision factor, ensuring that high-risk commands are automatically blocked or require secondary verification under specific user states, thereby improving user experience while ensuring security. An asynchronous first semantic rejection mechanism hides latency within the backbone link, significantly reducing end-to-end response latency. Through multi-dimensional perception and collaborative decision-making, this invention achieves a dynamic balance between user experience and system security while maintaining high rejection accuracy and significantly reducing system response latency. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating a smart home service execution method proposed in this invention. Detailed Implementation

[0014] refer to Figure 1 This invention proposes a method for executing smart home services, comprising: S1. Obtain the user's voice information, voice command descriptions of voice-operable elements in the current display view of the terminal, and the user's real-time physiological status data.

[0015] Specifically, the terminal is a mobile terminal in a home setting, such as a smart tablet or a home control screen. The real-time physiological status data includes the user's heart rate, respiratory rate, body movement amplitude, and posture information, such as standing, lying down, walking, or falling. The voice command description information includes a list of voice commands and configuration information for each voice command.

[0016] S2. Based on the view description information of the voice-operable element, perform semantic recognition on the voice information to obtain the user's operation intention.

[0017] Specifically, a pre-trained acoustic model is used to predict the acoustic features of the audio signal of the speech information; a pre-trained language model is then used to dynamically decode the acoustic features based on the view description information to generate the corresponding query text. For example, in a movie recommendation view, if a user says "play this," the system dynamically decodes the currently highlighted movie element in the view to generate the query text "play Interstellar." The view description information directly defines the directionality of speech recognition, eliminating the inherent defect that the system cannot determine the referent when the user's expression is ambiguous in an open domain. View perception and speech recognition mutually reinforce each other, jointly solving the problem of low accuracy in intent recognition under a single modality. This is the first collaboration between view features and semantic recognition technology features.

[0018] S3. Determine the type of the current view based on the view description information, and dynamically adjust the rejection parameters of semantic rejection based on the view type and the real-time physiological state data.

[0019] Specifically, the rejection parameters include a rejection threshold and semantic rejection model parameters. The view type is determined by analyzing the view structure identifier or view name in the view description information; for example, if it contains a VideoPlayerView identifier, it is a video playback type, and if it contains a PaymentView identifier, it is a payment type. The dynamic adjustment rules are as follows: when the real-time physiological state data indicates that the user is asleep, regardless of the view type, the rejection threshold is increased to a first preset value, such as 0.95, to avoid false triggering; when the real-time physiological state data indicates that the user is in an emergency, such as a sudden increase in heart rate, abnormal body movement amplitude, or a detected fall posture, the rejection threshold is decreased to a second preset value, such as 0.1, to ensure that the emergency help command is approved; when the user is in a normal activity state, differentiated adjustments are made according to the view type; for example, the rejection threshold is decreased to 0.3 for video playback and increased to 0.95 for payment. When the real-time physiological state data indicates that the user is in an emergency, if the current view type is navigation or settings, the rejection threshold is decreased to a third preset value. The third preset value is lower than the default rejection threshold, aiming to balance the user's need for quick access to navigation or settings functions in emergency situations with system security. For example, it is set to 0.4 to ensure that emergency commands are quickly granted while retaining appropriate screening for non-critical operations. Furthermore, based on the combination of the view type and user state, a corresponding semantic rejection model is selected and dynamically loaded. For example, when the user is asleep and the view is media playback, a low-power, lenient rejection model is loaded; when the user is in an emergency and the view is communication, an emergency priority model is loaded. This step forms the core synergy of this application: the view type provides the operational risk level, and the user state provides the urgency of the interaction. The decision generated by the combination of these two factors is far more accurate than a single-dimensional decision. For example, based solely on the user being asleep, the system can only raise all rejection thresholds indiscriminately. However, combined with the view type being alarm clock settings, the system can accurately determine that the user may be performing an important operation while asleep, thus rejecting the request and avoiding accidental operation. Conversely, if the view type is an emergency call, the rejection threshold is lowered even in a sleep state. This combined decision-making evolves the rejection strategy from a rigid, one-size-fits-all approach to a highly intelligent decision-making process based on scene awareness.

[0020] S4. Based on the adjusted rejection parameters, perform asynchronous first semantic rejection on the voice request to obtain the first semantic rejection result; wherein, the asynchronous first semantic rejection is executed in parallel with the downstream logic processing of the voice request, and the delay of the first semantic rejection is hidden by the parallel execution.

[0021] Specifically, the asynchronous first semantic rejection sends an asynchronous request to the semantic rejection service through the central control service. This service makes a judgment based on the adjusted threshold and model. The asynchronous processing mechanism and the aforementioned dynamic rejection strategy form a process-level collaboration. The dynamic strategy specifies the direction and threshold for asynchronous rejection, while asynchronous processing ensures that this precise strategy can be executed quickly. The asynchronous first semantic rejection and the downstream logic processing of the voice request are set to be executed in parallel, so as to hide the latency of the first semantic rejection in the execution process of the downstream logic processing. This invention eliminates the long latency caused by the serial rejection process through this parallel architecture, directly improving the smoothness of user interaction.

[0022] S5. After receiving the downstream logic processing result, perform synchronous second semantic rejection based on the downstream logic processing result and the first semantic rejection result to obtain the second semantic rejection result.

[0023] Specifically, this step integrates the asynchronous rejection result with the downstream logic result to form a final and authoritative decision basis, preventing divergence in the asynchronous process from leading to erroneous execution.

[0024] S6. Match the target service adapter according to the operation intention and the preset extensible service registry, and convert the operation intention into an executable instruction sequence containing deep link template parameters.

[0025] Specifically, the scalable service registry and adapter pattern provide architectural decoupling for intent execution, ensuring that the system can flexibly access new services, and together with the dynamic rejection policy, form an adaptive service delivery closed loop.

[0026] S7. Based on the second semantic rejection result and the real-time physiological state data, decide whether to execute the instruction sequence.

[0027] Specifically, when the second semantic rejection result indicates a valid input, and the real-time physiological state data does not indicate a high-risk state (e.g., the user is in a normal activity state), the instruction sequence is executed. When the second semantic rejection result indicates a valid input, but the real-time physiological state data indicates the user is in a high-risk state (e.g., the user is asleep and the instruction involves turning on a high-power appliance), execution is abandoned and a confirmation request is sent to the user. When the second semantic rejection result indicates an invalid input, execution is abandoned. This step constitutes the core synergy point between experience and safety decision-making. Semantic rejection ensures the semantic correctness of the instruction, while the user's state assesses the risk of executing the correct instruction. The two work together to achieve intelligent decision-making logic where correct instructions are not necessarily executed immediately, breaking through the traditional system's instruction-only framework. For example, if a user says "turn on the electric blanket" while half-asleep, although semantic rejection determines it as valid input, combined with the user's high-risk state, the system abandons execution, thereby eliminating the safety risk of fire or overheating. In this scenario, the synergy between semantic rejection and physiological state perception generates an overall safety gain. The view perception provides scene information, and the user state perception provides physiological evidence of the user's intent. Together, they drive the precise adjustment of the rejection strategy, while the asynchronous rejection process executes this decision with low latency. These three elements are interdependent and mutually reinforcing. For example, when a user is detected falling and making a distress call, the system automatically lowers the rejection threshold to the minimum, ensuring the emergency command is quickly processed and executed, while simultaneously linking with the community emergency assistance system. When a user is detected in deep sleep and making indistinct sounds, the system raises the rejection threshold to avoid accidental device activation, thus providing uninterrupted intelligent services while ensuring safety.

[0028] Furthermore, the step of dynamically adjusting the semantic rejection parameters based on the view type and the real-time physiological state data specifically includes: when the current view type is media playback and the real-time physiological state data indicates that the user is in a static state, lowering the rejection threshold to a first preset value; when the current view type is payment or security sensitive and the real-time physiological state data indicates that the user is in an active state, raising the rejection threshold to a second preset value; and when the current view type is navigation or settings and the real-time physiological state data indicates that the user is in an emergency state, lowering the rejection threshold to a third preset value. The value of the third preset value is lower than the default rejection threshold, used to ensure that users can quickly access navigation or settings services in an emergency state while retaining the ability to prevent operations that are accidentally triggered in a very low probability. The first preset value is lower than the default value, and the second preset value is higher than the default value.

[0029] Furthermore, the step of determining whether to execute the instruction sequence based on the second semantic rejection result and the real-time physiological state data includes: when the second semantic rejection result indicates valid input and the real-time physiological state data indicates that the user is in a normal activity state, constructing a call instruction based on the deep link template parameters and sending it to the target service; when the second semantic rejection result indicates valid input but the real-time physiological state data indicates that the user is in a sleep or high-risk state, generating a secondary confirmation request and waiting for user confirmation before execution; when the second semantic rejection result indicates invalid input, abandoning execution. Secondary confirmation itself is also a manifestation of collaborative effect: when the two decisions conflict, with semantic determination being feasible but state determination being high-risk, the system does not default to execution but introduces secondary confirmation as an intermediate strategy. This compromise strategy further optimizes the balance between user experience and security, forming the final closed loop of the overall collaborative decision-making chain.

[0030] Furthermore, the voice command description information also includes voice tags, which are used to describe the voice-operable element information on the view.

[0031] Furthermore, the present invention also proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned smart home service execution method.

[0032] Furthermore, this invention proposes a smart home service execution system, comprising: a multimodal perception module configured to acquire user voice information, voice command description information of the currently displayed view on the terminal, and real-time physiological state data of the user; an intent parsing module configured to perform semantic recognition on the voice information based on the view description information to obtain the user's operation intent; a view type recognition module configured to determine the type of the current view based on the view description information; a rejection strategy adjustment module configured to dynamically adjust the rejection parameters of semantic rejection based on the view type and the real-time physiological state data; an asynchronous rejection module configured to perform asynchronous first semantic rejection on the voice request based on the adjusted rejection parameters; a synchronous rejection module configured to perform synchronous second semantic rejection after receiving the downstream logic processing result to obtain a second semantic rejection result; a service orchestration engine configured to match service adapters and generate instruction sequences based on the operation intent; and an execution decision module configured to decide whether to execute the instruction sequence based on the second semantic rejection result and the real-time physiological state data. The multimodal perception module includes a microphone array and a millimeter-wave radar sensor. The execution decision module maintains the mapping relationship between user state and instruction execution strategy. In this system architecture, the service relationships between the modules form a closed-loop collaborative network: multimodal perception provides input for parsing and decision-making; intent parsing and rejection strategy adjustment are designed to operate in parallel, analyzing the input from their respective dimensions. The final results converge in the execution decision module, which in turn triggers the next round of rejection parameter adjustments, forming a continuously optimizing and adaptive intelligent system. This collaborative working relationship between modules makes the overall system's intelligence level, including rejection accuracy, response latency, and service security, far higher than the simple sum of the individual modules working independently; this is the effect of system-level collaboration.

[0033] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for executing smart home services, characterized in that, include: S1. Obtain the user's voice information, voice command descriptions of voice-operable elements in the current display view of the terminal, and the user's real-time physiological status data. S2. Based on the view description information of the voice-operable element, perform semantic recognition on the voice information to obtain the user's operation intention; S3. Determine the type of the current view based on the view description information, and dynamically adjust the rejection parameters of semantic rejection based on the view type and the real-time physiological state data. S4. Based on the adjusted rejection parameters, perform asynchronous first semantic rejection on the voice request to obtain the first semantic rejection result; wherein, the asynchronous first semantic rejection is executed in parallel with the downstream logic processing of the voice request, and the delay of the first semantic rejection is hidden by the parallel execution. S5. After receiving the downstream logic processing result, perform synchronous second semantic rejection based on the downstream logic processing result and the first semantic rejection result to obtain the second semantic rejection result; S6. Match the target service adapter according to the operation intention and the preset extensible service registry, and convert the operation intention into an executable instruction sequence; S7. Based on the second semantic rejection result and the real-time physiological state data, decide whether to execute the instruction sequence.

2. The smart home service execution method according to claim 1, characterized in that, The real-time physiological data includes heart rate, respiratory rate, body movement amplitude, and posture information.

3. The smart home service execution method according to claim 1, characterized in that, The step of dynamically adjusting the rejection parameters of semantic rejection based on the view type and the real-time physiological state data includes: increasing the rejection threshold to a first preset value when the real-time physiological state data indicates that the user is in a sleep state; and decreasing the rejection threshold to a second preset value when the real-time physiological state data indicates that the user is in an emergency state.

4. The smart home service execution method according to claim 1, characterized in that, The step of dynamically adjusting the rejection parameters of semantic rejection based on the view type and the real-time physiological state data further includes: selecting the corresponding semantic rejection model based on the combination of the view type and the real-time physiological state data, and dynamically loading and switching to the model for rejection judgment.

5. The smart home service execution method according to claim 1, characterized in that, The step of determining whether to execute the instruction sequence based on the second semantic rejection result and the real-time physiological state data includes: executing the instruction sequence when the second semantic rejection result indicates valid input and the real-time physiological state data indicates that the user is in a normal activity state; and generating a secondary confirmation request when the second semantic rejection result indicates valid input but the real-time physiological state data indicates that the user is in a high-risk state.

6. The smart home service execution method according to claim 1, characterized in that, The step of performing semantic recognition on the speech information based on the view description information of the voice-operable element includes: predicting the acoustic features of the speech information using an acoustic model; and dynamically decoding the acoustic features based on the view description information using a language model to generate query text.

7. The smart home service execution method according to claim 1, characterized in that, The step of matching a target service adapter based on the operation intent and a preset extensible service registry includes: receiving a standardized abstract intent object, which contains an intent identifier and a parameter mapping table; matching a target service adapter based on the intent identifier and service priority according to the extensible service registry; and invoking the adapter to convert the abstract intent object into an instruction sequence containing deep link template parameters.

8. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the smart home service execution method as described in any one of claims 1 to 7.

9. A smart home service execution system, characterized in that, include: The multimodal perception module is configured to acquire the user's voice information, the voice command description information of the terminal's current display view, and the user's real-time physiological state data; The intent parsing module is configured to perform semantic recognition on the voice information based on the view description information to obtain the user's operation intent; The view type identification module is configured to determine the type of the current view based on the view description information. The rejection strategy adjustment module is configured to dynamically adjust the rejection parameters of semantic rejection based on the view type and the real-time physiological state data. The asynchronous rejection module is configured to perform asynchronous first semantic rejection of voice requests based on the adjusted rejection parameters. The synchronous rejection module is configured to perform synchronous second semantic rejection after receiving the downstream logic processing result, and obtain the second semantic rejection result. A service orchestration engine is configured to match target service adapters and generate instruction sequences based on the stated operational intent. The execution decision module is configured to determine whether to execute the instruction sequence based on the second semantic rejection result and the real-time physiological state data.

Images