Smart home interaction method, home sprite interaction system, medium and product

Abstract

A smart home interaction method, a home sprite interaction system, a medium and a product relate to the field of general control or regulation systems, and the method comprises the following steps: receiving a voice instruction of a user, and determining an operation type and a target position descriptor; performing position mapping in a pre-established home three-dimensional virtual space to obtain a candidate device identifier located at a corresponding position; calculating the relevancy score of each candidate device and the historical interaction record of the user, and generating a candidate device sequence; generating an option display track of the interaction elf in the home three-dimensional virtual space; based on the option display track, controlling an interaction elf to sequentially move to each candidate device position in the home three-dimensional virtual space and execute a pointing action, and determining a target device identifier; generating an operation rehearsal animation; and executing the operation preview animation, and sending a device control instruction to a corresponding target device after receiving a user confirmation instruction. By implementing the application, the accuracy of feeding back the user instruction in smart home interaction can be improved.

Description

Technical Field

[0001] This application relates to the field of general control or regulation systems, and particularly to a smart home interaction method, a home wizard interaction system, a medium, and a product. Background Art

[0002] With the rapid development of smart home technologies, users' requirements for home interaction experiences are constantly increasing. A smart home system needs to be able to understand users' needs at different spatial locations and provide a natural and continuous interaction experience. Especially in the trend of whole-house intelligence, users expect to obtain a more immersive and context-aware smart home interaction method. Smart home interaction has gradually evolved from single command control to a natural interaction method that simulates real-life scenarios.

[0003] In related technologies, smart home interaction mainly uses smart speakers and mobile phone applications as control terminals. The smart speaker is fixedly installed at a specific location to receive users' voice commands and execute corresponding control operations. The mobile phone application allows users to remotely control home appliances through the mobile terminal interface. Some systems also introduce preset scenario modes, and users can execute multiple device linkages configured in advance by simply triggering a command.

[0004] However, in actual application scenarios, users' voice commands are often vaguely directed, and their interaction requirements are often strongly context-dependent; for example, when a user issues a command like "turn off the lights here", related technologies are difficult to effectively distinguish the specific range of the user's pointing, and there will be delays or deviations in command execution. Summary of the Invention

[0005] This application provides a smart home interaction method, a home wizard interaction system, a medium, and a product, which are used to improve the accuracy of feedback to users' commands in smart home interaction.

[0006] Firstly, this application provides a smart home interaction method applied to a home assistant interaction system. The method includes: receiving a user's voice command; determining the operation type and target location descriptor through voice recognition; mapping the target location descriptor in a pre-established three-dimensional virtual space of the home to obtain candidate device identifiers located at the positions corresponding to the target location descriptor; when the number of candidate device identifiers is greater than one, calculating the relevance score between each candidate device and the user's historical interaction records, and prioritizing the candidate devices based on the relevance score to generate a candidate device sequence; and generating an option display trajectory for the interactive assistant in the three-dimensional virtual space of the home based on the candidate device sequence. The display trajectory includes a set of movement path points of the interactive sprite in the 3D virtual space of the home and the pointing action parameters for each candidate device. Based on the display trajectory, the interactive sprite is controlled to move sequentially to the positions of each candidate device in the 3D virtual space of the home and perform pointing actions until it receives the user's device selection command and determines the target device identifier. According to the target device identifier and operation type, an operation pre-show animation is generated. This operation pre-show animation includes the displacement path of the interactive sprite, operation action parameters, and device status change parameters. The operation pre-show animation is executed, and after receiving the user's confirmation command, the operation pre-show animation is converted into a device control command and sent to the corresponding target device.

[0007] In the above embodiments, after receiving voice commands, the home assistant interactive system achieves accurate understanding and feedback of user commands through multiple steps such as three-dimensional virtual space position mapping, historical interaction record correlation calculation, dynamic display of the interactive assistant, and operation rehearsal. In particular, it generates a candidate device sequence by combining the user's historical interaction habits, intuitively displays the operation target through the movement and pointing actions of the interactive assistant, and demonstrates the operation effect in advance through the rehearsal animation, thereby improving the accuracy of smart home control and user experience.

[0008] In conjunction with some embodiments of the first aspect, in some embodiments, the step of controlling the interactive sprite to move sequentially to each candidate device location and perform pointing actions in the three-dimensional virtual space of the home based on the option display trajectory, until a user's device selection instruction is received, and determining the target device identifier, specifically includes: controlling the interactive sprite to move sequentially to each candidate device location and perform pointing actions in the three-dimensional virtual space of the home based on the option display trajectory; playing device identifier prompt information and starting a timer with a preset waiting time when the interactive sprite moves to each candidate device; determining the candidate device currently selected by the interactive sprite as the target device identifier when a user's selection confirmation instruction is received; continuing to move the interactive sprite to the next candidate device location when the timer expires and no user instruction is received; and controlling the interactive sprite to return to the candidate device with the highest relevance score and wait for the user's selection when all candidate devices have been traversed and no user confirmation instruction has been received.

[0009] In the above embodiments, the Home Assistant Interactive System adopts a timer and user confirmation mechanism to realize the automatic switching process of the interactive assistant displaying candidate devices in priority order; when no user confirmation is received, it automatically switches to the next candidate device, and after traversing all devices, it returns to the device with the highest relevance to wait, which not only ensures the continuity of interaction, but also avoids the user missing the best choice.

[0010] In conjunction with some embodiments of the first aspect, in some embodiments, the step of generating an operation preview animation based on the target device identifier and operation type specifically includes: obtaining historical control records of the target device and extracting execution parameters corresponding to the operation type; generating a device state change sequence and a sprite action sequence based on the execution parameters; and combining the state change sequence and the sprite action sequence in chronological order to generate an operation preview animation.

[0011] In the above embodiments, the Home Assistant Interactive System analyzes the historical control records of the device, extracts the execution parameters corresponding to the operation type, and generates a pre-show animation sequence that includes changes in device status and the actions of the assistant. Based on the historical data, the animation pre-show allows users to intuitively understand the operation effect and reduces the risk of misoperation.

[0012] In conjunction with some embodiments of the first aspect, in some embodiments, after the step of performing location mapping in a pre-established three-dimensional virtual space of the home based on the target location descriptor to obtain the candidate device identifier located at the position corresponding to the target location descriptor, the method further includes: obtaining user interaction records and device operation records for each candidate device; extracting device usage time period, usage frequency, and response latency features based on the user interaction records and device operation records; and adjusting the movement speed, waiting time, and prompting method of the interactive sprite according to the device usage time period, usage frequency, and response latency features.

[0013] In the above embodiments, the Home Assistant Interactive System dynamically adjusts the behavioral parameters of the interactive assistant based on the device's usage records and operating status; by analyzing the device's usage time, frequency, and response characteristics, it optimizes the assistant's movement speed, waiting time, and prompting methods, thus achieving an interactive experience that is more in line with user habits.

[0014] In conjunction with some embodiments of the first aspect, in some embodiments, the step of adjusting the movement speed, waiting time, and prompting method of the interactive wizard based on the device usage period, usage frequency, and response latency characteristics specifically includes: slowing down the movement speed of the interactive wizard and extending the action time pointing to the device when the device usage frequency is higher than a preset frequency threshold; setting the waiting time of the interactive wizard according to the device's response latency characteristics, and extending the waiting time of the interactive wizard when the device response latency is detected to exceed the historical average response latency; and selecting a prompting method with different prompting intensities based on the device usage period.

[0015] In the above embodiments, the Home Assistant Interaction System intelligently adjusts interaction parameters based on device usage characteristics, slows down the assistant's movement speed at frequently used devices, dynamically adjusts the waiting time based on response latency, and selects the intensity of prompts based on the usage period; this adaptive interaction method improves the system's accuracy in understanding user intent.

[0016] In conjunction with some embodiments of the first aspect, in some embodiments, after executing the operation preview animation and, upon receiving a user confirmation instruction, converting the operation preview animation into a device control instruction and sending the device control instruction to the corresponding target device, the method further includes: recording the playback response difference value between the playback speed of the operation preview animation and the actual device response speed in the historical control record of the target device; when the playback response difference value exceeds a preset deviation threshold, adjusting the playback speed of the preview animation in the control sequence according to the actual device response speed.

[0017] In the above embodiments, the Home Assistant interactive system dynamically optimizes the playback parameters of the pre-show animation by monitoring the difference between the playback speed of the pre-show animation and the actual device response speed; when the difference exceeds a threshold, adjustments are made to ensure the consistency between the pre-show effect and the actual operating experience.

[0018] In conjunction with some embodiments of the first aspect, in some embodiments, after adjusting the playback speed of the preview animation in the control sequence according to the actual device response speed, the method further includes: recording the user's confirmation waiting time during the playback of the preview animation; canceling the playback of the preview animation when the confirmation waiting time exceeds a first preset time; marking the operation type as a fast execution type when the number of consecutive cancellations of the same operation type by the user reaches a preset number; when the operation type is a fast execution type, the home assistant interactive system will skip the playback step of the preview animation and directly send device control commands.

[0019] In the above embodiments, the Home Assistant interactive system records the user's waiting time and cancellation operation for the preview animation, and intelligently identifies the operation needs of quick execution types; for operation types where users frequently cancel the preview, the system directly executes the device control command, thereby improving the system's interactive efficiency.

[0020] Secondly, embodiments of this application provide a home improvement interactive system, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, which includes computer instructions, and the one or more processors call the computer instructions to cause the home improvement interactive system to perform the method described in the first aspect and any possible implementation thereof.

[0021] Thirdly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on a home improvement interactive system, cause the home improvement interactive system to execute the method described in the first aspect and any possible implementation thereof.

[0022] Fourthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a home improvement interactive system, cause the home improvement interactive system to perform the method described in the first aspect and any possible implementation thereof.

[0023] Understandably, the home automation interactive system provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the methods provided in the embodiments of this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here.

[0024] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. By adopting a multi-level interaction scheme based on voice recognition, 3D spatial mapping, historical interaction analysis, and dynamic display of interactive wizards, the system can accurately extract users' fuzzy commands and provide intuitive operation feedback. This effectively solves the problem in existing technologies where it is difficult to accurately determine the user's pointing range by simply relying on voice recognition, thereby achieving high accuracy and a good user experience in smart home control. The system optimizes the sorting of candidate devices through analysis of historical interaction records, and combined with the dynamic pointing and pre-show animation of the interactive wizard, allows users to intuitively confirm the operation target and effect, significantly reducing the risk of misoperation.

[0025] 2. Because it adopts an adaptive interaction mechanism based on device usage records and operating status, the system can dynamically adjust the behavior parameters of the interactive sprite according to device characteristics, effectively solving the problem of poor user experience caused by fixed interaction parameters in existing technologies, and thus achieving a more natural interactive experience that conforms to user habits; by analyzing the device usage time period, frequency and response characteristics, the system intelligently adjusts the sprite's movement speed, waiting time and prompting method, making the interaction process smoother and more natural.

[0026] 3. Due to the adoption of a difference monitoring and dynamic optimization mechanism between the pre-show animation and the actual device response, the system can maintain the consistency between the pre-show effect and the actual operation experience, effectively solving the problem of deviation between the pre-show effect and the actual operation in the existing technology, and thus achieving more accurate operation expectation management; by recording the playback response difference value and dynamically adjusting the pre-show animation parameters, the system ensures that users can obtain the real operation expectation and improves the continuity of the interactive experience. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of a scene of the Home Assistant interactive system in an embodiment of this application; Figure 2 This is a scene interaction diagram of the home furnishing elf interaction method in the embodiments of this application; Figure 3 This is a flowchart illustrating a smart home interaction method in an embodiment of this application; Figure 4 This is another flowchart illustrating the smart home interaction method in the embodiments of this application; Figure 5 This is a schematic diagram of the physical device structure of the Home Assistant Interactive System in this application embodiment. Detailed Implementation

[0028] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification of this application, the singular expressions “a,” “an,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to any or all possible combinations including one or more of the listed items.

[0029] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0030] To facilitate understanding, the application scenarios of the embodiments of this application are described below.

[0031] In smart home environments, as the number of devices increases, users often encounter ambiguous commands when controlling devices via voice. For example, when a user says, "Turn off this light," if there are multiple lights in the room, the system may struggle to accurately understand the user's intention. This is especially true in open spaces like living rooms, where multiple lights such as chandeliers, wall lamps, and table lamps may be present, some of which are close together. Users need to clarify the specific device through multiple rounds of dialogue, such as "Not this one, but the one over there," reducing interaction efficiency. Furthermore, different devices have varying response characteristics; some respond quickly, while others require longer startup times, making it difficult for users to predict the desired effect and increasing the risk of misoperation.

[0032] In related technologies, smart home device selection and control can be achieved by using fixed voice command matching and device list display methods. The following describes scenarios using smart home interaction methods from these technologies.

[0033] Traditional voice control systems typically employ a fixed interaction flow: receiving voice commands, matching device names, and executing control commands. For example, when a user says "turn off the living room lights," the system lists all possible light fixtures, requiring the user to select one using options like "first," "second," etc. This approach has several problems: First, users need to remember the device's serial number or full name; second, the display order of the device list is fixed, failing to consider user habits; third, control commands are executed directly without any previewing, leaving users unable to anticipate the outcome. For instance, if a user wants to turn off some lights at night to adjust the ambiance but is unsure which light to control, they might need to repeatedly try different combinations, impacting the user experience.

[0034] The smart home interaction method described in this application, by displaying a dynamic device selection process in a virtual space, enables intuitive device location and operation preview, which not only improves the accuracy of user operations but also effectively reduces the cognitive burden on users. The following describes scenarios where the smart home interaction method of this application is used.

[0035] First, let me introduce the home automation interactive system in this application; please refer to [link / reference]. Figure 1 , Figure 1 This is a schematic diagram of a scenario of the Home Elf interactive system in this application embodiment, showing the Home Elf interactive interface presented on the mobile terminal after the user wakes up the system with a voice command. The Home Elf can adopt anthropomorphic images of animals or plants to enhance the user experience through emotional interaction.

[0036] In traditional smart home interaction models, the interaction between users and the system is often limited to simple command sending and execution feedback, lacking emotional connection and motivation for continued participation. This application introduces the concept of a home sprite (also known as an interactive sprite) to construct a completely new emotional interaction paradigm. For example... Figure 1 As shown, the user establishes an initial connection with the system via voice, and the system then displays a dynamic sprite image on the mobile terminal interface. This sprite is not simply a visual decoration, but an intelligent agent that carries the logic of home interaction.

[0037] In its design and implementation, the system offers users two basic forms: animal and plant. This choice considers user preferences and provides matching interactive platforms for different types of emotional needs. Each form is equipped with a unique growth system and interaction rules, using a carefully designed mathematical model to translate users' daily behaviors into the development trajectory of the virtual life form. The system collects and analyzes users' home usage data in real time, including device operation frequency, usage time distribution, energy consumption trends, and other multi-dimensional information. After standardization, this data is mapped onto various attribute parameters of the virtual life form, thereby driving dynamic changes in its appearance and behavior.

[0038] Through real-time display on the terminal screen, users can intuitively perceive the impact of their actions on their virtual companions. The system incorporates a "check-in" mechanism, gamifying everyday home use behaviors, with each task assigned clear growth incentives. For example, when users develop regular energy-saving habits, the electronic plant receives the "water droplets" needed for growth, exhibiting a more lush appearance; the electronic pet displays more affectionate interactive behaviors. This instant feedback mechanism greatly enhances users' sense of participation and accomplishment.

[0039] Furthermore, the Home Assistant interactive system continuously senses and analyzes the user's emotional state, enabling the assistant to provide contextualized emotional support. When it detects that a user is fatigued or stressed, the assistant proactively adjusts its behavior, conveying care through gentle interactions. This intelligent interaction based on emotion computing transforms the home system from a mere functional tool into a true life companion. As the interaction deepens, the system accumulates rich user profile data, continuously optimizing its emotional response strategies to create a more personalized and precise service experience. It should be noted that all user information is collected with the user's authorization, and the user is aware of the use and processing of the information.

[0040] In this embodiment, the home automation system described in this solution enables intuitive device selection and operation preview through a virtual interactive wizard. For example, when a user says, "Dim the light next to the sofa," the interactive wizard will move to the location of the light fixture in the sofa area in the virtual space, using pointing gestures and lighting effects to help the user identify the target device. The system will prioritize displaying the most frequently used light fixtures based on the user's usual usage habits. After the user confirms the device, the interactive wizard will demonstrate the dimming effect, allowing the user to intuitively see the expected change in lighting. The entire process is natural and smooth; the user does not need to remember device names or numbers, and can complete the operation directly through observation and confirmation.

[0041] As can be seen, the smart home interaction method in this application embodiment can not only achieve accurate device selection, but also effectively solve the problem of interaction experience caused by inconsistent device response characteristics, thereby realizing personalized optimization of smart home control.

[0042] For ease of understanding, the interaction logic in the above scenario is briefly described below.

[0043] Please see Figure 2 , Figure 2 This is a scene interaction diagram of the home assistant interaction method in this application embodiment. The diagram illustrates how the system accurately understands and executes the user's voice commands through three consecutive scenes. In the top scene, a typical home floor plan is shown, including functional areas such as a bathroom, bedroom, and living room. The user issues a voice command from bedroom 1. The system first needs to determine the user's position and perspective in this three-dimensional space, which is the basis for subsequent spatial mapping.

[0044] The central scene demonstrates the system's device recognition process. Based on the user's location and viewpoint information, the system establishes a dynamic field-of-view model. Within this range, all possible target devices are labeled as candidate objects. The system uses ray tracing to simulate the user's line of sight and combines this with the location description in the voice command to calculate the matching degree between each candidate device and the user's intent. This process is not a simple distance calculation, but rather integrates features from multiple dimensions such as spatial relationships, usage frequency, and temporal relevance.

[0045] The bottom scene highlights the dynamic selection process of the interactive sprite. Based on preliminary calculations, the system generates an optimal device access path. The interactive sprite moves sequentially along this path to each candidate device, performing standardized pointing actions. At each stop, the system starts a preset waiting timer, giving the user sufficient time to confirm or reject. The entire process is continuous and smooth; even if the user misses a confirmation opportunity, the system will still display all possible options and finally return to the device with the highest relevance, awaiting the user's final selection.

[0046] This dynamic and visual selection process not only improves the accuracy of device selection but also greatly enhances the intuitiveness of the interaction and the user experience. Through carefully designed animation effects and interactive rhythm, the system allows users to easily follow the wizard's guidance to complete device selection, avoiding the misunderstandings and operational errors that are common in traditional methods.

[0047] It should be noted that the above illustrations are only for the purpose of helping to understand the technical solution of this application and do not constitute a specific limitation of this application. Figure 1 and Figure 2 The home furnishing sprite images, device layouts, and interactive scenarios shown are for illustrative purposes only. In actual applications, different sprite designs and interaction methods can be selected according to specific needs.

[0048] To facilitate understanding, the method provided in this implementation will be described in detail below, using the above scenario as an example. Please refer to [link / reference]. Figure 3 This is a flowchart illustrating a smart home interaction method in an embodiment of this application.

[0049] S301: Receive the user's voice command and determine the operation type and target location descriptor through voice recognition.

[0050] Among them, voice commands represent the control intentions expressed by the user through voice, including information such as operation requirements and location descriptions; operation type refers to the specific control action that the user expects to perform, such as turning on, turning off, adjusting, etc.; target location descriptors are used to represent the spatial location information mentioned in the user's voice, such as location descriptions such as "corner of the living room" or "next to the dining table".

[0051] The Home Assistant interactive system executes this step when it detects a voice command from the user. Specifically, the Home Assistant interactive system first collects the user's voice signal through a microphone array, performs noise reduction and segmentation processing on the voice signal; then it calls the natural language processing module to convert the speech into text and perform semantic analysis; next, it extracts the operation intent information from the text and identifies the specific operation type; finally, it locates location-related words in the text and extracts the target location descriptive words.

[0052] In some embodiments, voice command processing and information extraction can be implemented in multiple ways: Optionally, the home assistant interaction system can employ a deep learning-based speech recognition model, extracting speech features through a multi-layer neural network, recognizing keywords using an attention mechanism, and finally labeling operation types and location descriptive words using a conditional random field model; Optionally, the home assistant interaction system can employ a rule-based method, pre-establishing an operation type dictionary and a location description dictionary, and extracting corresponding information from the speech text through pattern matching. It is understood that other speech processing and information extraction methods can also be used, and this application does not limit the specific implementation method.

[0053] During the implementation of the solution, ambiguous user speech may arise. For example, when a user says "Turn off the light over here," the specific location of "over here" is unclear. The Home Assistant interactive system can address this by: first, combining information such as the user's gaze direction and body orientation to determine the approximate spatial range the user is interested in; then, analyzing the recent device usage records within that range to identify the most likely target device; and finally, clarifying the user's intention through interactive confirmation.

[0054] S302. Based on the target location descriptor, perform location mapping in the pre-established 3D virtual space of the home to obtain the candidate device identifier located at the position corresponding to the target location descriptor.

[0055] Among them, the target location descriptor refers to the spatial location-related description extracted from the user's voice command, such as "next to the sofa" or "near the window"; the three-dimensional virtual space of the home refers to the digital modeling expression of the actual living environment, including information such as spatial layout, furniture position and equipment distribution; the location mapping is used to represent the conversion of the location information described in natural language into a specific coordinate area in the virtual space; the candidate device identifier refers to the unique identification code of the controllable device located in the target area.

[0056] The Home Assistant Interactive System executes this step after obtaining the target location descriptor. Specifically, the system first accesses a pre-built 3D virtual space model of the home, which includes spatial information such as room layout, furniture placement, and equipment installation locations. Then, it parses spatial relationship words contained in the target location descriptor, such as "beside," "above," and "middle," converting them into specific spatial range parameters. Next, it locates the positions of relevant reference objects (such as furniture and walls) in the virtual space and determines the specific range of the target area based on the spatial relationship parameters. Finally, it scans all smart devices within the area to generate a candidate device list.

[0057] In some embodiments, the mapping from location descriptors to virtual spatial locations can be achieved in multiple ways: Optionally, the Home Assistant interactive system can establish a semantic spatial relationship knowledge base, converting location descriptions in natural language into standardized spatial relationship expressions, and determining specific areas through spatial calculations; alternatively, the Home Assistant interactive system can employ machine learning-based methods, training a location mapping model with a large amount of labeled data, and directly outputting the spatial coordinates of the target area. It is understood that other spatial mapping methods can also be used to achieve the conversion from location descriptions to virtual space, and this application does not limit the specific implementation method.

[0058] During implementation, the same location description may correspond to multiple possible areas. For example, when a user says "turn on the light at the door," there may be multiple doors in the room. The Home Assistant Interactive System can address this by: first, analyzing the user's current location and behavior to determine the most likely doorway area; then, considering the room's usage scenario and time period, assessing the probability of use for each doorway area; finally, if a unique area still cannot be determined, adding all devices within all possible areas to the candidate list for further filtering in subsequent steps.

[0059] S303. When the number of candidate device identifiers is greater than 1, calculate the relevance score between each candidate device and the user's historical interaction records, and prioritize the candidate devices based on the relevance scores to generate a candidate device sequence.

[0060] Among them, the candidate device identifier represents the unique code of the controllable intelligent device identified in the target area; the historical interaction record refers to the historical operation data between the user and each device, including information such as operation time, operation type, and operation frequency; the relevance score is used to represent the degree of matching between the candidate device and the current operation scenario; the priority ranking refers to the ranking of the candidate devices according to the importance based on the relevance score; and the candidate device sequence represents the ranked list of devices.

[0061] The Home Assistant Interactive System executes this step after acquiring multiple candidate device identifiers. Specifically, the system first accesses the user's historical interaction database to extract the historical operation records of each candidate device; then it analyzes environmental factors such as the current time and scene to establish an operation scene feature vector; next, it calculates the similarity between the historical interaction features of each candidate device and the current scene features, while also considering factors such as device usage frequency and recent operation time to generate a comprehensive relevance score; finally, it sorts the candidate devices in descending order based on the relevance scores to form a candidate device sequence.

[0062] In some embodiments, device relevance can be calculated and ranked in multiple ways: Optionally, the Home Assistant interaction system can employ a time-decay-based weighted calculation method, setting different weights according to the time intervals of historical operations, and combining operation frequency and scene similarity to calculate the final score; Optionally, the Home Assistant interaction system can establish a user behavior model to analyze the user's device usage patterns in different scenarios and predict the device most likely to be operated in the current scenario. It is understood that other methods can also be used to calculate and rank device relevance, and this application does not limit the specific implementation method.

[0063] During implementation, changes in user habits may lead to inaccurate historical data. For example, users may change room functions and equipment usage after renovations. The Home Assistant interactive system addresses this by: first, setting an expiration period for historical data and reducing the weight of earlier records; second, identifying abrupt changes in user habits and assigning higher weight to data after that point; and finally, introducing a real-time learning mechanism to quickly adapt to new user habits and dynamically update the relevance calculation model.

[0064] It should be noted that the device relevance calculation model uses a deep neural network combined with an attention mechanism to calculate the relevance between candidate devices and the current operational intent. The model input includes three types of features: temporal features (mapping operation time to continuous values ​​over a 24-hour period, calculating the time interval and frequency statistics of recent operations), spatial features (the device's three-dimensional coordinates and its relative position vector to a reference object), and user features (embedded representations of historical operation sequences). These features undergo nonlinear transformation via a multilayer perceptron, and then the importance weights of different features are calculated through an attention layer, resulting in a weighted fusion to obtain a comprehensive representation of the device. Finally, the output is mapped to a relevance score between 0 and 1 using a sigmoid function. Model training uses records of users actually selecting devices as supervision signals to minimize the cross-entropy loss between predicted relevance and actual selection. For example, when a user says "turn off this light," the model will rank the relevance of each light fixture based on the user's location, viewing direction (which can be determined based on the user's mobile terminal (phone),) and historical usage habits.

[0065] S304. Based on the candidate device sequence, generate the option display trajectory of the interactive sprite in the three-dimensional virtual space of the home.

[0066] Among them, the candidate device sequence represents a list of devices to be confirmed, sorted by relevance; the interactive sprite is a virtual avatar representing the system in the virtual space, used to display the system status and perform interactive actions; the home 3D virtual space refers to a digital indoor environment model; the option display trajectory represents the movement path and action sequence required for the interactive sprite to complete the candidate device display, including information such as spatial coordinates, movement speed, and posture parameters; the movement path point set refers to the sequence of key location points of the interactive sprite in the virtual space; and the pointing action parameters represent the specific action details performed at each device location.

[0067] The Home Assistant Interactive System executes this step after obtaining the sorted sequence of candidate devices. Specifically, the system first obtains the precise coordinates of all candidate devices in the virtual space; then it analyzes the spatial distribution relationship between the devices, plans the optimal display order, and avoids repetitive back-and-forth movements and occlusion interference; next, it calculates the appropriate viewing angle and pointing posture for each device position to ensure that the user can clearly see the interactive sprite's instructions; then, it generates the interactive sprite's movement path points based on the device distribution, while considering obstacle avoidance and smooth transitions; finally, it sets the corresponding dwell time and motion amplitude parameters based on the importance of each device to generate a complete display trajectory.

[0068] In some embodiments, the generation of the option display trajectory can be achieved in multiple ways: Optionally, the Home Assistant interactive system can employ a graph-based path planning algorithm to construct a node graph of device locations, comprehensively considering path length, number of turns, and visual effects to calculate the optimal display path; combined with cubic spline interpolation, a smooth motion trajectory can be generated; and combined with preset instruction actions in the action library, a smooth animation sequence can be generated. Optionally, the Home Assistant interactive system can employ a heuristic search method, setting display effect scoring criteria, and searching for the best display scheme through optimization methods such as genetic algorithms; including setting multiple alternative display path schemes and selecting the most suitable display trajectory based on the real-time scene. It is understood that other methods can also be used to plan and generate the display trajectory, and this application does not limit the specific implementation method.

[0069] S305. Based on the option display trajectory, control the interactive wizard to move sequentially to the positions of each candidate device in the three-dimensional virtual space of the home and perform pointing actions until it receives the user's device selection instruction and determines the target device identifier.

[0070] Among them, the option display trajectory represents the complete action route required for the interactive sprite to complete the device instruction; the interactive sprite movement represents the process of the virtual image's position change in space; the pointing action refers to a specific gesture or light effect used to highlight the target device; the device selection instruction represents the user's confirmation signal for the currently indicated device; the target device identifier is a unique code used to represent the finally determined operation object; the waiting time parameter represents the time spent at each device; and the confirmation feedback signal includes multiple confirmation methods such as voice and gesture.

[0071] The Home Assistant Interactive System executes this step after generating the display trajectory. Specifically, the system first calculates the initial position and orientation of the interactive sprite based on the display trajectory; then, according to preset movement speed and acceleration parameters, it controls the interactive sprite to move smoothly along the trajectory; when it reaches the candidate device position, it executes a preset pointing action while playing the device name and function prompts; next, it starts a waiting timer to detect the user's confirmation signal within a preset time; if a confirmation command is received, the current device is marked as the target device; if the wait times out, it continues to move to the next candidate device position; this continues until all devices are displayed or user confirmation is obtained.

[0072] In some embodiments, the movement control and user confirmation detection of the interactive sprite can be implemented in multiple ways: Optionally, the home sprite interaction system can adopt a physics engine-based motion control method, setting physical attributes such as mass and inertia for the interactive sprite, and generating natural and smooth motion effects through mechanical calculations; at the same time, particle effects and sound prompts can be used to enhance the pointing effect; and the effect parameters can be dynamically adjusted according to the ambient brightness and user distance. Optionally, the home sprite interaction system can adopt a multimodal interaction scheme, supporting multiple confirmation methods such as voice, gesture, and gaze, and comprehensively judging the user's intent through a confidence fusion algorithm; including setting an anti-accidental touch mechanism, requiring a high confidence level to trigger confirmation. It is understood that other methods can also be used to implement interactive control and confirmation detection, and this application does not limit the specific implementation method.

[0073] During implementation, issues may arise where users' attention wanders, causing them to miss confirmation opportunities. For example, users might answer a phone call or handle other tasks. The Home Assistant Interactive System addresses this by: first, using sensors such as cameras to detect the user's attention state and identify distractions or absences; then, automatically pausing the presentation while keeping the interactive assistant in its current position; simultaneously recording the presentation progress for quick replay upon the user's return; prompting the user to resume interaction in a prominent but non-intrusive manner when their attention is detected again; and automatically saving the current state for quick resumption if the wait is too long. Furthermore, the system learns the user's attention patterns to optimize the presentation rhythm and prompting strategies.

[0074] S306. Generate an operation preview animation based on the target device identifier and operation type.

[0075] Among them, the target device identifier represents the unique identification code of the finally determined operation object; the operation type refers to the specific control action that the user expects to perform; the operation pre-show animation represents the virtual effect preview shown before actual control; the animation sequence includes the time sequence display of device state changes and interactive sprite actions; the execution parameters represent the specific numerical settings of the control actions; the state change sequence refers to the transition process of the device from the current state to the target state; and the scene lighting effects are environmental rendering parameters used to enhance the pre-show effect.

[0076] The Home Assistant Interactive System executes this step after identifying the target device. Specifically, the system first obtains the target device's current operating status and available operation set; then, based on the operation type, it queries the device's control parameter range and default settings; next, it analyzes historical operation records to extract user-preferred parameter settings; based on this information, it constructs a state transition model and calculates the optimal transition path from the current state to the target state; simultaneously, it generates corresponding interactive sprite actions, including operation gestures and facial feedback; finally, it synchronizes the state changes and action sequences in time, adds necessary lighting and sound effects, and synthesizes a complete pre-show animation.

[0077] In some embodiments, the generation of pre-show animations can be achieved in multiple ways: Optionally, the Home Assistant interactive system can establish a device state transition model library, preset standard animation templates for different types of operations, and generate personalized pre-show effects through parameterized adjustments; this includes setting multi-level animation details, selecting appropriate display methods based on the importance of the device and the complexity of the operation; and combining a physical simulation engine to calculate the real state change process. Optionally, the Home Assistant interactive system can adopt a deep learning-based animation generation method, training the generation model through a large number of real operation videos to achieve a more natural and smooth pre-show effect; this includes introducing style transfer technology to make the pre-show animation conform to the user's aesthetic preferences. It is understood that other methods can also be used to generate and optimize pre-show animations, and this application does not limit the specific implementation method.

[0078] It should be noted that the pre-show animation generation can be implemented using an algorithm; this algorithm generates a smooth pre-show animation sequence based on physical simulation and interpolation calculations. First, a state transition model is established based on the physical characteristics of the device, including the variation patterns of parameters such as position, angle, and brightness. The algorithm calculates the evolution trajectory of state parameters over time by solving differential equations, obtaining a sequence of key state points. Simultaneously, the motion sequence of the interactive sprite is planned based on kinematic principles, including path planning and posture interpolation. Then, the state change sequence and motion sequence are aligned on the time axis, and intermediate frames are generated through cubic spline interpolation to ensure the continuity and smoothness of the animation. For example, when adjusting light brightness, the algorithm calculates the gradual change process of light intensity based on the current brightness value and the target brightness value, and simultaneously generates the sprite's adjustment action, making the entire process natural and smooth.

[0079] S307. Execute the operation preview animation, and after receiving the user's confirmation instruction, convert the operation preview animation into a device control instruction and send the device control instruction to the corresponding target device.

[0080] Among them, the operation pre-show animation represents a visual preview of the control effect; the user confirmation instruction is a signal of user approval of the pre-show effect; the device control instruction represents the standard control command sent to the actual device; the animation conversion module is used to convert the pre-show effect into specific control parameters; the instruction adapter is used to handle the communication protocols of different devices; the execution feedback signal represents the status return of the device receiving and executing the instruction; and the device response confirmation is used to verify whether the control instruction has been successfully executed.

[0081] The Home Assistant Interactive System executes this step after completing the pre-show animation. Specifically, the system first plays the pre-show animation to demonstrate the expected control effect; then it initiates user confirmation detection, supporting multiple confirmation methods such as voice and gestures; upon receiving a confirmation command, the system parses the state change sequence in the pre-show animation and extracts key control parameters; next, based on the device type and communication protocol, it packages the control parameters into standard control commands; the commands are then sent to the target device through the device gateway; simultaneously, the system monitors the device's response status to ensure the commands are correctly received and executed; finally, based on the execution results returned by the device, the system updates the device status in the virtual space.

[0082] In some embodiments, the conversion from pre-rendered animation to control commands can be achieved in multiple ways: Optionally, the Home Assistant interactive system can establish a mapping rule base between animation parameters and control commands, and use a state machine model to process complex control logic to achieve accurate conversion from animation effects to specific parameters; this includes setting parameter validity checks to ensure that the generated control commands are within the range supported by the device; and optimizing parameters when necessary to make the control effect closer to the pre-rendered effect. Optionally, the Home Assistant interactive system can use intelligent analysis methods to directly extract control intent from the pre-rendered animation through a deep learning model and automatically generate the optimal control command sequence; this includes considering the current state of the device and environmental conditions to dynamically adjust the control strategy. It is understood that other methods can also be used to achieve the animation-to-command conversion process, and this application does not limit the specific implementation method.

[0083] During implementation, discrepancies may arise between the actual equipment performance and the pre-simulation results. For example, equipment malfunctions or environmental interference may cause the actual control effect to deviate from expectations. The Home Assistant Interactive System addresses this by: first, establishing a real-time monitoring mechanism to track the execution effect through sensors and equipment feedback signals; immediately alerting the user when a significant deviation is detected, explaining the specific cause; simultaneously, initiating a compensation control strategy to attempt to adjust control parameters to bring the actual effect closer to the expectation; if the expected effect cannot be achieved through adjustment, alternative solutions are provided for the user to choose from; the system also records such anomalies to optimize the pre-simulation model and control strategy, improving the accuracy of subsequent operations. Furthermore, the system regularly checks equipment status to proactively identify potential problems and reduce the probability of execution deviations.

[0084] The following provides a more detailed description of the process of the method provided in this implementation. Please refer to [link / reference]. Figure 4 This is another flowchart illustrating the smart home interaction method in this application embodiment.

[0085] S401: Receive the user's voice command and determine the operation type and target location descriptor through voice recognition.

[0086] Referring to step S301, the Home Assistant Interactive System will use voice recognition technology to convert the user's voice commands into specific operation intent information.

[0087] S402. Based on the target location descriptor, perform location mapping in the pre-established 3D virtual space of the home to obtain the candidate device identifier located at the position corresponding to the target location descriptor.

[0088] Referring to step S302, the Home Assistant interactive system will locate the target area described by the user in the virtual space and filter out the controllable devices in that area.

[0089] S403. Obtain user interaction records and device operation records for each candidate device.

[0090] Among them, user interaction records represent historical operation data between users and devices, including information such as operation time, operation type, and operation parameters; device operation records refer to the working status data of devices, including switch status, operating parameters, and fault information; data acquisition cycle represents the time interval for record updates; data storage format is used to standardize the storage structure of records; and data cleaning rules are used to filter invalid or abnormal data.

[0091] The Home Assistant Interaction System performs this step after identifying candidate devices. Specifically, the Home Assistant Interaction System first connects to the smart home data center and accesses the device data storage module; then, it queries the historical database according to the device identifier to extract the user interaction logs for each candidate device; simultaneously, it obtains the operational data recorded by the device status monitoring system; next, it cleans and preprocesses the raw data, removing outliers and invalid records; finally, it organizes the processed interaction records and operational records according to time series to generate a standardized dataset.

[0092] In some embodiments, the acquisition and processing of historical device data can be achieved in multiple ways: Optionally, the Home Assistant interactive system can establish a distributed data acquisition network, collect and preprocess device data in real time through edge computing nodes, and periodically synchronize it to a central database; this includes setting up a multi-level caching mechanism to optimize data access efficiency; and implementing incremental updates and cold / hot data separation storage. Optionally, the Home Assistant interactive system can adopt a real-time stream processing architecture, receive device status change events through a message queue, and update device operation records in real time; this includes establishing a data quality assessment model and automatically marking trustworthiness levels. It is understood that other methods can also be used to achieve data acquisition and management, and this application does not limit the specific implementation method.

[0093] During the implementation of the solution, issues such as intermittent or incomplete device data may be encountered. For example, data recording may be interrupted due to network failures or device offline. The Home Assistant Interactive System can address this by: first, establishing a data integrity detection mechanism to identify missing data intervals; then, based on the temporal correlation of the data, using interpolation or model prediction methods to supplement the missing data; simultaneously, recording data reliability markers for weight adjustment in subsequent analysis; for devices that frequently experience data loss, the system will optimize data acquisition strategies, add backup mechanisms, or adjust the sampling frequency; and, if necessary, combining multi-source data cross-validation can further improve data reliability.

[0094] S404. Based on user interaction records and device operation records, extract device usage time periods, usage frequency, and response latency characteristics.

[0095] Among them, the equipment usage period represents the time interval characteristics of the equipment being frequently operated; the usage frequency refers to the number of times the equipment is operated per unit time; the response delay characteristic is used to represent the time interval distribution from receiving the instruction to completing the operation; the time period division granularity represents the smallest time unit for analyzing usage patterns; the frequency statistics period refers to the time window for calculating the usage frequency; and the delay reference value represents the reference time for the normal response of the equipment.

[0096] The Home Assistant interactive system performs this step after acquiring historical records. Specifically, the Home Assistant interactive system first segments the historical data according to the time dimension to create a 24-hour usage time distribution map; then it counts the number of operations within each time period to identify high-frequency and low-frequency usage periods; next, it analyzes the start and end timestamps of each operation to calculate the response latency of the command execution; it performs statistical analysis on the response latency data to obtain the average response time and fluctuation range; finally, it performs correlation analysis on the time period characteristics, frequency characteristics, and latency characteristics to establish a device usage characteristic model.

[0097] In some embodiments, device usage characteristics can be extracted in multiple ways: Optionally, the Home Assistant interactive system can employ time series analysis methods, calculating local statistical features through a sliding window and combining periodic detection to identify user usage patterns; this includes considering the differences between weekdays and rest days to establish a multi-mode usage characteristic model; and supporting adaptive adjustment of feature extraction parameters. Optionally, the Home Assistant interactive system can use machine learning algorithms to automatically learn time period division methods and feature extraction rules from historical data; this includes introducing anomaly detection mechanisms to identify and handle atypical usage behaviors. It is understood that other methods can also be used to analyze and model usage characteristics, and this application does not limit the specific implementation method.

[0098] During the implementation of the solution, the dynamic changes in device usage patterns may be encountered. For example, changes in family members' schedules may cause shifts in usage periods. The Home Assistant interactive system can address this by: first, establishing a feature drift detection mechanism to monitor the changing trends of usage patterns in real time; then, employing incremental learning to dynamically update the feature model parameters; simultaneously, maintaining a memory mechanism for historical features to support switching between multiple usage modes; the system will also periodically evaluate the accuracy of the feature model and trigger model retraining when significant deviations are detected; and, if necessary, requesting user confirmation to assist in determining the validity of the mode change.

[0099] S405. Adjust the movement speed, waiting time, and prompting method of the interactive sprite according to the device's usage period, frequency of use, and response latency characteristics.

[0100] Among them, movement speed represents the rate of change of the interactive sprite's position in the virtual space; waiting time refers to the time spent waiting for user confirmation at each device; prompting method is used to represent the visual and sound effects that guide the user's attention; speed adjustment coefficient represents the speed scaling ratio dynamically calculated based on usage frequency; waiting time baseline value refers to the standard waiting time determined based on response latency characteristics; prompt intensity level is used to distinguish the prompting effect at different times.

[0101] The Home Assistant Interactive System executes this step after acquiring device usage characteristics. Specifically, the system first calculates a speed adjustment coefficient based on device usage frequency, with higher-frequency devices corresponding to lower movement speeds. Then, it sets a waiting time based on the device's response latency characteristics, considering the device's average response time and fluctuation range. Next, it selects an appropriate prompting method based on the usage characteristics of the current time period, using prominent prompts during high-frequency usage and gentler prompts during low-frequency usage. Finally, these parameters are applied to the interactive system's behavior control to achieve an adaptive interactive experience.

[0102] In some embodiments, the dynamic adjustment of interaction parameters can be achieved in multiple ways: Optionally, the Home Assistant interaction system can establish a fuzzy control rule base, dynamically calculate interaction parameters based on the fuzzy quantization value of usage features, and achieve smooth parameter adjustment; including setting multi-level adjustment thresholds to avoid frequent parameter fluctuations; and supporting personalized parameter configuration templates. Optionally, the Home Assistant interaction system can adopt reinforcement learning methods to continuously optimize the selection strategy of interaction parameters through user feedback; including establishing a user satisfaction evaluation model as the optimization target for parameter adjustment; and achieving continuous improvement of the interactive experience. It is understood that other methods can also be used to optimize interaction parameters, and this application does not limit the specific implementation method. It should be noted that when adjusting interaction parameters, contextual factors such as the user's operational proficiency and the degree of environmental interference also need to be considered.

[0103] During implementation, the system may encounter situations where different users have varying requirements for the interactive experience. For example, elderly users may require slower display speeds and more prominent prompts. The Home Assistant Interactive System addresses this by: first, establishing a multi-user configuration system and maintaining an independent set of interactive parameters for each user; then, using user recognition technology to switch the current configuration in real time; simultaneously, collecting feedback data during user interaction, including metrics such as operation latency and repetition counts; regularly analyzing user usage patterns and automatically adjusting personalized configurations; and, when necessary, providing a parameter adjustment interface to allow users to manually fine-tune the interactive effects.

[0104] In some embodiments, the Home Assistant Interaction System optimizes the interaction experience based on device usage characteristics. Specifically, when the frequency of device use exceeds a preset frequency threshold, the movement speed of the interaction assistant is slowed down, and the action time for pointing at the device is extended. The waiting time of the interaction assistant is set according to the device's response latency characteristics. When the device's response latency is detected to exceed the historical average response latency, the waiting time of the interaction assistant is extended. Different prompting intensities are selected based on the device's usage period.

[0105] Among them, device usage frequency represents the number of times the device is operated per unit time; preset frequency threshold refers to the standard number of times to judge high-frequency use; movement speed represents the rate of change of the interactive sprite's position in the virtual space; action time is used to represent the duration of the pointing action; response latency characteristic represents the time characteristics of the device from receiving the instruction to completing the operation; historical average response latency refers to the reference time for the device to respond normally; waiting time represents the time the system waits for the device to respond; usage time period characteristic is used to describe the usage pattern of the device in different time periods; prompt intensity refers to the prominence of the system prompt information.

[0106] The Home Assistant Interactive System executes this section after analyzing device usage characteristics. Specifically, the system first calculates the current usage frequency of the device and compares it with a preset threshold. For frequently used devices, it reduces the interactive wizard's movement speed parameter and extends the execution time of pointing actions, allowing users to more clearly perceive the interaction process. Next, it analyzes the device's response latency data, including average latency and fluctuation range. When an abnormal response latency is detected, the interactive wizard's waiting time is extended accordingly. Finally, based on the usage characteristics of the current time period, it selects an appropriate intensity of prompt from the prompt scheme library, ensuring the prompt effect while avoiding interference with the user.

[0107] In some embodiments, the dynamic adjustment of interaction parameters can be achieved in several ways: Optionally, the Home Assistant interaction system can establish a parameter adjustment scheme based on fuzzy control, converting features such as usage frequency and response latency into fuzzy quantized values; calculate the optimal combination of interaction parameters through fuzzy rule inference; set a smooth transition mechanism for parameter adjustment to avoid sudden changes causing inconvenience to users; and support scenario-based parameter configuration templates. Optionally, the Home Assistant interaction system can adopt an adaptive learning method to continuously optimize the parameter adjustment strategy by collecting user operation feedback; establish a user satisfaction evaluation model; dynamically update the weight coefficients of parameter adjustment; and achieve continuous improvement of the interactive experience. It is understood that other methods can also be used to optimize interaction parameters, and this application does not limit the specific implementation method.

[0108] During implementation, issues may arise where dynamic changes in device usage characteristics lead to mismatched interaction parameters. The Home Assistant Interaction System incorporates an adaptive interaction parameter algorithm, employing fuzzy control theory to dynamically adjust these parameters. First, features such as device usage frequency, response latency, and time period distribution are converted into fuzzy sets, defining linguistic variables like "high," "medium," and "low." Then, a fuzzy rule base is constructed, such as "IF high usage frequency AND currently in a high-frequency period THEN reduce movement speed." The algorithm calculates the membership degree of the output variables through fuzzy inference and finally defuzzifies using the centroid method to obtain specific control parameter values. The system also calculates dynamic changes in features using a sliding time window, continuously updating the weights of the fuzzy rules. For example, when a device is detected to be frequently used at night and has a slow response, the system automatically adjusts the movement speed and waiting time for that period.

[0109] S406. When the number of candidate device identifiers is greater than 1, calculate the relevance score between each candidate device and the user's historical interaction records, and prioritize the candidate devices based on the relevance scores to generate a candidate device sequence.

[0110] Referring to step S303, the Home Assistant interactive system will analyze the user's historical operating habits and assign priority weights to candidate devices.

[0111] S407. Based on the candidate device sequence, generate the option display trajectory of the interactive sprite in the three-dimensional virtual space of the home.

[0112] Referring to step S304, the Home Assistant Interactive System will plan the optimal movement path and action sequence for the interactive sprite.

[0113] S408: Based on the option display trajectory, control the interactive wizard to move sequentially to the positions of each candidate device in the three-dimensional virtual space of the home and perform pointing actions until it receives the user's device selection instruction and determines the target device identifier.

[0114] Referring to step S305, the Home Assistant interactive system will display each candidate device in sequence, waiting for the user to confirm the selection.

[0115] In some embodiments, the home automation system determines the user's target device through multiple rounds of interaction. Specifically, based on the option display trajectory, the system controls the interactive sprite to move sequentially to each candidate device location in the three-dimensional virtual space of the home and perform pointing actions. When the interactive sprite moves to each candidate device, it plays a device identification prompt and starts a timer with a preset waiting time. When the user's selection confirmation instruction is received, the candidate device currently selected by the interactive sprite is identified as the target device identifier. If the timer expires without receiving a user instruction, the system continues to move the interactive sprite to the next candidate device location. When all candidate devices have been traversed and no user confirmation instruction has been received, the system controls the interactive sprite to return to the candidate device with the highest relevance score and wait for the user to select it.

[0116] Among them, the option display trajectory represents the movement path and action sequence of the interactive sprite in the virtual space; the interactive sprite refers to the system's visual agent in the virtual space; the candidate device location represents the coordinate position of the candidate device in the virtual space; the pointing action refers to a specific gesture or light effect used to highlight the target device; the device identification prompt information represents text, voice, or icon prompts used to assist users in identifying the device; the preset waiting time is the standard time interval for the system to wait for user confirmation; the selection confirmation instruction represents the user's confirmation signal for the current option; the target device identifier is used to represent the unique code of the finally determined operation object; and the relevance score is the score of the degree of matching between the device and the current operation scenario.

[0117] The Home Assistant Interactive System executes this section of content after generating the option display trajectory. Specifically, the system first initializes the interactive sprite's position and orientation parameters based on the display trajectory; then, it controls the interactive sprite to move to the first candidate device position at a preset speed; upon arrival, it performs a standardized pointing action while playing the device's name and function prompts; next, it starts a waiting timer, continuously detecting the user's confirmation signal within a preset duration; if a confirmation command is received, the current device is marked as the target device; if the timer expires, it continues to move to the next candidate device; this process is repeated until all candidate devices have been traversed; if no confirmation command is received throughout the process, it returns to the device with the highest relevance to continue waiting for the user to select; during this process, it continuously monitors the user's attention state and adjusts the display pace as necessary.

[0118] In some embodiments, the movement control and user confirmation detection of the interactive sprite can be implemented in multiple ways: Optionally, the home sprite interaction system can adopt a physics engine-based motion control method, setting physical attributes such as mass and inertia for the interactive sprite to generate a smooth acceleration and deceleration process; during movement, collision detection is calculated in real time to automatically avoid obstacles; after reaching the device position, a pointing action from a preset action library is executed, combined with particle effects to enhance the visual effect. Optionally, the home sprite interaction system can adopt a multimodal interaction scheme, simultaneously supporting multiple confirmation methods such as voice, gesture, and eye tracking; user feedback signals are collected through a sensor array, and the validity of confirmation is judged by combining a confidence evaluation model; dynamic prompts are used to maintain user attention during the waiting process. It is understood that other methods can also be used to implement interactive control and confirmation detection, and this application does not limit the specific implementation method.

[0119] During implementation, users may encounter the problem of difficulty in following the interactive sprite's movement. For example, when candidate devices are scattered, users need to frequently shift their gaze or even turn around, easily missing the confirmation opportunity. The Home Assistant Interactive System addresses this by: first, grouping candidate devices using a spatial clustering algorithm and arranging adjacent devices in a continuous display sequence; then, calculating the optimal viewing angle based on the user's position and optimizing the sprite's movement path; simultaneously, dynamically adjusting the movement speed and dwell time to ensure the user can keep up with the display rhythm; the system also uses eye-tracking technology to detect shifts in user attention and pauses appropriately to wait for the user's gaze to align; and, if necessary, adding guiding markers to help users anticipate the next display location.

[0120] S409. Generate an operation preview animation based on the target device identifier and operation type.

[0121] Referring to step S306, the Home Assistant interactive system will generate an intuitive preview effect based on historical operation data.

[0122] In some embodiments, the home assistant interactive system generates a preview effect based on historical data. Specifically, it obtains the historical control records of the target device and extracts the execution parameters corresponding to the operation type. Based on the execution parameters, it generates a device state change sequence and an assistant action sequence. The state change sequence and the assistant action sequence are combined in chronological order to generate an operation preview animation.

[0123] Among them, historical control records represent the device's operation execution logs and state change data; operation type refers to the specific control action that the user expects to execute; execution parameters represent the specific configuration values ​​and constraints of the control action; device state change sequence refers to the transition process of the device from the current state to the target state; sprite action sequence represents the action combination when the interactive sprite executes the operation; operation preview animation is used to represent the virtual effect preview shown before actual control; timing combination rules refer to the synchronization scheme of state changes and action sequences; and animation frame rate parameters are used to control the smoothness of the preview effect.

[0124] The Home Assistant Interactive System executes this section after determining the target device and operation type. Specifically, the system first accesses the device's historical control database and extracts historical records of the same operation type. Then, it analyzes the distribution of operation parameters in the historical records, including parameter value ranges, common configurations, and constraints. Next, based on the current device state and target parameters, it calculates the transition path for state changes and generates a sequence of key state points. Simultaneously, it selects matching sprite action combinations from the action template library, adjusts the action parameters to synchronize them with the state changes, aligns the timelines of the two sequences, inserts necessary transition frames, and finally synthesizes a smooth pre-show animation. During this process, the physical characteristics of the device and operational safety limitations must also be considered.

[0125] In some embodiments, the generation of pre-show animations can be achieved in multiple ways: Optionally, the Home Assistant interactive system can establish an animation generation scheme based on physical simulation, calculating the real state change process through the mechanical model of the device; automatically adjusting the change speed and transition curve according to physical constraints; coordinating with the physical simulation of the interactive sprite's movements to achieve a natural and smooth operation effect; and adding ambient lighting effects and sound feedback when necessary. Optionally, the Home Assistant interactive system can adopt a data-driven animation generation method, training the animation model through a large number of real operation videos; extracting key features and timing patterns in the operation process; optimizing the animation style by combining user preference data; and supporting real-time adjustment of playback parameters. It is understood that other methods can also be used to generate pre-show animations, and this application does not limit the specific implementation method. It should be noted that when generating pre-show animations, the characteristic differences and operational complexity of different device types also need to be considered.

[0126] During implementation, a mismatch may arise between the pre-show animation and the actual device behavior. For example, the actual response process of some devices involves complex internal state transitions, which are difficult to fully demonstrate in the pre-show. The Home Assistant Interactive System addresses this by: first, constructing a multi-layered state model of the device, distinguishing between visible and internal states; then, highlighting the visible state changes that users care about in the pre-show animation; simultaneously displaying the internal processing through progress bars or status prompts; the system also records the differences between the pre-show effect and the actual behavior, continuously optimizing the state transition model; and, when necessary, providing a tiered pre-show mode, allowing users to choose to view a more detailed state change process.

[0127] S410: Execute the operation preview animation, and after receiving the user confirmation instruction, convert the operation preview animation into a device control instruction and send the device control instruction to the corresponding target device.

[0128] Referring to step S307, the Home Assistant interactive system will convert the pre-rendered actions into actual control commands after user confirmation.

[0129] S411. Record the difference between the playback speed of the operation preview animation and the actual device response speed in the historical control record of the target device.

[0130] Among them, historical control records represent the execution logs of device operations; playback speed refers to the time elapsed rate of the preview animation; actual device response speed represents the actual completion rate of the device's operations; playback response difference value is used to quantify the time difference between preview and actual execution; timestamp accuracy represents the time granularity of the record; and speed matching degree refers to the score of the degree of consistency between the preview speed and the actual speed.

[0131] The Home Assistant Interactive System executes this step after completing the device control operation. Specifically, the Home Assistant Interactive System first records the start and end times of the pre-playback animation and calculates the actual duration of the animation playback; then it monitors the start and end times of changes in device status to obtain the actual execution time of the operation; next, it calculates the time difference between the two and converts the difference value into a standardized speed ratio; at the same time, it records relevant factors such as operation type and environmental conditions; finally, it stores this data in the device's historical control record for subsequent animation optimization.

[0132] In some embodiments, recording and analyzing playback response differences can be achieved in multiple ways: Optionally, the Home Assistant interactive system can establish a multi-layered time recording mechanism, including detailed indicators such as animation rendering time, network transmission latency, and device response time; identify key factors affecting response speed through time series analysis; and establish a response time prediction model. Optionally, the Home Assistant interactive system can adopt a real-time monitoring scheme to continuously collect device status data during animation playback, achieving more accurate time synchronization; including considering the device's acceleration process and buffering time, and fully recording the status change curve. It is understood that other methods can also be used to measure and record response differences, and this application does not limit the specific implementation method.

[0133] During the implementation of the solution, issues such as unstable device response times may be encountered. For example, the execution time of the same operation may fluctuate significantly under different conditions. The Home Assistant Interactive System can address this by: firstly, performing statistical analysis on the response time data to calculate the average and standard deviation; then, setting dynamic difference thresholds based on the degree of fluctuation; simultaneously identifying environmental factors causing response fluctuations and establishing a condition-dependent time model; the system also periodically cleans up abnormal data to maintain the validity of records; and, if necessary, introducing adaptive prediction algorithms to estimate potential response delays in advance.

[0134] S412. When the playback response difference value exceeds the preset deviation threshold, adjust the playback speed of the pre-show animation in the control sequence according to the actual device response speed.

[0135] Among them, the playback response difference value represents the time deviation between the pre-show animation and the actual execution; the preset deviation threshold refers to the maximum allowable time difference range; the pre-show animation playback speed represents the rate factor of animation time elapsed; the control sequence refers to a set of instructions containing multiple consecutive operations; the speed adjustment coefficient represents the correction parameters calculated based on the actual response; and the animation compensation frame rate is used to optimize playback smoothness.

[0136] The Home Assistant interactive system executes this step when it detects a significant difference in playback response. Specifically, the Home Assistant interactive system first compares the current playback response difference value with a preset threshold; when the difference value exceeds the threshold, the system calculates the ratio of the actual device response speed to the current animation playback speed; then, based on this ratio, it generates a speed adjustment coefficient; next, it applies the adjusted playback speed to the timing control of the pre-playback animation; simultaneously, it updates the keyframe timestamps in the animation sequence; finally, it saves the adjusted animation parameters to the device configuration for subsequent pre-playback.

[0137] In some embodiments, the dynamic adjustment of the preview animation speed can be achieved in several ways: Optionally, the Home Assistant interactive system can establish a segmented speed adjustment model, using different playback speeds for different stages of the device response process; this includes setting transition stages such as acceleration, constant speed, and deceleration to achieve a more natural animation effect; and supporting dynamic frame interpolation technology to optimize playback smoothness. Optionally, the Home Assistant interactive system can adopt a predictive adjustment strategy, predicting the device's response trend based on historical data and adjusting the animation speed in advance; this includes establishing a correlation model between device status and response speed to achieve more accurate speed matching. It is understood that other methods can also be used to optimize the animation speed, and this application does not limit the specific implementation method.

[0138] During implementation, the speed of multiple consecutive operation preview animations may need to be coordinated. For example, significant differences in the response characteristics of different devices can lead to inconsistent rhythms in consecutive operations. The Home Assistant Interactive System addresses this by: first, analyzing the dependencies between operations in the control sequence to identify operation groups requiring synchronization; then, calculating the optimal overall playback speed for each operation group to ensure timing coordination between operations; setting synchronization checkpoints at key nodes to ensure synchronization between preview effects and actual execution; the system also learns user habits regarding different operation combinations to establish scenario-based speed adjustment templates; and, when necessary, provides a manual fine-tuning interface, allowing users to adjust the preview rhythm according to their preferences.

[0139] In some embodiments, the Home Assistant interactive system optimizes interactions for frequent operations. Specifically, it records the user's confirmation waiting time during the preview animation playback. If the confirmation waiting time exceeds a first preset time, the playback of the preview animation is canceled. When the number of consecutive cancellations of the same type of preview animation by the user reaches a preset number, the operation type is marked as a quick execution type. When the operation type is a quick execution type, the Home Assistant interactive system skips the playback steps of the preview animation and directly sends device control commands.

[0140] Among them, the confirmation waiting time represents the time interval between the user watching the preview animation and performing the confirmation operation; the first preset time is the time threshold at which the system determines that the user does not need the preview; the preview animation playback represents the process of showing the user a preview of the operation effect; the consecutive cancellation count is used to record the frequency at which the user skips similar previews; the preset number of times is the threshold for the number of cancellations for the quick execution type; the quick execution type represents the type of operation that the user wants to execute directly without a preview; and the device control command refers to the specific control command sent to the device for execution.

[0141] The Home Assistant interactive system executes this segment of content during the preview animation. Specifically, the system first starts the preview animation's playback timer and records the user's viewing time. When the waiting time exceeds a preset threshold, it determines that the user may not need the preview and automatically cancels the current preview animation. Then, it updates the cancellation counter for this operation type and analyzes the user's preview preferences. When the number of consecutive cancellations of a certain type of operation reaches a preset threshold, the operation is marked as a quick execution type. When the same type of operation request is encountered subsequently, the system skips the preview stage and immediately sends the control command. At the same time, it maintains a dynamic classification of operation types, supporting adjustments to the execution strategy based on changes in user habits.

[0142] It's worth noting that the Home Assistant interactive system uses a fast execution type judgment model, which employs a random forest classifier to determine whether an operation requires a pre-show presentation. The model's input features include the exponentially decaying cumulative value of historical cancellations, the statistical distribution of confirmation wait times, and the environmental feature vector of the current scene. The random forest classifies the operation through a voting mechanism of multiple decision trees, with each tree selecting the optimal split point based on the information gain of the features. The model's prediction confidence is calculated using the voting ratio of the trees; the classification result is only executed when the confidence exceeds a threshold. The system periodically updates the model with new user operation data to maintain the classifier's adaptability to changes in user habits. For example, if a user repeatedly cancels the pre-show animation when turning on the living room light, the model will learn to label this type of operation as a fast execution type.

[0143] In this embodiment, by employing virtual space-based interactive sprite visualization technology and combining it with an adaptive control strategy based on device usage characteristics, the system provides users with an intuitive device selection and operation preview experience, while dynamically optimizing interaction parameters. The system automatically adjusts the interactive sprite's movement speed, waiting time, and prompting methods by analyzing device usage frequency, response latency, and usage time characteristics, and establishes a rapid execution mechanism based on user operating habits. This solution effectively solves problems such as unclear device selection, unpredictable operation effects, and inconsistent interaction experiences in traditional voice control, thereby achieving a more natural and fluid smart home control experience. By introducing pre-rendering animations and dynamic parameter adjustments, the system ensures both the accuracy and predictability of operation while providing a personalized interactive experience, improving user efficiency and satisfaction. In practical applications, this solution also has good scalability and can be optimized according to different scenarios and user needs, providing a new solution for human-computer interaction in smart homes.

[0144] The following describes the home automation system in the embodiments of this invention from the perspective of hardware processing. Please refer to [link / reference]. Figure 5 This is a schematic diagram of the physical device structure of the Home Assistant Interactive System in this application embodiment.

[0145] It should be noted that, Figure 5 The structure of the Home Assistant interactive system shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0146] like Figure 5 As shown, the home automation interactive system includes a CPU 501, which can perform various appropriate actions and processes according to a program stored in ROM 502 or a program loaded into RAM 503 from storage section 508, such as executing the methods described in the above embodiments. RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via bus 504. I / O interface 505 is also connected to bus 504.

[0147] The following components are connected to I / O interface 505: input section 506 including audio input devices, push-button switches, etc.; output section 507 including liquid crystal display (LCD) and audio output devices, indicator lights, etc.; storage section 508 including hard disks, etc.; and communication section 509 including network interface cards such as LAN (Local Area Network) cards, modems, etc. Communication section 509 performs communication processing via a network such as the Internet. Drive 510 is also connected to I / O interface 505 as needed. Removable media 511, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on drive 510 as needed so that computer programs read from them can be installed into storage section 508 as needed.

[0148] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by CPU 501, it performs the various functions defined in the present invention.

[0149] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings.

[0150] Specifically, the home smart interaction system of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, it implements the smart home interaction method provided in the above embodiment.

[0151] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the home smart interaction system described in the above embodiments; or it may exist independently and not assembled into the home smart interaction system. The storage medium carries one or more computer programs, which, when executed by a processor of the home smart interaction system, enable the home smart interaction system to implement the smart home interaction method provided in the above embodiments.

[0152] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0153] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

Claims

1. A smart home interaction method, characterized in that, The method, applied to a home automation interactive system, includes: Receive user voice commands and determine the operation type and target location descriptor through voice recognition; Based on the target location descriptor, a location mapping is performed in a pre-established 3D virtual space of the home to obtain candidate device identifiers located at the positions corresponding to the target location descriptor; When the number of candidate device identifiers is greater than 1, calculate the relevance score between each candidate device and the user's historical interaction records, and prioritize the candidate devices based on the relevance scores to generate a candidate device sequence; Based on the candidate device sequence, an option display trajectory is generated for the interactive sprite in the 3D virtual space of the home; the option display trajectory includes a set of movement path points of the interactive sprite in the 3D virtual space of the home and pointing action parameters corresponding to each candidate device; Based on the displayed trajectory of the options, the interactive sprite is controlled to move sequentially to the positions of each candidate device in the three-dimensional virtual space of the home and perform pointing actions until it receives the user's device selection instruction and determines the target device identifier. An operation preview animation is generated based on the target device identifier and the operation type; the operation preview animation includes the displacement path of the interactive sprite, operation action parameters, and device state change parameters; The operation preview animation is executed, and after receiving the user's confirmation instruction, the operation preview animation is converted into a device control instruction and sent to the corresponding target device.

2. The method according to claim 1, characterized in that, The step of controlling the interactive sprite to move sequentially to the positions of each candidate device in the three-dimensional virtual space of the home, based on the displayed trajectory of the options and to perform pointing actions, until the user's device selection instruction is received, and to determine the target device identifier, specifically includes: Based on the displayed trajectory of the options, control the interactive sprite to move sequentially to the positions of each candidate device in the three-dimensional virtual space of the home and perform pointing actions; When the interactive sprite moves to each of the candidate devices, a device identification prompt message is played, and a timer with a preset waiting time is started; Upon receiving the user's selection confirmation instruction, the candidate device currently selected by the interactive wizard is identified as the target device identifier; If the timer expires without receiving a user instruction, the interactive sprite continues to move to the next candidate device location; If no user confirmation command is received after traversing all candidate devices, the interactive sprite is controlled to return to the candidate device with the highest relevance score and wait for the user to select it.

3. The method according to claim 1, characterized in that, The step of generating an operation preview animation based on the target device identifier and the operation type specifically includes: Obtain the historical control records of the target device and extract the execution parameters corresponding to the operation type; Generate a device state change sequence and a sprite action sequence based on the execution parameters; The state change sequence and the sprite action sequence are combined in chronological order to generate an operation preview animation.

4. The method according to claim 1, characterized in that, After the step of performing location mapping in a pre-established 3D virtual space of the home based on the target location descriptor to obtain the candidate device identifier located at the position corresponding to the target location descriptor, the method further includes: Obtain user interaction records and device operation records for each candidate device; Based on the user interaction records and the device operation records, extract the device usage time period, usage frequency, and response latency characteristics; The movement speed, waiting time, and prompting method of the interactive sprite are adjusted based on the device usage period, usage frequency, and response latency characteristics.

5. The method according to claim 4, characterized in that, The step of adjusting the movement speed, waiting time, and prompting method of the interactive sprite based on the device usage period, usage frequency, and response latency characteristics specifically includes: When the frequency of device use exceeds a preset frequency threshold, the movement speed of the interactive sprite is slowed down, and the action time for pointing at the device is extended. The waiting time of the interactive wizard is set according to the device's response latency characteristics. When the device's response latency is detected to exceed the historical average response latency, the waiting time of the interactive wizard is extended. Different prompt intensities are selected based on the time period during which the device is used.

6. The method according to claim 1, characterized in that, After the steps of executing the operation preview animation, converting the operation preview animation into a device control command after receiving a user confirmation command, and sending the device control command to the corresponding target device, the method further includes: Record the difference between the playback speed of the operation preview animation and the actual device response speed in the historical control records of the target device; When the playback response difference value exceeds a preset deviation threshold, the playback speed of the pre-show animation in the control sequence is adjusted according to the actual device response speed.

7. The method according to claim 6, characterized in that, After the step of adjusting the playback speed of the pre-show animation in the control sequence according to the actual device response speed, the method further includes: Record the user's confirmation waiting time during the preview animation playback. If the confirmation waiting time exceeds a first preset time, cancel the playback of the preview animation. When the user cancels the preview animation of the same operation type a preset number of times, the operation type is marked as a quick execution type; when the operation type is the quick execution type, the home assistant interactive system will skip the playback step of the preview animation and directly send device control commands.

8. A home automation interactive system, characterized in that, The Home Assistant interactive system includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the Home Assistant interactive system to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that, When the instruction is executed on the Home Assistant interactive system, the Home Assistant interactive system performs the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, When the computer program product is run on the Home Assistant interactive system, the Home Assistant interactive system performs the method as described in any one of claims 1-7.

Images