Method and device for generating smart butler, electronic device, storage medium

Abstract

The application relates to the technical field of smart homes, and discloses a method for generating a smart housekeeper, which comprises the following steps: constructing a model of a virtual housekeeper; generating a virtual data space; the virtual data space is used for feeding back the state of an entity home appliance in a preset area; the model of the virtual housekeeper is integrated into the virtual digital space to obtain a smart housekeeper. The smart housekeeper is a digital virtual person. The model of the virtual housekeeper is constructed, then a virtual data space for feeding back the state of an entity home appliance in a preset area is generated, and the model of the virtual housekeeper is integrated into the virtual digital space to obtain a smart housekeeper. In this way, a user can know the state of each entity home appliance in the preset area through the smart housekeeper, and does not need to go to the front of the home appliance to be known to check, so that the use experience of the user is improved. The application further discloses an apparatus for generating a smart housekeeper, an electronic device and a storage medium.

Description

Technical Field

[0001] This application relates to the field of smart home technology, such as a method and apparatus for generating a smart home manager, electronic devices, and storage media. Background Technology

[0002] Currently, with the development of IoT technology, users' demand for smart home systems is gradually increasing. Initially, users controlled smart home appliances via manual buttons or remote controls. Now, users control appliances via voice commands. However, regardless of the control method, once the status of an appliance changes due to control, users cannot accurately grasp the status of that appliance or other appliances. To ensure the normal operation of appliances, users must physically go to the appliance they want to check. This results in a poor user experience with smart home systems.

[0003] In implementing the embodiments of this disclosure, it was found that at least the following problems exist in the related technology: users must walk up to the home appliance they want to check in order to understand its status. This results in a poor user experience.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0006] This disclosure provides a method, apparatus, electronic device, and storage medium for generating a smart home manager, thereby improving the user experience.

[0007] In some embodiments, the method for generating a smart butler includes: constructing a model of a virtual butler; generating a virtual data space; the virtual data space being used to provide feedback on the status of physical home appliances within a preset area; and integrating the model of the virtual butler into the virtual digital space to obtain a smart butler; the smart butler is a digital virtual human.

[0008] In some embodiments, constructing the virtual butler model includes: establishing a three-dimensional model of the virtual butler; determining the actions of the virtual butler; binding preset semantics to the actions of the virtual butler; and combining the semantically bound actions with the three-dimensional model to obtain the virtual butler model.

[0009] In some embodiments, generating the virtual data space includes: acquiring a real-world home scene; the real-world home scene includes multiple physical home appliances. Modeling each object and the home space structure in the real-world home scene based on the real-world home scene to obtain a spatial geometric model. The spatial geometric model contains virtual home appliances corresponding to each of the physical home appliances. The home space structure is obtained from the real-world home scene. Establishing a state recognition model corresponding to each of the physical home appliances; the state recognition model is used to identify the state of each of the physical home appliances. Binding each virtual home appliance to its corresponding state recognition model to obtain a virtual data space. In the virtual data space, the state of the virtual home appliance is the same as the state of its corresponding physical home appliance.

[0010] In some embodiments, after obtaining the smart butler, the process further includes: controlling the smart butler to interact with the user.

[0011] In some embodiments, controlling the smart home assistant to interact with the user includes: determining the user's real-time location when it is determined that the user has returned home; waking up the smart home assistant on the nearest physical home appliance with a screen to the user's real-time location; acquiring the user's voice; and controlling the smart home assistant to communicate with the user based on the user's voice.

[0012] In some embodiments, after waking up the smart home manager, the method further includes: controlling the smart home manager to identify the status of each physical home appliance in the virtual data space; and controlling the smart home manager to broadcast an abnormal status when it detects an abnormal status of the physical home appliance.

[0013] In some embodiments, the device for generating a smart butler includes: a construction module configured to construct a model of a virtual butler; a generation module configured to generate a virtual data space; the virtual data space being used to provide feedback on the status of physical home appliances within a preset area; and an integration module configured to integrate the model of the virtual butler into the virtual digital space to obtain a smart butler; the smart butler is a digital virtual human.

[0014] In some embodiments, the apparatus for generating a smart home includes a processor and a memory storing program instructions, the processor being configured to execute the method for generating a smart home as described above when the program instructions are executed.

[0015] In some embodiments, the electronic device includes: an electronic device body; and the aforementioned device for generating a smart home assistant is installed on the electronic device body.

[0016] In some embodiments, the storage medium stores program instructions that, when executed, perform the method described above for generating a smart home manager.

[0017] The method, apparatus, electronic device, and storage medium for generating a smart home manager provided in this disclosure can achieve the following technical effects: By constructing a model of a virtual home manager, a virtual data space is generated to provide feedback on the status of physical home appliances within a preset area. The model of the virtual home manager is integrated into the virtual digital space to obtain a smart home manager. In this way, because the virtual digital space of the smart home manager can provide feedback on the status of physical home appliances within the preset area, users can understand the status of each physical home appliance within the preset area through the smart home manager without having to physically go to the specific appliance to check, thus improving the user experience.

[0018] The above general description and the description below are exemplary and illustrative only and are not intended to limit this application. Attached Figure Description

[0019] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are shown as similar elements. The drawings are not to be scaled. And wherein:

[0020] Figure 1 This is a schematic diagram of a method for generating a smart home assistant provided in an embodiment of this disclosure;

[0021] Figure 2 This is a schematic diagram of another method for generating a smart home assistant provided in an embodiment of this disclosure;

[0022] Figure 3-1 This is a schematic diagram of the smart home assistant's cross-screen front screen display provided in an embodiment of this disclosure;

[0023] Figure 3-2 This is a schematic diagram of the screen display during the cross-screen process of the smart butler provided in this embodiment of the disclosure;

[0024] Figure 3-3 This is a schematic diagram of the screen display after the smart butler crosses screens, as provided in an embodiment of this disclosure;

[0025] Figure 4 This is a schematic diagram of a method for controlling the interaction of a smart home assistant, provided in an embodiment of this disclosure;

[0026] Figure 5 This is a schematic diagram of a device for generating a smart home assistant, provided in an embodiment of this disclosure;

[0027] Figure 6This is a schematic diagram of another device for generating a smart home assistant provided in an embodiment of this disclosure;

[0028] Figure 7 This is a schematic diagram of an electronic device provided in an embodiment of this disclosure;

[0029] Figure 8 This is a schematic diagram of another electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0030] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0031] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0032] Unless otherwise stated, the term "multiple" means two or more.

[0033] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0034] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0035] The term "correspondence" can refer to an association or binding relationship. The correspondence between A and B means that there is an association or binding relationship between A and B.

[0036] The relay communication method provided in this disclosure is applied to an electronic device. The electronic device constructs a virtual butler model, generates a virtual data space for feedback on the status of physical home appliances within a preset area, and then integrates the virtual butler model into the virtual digital space to obtain a smart butler. In this way, because the smart butler's virtual digital space can provide feedback on the status of physical home appliances within the preset area, users can understand the status of each physical home appliance within the preset area through the smart butler without having to physically go to the desired appliance, thus improving the user experience.

[0037] In some embodiments, the electronic device serves as the central control unit for a smart home system. For example, it may be a server that controls the smart home system. This central control unit is capable of controlling physical home appliances within the house.

[0038] Combination Figure 1 As shown in the embodiments of this disclosure, a method for generating a smart home manager is provided, including:

[0039] Step S101: The electronic device constructs a model of a virtual butler.

[0040] In step S102, the electronic device generates a virtual data space. This virtual data space is used to provide feedback on the status of physical home appliances within a preset area. The preset area is the user's house.

[0041] In step S103, the electronic device integrates the virtual butler model into the virtual digital space to obtain an intelligent butler. This intelligent butler is a digital virtual human.

[0042] The method for generating a smart home manager provided in this disclosure involves constructing a virtual home manager model and then generating a virtual data space to provide feedback on the status of physical home appliances within a preset area. The virtual home manager model is then integrated into this virtual digital space to obtain the smart home manager. Because the smart home manager's virtual digital space can provide feedback on the status of physical home appliances within the preset area, users can easily understand the status of each appliance without having to physically visit the specific appliance, thus improving the user experience.

[0043] Optionally, a virtual butler model is constructed, including: creating a 3D model of the virtual butler; determining the actions of the virtual butler; binding preset semantics with the actions of the virtual butler; and combining the semantically bound actions with the 3D model to obtain the virtual butler model. This allows for the creation of a virtual butler model capable of performing different actions based on semantics, ensuring that the final intelligent butler can also perform different actions based on semantics. This improves the user's interactive experience.

[0044] Furthermore, a 3D model of the virtual butler is established, including: acquiring the virtual butler's 3D data; and creating a 3D model of the virtual butler based on this data. The 3D data of the virtual butler is used to characterize its physical features, such as one or more of the following: height, chest circumference, waist circumference, hip circumference, hair color, hair length, skin tone, clothing, facial features, body proportions, and facial expressions.

[0045] Furthermore, acquiring the virtual butler's 3D data includes: acquiring the 3D data of the target person as a reference. In some embodiments, the target person is a real person customized by the user or preset by the manufacturer.

[0046] Furthermore, modeling is performed based on the 3D data to obtain a 3D model of the virtual butler, including: using a preset first modeling tool to model based on the 3D data to obtain a 3D model of the virtual butler. In some embodiments, the preset modeling tool includes one or more of Mirauge3D, Smart3D, UnityArtEngine, Autodesk-Maya, Mudbox, and Zbrush.

[0047] Furthermore, after modeling the 3D data using preset modeling tools, the process also includes rendering the feature details of the 3D model using a preset rendering engine. The preset rendering engines include Unity 3D and / or Unreal Engine, among others. The feature details of the 3D model include the texture details of clothing, hair, facial features, and skin. This results in a hyper-realistic virtual butler model, allowing the intelligent butler created by integrating this model into the virtual digital space to closely resemble a real person.

[0048] Furthermore, the actions of the virtual butler are determined, including: determining the basic actions of the virtual human through motion capture of the target person.

[0049] Furthermore, the preset semantics are bound to the actions of the virtual butler, including: establishing a correspondence between the actions of the virtual butler and the preset semantics; and binding the preset semantics to their corresponding actions of the virtual butler. The preset semantics include preset user semantics and preset butler semantics. User semantics is used to represent the semantics in the user's speech. Butler semantics is used to represent the semantics in the virtual butler's speech.

[0050] In some embodiments, the preset semantics are preset user semantics. When the user semantics are used to represent wake-up, a correspondence is established between the action of opening the eyes and the user semantics used to represent wake-up, and then the action of opening the eyes is bound to the user semantics used to represent wake-up.

[0051] In some embodiments, the preset semantics are preset butler semantics. When the butler semantics is used to represent the emotion of happiness, a correspondence is established between the jumping action and the butler semantics used to represent the emotion of happiness. Then, the jumping action is bound to the butler semantics used to represent the emotion of happiness.

[0052] Furthermore, the action with bound semantics is combined with the 3D model to obtain the model of the virtual butler, including: rendering the 3D model with the action with bound semantics to obtain the model of the virtual butler.

[0053] Furthermore, after obtaining the virtual butler model, the process includes: semantically training the virtual butler model using a pre-defined speech semantic recognition algorithm. This allows the final intelligent butler to parse different speech and then provide the corresponding actions based on the parsed semantics. Thus, the final intelligent butler can display different actions in different semantic contexts.

[0054] Optionally, generating a virtual data space includes: acquiring a real-world home scene; the real-world home scene includes multiple physical home appliances. Modeling each object and the home space structure based on the real-world home scene yields a spatial geometric model. The spatial geometric model contains virtual home appliances corresponding to each physical home appliance. The home space structure is obtained from the real-world home scene. Establishing a state recognition model for each physical home appliance. This state recognition model is used to identify the state of each physical home appliance. Binding each virtual home appliance to its corresponding state recognition model yields a virtual data space. In this virtual data space, the state of the virtual home appliance is the same as the state of its corresponding physical home appliance. Thus, the virtual data space can be generated based on the real-world home scene, enabling the virtual data space to respond to objects within the real-world home scene.

[0055] Furthermore, the home reality scene is captured using a 3D (3D) camera. The home reality scene represents the actual view of the user's house. Therefore, it can reflect the spatial structure of the user's house and the various objects within it. These objects include multiple pieces of furniture, multiple appliances, and decorative items. This allows the spatial geometric model obtained by modeling the objects and spatial structure of the home reality scene to reflect the actual layout of the user's house.

[0056] Furthermore, based on the real-world home environment, models are created for each object and the spatial structure to obtain a spatial geometric model. This includes: using a pre-set second modeling tool to perform multi-layered modeling of each object in the home environment, obtaining multiple object models. These object models represent the shape of the objects and their positions within the room. The pre-set second modeling tool includes one or more of Mirauge3D, Smart3D, and / or UnityArtEngine. A pre-set third modeling tool is then used to construct a digital space virtual scene based on the home spatial structure using these multiple object models, obtaining a digital space virtual scene model. Finally, the digital space virtual scene model is improved to obtain a spatial geometric model.

[0057] Furthermore, the digital space virtual scene model is improved, including: modifying the parts of the digital space virtual scene model that differ from the real home scene; re-rendering the colors and lighting in the digital space virtual scene model that differ from the real home scene; and optimizing the modified and rendered digital space virtual scene model as a whole to make the optimized virtual space scene model closer to the real home scene.

[0058] Furthermore, establish a status recognition model for each physical household appliance, including: establishing a status recognition model based on the type, location, shape, and / or operating parameters of each physical household appliance.

[0059] In some embodiments, when the physical home appliance is a smart light, a state recognition model corresponding to the smart light is established based on the smart light's operating parameters. These operating parameters include the smart light's on / off status, color temperature, and brightness. Multiple image samples of the smart light under different state parameters are acquired. State parameter samples of the smart light are extracted from the image samples using a preset deep learning framework to generate the state recognition model. The preset deep learning framework includes PaddlePaddle, Tensorflow, Caffe (Convolutional Architecture for Fast Feature Embedding), Theano, MXNet, Torch, or PyTorch. This enables the obtained state recognition model to accurately identify the state of the smart light, namely its on / off status, color temperature, and brightness.

[0060] In some embodiments, when the physical household appliance is a robotic vacuum cleaner, a state recognition model corresponding to the robotic vacuum cleaner is established based on the robotic vacuum cleaner's operating parameters and location. The robotic vacuum cleaner's operating parameters include cleaning time, remaining battery power, etc. This enables the obtained state recognition model to accurately identify the state of the robotic vacuum cleaner, i.e., its operating parameters and location.

[0061] Furthermore, the virtual butler model is integrated into the virtual digital space to obtain an intelligent butler. This includes: integrating the virtual butler model into the virtual digital space using a preset graphical application; and simultaneously driving the virtual butler model using a preset driving engine. This enables the virtual butler model to move within the virtual data space, creating an intelligent butler capable of interacting with the user.

[0062] The pre-defined graphics application includes OpenGL ES (Open Graphics Library for Embedded Systems). Since OpenGL ES is compatible with various 3D model files, the smart butler is displayed using a 3D model display framework within OpenGL ES. The 3D model display framework first draws the virtual butler's model and virtual data space, and then displays the drawn virtual butler model and virtual data space. If the virtual data space is a model trained based on the TensorFlow Lite framework, the 3D model display framework runs the spatial geometry model, and TensorFlow Lite is used to recognize the model's running state, thus realizing the operation of the virtual data space.

[0063] Combination Figure 2 As shown in the embodiments of this disclosure, a method for generating a smart home manager is provided, including:

[0064] Step S201: The electronic device establishes a 3D model of the virtual butler.

[0065] In step S202, the electronic device determines the virtual manager's actions.

[0066] In step S203, the electronic device binds the preset semantics with the actions of the virtual butler.

[0067] In step S204, the electronic device combines the semantically bound actions with the 3D model to obtain the model of the virtual butler.

[0068] Step S205: The electronic device acquires a real-world view of the home. This real-world view includes multiple physical home appliances.

[0069] In step S206, the electronic device models the objects and spatial structure of the home environment based on the real-world home scene, obtaining a spatial geometric model. This spatial geometric model contains virtual home appliances corresponding to each physical appliance. The home spatial structure is obtained from the real-world home scene.

[0070] Step S207: The electronic device establishes a state recognition model corresponding to each physical home appliance. The state recognition model is used to identify the state of each physical home appliance.

[0071] In step S208, the electronic device binds each virtual home appliance to its corresponding state recognition model to obtain a virtual data space. In this virtual data space, the state of the virtual home appliance is the same as the state of its corresponding physical home appliance.

[0072] In step S209, the electronic device integrates the virtual butler model into the virtual digital space to obtain an intelligent butler. This intelligent butler is a digital virtual human.

[0073] The method for generating a smart home assistant provided in this disclosure involves binding preset semantics with the actions of a virtual home assistant, and then combining the semantically bound actions with a 3D model of the virtual home assistant to obtain a model of the virtual home assistant. Next, models are created based on objects and the spatial structure of the home environment, and state recognition models are established for each physical home appliance. Each virtual home appliance is then bound to its corresponding state recognition model to obtain a virtual data space. Finally, the model of the virtual home assistant is integrated into the virtual digital space to obtain the smart home assistant. This allows the smart home assistant to perform different actions based on semantics in the virtual digital space while simultaneously providing feedback on the status of physical home appliances within a preset area, improving the user experience and interaction.

[0074] Optionally, after obtaining the smart butler, the system also includes controlling the smart butler to interact with the user. This allows users to experience the pleasure of interacting with a real person, enhancing their interactive experience.

[0075] Optionally, the smart home assistant can be controlled to interact with the user, including: determining the user's real-time location upon confirming the user has returned home; waking up the smart home assistant on the nearest physical appliance with a screen to the user's real-time location; acquiring the user's voice; and controlling the smart home assistant to communicate with the user based on the voice. This allows the smart home assistant to move according to the user's real-time location, enabling seamless dialogue and communication with the user, thus improving the user's interactive experience.

[0076] Furthermore, the user's return home is determined by the following method: upon receiving the door opening information sent by the smart door lock and the light turning on information sent by the smart light, the user is confirmed to have returned home.

[0077] Furthermore, upon confirming that the user has returned home, the system also includes: activating preset home-return scenarios to configure preset physical home appliances. For example, one or more smart lights, smart curtains, and smart air conditioners can be configured to adjust the light and temperature inside the house. This allows users to organize their home life efficiently based on their habits.

[0078] Furthermore, determining the user's real-time location includes: capturing images of the user using camera devices in different areas; and determining the area of ​​the camera device that captured the user's image as the user's real-time location. Alternatively, capturing the user's voice using microphones in different areas; and determining the area of ​​the microphone that captured the user's voice as the user's real-time location. Or, capturing the user's voice using microphones in different areas. Based on the user's voice captured by each microphone, the distance between the user and each microphone is determined to determine the user's location within the building.

[0079] Furthermore, waking up the smart home assistant on the physical appliance with a screen closest to the user's real-time location includes: if no physical appliance with a screen in the house can wake up the smart home assistant, waking it up directly on the physical appliance with a screen closest to the user's real-time location; and / or, if a physical appliance with a screen in the house can wake up the smart home assistant, controlling the smart home assistant to perform cross-screen operations between that physical appliance and the physical appliance with a screen closest to the user's real-time location, so as to wake up the smart home assistant on the physical appliance with a screen closest to the user's real-time location.

[0080] Furthermore, if there are no screen-equipped physical appliances in the house to wake up the smart home manager, waking up the smart home manager directly from the screen-equipped physical appliance closest to the user's real-time location also includes: broadcasting preset information in a broadcasting manner. For example, broadcasting the day's visitors, news, weather, and the status of various physical appliances in the preset area.

[0081] In some embodiments, such as Figures 3-1 to 3-3 As shown, Figure 3-1 This is a schematic diagram of the front screen display for the smart home assistant across screens. Figure 3-2 This is a diagram illustrating the screen display during the cross-screen process of the smart home assistant. Figure 3-3 This is a diagram illustrating the screen display after the smart home assistant crosses screens. Before the cross-screen transition, as shown below... Figure 3-1 As shown, when the user is closest to the first physical home appliance with a screen (1), the electronic device controls the activation of the smart home manager on the first physical home appliance with a screen (1). The smart home manager then appears on the first screen (3) of the first physical home appliance with a screen (1). The smart home manager is not displayed on the second screen (4) of the second physical home appliance with a screen (2). After the user moves, they become closest to the second physical home appliance with a screen (2). Then... Figure 3-2As shown, the electronic device controls the smart home assistant to perform cross-screen operations between the first physical home appliance with a screen 1 and the second physical home appliance with a screen 2. After the cross-screen operation, as shown... Figure 3-3 As shown, the smart assistant appears on the second screen 4 of the second physical home appliance 2, while the smart assistant on the first screen 1 of the first physical home appliance 1 has disappeared. In this way, the smart assistant can follow the user from the first screen of the first physical home appliance to the second screen of the second physical home appliance, allowing the smart assistant to cross screens to the closest smart device to the user and continue face-to-face communication, thus improving the user's interactive experience.

[0082] Furthermore, the intelligent butler can communicate with the user based on their voice, including: using a preset voice semantic recognition algorithm to perform semantic recognition on the user's voice; and executing corresponding actions based on the recognized semantic feedback. This enables communication between the user and the intelligent butler. Simultaneously, through continuous training of the voice semantic recognition algorithm, the virtual butler can grow and improve along with the duration of user interactions, providing a nurturing experience for the user.

[0083] Optionally, after waking up the smart home manager, the system also includes: controlling the smart home manager to identify the status of each physical home appliance in the virtual data space; and controlling the smart home manager to broadcast abnormal status when abnormal status of a physical home appliance is detected. This allows the system to broadcast abnormal status of physical home appliances to the user, improving home security.

[0084] Furthermore, the smart home manager is controlled to identify the status of each physical home appliance in the virtual data space, including: the smart home manager is controlled to identify the status of each physical home appliance using the status recognition model corresponding to each physical home appliance.

[0085] Furthermore, after the smart home manager identifies the state of each physical home appliance using its corresponding state recognition model, it also includes updating the state of each virtual home appliance in the spatial geometric model of the virtual data space to match the state of the physical home appliances. This ensures that the state of the virtual home appliances is the same as the state of their corresponding physical home appliances.

[0086] Furthermore, after waking up the smart home assistant, it also includes: when the physical home appliance with a screen is set to sleep mode, controlling the smart home assistant not to appear in the physical home appliance with a screen.

[0087] Furthermore, after waking up the smart home assistant, the system also includes: when it is determined that the user has left home, controlling the smart home assistant to activate the "away mode" and execute preset away scenarios. For example, controlling the smart home assistant to turn off all physical home appliances in a preset area, and the smart home assistant entering a wake-up mode, waiting to be woken up again.

[0088] Furthermore, the user's departure from home is determined by the following method: upon receiving a door-closing notification from the smart lock and a light-off notification from the smart light, the user is confirmed to have left home.

[0089] Furthermore, after waking up the smart home assistant, it also includes: if the user does not interact with the smart home assistant, controlling the smart home assistant to entertain itself on physical home appliances with screens. For example, controlling the smart home assistant to sing or dance.

[0090] Combination Figure 4 As shown in the embodiments of this disclosure, a method for controlling the interaction of a smart home assistant is provided, including:

[0091] In step S301, the electronic device determines the user's real-time location if it determines that the user has returned home.

[0092] In step S302, the electronic device wakes up the smart home assistant on the physical home appliance with a screen closest to the user's real-time location. The physical home appliance with a screen is equipped with a smart home assistant, which is a digital virtual human.

[0093] Step S303: The electronic device acquires the user's voice.

[0094] Step S304: The electronic device controls the smart home assistant to communicate with the user based on the user's voice.

[0095] Step S305: The electronic device controls the smart home manager to identify the status of each physical home appliance in the virtual data space.

[0096] Step S306: When the electronic device controls the smart home manager to broadcast an abnormal status when it detects an abnormal status of the physical home appliance.

[0097] The method for controlling smart home interaction provided in this disclosure wakes up the smart home assistant based on the user's real-time location when the user is confirmed to have returned home. This allows the smart home assistant to follow the user and interact with them at close range, improving the user's interactive experience. Furthermore, because it can follow the user, it can promptly alert the user to any malfunctions in physical home appliances, facilitating quick resolution of the problem and enhancing user safety.

[0098] Combination Figure 5As shown in the figure, this disclosure provides a device 5 for generating a smart home manager, including a construction module 6, a generation module 7, and an integration module 8. The construction module 6 is configured to construct a model of the virtual home manager. The generation module 7 is configured to generate a virtual data space. The virtual data space is used to provide feedback on the status of physical home appliances within a preset area. The integration module 8 is configured to integrate the model of the virtual home manager into the virtual digital space to obtain a smart home manager. The smart home manager is a digital virtual human.

[0099] The apparatus for generating a smart home manager provided in this embodiment constructs a virtual home manager model and then generates a virtual data space to provide feedback on the status of physical home appliances within a preset area. The virtual home manager model is integrated into this virtual digital space to obtain the smart home manager. Thus, because the virtual digital space of the smart home manager can provide feedback on the status of physical home appliances within the preset area, users can understand the status of each physical home appliance within the preset area through the smart home manager without having to physically visit the specific appliance, thereby improving the user experience.

[0100] Optionally, the construction module is configured to build the virtual butler model by: creating a 3D model of the virtual butler; determining the actions of the virtual butler; binding preset semantics to the actions of the virtual butler; and combining the semantically bound actions with the 3D model to obtain the virtual butler model.

[0101] Optionally, the generation module is configured to generate a virtual data space by: acquiring a real-world home scene, which includes multiple physical home appliances; modeling each object and the home space structure based on the real-world scene to obtain a spatial geometric model; containing virtual home appliances corresponding to each physical home appliance within the spatial geometric model; the home space structure being obtained from the real-world scene; establishing a state recognition model for each physical home appliance; and binding each virtual home appliance to its corresponding state recognition model to obtain a virtual data space, wherein the state of the virtual home appliance in the virtual data space is the same as the state of its corresponding physical home appliance.

[0102] Optionally, the device for generating a smart butler also includes a control module. The control module is configured to control the smart butler to interact with the user after the smart butler is acquired.

[0103] Optionally, the control module is configured to control the smart home assistant to interact with the user by: determining the user's real-time location upon confirming the user has returned home; waking up the smart home assistant on the nearest physical appliance with a screen to the user's real-time location; the smart home assistant being installed on the physical appliance with a screen; acquiring the user's voice; and controlling the smart home assistant to communicate with the user based on the user's voice.

[0104] Optionally, the device for generating the smart home manager also includes a broadcast module. The broadcast module is configured to, after waking up the smart home manager, control the smart home manager to identify the status of each physical home appliance in the virtual data space. When the smart home manager detects an abnormal status of a physical home appliance, it broadcasts the abnormal status.

[0105] The apparatus for generating a smart home manager provided in this embodiment constructs a virtual home manager model and then generates a virtual data space to provide feedback on the status of physical home appliances within a preset area. The virtual home manager model is integrated into this virtual digital space to obtain the smart home manager. Thus, because the virtual digital space of the smart home manager can provide feedback on the status of physical home appliances within the preset area, users can understand the status of each physical home appliance within the preset area through the smart home manager without having to physically visit the specific appliance, thereby improving the user experience.

[0106] In some embodiments, virtual technologies such as AR (Augmented Reality) and VR (Virtual Reality) and smart home systems are areas that have attracted considerable attention. While existing smart home systems are actively improving user experience and device management intelligence, and enhancing UI (User Interface) rendering, they still haven't broken free from traditional thinking patterns or disrupted user operating habits; their interaction and operation with users are relatively conventional. Digital virtual human technology, however, is a complex technological system requiring collaboration across multiple disciplines and fields. For example, externally, digital virtual humans require character modeling, speech synthesis technology, animation generation, and computer animation rendering technology; internally, it requires artificial intelligence or human drivers, involving technologies such as 3D modeling, motion capture, real-time rendering, and deep learning. Currently, existing digital virtual humans are mainly used in television, e-commerce, virtual anchors, and live-streaming sales. By integrating the two technologies—virtual technology and smart home systems—which are distinct fields, this application aims to bring a completely new experience and usage to users and the industry. Furthermore, this application integrates digital virtual humans into smart homes to create intelligent butlers. By combining voice and semantic recognition models, intelligent communication between users and smart home assistants can be achieved, overcoming the rigid and indirect communication between users and machines. The smart home assistant can also serve as a visual interface for interaction between the virtual and real worlds, providing companionship and human-like support. Furthermore, it can assist users in managing physical home appliances, monitoring their status, and offering a unique and enjoyable experience that conventional interaction methods cannot provide.

[0107] Combination Figure 6As shown, this disclosure provides an apparatus 9 for generating a smart home manager, including a processor 10 and a memory 11. Optionally, the apparatus may further include a communication interface 12 and a bus 13. The processor 10, communication interface 12, and memory 11 can communicate with each other via the bus 13. The communication interface 12 can be used for information transmission. The processor 10 can call logical instructions in the memory 11 to execute the method for generating a smart home manager described in the above embodiments.

[0108] Furthermore, the logical instructions in the aforementioned memory 11 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0109] The memory 11, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor 10 executes functional applications and data processing by running the program instructions / modules stored in the memory 11, that is, it implements the method for generating a smart home in the above embodiments.

[0110] The memory 11 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 11 may include high-speed random access memory and may also include non-volatile memory.

[0111] Combination Figure 7 As shown, this disclosure provides an electronic device 14, including: an electronic device body, and the aforementioned device 5 for generating a smart home manager. The device 5 for generating a smart home manager is installed in the electronic device body. The installation relationship described herein is not limited to placement inside the electronic device, but also includes installation connections with other components of the electronic device, including but not limited to physical connections, electrical connections, or signal transmission connections. Those skilled in the art will understand that the device 5 for generating a smart home manager can be adapted to feasible electronic device bodies to achieve other feasible embodiments.

[0112] Combination Figure 8As shown, this disclosure provides an electronic device 14, including: an electronic device body, and the aforementioned device 9 for generating a smart home manager. The device 9 for generating a smart home manager is installed in the electronic device body. The installation relationship described herein is not limited to placement inside the electronic device, but also includes installation connections with other components of the electronic device, including but not limited to physical connections, electrical connections, or signal transmission connections. Those skilled in the art will understand that the device 9 for generating a smart home manager can be adapted to feasible electronic device bodies to achieve other feasible embodiments.

[0113] This disclosure provides a computer-readable storage medium storing computer-executable instructions configured to perform the above-described method for generating a smart home manager.

[0114] The aforementioned computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0115] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this disclosure. The aforementioned storage medium can be a non-transitory storage medium, including: a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media capable of storing program code; it can also be a transient storage medium.

[0116] The foregoing description and accompanying drawings fully illustrate embodiments of this disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used in this application means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0117] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0118] The methods and products (including but not limited to devices and equipment) disclosed in the embodiments herein can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to implement this embodiment according to actual needs. Furthermore, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0119] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A method for generating a smart home assistant, characterized in that, include: Build a model for a virtual butler; Get real-life photos of the home; The described home scene includes multiple physical home appliances; Based on the real-world home scene, models are created for each object and the home space structure to obtain a spatial geometric model; the spatial geometric model contains virtual home appliances corresponding to each of the physical home appliances. The home space structure is obtained through the real-world home scene; a state recognition model is established for each physical home appliance; the state recognition model is used to identify the state of each physical home appliance; each virtual home appliance is bound to its corresponding state recognition model to obtain a virtual data space; wherein, in the virtual data space, the state of the virtual home appliance is the same as the state of its corresponding physical home appliance; the virtual data space is used to provide feedback on the state of physical home appliances within a preset area. The virtual butler model is integrated into a virtual digital space to obtain an intelligent butler; the intelligent butler is a digital virtual human. Control the interaction between the smart home assistant and the user; The process of controlling the smart home assistant to interact with the user includes: determining the user's real-time location when the user has returned home; waking up the smart home assistant on the nearest physical home appliance with a screen to the user's real-time location; the physical home appliance with a screen is equipped with the smart home assistant; acquiring the user's voice; and controlling the smart home assistant to communicate with the user based on the user's voice.

2. The method according to claim 1, characterized in that, The model for building a virtual butler includes: Establish a 3D model of the virtual butler; Determine the actions of the virtual butler; Bind the preset semantics to the actions of the virtual butler; By combining the semantically bound actions with the three-dimensional model, a model of the virtual butler is obtained.

3. The method according to claim 1, characterized in that, The process involves modeling the objects and spatial structure of the home scene to obtain a spatial geometric model, including: using a preset second modeling tool to perform multi-layer modeling of the objects in the home scene to obtain multiple object models; using a preset third modeling tool to build a digital space virtual scene based on the home spatial structure to obtain a digital space virtual scene model; and improving the digital space virtual scene model to obtain a spatial geometric model.

4. The method according to claim 1, characterized in that, After waking up the smart home assistant, it also includes: The smart home assistant is controlled to identify the status of each physical home appliance in the virtual data space; The smart home assistant is controlled to broadcast an abnormal status report when it detects an abnormal status of the physical home appliance.

5. A device for generating a smart home manager, characterized in that, include: The building module is configured to build a model for the virtual butler; The generation module is configured to capture a real-world view of the home. The real-world home scene includes multiple physical home appliances; based on the real-world home scene, each object and the home space structure in the real-world home scene are modeled to obtain a spatial geometric model; the spatial geometric model contains virtual home appliances corresponding to each of the physical home appliances; The home space structure is obtained through the real-world home scene; a state recognition model is established for each physical home appliance; the state recognition model is used to identify the state of each physical home appliance; each virtual home appliance is bound to its corresponding state recognition model to obtain a virtual data space; wherein, in the virtual data space, the state of the virtual home appliance is the same as the state of its corresponding physical home appliance; the virtual data space is used to provide feedback on the state of physical home appliances within a preset area. The integration module is configured to integrate the model of the virtual butler into the virtual digital space to obtain an intelligent butler; the intelligent butler is a digital virtual human. The control module is configured to control the smart butler to interact with the user after obtaining the smart butler; The control module is configured to control the smart home assistant to interact with the user in the following ways: when the user is confirmed to be home, determine the user's real-time location; wake up the smart home assistant on the physical home appliance with a screen closest to the user's real-time location; the physical home appliance with a screen has the smart home assistant installed; acquire the user's voice; and control the smart home assistant to communicate with the user based on the user's voice.

6. An apparatus for generating a smart home manager, comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to, when running the program instructions, execute the method for generating a smart home as described in any one of claims 1 to 4.

7. An electronic device, characterized in that, include: The electronic device itself; The device for generating a smart home manager as described in claim 5 or 6 is installed on the electronic device body.

8. A storage medium storing program instructions, characterized in that, When the program instructions are executed, they perform the method for generating a smart home as described in any one of claims 1 to 4.

Images