A control method and device of a home device, an electronic device and a readable medium

Abstract

The application provides a control method and device of a household equipment, an electronic equipment and a readable medium, which are applied to a household control system, historical action time and a historical action area of at least one action of a user are acquired, and associated information of the action is acquired; according to the associated information, actions with an associated relationship are combined to obtain at least one action set; for any action set, the historical action area of the action in the action set is regarded as a to-be-processed action area, and a predicted time of the user to perform the action in the action set is acquired according to the historical action time; the to-be-processed action area of the predicted time is regarded as a target action area, and a target household equipment is allocated to the target action area; the target household equipment is a household equipment that provides a service for the action in the action set; a user instruction is acquired, and based on the instruction, the target household equipment in at least one target action area is controlled to provide a service for the user, so that a management difficulty caused by an increase in the number of household equipments is solved.

Description

Technical Field

[0001] This invention relates to the field of smart home technology, and in particular to a control method for home appliances, a control device for home appliances, an electronic device, and a computer-readable medium. Background Technology

[0002] With the continuous advancement of technology, the smart home market is developing rapidly, and more and more families are adopting smart home devices to improve the convenience and comfort of their lives. However, in the process of using smart home systems to control the operation of smart home devices, as the number of smart home devices increases, the smart home system struggles to effectively manage and control the numerous smart home devices. Summary of the Invention

[0003] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for controlling home devices, in order to solve the problem that with the increasing number of smart home devices, smart home systems find it difficult to effectively manage and control numerous smart home devices.

[0004] This invention discloses a method for controlling home appliances, applied to a home control system. The home control system is used to control at least one home appliance in a preset home environment, providing services for at least one action of a user in the home environment. The method includes:

[0005] Obtain the historical action time and historical action area of ​​the at least one action, and obtain the association information of the at least one action; the association information is used to indicate whether there is an association relationship between the at least one action;

[0006] Based on the association information, the actions that are related are combined to obtain at least one set of actions;

[0007] For any of the action sets, the historical action area of ​​the actions in the action set is taken as the action area to be processed, and the predicted time for the user to perform the action in the action set is obtained based on the historical action time.

[0008] The region of the action to be processed at the predicted time is taken as the target action region, and a target home device is assigned to the target action region from the at least one home device; the target home device is a home device that provides services for the actions in the action set;

[0009] Obtain the user's instructions, and based on the instructions, control at least one target home appliance in the target action area to provide services to the user.

[0010] Optionally, the instruction to obtain the user includes:

[0011] Obtain the voice command issued by the user, and the user image when the user issues the voice command;

[0012] The voice command is input into a preset voice recognition model to obtain the user's voice recognition command;

[0013] The user's image is input into a preset image recognition model to obtain the user's image recognition instruction;

[0014] Based on the voice recognition command and the image recognition command, the user's command is obtained.

[0015] Optionally, the home control system includes at least one voice acquisition device; the step of controlling at least one target home device in the target action area to provide services to the user based on the instructions includes:

[0016] The device distance between the at least one voice acquisition device and the acquisition time of the voice acquisition device for the voice command are obtained.

[0017] The horizontal distance between the user and the voice acquisition device is obtained using the device distance and the acquisition time.

[0018] Optionally, the voice command is a time-domain signal; the step of controlling at least one target home appliance in the target action area to provide services to the user based on the command includes:

[0019] The time-domain signal is converted into a frequency-domain signal using a preset fast Fourier transform algorithm;

[0020] Extract the phase information of the sound wave frequency of the voice command from the frequency domain signal;

[0021] The phase difference of the at least one voice acquisition device is obtained using the phase information corresponding to the at least one voice acquisition device;

[0022] Acquire the sound intensity of the voice command acquired by the at least one voice acquisition device;

[0023] The vertical distance between the user and the voice acquisition device is obtained using the phase difference and the sound intensity.

[0024] Optionally, controlling at least one target home appliance in the target action area to provide services to the user based on the instructions includes:

[0025] Based on the horizontal distance and the vertical distance, the distance between the user and the voice acquisition device is obtained;

[0026] Obtain the time when the voice command was issued;

[0027] Based on the distance and the time of issuance of the voice command, a processable action area is determined from at least one of the target action areas;

[0028] Control the target home appliances in the manageable action area to provide services to the user.

[0029] Optionally, controlling at least one target home appliance in the target action area to provide services to the user based on the instructions includes:

[0030] Based on the instructions, processable home devices are determined from at least one of the target home devices;

[0031] The user information corresponding to the voice command is obtained using the speech recognition model and / or the image recognition model.

[0032] Based on the user information, determine whether the user has the right to use the manageable home appliances;

[0033] If the user has permission to use the manageable home appliance, then control the manageable home appliance to provide services to the user.

[0034] Optionally, the method includes:

[0035] Obtain at least one of the user’s historical usage frequency and / or historical usage success rate for the manageable home device;

[0036] Based on the historical usage frequency and / or the historical usage success rate, determine at least one user's usage rights for the manageable home device.

[0037] Optionally, the method includes:

[0038] Obtain at least one training voice command from the user, a training user image of the user issuing the training voice command, training user information corresponding to the training voice command, and training text command corresponding to the training voice command.

[0039] The speech recognition model is obtained by training a preset first machine learning model using the training voice command, the training user information, and the training text command.

[0040] The image recognition model is obtained by training a preset second machine learning model using the training user images, the training user information, and the training text instructions.

[0041] Optionally, the method includes:

[0042] Configure preset identity recognition voice commands for the user;

[0043] By determining whether the voice command is the same as the identity recognition voice command, the user information corresponding to the voice command can be obtained.

[0044] This invention also discloses a control device for home appliances, applied to a home control system. The home control system is used to control at least one home appliance in a preset home environment, providing services for at least one action of a user in the home environment. The device includes:

[0045] The historical action information acquisition module is used to acquire the historical action time and historical action area of ​​the at least one action, and to acquire the association information of the at least one action; the association information is used to indicate whether there is an association relationship between the at least one action;

[0046] The action set acquisition module is used to combine the actions that have a relationship based on the association information to obtain at least one action set;

[0047] The pending action area is a module used to, for any set of actions, take the historical action area of ​​the actions in the action set as the pending action area, and obtain the predicted time when the user will perform the action in the action set based on the historical action time.

[0048] The target action area is a module used to identify the action area to be processed at the predicted time as the target action area, and to assign a target home device to the target action area from the at least one home device; the target home device is a home device that provides services for the actions in the action set.

[0049] The device control module is used to acquire the user's instructions and, based on the instructions, control at least one target home device in the target action area to provide services to the user.

[0050] Optionally, the device control module includes:

[0051] The voice command acquisition submodule is used to acquire the voice command issued by the user, as well as the user image when the user issues the voice command;

[0052] The voice recognition instruction acquisition submodule is used to input the voice instruction into a preset voice recognition model to obtain the user's voice recognition instruction;

[0053] The image recognition instruction acquisition submodule is used to input the user's image into a preset image recognition model to obtain the user's image recognition instruction;

[0054] The instruction acquisition submodule is used to acquire the user's instruction based on the voice recognition instruction and the image recognition instruction.

[0055] Optionally, the home control system includes at least one voice acquisition device; the device control module includes:

[0056] The device distance acquisition submodule is used to acquire the device distance between the at least one voice acquisition device and the acquisition time of the voice acquisition device in response to the voice command.

[0057] The horizontal distance acquisition submodule is used to acquire the horizontal distance between the user and the voice acquisition device using the device distance and the acquisition time.

[0058] Optionally, the voice command is a time-domain signal; the device control module includes:

[0059] The signal conversion submodule is used to convert the time-domain signal into a frequency-domain signal using a preset fast Fourier transform algorithm;

[0060] A phase information extraction submodule is used to extract the phase information of the sound wave frequency of the voice command from the frequency domain signal;

[0061] The phase difference acquisition submodule is used to acquire the phase difference of the at least one voice acquisition device using the phase information corresponding to the at least one voice acquisition device.

[0062] A sound intensity acquisition submodule is used to acquire the sound intensity of the voice command acquired by the at least one voice acquisition device;

[0063] The vertical distance acquisition submodule is used to acquire the vertical distance between the user and the voice acquisition device using the phase difference and the sound intensity.

[0064] Optionally, the device control module includes:

[0065] The distance acquisition submodule is used to acquire the distance between the user and the voice acquisition device based on the horizontal distance and the vertical distance;

[0066] The time of issuance acquisition submodule is used to acquire the time of issuance of the voice command;

[0067] A processable action area determination submodule is used to determine a processable action area from at least one of the target action areas based on the distance and the time of issuance of the voice command;

[0068] The device control submodule is used to control the target home devices in the processable action area to provide services to the user.

[0069] Optionally, the device control module includes:

[0070] A processable home device determination submodule is configured to determine processable home devices from at least one of the target home devices based on the instructions.

[0071] The user information acquisition submodule is used to acquire user information corresponding to the voice command by using the speech recognition model and / or the image recognition model.

[0072] The permission determination submodule is used to determine, based on the user information, whether the user has the permission to use the manageable home appliances;

[0073] The manageable home device control submodule is used to control the manageable home device to provide services to the user if the user has the right to use the manageable home device.

[0074] Optionally, the device includes:

[0075] A historical usage success rate acquisition module is used to acquire the historical usage frequency and / or historical usage success rate of at least one of the users for the processable home devices;

[0076] The permission determination module is used to determine the usage permission of at least one user for the manageable home device based on the historical usage frequency and / or the historical usage success rate.

[0077] Optionally, the device includes:

[0078] The training data acquisition module is used to acquire at least one training voice command of the user, a training user image of the user issuing the training voice command, training user information corresponding to the training voice command, and training text command corresponding to the training voice command.

[0079] The first training module is used to train a preset first machine learning model using the training voice command, the training user information, and the training text command to obtain the speech recognition model.

[0080] The second training module is used to train a preset second machine learning model using the training user images, the training user information, and the training text instructions to obtain the image recognition model.

[0081] Optionally, the device includes:

[0082] The configuration module is used to configure preset identity recognition voice commands for the user;

[0083] The user information acquisition module is used to acquire user information corresponding to the voice command by determining whether the voice command is the same as the identity recognition voice command.

[0084] This invention also discloses an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0085] The memory is used to store computer programs;

[0086] When the processor executes a program stored in the memory, it implements the method described in the embodiments of the present invention.

[0087] This invention also discloses one or more computer-readable media storing instructions that, when executed by one or more processors, cause the processors to perform the methods described in this invention.

[0088] The embodiments of the present invention have the following advantages:

[0089] In this embodiment of the invention, a home control system is used to control at least one home device in a preset home environment to provide services for at least one action of the user in the home environment. The home control system obtains the historical action time and historical action area of ​​at least one action, and obtains the association information of at least one action; the association information is used to indicate whether there is a relationship between at least one action; based on the association information, actions with relationships are combined to obtain at least one action set; for any action set, the historical action area of ​​the actions in the action set is taken as the action area to be processed, and the predicted time for the user to perform the action in the action set is obtained based on the historical action time; the action area to be processed at the predicted time is taken as the target action area, and a target home device is assigned to the target action area from at least one home device; the target home device is the home device that provides services for the action in the action set; the user's instructions are obtained, and based on the instructions, the target home devices in at least one target action area are controlled to provide services to the user. By determining different target action areas based on the functional correlation between the user's actions and home devices, and using the target home devices in the target action areas to respond to user instructions and provide services to the user, the management difficulties caused by the increasing number of smart home devices are solved. By using partitioned networking, home devices are assigned to different areas, making it convenient for users to find and control the devices. Attached Figure Description

[0090] Figure 1 This is a flowchart illustrating the steps of a home appliance control method provided in an embodiment of the present invention;

[0091] Figure 2 This is a flowchart of the steps of another home appliance control method provided in this embodiment of the invention;

[0092] Figure 3 This is a structural block diagram of a control device for a home appliance provided in an embodiment of the present invention;

[0093] Figure 4 This is a block diagram of an electronic device provided in an embodiment of the present invention;

[0094] Figure 5 This is a schematic diagram of a computer-readable medium provided in an embodiment of the present invention. Detailed Implementation

[0095] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0096] To facilitate understanding of the technical solutions and effects of the embodiments of the present invention, the relevant technologies of the present invention will be briefly described below.

[0097] With the continuous advancement of technology, the smart home market is developing rapidly, and more and more families are adopting smart home devices to improve the convenience and comfort of their lives. However, the use of smart home systems to control the operation of smart home devices faces at least the following problems: First, with the increasing number of smart home devices, smart home systems struggle to effectively manage and control so many devices. Second, different family members have different usage needs and permission requirements for smart home devices, making it difficult for smart home systems to meet the personalized needs of different users.

[0098] Reference Figure 1 This diagram illustrates a flowchart of a home appliance control method provided in an embodiment of the present invention. The method is applied to a home control system, which controls at least one home appliance in a preset home environment to provide services for at least one action taken by a user within that environment. Specifically, it may include the following steps:

[0099] Step 101: Obtain the historical action time and historical action area of ​​the at least one action, and obtain the association information of the at least one action; the association information is used to indicate whether there is an association relationship between the at least one action;

[0100] In this embodiment of the invention, the home environment includes at least one home appliance that can provide services to users within the home environment. The home appliance can be a television, lighting fixtures, curtain motors, and air purifiers, etc. Users can perform daily activities within the home environment, and the home control system can control the home appliances within the home environment to provide services for at least one of the user's actions within the home environment.

[0101] In this embodiment of the invention, the home appliances can be smart devices. Depending on the needs of different areas of the home environment, these devices can be deployed in different areas. For example, the living room can be equipped with a smart TV, dimmable and color-adjustable smart lights, smart curtain motors, and smart air purifiers; the bedroom can be equipped with smart lights, smart air conditioners, and smart humidifiers; the kitchen can be equipped with smart lights, a smart refrigerator, and a smart rice cooker; and the bathroom can be equipped with smart lights, a smart water heater, and a smart toilet.

[0102] In this embodiment of the invention, the home control system may also include a gateway. Specifically, if there are home devices using ZigBee or Bluetooth protocols in the home environment, a stable and reliable gateway needs to be selected to ensure that its coverage can cover the area where all devices are located. ZigBee is a wireless communication protocol. In this embodiment of the invention, home devices need to be connected to the home network. Specifically, the home devices can be powered on, enter the device settings mode, select the home Wi-Fi network, and enter the password to connect the home devices to Wi-Fi (wireless network). For some smart devices, auxiliary connection settings may be required through a mobile app. For home devices using ZigBee or Bluetooth protocols, the gateway device can be connected to the home network, and then the ZigBee or Bluetooth device can be paired with the gateway according to the device manual. Typically, these operations require adding and pairing devices through a mobile app.

[0103] In this embodiment of the invention, the home control system can obtain the historical action time and historical action area of ​​at least one user action, and obtain the association information of at least one action. The association information indicates whether there is a correlation between at least one user action.

[0104] Specifically, the home control system can record and analyze in detail the movement trajectory and behavior patterns of at least one family member at home on a certain day, and identify the historical time and historical action area of ​​frequent and consistent actions.

[0105] In a specific example, a user gets up one morning and performs actions such as getting up, getting dressed, and washing up in the bedroom, then goes to the kitchen to prepare breakfast, and finally eats in the dining room. This series of actions is related.

[0106] Step 102: Based on the association information, combine the actions that have an association relationship to obtain at least one action set;

[0107] In this embodiment of the invention, the correlation information between actions is used to determine whether there is a correlation between them. If there is a correlation between the actions, the actions with the correlation are combined to obtain at least one set of actions.

[0108] Step 103: For any of the action sets, take the historical action area of ​​the actions in the action set as the action area to be processed, and obtain the predicted time when the user will perform the action in the action set based on the historical action time.

[0109] In this embodiment of the invention, for any set of actions, the historical action area of ​​the actions in the set of actions can be used as the action area to be processed, and the predicted time for the user to perform the action in the set of actions can be determined based on the historical action time of all the actions obtained.

[0110] In a specific example, a user wakes up one morning and performs actions such as getting up, dressing, and washing up in the bedroom. Then, they go to the kitchen to prepare breakfast and finally eat in the dining room. This series of actions is related, and combining these actions results in an action set. The historical action areas involved in this action set include the bedroom, kitchen, and dining room; therefore, the bedroom, kitchen, and dining room can be considered as the areas to be processed within this action set. Since the historical action time of the actions in this action set is in the morning, it can be predicted that the user will perform any action in this action set on a future morning.

[0111] Step 104: The region of the action to be processed at the predicted time is taken as the target action region, and a target home device is assigned to the target action region from the at least one home device; the target home device is a home device that provides services for the actions in the action set;

[0112] In this embodiment of the invention, the region of actions to be processed at the predicted time can be used as the target action region, and the target action region can also be named. For any set of actions, the corresponding target action region in the home environment is determined, realizing partitioning based on the behavioral flow of home users and the functional correlation between home devices.

[0113] In a specific example, a user wakes up one morning and performs actions such as getting up, dressing, and washing up in the bedroom. Then, they go to the kitchen to prepare breakfast and finally eat in the dining room. This series of actions is related, and combining these actions results in an action set. For this action set, the bedroom, kitchen, and dining room on any future morning can be designated as the target action area and named the "Morning Vitality Zone (MVZ)." MVZ can serve as the identifier for the Morning Vitality Zone.

[0114] In another specific example, a user can watch a movie on a smart TV in the living room in the evening, or use a gaming device for entertainment. Therefore, the living room on any future evening can be designated as the target action area and named the "Leisure Core Area (LCA)".

[0115] In another specific example, family members can communicate, discuss family matters, hold family meetings, and engage in parent-child activities in the living room during the day. Therefore, the living room on any future daytime can be designated as the target activity area and named the "Family Communication and Interaction Zone".

[0116] In another specific example, a user can use a smart makeup mirror to apply makeup and use an electric toothbrush to brush their teeth in the bathroom in the morning. Therefore, the bathroom on any future morning can be designated as the target action area and named the "Morning Washing and Dressing Area".

[0117] In another specific example, users can use a smart water heater and a jacuzzi in the bathroom at night. Therefore, the bathroom can be used as a relaxing evening bathing area on any future night.

[0118] In another specific example, users typically spend their afternoons in their study using home appliances such as professional drawing tablets and high-resolution monitors for creative work; therefore, the study on any future afternoon can be designated as the target activity area and named the "Focused Creativity Zone." Users typically spend their evenings in their study reading books and searching for information using smart bookshelves and electronic data storage devices; therefore, the study on any future evening can be designated as the target activity area and named the "Research Zone," thus dividing the study into different areas based on the type of work task and device collaboration.

[0119] In this embodiment of the invention, the home control system can assign a target home device to any target action area from at least one home device in the home environment. The target home device is a home device that can provide services for actions in the action set corresponding to the target action area. For example, the home control system can assign a smart makeup mirror and an electric toothbrush as target home devices to a "morning grooming area".

[0120] In this embodiment of the invention, the home control system includes an intelligent planning module, which also provides an interactive interface for users. This interface displays a button called "Behavior-Based Zone Creation," which users can click to access the intelligent planning module. Within the intelligent planning module, users can divide target action areas, name these areas, and configure corresponding identifiers. After the target action areas are created, the home control system can automatically recommend and assign target home devices that can provide services to actions in the action set corresponding to the target action area to the corresponding zones based on a pre-set association algorithm between home devices and behavioral functions. Users can also manually adjust the settings according to their needs, for example, assigning home devices related to morning wake-up scenarios, such as smart alarm clocks and smart light controllers in the bedroom, to the morning activity zone, and assigning home devices related to entertainment scenarios, such as smart TVs, game consoles, and smart sofa control systems in the living room, to the leisure and entertainment core zone.

[0121] Step 105: Obtain the user's instructions, and based on the instructions, control at least one target home appliance in the target action area to provide services to the user.

[0122] In this embodiment of the invention, the home control system may include at least one voice acquisition device, such as a microphone. These microphones may form a microphone array. Specifically, the home control system may include a smart speaker with a microphone array, which enhances sound pickup through the microphone array. Furthermore, the smart speaker is equipped with a powerful voice processing chip and has broad smart home ecosystem compatibility. It also boasts high voice recognition accuracy and supports connection to multiple smart home protocols.

[0123] Home control systems can use smart speakers to receive commands from users within their home environment. Based on these commands, they can then control targeted home devices in at least one target activity area to provide services to the user.

[0124] In some embodiments of the present invention, the instruction to obtain the user includes:

[0125] Obtain the voice command issued by the user, and the user image when the user issues the voice command;

[0126] The voice command is input into a preset voice recognition model to obtain the user's voice recognition command;

[0127] The user's image is input into a preset image recognition model to obtain the user's image recognition instruction;

[0128] Based on the voice recognition command and the image recognition command, the user's command is obtained.

[0129] In this embodiment of the invention, the home control system further includes at least one smart camera. It should be noted that the smart camera can be mounted on a smart speaker.

[0130] In this embodiment of the invention, the home control system can utilize a smart speaker to acquire voice commands issued by the user and a smart camera to capture an image of the user when issuing the voice commands. The smart speaker in the home control system can input the voice commands into a voice recognition model to obtain the user's voice recognition commands. The smart speaker can also input the user's image captured by the smart camera into an image recognition model to obtain the user's image recognition commands. Both the voice recognition commands and the image recognition commands are in text format.

[0131] In this embodiment of the invention, the smart speaker in the home control system can fuse and analyze voice recognition commands and image recognition commands to obtain the user's commands. The smart speaker can employ a federated learning mechanism, enabling it to share the learning results of voice recognition and image recognition models with smart speakers in other households.

[0132] It's worth noting that when a smart speaker receives a voice command from a user, it can automatically adjust the microphone's gain and frequency response. Specifically, the smart speaker can dynamically optimize the sound signal acquisition based on the distance between the voice command source and the microphone, as well as the ambient noise level. The smart speaker can convert the received sound signal into a digital signal and preprocess it using an artificial intelligence noise reduction model. This noise reduction model, by learning from numerous samples of home ambient noise, can effectively remove common noise interference such as appliance operation noise and traffic noise from outside the window, thereby improving the quality of the voice signal.

[0133] In this embodiment of the invention, the smart speaker employs a multimodal fusion speech recognition algorithm. This algorithm not only analyzes traditional acoustic features but also integrates visual features of the speech. When a user speaks, if the smart speaker is equipped with a camera or linked with other smart cameras in the home, it will capture visual information such as the user's lip movements, facial expressions, and body language, and fuse and analyze this information with the speech signal. This significantly improves the accuracy of command recognition in complex environments or when speech is unclear. That is, even in noisy settings such as family gatherings, where some parts of the voice command are masked by noise, the smart speaker can still work in conjunction with the smart camera to accurately understand the user's intent and obtain the user's command by analyzing the user's lip movements and related facial expressions and body language.

[0134] In some embodiments of the present invention, the method includes:

[0135] Obtain at least one training voice command from the user, a training user image of the user issuing the training voice command, training user information corresponding to the training voice command, and training text command corresponding to the training voice command.

[0136] The speech recognition model is obtained by training a preset first machine learning model using the training voice command, the training user information, and the training text command.

[0137] The image recognition model is obtained by training a preset second machine learning model using the training user images, the training user information, and the training text instructions.

[0138] In this embodiment of the invention, the home control system can acquire at least one user's training voice commands, a training user image showing the user issuing the training voice commands, training user information corresponding to the training voice commands, and training text commands corresponding to the training voice commands. The training voice commands, training user information, and training text commands can be used to train a preset first machine learning model to obtain a speech recognition model, and the training user image, training user information, and training text commands can be used to train a preset second machine learning model to obtain an image recognition model. The training voice commands may include the user's speech habits, and the training text commands may include vocabulary corresponding to those speech habits.

[0139] It's important to note that when training the model, a quiet environment should be chosen to avoid background noise interference. Specifically, training can be conducted in a relatively enclosed room, such as a bedroom or study. Ensure the smart speaker's microphone is in good working order and not obstructed or damaged. This can be checked by observing the smart speaker's indicator lights or via the mobile app. To train the model, you can find the "Voice Training" option in the smart speaker's settings menu or the mobile app. After entering, follow the prompts. Each family member can take turns speaking a series of commonly used commands and words into the smart speaker, such as "turn on the lights," "close the curtains," "turn up the temperature," and "play music." The smart speaker will record the voice characteristics of each family member and analyze and learn from them. To improve the accuracy of speech recognition, you can repeatedly record voice samples. It is recommended that each family member conduct at least three to five training sessions, with a period of time between each session to allow the smart speaker sufficient time to learn and optimize.

[0140] In this embodiment of the invention, the model is trained using the user's unique speech habits and vocabulary, which enables the rapid and accurate conversion of digital signals into text instructions.

[0141] In some embodiments of the present invention, the home control system includes at least one voice acquisition device; the step of controlling at least one target home device in the target action area to provide services to the user based on the instructions includes:

[0142] The device distance between the at least one voice acquisition device and the acquisition time of the voice acquisition device for the voice command are obtained.

[0143] The horizontal distance between the user and the voice acquisition device is obtained using the device distance and the acquisition time.

[0144] In this embodiment of the invention, the home control system includes a smart speaker, which in turn includes a microphone array. The microphone array includes at least one microphone, which can be considered a voice acquisition device. The microphone array is rationally arranged, with a reasonable number, position, and spacing of microphones. The microphone array can be arranged in a planar array, a three-dimensional array, or similar manner.

[0145] In this embodiment of the invention, when the home control system receives a command from a user, the smart speaker can utilize a microphone array and spatial positioning technology to accurately determine the three-dimensional coordinates of the voice command source, i.e., the user's three-dimensional coordinates. Specifically, the smart speaker can perform complex calculations by analyzing the time difference, phase difference, and sound intensity differences between sound signals received from multiple microphones to determine which room area the sound originates from and to pinpoint its exact location within the room. For example, it can determine whether the command was issued from the left or right side of the living room sofa.

[0146] Specifically, when a user issues a voice command, each microphone in the microphone array simultaneously receives the sound signal. The home control system can preprocess the acquired sound signal, including noise reduction and filtering, to remove background noise and other interference signals, improving signal quality and reliability. Simultaneously, the home control system can also perform sampling rate conversion and normalization on the sound signal. Finally, the home control system can convert the sound signal into an electrical signal.

[0147] In this embodiment of the invention, the home control system can perform frame-based processing on the pre-processed sound data. Each frame typically has a certain duration, such as 20ms-30ms (milliseconds). Then, a window function, such as a Hamming window or a rectangular window, is applied to each frame to reduce the spectral leakage problem caused by signal truncation.

[0148] In this embodiment of the invention, the voice command is a time-domain signal. The home control system can perform a Fast Fourier Transform (FFT) on the framed and windowed audio data to convert the time-domain signal into a frequency-domain signal, obtaining the spectral information of each microphone at different frequencies. Then, the cross-power spectrum between any two microphones can be calculated by multiplying the frequency-domain signals of the two microphones by their conjugates, thereby obtaining their correlation information at various frequencies. Next, the cross-power spectrum can be weighted to enhance the time delay information in the signal, followed by an Inverse Fast Fourier Transform (IFFT) to obtain the generalized cross-correlation function. The weighting function for the weighting process can be PHAT (Phase Transform Generalized Cross-Correlation), etc.

[0149] In this embodiment of the invention, the home control system can find the maximum point of the generalized cross-correlation function, and the sample difference corresponding to this maximum point is the time difference between the sound signals received by the two microphones. By performing the above calculation on different microphone combinations, multiple sets of time difference data can be obtained.

[0150] In this embodiment of the invention, the home control system can obtain the device distance between at least one voice acquisition device and the acquisition time when the at least one voice acquisition device acquires a voice command. Using the device distance between the at least one voice acquisition device and the acquisition time, the horizontal distance between the user and the voice acquisition device can be obtained.

[0151] Specifically, the user's coordinates on a two-dimensional plane can be calculated using geometric relationships (such as triangulation). For example, if the coordinates of two microphones on a preset two-dimensional plane are known to be (x1, y1) and (x2, y2), and the speed of sound is c, then the user's coordinates (x, y) on the two-dimensional plane can be calculated using the following formula:

[0152]

[0153] Where △t1 and △t2 are the time differences between the time when the user issues a voice command and the time when the two microphones capture the voice command.

[0154] After obtaining the user's coordinate position in a two-dimensional plane, the horizontal distance between the user and the voice acquisition device can then be obtained.

[0155] In some embodiments of the present invention, the voice command is a time-domain signal; the step of controlling at least one target home appliance in the target action area to provide services to the user based on the command includes:

[0156] The time-domain signal is converted into a frequency-domain signal using a preset fast Fourier transform algorithm;

[0157] Extract the phase information of the sound wave frequency of the voice command from the frequency domain signal;

[0158] The phase difference of the at least one voice acquisition device is obtained using the phase information corresponding to the at least one voice acquisition device;

[0159] Acquire the sound intensity of the voice command acquired by the at least one voice acquisition device;

[0160] The vertical distance between the user and the voice acquisition device is obtained using the phase difference and the sound intensity.

[0161] In this embodiment of the invention, the voice command is a time-domain signal. The home control system can use a preset Fast Fourier Transform algorithm to convert the time-domain signal into a frequency-domain signal.

[0162] In this embodiment of the invention, the home control system can extract the phase information of each microphone at various frequencies from the frequency domain signal transformed by the Fast Fourier Transform (FFT) algorithm. For any two microphones, the phase difference between the two microphones at the same frequency can be calculated. By analyzing and processing the phase differences at multiple frequencies, the accurate phase difference information of the two microphones can be obtained. Due to factors such as multipath effects that may exist during sound propagation, this may cause deviations in the phase difference. Therefore, phase correction algorithms, such as the phase correction method in GCC-PHAT (Generalized Cross-Correlation with Phase Transform), can be used to correct the phase difference, thereby eliminating the influence of multipath effects and improving the accuracy of the phase difference.

[0163] In this embodiment of the invention, the sound intensity of a voice command collected by at least one voice acquisition device can be obtained, specifically by summing the squares of the time-domain signal or summing the squares of the amplitude of the frequency-domain signal.

[0164] In this embodiment of the invention, the vertical distance between the user and the voice acquisition device can be obtained by utilizing phase difference and sound intensity, and the user's height coordinates can be calculated. For example, by analyzing the phase difference and intensity differences of the sound signals received by microphones at different height positions, a corresponding mathematical model can be established to solve for the user's height position.

[0165] It should be noted that by comparing the sound energy levels of different microphones, the difference in sound intensity can be obtained. The difference in sound intensity reflects the distance relationship between the sound source and each microphone; the microphone closer to the sound source receives a greater sound intensity.

[0166] In some embodiments of the present invention, controlling at least one target home appliance in the target action area to provide services to the user based on the instructions includes:

[0167] Based on the horizontal distance and the vertical distance, the distance between the user and the voice acquisition device is obtained;

[0168] Obtain the time when the voice command was issued;

[0169] Based on the distance and the time of issuance of the voice command, a processable action area is determined from at least one of the target action areas;

[0170] Control the target home appliances in the manageable action area to provide services to the user.

[0171] In this embodiment of the invention, the distance between the user and the voice acquisition device can be obtained by utilizing the horizontal and vertical distances between the user and the voice acquisition device, thereby determining the user's location.

[0172] Since the target action area is determined based on the action area to be processed at the predicted time, the time of voice command issuance can be obtained. Based on the voice command issuance time and the user's location, a processable action area can be determined from at least one target action area. After determining the processable action area, the user's commands can be analyzed to identify processable home devices from the target home devices within the processable action area, and services can be provided to the user using these processable home devices.

[0173] In a specific example, after a user issues a voice command "I want to take a shower" in the bathroom at night, the target action area of ​​the "Evening Bathing Relaxation Zone" can be designated as the actionable area, and the smart water heater and whirlpool bathtub in the "Evening Bathing Relaxation Zone" can be designated as actionable home appliances.

[0174] In this embodiment of the invention, when the home control system analyzes user commands, in addition to identifying keywords in the commands, the user's location, and the time the voice commands were issued, it also performs semantic reasoning based on factors such as the family user's daily routine, recent home appliance usage history, and seasonal climate. For example, after 10 p.m., if the user says "dim the lights," the smart speaker will automatically associate the command with the smart lights in the bedroom area and dim them to a suitable brightness for sleep; if in summer the user says "turn the temperature down," the smart speaker will prioritize associating the command with the air conditioning unit in the living room or bedroom, and determine an appropriate cooling level based on historical settings and the current outdoor temperature.

[0175] Therefore, when analyzing user commands to determine manageable action areas and manageable home appliances, the home control system considers both the user's location and the time the command was issued. For example, if a target action area is in the bedroom and the predicted time is evening, and the user is in the bedroom at the predicted time for that target action area, and the user's voice command involves sleep-related operations, the home control system can infer that the command is likely directed at a manageable home appliance in the bedroom, and that this manageable home appliance is one that provides services to the user's sleep.

[0176] On the other hand, the user's location and the time of the command are not absolute criteria for determining the manageable action area. Even outside the bedroom area, if the user mentions a sleep-related command, the home control system needs to consider whether to operate related devices in other areas. For example, if a user in the living room requests to turn off all lights to prepare for sleep, the home control system cannot ignore the command and not control manageable home devices in the bedroom simply because it's not in the bedroom zone. Therefore, the home control system can use semantic reasoning based on recent device usage history and seasonal climate factors to transcend zone boundaries and determine manageable action areas and manageable home devices. For instance, if the living room air conditioner has been used frequently recently, when the user commands "turn the temperature down further," the home control system will prioritize adjusting the living room air conditioner, rather than limiting itself to devices in a specific zone. Regarding seasonal climate, in summer, regardless of the zone from which the command is issued, as long as it mentions cooling or ventilation, the home control system will comprehensively consider the status and usage of cooling equipment in various zones (such as bedrooms and living rooms) to process the command, rather than simply determining the device operation based on zone.

[0177] In this embodiment of the invention, after identifying manageable home devices, the smart speaker of the home control system can send instructions to the manageable home devices within the manageable activity area. Specifically, the home control system can employ a blockchain-based distributed smart contract mechanism, where each home device acts as a node in the blockchain network. The smart speaker can encapsulate the instructions into a smart contract transaction and send the smart contract transaction to the blockchain network.

[0178] After a node capable of handling home devices receives a smart contract transaction, it automatically executes the operation according to the instructions in the contract and its own functional permissions, while recording the execution result on the blockchain to ensure the immutability and traceability of the instruction execution. For example, when a user issues the command "Open the smart curtains of Family Hub," the smart speaker creates and broadcasts a smart contract transaction for opening the curtains. The smart curtain device node in the living room receives the transaction, executes the opening operation, and records the successful execution information in the blockchain. Other related devices (such as the smart lighting devices in the living room) can also adjust their own status synchronously based on the information on the blockchain, such as automatically brightening the lights to adapt to the change in light after the curtains are opened. After the device executes the instruction, it sends feedback information to the smart speaker through the blockchain network. The smart speaker uses natural language generation technology to synthesize personalized voice feedback to the user based on the feedback information and preset voice templates. For example, if the smart curtains open successfully, the smart speaker will generate voice feedback such as "Dear, the smart curtains of Family Hub have been opened for you. The sun is shining brightly outside. Have a pleasant time." based on the current time, weather, and user preferences.

[0179] In some embodiments of the present invention, the method includes:

[0180] Configure preset identity recognition voice commands for the user;

[0181] By determining whether the voice command is the same as the identity recognition voice command, the user information corresponding to the voice command can be obtained.

[0182] In this embodiment of the invention, preset identity recognition voice commands can be configured for at least one user. The home control system can obtain the user information corresponding to the voice command by determining whether the voice command is the same as the identity recognition voice command. Specifically, the identity recognition voice command can also be called a wake word. The father can set the wake word to "Smart Butler Dad", the mother can set it to "Smart Assistant Mom", and the child can set it to "Magic Elf", etc. This allows the smart speaker to quickly recognize the commands of different users.

[0183] In some embodiments of the present invention, controlling at least one target home appliance in the target action area to provide services to the user based on the instructions includes:

[0184] Based on the instructions, processable home devices are determined from at least one of the target home devices;

[0185] The user information corresponding to the voice command is obtained using the speech recognition model and / or the image recognition model.

[0186] Based on the user information, determine whether the user has the right to use the manageable home appliances;

[0187] If the user has permission to use the manageable home appliance, then control the manageable home appliance to provide services to the user.

[0188] In an embodiment of the invention, according to instructions, the home control system can determine manageable home devices from at least one target home device in a manageable action area.

[0189] In this embodiment of the invention, the home control system can utilize a voice recognition model and / or an image recognition model to obtain user information of the user who issues the voice command. Specifically, the voice command can be input into the voice recognition model to obtain user information; alternatively, the user's image at the time of issuing the voice command can be input into the image recognition model to obtain user information.

[0190] In this embodiment of the invention, based on user information, it can be determined whether the user has permission to use the manageable home devices. If the user has permission to use the manageable home devices, the manageable home devices are controlled to provide services to the user.

[0191] Specifically, when a user issues a voice command, the smart speaker first performs an access verification. The smart speaker identifies the user by analyzing their voice characteristics and personalized wake word. Then, the smart speaker checks whether the home device and operation involved in the command are within the user's authorized scope. If the user does not have permission to operate the home device or perform the corresponding operation, the smart speaker will refuse to execute the command and prompt the user, "You do not have permission to operate this device." If the command is within the user's authorized scope, the smart speaker will continue with the command recognition and execution process.

[0192] In this embodiment of the invention, in addition to using voice features and wake words, various other methods can be employed to verify user identity, thereby enhancing system security. For example, passwords can be set, fingerprint recognition, and facial recognition can be used. When a user issues a voice command, the smart speaker may require the user to verify their identity; the command will only be executed after successful verification. For sensitive devices or operations, the system will employ a permission verification method that combines biometrics and environmental factors. For example, for a smart safe, in addition to standard fingerprint recognition, changes in ambient temperature and humidity around the safe, as well as the user's heart rate, skin conductance, and other physiological characteristics will be considered for comprehensive verification. Operation is only permitted when these biometrics match preset normal ranges and environmental factors do not exhibit abnormal fluctuations.

[0193] In terms of data transmission in home control systems, encryption technology should be employed to ensure that user commands and device feedback information are not stolen or tampered with during transmission. For example, SSL / TLS (Secure Sockets Layer / Transport Layer Security) encryption protocols can be used to encrypt data. Furthermore, the software and firmware of the home control system need to be updated regularly to patch any potential security vulnerabilities. Simultaneously, the security of the home network should be ensured, such as by setting complex passwords and enabling firewalls, to prevent hackers from intruding into the home control system. Home appliances should be regularly inspected and maintained to ensure their proper functioning and security. If any security vulnerabilities are found in the equipment, measures should be taken immediately to repair or replace it.

[0194] In some embodiments of the present invention, the method includes:

[0195] Obtain at least one of the user’s historical usage frequency and / or historical usage success rate for the manageable home device;

[0196] Based on the historical usage frequency and / or the historical usage success rate, determine at least one user's usage rights for the manageable home device.

[0197] In this embodiment of the invention, the historical usage frequency and / or historical success rate of at least one user for manageable home devices can be obtained, and the usage rights of at least one user for manageable home devices can be determined based on the historical usage frequency and / or historical success rate.

[0198] Specifically, the home control system includes a dynamic permission allocation model. This model can set usage permissions for family members based on diverse dynamic data such as their daily activity patterns, proficiency in using home devices, and contributions to family affairs management. For example, if a family member has recently been frequently and successfully operating complex smart kitchen devices to complete various cooking tasks, their permissions on kitchen-related smart devices will be appropriately enhanced, such as unlocking access to certain advanced cooking modes. Similarly, if a family member actively participates in the daily inspection and maintenance of home security devices, some of their usage permissions within the smart security system will be correspondingly enhanced, such as temporarily adjusting the coverage area of ​​the monitoring zone. In this embodiment of the invention, a temporary permission mechanism is also designed. When a visitor needs to use a specific smart device (such as the screen mirroring function of a smart TV), the administrator can quickly generate a time-limited and function-limited temporary permission QR code via a mobile phone. After scanning the code, the visitor can use the corresponding device within the specified time and range.

[0199] In this embodiment of the invention, a home control system is used to control at least one home device in a preset home environment to provide services for at least one action of the user in the home environment. The home control system obtains the historical action time and historical action area of ​​at least one action, and obtains the association information of at least one action; the association information is used to indicate whether there is a relationship between at least one action; based on the association information, actions with relationships are combined to obtain at least one action set; for any action set, the historical action area of ​​the actions in the action set is taken as the action area to be processed, and the predicted time for the user to perform the action in the action set is obtained based on the historical action time; the action area to be processed at the predicted time is taken as the target action area, and a target home device is assigned to the target action area from at least one home device; the target home device is the home device that provides services for the action in the action set; the user's instructions are obtained, and based on the instructions, the target home devices in at least one target action area are controlled to provide services to the user. By determining different target action areas based on the functional correlation between the user's actions and home devices, and utilizing the target home devices in the target action areas to respond to user instructions and provide services to the user, the management difficulties caused by the increasing number of smart home devices are solved. By using zoned networking, home appliances are assigned to different areas, making it easier for users to find and control them. This improves the targetedness and accuracy of appliance control, avoids indiscriminate control of the entire home space, and enhances the user experience.

[0200] In this embodiment of the invention, usage permissions for home devices are defined for different users within a household. This multi-user permission division prevents accidental or malicious user operations, improving device security and stability. Different users can control devices in their respective areas according to their permissions, meeting the personalized needs of different family members for smart home devices and improving user satisfaction and experience. Administrators within the household can comprehensively manage and control the entire smart home system, while ordinary users can only control devices in their designated areas, enhancing system security and management efficiency.

[0201] In this embodiment of the invention, the voice recognition technology of the smart speaker allows users to interact with the smart device through natural language without manual operation, improving user convenience and comfort. At the same time, the accuracy and stability of voice recognition are also improved, reducing the occurrence of erroneous operations. Through voice recognition training and personalized wake-up word settings, the accuracy of voice recognition and the user experience are further enhanced.

[0202] Reference Figure 2 The diagram illustrates a flowchart of another home appliance control method provided in an embodiment of the present invention, which may specifically include the following steps:

[0203] Step 201, Equipment preparation and connection.

[0204] Step 202, Partition settings.

[0205] The home control system can acquire the historical action time and historical action area of ​​at least one action, and acquire the association information of at least one action; the association information is used to indicate whether there is a relationship between at least one action. Based on the association information, the actions with relationships are combined to obtain at least one action set; for any action set, the historical action area of ​​the actions in the action set is taken as the action area to be processed, and the predicted time for the user to perform the action in the action set is obtained based on the historical action time; the action area to be processed at the predicted time is taken as the target action area, and a target home device is assigned to the target action area from at least one home device; the target home device is the home device that provides services for the actions in the action set.

[0206] Step 203, speech recognition training.

[0207] The home control system acquires at least one user's training voice command, a training user image of the user issuing the training voice command, training user information corresponding to the training voice command, and training text command corresponding to the training voice command; it uses the training voice command, training user information, and training text command to train a preset first machine learning model to obtain a speech recognition model; and it uses the training user image, training user information, and training text command to train a preset second machine learning model to obtain an image recognition model.

[0208] Step 204, User Permission Division.

[0209] The home control system acquires the historical usage frequency and / or historical success rate of at least one user for manageable home devices; based on the historical usage frequency and / or historical success rate, it determines the usage rights of at least one user for manageable home devices.

[0210] Step 205: Voice command recognition and execution.

[0211] The home control system acquires voice commands issued by the user, as well as an image of the user when the voice command is issued; it inputs the voice command into a preset voice recognition model to obtain the user's voice recognition command; it inputs the user's image into a preset image recognition model to obtain the user's image recognition command; based on the voice recognition command and the image recognition command, it obtains the user's instructions. Based on the instructions, it controls at least one target home device in a target action area to provide services to the user.

[0212] Step 206, Permission verification and security management.

[0213] The home control system uses voice recognition models and / or image recognition models to obtain user information corresponding to voice commands; based on the user information, it determines whether the user has the right to use the manageable home devices; if the user has the right to use the manageable home devices, it controls the manageable home devices to provide services to the user.

[0214] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0215] Reference Figure 3 This diagram illustrates a structural block diagram of a control device for home appliances provided in an embodiment of the present invention. The device is applied to a home control system, which controls at least one home appliance in a preset home environment to provide services for at least one action taken by a user within that environment. Specifically, it may include the following modules:

[0216] The historical action information acquisition module 301 is used to acquire the historical action time and historical action area of ​​the at least one action, and to acquire the association information of the at least one action; the association information is used to indicate whether there is an association relationship between the at least one action;

[0217] Action set acquisition module 302 is used to combine the actions that have a relationship according to the association information to obtain at least one action set;

[0218] The pending action area is module 303, which is used to take the historical action area of ​​the action in the action set as the pending action area for any action set, and obtain the predicted time when the user will perform the action in the action set based on the historical action time.

[0219] The target action area, as module 304, is used to take the action area to be processed at the predicted time as the target action area, and to assign a target home device to the target action area from the at least one home device; the target home device is a home device that provides services for the actions in the action set;

[0220] The device control module 305 is used to acquire the user's instructions and, based on the instructions, control at least one target home device in the target action area to provide services to the user.

[0221] In an optional embodiment of the present invention, the device control module includes:

[0222] The voice command acquisition submodule is used to acquire the voice command issued by the user, as well as the user image when the user issues the voice command;

[0223] The voice recognition instruction acquisition submodule is used to input the voice instruction into a preset voice recognition model to obtain the user's voice recognition instruction;

[0224] The image recognition instruction acquisition submodule is used to input the user's image into a preset image recognition model to obtain the user's image recognition instruction;

[0225] The instruction acquisition submodule is used to acquire the user's instruction based on the voice recognition instruction and the image recognition instruction.

[0226] In an optional embodiment of the present invention, the home control system includes at least one voice acquisition device; the device control module includes:

[0227] The device distance acquisition submodule is used to acquire the device distance between the at least one voice acquisition device and the acquisition time of the voice acquisition device in response to the voice command.

[0228] The horizontal distance acquisition submodule is used to acquire the horizontal distance between the user and the voice acquisition device using the device distance and the acquisition time.

[0229] In an optional embodiment of the present invention, the voice command is a time-domain signal; the device control module includes:

[0230] The signal conversion submodule is used to convert the time-domain signal into a frequency-domain signal using a preset fast Fourier transform algorithm;

[0231] A phase information extraction submodule is used to extract the phase information of the sound wave frequency of the voice command from the frequency domain signal;

[0232] The phase difference acquisition submodule is used to acquire the phase difference of the at least one voice acquisition device using the phase information corresponding to the at least one voice acquisition device.

[0233] A sound intensity acquisition submodule is used to acquire the sound intensity of the voice command acquired by the at least one voice acquisition device;

[0234] The vertical distance acquisition submodule is used to acquire the vertical distance between the user and the voice acquisition device using the phase difference and the sound intensity.

[0235] In an optional embodiment of the present invention, the device control module includes:

[0236] The distance acquisition submodule is used to acquire the distance between the user and the voice acquisition device based on the horizontal distance and the vertical distance;

[0237] The time of issuance acquisition submodule is used to acquire the time of issuance of the voice command;

[0238] A processable action area determination submodule is used to determine a processable action area from at least one of the target action areas based on the distance and the time of issuance of the voice command;

[0239] The device control submodule is used to control the target home devices in the processable action area to provide services to the user.

[0240] In an optional embodiment of the present invention, the device control module includes:

[0241] A processable home device determination submodule is configured to determine processable home devices from at least one of the target home devices based on the instructions.

[0242] The user information acquisition submodule is used to acquire user information corresponding to the voice command by using the speech recognition model and / or the image recognition model.

[0243] The permission determination submodule is used to determine, based on the user information, whether the user has the permission to use the manageable home appliances;

[0244] The manageable home device control submodule is used to control the manageable home device to provide services to the user if the user has the right to use the manageable home device.

[0245] In an optional embodiment of the present invention, the device includes:

[0246] A historical usage success rate acquisition module is used to acquire the historical usage frequency and / or historical usage success rate of at least one of the users for the processable home devices;

[0247] The permission determination module is used to determine the usage permission of at least one user for the manageable home device based on the historical usage frequency and / or the historical usage success rate.

[0248] In an optional embodiment of the present invention, the device includes:

[0249] The training data acquisition module is used to acquire at least one training voice command of the user, a training user image of the user issuing the training voice command, training user information corresponding to the training voice command, and training text command corresponding to the training voice command.

[0250] The first training module is used to train a preset first machine learning model using the training voice command, the training user information, and the training text command to obtain the speech recognition model.

[0251] The second training module is used to train a preset second machine learning model using the training user images, the training user information, and the training text instructions to obtain the image recognition model.

[0252] In an optional embodiment of the present invention, the device includes:

[0253] The configuration module is used to configure preset identity recognition voice commands for the user;

[0254] The user information acquisition module is used to acquire user information corresponding to the voice command by determining whether the voice command is the same as the identity recognition voice command.

[0255] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0256] In addition, embodiments of the present invention also provide an electronic device, such as... Figure 4 As shown, it includes a processor 401, a communication interface 402, a memory 403, and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404.

[0257] Memory 403 is used to store computer programs;

[0258] When processor 401 executes the program stored in memory 403, it performs the following steps:

[0259] Obtain the historical action time and historical action area of ​​the at least one action, and obtain the association information of the at least one action; the association information is used to indicate whether there is an association relationship between the at least one action;

[0260] Based on the association information, the actions that are related are combined to obtain at least one set of actions;

[0261] For any of the action sets, the historical action area of ​​the actions in the action set is taken as the action area to be processed, and the predicted time for the user to perform the action in the action set is obtained based on the historical action time.

[0262] The region of the action to be processed at the predicted time is taken as the target action region, and a target home device is assigned to the target action region from the at least one home device; the target home device is a home device that provides services for the actions in the action set;

[0263] Obtain the user's instructions, and based on the instructions, control at least one target home appliance in the target action area to provide services to the user.

[0264] In an optional embodiment of the present invention, the instruction to obtain the user includes:

[0265] Obtain the voice command issued by the user, and the user image when the user issues the voice command;

[0266] The voice command is input into a preset voice recognition model to obtain the user's voice recognition command;

[0267] The user's image is input into a preset image recognition model to obtain the user's image recognition instruction;

[0268] Based on the voice recognition command and the image recognition command, the user's command is obtained.

[0269] In an optional embodiment of the present invention, the home control system includes at least one voice acquisition device; the step of controlling at least one target home device in the target activity area to provide services to the user based on the instructions includes:

[0270] The device distance between the at least one voice acquisition device and the acquisition time of the voice acquisition device for the voice command are obtained.

[0271] The horizontal distance between the user and the voice acquisition device is obtained using the device distance and the acquisition time.

[0272] In an optional embodiment of the present invention, the voice command is a time-domain signal; the step of controlling at least one target home appliance in the target action area to provide services to the user based on the command includes:

[0273] The time-domain signal is converted into a frequency-domain signal using a preset fast Fourier transform algorithm;

[0274] Extract the phase information of the sound wave frequency of the voice command from the frequency domain signal;

[0275] The phase difference of the at least one voice acquisition device is obtained using the phase information corresponding to the at least one voice acquisition device;

[0276] Acquire the sound intensity of the voice command acquired by the at least one voice acquisition device;

[0277] The vertical distance between the user and the voice acquisition device is obtained using the phase difference and the sound intensity.

[0278] In an optional embodiment of the present invention, controlling at least one target home appliance in the target activity area to provide services to the user based on the instructions includes:

[0279] Based on the horizontal distance and the vertical distance, the distance between the user and the voice acquisition device is obtained;

[0280] Obtain the time when the voice command was issued;

[0281] Based on the distance and the time of issuance of the voice command, a processable action area is determined from at least one of the target action areas;

[0282] Control the target home appliances in the manageable action area to provide services to the user.

[0283] In an optional embodiment of the present invention, controlling at least one target home appliance in the target activity area to provide services to the user based on the instructions includes:

[0284] Based on the instructions, processable home devices are determined from at least one of the target home devices;

[0285] The user information corresponding to the voice command is obtained using the speech recognition model and / or the image recognition model.

[0286] Based on the user information, determine whether the user has the right to use the manageable home appliances;

[0287] If the user has permission to use the manageable home appliance, then control the manageable home appliance to provide services to the user.

[0288] In an optional embodiment of the present invention, the method includes:

[0289] Obtain at least one of the user’s historical usage frequency and / or historical usage success rate for the manageable home device;

[0290] Based on the historical usage frequency and / or the historical usage success rate, determine at least one user's usage rights for the manageable home device.

[0291] In an optional embodiment of the present invention, the method includes:

[0292] Obtain at least one training voice command from the user, a training user image of the user issuing the training voice command, training user information corresponding to the training voice command, and training text command corresponding to the training voice command.

[0293] The speech recognition model is obtained by training a preset first machine learning model using the training voice command, the training user information, and the training text command.

[0294] The image recognition model is obtained by training a preset second machine learning model using the training user images, the training user information, and the training text instructions.

[0295] In an optional embodiment of the present invention, the method includes:

[0296] Configure preset identity recognition voice commands for the user;

[0297] By determining whether the voice command is the same as the identity recognition voice command, the user information corresponding to the voice command can be obtained.

[0298] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0299] The communication interface is used for communication between the aforementioned terminal and other devices.

[0300] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0301] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0302] like Figure 5 As shown, in another embodiment of the present invention, a computer-readable storage medium 501 is also provided, which stores instructions that, when executed on a computer, cause the computer to perform a home appliance control method as described in the above embodiment.

[0303] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute a home appliance control method described in the above embodiments.

[0304] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0305] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0306] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0307] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for controlling home appliances, characterized in that, An application to a home control system, wherein the home control system controls at least one home appliance in a preset home environment and provides services for at least one action of a user in the home environment; the method includes: Obtain the historical action time and historical action area of ​​the at least one action, and obtain the association information of the at least one action; the association information is used to indicate whether there is an association relationship between the at least one action; Based on the association information, the actions that are related are combined to obtain at least one set of actions; For any of the action sets, the historical action area of ​​the actions in the action set is taken as the action area to be processed, and the predicted time for the user to perform the action in the action set is obtained based on the historical action time. The region of the action to be processed at the predicted time is taken as the target action region, and a target home device is assigned to the target action region from the at least one home device; the target home device is a home device that provides services for the actions in the action set; Obtain the user's instructions, and based on the instructions, control at least one target home appliance in the target action area to provide services to the user.

2. The method according to claim 1, characterized in that, The instruction to obtain the user includes: Obtain the voice command issued by the user, and the user image when the user issues the voice command; The voice command is input into a preset voice recognition model to obtain the user's voice recognition command; The user's image is input into a preset image recognition model to obtain the user's image recognition instruction; Based on the voice recognition command and the image recognition command, the user's command is obtained.

3. The method according to claim 2, characterized in that, The home control system includes at least one voice acquisition device; the step of controlling at least one target home device in the target action area to provide services to the user based on the instructions includes: The device distance between the at least one voice acquisition device and the acquisition time of the voice acquisition device for the voice command are obtained. The horizontal distance between the user and the voice acquisition device is obtained using the device distance and the acquisition time.

4. The method according to claim 3, characterized in that, The voice command is a time-domain signal; the step of controlling at least one target home appliance in the target action area to provide services to the user based on the command includes: The time-domain signal is converted into a frequency-domain signal using a preset fast Fourier transform algorithm; Extract the phase information of the sound wave frequency of the voice command from the frequency domain signal; The phase difference of the at least one voice acquisition device is obtained using the phase information corresponding to the at least one voice acquisition device; Acquire the sound intensity of the voice command acquired by the at least one voice acquisition device; The vertical distance between the user and the voice acquisition device is obtained using the phase difference and the sound intensity.

5. The method according to claim 4, characterized in that, The step of controlling at least one target home appliance in the target action area to provide services to the user based on the instructions includes: Based on the horizontal distance and the vertical distance, the distance between the user and the voice acquisition device is obtained; Obtain the time when the voice command was issued; Based on the distance and the time of issuance of the voice command, a processable action area is determined from at least one of the target action areas; Control the target home appliances in the manageable action area to provide services to the user.

6. The method according to claim 2, characterized in that, The step of controlling at least one target home appliance in the target action area to provide services to the user based on the instructions includes: Based on the instructions, processable home devices are determined from at least one of the target home devices; The user information corresponding to the voice command is obtained using the speech recognition model and / or the image recognition model. Based on the user information, determine whether the user has the right to use the manageable home appliances; If the user has permission to use the manageable home appliance, then control the manageable home appliance to provide services to the user.

7. The method according to claim 6, characterized in that, The method includes: Obtain at least one of the user’s historical usage frequency and / or historical usage success rate for the manageable home device; Based on the historical usage frequency and / or the historical usage success rate, determine at least one user's usage rights for the manageable home device.

8. The method according to claim 2, characterized in that, The method includes: Obtain at least one training voice command from the user, a training user image of the user issuing the training voice command, training user information corresponding to the training voice command, and training text command corresponding to the training voice command. The speech recognition model is obtained by training a preset first machine learning model using the training voice command, the training user information, and the training text command. The image recognition model is obtained by training a preset second machine learning model using the training user images, the training user information, and the training text instructions.

9. The method according to claim 2, characterized in that, The method includes: Configure preset identity recognition voice commands for the user; By determining whether the voice command is the same as the identity recognition voice command, the user information corresponding to the voice command can be obtained.

10. A control device for a home appliance, characterized in that, An application to a home control system, wherein the home control system controls at least one home appliance in a preset home environment and provides services for at least one action of a user in the home environment; the device includes: The historical action information acquisition module is used to acquire the historical action time and historical action area of ​​the at least one action, and to acquire the association information of the at least one action; the association information is used to indicate whether there is an association relationship between the at least one action; The action set acquisition module is used to combine the actions that have a relationship based on the association information to obtain at least one action set; The pending action area is a module used to, for any set of actions, take the historical action area of ​​the actions in the action set as the pending action area, and obtain the predicted time when the user will perform the action in the action set based on the historical action time. The target action area is a module used to identify the action area to be processed at the predicted time as the target action area, and to assign a target home device to the target action area from the at least one home device; the target home device is a home device that provides services for the actions in the action set. The device control module is used to acquire the user's instructions and, based on the instructions, control at least one target home device in the target action area to provide services to the user.

11. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes a program stored in the memory, it implements the method as described in any one of claims 1-9.

12. One or more computer-readable media having instructions stored thereon that, when executed by one or more processors, cause the processors to perform the method as described in any one of claims 1-9.

Images