Household equipment control method and device, electronic equipment and readable medium

Abstract

The invention provides a home equipment control method and device, electronic equipment and a readable medium, which are applied to a home control system, and the method comprises the steps: obtaining the historical action time and historical action area of at least one action of a user, and obtaining the associated information of the action; according to the association information, combining the actions with the association relationship to obtain at least one action set; for any action set, taking the historical action area of the action in the action set as a to-be-processed action area, and obtaining the prediction time of the action in the action set by the user according to the historical action time; taking the to-be-processed action area of the predicted time as a target action area, and distributing target home equipment for the target action area; the target home device is a home device providing services for the actions in the action set; and obtaining a user instruction, and based on the instruction, controlling the target home equipment in the at least one target action area to provide services for the user, thereby solving a management problem caused by increase of the number of the home equipment.

Description

Technical Field

[0001] The present invention relates to the field of smart home technology, and in particular to a control method for home appliances, a control device for home appliances, an electronic device and a computer-readable medium. Background Art

[0002] With the continuous advancement of technology, the smart home market has developed rapidly, and more and more families have begun to use smart home devices to improve the convenience and comfort of life. However, in the process of using smart home systems to control the operation of smart home devices in related technologies, with the increase in the number of smart home devices, it is difficult for smart home systems to effectively manage and control numerous smart home devices. Summary of the invention

[0003] The embodiments of the present invention provide a control method, device, electronic device and computer-readable storage medium for home appliances to solve the problem that as the number of smart home appliances increases, it is difficult for a smart home system to effectively manage and control numerous smart home appliances.

[0004] An embodiment of the present invention discloses a method for controlling a home appliance, which is applied to a home control system, wherein the home control system is used to control at least one home appliance in a preset home environment and provide services for at least one action of a user in the home environment; the method comprises:

[0005] Acquire the historical action time and historical action area of ​​the at least one action, and acquire the association information of the at least one action; the association information is used to indicate whether there is an association relationship between the at least one action;

[0006] Combining the actions that have an associated relationship according to the associated information to obtain at least one action set;

[0007] For any of the action sets, a historical action region of the actions in the action set is used as an action region to be processed, and a predicted time for the user to perform the actions in the action set is obtained according to the historical action time;

[0008] The to-be-processed action area at the predicted time is used as a target action area, and a target home device is allocated to the target action area from the at least one home device; the target home device is a home device that provides services for the actions in the action set;

[0009] Obtain instructions from the user, and based on the instructions, control at least one target home device in the target action area to provide services to the user.

[0010] Optionally, obtaining the instruction of the user includes:

[0011] Acquire the voice command issued by the user and the user picture when the user issues the voice command;

[0012] Inputting the voice command into a preset voice recognition model to obtain the voice recognition command of the user;

[0013] Inputting the user's picture into a preset picture recognition model to obtain the user's picture recognition instruction;

[0014] Based on the voice recognition instruction and the picture recognition instruction, the instruction of the user is obtained.

[0015] Optionally, the home control system includes at least one voice collection device; and based on the instruction, controlling at least one target home device in the target action area to provide services for the user includes:

[0016] Acquire the device distance between the at least one voice collection device and the collection time of the voice collection device for the voice command;

[0017] The horizontal distance between the user and the voice collection device is acquired by using the device distance and the collection time.

[0018] Optionally, the voice command is a time domain signal; and based on the command, controlling at least one target home device in the target action area to provide a service for the user includes:

[0019] Converting the time domain signal into a frequency domain signal using a preset fast Fourier transform algorithm;

[0020] Extracting phase information of the sound wave frequency of the voice command from the frequency domain signal;

[0021] Using the phase information corresponding to the at least one voice collection device, obtaining the phase difference of the at least one voice collection device;

[0022] Acquiring the sound intensity of the voice command collected by the at least one voice collection device;

[0023] The vertical distance between the user and the voice collection device is acquired by using the phase difference and the sound intensity.

[0024] Optionally, controlling at least one target home device in the target action area to provide a service for the user based on the instruction includes:

[0025] Based on the horizontal distance and the vertical distance, obtaining the distance between the user and the voice collection device;

[0026] Obtaining the time when the voice command was issued;

[0027] determining a processable action area from at least one of the target action areas based on the distance and the time when the voice command is issued;

[0028] The target home device in the processable action area is controlled to provide services for the user.

[0029] Optionally, controlling at least one target home device in the target action area to provide a service for the user based on the instruction includes:

[0030] Based on the instruction, determining a processable home device from at least one of the target home devices;

[0031] Using the voice recognition model and / or the image recognition model, obtaining user information corresponding to the voice command;

[0032] Based on the user information, determining whether the user has permission to use the processable household device;

[0033] If the user has the permission to use the processable household device, the processable household device is controlled to provide services to the user.

[0034] Optionally, the method comprises:

[0035] Obtaining a historical usage frequency and / or a historical usage success rate of at least one of the users for the processable household device;

[0036] The usage permission of at least one of the users for the processable household device is determined according to the historical usage frequency and / or the historical usage success rate.

[0037] Optionally, the method comprises:

[0038] Obtaining a training voice instruction of at least one of the users, a training user picture of the user issuing the training voice instruction, training user information corresponding to the training voice instruction, and a training text instruction corresponding to the training voice instruction;

[0039] Using the training voice instruction, the training user information and the training text instruction, a preset first machine learning model is trained to obtain the speech recognition model;

[0040] The preset second machine learning model is trained using the training user pictures, the training user information and the training text instructions to obtain the picture recognition model.

[0041] Optionally, the method comprises:

[0042] Configuring a preset identity recognition voice command for the user;

[0043] By determining whether the voice instruction is the same as the identity recognition voice instruction, user information corresponding to the voice instruction is obtained.

[0044] The embodiment of the present invention further discloses a control device for home appliances, which is applied to a home control system. The home control system is used to control at least one home appliance in a preset home environment and provide services for at least one action of a user in the home environment. The device includes:

[0045] A historical action information acquisition module, used to acquire the historical action time and historical action area of ​​the at least one action, and acquire the associated information of the at least one action; the associated information is used to indicate whether there is an associated relationship between the at least one action;

[0046] An action set obtaining module, used for combining the actions having an associated relationship according to the associated information to obtain at least one action set;

[0047] The action area to be processed is used as a module, for any action set, taking the historical action area of ​​the action in the action set as the action area to be processed, and obtaining the predicted time when the user performs the action in the action set according to the historical action time;

[0048] The target action area is used as a module, which is used to take the action area to be processed at the predicted time as the target action area, and allocate a target home device to the target action area from the at least one home device; the target home device is a home device that provides services for the actions in the action set;

[0049] The device control module is used to obtain the user's instructions and, based on the instructions, control at least one target home device in the target action area to provide services for the user.

[0050] Optionally, the device control module includes:

[0051] A voice command acquisition submodule, used to acquire the voice command issued by the user and the user picture when the user issues the voice command;

[0052] A voice recognition instruction obtaining submodule, used for inputting the voice instruction into a preset voice recognition model to obtain the voice recognition instruction of the user;

[0053] A picture recognition instruction obtaining submodule, used for inputting the user picture into a preset picture recognition model to obtain the user's picture recognition instruction;

[0054] The instruction acquisition submodule is used to acquire the instruction of the user based on the voice recognition instruction and the picture recognition instruction.

[0055] Optionally, the home control system includes at least one voice acquisition device; the device control module includes:

[0056] A device distance acquisition submodule, used to acquire the device distance between the at least one voice collection device and the collection time of the voice collection device for the voice command;

[0057] The horizontal distance acquisition submodule is used to acquire the horizontal distance between the user and the voice acquisition device by using the device distance and the acquisition time.

[0058] Optionally, the voice command is a time domain signal; and the device control module includes:

[0059] A signal conversion submodule, used to convert the time domain signal into a frequency domain signal using a preset fast Fourier transform algorithm;

[0060] A phase information extraction submodule, used to extract the phase information of the sound wave frequency of the voice command from the frequency domain signal;

[0061] A phase difference acquisition submodule, used to acquire the phase difference of the at least one voice collection device by using the phase information corresponding to the at least one voice collection device;

[0062] A sound intensity acquisition submodule, used to acquire the sound intensity of the voice command collected by the at least one voice collection device;

[0063] The vertical distance acquisition submodule is used to acquire the vertical distance between the user and the voice collection device by using the phase difference and the sound intensity.

[0064] Optionally, the device control module includes:

[0065] A distance acquisition submodule, used to acquire the distance between the user and the voice collection device based on the horizontal distance and the vertical distance;

[0066] The issuing time acquisition submodule is used to obtain the issuing time of the voice command;

[0067] a processable action area determination submodule, configured to determine a processable action area from at least one of the target action areas based on the distance and the issuance time of the voice command;

[0068] The device control submodule is used to control the target home devices in the processable action area to provide services for the user.

[0069] Optionally, the device control module includes:

[0070] A processable household device determination submodule, configured to determine a processable household device from at least one of the target household devices based on the instruction;

[0071] A user information acquisition submodule, used to acquire user information corresponding to the voice command using the voice recognition model and / or the image recognition model;

[0072] A permission judgment submodule, used for judging whether the user has permission to use the processable household device based on the user information;

[0073] The processable household equipment control submodule is used to control the processable household equipment to provide services to the user if the user has the permission to use the processable household equipment.

[0074] Optionally, the device comprises:

[0075] A historical usage success rate acquisition module, used to acquire the historical usage frequency and / or historical usage success rate of at least one of the users for the processable household device;

[0076] The usage permission determination module is used to determine the usage permission of at least one of the users for the processable household device according to the historical usage frequency and / or the historical usage success rate.

[0077] Optionally, the device comprises:

[0078] A training data acquisition module, used to acquire a training voice instruction of at least one of the users, a training user picture of the user issuing the training voice instruction, training user information corresponding to the training voice instruction, and a training text instruction corresponding to the training voice instruction;

[0079] A first training module, used to train a preset first machine learning model using the training voice instruction, the training user information and the training text instruction to obtain the speech recognition model;

[0080] The second training module is used to train a preset second machine learning model using the training user pictures, the training user information and the training text instructions to obtain the picture recognition model.

[0081] Optionally, the device comprises:

[0082] A configuration module, used to configure a preset identity recognition voice instruction for the user;

[0083] The user information acquisition module is used to acquire the user information corresponding to the voice instruction by determining whether the voice instruction is the same as the identity recognition voice instruction.

[0084] The embodiment of the present invention further discloses an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0085] The memory is used to store computer programs;

[0086] The processor is used to implement the method described in the embodiment of the present invention when executing the program stored in the memory.

[0087] The embodiment of the present invention further discloses one or more computer-readable media on which instructions are stored. When executed by one or more processors, the processors are enabled to execute the method described in the embodiment of the present invention.

[0088] The embodiments of the present invention include the following advantages:

[0089] In an embodiment of the present invention, a home control system is used to control at least one home appliance in a preset home environment and provide services for at least one action of a user in the home environment. The home control system obtains the historical action time and historical action area of ​​at least one action, and obtains the association information of at least one action; the association information is used to indicate whether there is an association relationship between at least one action; according to the association information, the actions with an association relationship are combined to obtain at least one action set; for any action set, the historical action area of ​​the action in the action set is used as the action area to be processed, and according to the historical action time, the predicted time for the user to perform the action in the action set is obtained; the action area to be processed at the predicted time is used as the target action area, and a target home appliance is allocated to the target action area from at least one home appliance; the target home appliance is a home appliance that provides services for the actions in the action set; the user's instructions are obtained, and based on the instructions, the target home appliance in at least one target action area is controlled to provide services to the user. Different target action areas are determined based on the functional association between the actions of home users and home appliances, and the target home appliances in the target action area are used to respond to user instructions and provide services to users, thereby solving the management difficulties caused by the increase in the number of smart home appliances. Through partition networking, home appliances are allocated to different areas, which facilitates users to find and control the devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 is a flowchart of a method for controlling a household appliance provided in an embodiment of the present invention;

[0091] Figure 2 is a flowchart of steps of another method for controlling a household appliance provided in an embodiment of the present invention;

[0092] Figure 3 is a structural block diagram of a control device for household appliances provided in an embodiment of the present invention;

[0093] Figure 4 is a block diagram of an electronic device provided in an embodiment of the present invention;

[0094] Figure 5 is a schematic diagram of a computer-readable medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0095] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0096] To facilitate understanding of the technical solutions and technical effects of the embodiments of the present invention, the relevant technologies of the present invention are briefly described below.

[0097] With the continuous advancement of technology, the smart home market has developed rapidly, and more and more families have begun to use smart home devices to improve the convenience and comfort of life. However, in the process of using smart home systems to control the operation of smart home devices in related technologies, there are at least the following problems: First, with the increase in the number of smart home devices, it is difficult for smart home systems to effectively manage and control a large number of smart home devices. Second, different family members have different usage requirements and authority requirements for smart home devices, and it is difficult for smart home systems to meet the personalized needs of different users.

[0098] Reference Figure 1 , shows a flowchart of a method for controlling a home appliance provided in an embodiment of the present invention, which is applied to a home control system, wherein the home control system is used to control at least one home appliance in a preset home environment and provide services for at least one action of a user in the home environment; specifically, the following steps may be included:

[0099] Step 101, obtaining the historical action time and historical action area of ​​the at least one action, and obtaining the association information of the at least one action; the association information is used to indicate whether there is an association relationship between the at least one action;

[0100] In an embodiment of the present invention, a home environment includes at least one home appliance, and the home appliance can provide services for users in the home environment. The home appliance can be a television, a lamp, a curtain motor, an air purifier, etc. The user can perform daily actions in the home environment, and the home control system can control the home appliance in the home environment to provide services for at least one action of the user in the home environment.

[0101] In the embodiment of the present invention, the home appliance may be a smart device. The home appliance may be deployed in different areas according to the needs of different areas of the home environment. For example, a smart TV, a smart lamp with dimming and color adjustment, a smart curtain motor, and a smart air purifier may be installed in the living room; smart lamps, smart air conditioners, and smart humidifiers may be installed in the bedroom; smart lamps, smart refrigerators, and smart rice cookers may be installed in the kitchen; and smart lamps, smart water heaters, and smart toilets may be installed in the bathroom.

[0102] In an embodiment of the present invention, a home control system may also include a gateway. Specifically, if there are home appliances that use ZigBee or Bluetooth protocols in the home environment, it is necessary to select a stable and reliable gateway to ensure that its coverage can cover the area where all devices are located. ZigBee is a wireless communication protocol. In an embodiment of the present invention, the home appliances need to be connected to the home network. Specifically, you can turn on the power of the home appliance, enter the device setting mode, select the home Wi-Fi network and enter the password to connect the home appliance to Wi-Fi (wireless network). For some smart devices, you may need to use a mobile phone APP (application) for auxiliary connection settings. For home appliances that use ZigBee or Bluetooth protocols, you can connect the gateway device to the home network, and then pair the ZigBee or Bluetooth device with the gateway according to the device manual. Usually, these operations require adding and pairing devices on the mobile phone APP.

[0103] In an embodiment of the present invention, the home control system may obtain the historical action time and historical action area of ​​at least one action of the user, and obtain the association information of at least one action, wherein the association information may indicate whether there is an association between at least one action of the user.

[0104] Specifically, the home control system can record in detail and conduct big data analysis on the movement trajectory and behavior pattern of at least one family member at home on a certain day, and find out the historical action time and historical action area of ​​frequent and consistent actions.

[0105] In a specific example, after waking up one morning, the user performs actions such as getting up, getting dressed, and washing in the bedroom, then goes to the kitchen to prepare breakfast, and finally eats in the restaurant. This series of actions are related actions.

[0106] Step 102, combining the actions that have an associated relationship according to the associated information to obtain at least one action set;

[0107] In the embodiment of the present invention, whether the actions are associated is determined based on the association information between the actions. If the actions are associated, the associated actions are combined to obtain at least one action set.

[0108] Step 103, for any of the action sets, taking the historical action region of the action in the action set as the action region to be processed, and obtaining the predicted time for the user to perform the action in the action set according to the historical action time;

[0109] In the embodiment of the present invention, for any action set, the historical action region of the actions in the action set can be used as the action region to be processed, and the predicted time for the user to perform the actions in the action set can be determined based on the acquired historical action time of all actions.

[0110] In a specific example, after getting up one morning, the user gets up, dresses, washes, etc. in the bedroom, then goes to the kitchen to prepare breakfast, and finally eats in the dining room. This series of actions is related, and this series of actions is combined to obtain an action set. The historical action areas involved in the action set include the bedroom, kitchen, and dining room, so the bedroom, kitchen, and dining room can be used as the pending action areas of the action set. The historical action time of the actions in the action set is morning, so it can be predicted that the predicted time for the user to perform the actions in the action set is the morning of any future day.

[0111] Step 104, taking the action area to be processed at the predicted time as a target action area, and assigning a target home device to the target action area from the at least one home device; the target home device is a home device that provides services for the actions in the action set;

[0112] In the embodiment of the present invention, the action area to be processed at the predicted time can be used as the target action area, and the target action area can also be named. For any action set, the corresponding target action area in the home environment is determined, and the partitioning is realized based on the behavior flow of the home user and the functional association between the home devices.

[0113] In a specific example, after getting up one morning, the user gets up, dresses, washes in the bedroom, then goes to the kitchen to prepare breakfast, and finally eats in the dining room. This series of actions is related, and this series of actions is combined to obtain an action set. For this action set, the bedroom, kitchen, and dining room of any future morning can be used as the target action area and named "Morning Vitality Zone (MVZ)". MVZ can be used as an identifier for the morning vitality zone.

[0114] In another specific example, the user can watch movies on a smart TV or use a gaming device for entertainment in the living room at night. Therefore, the living room at night on any future day can be used as the target action area and named "Leisure Core Area (LCA)".

[0115] In another specific example, family members can communicate, discuss family affairs, hold family meetings, and carry out parent-child activities in the living room during the day. Therefore, the living room during the daytime of any future day can be used as the target action area and named "family communication and interaction area".

[0116] In another specific example, the user can use a smart makeup mirror to put on makeup and use an electric toothbrush to brush teeth in the bathroom in the morning. Therefore, the bathroom in the morning of any future day can be used as the target action area and named "morning washing and dressing area".

[0117] In another specific example, the user can use the smart water heater and the massage bathtub in the bathroom at night. Therefore, the bathroom at any night in the future can be used as a night bathing relaxation area.

[0118] In another specific example, users usually use professional drawing boards, high-resolution displays and other home devices in the study room in the afternoon for creative work; therefore, the study room in the afternoon of any future day can be used as the target action area and named "Focus Creation Area". Users usually read books and search for information in the study room at night through smart bookshelves and electronic data storage devices; therefore, the study room in the evening of any future day can be used as the target action area and named "Information Consultation Area", realizing the division of the study room at different times into different areas according to the work task type and equipment collaboration relationship.

[0119] In an embodiment of the present invention, the home control system can allocate a target home device to any target action area from at least one home device in the home environment. The target home device is a home device that can provide services for the actions in the action set corresponding to the target action area. For example, the home control system can allocate a smart beauty mirror and an electric toothbrush as target home devices for the "morning washing and dressing area".

[0120] In an embodiment of the present invention, the home control system includes an intelligent planning module, and the system can also provide an interface for interacting with the user. On this interface, a button named "Behavior-based Partition Creation" will be displayed, and the user can click this button to enter the intelligent planning module. In the intelligent planning module, the user can divide the target action area, name these areas, and configure corresponding identifiers. After the target action area is created, the home control system can automatically recommend and assign target home devices that can provide services for the actions in the action set corresponding to the target action area to the corresponding partitions according to the pre-set association algorithm between home appliances and behavioral functions. Users can also make manual adjustments according to actual conditions, such as allocating home appliances related to morning wake-up scenes such as smart alarm clocks and smart light controllers in the bedroom to the morning vitality zone, and allocating home appliances related to entertainment scenes such as smart TVs, game consoles, and smart sofa control systems in the living room to the leisure and entertainment core area.

[0121] Step 105: Obtain an instruction from the user, and based on the instruction, control at least one target home device in the target action area to provide services to the user.

[0122] In an embodiment of the present invention, the home control system may include at least one voice collection device, such as a microphone. These microphones may form a microphone array. Specifically, the home control system may include a smart speaker with a microphone array, which enhances the sound collection effect through the microphone array. In addition, the smart speaker is equipped with a powerful voice processing chip and has a wide range of smart home ecological compatibility. It also has high voice recognition accuracy and supports connection to multiple smart home protocols.

[0123] The home control system can use the smart speaker to obtain the instructions issued by the user in the home environment. Then, based on the instructions, it controls the target home devices in at least one target action area to provide services to the user.

[0124] In some embodiments of the present invention, the obtaining the instruction of the user includes:

[0125] Acquire the voice command issued by the user and the user picture when the user issues the voice command;

[0126] Inputting the voice command into a preset voice recognition model to obtain the voice recognition command of the user;

[0127] Inputting the user's picture into a preset picture recognition model to obtain the user's picture recognition instruction;

[0128] Based on the voice recognition instruction and the picture recognition instruction, the instruction of the user is obtained.

[0129] In an embodiment of the present invention, the home control system further includes at least one smart camera. It should be noted that the smart camera can be set on a smart speaker.

[0130] In an embodiment of the present invention, the home control system can use a smart speaker to obtain a voice command issued by a user, and use a smart camera to collect a user image when the user issues a voice command. The smart speaker of the home control system can input the voice command into a voice recognition model to obtain the user's voice recognition command. The smart speaker of the home control system can also input the user image collected by the smart camera into a picture recognition model to obtain the user's picture recognition command. Among them, the voice recognition command and the picture recognition command are both text-based commands.

[0131] In an embodiment of the present invention, the smart speaker of the home control system can integrate and analyze the voice recognition command and the image recognition command to obtain the user's command. The smart speaker can adopt a federated learning mechanism to share the learning results of the voice recognition model and the image recognition model with the smart speakers of other families.

[0132] It should be noted that when the smart speaker receives the voice command from the user, it can automatically adjust the gain and frequency response of the microphone. Specifically, the smart speaker can dynamically optimize the sound signal acquisition effect according to the distance between the voice command source and the microphone and the ambient noise level. The smart speaker can convert the received sound signal into a digital signal and pre-process it using an artificial intelligence noise reduction model. By learning from many home environment noise samples, the noise reduction model can effectively remove common noise interference such as the sound of electrical appliances running and traffic noise outside the window, thereby improving the quality of the voice signal.

[0133] In an embodiment of the present invention, the smart speaker adopts a multimodal fusion speech recognition algorithm, which not only analyzes traditional acoustic features, but also integrates the visual features of speech. When the user speaks, if the smart speaker is equipped with a camera or linked with other smart cameras in the home, it will capture the user's lip shape, facial expressions, body movements and other visual information, and fuse and analyze it with the voice signal, thereby significantly improving the accuracy of command recognition in complex environments or when the voice is unclear. That is, in occasions with high background noise such as family gatherings, even if part of the content of the voice command is masked by the noise, the smart speaker can still work with the smart camera to accurately understand the command intention and obtain the user's command by analyzing the user's lip shape and related facial expressions and body language.

[0134] In some embodiments of the invention, the method comprises:

[0135] Obtaining a training voice instruction of at least one of the users, a training user picture of the user issuing the training voice instruction, training user information corresponding to the training voice instruction, and a training text instruction corresponding to the training voice instruction;

[0136] Using the training voice instruction, the training user information and the training text instruction, a preset first machine learning model is trained to obtain the speech recognition model;

[0137] The preset second machine learning model is trained using the training user pictures, the training user information and the training text instructions to obtain the picture recognition model.

[0138] In an embodiment of the present invention, the home control system can obtain at least one user's training voice instructions, a training user picture of the user issuing the training voice instructions, training user information corresponding to the training voice instructions, and training text instructions corresponding to the training voice instructions. The training voice instructions, the training user information, and the training text instructions can be used to train a preset first machine learning model to obtain a voice recognition model, and the training user picture, the training user information, and the training text instructions can be used to train a preset second machine learning model to obtain a picture recognition model. The training voice instructions can include the user's voice habits, and the training text instructions can include vocabulary corresponding to the voice habits.

[0139] It should be noted that when training the model, you can choose a quiet environment for speech recognition training to avoid background noise interference. Specifically, you can train in a relatively closed room, such as a bedroom or study. At the same time, make sure that the microphone of the smart speaker is in good working condition and is not blocked or damaged. Specifically, you can check the indicator light of the smart speaker or check the microphone status through the mobile phone APP. When training the model, you can find the "Voice Training" option through the settings menu of the smart speaker or the mobile phone APP. After clicking to enter, follow the prompts. Each family member can say a series of commonly used commands and words to the smart speaker in turn, such as "turn on the lights", "close the curtains", "raise the temperature", "play music", etc. The smart speaker will record the voice characteristics of each family member, analyze and learn. In order to improve the accuracy of speech recognition, you can repeatedly enter voice samples. It is recommended that each family member conduct at least three to five trainings, with a period of time between each training to allow the smart speaker enough time to learn and optimize.

[0140] In the embodiment of the present invention, the model is personalized trained by utilizing the unique voice usage habits and vocabulary of the user, so that the conversion of digital signals into text instructions can be realized quickly and accurately.

[0141] In some embodiments of the present invention, the home control system includes at least one voice collection device; and based on the instruction, controlling at least one target home device in the target action area to provide services to the user includes:

[0142] Acquire the device distance between the at least one voice collection device and the collection time of the voice collection device for the voice command;

[0143] The horizontal distance between the user and the voice collection device is acquired by using the device distance and the collection time.

[0144] In an embodiment of the present invention, the home control system includes a smart speaker, the smart speaker includes a microphone array, the microphone array includes at least one microphone, and the microphone can be regarded as a voice collection device. The microphone array is reasonably arranged, with a reasonable number, position and spacing of microphones. The arrangement of the microphone array can be a plane array, a three-dimensional array, etc.

[0145] In the embodiment of the present invention, when the home control system obtains the command issued by the user, the smart speaker can use the microphone array and combine it with the spatial positioning technology to accurately determine the three-dimensional coordinate position of the voice command source, that is, the three-dimensional coordinate position of the user. That is, the smart speaker can analyze the time difference, phase difference and sound intensity difference between the sound signals received by multiple microphones, perform complex calculations, and can determine which room area the sound comes from, and can also accurately locate the specific position in the room. For example, it can be determined whether the command is issued on the left or right side of the sofa in the living room.

[0146] Specifically, when the user issues a voice command, each microphone in the microphone array receives the sound signal at the same time, and the home control system can pre-process the collected sound signal, including noise reduction, filtering and other operations, to remove background noise and other interference signals, and improve the quality and reliability of the signal. At the same time, the home control system can also perform sampling rate conversion, normalization and other processing on the sound signal. The home control system can convert the sound signal into an electrical signal.

[0147] In an embodiment of the present invention, the home control system can process the pre-processed sound data by framing, where each frame of data usually has a certain time length, such as 20ms-30ms (milliseconds). Then, a window function, such as a Hamming window, a rectangular window, etc., is applied to each frame of data to reduce the spectrum leakage problem caused by signal truncation.

[0148] In an embodiment of the present invention, the voice command is a time domain signal. The home control system can perform a fast Fourier transform on the audio data after frame windowing, convert the time domain signal into a frequency domain signal, and obtain the spectrum information of each microphone at different frequencies. Then, the cross-power spectrum between any two microphones can be calculated, which is achieved by conjugating and multiplying the frequency domain signals of the two microphones to obtain their correlation information at each frequency. Then, the cross-power spectrum can be weighted to enhance the time delay information in the signal, and then an inverse fast Fourier transform (IFFT) is performed to obtain a generalized cross-correlation function. Among them, the weighted function of the weighted processing can be PHAT (Phase Transform Generalized Cross-Correlation, phase transform generalized cross-correlation) and the like.

[0149] In the embodiment of the present invention, the home control system can find the maximum value of the generalized cross-correlation function, and the sample difference corresponding to the maximum value is the time difference between the two microphones receiving the sound signal. By performing the above calculation on different microphone combinations, multiple groups of time difference data can be obtained.

[0150] In an embodiment of the present invention, the home control system can obtain the device distance between at least one voice collection device and the collection time when at least one voice collection device collects the voice command. The horizontal distance between the user and the voice collection device can be obtained by using the device distance and collection time between at least one voice collection device.

[0151] Specifically, the coordinate position of the user on the two-dimensional plane can be calculated using geometric relationships (such as triangulation). For example, if the coordinates of two microphones on the preset two-dimensional plane are known to be (x1, y1) and (x2, y2), and the speed of sound is c, the coordinates of the user on the two-dimensional plane (x, y) can be calculated using the following formula:

[0152]

[0153] Among them, △t1 and △t2 are the time differences between the time when the user issues a voice command and the time when the two microphones collect the voice command.

[0154] After obtaining the coordinate position of the user in the two-dimensional plane, the horizontal distance between the user and the voice collection device can be obtained.

[0155] In some embodiments of the present invention, the voice command is a time domain signal; and based on the command, controlling at least one target home device in the target action area to provide services to the user includes:

[0156] Converting the time domain signal into a frequency domain signal using a preset fast Fourier transform algorithm;

[0157] Extracting phase information of the sound wave frequency of the voice command from the frequency domain signal;

[0158] Using the phase information corresponding to the at least one voice collection device, obtaining the phase difference of the at least one voice collection device;

[0159] Acquiring the sound intensity of the voice command collected by the at least one voice collection device;

[0160] The vertical distance between the user and the voice collection device is acquired by using the phase difference and the sound intensity.

[0161] In the embodiment of the present invention, the voice command is a time domain signal. The home control system can convert the time domain signal into a frequency domain signal using a preset fast Fourier transform algorithm.

[0162] In an embodiment of the present invention, the home control system can extract the phase information of each microphone at each frequency from the frequency domain signal converted by the fast Fourier transform (FFT) algorithm. For any two microphones, the phase difference between the two microphones at the same frequency can be calculated. By analyzing and processing the phase difference at multiple frequencies, the accurate phase difference information of the two microphones can be obtained. Since there may be factors such as multipath effects in the sound propagation process, this may cause deviations in the phase difference. Therefore, a phase correction algorithm, such as the phase correction method in GCC-PHAT (Generalized Cross-Correlation with Phase Transform), can be used to correct the phase difference to eliminate the influence of the multipath effect and improve the accuracy of the phase difference.

[0163] In an embodiment of the present invention, the sound intensity of the voice command collected by at least one voice collection device may be obtained, specifically by summing the squares of time domain signals or summing the amplitude squares of frequency domain signals.

[0164] In the embodiment of the present invention, the vertical distance between the user and the voice collection device can be obtained by using the phase difference and the sound intensity, and the height coordinate of the user can be calculated. For example, by analyzing the phase difference and intensity difference of the sound signals received by microphones at different heights, a corresponding mathematical model can be established to solve the height position of the user.

[0165] It should be noted that by comparing the sound energy between different microphones, the difference in sound intensity can be obtained. The difference in sound intensity can reflect the distance relationship between the sound source and each microphone. The closer the microphone is to the sound source, the greater the sound intensity it receives.

[0166] In some embodiments of the present invention, controlling at least one target home device in the target action area to provide services to the user based on the instruction includes:

[0167] Based on the horizontal distance and the vertical distance, obtaining the distance between the user and the voice collection device;

[0168] Obtaining the time when the voice command was issued;

[0169] determining a processable action area from at least one of the target action areas based on the distance and the time when the voice command is issued;

[0170] The target home device in the processable action area is controlled to provide services for the user.

[0171] In the embodiment of the present invention, the distance between the user and the voice collection device can be obtained by using the horizontal distance and the vertical distance between the user and the voice collection device, so as to determine the location of the user.

[0172] Since the target action area is determined based on the action area to be processed at the predicted time, the time when the voice command is issued can be obtained, and a processable action area can be determined from at least one target action area based on the time when the voice command is issued and the user's location. After determining the processable action area, the instructions issued by the user can be analyzed, and processable home appliances can be determined from the target home appliances in the processable action area, and services can be provided to the user using the processable home appliances.

[0173] In a specific example, after the user issues a voice command "I want to take a bath" in the bathroom at night, the target action area of ​​​​"evening bath soothing area" can be used as the processable action area, and then the smart water heater and jacuzzi in the "evening bath soothing area" can be used as processable home equipment.

[0174] In the embodiment of the present invention, when the home control system analyzes the instructions issued by the user, in addition to identifying the keywords in the instructions, the user's location and the time when the voice command is issued, it will also perform semantic reasoning based on factors such as the daily schedule of the home user, the recent history of home equipment use, and seasonal climate. For example, after 10 o'clock in the evening, if the user says "dim the lights", the smart speaker will automatically associate the instruction with the smart lights in the bedroom area and dim them to a brightness suitable for sleep; if the user says "lower the temperature" in summer, the smart speaker will give priority to corresponding the instruction to the air conditioning equipment in the living room or bedroom, and determine the appropriate cooling range based on historical setting habits and the current outdoor temperature.

[0175] Therefore, when analyzing the instructions issued by the user to determine the processable action area and processable home devices, the home control system will determine the processable action area based on the user's location and the time when the user issued the instruction. For example, if a target action area is in the bedroom and the predicted time is at night; at the predicted time of the target action area, when the user is in the bedroom and the user's voice instruction involves sleep-related operations, the home control system can infer that the instruction is most likely for the processable home device in the bedroom, and the processable home device is a device that can provide services for the user's sleep.

[0176] On the other hand, the user's location and the time when the user issues the command are not absolute criteria for determining the processable action area. Even in non-bedroom areas, if the user mentions a command related to sleep, the home control system needs to consider whether to operate related devices in other areas. For example, if the user asks to turn off all lights in the living room to prepare for bed, the home control system cannot ignore the command simply because it is not in the bedroom partition and not control the processable home devices in the bedroom. Therefore, the home control system can cross the partition boundary based on semantic reasoning of recent device usage history and seasonal climate factors to determine the processable action area and processable home devices. For example, if the air conditioner in the living room has been used frequently recently, when the user issues the command "lower the temperature", the home control system will give priority to adjusting the air conditioner in the living room, rather than just limiting it to the devices in a specific partition. In terms of seasonal climate, in summer, no matter which area the command is issued in, as long as the command related to cooling and ventilation is mentioned, the home control system will comprehensively consider the status and usage of the refrigeration equipment in each area (such as bedroom, living room, etc.) to process it, rather than simply determining the operation mode of the device based on the partition.

[0177] In an embodiment of the present invention, after determining the processable home devices, the smart speaker of the home control system can send instructions to the processable home devices in the processable action area. Specifically, the home control system can adopt a distributed smart contract mechanism based on blockchain, and each home device serves as a node in the blockchain network. The smart speaker can encapsulate the instructions into smart contract transactions and send the smart contract transactions to the blockchain network.

[0178] After the node that can process home devices receives the smart contract transaction, it automatically executes the operation according to the instruction requirements in the contract and its own functional permissions, and records the execution results on the blockchain to ensure that the execution of the instruction cannot be tampered with and is traceable. For example, when the user issues the command "Open the smart curtains of Family Hub", the smart speaker creates and broadcasts the smart contract transaction for opening the curtains. After receiving the transaction, the smart curtain device node in the living room executes the opening operation and records the successful execution information in the blockchain. Other related devices (such as the smart lighting device in the living room) can also adjust their own status synchronously according to the information on the blockchain, such as automatically brightening the lights to adapt to the light changes after the curtains are opened. After the device executes the instruction, it will send feedback information to the smart speaker through the blockchain network. The smart speaker uses natural language generation technology to synthesize personalized voice feedback to the user based on the feedback information and the preset voice template. For example, if the smart curtains are successfully opened, the smart speaker will generate voice feedback such as "Dear, the smart curtains of Family Hub have been opened for you. The sun is shining outside. I wish you a good time" according to the current time, weather and user preferences.

[0179] In some embodiments of the invention, the method comprises:

[0180] Configuring a preset identity recognition voice command for the user;

[0181] By determining whether the voice instruction is the same as the identity recognition voice instruction, user information corresponding to the voice instruction is obtained.

[0182] In an embodiment of the present invention, a preset identity recognition voice instruction can be configured for at least one user. The home control system can obtain the user information corresponding to the voice instruction by determining whether the voice instruction is the same as the identity recognition voice instruction. Specifically, the identity recognition voice instruction can also be called a wake-up word. The father can set the wake-up word to "Smart Butler Dad", the mother can set it to "Smart Assistant Mom", the child can set it to "Magic Elf", etc. This makes it convenient for the smart speaker to quickly recognize the instructions of different users.

[0183] In some embodiments of the present invention, controlling at least one target home device in the target action area to provide services to the user based on the instruction includes:

[0184] Based on the instruction, determining a processable home device from at least one of the target home devices;

[0185] Using the voice recognition model and / or the image recognition model, obtaining user information corresponding to the voice command;

[0186] Based on the user information, determining whether the user has permission to use the processable household device;

[0187] If the user has the permission to use the processable household device, the processable household device is controlled to provide services to the user.

[0188] In the embodiment of the present invention, according to the instruction, the home control system can determine the processable home device from at least one target home device in the processable action area.

[0189] In the embodiment of the present invention, the home control system can use the voice recognition model and / or the picture recognition model to obtain the user information of the user who issued the voice command. Specifically, the voice command can be input into the voice recognition model to obtain the user information; the user picture when the user issues the voice command can also be input into the picture recognition model to obtain the user information.

[0190] In the embodiment of the present invention, it can be determined whether the user has the authority to use the processable household device according to the user information. If the user has the authority to use the processable household device, the processable household device is controlled to provide services for the user.

[0191] Specifically, when a user issues a voice command, the smart speaker will first perform permission verification. The smart speaker identifies the user by analyzing the user's voice characteristics and personalized wake-up words. Subsequently, the smart speaker will check whether the home devices and operations involved in the command are within the user's authority. If the user does not have the authority to operate the home device or perform the corresponding operation, the smart speaker will refuse to execute the command and prompt the user "You do not have permission to operate this device." If the command is within the user's authority, the smart speaker will continue the command recognition and execution process.

[0192] In an embodiment of the present invention, in addition to using voice features and wake-up words, a variety of methods can be used to verify the identity of the user, thereby enhancing the security of the system. For example, a password can be set, fingerprint recognition, facial recognition, etc. can be performed. When a user issues a voice command, the smart speaker may ask the user to authenticate the identity, and the command will only be executed after the verification is successful. For sensitive devices or operations, the system will adopt a permission verification method that combines biometrics and environmental factors. For example, for a smart safe, in addition to standard fingerprint recognition, the ambient temperature and humidity changes around the safe, as well as the user's heart rate, skin conductance response and other physiological characteristics will be considered for comprehensive verification. Operations are allowed only when these biometrics match the preset normal range and there are no abnormal fluctuations in environmental factors.

[0193] In terms of data transmission of home control systems, encryption technology should be used to ensure that user instructions and device feedback information are not stolen or tampered with during transmission. For example, the SSL / TLS (Secure Sockets Layer / Transport Layer Security) encryption protocol can be used to encrypt data. In addition, the software and firmware of the home control system need to be updated regularly to fix possible security vulnerabilities. At the same time, the security of the home network should be ensured, such as setting complex passwords, enabling firewalls and other measures to prevent hackers from invading the home control system. Home equipment should be regularly inspected and maintained to ensure its normal operation and safety. If a device is found to have a security risk, measures should be taken immediately to repair or replace it.

[0194] In some embodiments of the invention, the method comprises:

[0195] Obtaining a historical usage frequency and / or a historical usage success rate of at least one of the users for the processable household device;

[0196] The usage permission of at least one of the users for the processable household device is determined according to the historical usage frequency and / or the historical usage success rate.

[0197] In an embodiment of the present invention, the historical usage frequency and / or historical usage success rate of at least one user for the processable household device can be obtained, and the usage rights of at least one user for the processable household device can be determined based on the historical usage frequency and / or historical usage success rate.

[0198] Specifically, a dynamic permission allocation model is included in the home control system. The model can set the use rights of family members to home devices based on diversified dynamic data such as the daily activity patterns of family members, the proficiency of using home devices, and the contribution of participating in the management of family affairs. For example, if a family member has recently frequently and successfully operated complex smart kitchen equipment to complete a variety of cooking tasks, then the member's permissions on kitchen-related smart devices will be appropriately improved, such as unlocking the use rights of certain advanced cooking modes. Similarly, if a family member actively participates in the daily inspection and maintenance of home security equipment, then the member's partial use rights in the smart security system will also be improved accordingly, such as temporarily adjusting the coverage of the monitoring area. In an embodiment of the present invention, a temporary permission mechanism is also designed. When a visitor needs to use a specific smart device (such as the screen projection function of a smart TV), the administrator can quickly generate a temporary permission QR code with a limited time and limited function through the mobile phone. After scanning, the visitor can use the corresponding device within the specified time and scope.

[0199] In an embodiment of the present invention, a home control system is used to control at least one home appliance in a preset home environment and provide services for at least one action of a user in the home environment. The home control system obtains the historical action time and historical action area of ​​at least one action, and obtains the association information of at least one action; the association information is used to indicate whether there is an association relationship between at least one action; according to the association information, the actions with an association relationship are combined to obtain at least one action set; for any action set, the historical action area of ​​the action in the action set is used as the action area to be processed, and according to the historical action time, the predicted time for the user to perform the action in the action set is obtained; the action area to be processed at the predicted time is used as the target action area, and a target home appliance is allocated to the target action area from at least one home appliance; the target home appliance is a home appliance that provides services for the actions in the action set; the user's instructions are obtained, and based on the instructions, the target home appliance in at least one target action area is controlled to provide services to the user. Different target action areas are determined based on the functional association between the actions of home users and home appliances, and the target home appliances in the target action area are used to respond to user instructions and provide services to users, thereby solving the management problems caused by the increase in the number of smart home appliances. Through partition networking, home devices are allocated to different areas, making it easier for users to find and control devices, improving the pertinence and accuracy of device control, avoiding indiscriminate control of the entire home space, and improving user experience.

[0200] In the embodiment of the present invention, the use rights for home appliances are determined for different users in the family. The division of multi-user rights can avoid user misoperation or malicious operation, thereby improving the security and stability of the equipment. Different users can control the equipment in the corresponding area according to their own rights, which meets the personalized needs of different family members for smart home appliances and improves user satisfaction and usage experience. The administrator among the family users can comprehensively manage and control the entire smart home system, while ordinary users can only control the equipment in their own area, thereby improving the security and management efficiency of the system.

[0201] In the embodiment of the present invention, the voice recognition technology of the smart speaker allows users to interact with smart devices through natural language without manual operation, which improves the convenience and comfort of users. At the same time, the accuracy and stability of voice recognition are also improved, reducing the occurrence of misoperation. Through voice recognition training and the setting of personalized wake-up words, the accuracy of voice recognition and user experience are improved.

[0202] Reference Figure 2 , shows a flowchart of the steps of another method for controlling a household appliance provided in an embodiment of the present invention, which may specifically include the following steps:

[0203] Step 201, equipment preparation and connection.

[0204] Step 202, partition setting.

[0205] The home control system can obtain the historical action time and historical action area of ​​at least one action, and obtain the association information of at least one action; the association information is used to indicate whether there is an association relationship between at least one action. According to the association information, the actions with association relationships are combined to obtain at least one action set; for any action set, the historical action area of ​​the actions in the action set is used as the action area to be processed, and according to the historical action time, the predicted time when the user performs the actions in the action set is obtained; the action area to be processed at the predicted time is used as the target action area, and a target home device is allocated to the target action area from at least one home device; the target home device is a home device that provides services for the actions in the action set.

[0206] Step 203: speech recognition training.

[0207] The home control system obtains training voice instructions of at least one user, a training user picture of the user issuing the training voice instructions, training user information corresponding to the training voice instructions, and training text instructions corresponding to the training voice instructions; uses the training voice instructions, training user information, and training text instructions to train a preset first machine learning model to obtain a voice recognition model; uses the training user picture, training user information, and training text instructions to train a preset second machine learning model to obtain an image recognition model.

[0208] Step 204: user rights division.

[0209] The home control system obtains the historical usage frequency and / or historical usage success rate of at least one user for the processable home equipment; and determines the usage authority of at least one user for the processable home equipment based on the historical usage frequency and / or historical usage success rate.

[0210] Step 205: Voice command recognition and execution.

[0211] The home control system obtains the voice command issued by the user and the user's picture when the user issues the voice command; inputs the voice command into a preset voice recognition model to obtain the user's voice recognition command; inputs the user's picture into a preset picture recognition model to obtain the user's picture recognition command; obtains the user's command based on the voice recognition command and the picture recognition command. Based on the command, the target home device in at least one target action area is controlled to provide services to the user.

[0212] Step 206: authority verification and security management.

[0213] The home control system uses a voice recognition model and / or an image recognition model to obtain user information corresponding to the voice command; based on the user information, it determines whether the user has the authority to use the processable home equipment; if the user has the authority to use the processable home equipment, the processable home equipment is controlled to provide services to the user.

[0214] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0215] Reference Figure 3 , shows a structural block diagram of a control device for home appliances provided in an embodiment of the present invention, which is applied to a home control system, wherein the home control system is used to control at least one home appliance in a preset home environment and provide services for at least one action of a user in the home environment; specifically, it may include the following modules:

[0216] The historical action information acquisition module 301 is used to acquire the historical action time and historical action area of ​​the at least one action, and acquire the associated information of the at least one action; the associated information is used to indicate whether there is an associated relationship between the at least one action;

[0217] An action set obtaining module 302 is used to combine the actions having an associated relationship according to the associated information to obtain at least one action set;

[0218] The action area to be processed is used as a module 303, for taking the historical action area of ​​the action in the action set as the action area to be processed for any action set, and obtaining the predicted time when the user performs the action in the action set according to the historical action time;

[0219] The target action area is used as a module 304, which is used to take the action area to be processed at the predicted time as the target action area, and allocate a target home device to the target action area from the at least one home device; the target home device is a home device that provides services for the actions in the action set;

[0220] The device control module 305 is used to obtain the user's instructions and, based on the instructions, control at least one target home device in the target action area to provide services for the user.

[0221] In an optional embodiment of the present invention, the device control module includes:

[0222] A voice command acquisition submodule, used to acquire the voice command issued by the user and the user picture when the user issues the voice command;

[0223] A voice recognition instruction obtaining submodule, used for inputting the voice instruction into a preset voice recognition model to obtain the voice recognition instruction of the user;

[0224] A picture recognition instruction obtaining submodule, used for inputting the user picture into a preset picture recognition model to obtain the user's picture recognition instruction;

[0225] The instruction acquisition submodule is used to acquire the instruction of the user based on the voice recognition instruction and the picture recognition instruction.

[0226] In an optional embodiment of the present invention, the home control system includes at least one voice acquisition device; the device control module includes:

[0227] A device distance acquisition submodule, used to acquire the device distance between the at least one voice collection device and the collection time of the voice collection device for the voice command;

[0228] The horizontal distance acquisition submodule is used to acquire the horizontal distance between the user and the voice acquisition device by using the device distance and the acquisition time.

[0229] In an optional embodiment of the present invention, the voice instruction is a time domain signal; and the device control module includes:

[0230] A signal conversion submodule, used to convert the time domain signal into a frequency domain signal using a preset fast Fourier transform algorithm;

[0231] A phase information extraction submodule, used to extract the phase information of the sound wave frequency of the voice command from the frequency domain signal;

[0232] A phase difference acquisition submodule, used to acquire the phase difference of the at least one voice collection device by using the phase information corresponding to the at least one voice collection device;

[0233] A sound intensity acquisition submodule, used to acquire the sound intensity of the voice command collected by the at least one voice collection device;

[0234] The vertical distance acquisition submodule is used to acquire the vertical distance between the user and the voice collection device by using the phase difference and the sound intensity.

[0235] In an optional embodiment of the present invention, the device control module includes:

[0236] A distance acquisition submodule, used to acquire the distance between the user and the voice collection device based on the horizontal distance and the vertical distance;

[0237] The issuing time acquisition submodule is used to obtain the issuing time of the voice command;

[0238] a processable action area determination submodule, configured to determine a processable action area from at least one of the target action areas based on the distance and the issuance time of the voice command;

[0239] The device control submodule is used to control the target home devices in the processable action area to provide services for the user.

[0240] In an optional embodiment of the present invention, the device control module includes:

[0241] A processable household device determination submodule, configured to determine a processable household device from at least one of the target household devices based on the instruction;

[0242] A user information acquisition submodule, used to acquire user information corresponding to the voice command using the voice recognition model and / or the image recognition model;

[0243] A permission judgment submodule, used for judging whether the user has permission to use the processable household device based on the user information;

[0244] The processable household equipment control submodule is used to control the processable household equipment to provide services to the user if the user has the permission to use the processable household equipment.

[0245] In an optional embodiment of the present invention, the device comprises:

[0246] A historical usage success rate acquisition module, used to acquire the historical usage frequency and / or historical usage success rate of at least one of the users for the processable household device;

[0247] The usage permission determination module is used to determine the usage permission of at least one of the users for the processable household device according to the historical usage frequency and / or the historical usage success rate.

[0248] In an optional embodiment of the present invention, the device comprises:

[0249] A training data acquisition module, used to acquire a training voice instruction of at least one of the users, a training user picture of the user issuing the training voice instruction, training user information corresponding to the training voice instruction, and a training text instruction corresponding to the training voice instruction;

[0250] A first training module, used to train a preset first machine learning model using the training voice instruction, the training user information and the training text instruction to obtain the speech recognition model;

[0251] The second training module is used to train a preset second machine learning model using the training user pictures, the training user information and the training text instructions to obtain the picture recognition model.

[0252] In an optional embodiment of the present invention, the device comprises:

[0253] A configuration module, used to configure a preset identity recognition voice instruction for the user;

[0254] The user information acquisition module is used to acquire the user information corresponding to the voice instruction by determining whether the voice instruction is the same as the identity recognition voice instruction.

[0255] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0256] In addition, an embodiment of the present invention further provides an electronic device, such as Figure 4 As shown, it includes a processor 401, a communication interface 402, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404.

[0257] Memory 403, used for storing computer programs;

[0258] The processor 401 is used to execute the program stored in the memory 403 to implement the following steps:

[0259] Acquire the historical action time and historical action area of ​​the at least one action, and acquire the association information of the at least one action; the association information is used to indicate whether there is an association relationship between the at least one action;

[0260] Combining the actions that have an associated relationship according to the associated information to obtain at least one action set;

[0261] For any of the action sets, a historical action region of the actions in the action set is used as an action region to be processed, and a predicted time for the user to perform the actions in the action set is obtained according to the historical action time;

[0262] The to-be-processed action area at the predicted time is used as a target action area, and a target home device is allocated to the target action area from the at least one home device; the target home device is a home device that provides services for the actions in the action set;

[0263] Obtain instructions from the user, and based on the instructions, control at least one target home device in the target action area to provide services to the user.

[0264] In an optional embodiment of the present invention, the obtaining the instruction of the user includes:

[0265] Acquire the voice command issued by the user and the user picture when the user issues the voice command;

[0266] Inputting the voice command into a preset voice recognition model to obtain the voice recognition command of the user;

[0267] Inputting the user's picture into a preset picture recognition model to obtain the user's picture recognition instruction;

[0268] Based on the voice recognition instruction and the picture recognition instruction, the instruction of the user is obtained.

[0269] In an optional embodiment of the present invention, the home control system includes at least one voice collection device; and based on the instruction, controlling at least one target home device in the target action area to provide services to the user includes:

[0270] Acquire the device distance between the at least one voice collection device and the collection time of the voice collection device for the voice command;

[0271] The horizontal distance between the user and the voice collection device is acquired by using the device distance and the collection time.

[0272] In an optional embodiment of the present invention, the voice instruction is a time domain signal; and based on the instruction, controlling at least one target home device in the target action area to provide services to the user includes:

[0273] Converting the time domain signal into a frequency domain signal using a preset fast Fourier transform algorithm;

[0274] Extracting phase information of the sound wave frequency of the voice command from the frequency domain signal;

[0275] Using the phase information corresponding to the at least one voice collection device, obtaining the phase difference of the at least one voice collection device;

[0276] Acquiring the sound intensity of the voice command collected by the at least one voice collection device;

[0277] The vertical distance between the user and the voice collection device is acquired by using the phase difference and the sound intensity.

[0278] In an optional embodiment of the present invention, controlling at least one target home device in the target action area to provide services to the user based on the instruction includes:

[0279] Based on the horizontal distance and the vertical distance, obtaining the distance between the user and the voice collection device;

[0280] Obtaining the time when the voice command was issued;

[0281] determining a processable action area from at least one of the target action areas based on the distance and the time when the voice command is issued;

[0282] The target home device in the processable action area is controlled to provide services for the user.

[0283] In an optional embodiment of the present invention, controlling at least one target home device in the target action area to provide services to the user based on the instruction includes:

[0284] Based on the instruction, determining a processable home device from at least one of the target home devices;

[0285] Using the voice recognition model and / or the image recognition model, obtaining user information corresponding to the voice command;

[0286] Based on the user information, determining whether the user has permission to use the processable household device;

[0287] If the user has the permission to use the processable household device, the processable household device is controlled to provide services to the user.

[0288] In an optional embodiment of the present invention, the method includes:

[0289] Obtaining a historical usage frequency and / or a historical usage success rate of at least one of the users for the processable household device;

[0290] The usage permission of at least one of the users for the processable household device is determined according to the historical usage frequency and / or the historical usage success rate.

[0291] In an optional embodiment of the present invention, the method includes:

[0292] Obtaining a training voice instruction of at least one of the users, a training user picture of the user issuing the training voice instruction, training user information corresponding to the training voice instruction, and a training text instruction corresponding to the training voice instruction;

[0293] Using the training voice instruction, the training user information and the training text instruction, a preset first machine learning model is trained to obtain the speech recognition model;

[0294] The preset second machine learning model is trained using the training user pictures, the training user information and the training text instructions to obtain the picture recognition model.

[0295] In an optional embodiment of the present invention, the method includes:

[0296] Configuring a preset identity recognition voice command for the user;

[0297] By determining whether the voice instruction is the same as the identity recognition voice instruction, user information corresponding to the voice instruction is obtained.

[0298] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0299] The communication interface is used for communication between the above terminal and other devices.

[0300] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0301] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0302] like Figure 5 As shown, in another embodiment provided by the present invention, a computer-readable storage medium 501 is also provided, in which instructions are stored. When the computer-readable storage medium 501 is run on a computer, the computer executes a method for controlling a household appliance described in the above embodiment.

[0303] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is run on a computer, the computer executes a method for controlling a household appliance described in the above embodiment.

[0304] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0305] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0306] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0307] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for controlling a household appliance, characterized in that: Applied to a home control system, the home control system is used to control at least one home device in a preset home environment and provide services for at least one action of a user in the home environment; the method comprises: Acquire the historical action time and historical action area of ​​the at least one action, and acquire the association information of the at least one action; the association information is used to indicate whether there is an association relationship between the at least one action; Combining the actions that have an associated relationship according to the associated information to obtain at least one action set; For any of the action sets, a historical action region of the actions in the action set is used as an action region to be processed, and a predicted time for the user to perform the actions in the action set is obtained according to the historical action time; The to-be-processed action area at the predicted time is used as a target action area, and a target home device is allocated to the target action area from the at least one home device; the target home device is a home device that provides services for the actions in the action set; Obtain instructions from the user, and based on the instructions, control at least one target home device in the target action area to provide services to the user.

2. The method according to claim 1, characterized in that The obtaining of the user's instruction includes: Acquire the voice command issued by the user and the user picture when the user issues the voice command; Inputting the voice command into a preset voice recognition model to obtain the voice recognition command of the user; Inputting the user's picture into a preset picture recognition model to obtain the user's picture recognition instruction; Based on the voice recognition instruction and the picture recognition instruction, the instruction of the user is obtained.

3. The method according to claim 2, characterized in that The home control system includes at least one voice collection device; and based on the instruction, controlling at least one target home device in the target action area to provide services to the user includes: Acquire the device distance between the at least one voice collection device and the collection time of the voice collection device for the voice command; The horizontal distance between the user and the voice collection device is acquired by using the device distance and the collection time.

4. The method according to claim 3, characterized in that The voice command is a time domain signal; and based on the command, controlling at least one target home device in the target action area to provide services to the user includes: Converting the time domain signal into a frequency domain signal using a preset fast Fourier transform algorithm; Extracting phase information of the sound wave frequency of the voice command from the frequency domain signal; Using the phase information corresponding to the at least one voice collection device, obtaining the phase difference of the at least one voice collection device; Acquiring the sound intensity of the voice command collected by the at least one voice collection device; The vertical distance between the user and the voice collection device is acquired by using the phase difference and the sound intensity.

5. The method according to claim 4, characterized in that The controlling, based on the instruction, at least one target home device in the target action area to provide services to the user comprises: Based on the horizontal distance and the vertical distance, obtaining the distance between the user and the voice collection device; Obtaining the time when the voice command was issued; determining a processable action area from at least one of the target action areas based on the distance and the time when the voice command is issued; The target home device in the processable action area is controlled to provide services for the user.

6. The method according to claim 2, characterized in that The controlling, based on the instruction, at least one target home device in the target action area to provide services to the user comprises: Based on the instruction, determining a processable home device from at least one of the target home devices; Using the voice recognition model and / or the image recognition model, obtaining user information corresponding to the voice command; Based on the user information, determining whether the user has permission to use the processable household device; If the user has the permission to use the processable household device, the processable household device is controlled to provide services to the user.

7. The method according to claim 6, characterized in that The method comprises: Obtaining a historical usage frequency and / or a historical usage success rate of at least one of the users for the processable household device; The usage permission of at least one of the users for the processable household device is determined according to the historical usage frequency and / or the historical usage success rate.

8. The method according to claim 2, characterized in that: The method comprises: Obtaining a training voice instruction of at least one of the users, a training user picture of the user issuing the training voice instruction, training user information corresponding to the training voice instruction, and a training text instruction corresponding to the training voice instruction; Using the training voice instructions, the training user information and the training text instructions, a preset first machine learning model is trained to obtain the speech recognition model; The preset second machine learning model is trained using the training user pictures, the training user information and the training text instructions to obtain the picture recognition model.

9. The method according to claim 2, characterized in that: The method comprises: Configuring a preset identity recognition voice command for the user; By determining whether the voice instruction is the same as the identity recognition voice instruction, user information corresponding to the voice instruction is obtained.

10. A control device for household appliances, characterized in that: Applied to a home control system, the home control system is used to control at least one home device in a preset home environment and provide services for at least one action of a user in the home environment; the device comprises: A historical action information acquisition module, used to acquire the historical action time and historical action area of ​​the at least one action, and acquire the associated information of the at least one action; the associated information is used to indicate whether there is an associated relationship between the at least one action; an action set obtaining module, configured to combine the actions having an associated relationship according to the associated information to obtain at least one action set; The action area to be processed is used as a module, for any action set, taking the historical action area of ​​the action in the action set as the action area to be processed, and obtaining the predicted time when the user performs the action in the action set according to the historical action time; The target action area is used as a module, which is used to take the action area to be processed at the predicted time as the target action area, and allocate a target home device to the target action area from the at least one home device; the target home device is a home device that provides services for the actions in the action set; The device control module is used to obtain the user's instructions and, based on the instructions, control at least one target home device in the target action area to provide services for the user.

11. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 9 when executing the program stored in the memory.

12. One or more computer-readable media having instructions stored thereon, which when executed by one or more processors cause the processors to perform the method according to any one of claims 1 to 9.

Images