Scene-based smart home control method, device, equipment and storage medium

Abstract

The embodiment of the present invention provides a scene-based smart home control method, device, equipment and storage medium, the method comprising: obtaining user voice data, and performing recognition processing on the voice data to obtain corresponding text data; recognizing the text data to determine whether the text data is a control text; when the text data is determined to be a control text, determining whether the text data is a scene control text at least based on a pre-built scene dictionary and classification model; when the text data is determined to be a scene control text, obtaining scene information corresponding to the scene control text, and controlling the working state of at least one corresponding smart home according to the scene information. The present invention sets a mapping relationship between different scenes and each smart home, so that multiple associated smart homes can be controlled simultaneously by issuing a voice control instruction including a scene, thereby improving the user experience.

Description

Technical Field

[0001] The present invention relates to the field of smart homes, and in particular to a scene-based smart home control method, device, equipment and storage medium. Background Art

[0002] With the advancement of science and technology, the concept of smart home is being promoted continuously, and various smart home products are increasingly entering the lives of ordinary people, thus providing people with a more comfortable and convenient living environment. At present, smart home control systems are generally implemented based on voice technology, including voice recognition and semantic understanding. Voice recognition (ASR) converts voice signals into text, and semantic parsing uses natural language processing (NLP) technology in the field of artificial intelligence (AI) to identify and extract corresponding control commands from text, and then sends control commands to IoT devices through the big data platform so that the corresponding devices can respond to relevant operations.

[0003] Current smart home systems generally only support control of a single device, which means that in some cases users need to perform multiple voice interactions to control the corresponding smart device, resulting in a poor user experience. Summary of the invention

[0004] In view of this, an object of the present invention is to provide a scene-based smart home control method, device, equipment and storage medium to improve the above-mentioned problems.

[0005] An embodiment of the present invention provides a scenario-based smart home control method, which includes:

[0006] Acquire the user's voice data, and perform recognition processing on the voice data to obtain corresponding text data;

[0007] Identifying the text data to determine whether the text data is a control text;

[0008] When it is determined that the text data is a control text, determining whether the text data is a scene control text at least based on a pre-built scene dictionary and a classification model;

[0009] When it is determined that the text data is a scene control text, scene information corresponding to the scene control text is obtained, and the working state of at least one corresponding smart home is controlled according to the scene information.

[0010] Preferably, obtaining the user's voice data and performing recognition processing on the voice data to obtain corresponding text data specifically includes:

[0011] Receive voice data sent by users through the channel end;

[0012] The voice data is subjected to text recognition, and the recognized text is subjected to text error correction and device normalization to obtain text data.

[0013] Preferably, identifying the text data to determine whether the text data is a control text specifically includes:

[0014] Performing rule matching on the text data through a pre-built control dictionary;

[0015] If the match is successful, the text data is determined to be a control text;

[0016] If the match is unsuccessful, the text data is input into a pre-trained first classification model to obtain a classification result; wherein the first classification model is a binary classification model trained using existing corpus based on TF-IDF and K nearest neighbor algorithm;

[0017] It is determined whether the text data is a control text according to the classification result.

[0018] Preferably, when it is determined that the text data is a control text, determining whether the text data is a scene control text is performed at least based on a pre-built scene dictionary and a classification model, specifically including:

[0019] When it is determined that the text data is a control text, rule matching is performed on the text data based on a pre-built scenario dictionary;

[0020] If the match is successful, the text data is marked as scene text, and scene information matching the text data is obtained;

[0021] If the match is unsuccessful, the text data is input into the trained second classification model to obtain the corresponding classification results and the probability value of each classification result; wherein the second classification model is a multi-classification model trained based on the pre-trained scene word vector and the K nearest neighbor algorithm;

[0022] Determine whether the probability value of the classification result with the largest probability value is greater than a preset threshold;

[0023] If it is greater than, output the classification result with the largest probability value to obtain the scene information of the text data;

[0024] If not, the output is a null value, and entity recognition is performed to obtain a control instruction corresponding to the text data.

[0025] Preferably, performing entity recognition to obtain a control instruction corresponding to the text data includes:

[0026] Marking each character in the text data according to a pre-trained marking model;

[0027] For the labeled text data, the deep learning model trained by the network structure combining BERT and CRF is used to extract entities to obtain the corresponding entities;

[0028] Normalize the entities and generate corresponding control commands.

[0029] Preferably, when marking each character, the marker includes:

[0030] The first marker indicating the start of the entity;

[0031] Indicates the second marker located inside the entity;

[0032] Represents a third tag outside of an entity.

[0033] Preferably, the type of the entity includes category, function, command, value and brand.

[0034] The embodiment of the present invention further provides a scene-based smart home control device, which includes:

[0035] A voice processing unit, used to obtain the user's voice data and perform recognition processing on the voice data to obtain corresponding text data;

[0036] an identification unit, used for identifying the text data to determine whether the text data is a control text;

[0037] A scene judgment unit, configured to judge whether the text data is a scene control text at least according to a pre-built scene dictionary and a classification model when judging that the text data is a control text;

[0038] A scene control unit is used to obtain scene information corresponding to the scene control text when it is determined that the text data is a scene control text, and control the working state of at least one corresponding smart home according to the scene information.

[0039] An embodiment of the present invention further provides a scene-based smart home control device, which includes a processor and a memory, wherein the memory stores a computer program, and the computer program can be executed by the processor to implement the scene-based smart home control method as described above.

[0040] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, wherein the computer program can be executed by a processor of a device where the computer-readable storage medium is located to implement the scene-based smart home control method as described above.

[0041] To sum up, this embodiment sets a mapping relationship between different scenes and each smart home, so that multiple associated smart homes can be controlled simultaneously by issuing a voice control command including the scene, avoiding the cumbersome operation problems caused by controlling multiple smart homes separately, and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 It is a flowchart of the scene-based smart home control method provided by the first embodiment of the present invention.

[0044] Figure 2 It is a structural diagram of a scene-based smart home control device provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] In order to better understand the technical solution of the present invention, the embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0047] It should be clear that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0049] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments:

[0050] See also Figure 1The first embodiment of the present invention provides a scene-based smart home control method, which can be executed by a scene-based smart home control device (hereinafter referred to as a control device), and in particular, is executed by one or more processors in the control device to implement the following steps:

[0051] S101, acquiring user's voice data, and performing recognition processing on the voice data to obtain corresponding text data.

[0052] In this embodiment, the smart home refers to various smart devices set in a home environment, such as smart TVs, smart lighting, smart toilets, smart refrigerators, smart speakers, smart water heaters, smart fans, etc., which are not specifically limited in the present invention. Among them, these smart devices generally have networking capabilities and can control their functions through network connections. For example, these smart devices can all be connected to the router of the home device, and then connected to the control device in the home or the server in the cloud through the router.

[0053] In this embodiment, the user can send voice data through a channel end, such as various smart speakers, etc. After the channel end receives the voice data, it sends it to the control device through the network.

[0054] In this embodiment, the control device may be a control device installed locally at home, such as a user's smart phone, smart router, etc., or may be a server installed in the cloud, which is not specifically limited in the present invention.

[0055] In this embodiment, after obtaining the voice data, the control device may perform recognition processing on the voice data to obtain corresponding text data.

[0056] Specifically, after receiving the voice data, the control device performs voice recognition on it to obtain the corresponding text information, and then performs text error correction and device normalization on the text information to obtain the final text data. Among them, text error correction is mainly to correct the problem that the text recognized by voice is similar in pinyin but incorrect in words. The error correction scheme can be based on the kenLM language model, which can add specific scene error sets in real time to further improve the error correction effect. The tool used is pycorrector. Device normalization refers to type normalization based on the constructed device name dictionary, such as "Lake Smart Circulation Fan" -> "Fan", "Light Strip" -> "Light Bulb".

[0057] S102: Identify the text data to determine whether the text data is a control text.

[0058] In this embodiment, after obtaining the text data, it is necessary to determine the field to which the text data belongs, such as whether it is control text, chat text, or invalid content text.

[0059] Specifically:

[0060] S1021, construct a domain dictionary, and use the existing corpus to train a binary classification model based on TF-IDF (term frequency–inverse document frequency) and K nearest neighbor (KNN) algorithm.

[0061] Among them, TF-IDF means term frequency - inverse document frequency, where TF means term frequency, that is, the frequency of a word appearing in a text, and IDF means inverse document frequency, which is inversely proportional to the number of documents containing the word. By multiplying the TF and IDF values, we can get the TF-IDF value of a word. The larger the TF-IDF of a word in an article, the more important the word is in the article. The K nearest neighbor algorithm is one of the machine learning algorithms and a classification model. The classification criterion is that if most of the k nearest samples near a sample belong to a certain category, then the sample also belongs to this category.

[0062] S1022, first determine whether the text data is a control text by combining rule matching + model judgment method.

[0063] Among them, the input text data is first matched with rules through a dictionary including control words. If it matches, it means that the text data is control text. Otherwise, the text data is input into the first classification model of the above-mentioned trained binary classification to obtain the first classification result. If the first classification result is control text, it means that the current text data is control text, otherwise it is other text, such as chat text or invalid text.

[0064] In this embodiment, rule matching has a higher priority mainly because the voice text is short text with strong domain specificity. Using rule matching first is faster and more accurate, and then adding the first classification model judgment further improves the recall rate.

[0065] S103, when it is determined that the text data is a control text, determining whether the text data is a scene control text at least based on a pre-built scene dictionary and a classification model.

[0066] In this embodiment, for the control text belonging to the control field, scene recognition is performed to determine whether it is scene control or device control.

[0067] First, build a scene dictionary.

[0068] A set of matching keywords is constructed for each scenario. For example, the keywords for "sleep mode" are ["rest", "sleep", "good night"]. Then, the pre-trained word vector + K nearest neighbor algorithm is used to train a multi-classification second classification model. The classification results are different scenarios such as going home, leaving home, sleeping, and getting up.

[0069] Then, it is determined whether it is scene control or device control by combining rule matching + model judgment.

[0070] Or firstly, the input text data is matched with the rules through the dictionary including the scene, and the text data is marked as the scene text, and the scene information matching it is obtained. If the match is unsuccessful, the text data is input into the trained second classification model to obtain the corresponding classification result and the probability value of each classification result. Then, it is judged whether the probability value of the classification result with the largest probability value is greater than the preset threshold value; if it is greater, the classification result with the largest probability value is output to obtain the scene information of the text data; if it is not greater, the output is a null value, and entity recognition is performed to obtain the control instruction corresponding to the text data.

[0071] The performing of entity recognition to obtain a control instruction corresponding to the text data specifically includes:

[0072] First, each character in the text data is labeled according to a pre-trained labeling model.

[0073] Then, the labeled text data is subjected to entity extraction through a deep learning model trained by a network structure combining BERT and CRF to obtain the corresponding entities.

[0074] In this embodiment, the markers include:

[0075] The first marker B (abbreviation of begin) indicating the start position of the entity;

[0076] The second marker I (abbreviation for Inside) indicates that the entity is located inside.

[0077] The third marker O (abbreviation for Outside) indicates that it is located outside the entity.

[0078] In this embodiment, the type of entity may include category, skill, order, value, brand, etc., which may be set according to actual needs.

[0079] For example, if the text data is "Help me turn on Huawei's fan and set the fan speed to medium", its tags and entity recognition results are shown in Table 1:

[0080] Table 1

[0081] text Label text Label help 0 set up B-order I 0 Place I-order beat B-order wind B-category open I-order fan I-category magnificent B-brand wind B-skill for I-brand speed I-skill of O for O wind B-category middle B-value fan I-category files I-value

[0082] Finally, the entities are normalized and the corresponding control commands are generated.

[0083] Then the entities are normalized, for example, "on" -> "on", as shown in Table 2:

[0084] Table 2

[0085]

[0086]

[0087] Generate the corresponding control commands as follows:

[0088] {“category”:“Electric Fan”,“brand”:“Huawei”,“command”:{“skill”:“switch”,“order”:“set”,“value”:“1”}}

[0089] S104, when it is determined that the text data is a scene control text, scene information corresponding to the scene control text is obtained, and a working state of at least one corresponding smart home is controlled according to the scene information.

[0090] In this embodiment, for example, if the scene information is sleep, the control device obtains the working status of each home corresponding to the scene information, and controls each smart home according to the working status.

[0091] For example, assuming that the scene information is sleeping, the corresponding smart home working state is: {"category":"air conditioning","brand":"XX","command":{"skill":"switch","order":"set","value":"27"}}

[0092] {"category":"Lamp","brand":"XX","command":{"skill":"switch","order":"OFF"}}

[0093] {“category”:“TV”,“brand”:“XX”,“command”:{“skill”:“switch”,“order”:“OFF”}}.

[0094] The control device will then perform the following actions simultaneously: turn on the bedroom air conditioner and adjust it to 27 degrees, and turn off the lights and the TV at the same time.

[0095] To sum up, this embodiment sets a mapping relationship between different scenes and each smart home, so that multiple associated smart homes can be controlled simultaneously by issuing a voice control command including the scene, avoiding the cumbersome operation problems caused by controlling multiple smart homes separately, and improving the user experience.

[0096] In addition, for the home appliance field, this embodiment creatively proposes a complete solution by combining domain dictionaries, statistical learning methods, and deep learning algorithms. It can recognize scenes, significantly improve the recognition rate of voice control commands, adapt to input voice texts in different situations, and enhance user experience.

[0097] See also Figure 2 The second embodiment of the present invention further provides a scene-based smart home control device, which includes:

[0098] The voice processing unit 210 is used to obtain the user's voice data and perform recognition processing on the voice data to obtain corresponding text data;

[0099] The recognition unit 220 is used to recognize the text data to determine whether the text data is a control text;

[0100] A scene determination unit 230, configured to determine whether the text data is a scene control text at least based on a pre-built scene dictionary and a classification model when determining that the text data is a control text;

[0101] The scene control unit 240 is used to obtain scene information corresponding to the scene control text when it is determined that the text data is a scene control text, and control the working state of at least one corresponding smart home according to the scene information.

[0102] Preferably, the speech processing unit 210 is specifically used for:

[0103] Receive voice data sent by users through the channel end;

[0104] The voice data is subjected to text recognition, and the recognized text is subjected to text error correction and device normalization to obtain text data.

[0105] Preferably, the identification unit 220 is specifically used for:

[0106] Performing rule matching on the text data through a pre-built control dictionary;

[0107] If the match is successful, the text data is determined to be a control text;

[0108] If the match is unsuccessful, the text data is input into a pre-trained first classification model to obtain a classification result; wherein the first classification model is a binary classification model trained using existing corpus based on TF-IDF and K nearest neighbor algorithm;

[0109] It is determined whether the text data is a control text according to the classification result.

[0110] Preferably, the scene determination unit 230 is specifically used for:

[0111] When it is determined that the text data is a control text, rule matching is performed on the text data based on a pre-built scenario dictionary;

[0112] If the match is successful, the text data is marked as scene text, and scene information matching the text data is obtained;

[0113] If the match is unsuccessful, the text data is input into the trained second classification model to obtain the corresponding classification results and the probability value of each classification result; wherein the second classification model is a multi-classification model trained based on the pre-trained scene word vector and the K nearest neighbor algorithm;

[0114] Determine whether the probability value of the classification result with the largest probability value is greater than a preset threshold;

[0115] If it is greater than, output the classification result with the largest probability value to obtain the scene information of the text data;

[0116] If not, the output is a null value, and entity recognition is performed to obtain a control instruction corresponding to the text data.

[0117] The performing of entity recognition to obtain a control instruction corresponding to the text data includes:

[0118] Marking each character in the text data according to a pre-trained marking model;

[0119] For the labeled text data, the deep learning model trained by the network structure combining BERT and CRF is used to extract entities to obtain the corresponding entities;

[0120] Normalize the entities and generate corresponding control commands.

[0121] Preferably, when marking each character, the marker includes:

[0122] The first marker indicating the start of the entity;

[0123] Indicates the second marker located inside the entity;

[0124] Represents a third tag outside of an entity.

[0125] Preferably, the type of the entity includes category, function, command, value and brand.

[0126] The third embodiment of the present invention further provides a scene-based smart home control device, which includes a processor and a memory, wherein the memory stores a computer program, and the computer program can be executed by the processor to implement the scene-based smart home control method as described above.

[0127] The fourth embodiment of the present invention further provides a computer-readable storage medium storing a computer program, wherein the computer program can be executed by a processor of a device where the computer-readable storage medium is located to implement the scene-based smart home control method as described above.

[0128] In several embodiments provided in the embodiments of the present invention, it should be understood that the disclosed method can also be implemented in other ways. The above-described device and method embodiments are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0129] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0130] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention is essentially or partly contributed to the prior art or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk. It should be noted that, in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0131] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A scene-based smart home control method, characterized in that: include: Acquire the user's voice data, and perform recognition processing on the voice data to obtain corresponding text data; Identifying the text data to determine whether the text data is a control text; When it is determined that the text data is a control text, whether the text data is a scene control text is determined at least based on a pre-constructed scene dictionary and a classification model; wherein, specifically, it includes: when it is determined that the text data is a control text, the text data is matched with rules based on a pre-constructed scene dictionary; if the match is successful, the text data is marked as a scene text, and the scene information matching it is obtained; if the match is unsuccessful, the text data is input into a trained second classification model to obtain its corresponding classification result and the probability value of each classification result; wherein, the second classification model is a multi-classification model trained based on pre-trained scene word vectors and K nearest neighbor algorithm; it is determined whether the probability value of the classification result with the largest probability value is greater than a preset threshold; if it is greater, the classification result with the largest probability value is output to obtain the scene information of the text data; if it is not greater, the output is a null value, and entity recognition is performed to obtain the control instruction corresponding to the text data; When it is determined that the text data is a scene control text, scene information corresponding to the scene control text is obtained, and the working state of at least one corresponding smart home is controlled according to the scene information.

2. The scene-based smart home control method according to claim 1, characterized in that: Acquiring the user's voice data and performing recognition processing on the voice data to obtain corresponding text data specifically includes: Receive voice data sent by users through the channel end; The voice data is subjected to text recognition, and the recognized text is subjected to text error correction and device normalization to obtain text data.

3. The scene-based smart home control method according to claim 1, characterized in that: Identifying the text data to determine whether the text data is a control text specifically includes: Performing rule matching on the text data through a pre-built control dictionary; If the match is successful, the text data is determined to be a control text; If the match is unsuccessful, the text data is input into a pre-trained first classification model to obtain a classification result; wherein the first classification model is a binary classification model trained using existing corpus based on TF-IDF and K nearest neighbor algorithm; It is determined whether the text data is a control text according to the classification result.

4. The scene-based smart home control method according to claim 1, characterized in that: Performing entity recognition to obtain a control instruction corresponding to the text data includes: Marking each character in the text data according to a pre-trained marking model; For the labeled text data, the deep learning model trained by the network structure combining BERT and CRF is used to extract entities to obtain the corresponding entities; Normalize the entities and generate corresponding control commands.

5. The scene-based smart home control method according to claim 4, characterized in that: When marking each character, the marker includes: The first marker indicating the start of the entity; Indicates the second marker located inside the entity; Represents a third tag outside of an entity.

6. The scene-based smart home control method according to claim 4, characterized in that: The types of the entities include category, function, command, value, and brand.

7. A scene-based smart home control device, characterized in that: include: A voice processing unit, used to obtain the user's voice data and perform recognition processing on the voice data to obtain corresponding text data; an identification unit, used for identifying the text data to determine whether the text data is a control text; A scene judgment unit, used for judging whether the text data is a scene control text at least according to a pre-built scene dictionary and a classification model when judging that the text data is a control text; wherein the scene judgment unit is specifically used for: when judging that the text data is a control text, performing rule matching on the text data based on a pre-built scene dictionary; if the match is successful, marking the text data as a scene text, and obtaining the scene information matching it; if the match is unsuccessful, inputting the text data into a trained second classification model, obtaining the corresponding classification result and the probability value of each classification result; wherein the second classification model is a multi-classification model trained based on pre-trained scene word vectors and a K nearest neighbor algorithm; judging whether the probability value of the classification result with the largest probability value is greater than a preset threshold; if greater than, outputting the classification result with the largest probability value to obtain the scene information of the text data; if not greater than, outputting a null value, and performing entity recognition to obtain the control instruction corresponding to the text data; A scene control unit is used to obtain scene information corresponding to the scene control text when it is determined that the text data is a scene control text, and control the working state of at least one corresponding smart home according to the scene information.

8. A scene-based smart home control device, characterized in that: It includes a processor and a memory, wherein the memory stores a computer program, and the computer program can be executed by the processor to implement the scene-based smart home control method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: A computer program is stored, and the computer program can be executed by a processor of the device where the readable storage medium is located to implement the scene-based smart home control method as described in any one of claims 1 to 6.

Images