Control instruction generation method and device, storage medium and electronic device

Abstract

The invention discloses a control instruction generation method and device, a storage medium and an electronic device, and relates to the technical field of smart home / smart home, and the control instruction generation method comprises the steps: receiving a voice instruction of a user, and extracting entity relation information according to the voice instruction and a preset extraction model, the preset extraction model comprises a natural language processing model fusing user preferences and daily knowledge, and generating a control instruction based on the extracted entity relationship information. According to the method, the user preference and the daily knowledge are fused into the preset extraction model, the entity relationship information is extracted from the voice instruction of the user by using the preset extraction model, and the control instruction is generated, so that the purpose of improving the user intention understanding accuracy is achieved, the effect of improving the instruction generation accuracy is achieved, and the user experience is improved. Therefore, a problem that the control instruction generated by the smart home control system when facing the non-standard voice command is not accurate enough is solved.

Description

Technical Field

[0001] The present application relates to the field of smart home / intelligent family technology, and more specifically, to a control instruction generation method, device, storage medium and electronic device. Background Art

[0002] At present, the voice recognition technology of most smart home control systems is mainly based on a series of pre-set command modes, but the instructions issued by users to the control system may not conform to conventional and standard settings. For example, if the user issues the instruction "pause this operation and use my usual operation", the control system is difficult to accurately understand the user's intention, resulting in inaccurate instructions.

[0003] Accordingly, the art needs a new instruction generation solution to solve the above problems. Summary of the invention

[0004] In order to overcome the above-mentioned defects, the present application is proposed to solve or at least partially solve the problem that the control instructions generated by the smart home control system when facing non-standard voice commands are not accurate enough.

[0005] In a first aspect, a control instruction generation method is provided, the method comprising: receiving a user's voice instruction; extracting entity relationship information according to the voice instruction and a preset extraction model, the preset extraction model comprising a natural language processing model that integrates user preferences and daily knowledge; and generating a control instruction based on the entity relationship information.

[0006] In a technical solution of the above-mentioned control instruction generation method, the entity relationship information is extracted according to the voice instruction and the preset extraction model, including: converting the voice instruction into a text instruction; extracting multiple groups of entities and relationships from the text instruction using the preset extraction model; calculating the confidence of each group of entities and relationships; and selecting the entities and relationships with the highest confidence as the entity relationship information.

[0007] In a technical solution of the above-mentioned control instruction generation method, the use of the preset extraction model to extract multiple groups of entities and relationships from the text instructions includes: using the preset extraction model to identify and classify entities in the text instructions; and constructing multiple groups of entities and relationships based on the identified and classified entities, user preferences, and daily knowledge.

[0008] In a technical solution of the above-mentioned control instruction generation method, the calculation of the confidence of each group of entities and relationships includes: determining the user intention based on the text instruction; calculating the confidence of each group of entities and relationships according to the matching degree between each group of entities and relationships and preset information, and the preset information includes pre-acquired device information, the user intention, the user preferences and daily knowledge.

[0009] In a technical solution of the above control instruction generation method, the generating of the control instruction based on the entity relationship information includes: filling slots based on the entity relationship information; and generating the control instruction according to the slot filling result.

[0010] In a technical solution of the above control instruction generation method, the method further includes: sending the control instruction to the target device and generating reply information.

[0011] In a technical solution of the above-mentioned control instruction generation method, the method further includes: obtaining user feedback information on the control instruction; and optimizing and updating the preset extraction model based on the entity relationship information, the control instruction and the feedback information.

[0012] In a second aspect, a control instruction generating device is provided, the device comprising: a voice receiving module for receiving a user's voice instruction; an information extraction module for extracting entity relationship information according to the voice instruction and a preset extraction model, the preset extraction model comprising a natural language processing model that integrates user preferences and daily knowledge; an instruction generating module for generating control instructions based on the entity relationship information.

[0013] In a technical solution of the above-mentioned control instruction generation device, the information extraction module includes: an instruction conversion unit, used to convert the voice instruction into a text instruction; an entity relationship extraction unit, used to extract multiple groups of entities and relationships from the text instruction using the preset extraction model; a confidence calculation unit, used to calculate the confidence of each group of entities and relationships; and an entity relationship selection unit, used to select entities and relationships with the highest confidence as entity relationship information.

[0014] In a technical solution of the above-mentioned control instruction generating device, the entity relationship extraction unit includes: an identification and classification subunit, which is used to identify and classify entities in the text instructions using the preset extraction model; and an entity relationship construction subunit, which is used to construct multiple groups of entities and relationships based on the identified and classified entities, user preferences and daily knowledge.

[0015] In a technical solution of the above-mentioned control instruction generating device, the confidence calculation unit includes: an intention determination subunit, used to determine the user intention based on the text instruction; a confidence calculation subunit, used to calculate the confidence of each group of entities and relationships based on the matching degree between each group of entities and relationships and preset information, and the preset information includes pre-acquired device information, the user intention, the user preferences and daily knowledge.

[0016] In a technical solution of the above-mentioned control instruction generating device, the instruction generating module includes: a slot filling unit, used to fill slots based on the entity relationship information; and a control instruction generating unit, used to generate control instructions according to the slot filling results.

[0017] In a technical solution of the above control instruction generating device, the device further includes: an instruction sending and reply module, which is used to send the control instruction to the target device and generate reply information.

[0018] In a technical solution of the above-mentioned control instruction generating device, the device also includes: a feedback acquisition module, used to obtain user feedback information on the control instruction; an optimization and update module, used to optimize and update the preset extraction model based on the entity relationship information, the control instruction and the feedback information.

[0019] In a third aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is run, the control instruction generation method in the above-mentioned first aspect or any corresponding technical solution thereof is executed.

[0020] In a fourth aspect, an electronic device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the control instruction generation method in the above-mentioned first aspect or any corresponding technical solution thereof through the computer program.

[0021] The above one or more technical solutions of the present application have at least one or more of the following beneficial effects:

[0022] In the technical solution for implementing the present application, user preferences and daily knowledge are integrated with a preset extraction model, and the preset extraction model is used to extract entity relationship information from the user's voice commands and generate control commands, thereby achieving the purpose of improving the accuracy of understanding user intentions and the effect of improving the accuracy of command generation, thereby solving the problem of inaccurate control commands generated by smart home control systems when faced with non-standard voice commands.

[0023] In the technical solution for implementing the present application, by calculating the confidence of multiple groups of entities and relationships extracted by a preset extraction model, the entities and relationships with the highest confidence are selected as the extracted entity relationship information, thereby achieving the purpose of improving the reliability and accuracy of the entity relationship information and further improving the accuracy of control instruction generation.

[0024] In the technical solution for implementing the present application, the confidence of each group of entities and relationships is calculated based on pre-acquired device information, user intentions, user preferences, and daily knowledge, thereby improving the reliability of confidence calculation and thus improving the accuracy of entity relationship information extraction.

[0025] In implementing the technical solution of the present application, the preset extraction model is optimized and updated using entity relationship information, control instructions, and feedback information, thereby achieving the purpose of improving the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0027] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0028] Figure 1 is a schematic diagram of a hardware environment of an interaction method of a smart device according to an embodiment of the present application;

[0029] Figure 2 It is a schematic flow chart of the main steps of a control instruction generation method according to an embodiment of the present application;

[0030] Figure 3 is a schematic diagram of the overall framework of a control instruction generation method according to an embodiment of the present application;

[0031] Figure 4 It is a schematic diagram of the overall steps of a control instruction generation method according to an embodiment of the present application;

[0032] Figure 5 is a schematic diagram of the main structural block diagram of an instruction generation device according to an embodiment of the present application;

[0033] Figure 6 It is a schematic diagram of the connection relationship between a processor and a memory of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. The terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected, it can be indirectly connected through an intermediate medium, it can also be the internal communication of two elements, it can be a wireless connection, or it can be a wired connection.

[0036] In addition, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, and memories, and may also include software parts, such as program codes, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware, or a combination of the two. Computer-readable storage media include any suitable media that can store program codes, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc.

[0037] In addition, if the meaning of "and / or" appears in this application, it includes three parallel solutions. Taking "A and / or B" as an example, it includes solution A, solution B, or a solution that satisfies both A and B. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in this field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by this application. The term "at least one A or B" or "at least one of A and B" has a similar meaning to "A and / or B" and can include only A, only B, or A and B. The singular terms "one" and "this" can also include plural forms.

[0038] The relevant user personal information that may be involved in the various embodiments of this application is strictly in accordance with the requirements of laws and regulations, following the principles of legality, legitimacy and necessity, based on the reasonable purposes of business scenarios, to process the personal information that users actively provide during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the user's authorization.

[0039] This application attaches great importance to the security of user personal information and has taken reasonable and feasible security protection measures that meet industry standards to protect user information and prevent personal information from being accessed, disclosed, used, modified, damaged or lost without authorization.

[0040] According to one aspect of an embodiment of the present application, a method for interacting with a smart home device is provided. The method for interacting with a smart home device is widely used in smart home, smart home, smart home device ecosystem, smart home ecosystem and other whole-house intelligent digital control application scenarios. Optionally, in this embodiment, the method for interacting with the smart home device can be applied to Figure 1 In the hardware environment composed of the terminal device 102 and the server 104 shown in FIG. Figure 1 As shown, the server 104 is connected to the terminal device 102 via a network, and can be used to provide services (such as application services, etc.) for the terminal or a client installed on the terminal. A database can be set on the server or independently of the server to provide data storage services for the server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data computing services for the server 104.

[0041] The network may include but is not limited to at least one of the following: wired network, wireless network. The wired network may include but is not limited to at least one of the following: wide area network, metropolitan area network, local area network, and the wireless network may include but is not limited to at least one of the following: WIFI (Wireless Fidelity), Bluetooth. The terminal device 102 may be but is not limited to a PC, a mobile phone, a tablet computer, a smart air conditioner, a smart range hood, a smart refrigerator, a smart oven, a smart stove, a smart washing machine, a smart water heater, a smart washing device, a smart dishwasher, a smart projection device, a smart TV, a smart clothes drying rack, a smart curtain, a smart audio and video, a smart socket, a smart speaker, a smart fresh air device, a smart kitchen and bathroom device, a smart bathroom device, a smart sweeping robot, a smart window cleaning robot, a smart mopping robot, a smart air purification device, a smart steamer, a smart microwave oven, a smart kitchen treasure, a smart purifier, a smart water dispenser, a smart door lock, etc.

[0042] Smart home control systems often have the following problems when processing atypical sentences or commands with complex intentions: due to the complexity and fluidity of the language itself, it is difficult to accurately capture all the details, the accuracy of speech recognition is restricted by the extensiveness of the vocabulary, it is unable to adapt to and learn the usage preferences of different users, lacks the use of user historical interaction data, lacks the effective integration of context clues and common sense, and is difficult to accurately understand the user's true intentions. Accordingly, this field needs a new command generation solution to solve the above problems.

[0043] According to one aspect of the embodiments of the present application, a control instruction generation method is provided. Figure 2 , Figure 2 FIG. 1 is a flow chart of the main steps of a control instruction generation method according to an embodiment of the present application. Figure 2 As shown, the control instruction generation method mainly includes the following steps S201 to S203:

[0044] Step S201, receiving a user's voice command.

[0045] In this embodiment, the user's voice command refers to a command or instruction issued by the user in the form of natural language, such as the user issuing an instruction to the system by speaking, requiring it to control the smart home device to perform a certain operation.

[0046] Step S202, extracting entity relationship information according to the voice instruction and a preset extraction model, wherein the preset extraction model includes a natural language processing model that integrates user preferences and daily knowledge.

[0047] In this embodiment, the preset extraction model is a pre-trained entity relationship extraction model, which can be used to extract entity relationship information from voice commands. For example, in a smart home control scenario, entities include (smart home) devices or other objects, and relationships include relationships between entities, such as actions such as controlling and managing devices.

[0048] In one embodiment, the user preferences and daily knowledge are integrated with the preset extraction model, for example, by collecting the user's historical data and daily knowledge, and using the historical data and daily knowledge to train the natural language processing model to perform named entity recognition and relationship extraction tasks. In this way, the model can learn the user's interests and habits (preferences), thereby improving the accuracy of entity recognition and relationship extraction.

[0049] In one implementation, user usage behaviors, such as click-through rate, dwell time, etc., can be collected and analyzed to obtain user usage preferences and interests, and the above user preference information can be integrated into a preset extraction model to improve the accuracy of entity recognition and relationship extraction.

[0050] In one embodiment, an external knowledge base (such as Wikipedia or a pre-built knowledge graph) can also be used to provide more comprehensive and accurate entity and relationship information for the preset extraction model, thereby further improving the performance of the preset extraction model.

[0051] Chinese semantic expressions are highly complex and diverse. This embodiment introduces user usage habits and daily knowledge (common sense library) to assist in understanding and inferring user intentions, thereby achieving the purpose of improving the accuracy of entity relationship information extraction and further improving the accuracy of control instruction generation.

[0052] Step S203: Generate a control instruction based on the entity relationship information.

[0053] In this embodiment, the control instructions of the device are generated according to the entity relationship information extracted in step S202.

[0054] In one implementation, the identified entity relationship information may be classified (filled) into predefined slots, which typically represent certain specific information categories, such as equipment, action, location, time, etc., thereby helping the system understand the user's intent and generate corresponding control instructions.

[0055] Based on the above steps S201 to S203, user preferences and daily knowledge are integrated into the preset extraction model, and the preset extraction model is used to extract entity relationship information from the user's voice instructions and generate control instructions, thereby achieving the purpose of improving the accuracy of understanding user intentions and the effect of improving the accuracy of instruction generation, thereby solving the problem of inaccurate control instructions generated by the smart home control system when facing non-standard voice commands.

[0056] In one implementation of the embodiment of the present application, the above step S202 may further include the following steps S2021 to S2024:

[0057] Step S2021, converting the voice command into a text command.

[0058] In this embodiment, the voice instructions need to be converted into text instructions first, and then the text instructions are analyzed using natural language processing technology.

[0059] In one implementation, a speech recognition algorithm and model may be used to convert a user's voice command into a corresponding text command, wherein the speech recognition algorithm and model may refer to methods in the prior art and will not be described in detail herein.

[0060] Step S2022, extracting multiple groups of entities and relationships from the text instructions using a preset extraction model.

[0061] In this embodiment, the preset extraction model can extract entities and the relationship between entities from the text. Specifically, the model needs to identify entities from the text. The entities can be nouns, noun phrases or other language units with specific meanings; then determine the relationship between these entities, such as analyzing the vocabulary, phrases or syntactic structures in the text to determine the connection between entities, or using causal reasoning and context understanding to analyze the association, positional relationship, semantic attributes and other information between entities. In addition, a data structure in the form of triples can be constructed based on the identified entities and relationships, and then a knowledge graph or other semantic network can be constructed.

[0062] As an example, suppose the user's instruction is "Turn on the air conditioner to energy-saving mode, and then play "Interstellar". The model initially extracts entities and relationships from the instruction, including: "Air conditioner - Adjust to - Energy-saving mode" and "Play - Interstellar". Since the preset extraction model integrates user preferences and daily knowledge, further analysis shows that the temperature of the air conditioner energy-saving mode is usually set at 25-26 degrees; the device that plays Interstellar can be a TV or a projector. Therefore, the following groups of entities and relationships can be obtained: "Air conditioner - Adjust to -25 degrees" / "Air conditioner - Adjust to -26 degrees"; "TV - Play - Interstellar" / "Projector - Play - Interstellar".

[0063] Step S2023, calculating the confidence of each group of entities and relationships.

[0064] In this embodiment, there are multiple possible choices for each set of entities and relationships. At this time, the confidence corresponding to each possible entity and relationship can be calculated in combination with device information, user intent, user preference, etc.

[0065] In one implementation, the scores of each entity and relationship in terms of device information, user intent, and user preference can be calculated separately, and the weighted sum of the scores in multiple aspects can be used as the confidence. It should be noted that the device information includes information such as the category type and quantity of the devices in the user's home. It can be understood that if there are devices in the entities and relationships that are not included in the user's home, the confidence in the device information is 0; user intent can be analyzed from user instructions, specifically, the degree of match between entities and relationships and user intent is analyzed. The higher the degree of match, the more credible the entities and relationships are; user preferences can be analyzed from the user's historical usage data, such as which devices the user often uses and which operations are often performed on the devices. The more frequently used devices and operations in entities and relationships are, the higher the confidence.

[0066] For example, for the two optional entities and relationships "air conditioner - adjusted to -25 degrees" / "air conditioner - adjusted to -26 degrees", there is little difference in device information and user intent, but according to user preference, users more often adjust the air conditioner to 26 degrees, that is, "air conditioner - adjusted to -26 degrees" has the highest confidence.

[0067] Step S2024, selecting entities and relationships with the highest confidence as entity relationship information.

[0068] In this embodiment, for each set of entities and relationships, if there are multiple selectable entities and relationships, the confidences corresponding to the multiple entities and relationships are calculated respectively, and the entities and relationships with the highest confidences are used as the extracted entity relationship information. According to the above embodiment, for the "air conditioner - adjust to - energy saving mode" instruction issued by the user, "air conditioner - adjust to -26 degrees" has the highest confidence among all possible entities and relationships, so it is used as the final entity relationship information.

[0069] In one implementation of the embodiment of the present application, the above step S2022 may further include the following steps S20221 and S20222:

[0070] Step S20221, using a preset extraction model to identify and classify entities in the text instructions.

[0071] In this embodiment, by classifying the entities recognized by the model, it is helpful to generate control instructions that are more suitable for application scenarios and enhance the adaptability of the scenarios.

[0072] As an example, suppose the user's voice command is "turn on the oven and light". Although it does not specify where the "light" is in the house, by classifying the "oven", it can be known that the "oven" belongs to the kitchen, so it can be inferred that the "light" is most likely the kitchen light. The above method achieves the purpose of generating more accurate and targeted commands and enhancing scene adaptability.

[0073] Step S20222, construct multiple groups of entities and relationships based on the identified and classified entities, user preferences, and daily knowledge.

[0074] In this embodiment, since the model integrates user preferences and daily knowledge, when constructing entities and relationships, causal reasoning and context understanding technology can be used in combination with user preferences and daily knowledge to infer multiple groups of entities and relationships.

[0075] In one implementation of the embodiment of the present application, the above step S2023 may further include the following steps S20231 and S20232:

[0076] Step S20231, determine user intent based on text instructions.

[0077] In this embodiment, rule template analysis, deep learning models and other methods can be used to identify the user's intention from text instructions. For details, please refer to the intention recognition method in the prior art, which will not be repeated here.

[0078] Step S20232, calculating the confidence of each group of entities and relationships based on the matching degree of each group of entities and relationships with preset information, where the preset information includes pre-acquired device information, user intentions, user preferences, and daily knowledge.

[0079] In this embodiment, the confidence of entities and relationships can be calculated from multiple aspects such as device information, user intention, user preference, and daily knowledge. It should be noted that the preset information can also include other information such as time information. Since certain devices / instructions are more frequently used at a certain time, the time information can also reflect the confidence of entities and relationships to a certain extent. It is understandable that the preset information can be set according to the specific situation, and this embodiment does not limit this.

[0080] In one implementation of the embodiment of the present application, the above step S203 may further include the following steps S2031 and S2032:

[0081] Step S2031, filling slots based on entity relationship information.

[0082] In this embodiment, the slots may include equipment, actions, locations, etc. For example, if the user instruction is "turn on the bedroom chandelier", the filling result of the above slots is as follows:

[0083] Equipment: Chandelier;

[0084] Action: Open;

[0085] Location: Bedroom;

[0086] Step S2032, generating a control instruction according to the slot filling result.

[0087] In this embodiment, according to the slot filling result in step S2031, control instructions for corresponding devices may be generated according to a preset sequence or rule.

[0088] In one implementation of the embodiment of the present application, after step S203, the following step S204 may be further included:

[0089] Step S204: send the control instruction to the target device and generate reply information.

[0090] In this embodiment, the reply information is the result feedback of whether the command is executed successfully or not, and a pre-set corpus can be used. For example, if the control command issued is to turn on the air conditioner, the reply message can be "The air conditioner has been successfully turned on for you" if it is turned on successfully; if it fails to turn on, the reply message can be "The air conditioner failed to start successfully, please check whether the power is on, or try to restart it."

[0091] In one implementation of the embodiment of the present application, step S205 may be further included after step S203, specifically including the following steps S2051 and S2052:

[0092] Step S2051, obtaining user feedback information on the control instruction.

[0093] In this embodiment, user feedback refers to the user's positive / negative feedback on the generated control instruction. As an example, if after executing the generated control instruction, the user re-issues a new voice instruction or issues an instruction to stop the current operation, it means that the generated control instruction does not meet the user's expectations (negative feedback); if after executing the generated control instruction, the user does not issue other instructions or stop the current operation, it means that the generated control instruction is recognized by the user (positive feedback).

[0094] Step S2052, optimizing and updating the preset extraction model based on the entity relationship information, control instructions and feedback information.

[0095] In this embodiment, the preset extraction model is trained based on the user's historical interaction data such as historical voice commands, generated control commands, user feedback and other data. As an example, the self-supervised learning and reinforcement learning training models are combined. On the one hand, the user's historical interaction data is feature extracted and the preset extraction model is trained using self-supervised learning. On the other hand, user feedback is used as a reward signal to further train the preset extraction model through reinforcement learning. Based on the above steps, the preset extraction model is continuously optimized and updated, achieving the purpose of improving the generalization ability of the model and achieving the effect of dynamic optimization.

[0096] In this embodiment, by combining self-supervised learning with reinforcement learning, an efficient learning mechanism is formed, which enables the system to continuously optimize and update using historical user interaction data without manual labeling. This enhances the dynamic optimization and adaptive capabilities of the smart home control system, while achieving the purpose of adapting to the personalized needs and daily usage habits of users.

[0097] In an example of an application scenario of the present application, Figure 3 FIG. 1 is a schematic diagram of the overall framework of a control instruction generation method according to an embodiment of the present application. Figure 3As shown, first obtain the voice interaction (voice command) between the user and the smart home device, and then perform voice-to-text conversion, such as using the ASR (Automatic Speech Recognition, automatic speech recognition) tool to convert the voice command into a text command. Then perform natural language processing on the text command, including entity relationship extraction and slot filling, and the above natural language processing steps can be implemented using a natural language processing model. It can be understood that in order to ensure that the model can accurately understand the user's intention, the model integrates user preferences and daily knowledge, such as using user historical data and daily knowledge to train the model to achieve the purpose of integrating user preferences and daily knowledge, or using user preferences and daily knowledge as an external knowledge base to provide information for the model. In addition, the user's intention can be extracted from the text instruction, and the user's intention can be used to improve the accuracy of entity relationship extraction and slot filling. Finally, the control instruction is generated according to the slot filling result and sent down (to the target device) for execution. Finally, the (interaction result) is generated based on the execution result of the target device and replied to the user.

[0098] In an example of an application scenario of the present application, Figure 4 FIG. 1 is a schematic diagram of the overall steps of a control instruction generation method according to an embodiment of the present application. Figure 4 As shown, the method mainly includes the following steps S401 to S408:

[0099] Step S401, obtaining a voice command.

[0100] For details, please see Figure 2 Step S201 of the illustrated embodiment will not be described in detail here.

[0101] Step S402: data preprocessing.

[0102] In this embodiment, the voice instructions are converted into text instructions, and the text instructions are preliminarily processed by methods such as word segmentation, part-of-speech tagging, and dependency syntax analysis.

[0103] Step S403: entity extraction.

[0104] In this embodiment, a large model is used to identify keyword entities (action, equipment, location, etc.) from the preprocessed corpus.

[0105] Step S404: relationship building.

[0106] In this embodiment, we combine user preferences and daily knowledge base, analyze entity relationships in complex sentences through causal reasoning and context understanding technology, calculate the confidence of entity relationships, and select high-confidence entity relationship information. Figure 2 Step S202 of the illustrated embodiment will not be described in detail here.

[0107] Step S405, determining whether the entity relationship is in the user preference or daily knowledge base.

[0108] In this embodiment, it is determined whether the extracted entity relationship exists in the user preference or daily knowledge base. If so, step S406 is executed; if not, step S407 is executed.

[0109] Step S406: Generate a high-confidence control instruction.

[0110] In this embodiment, an entity relationship with a high confidence level is selected to generate a control instruction.

[0111] Step S407, dynamically update user preferences and daily knowledge base.

[0112] In this embodiment, the user preferences and the daily knowledge base are dynamically updated using entity relationships that are not in the user preferences and daily knowledge base.

[0113] Step S408, issuing instructions.

[0114] In this embodiment, slot filling is performed based on entity relationship information, that is, the entity relationship information is converted into specific control instructions and sent to the target device (related smart home device) for execution.

[0115] In one implementation, multi-threaded concurrent technology and real-time communication protocol are used to issue instructions to ensure rapid execution and feedback of instructions.

[0116] In this embodiment, the preset extraction model is integrated with the user's daily language interaction habits and rich common sense knowledge, achieving the purpose of accurately and efficiently extracting key entity information in text instructions, and achieving the effect of improving the accuracy of control instruction generation. It solves the problem that the smart home control system has insufficient generalization ability when facing complex Chinese contexts, and the difficulty in information extraction leads to inaccurate instruction generation. It is conducive to providing users with more intelligent and personalized help and support, improving the convenience and comfort of users when using, so as to better serve users.

[0117] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effect of the present application, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders. These adjusted schemes are equivalent to the technical schemes described in this application, and therefore will also fall within the scope of protection of this application.

[0118] Another aspect of the present application also provides a control instruction generating device, such as Figure 5As shown, the device includes: a voice receiving module 501, used to receive a user's voice command; an information extraction module 502, used to extract entity relationship information according to the voice command and a preset extraction model, the preset extraction model includes a natural language processing model that integrates user preferences and daily knowledge; an instruction generation module 503, used to generate control instructions based on the entity relationship information.

[0119] In a technical solution of the above-mentioned control instruction generation device, the information extraction module includes: an instruction conversion unit, which is used to convert voice instructions into text instructions; an entity relationship extraction unit, which is used to extract multiple groups of entities and relationships from text instructions using a preset extraction model; a confidence calculation unit, which is used to calculate the confidence of each group of entities and relationships; and an entity relationship selection unit, which is used to select entities and relationships with the highest confidence as entity relationship information.

[0120] In a technical solution of the above-mentioned control instruction generation device, the entity relationship extraction unit includes: an identification and classification subunit, which is used to identify and classify entities in text instructions using a preset extraction model; and an entity relationship construction subunit, which is used to construct multiple groups of entities and relationships based on the identified and classified entities, user preferences, and daily knowledge.

[0121] In a technical solution of the above-mentioned control instruction generating device, the confidence calculation unit includes: an intention determination subunit, which is used to determine the user intention based on the text instruction; a confidence calculation subunit, which is used to calculate the confidence of each group of entities and relationships based on the matching degree between each group of entities and relationships and preset information, and the preset information includes pre-acquired device information, user intention, user preferences and daily knowledge.

[0122] In a technical solution of the above-mentioned control instruction generating device, the instruction generating module includes: a slot filling unit, which is used to fill the slots based on the entity relationship information; and a control instruction generating unit, which is used to generate the control instruction according to the slot filling result.

[0123] In a technical solution of the above control instruction generating device, the device further includes: an instruction sending and reply module, which is used to send the control instruction to the target device and generate reply information.

[0124] In a technical solution of the above-mentioned control instruction generating device, the device also includes: a feedback acquisition module for obtaining user feedback information on the control instruction; an optimization and update module for optimizing and updating the preset extraction model based on entity relationship information, control instructions and feedback information.

[0125] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above-mentioned embodiment of the present application can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.

[0126] Another aspect of the present application also provides a computer-readable storage medium.

[0127] In an embodiment of a computer-readable storage medium according to the present application, the computer-readable storage medium may be configured to store a program for executing a control instruction generation method of the above-mentioned method embodiment, and the program may be loaded and run by a processor to implement the control instruction generation method in the above-mentioned first aspect or any technical solution corresponding thereto. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method section of the embodiment of the present application. The computer-readable storage medium may be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-temporary computer-readable storage medium.

[0128] Another aspect of the present application provides an electronic device.

[0129] In an embodiment of an electronic device according to the present application, the electronic device may include at least one processor; and a memory connected to the at least one processor; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the control instruction generation method in the first aspect or any corresponding technical solution thereof is implemented. The electronic device described in the present application may include a PC, a mobile phone, a tablet computer, a smart speaker, a smart washing machine, a smart projection device, a smart TV, etc. See the attached Figure 6 , Figure 6 FIG. 4 exemplarily shows that the memory 11 and the processor 12 are communicatively connected via a bus.

[0130] In some embodiments of the present application, the electronic device may further include at least one sensor, and the sensor is used to sense information. The sensor is connected to any type of processor mentioned in the present application. The electronic device described in the present application may be, but is not limited to, a PC, a mobile phone, a tablet computer, a smart air conditioner, a smart range hood, a smart refrigerator, a smart oven, a smart stove, a smart washing machine, a smart water heater, a smart washing equipment, a smart dishwasher, a smart projection equipment, a smart TV, a smart clothes drying rack, a smart curtain, a smart audio and video, a smart socket, a smart speaker, a smart speaker, a smart fresh air equipment, a smart kitchen and bathroom equipment, a smart bathroom equipment, a smart sweeping robot, a smart window cleaning robot, a smart mopping robot, a smart air purification equipment, a smart steamer, a smart microwave oven, a smart kitchen treasure, a smart purifier, a smart water dispenser, a smart door lock, etc.

[0131] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A control instruction generation method, characterized in that: The method comprises: Receive user's voice command; Extracting entity relationship information according to the voice instruction and a preset extraction model, wherein the preset extraction model includes a natural language processing model that integrates user preferences and daily knowledge; A control instruction is generated based on the entity relationship information.

2. The control instruction generation method according to claim 1, characterized in that: The extracting entity relationship information according to the voice instruction and a preset extraction model includes: Converting the voice command into a text command; Extracting multiple groups of entities and relationships from the text instructions using the preset extraction model; Calculate the confidence of each group of entities and relations; The entities and relations with the highest confidence are selected as entity relationship information.

3. The control instruction generation method according to claim 2, characterized in that: The extracting multiple groups of entities and relationships from the text instruction using the preset extraction model includes: Using the preset extraction model to identify and classify entities in the text instructions; Construct multiple groups of entities and relationships based on the identified and classified entities, user preferences, and daily knowledge.

4. The control instruction generation method according to claim 2, characterized in that: The calculation of the confidence of each group of entities and relations includes: determining user intent based on the text instruction; The confidence of each group of entities and relationships is calculated according to the matching degree of each group of entities and relationships with preset information, wherein the preset information includes pre-acquired device information, the user intention, the user preference and daily knowledge.

5. The control instruction generation method according to any one of claims 1 to 4, characterized in that: The generating a control instruction based on the entity relationship information includes: Filling slots based on the entity relationship information; Generate control instructions based on the slot filling results.

6. The control instruction generation method according to claim 1, characterized in that: The method further comprises: The control instruction is sent to the target device and a reply message is generated.

7. The control instruction generation method according to claim 1, characterized in that: The method further comprises: Obtaining user feedback information on the control instruction; The preset extraction model is optimized and updated based on the entity relationship information, the control instruction and the feedback information.

8. A control instruction generating device, characterized in that: The device comprises: A voice receiving module, used to receive a user's voice command; An information extraction module, configured to extract entity relationship information according to the voice instruction and a preset extraction model, wherein the preset extraction model includes a natural language processing model that integrates user preferences and daily knowledge; An instruction generation module is used to generate a control instruction based on the entity relationship information.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the control instruction generation method according to any one of claims 1 to 7 is executed when the program is executed.

10. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to execute the control instruction generation method according to any one of claims 1 to 7 through the computer program.

Images