Instruction processing method and device of intelligent equipment, electronic equipment and storage medium

The trigger-execution rules of smart devices are generated through semantic analysis of natural language instructions, which solves the problems of complexity and poor user experience in the creation of intelligent device automation rules in the existing technology, and realizes the automation control and user experience improvement of intelligent devices.

CN119067095APending Publication Date: 2024-12-03BEIJING XIAOMI MOBILE SOFTWARE CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202310628982.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-30
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing smart device automation rules creation methods are complex, with high time costs, poor applicability and poor user experience, making it difficult to meet the needs of smart device automation control in different scenarios.

Method used

By obtaining natural language instructions and performing semantic analysis, trigger-execution rules are generated, and logical trigger conditions and control instructions are automatically generated to realize automated control of smart devices without complex operations.

Benefits of technology

It simplifies the process of creating automation rules of smart devices, improves user experience, can meet the usage needs of most scenarios, and simplifies the automated control of smart devices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067095A_ABST
    Figure CN119067095A_ABST
Patent Text Reader

Abstract

The invention relates to an instruction processing method and device of intelligent equipment, electronic equipment and a storage medium. The instruction processing method of the intelligent equipment comprises the steps that a natural language instruction is acquired; semantic analysis is conducted on the natural language instruction, a semantic analysis result is obtained, and the semantic analysis result at least comprises a trigger condition matched with a function interface of the intelligent device and the first control instruction; based on the semantic analysis result, a trigger-execution rule is generated, and the trigger-execution rule is configured to control the intelligent device to execute the first control instruction when a trigger condition is met. By means of the instruction processing method, the obtained natural language instruction can be disassembled, the trigger condition and the control instruction in the trigger-execution rule can be generated, the trigger-execution rule conforming to logic is automatically generated, other complex operations are not needed, the simple trigger-execution rule can meet the use requirements of most scenes, and the user experience is improved. And the user experience is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent Internet of Things, and in particular to a method and device for processing instructions of an intelligent device, an electronic device, and a storage medium. Background Art

[0002] In recent years, with the development of artificial intelligence, natural language processing (NLP), Internet of Things (IOT), etc., people can control smart devices to execute specific instructions through electronic devices such as smartphones, speakers, and central gateways. In order to realize the automatic control of smart devices in different scenarios, it is necessary to create or customize the automation rules of smart devices according to actual needs.

[0003] However, the existing instruction processing methods require a high level of professionalism to create automation rules, and there are problems such as complex creation process, high time cost, poor applicability and poor user experience. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a method and apparatus for processing instructions of an intelligent device, an electronic device and a storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for processing an instruction of a smart device is provided, and the method for processing an instruction of a smart device includes:

[0006] Get natural language instructions;

[0007] Performing semantic parsing on the natural language instruction to obtain a semantic parsing result, wherein the semantic parsing result at least includes a trigger condition and a first control instruction that match a functional interface of the smart device;

[0008] Based on the semantic parsing result, a trigger-execution rule is generated, and the trigger-execution rule is configured to: when the trigger condition is met, control the smart device to execute the first control instruction.

[0009] In some embodiments of the present disclosure, performing semantic parsing on the natural language instruction to obtain a semantic parsing result includes:

[0010] identifying condition information and / or instruction information in the natural language instruction;

[0011] The semantic parsing result is determined according to the condition information and / or the instruction information.

[0012] In some embodiments of the present disclosure, determining the semantic parsing result according to the condition information and / or the instruction information includes:

[0013] When the natural language instruction includes the condition information and the instruction information, generating the trigger condition according to the condition information;

[0014] The first control instruction is generated according to the instruction information.

[0015] In some embodiments of the present disclosure, the condition information includes event condition information; or,

[0016] The condition information includes event condition information and state condition information.

[0017] In some embodiments of the present disclosure, determining the semantic parsing result according to the condition information and / or the instruction information includes: when the natural language instruction only includes the instruction information,

[0018] Generate a second control instruction according to the instruction information and control the smart device to execute the second control instruction;

[0019] and / or,

[0020] Acquire first supplementary information, and determine the condition information according to the first supplementary information;

[0021] generating the trigger condition according to the condition information;

[0022] The first control instruction is generated according to the instruction information.

[0023] In some embodiments of the present disclosure, the obtaining of the first supplementary information includes:

[0024] Based on the current scenario, determining the first supplementary information; and / or,

[0025] A first supplementary instruction is obtained, and the first supplementary information is determined based on the first supplementary instruction.

[0026] In some embodiments of the present disclosure, the current scenario includes one or more of time, location of the smart device, environment of the smart device, and usage habits of the smart device.

[0027] In some embodiments of the present disclosure, before generating the trigger-execution rule, the instruction processing method of the smart device further includes:

[0028] issuing a prompt for whether to supplement the conditional information, and upon receiving a second supplementary instruction, determining the second supplementary information according to the second supplementary instruction;

[0029] The second supplementary information is supplemented to the condition information.

[0030] In some embodiments of the present disclosure, when the instruction information includes multiple pieces of information, determining the semantic parsing result according to the condition information and / or the instruction information includes:

[0031] generating the trigger condition according to the condition information;

[0032] Generate a plurality of the first control instructions according to the plurality of instruction information;

[0033] The generating a trigger-execution rule based on the semantic parsing result includes:

[0034] Based on the trigger condition and the first control instructions, one or more trigger-execution rules are generated, and the trigger-execution rules correspond to the first control instructions one by one respectively;

[0035] When generating one of the trigger-execution rules, the trigger-execution rule is configured to: execute a plurality of the first control instructions when the trigger condition is met;

[0036] When a plurality of the trigger-execution rules are generated, each of the trigger-execution rules is configured to execute the corresponding first control instruction when the trigger condition is met.

[0037] In some embodiments of the present disclosure, obtaining a natural language instruction includes:

[0038] The natural language instructions are voice recognized and / or the natural language instructions are received as text input.

[0039] According to a second aspect of an embodiment of the present disclosure, a command processing device of an intelligent device is provided, and the command processing device of the intelligent device includes:

[0040] An acquisition module, the acquisition module is used to acquire natural language instructions;

[0041] A parsing module, the parsing module is used to perform semantic parsing on the natural language instruction to obtain a semantic parsing result, wherein the semantic parsing result at least includes a trigger condition and a first control instruction that match a functional interface of the smart device;

[0042] A generation module, the generation module is used to generate a trigger-execution rule based on the semantic parsing result, and the trigger-execution rule is configured to: when the trigger condition is met, control the smart device to execute the first control instruction.

[0043] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, the electronic device comprising:

[0044] processor;

[0045] a memory for storing processor-executable instructions;

[0046] Wherein, the processor is configured to:

[0047] Get natural language instructions;

[0048] Performing semantic parsing on the natural language instruction to obtain a semantic parsing result, wherein the semantic parsing result at least includes a trigger condition and a first control instruction that match a functional interface of the smart device;

[0049] Based on the semantic parsing result, a trigger-execution rule is generated, and the trigger-execution rule is configured to: when the trigger condition is met, control the smart device to execute the first control instruction.

[0050] According to a fourth aspect of an embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute an instruction processing method of an intelligent device. The instruction processing method of the intelligent device includes:

[0051] Get natural language instructions;

[0052] Performing semantic parsing on the natural language instruction to obtain a semantic parsing result, wherein the semantic parsing result at least includes a trigger condition and a first control instruction that match a functional interface of the smart device;

[0053] Based on the semantic parsing result, a trigger-execution rule is generated, and the trigger-execution rule is configured to: when the trigger condition is met, control the smart device to execute the first control instruction.

[0054] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: by acquiring natural language instructions and performing semantic analysis on the natural language instructions, a trigger-execution rule is generated based on the semantic analysis result to control the smart device to execute the first control instruction whenever the trigger condition is met. The instruction processing method of this smart device can disassemble the acquired natural language instructions and generate the trigger conditions and control instructions in the trigger-execution rule, automatically generate a logical trigger-execution rule, without the need for other complex operations, and the simple trigger-execution rule can meet the use requirements of most scenarios, thereby improving the user experience.

[0055] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0057] Figure 1 The figure is a flowchart of a method for processing instructions of a smart device according to an exemplary embodiment.

[0058] Figure 2 The present invention is a flowchart of performing semantic parsing on a natural language instruction and obtaining a semantic parsing result according to an exemplary embodiment.

[0059] Figure 3 The present invention is a flowchart of determining a semantic parsing result according to condition information and / or instruction information according to an exemplary embodiment.

[0060] Figure 4 The present invention is a flowchart of determining a semantic parsing result according to condition information and / or instruction information according to another exemplary embodiment.

[0061] Figure 5 The invention is a flowchart of a method for processing instructions of a smart device before generating a trigger-execution rule according to an exemplary embodiment.

[0062] Figure 6 It is a flowchart showing determining a semantic parsing result according to condition information and / or instruction information and generating a trigger-execution rule based on the semantic parsing result according to another exemplary embodiment.

[0063] Figure 7 The figure is a flowchart of a method for processing instructions of a smart device according to another exemplary embodiment.

[0064] Figure 8 The invention is a block diagram of a command processing apparatus of a smart device according to an exemplary embodiment.

[0065] Fig. 9 is a block diagram of an electronic device according to an exemplary embodiment.

[0066] In the figure:

[0067] 10 - acquisition module; 20 - parsing module; 30 - generation module; 101 - processing component; 102 - memory; 103 - power component; 104 - multimedia component; 105 - audio component; 106 - input / output interface; 107 - sensor component; 108 - communication component; 109 - processor. DETAILED DESCRIPTION

[0068] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0069] In related technologies, with the development of artificial intelligence, natural language processing (NLP), Internet of Things (IOT), etc., people can control smart devices to execute specific instructions through electronic devices such as smartphones, speakers, and central gateways. In order to realize the automatic control of smart devices in different scenarios, it is necessary to create or customize smart home automation rules according to actual needs. At present, the mainstream paradigm of automation rules is the trigger-action process (TAP), which usually corresponds to the "when-if-then" language rule sentence in the field of smart homes.

[0070] However, using existing instruction processing methods to create automated rules for trigger-execution processes usually requires complex operations such as filtering and sorting the logic and processes of specific program execution based on instructions. At the same time, it is necessary to consider scenario planning and device functions and characteristics, resulting in a complex creation process, difficulty in popularization, and poor user experience.

[0071] Based on this, the exemplary embodiment of the present disclosure provides a method for processing instructions for a smart device, which obtains natural language instructions and performs semantic analysis on the natural language instructions, and generates a trigger-execution rule based on the semantic analysis result, which controls the smart device to execute a first control instruction whenever a trigger condition is met. Through this method for processing instructions for a smart device, the obtained natural language instructions can be disassembled and the trigger conditions and control instructions in the trigger-execution rule can be generated, and a trigger-execution rule that conforms to logic can be automatically generated without performing other complex operations, and a simple trigger-execution rule can meet the use requirements of most scenarios, thereby improving the user experience.

[0072] In an exemplary embodiment, reference Figure 1As shown, a method for processing instructions for a smart device is provided. The smart device is a device with computing and processing capabilities, such as a smart TV, a smart speaker, a smart curtain, a smart air conditioner, a smart lamp, and the like. The smart device has a functional interface, and the functional interface of the smart device can receive control instructions that match its functional interface, so that the smart device can perform actions corresponding to the control instructions. For example, when the smart device is a smart TV, its functional interface may include a functional interface for turning on the smart TV, a functional interface for playing TV programs, and the like. For another example, when the smart device is a smart speaker, its functional interface may include a functional interface for turning on the smart speaker, a functional interface for adjusting the volume, and the like. When the smart device is a smart air conditioner, its functional interface may include a functional interface for turning on the smart air conditioner, a functional interface for adjusting the temperature, and the like. Figure 1 As shown, the instruction processing method of the smart device includes:

[0073] S100: Obtain natural language instructions.

[0074] In step S100, natural language generally refers to a language that evolves naturally with culture and is the language used by humans to communicate with each other in daily life. Natural language instructions are instructions issued by users according to actual scenarios and needs. Natural language instructions can be requests, commands, wishes, reminders, prohibitions, etc. that conform to imperative sentences, such as "I want to turn on the lights every time I enter the living room at night", "I want to watch a movie", "When the weather is like this in the future, please close the windows and turn on the purifier", etc.

[0075] S200: semantically parse the natural language instruction to obtain a semantic parsing result, where the semantic parsing result at least includes a trigger condition and a first control instruction that match a functional interface of the smart device.

[0076] In step S200, the acquired natural language instructions can be semantically parsed by, for example, natural language processing (NLP) technology, wherein the natural language processing technology includes but is not limited to text error correction, intent recognition, and slot extraction (keyword extraction), etc., to obtain semantic parsing results, so as to achieve human-computer interaction and convey user needs to the device. The obtained semantic parsing results include trigger conditions such as "the time is evening and the location is in the living room" and a first control instruction such as "turn on the light". The trigger conditions and the first control instruction that match the functional port of the smart lamp for turning on the lamp are obtained through semantic parsing. S300, based on the semantic parsing results, a trigger-execution rule is generated, and the trigger-execution rule is configured as follows: when the trigger condition is met, the smart device is controlled to execute the first control instruction.

[0077] In step S300, the semantic analysis result is used as the basis for generating a trigger-execution rule. For example, when the semantic analysis result includes the trigger condition of "the time is evening and the location is in the living room" and the first control instruction of "turn on the light", the trigger-execution rule of "turn on the light when the time is evening and the location is in the living room" is automatically generated. In this way, when the trigger condition of "the time is evening and the location is in the living room" is met, the first control instruction of "turn on the light" is sent to the function port of the smart lamp to turn on the lamp, so that the smart lamp is automatically turned on. The smart device corresponding to a semantic analysis result and a trigger-execution rule can be one or more. For example, when the trigger-execution rule is "turn on the air conditioner when the temperature reaches 30 degrees", the corresponding smart device is multiple, including a smart thermometer and a smart air conditioner. The functional ports involved include a function port for detecting temperature of the smart thermometer and a function port for turning on the cooling mode of the smart air conditioner. The temperature is detected by the smart thermometer, and the cooling mode of the smart air conditioner is turned on when the temperature detected by the smart thermometer reaches 30 degrees.

[0078] In this embodiment, by acquiring natural language instructions and performing semantic analysis on the natural language instructions, a trigger-execution rule is generated based on the semantic analysis result to control the smart device to execute the first control instruction whenever the trigger condition is met. This instruction processing method for smart devices can disassemble the acquired natural language instructions and generate the trigger conditions and control instructions in the trigger-execution rules, automatically generate logical trigger-execution rules, without the need for other complex operations, and simple trigger-execution rules can meet the usage requirements of most scenarios, thereby improving the user experience.

[0079] In one embodiment, reference Figure 2 As shown, the natural language instruction is semantically parsed, and the semantic parsing results include:

[0080] S210: Identify condition information and / or instruction information in the natural language instruction.

[0081] In step S210, the condition information and instruction information in the natural language instruction can be identified by keyword extraction in natural language processing technology, and the identification rule can be, for example, by exhaustive enumeration of keywords or other identification rules in advance. For example, in this step, the condition information in the natural language instruction "I want to turn on the light every time I enter the living room at night" is "night" and "living room", and the instruction information is "turn on the light".

[0082] It is understandable that the number of conditional information and the number of instruction information in each natural language instruction are not fixed. For example, in the natural language instruction "I want to turn on the lights every time I enter the living room at night", there are two conditional information and one instruction information; in the natural language instruction "Please close the windows and turn on the purifier", there is no conditional information but two instruction information.

[0083] S220: Determine a semantic parsing result according to the condition information and / or instruction information.

[0084] In step S220, the semantic parsing result is determined based on the identified conditional information and instruction information, that is, the semantic parsing result can characterize the conditional information and instruction information in the natural language instruction, the identified conditional information corresponds to the trigger condition in the semantic parsing result, and the identified instruction information corresponds to the first control instruction in the semantic parsing result, providing a basis for subsequent generation of trigger-execution rules based on the semantic parsing result.

[0085] In this embodiment, by identifying the condition information and instruction information in the natural language instructions, and determining the semantic parsing results that can characterize the condition information and instruction information based on the condition information and instruction information, and the semantic parsing results include the trigger conditions and the first control instructions for generating trigger-execution rules, the decomposition of the natural language instructions is achieved, which provides the necessary prerequisites for the subsequent generation of trigger-execution rules based on the semantic parsing results, without the need for other complex operations, and the simple trigger-execution rules can meet the usage requirements of most scenarios, thereby improving the user experience.

[0086] In one embodiment, reference Figure 3 As shown, determining the semantic parsing result according to the condition information and / or the instruction information includes:

[0087] S310: When the natural language instruction includes condition information and instruction information, a trigger condition is generated according to the condition information.

[0088] In step S310, when condition information and instruction information are identified in the natural language instruction, in this case, a trigger condition is generated according to the condition information. For example, when the semantic parsing result includes the condition information of "evening" and "living room", the trigger conditions of "time is evening" and "location is living room" are generated accordingly as a trigger condition of a trigger-execution rule.

[0089] S320: Generate a first control instruction according to the instruction information.

[0090] In step S320, a first control instruction is generated according to the instruction information. For example, when the semantic parsing result includes instruction information of "turn on the light", a first control instruction of "turn on the light" is generated accordingly as the first control instruction of the same trigger-execution rule.

[0091] It can be understood that after the trigger conditions and the first control instructions are generated, trigger-execution rules can be generated based on the generated trigger conditions and the first control instructions. For example, according to the trigger conditions of "the time is evening" and "the location is the living room" and the first control instruction of "turn on the lights", the corresponding trigger-execution rule of "turn on the lights when the time is evening and the location is in the living room" is generated.

[0092] In this embodiment, when the natural language instruction includes condition information and instruction information, the trigger condition and the first control instruction are generated respectively according to the condition information and the instruction information, and the corresponding trigger-execution rules are subsequently generated according to the generated trigger condition and the first control instruction. The condition information and execution information in the natural language instruction are automatically converted to the trigger condition and the first control instruction in the semantic parsing result, and the trigger-execution rules are automatically generated without performing other complex operations. The simple trigger-execution rules can meet the usage requirements of most scenarios, thereby improving the user experience.

[0093] In one embodiment, the condition information includes event condition information; or, the condition information includes event condition information and state condition information.

[0094] Among them, event condition information and state condition information are two different condition information. For the "when-if-then" language rule sentence, event condition information can correspond to the "when" therein, and a natural language instruction has condition information as a necessary condition for forming a trigger-execution rule, while state condition information can correspond to the "if" therein, and a natural language instruction has state information as a non-necessary condition for forming a trigger-execution rule, but it can improve the trigger-execution rule and provide more condition definitions and restrictions for it. Therefore, the condition information must include event condition information. These two types of condition information can be combined through logical relationships such as "and / and", "or", and "not" to form a composite trigger condition. For example, the condition information of "evening" and "living room" can form a composite trigger condition of "time is evening and location is living room".

[0095] It is understandable that when performing semantic analysis on some natural language instructions with multiple conditional information, the same conditional information may be identified as event conditional information or state conditional information. For example, when performing semantic analysis on the natural language instruction "I want to turn on the lights every time I enter the living room at night", "night" may be identified as event conditional information and "living room" as state conditional information. It is also possible that "living room" is identified as event conditional information and "night" is identified as state conditional information.

[0096] In this embodiment, by classifying event condition information and state condition information, it is possible to simultaneously judge the event and state of the trigger condition formed by the condition information, and to form a complex judgment condition through specific logical relationships. On the premise of having event condition information to ensure that trigger-execution rules can be formed, it is also possible to provide more condition definitions and restrictions for the trigger-execution rules through state condition information, thereby improving the trigger-execution rules, making the formed trigger-execution rules more in line with the actual needs of users, and enhancing the user experience.

[0097] In one embodiment, determining the semantic parsing result according to the condition information and / or the instruction information includes: when the natural language instruction only includes the instruction information,

[0098] S410: Generate a second control instruction according to the instruction information and control the smart device to execute the second control instruction.

[0099] In step S410, when the natural language instruction only includes instruction information and no condition information, a second control instruction is generated and the smart device is controlled to directly execute the second control instruction. The second control instruction is different from the first control instruction and is not used as a control instruction for generating a trigger-execution rule, but as a control instruction for direct execution. For example, after receiving the natural language instruction "I want to turn on the light", the natural language instruction only includes the instruction information "turn on the light" but no condition information, then the second control instruction "turn on the light" can be generated, and the relevant lighting device can be directly executed. This control instruction "turn on the light" is executed.

[0100] In this embodiment, when the natural language instruction only includes instruction information, a second control instruction can be generated according to the instruction information and the smart device can be controlled to directly execute the second control instruction, thereby realizing the direct execution of the control instruction in the case of unconditional information in the natural language instruction, avoiding the neglect of instruction information in the case of unconditional information, ensuring the applicability of the instruction processing method to different natural language instructions, and improving the user experience.

[0101] In one embodiment, reference Figure 4 As shown, determining the semantic parsing result according to the condition information and / or the instruction information includes: when the semantic parsing result only includes the instruction information,

[0102] S510: Acquire first supplementary information, and determine condition information according to the first supplementary information.

[0103] In step S510, when the semantic parsing result includes only instruction information and no condition information, first supplementary information is obtained, and the first supplementary information includes new supplementary condition information, and the condition information missing from the original natural language instruction can be determined based on the first supplementary information. For example, if there is no condition information in the natural language instruction "I want to turn on the light", but the first supplementary information obtained is "at night", then the condition information can be determined to be "at night" based on the first supplementary information.

[0104] S520, generating a trigger condition according to the condition information;

[0105] S530: Generate a first control instruction according to the instruction information.

[0106] In steps S520 to S530, trigger conditions and first control instructions are generated according to the condition information and instruction information respectively. Subsequently, trigger-execution rules can be generated according to the trigger conditions and the first control instructions. The process of generating trigger-execution rules is similar to step S300 described above and will not be repeated here.

[0107] In this embodiment, conditional information is determined based on the first supplementary information obtained, thereby supplementing the originally missing conditional information, so that the necessary conditions missing for generating trigger-execution rules are supplemented, and the trigger conditions and the first control instructions can be generated respectively according to the conditional information and instruction information obtained after the supplement, thereby ensuring that the generation of trigger-execution rules is ultimately realized when the conditional information is missing in the natural language instructions, thereby ensuring the fault tolerance of the instruction processing method for natural language instructions and improving the user experience.

[0108] In one embodiment, when the natural language instruction only includes instruction information, a second control instruction can be generated and the first supplementary information can be obtained at the same time. While controlling the smart device to execute the second control instruction, conditional information is determined based on the first supplementary information to generate a trigger-execution rule, thereby simultaneously realizing the direct execution of natural language instructions that lack conditional information and the supplementation of conditional information, thereby avoiding the user's actual needs not being met and improving the user experience.

[0109] In one embodiment, obtaining the first supplementary information includes:

[0110] Based on the current scenario, determining first supplementary information; and / or,

[0111] A first supplementary instruction is obtained, and first supplementary information is determined based on the first supplementary instruction.

[0112] Among them, the current scene for determining the first supplementary information based on the current scene includes but is not limited to time, the location of the smart device, the environment in which the smart device is located, the usage habits of the smart device, time such as morning, afternoon, 9 o'clock or 15 o'clock, etc., the location of the smart device such as the room attributes, the floor, etc., the environment such as brightness, sound, temperature, etc., and the usage habits such as the frequency and order of use of the device. That is, when the conditional information in the semantic parsing result is incomplete, the first supplementary information can be determined based on these fixed collocations and common sense logic to supplement and improve the conditional information. For example, in the natural language instruction "When the weather is like this in the future, please close the window and turn on the purifier", there is no clear and definite conditional information. The weather information such as the temperature is 10°C and the humidity is 50% will be queried and recorded according to the environment of the current scene, which is the first supplementary information obtained, and then the conditional information is determined as "the temperature is 10°C" and "the humidity is 50%" through the first supplementary information. A first supplementary instruction can also be obtained, and the first supplementary information can be determined based on the first supplementary instruction. The first supplementary instruction can, for example, be a natural language instruction re-issued by the user, or a keyword supplement for the missing conditional information. For example, when the first supplementary instruction obtained is "supplement in the living room", the first supplementary information is determined to be "living room" based on this first supplementary instruction.

[0113] In this embodiment, the first supplementary information is determined based on the current scenario and based on the acquired first supplementary instruction, thereby realizing the determination of the first supplementary information according to the actual situation or user needs, and further realizing the supplement of the conditional information that is missing for generating trigger-execution rules. The determination of the first supplementary information is reasonable and diverse, so that the instruction processing method can be applied to a variety of different situations, thereby improving the user experience.

[0114] It is understandable that when the condition information or instruction information in the natural language instruction is incomplete or lacks key factors, the condition information or instruction information can also be supplemented by obtaining the first supplementary information. The process of obtaining and determining the first supplementary information is as described in the above steps and will not be repeated here. For example, when the natural language instruction is "when the temperature reaches 30 degrees Celsius, turn on the air conditioner", the condition information "the temperature reaches 30 degrees Celsius" is incomplete, lacking the key factor of where the temperature reaches 30 degrees Celsius, and the condition information "turn on the air conditioner" is incomplete, lacking the key factor of which air conditioner to turn on. At this time, the location of obtaining the natural language instruction can be determined by, for example, the current scene, and the location information is used as the first supplementary information to supplement the condition information. If the current scene indicates that the user is in the living room, the condition information can be supplemented to "when the temperature in the living room reaches 30 degrees Celsius" through the first supplementary information "living room" and the instruction information can be supplemented to "turn on the air conditioner in the living room", so that the generated trigger condition and the first control instruction are clearer, which is conducive to controlling the smart device to execute the first control instruction when the trigger condition is met.

[0115] In one embodiment, reference Figure 5 As shown, before generating the trigger-execution rule, the instruction processing method of the smart device also includes:

[0116] S610: Issue a prompt to determine whether to supplement condition information, and upon receiving a second supplement instruction, determine second supplement information according to the second supplement instruction.

[0117] In step S610, a prompt is issued to ask whether to supplement the condition information. The prompt can be issued by the terminal device in the form of text or voice, for example, to prompt the user whether other reasonable condition information needs to be supplemented when the condition information is already available, so as to further improve the trigger condition in the trigger-execution rule generated subsequently. If a second supplementary instruction is received, the second supplementary information is determined according to the second supplementary instruction. Exemplarily, the terminal device issues a prompt of "whether to supplement other condition information" through voice. If the user feedbacks the second supplementary instruction "supplement at noon", the second supplementary information is determined to be "noon" according to the second supplementary instruction.

[0118] S620: Supplement the second supplementary information into the conditional information.

[0119] In step S620, after determining the second supplementary information, the second supplementary information is added to the conditional information, and subsequent trigger conditions and trigger-execution rules are generated based on the supplemented conditional information. For example, the second supplementary information is determined to be "noon", and this second supplementary information is added to the original conditional information of "kitchen", forming new conditional information of "noon" and "kitchen", so that the trigger condition of "the time is noon and the location is kitchen" is subsequently generated.

[0120] In this embodiment, a prompt is issued before generating a trigger-rule whether to supplement the conditional information, the second supplementary information is determined according to the second supplementary instruction, and the second supplementary information is supplemented to the conditional information. The second supplementary information can be obtained before the trigger-rule is finally generated, and then other reasonable conditional information can be supplemented to complete the supplement and restriction of the trigger condition in the final trigger-execution rule, so that the final generated trigger-execution rule is more complete and meets the needs, and provides a supplementary way for the generation of complex rules. Users do not need to express complex natural language instructions completely at one time, and can supplement the conditional information through the second supplementary instruction, thereby improving the user experience.

[0121] In one embodiment, reference Figure 6 As shown, when the instruction information includes multiple items, the semantic parsing result is determined according to the condition information and / or the instruction information, including:

[0122] S710, generating a trigger condition according to the condition information;

[0123] S720, generating a plurality of first control instructions according to a plurality of instruction information;

[0124] Generate trigger-execution rules based on the semantic analysis results, including:

[0125] S730, generating one or more trigger-execution rules based on the trigger condition and the multiple first control instructions, wherein the multiple trigger-execution rules correspond to the multiple first control instructions one by one;

[0126] When a trigger-execution rule is generated, the trigger-execution rule is configured to: execute a plurality of first control instructions when a trigger condition is met;

[0127] When multiple trigger-execution rules are generated, each trigger-execution rule is configured to execute the corresponding first control instruction when the trigger condition is met.

[0128] Among them, multiple command information is identified when performing semantic analysis on natural language commands. For example, in the natural language command "When the weather is like this in the future, please close the windows and turn on the purifier", the two command information "close the windows" and "turn on the purifier" are identified; or, in the natural language command "I want to watch a movie", multiple command information "turn off the lights", "turn on the TV", and "turn on the speakers" are identified through analysis of the keyword "watch a movie".

[0129] The process of generating the trigger condition in step S710 is the same as that in step S310. Step S720 generates multiple first control instructions according to multiple instruction information. For example, it is possible to generate multiple first control instructions of "close the window" and "turn on the purifier" according to the instruction information of "close the window" and "turn on the purifier", or generate multiple first control instructions of "turn off the lights", "turn on the TV", "turn on the speakers" according to the instruction information of "turn off the lights", "turn on the TV", "turn on the speakers".

[0130] In step S730, one or more trigger execution rules are generated based on the trigger condition and multiple first control instructions. When a trigger-execution rule is generated, multiple first control instructions are executed when the trigger condition of the trigger-execution rule is met. For example, a trigger-execution rule of "when the time is evening, turn off the lights, turn on the TV, and turn on the speakers" is generated based on the trigger condition of "the time is evening" and multiple first control instructions such as "turn off the lights", "turn on the TV", and "turn on the speakers". When the trigger condition of "when the time is evening" is met, multiple first control instructions such as "turn off the lights", "turn on the TV", and "turn on the speakers" are executed at the same time.

[0131] When multiple trigger-execution rules are generated, the multiple trigger-execution rules correspond one-to-one to the multiple first control instructions respectively. When the trigger condition is met, each trigger execution rule executes the corresponding first control instruction when its trigger condition is met. For example, according to the trigger condition of "the time is evening" and multiple first control instructions such as "turn off the lights", "turn on the TV", and "turn on the speakers", multiple trigger-execution rules of "when the time is evening, turn off the lights", "when the time is evening, turn on the TV", and "when the time is evening, turn on the speakers" are generated respectively. When the trigger condition of "when the time is evening" is met, each trigger-execution rule executes the corresponding first control instructions of "turn off the lights", "turn on the TV", and "turn on the speakers".

[0132] In this embodiment, when there are multiple instruction information, multiple first control instructions are generated, and one or more trigger-execution rules are generated based on the multiple first control instructions. Batch recognition of multiple instruction information and creation of trigger-execution rules are achieved through two methods: one trigger-execution rule with multiple first control instructions and multiple trigger-execution rules with one first control instruction. Trigger-execution rules that can implement complex control instructions are created in a simple way, which can meet the usage needs of most users in most scenarios and improve the user experience.

[0133] In one embodiment, obtaining the natural language instruction includes: voice recognition of the natural language instruction and / or receiving the natural language instruction of text input.

[0134] In this embodiment, the method of obtaining natural language instructions includes but is not limited to voice recognition and receiving text input, that is, the user can obtain natural language instructions by issuing voice instructions and inputting text, so that the instruction processing method has good applicability and popularity, and improves the user experience.

[0135] In an exemplary embodiment, reference Figure 7 As shown, a method for processing instructions of a smart device is provided, and the method for processing instructions of a smart device includes:

[0136] S10, obtaining a natural language instruction;

[0137] S20, identifying condition information and / or instruction information in the natural language instruction;

[0138] S30, when the natural language instruction includes condition information and instruction information, issuing a prompt whether to supplement the condition information, and upon receiving a second supplementary instruction, determining the second supplementary information according to the second supplementary instruction;

[0139] S31, adding the second supplementary information to the condition information;

[0140] S32, generating a trigger condition according to the condition information;

[0141] S33, when the instruction information includes one, generating a first control instruction according to the instruction information;

[0142] S34, generating a trigger-execution rule based on the trigger condition and the first control instruction;

[0143] S35, when the instruction information includes multiple instructions, generating multiple first control instructions according to the multiple instruction information;

[0144] S36, generating one or more trigger-execution rules based on the trigger condition and the multiple first control instructions, wherein the multiple trigger-execution rules correspond to the multiple first control instructions one by one;

[0145] S40: when the natural language instruction includes only instruction information, generating a second control instruction according to the instruction information and executing the second control instruction;

[0146] S50: when the natural language instruction only includes instruction information, obtaining first supplementary information, and determining condition information according to the first supplementary information;

[0147] S51, issuing a prompt for whether to supplement the conditional information, and upon receiving the second supplementary instruction, determining the second supplementary information according to the second supplementary instruction;

[0148] S52, adding the second supplementary information to the condition information;

[0149] S53, generating a trigger condition according to the condition information;

[0150] S54, when the instruction information includes one, generating a first control instruction according to the instruction information;

[0151] S55, generating a trigger-execution rule based on the trigger condition and the first control instruction;

[0152] S56, when the instruction information includes multiple instructions, generate multiple first control instructions according to the multiple instruction information;

[0153] S57. Generate one or more trigger-execution rules based on the trigger condition and the multiple first control instructions, where the multiple trigger-execution rules correspond one-to-one to the multiple first control instructions respectively.

[0154] In this embodiment, a natural language instruction is obtained and semantically parsed, and a trigger-execution rule is generated based on the semantic parsing result to execute the first control instruction whenever the trigger condition is met. Through this instruction processing method of an intelligent device, the obtained natural language instruction can be disassembled into trigger conditions and control instructions in the trigger-execution rule, and a logical trigger-execution rule can be automatically generated without performing other complex operations, and a simple trigger-execution rule can meet the use requirements of most scenarios. At the same time, the missing condition information is supplemented by the first supplementary information and the second supplementary information or other reasonable condition information is supplemented, which can ensure the fault tolerance of the instruction processing method for natural language instructions, and make the final generated trigger-execution rule more complete and meet the needs. When there are multiple instruction information, batch recognition of multiple instruction information and creation of trigger-execution rules are realized, which improves the user experience.

[0155] In an exemplary embodiment, reference Figure 8 As shown, a smart device instruction processing device is provided, and the instruction processing device includes: an acquisition module 10, a parsing module 20 and a generation module 30. The acquisition module 10 is used to acquire natural language instructions, and the parsing module 20 is used to perform semantic parsing on the natural language instructions to obtain a semantic parsing result. The semantic parsing result at least includes a trigger condition and a first control instruction that match the functional interface of the smart device. The generation module 30 is used to generate a trigger-execution rule based on the semantic parsing result. The trigger-execution rule is configured as follows: when the trigger condition is met, the smart device is controlled to execute the first control instruction.

[0156] In this embodiment, natural language instructions are acquired through the acquisition module 10, semantic analysis is performed on the natural language instructions through the parsing module 20, and a trigger-execution rule is generated through the generation module 30 for controlling the smart device to execute the first control instruction whenever the trigger condition is met. The acquired natural language instructions can be decomposed into trigger conditions and control instructions in the trigger-execution rule, and logical trigger-execution rules can be automatically generated without performing other complex operations. The simple trigger-execution rule can meet the usage requirements of most scenarios, thereby improving the user experience.

[0157] In one embodiment, the parsing module 20 is further used to: identify condition information and / or instruction information in the natural language instruction; and determine a semantic parsing result according to the condition information and / or instruction information.

[0158] In one embodiment, the parsing module 20 is further used to: when the natural language instruction includes condition information and instruction information, generate a trigger condition according to the condition information; and generate a first control instruction according to the instruction information.

[0159] In one embodiment, the parsing module 20 is also used to: when the natural language instruction only includes instruction information, generate a second control instruction and execute the second control instruction according to the instruction information; and / or obtain first supplementary information and determine conditional information according to the first supplementary information; generate a trigger condition according to the conditional information; and generate a first control instruction according to the instruction information.

[0160] In one embodiment, the analysis module 20 is further used to: issue a prompt whether to supplement the condition information, and upon receiving the second supplement instruction, determine the second supplement information according to the second supplement instruction; and supplement the second supplement information to the condition information.

[0161] In one embodiment, the analysis module 20 is further used to: generate a trigger condition according to the condition information when the instruction information includes multiple instructions; and generate multiple first control instructions according to the multiple instruction information.

[0162] In one embodiment, the generation module 30 is also used to: generate one or more trigger-execution rules based on the trigger condition and multiple first control instructions, and the multiple trigger-execution rules correspond one-to-one to the multiple first control instructions respectively; when generating a trigger-execution rule, the trigger-execution rule is configured to: execute multiple first control instructions when the trigger condition is met; when generating multiple trigger-execution rules, each trigger-execution rule is configured to: execute the corresponding first control instruction when the trigger condition is met.

[0163] In an exemplary embodiment, an electronic device is provided, and the electronic device may be, for example, a mobile phone, a speaker, a central gateway, etc.

[0164] refer to Fig. 9As shown, the electronic device may include one or more of the following components: a processing component 101 , a memory 102 , a power component 103 , a multimedia component 104 , an audio component 105 , an input / output (I / O) interface 106 , a sensor component 107 , and a communication component 108 .

[0165] The processing component 101 generally controls the overall operation of the electronic device, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 101 may include one or more processors 109 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 101 may include one or more modules to facilitate the interaction between the processing component 101 and other components. For example, the processing component 101 may include a multimedia module to facilitate the interaction between the multimedia component 104 and the processing component 101.

[0166] The memory 102 is configured to store various types of data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc. The memory 102 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0167] The power component 103 provides power to various components of the electronic device. The power component 103 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device.

[0168] The multimedia component 104 includes a screen that provides an output interface between the electronic device and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 104 includes a front camera and / or a rear camera. When the electronic device is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0169] The audio component 105 is configured to output and / or input audio signals. For example, the audio component 105 includes a microphone (MIC), and when the electronic device is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 102 or sent via the communication component 108. In some embodiments, the audio component 105 also includes a speaker for outputting audio signals.

[0170] I / O interface 106 provides an interface between processing component 101 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0171] The sensor assembly 107 includes one or more sensors for providing various aspects of status assessment for the electronic device. For example, the sensor assembly 107 can detect the on / off state of the electronic device, the relative positioning of components, such as the display and keypad of the electronic device, and the sensor assembly 107 can also detect the position change of the electronic device or a component of the electronic device, the presence or absence of user contact with the electronic device, the orientation or acceleration / deceleration of the electronic device and the temperature change of the electronic device. The sensor assembly 107 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 107 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 107 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0172] The communication component 108 is configured to facilitate wired or wireless communication between the electronic device and other devices. The device can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 108 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 108 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0173] In an exemplary embodiment, the electronic device may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned instruction processing method for smart devices applied to electronic devices.

[0174] In an exemplary embodiment, a non-temporary computer-readable storage medium including instructions is also provided, such as a memory 102 including instructions, and the instructions can be executed by a processor 109 of an electronic device to complete the instruction processing method applied to the electronic device. For example, the non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a tape, a floppy disk, and an optical data storage device. When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is enabled to execute the instruction processing method of the smart device shown in the above embodiment.

[0175] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0176] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for processing instructions of a smart device, It is characterized in that The instruction processing method of the smart device includes: Get natural language instructions; Performing semantic parsing on the natural language instruction to obtain a semantic parsing result, wherein the semantic parsing result at least includes a trigger condition and a first control instruction that match a functional interface of the smart device; Based on the semantic parsing result, a trigger-execution rule is generated, and the trigger-execution rule is configured to: when the trigger condition is met, control the smart device to execute the first control instruction.

2. The method for processing instructions of a smart device according to claim 1, It is characterized in that The performing semantic analysis on the natural language instruction to obtain a semantic analysis result includes: identifying condition information and / or instruction information in the natural language instruction; The semantic parsing result is determined according to the condition information and / or the instruction information.

3. The instruction processing method of the smart device according to claim 2, It is characterized in that The determining the semantic parsing result according to the condition information and / or the instruction information includes: When the natural language instruction includes the condition information and the instruction information, generating the trigger condition according to the condition information; The first control instruction is generated according to the instruction information.

4. The instruction processing method of the smart device according to claim 3, It is characterized in that The condition information includes event condition information; or, The condition information includes event condition information and state condition information.

5. The instruction processing method of the smart device according to claim 2, It is characterized in that The determining the semantic parsing result according to the condition information and / or the instruction information includes: when the natural language instruction only includes the instruction information, Generate a second control instruction according to the instruction information and control the smart device to execute the second control instruction; and / or, Acquire first supplementary information, and determine the condition information according to the first supplementary information; generating the trigger condition according to the condition information; The first control instruction is generated according to the instruction information.

6. The instruction processing method of the smart device according to claim 5, It is characterized in that The obtaining of the first supplementary information includes: Based on the current scenario, determining the first supplementary information; and / or, A first supplementary instruction is obtained, and the first supplementary information is determined based on the first supplementary instruction.

7. The instruction processing method of the smart device according to claim 6, It is characterized in that The current scenario includes one or more of time, location of the smart device, environment of the smart device, and usage habits of the smart device.

8. The instruction processing method of the smart device according to any one of claims 2 to 7, It is characterized in that Before generating the trigger-execution rule, the instruction processing method of the smart device further includes: issuing a prompt for whether to supplement the conditional information, and upon receiving a second supplementary instruction, determining the second supplementary information according to the second supplementary instruction; The second supplementary information is supplemented to the condition information.

9. The instruction processing method of the smart device according to any one of claims 2 to 7, It is characterized in that When the instruction information includes multiple items, determining the semantic parsing result according to the condition information and / or the instruction information includes: generating the trigger condition according to the condition information; Generate a plurality of the first control instructions according to the plurality of instruction information; The generating a trigger-execution rule based on the semantic parsing result includes: Based on the trigger condition and the first control instructions, one or more trigger-execution rules are generated, and the trigger-execution rules correspond to the first control instructions one by one respectively; When generating one of the trigger-execution rules, the trigger-execution rule is configured to: execute a plurality of the first control instructions when the trigger condition is met; When a plurality of the trigger-execution rules are generated, each of the trigger-execution rules is configured to execute the corresponding first control instruction when the trigger condition is met.

10. The instruction processing method of the smart device according to any one of claims 1 to 7, It is characterized in that The obtaining of natural language instructions comprises: The natural language instructions are voice recognized and / or the natural language instructions are received as text input.

11. A command processing device for an intelligent device, It is characterized in that The instruction processing device of the smart device includes: An acquisition module, the acquisition module is used to acquire natural language instructions; A parsing module, the parsing module is used to perform semantic parsing on the natural language instruction to obtain a semantic parsing result, wherein the semantic parsing result at least includes a trigger condition and a first control instruction that match a functional interface of the smart device; A generation module, the generation module is used to generate a trigger-execution rule based on the semantic parsing result, and the trigger-execution rule is configured to: when the trigger condition is met, control the smart device to execute the first control instruction.

12. An electronic device, It is characterized in that The electronic device comprises: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: Get natural language instructions; Performing semantic parsing on the natural language instruction to obtain a semantic parsing result, wherein the semantic parsing result at least includes a trigger condition and a first control instruction that match a functional interface of the smart device; Based on the semantic parsing result, a trigger-execution rule is generated, and the trigger-execution rule is configured to: when the trigger condition is met, control the smart device to execute the first control instruction.

13. A non-transitory computer-readable storage medium, It is characterized in that When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute an instruction processing method of an intelligent device, the instruction processing method of the intelligent device comprising: Get natural language instructions; Performing semantic parsing on the natural language instruction to obtain a semantic parsing result, wherein the semantic parsing result at least includes a trigger condition and a first control instruction that match a functional interface of the smart device; Based on the semantic parsing result, a trigger-execution rule is generated, and the trigger-execution rule is configured to: when the trigger condition is met, control the smart device to execute the first control instruction.

Citation Information

Cited By

  • Control method and device based on semantic perception and related equipment

    CN121500788A