Semantic understanding method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a semantic understanding method, device and equipment and a storage medium, which are applied to a smart home system, the smart home system is deployed with a hierarchical attention network model, and the method comprises the following steps: in response to a current voice instruction of a user, obtaining context information of the current voice instruction; encoding the current voice instruction and the context information through a hierarchical attention network model to obtain a semantic representation vector containing a context; performing semantic analysis based on the semantic representation vector containing the context to obtain a semantic analysis result; and executing corresponding home equipment control operation according to the semantic analysis result. The context information of the user voice instruction is effectively coded and modeled by using the constructed hierarchical attention network model, so that accurate semantic understanding of the voice instruction with long-term context dependence in the home scene is realized, the interaction experience of the smart home system is greatly improved, and the response quality of the smart home system is ensured.

Description

Technical Field

[0001] The present invention relates to the field of semantic understanding technology, and in particular to a semantic understanding method, a semantic understanding device, a corresponding electronic device, and a corresponding computer-readable storage medium. Background Art

[0002] With the rapid development of artificial intelligence technology, smart home systems are widely used in home life. Users can control home appliances, adjust the environment, query information, etc. by issuing natural voice commands, greatly improving the convenience of life.

[0003] However, due to the complexity of home scenes, users' voice commands usually involve long-term context dependencies. Traditional semantic understanding is difficult to accurately understand the full meaning of users' voice commands, which will affect the response quality of smart home systems. Summary of the invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a semantic understanding method, a semantic understanding device, a corresponding electronic device and a corresponding computer-readable storage medium that overcome the above problems or at least partially solve the above problems.

[0005] The embodiment of the present invention discloses a semantic understanding method, which is applied to a smart home system, wherein the smart home system is deployed with a hierarchical attention network model, and the method comprises:

[0006] Responding to a current voice command of a user, obtaining context information of the current voice command;

[0007] Encoding the current voice instruction and the context information through the hierarchical attention network model to obtain a semantic representation vector containing the context;

[0008] Performing semantic parsing based on the semantic representation vector containing the context to obtain a semantic parsing result;

[0009] Execute corresponding home appliance control operations according to the semantic parsing results.

[0010] In some embodiments of the present invention, encoding the current voice instruction and the context information by the hierarchical attention network model to obtain a semantic representation vector containing the context includes:

[0011] Performing voice recognition on the current voice command to obtain current natural language text information;

[0012] The current natural language text information and the context information are encoded through the hierarchical attention network model to obtain a semantic representation vector containing the context.

[0013] In some embodiments of the present invention, the context information is used to indicate a set of historical vectors; encoding the current natural language text information and the context information through the hierarchical attention network model to obtain a semantic representation vector containing the context includes:

[0014] Encoding the current natural language text information through the hierarchical attention network model to obtain a current word vector sequence;

[0015] Performing weighted summation on the current word vector sequence to obtain a vector representation for the current voice instruction;

[0016] A semantic representation vector containing context is obtained according to the vector representation of the current voice instruction and the historical vector set of the current voice instruction.

[0017] In some embodiments of the present invention, the hierarchical attention network model includes a word-level encoding layer, and encoding the current natural language text information to obtain a current word vector sequence includes:

[0018] The current natural language text information is input into the word-level encoding layer, and each word in the current natural language text information is encoded in the word-level encoding layer to obtain a current word vector sequence.

[0019] In some embodiments of the present invention, the hierarchical attention network model includes a sentence-level encoding layer, and the weighted summing of the current word vector sequence to obtain a vector representation for the current voice instruction includes:

[0020] The attention mechanism of the sentence-level encoding layer is used to perform weighted summation on the current word vector sequence to obtain a sentence-level vector representation of the current voice instruction.

[0021] In some embodiments of the present invention, the hierarchical attention network model includes a context encoding layer, and the semantic representation vector containing the context is obtained according to the vector representation of the current voice instruction and the historical vector set of the current voice instruction, including:

[0022] Inputting the vector representation of the current voice instruction and the historical vector set of the current voice instruction into the context encoding layer;

[0023] The attention mechanism of the context encoding layer is used to obtain the weights of different historical vectors in the historical vector set, and the weights of the different historical vectors are used to weight the vector representation of the current voice instruction to obtain a semantic representation vector containing the context.

[0024] In some embodiments of the present invention, the smart home system deploys a hierarchical attention network model and a semantic parsing model, and the semantic parsing is performed based on the semantic representation vector containing the context to obtain a semantic parsing result, including:

[0025] Mapping the context-containing semantic representation vector into an understanding state vector through the semantic parsing model;

[0026] Outputting a semantic parsing result based on the understanding state vector.

[0027] In some embodiments of the present invention, the semantic parsing model includes an encoder network, and mapping the context-containing semantic representation vector to an understanding state vector includes:

[0028] Mapping the context-containing semantic representation vector into an understanding state vector through the encoder network;

[0029] The semantic parsing model includes a decoder network, and the outputting of the semantic parsing result based on the understanding state vector includes:

[0030] The decoder network outputs the intent category label, slot type label and corresponding slot value sequence according to the understanding state vector.

[0031] In some embodiments of the present invention, the performing corresponding home appliance control operations according to the semantic parsing results includes:

[0032] Obtaining a corresponding task execution strategy based on the semantic analysis result;

[0033] The task execution strategy is adjusted to obtain a home appliance control operation, and the home appliance control operation is executed.

[0034] In some embodiments of the present invention, after executing the home device control operation, the method further includes:

[0035] The control instruction execution result is obtained, and the control instruction execution result is input into the context encoder in the hierarchical attention network model as the context information for the next round of voice instructions.

[0036] The embodiment of the present invention further discloses a semantic understanding device, which is applied to a smart home system. The smart home system is deployed with a hierarchical attention network model. The device includes:

[0037] A voice command response module, used to respond to the user's current voice command and obtain context information of the current voice command;

[0038] A semantic representation vector generation module, used for encoding the current voice instruction and the context information through the hierarchical attention network model to obtain a semantic representation vector containing the context;

[0039] A semantic parsing module, used for performing semantic parsing based on the semantic representation vector containing the context to obtain a semantic parsing result;

[0040] The device control module is used to perform corresponding home device control operations according to the semantic parsing results.

[0041] In some embodiments of the present invention, the semantic representation vector generation module includes:

[0042] The semantic representation vector generation submodule is used to perform speech recognition on the current voice instruction to obtain current natural language text information; encode the current natural language text information and the context information through the hierarchical attention network model to obtain a semantic representation vector containing the context.

[0043] In some embodiments of the present invention, the context information is used to indicate a historical vector set; and the semantic representation vector generation submodule includes:

[0044] The semantic representation vector generating unit is used to encode the current natural language text information through the hierarchical attention network model to obtain a current word vector sequence; perform weighted summation on the current word vector sequence to obtain a vector representation for the current voice instruction; and obtain a semantic representation vector containing context based on the vector representation of the current voice instruction and the historical vector set of the current voice instruction.

[0045] In some embodiments of the present invention, the hierarchical attention network model includes a word-level encoding layer, and the semantic representation vector generation unit includes:

[0046] The word vector sequence generating subunit is used to input the current natural language text information into the word-level encoding layer, encode each word in the current natural language text information at the word-level encoding layer, and obtain the current word vector sequence.

[0047] In some embodiments of the present invention, the hierarchical attention network model includes a sentence-level encoding layer, and the semantic representation vector generation unit includes:

[0048] The vector representation generating subunit is used to adopt the attention mechanism of the sentence-level encoding layer to perform weighted summation on the current word vector sequence to obtain the sentence-level vector representation of the current voice instruction.

[0049] In some embodiments of the present invention, the hierarchical attention network model includes a context encoding layer, and the semantic representation vector generation unit includes:

[0050] The semantic representation vector generating subunit is used to input the vector representation of the current voice instruction and the historical vector set of the current voice instruction into the context coding layer; adopt the attention mechanism of the context coding layer to obtain the weights of different historical vectors in the historical vector set, and adopt the weights of the different historical vectors to weight the vector representation of the current voice instruction to obtain a semantic representation vector containing the context.

[0051] In some embodiments of the present invention, the smart home system deploys a hierarchical attention network model and a semantic parsing model, and the semantic parsing module includes:

[0052] The semantic parsing submodule is used to map the semantic representation vector containing the context into an understanding state vector through the semantic parsing model; and output a semantic parsing result based on the understanding state vector.

[0053] In some embodiments of the present invention, the semantic parsing model includes an encoder network, and the semantic parsing submodule includes:

[0054] An understanding state vector mapping unit, used for mapping the semantic representation vector containing the context into an understanding state vector through the encoder network;

[0055] In some embodiments of the present invention, the semantic parsing model includes a decoder network, and the semantic parsing submodule includes:

[0056] A semantic parsing unit is used to output an intent category label, a slot type label and a corresponding slot value sequence according to the understanding state vector through the decoder network.

[0057] In some embodiments of the present invention, the device control module includes:

[0058] The device control submodule is used to query and obtain the corresponding task execution strategy according to the semantic analysis result; adjust the task execution strategy to obtain the home device control operation, and execute the home device control operation.

[0059] In some embodiments of the present invention, after performing the home appliance control operation, the apparatus further includes:

[0060] The context information input module is used to obtain the control instruction execution result and input the control instruction execution result into the context encoder in the hierarchical attention network model as the context information for the next round of voice instructions.

[0061] An embodiment of the present invention further discloses an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements any one of the semantic understanding methods when executed by the processor.

[0062] An embodiment of the present invention further discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, any one of the semantic understanding methods is implemented.

[0063] The embodiments of the present invention include the following advantages:

[0064] In an embodiment of the present invention, a smart home system deployed with a hierarchical attention network model can respond to a user's current voice command, encode the current voice command and the context information of the current voice command, obtain a semantic representation vector containing the context, and then perform semantic analysis based on the semantic representation vector containing the context to obtain a semantic analysis result, and then perform corresponding home appliance control operations according to the semantic analysis result. By using the constructed hierarchical attention network model to effectively encode and model the context information of the user's voice command, accurate semantic understanding of voice commands with long-term context dependence in home scenarios is achieved, which greatly improves the interactive experience of the smart home system and ensures the response quality of the smart home system. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 is a flowchart of steps of an embodiment of a semantic understanding method of the present invention;

[0066] Figure 2 is a flowchart of steps of another semantic understanding method embodiment of the present invention;

[0067] Figure 3 It is a schematic diagram of a process of smart home semantic understanding based on modeling long-term scene context information provided by an embodiment of the present invention;

[0068] Figure 4 It is a structural block diagram of an embodiment of a semantic understanding device of the present invention. DETAILED DESCRIPTION

[0069] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0070] The embodiment of the present invention uses the constructed hierarchical attention network model to effectively encode and model the context information of the user's voice commands, thereby achieving accurate semantic understanding of voice commands with long-term context dependencies in home scenarios, greatly improving the interactive experience of the smart home system, and ensuring the response quality of the smart home system. Specifically, the context encoder module adaptively models the long-term interaction history and environmental changes through the hierarchical attention network model, and inputs high-quality context semantic representations into the semantic parsing model, thereby improving the accuracy of understanding complex commands. Furthermore, the execution results of the control commands after executing the home device control operation can also be used as the context information of the next round of voice commands to achieve closed-loop intelligent interaction.

[0071] Reference Figure 1 , shows a flowchart of a semantic understanding method embodiment of the present invention, which is applied to a smart home system and may specifically include the following steps:

[0072] Step 101, responding to the user's current voice command and obtaining context information of the current voice command;

[0073] The semantic understanding method provided by the embodiment of the present invention can be applied to home scenes with voice entry devices, and specifically can be applied to smart home systems, so that the smart home system can accurately understand the user's voice commands based on modeling long-term scene context information, thereby ensuring the response quality of the smart home system.

[0074] In order to effectively encode and model the contextual information of the user's voice commands and achieve accurate semantic understanding of voice commands with long-term context dependencies in home scenarios, the smart home system can respond to the user's current voice commands and obtain the contextual information of the current voice commands.

[0075] The context information of the current voice command obtained can be multi-dimensional long-term context information, which can include multi-dimensional context information such as the text content of the user's voice command, historical interactions and scene status, to help the smart home system understand and respond to the user's commands more accurately.

[0076] Among them, the text content of the current voice command is the most direct contextual information. The smart home system can understand the user's current needs and intentions by analyzing the text; historical interaction can refer to the previous interaction records between the user and the smart home system, which can provide important contextual information. For example, if the user has turned on the air conditioner before, when the user's next step is to adjust the temperature, the target device for the voice command issued by the user can be assumed to be the air conditioner; the scene state can refer to the scene state of the smart home system, such as time, location, device status, etc., which can also be used as contextual information. For example, if the home device where the smart home system is located knows that it is currently night, when the user issues a voice command to turn on the lights, it may tend to prefer the command to turn on the lights in the room.

[0077] Step 102, encoding the current voice command and context information through a hierarchical attention network model to obtain a semantic representation vector containing the context;

[0078] In some embodiments of the present invention, the smart home system can be deployed with a hierarchical attention network model. The constructed hierarchical attention network model can perform multi-level encoding modeling on the user's current voice command and its related context information, so as to obtain a semantic representation vector that integrates multi-dimensional context information.

[0079] The semantic representation vector containing context can be used for subsequent tasks, such as command understanding, dialogue management, or response generation. The aforementioned vector is mainly obtained by capturing and encoding the semantic information of text or voice instructions and related contextual information.

[0080] Specifically, the hierarchical attention network model can process data containing multiple hierarchical structures. When processing voice commands and contextual information, the hierarchical attention network model can effectively capture semantic information at different levels and generate a semantic representation vector containing context.

[0081] In the specific implementation, the processing object of the hierarchical attention network model is usually text information. The smart home system can also be deployed with a speech recognition model. First, the current voice command can be recognized by the speech recognition model to obtain the current natural language text information, and then the current natural language text information and context information are encoded through the hierarchical attention network model to obtain a semantic representation vector containing the context.

[0082] Among them, the hierarchical attention network model can include a word-level encoding layer, a sentence-level encoding layer, and a context encoding layer, mainly based on the aforementioned multi-level structure, to achieve the capture of semantic information at different levels to generate a semantic representation vector that integrates the context. It should be noted that the specific semantic representation vector process implemented by the aforementioned hierarchical structures is not limited by the embodiment of the present invention.

[0083] Step 103, performing semantic analysis based on the semantic representation vector containing the context to obtain a semantic analysis result;

[0084] In order to identify the user intention represented by the user's current voice command, semantic parsing may be performed on the semantic representation vector containing the context.

[0085] Specifically, semantic parsing based on a semantic representation vector containing context involves converting abstract representations in a high-dimensional vector space into specific semantic information and structured data.

[0086] In some embodiments of the present invention, the smart home system may also be deployed with a semantic parsing model. In this case, the semantic parsing model may be used to map a semantic representation vector containing context into an understanding state vector, and then the semantic parsing result may be output based on the understanding state vector to realize the recognition of user intention.

[0087] Step 104: Execute corresponding home appliance control operations according to the semantic analysis results.

[0088] After obtaining the semantic parsing result, that is, identifying the user's specific intention from the semantic parsing result, the relevant home appliances and operation instructions can be extracted from the semantic parsing result to execute the corresponding home appliance control operations and realize the response of the smart home system to the user's voice commands.

[0089] Among them, the user intention obtained by parsing based on the semantic representation vector containing context can often capture the contextual connection between the user command and the previous interaction or the current scene, that is, the user intention obtained by the aforementioned analysis is mainly the user intention obtained based on the contextual connection. The home device control operation determined based on the user intention can also be a related operation instruction with contextual association. The smart home system not only parses the user's current command, but also considers the contextual information such as previous conversations, time, scenes or historical interactions, so as to provide more personalized and coherent services.

[0090] It should be noted that the embodiment of the present invention uses a context encoder for obtaining a context semantic representation vector, a semantic parsing model for outputting a semantic parsing result, and a control execution module for performing corresponding operations after optimizing and adjusting the home environment state, so as to realize the efficient modeling of complex multi-dimensional long-term context dependency information using a hierarchical attention network model and integrate it into the process of semantic understanding, thereby breaking through the bottleneck of traditional methods that are difficult to deal with long-term context dependency problems, and improving the accuracy of understanding voice commands in home scenarios. The above-mentioned hierarchical attention network model for smart home semantic understanding can adaptively capture the importance of contexts of different granularities, form a high-quality semantic representation containing comprehensive context information, and lay the foundation for smart home systems to achieve high-quality voice interaction.

[0091] In an embodiment of the present invention, a smart home system deployed with a hierarchical attention network model can respond to a user's current voice command, encode the current voice command and the context information of the current voice command, obtain a semantic representation vector containing the context, and then perform semantic analysis based on the semantic representation vector containing the context to obtain a semantic analysis result, and then perform corresponding home appliance control operations according to the semantic analysis result. By using the constructed hierarchical attention network model to effectively encode and model the context information of the user's voice command, accurate semantic understanding of voice commands with long-term context dependence in home scenarios is achieved, which greatly improves the interactive experience of the smart home system and ensures the response quality of the smart home system.

[0092] Reference Figure 2 , shows a flowchart of another semantic understanding method embodiment of the present invention, which may specifically include the following steps:

[0093] Step 201, performing voice recognition on the current voice command to obtain current natural language text information;

[0094] The semantic understanding method provided by the embodiment of the present invention can be applied to home scenes with voice entry devices, and specifically can be applied to smart home systems, so that the smart home system can accurately understand the user's voice commands based on modeling long-term scene context information, thereby ensuring the response quality of the smart home system.

[0095] In some embodiments of the present invention, the smart home system can be deployed with a hierarchical attention network model. The constructed hierarchical attention network model can perform multi-level encoding modeling on the user's current voice command and its related context information, so as to obtain a semantic representation vector that integrates multi-dimensional context information.

[0096] In the specific implementation, the processing object of the hierarchical attention network model is usually text information. The smart home system can also deploy a speech recognition model, such as Figure 3As shown, first, the current voice command can be recognized by the voice recognition model to obtain the user's current natural language text information. For example, this can be achieved by obtaining the user's current voice command and converting the voice signal into text utterance through the voice recognition model.

[0097] Step 202, encoding the current natural language text information and context information through a hierarchical attention network model to obtain a semantic representation vector containing the context;

[0098] Specifically, the hierarchical attention network model can process data containing multiple hierarchical structures. When processing voice commands and contextual information, the hierarchical attention network model can effectively capture semantic information at different levels and generate a semantic representation vector containing context.

[0099] The context information of the current voice command obtained can be multi-dimensional long-term context information, which can include multi-dimensional context information such as the text content of the user's voice command, historical interactions and scene status, to help the smart home system understand and respond to the user's commands more accurately.

[0100] Among them, the text content of the current voice command is the most direct contextual information. The smart home system can understand the user's current needs and intentions by analyzing the text; historical interaction can refer to the previous interaction records between the user and the smart home system, which can provide important contextual information. For example, if the user has turned on the air conditioner before, when the user's next step is to adjust the temperature, the target device for the voice command issued by the user can be assumed to be the air conditioner; the scene state can refer to the scene state of the smart home system, such as time, location, device status, etc., which can also be used as contextual information. For example, if the home device where the smart home system is located knows that it is currently night, when the user issues a voice command to turn on the lights, it may tend to prefer the command to turn on the lights in the room.

[0101] Optionally, the context information is text information, which may have a corresponding history vector set.

[0102] In some embodiments of the present invention, the current natural language text information can be encoded through a hierarchical attention network model to obtain a current word vector sequence, and then a vector representation for the current voice instruction is obtained by weighted summing the current word vector sequence. Then, based on the vector representation of the current voice instruction and the historical vector set of the current voice instruction, a semantic representation vector containing the context is obtained.

[0103] Specifically, the hierarchical attention network model can be deployed on the context encoder, such as Figure 3As shown, the current natural language text information converted by the speech recognition model can be input into the context encoder to generate a contextual semantic representation.

[0104] In practical applications, the hierarchical attention network model can include a word-level encoding layer, a sentence-level encoding layer, and a context encoding layer. It is mainly based on the aforementioned multi-level structure to capture semantic information at different levels to generate a semantic representation vector that integrates the context.

[0105] Optionally, the acquisition of the current word vector sequence can be achieved through the first-level word-level encoding layer, which can be specifically performed by inputting the current natural language text information into the word-level encoding layer, encoding each word in the current natural language text information at the word-level encoding layer, and obtaining the current word vector sequence.

[0106] For example, the word vector representation pre-trained by the word-level encoding layer and the bidirectional LSTM network (Long Short-Term Memory, a special recurrent neural network) can be used to encode each word in the utterance (user single sentence instruction) to generate a word vector sequence of the current utterance.

[0107] Optionally, the acquisition of sentence-level vector representation can be achieved through the second-level sentence-level encoding layer. Specifically, it can be manifested by using the attention mechanism of the sentence-level encoding layer to perform weighted summation on the current word vector sequence to obtain the sentence-level vector representation of the current voice command.

[0108] For example, based on the attention mechanism of the sentence-level encoding layer, that is, the utterance-level encoding layer, the word vector sequence can be weighted summed to obtain the utterance-level vector representation, thereby capturing the semantics of a single sentence instruction.

[0109] Optionally, the acquisition of the semantic representation vector containing the context can be achieved through the context encoding layer of the third level. Specifically, the vector representation of the current voice command and the historical vector set of the current voice command are input into the context encoding layer, and then the attention mechanism of the context encoding layer is used to obtain the weights of different historical vectors in the historical vector set, so as to weight the vector representation of the current voice command with the weights of different historical vectors to obtain the semantic representation vector containing the context.

[0110] For example, the context encoding layer can use the attention mechanism to model and weight the current utterance and its historical context utterance, so as to obtain a semantic representation vector that integrates the long-term context. Specifically, in the context encoding layer, the current utterance vector and its related historical utterance vector set can be used as input, and the attention mechanism is used to adaptively capture the weight contribution of different historical utterances, and these weighted historical utterance vectors are integrated to obtain the context semantic representation vector of the current utterance, which can integrate the long-term context information such as utterance text semantics, conversation history, and scene status.

[0111] Step 203, mapping the semantic representation vector containing the context into an understanding state vector through a semantic parsing model, and outputting a semantic parsing result based on the understanding state vector;

[0112] In order to identify the user intention represented by the user's current voice command, semantic parsing may be performed on the semantic representation vector containing the context.

[0113] Specifically, semantic parsing based on a semantic representation vector containing context involves converting abstract representations in a high-dimensional vector space into specific semantic information and structured data.

[0114] In some embodiments of the present invention, the smart home system may also be deployed with a semantic parsing model, and semantic parsing may be implemented through the semantic parsing model. The semantic parsing model may adopt a sequence-to-sequence based neural network architecture. At this time, the semantic representation vector containing the context may be mapped into an understanding state vector through the semantic parsing model, and then the semantic parsing result may be output based on the understanding state vector to realize the recognition of user intention.

[0115] Optionally, the semantic parsing model may include an encoder network, which may first map the semantic representation vector containing the context into an understanding state vector through the encoder network; the semantic parsing model may also include a decoder network, which may then output the intent category label, slot type label and corresponding slot value sequence according to the understanding state vector through the decoder network, and the aforementioned information output in sequence may be used as the semantic parsing result of the current utterance.

[0116] Exemplarily, in a dialogue system, the decoder network is typically used to extract structured semantic parsing results from the understanding state vector generated by the encoder (usually a semantic representation vector containing context). These semantic parsing results may include intent category labels, slot type labels, and corresponding slot value sequences.

[0117] Among them, the intent class label (Intent Class Label) represents the main purpose or action of the user input. For example, in the smart home control scenario, the user's intention may be to turn on the lights, adjust the temperature or play music, etc. For example, the system can determine the user's intent category through a classifier; the slot type (Slot Type Labels) is a predefined variable or label, which can be mainly used to capture and represent specific information in the user input. For example, in the dialogue system, the slot type can usually be used to extract and store the key information required for the user's intent. These slot types are usually associated with the user's intent and used to extract key information. For example, in the intent to play music, the slot type may include artist, album, song name, etc.; the slot value sequence (Slot Value Sequences) is a specific value sequence corresponding to the slot type. These values ​​can mainly be extracted from the natural language text information input by the user and used to fill the dialogue state.

[0118] In actual applications, through the decoder network, the system can output the following information in sequence as the semantic parsing result of the current utterance: an intent category label for determining the user's overall intent, a slot type label for identifying the type of key information that needs to be extracted from the user input, and a slot value sequence for extracting specific values ​​corresponding to the slot type from the user input, so as to provide the system with specific instructions and parameters required to perform subsequent operations based on the aforementioned semantic parsing results output in sequence.

[0119] Step 204: Execute corresponding home appliance control operations according to the semantic analysis results.

[0120] After obtaining the semantic parsing result, that is, identifying the user's specific intention from the semantic parsing result, the relevant home appliances and operation instructions can be extracted from the semantic parsing result to execute the corresponding home appliance control operations and realize the response of the smart home system to the user's voice commands.

[0121] The semantic analysis results can provide the system with specific instructions and parameters required to perform subsequent operations.

[0122] In some embodiments of the present invention, firstly, the corresponding task execution strategy, such as the task execution plan, etc., can be obtained according to the semantic analysis result query.

[0123] Specifically, Figure 3 As shown, the corresponding domain knowledge base or rule base can be queried according to the semantic parsing results to obtain the corresponding home appliance control operation instructions or task execution plans.

[0124] A simple example is as follows:

[0125] Assume that the user input is "Set the bedroom air conditioner to 23 degrees and turn on the music in the living room". The smart home system can identify the following two intents during intent recognition: adjust_temperature and control_music. The intent recognition results are as follows:

[0126]

[0127]

[0128] At this point you can fill in the slots for the two intents separately:

[0129]

[0130] The generated task plan can be expressed as follows:

[0131]

[0132] Furthermore, the task execution strategy may be adjusted to obtain the home appliance control operation and execute the home appliance control operation.

[0133] Optional, such as Figure 3 As shown, the queried operation instructions or plans can be further tailored and optimized in combination with the contextual state of the current home environment. For example, assuming that the user's current voice command is "It's too hot", the task plan can be adjusted to "Set the air conditioner to 23 degrees" in combination with the user's historical behavior, current ambient temperature, etc.; after optimization and adjustment in combination with the home environment state, the adjusted home device control operation or task execution plan can be executed to achieve command execution such as controlling home appliance switches, adjusting environmental parameters, and providing information query results.

[0134] In a preferred embodiment of the present invention, after executing the home appliance control operation, Figure 3 As shown, the control instruction execution result can also be obtained and input into the context encoder in the hierarchical attention network model as the context information for the next round of voice instructions to achieve closed-loop intelligent interaction. Based on the aforementioned feedback loop mechanism, it is ensured that the smart home system can learn from each interaction and continuously improve its performance. Through continuous optimization and adjustment, it can provide more intelligent and personalized services and enhance user experience.

[0135] In practical applications, the above-mentioned steps of voice signal conversion, context coding processing, semantic analysis, control execution, optimization adjustment, command execution, and feedback result input to the context encoder can be repeated, so that the semantic understanding and execution of user voice commands can be cyclically performed to achieve continuous human-computer dialogue interaction.

[0136] It should be noted that the semantic understanding method provided by the embodiment of the present invention can optimize and configure the model structure, parameters, etc. according to different scenarios and requirements to obtain a better understanding effect, and the embodiment of the present invention is not limited to this; and the embodiment of the present invention uses a context encoder for obtaining a context semantic representation vector, a semantic parsing model for outputting a semantic parsing result, and a control execution module for performing corresponding operations after optimizing and adjusting the home environment state to achieve efficient modeling of complex multi-dimensional long-term context dependency information using a hierarchical attention network model, and integrate it into the process of semantic understanding, thereby breaking through the bottleneck that traditional methods are difficult to handle long-term context dependency problems, and improving the understanding accuracy of voice commands in home scenarios. The hierarchical attention network model for semantic understanding of smart homes can adaptively capture the importance of contexts of different granularities, form a high-quality semantic representation containing comprehensive context information, and lay the foundation for smart home systems to achieve high-quality voice interaction. In addition, the various specific implementations of the context encoder and semantic parsing model provided in the embodiment of the present invention can be flexibly configured and optimized according to different scenarios and requirements to further improve practicality, and the embodiment of the present invention is not limited to this.

[0137] In an embodiment of the present invention, by using the constructed hierarchical attention network model to effectively encode and model the context information of the user's voice command, accurate semantic understanding of voice commands with long-term context dependence in home scenarios is achieved, which greatly improves the interactive experience of the smart home system and ensures the response quality of the smart home system. Specifically, the context encoder module adaptively models the long-term interaction history and environmental changes through the hierarchical attention network model, and inputs high-quality context semantic representations into the semantic parsing model, thereby improving the accuracy of understanding complex commands. Furthermore, the execution result of the control command after executing the home device control operation can also be used as the context information of the next round of voice commands to achieve closed-loop intelligent interaction.

[0138] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0139] Reference Figure 4, shows a structural block diagram of an embodiment of a semantic understanding device of the present invention, which is applied to a smart home system. The smart home system deploys a hierarchical attention network model, which may specifically include the following modules:

[0140] The voice command response module 401 is used to respond to the user's current voice command and obtain context information of the current voice command;

[0141] A semantic representation vector generation module 402, configured to encode the current voice instruction and the context information through the hierarchical attention network model to obtain a semantic representation vector including the context;

[0142] A semantic parsing module 403, configured to perform semantic parsing based on the semantic representation vector containing the context to obtain a semantic parsing result;

[0143] The device control module 404 is used to perform corresponding home device control operations according to the semantic parsing result.

[0144] In some embodiments of the present invention, the semantic representation vector generation module 402 may include the following submodules:

[0145] The semantic representation vector generation submodule is used to perform speech recognition on the current voice instruction to obtain current natural language text information; encode the current natural language text information and the context information through the hierarchical attention network model to obtain a semantic representation vector containing the context.

[0146] In some embodiments of the present invention, the context information is used to indicate a historical vector set; the semantic representation vector generation submodule may include the following units:

[0147] The semantic representation vector generating unit is used to encode the current natural language text information through the hierarchical attention network model to obtain a current word vector sequence; perform weighted summation on the current word vector sequence to obtain a vector representation for the current voice instruction; and obtain a semantic representation vector containing context based on the vector representation of the current voice instruction and the historical vector set of the current voice instruction.

[0148] In some embodiments of the present invention, the hierarchical attention network model includes a word-level encoding layer, and the semantic representation vector generation unit may include the following subunits:

[0149] The word vector sequence generating subunit is used to input the current natural language text information into the word-level encoding layer, encode each word in the current natural language text information at the word-level encoding layer, and obtain the current word vector sequence.

[0150] In some embodiments of the present invention, the hierarchical attention network model includes a sentence-level encoding layer, and the semantic representation vector generation unit may include the following subunits:

[0151] The vector representation generating subunit is used to adopt the attention mechanism of the sentence-level encoding layer to perform weighted summation on the current word vector sequence to obtain the sentence-level vector representation of the current voice instruction.

[0152] In some embodiments of the present invention, the hierarchical attention network model includes a context encoding layer, and the semantic representation vector generation unit may include the following subunits:

[0153] The semantic representation vector generating subunit is used to input the vector representation of the current voice instruction and the historical vector set of the current voice instruction into the context coding layer; adopt the attention mechanism of the context coding layer to obtain the weights of different historical vectors in the historical vector set, and adopt the weights of the different historical vectors to weight the vector representation of the current voice instruction to obtain a semantic representation vector containing the context.

[0154] In some embodiments of the present invention, the smart home system deploys a hierarchical attention network model and a semantic parsing model, and the semantic parsing module 403 may include the following submodules:

[0155] The semantic parsing submodule is used to map the semantic representation vector containing the context into an understanding state vector through the semantic parsing model; and output a semantic parsing result based on the understanding state vector.

[0156] In some embodiments of the present invention, the semantic parsing model includes an encoder network, and the semantic parsing submodule may include the following units:

[0157] An understanding state vector mapping unit, used for mapping the semantic representation vector containing the context into an understanding state vector through the encoder network;

[0158] In some embodiments of the present invention, the semantic parsing model includes a decoder network, and the semantic parsing submodule may include the following units:

[0159] A semantic parsing unit is used to output an intent category label, a slot type label and a corresponding slot value sequence according to the understanding state vector through the decoder network.

[0160] In some embodiments of the present invention, the device control module 404 may include the following submodules:

[0161] The device control submodule is used to query and obtain the corresponding task execution strategy according to the semantic analysis result; adjust the task execution strategy to obtain the home device control operation, and execute the home device control operation.

[0162] In some embodiments of the present invention, after performing the home appliance control operation, the apparatus provided by the embodiment of the present invention may further include the following modules:

[0163] The context information input module is used to obtain the control instruction execution result and input the control instruction execution result into the context encoder in the hierarchical attention network model as the context information for the next round of voice instructions.

[0164] In an embodiment of the present invention, through the semantic understanding device provided by the embodiment of the present invention, a smart home system deployed with a hierarchical attention network model can respond to the user's current voice command, encode the current voice command and the context information of the current voice command, obtain a semantic representation vector containing the context, and then perform semantic analysis based on the semantic representation vector containing the context to obtain a semantic analysis result, and then perform corresponding home appliance control operations according to the semantic analysis result. By using the constructed hierarchical attention network model to effectively encode and model the context information of the user's voice command, accurate semantic understanding of voice commands with long-term context dependence in home scenarios is achieved, which greatly improves the interactive experience of the smart home system and ensures the response quality of the smart home system.

[0165] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0166] An embodiment of the present invention further provides an electronic device, including:

[0167] It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned semantic understanding method embodiment are implemented. To avoid repetition, they are not described here.

[0168] In an embodiment of the present invention, the electronic device provided by the embodiment of the present invention enables a smart home system deployed with a hierarchical attention network model to respond to a user's current voice command, encode the current voice command and the context information of the current voice command, obtain a semantic representation vector containing the context, and then perform semantic analysis based on the semantic representation vector containing the context to obtain a semantic analysis result, and then perform corresponding home appliance control operations according to the semantic analysis result. By using the constructed hierarchical attention network model to effectively encode and model the context information of the user's voice command, accurate semantic understanding of voice commands with long-term context dependence in home scenarios is achieved, which greatly improves the interactive experience of the smart home system and ensures the response quality of the smart home system.

[0169] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned semantic understanding method embodiment are implemented. To avoid repetition, they are not described here.

[0170] In an embodiment of the present invention, the computer-readable storage medium provided by the embodiment of the present invention enables a smart home system deployed with a hierarchical attention network model to respond to a user's current voice command, encode the current voice command and the context information of the current voice command, obtain a semantic representation vector containing the context, and then perform semantic analysis based on the semantic representation vector containing the context to obtain a semantic analysis result, and then perform corresponding home appliance control operations according to the semantic analysis result. By using the constructed hierarchical attention network model to effectively encode and model the context information of the user's voice command, accurate semantic understanding of voice commands with long-term context dependence in home scenarios is achieved, which greatly improves the interactive experience of the smart home system and ensures the response quality of the smart home system.

[0171] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0172] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0173] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0174] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0175] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0176] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0177] Finally, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0178] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0179] The above is a detailed introduction to a semantic understanding method, a semantic understanding device, a corresponding electronic device and a corresponding computer-readable storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for a general technician in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A semantic understanding method, characterized in that: Applied to a smart home system, the smart home system deploys a hierarchical attention network model, and the method includes: Responding to a current voice command of a user, obtaining context information of the current voice command; Encoding the current voice instruction and the context information through the hierarchical attention network model to obtain a semantic representation vector containing the context; Performing semantic parsing based on the semantic representation vector containing the context to obtain a semantic parsing result; Execute corresponding home appliance control operations according to the semantic parsing results.

2. The method according to claim 1, characterized in that The step of encoding the current voice instruction and the context information by the hierarchical attention network model to obtain a semantic representation vector containing the context includes: Performing voice recognition on the current voice command to obtain current natural language text information; The current natural language text information and the context information are encoded through the hierarchical attention network model to obtain a semantic representation vector containing the context.

3. The method according to claim 2, characterized in that The context information is used to indicate a historical vector set; the current natural language text information and the context information are encoded by the hierarchical attention network model to obtain a semantic representation vector containing the context, including: Encoding the current natural language text information through the hierarchical attention network model to obtain a current word vector sequence; Performing weighted summation on the current word vector sequence to obtain a vector representation for the current voice instruction; A semantic representation vector containing context is obtained according to the vector representation of the current voice instruction and the historical vector set of the current voice instruction.

4. The method according to claim 3, characterized in that The hierarchical attention network model includes a word-level encoding layer, and the encoding of the current natural language text information to obtain a current word vector sequence includes: The current natural language text information is input into the word-level encoding layer, and each word in the current natural language text information is encoded in the word-level encoding layer to obtain a current word vector sequence.

5. The method according to claim 3, characterized in that: The hierarchical attention network model includes a sentence-level encoding layer, and the weighted summation of the current word vector sequence to obtain a vector representation for the current voice instruction includes: The attention mechanism of the sentence-level encoding layer is used to perform weighted summation on the current word vector sequence to obtain a sentence-level vector representation of the current voice instruction.

6. The method according to claim 3, characterized in that: The hierarchical attention network model includes a context encoding layer, and the semantic representation vector containing context is obtained according to the vector representation of the current voice instruction and the historical vector set of the current voice instruction, including: Inputting the vector representation of the current voice instruction and the historical vector set of the current voice instruction into the context encoding layer; The attention mechanism of the context encoding layer is used to obtain the weights of different historical vectors in the historical vector set, and the weights of the different historical vectors are used to weight the vector representation of the current voice instruction to obtain a semantic representation vector containing the context.

7. The method according to claim 1, characterized in that The smart home system is deployed with a hierarchical attention network model and a semantic parsing model, and the semantic parsing is performed based on the semantic representation vector containing the context to obtain a semantic parsing result, including: Mapping the context-containing semantic representation vector into an understanding state vector through the semantic parsing model; Outputting a semantic parsing result based on the understanding state vector.

8. The method according to claim 7, characterized in that The semantic parsing model includes an encoder network, and mapping the context-containing semantic representation vector to an understanding state vector includes: Mapping the context-containing semantic representation vector into an understanding state vector through the encoder network; The semantic parsing model includes a decoder network, and the outputting of the semantic parsing result based on the understanding state vector includes: The decoder network outputs the intent category label, slot type label and corresponding slot value sequence according to the understanding state vector.

9. The method according to claim 1, characterized in that: The performing corresponding home appliance control operations according to the semantic parsing results includes: Obtaining a corresponding task execution strategy based on the semantic analysis result; The task execution strategy is adjusted to obtain a home appliance control operation, and the home appliance control operation is executed.

10. The method according to claim 9, characterized in that After executing the home device control operation, the method further includes: The control instruction execution result is obtained, and the control instruction execution result is input into the context encoder in the hierarchical attention network model as the context information for the next round of voice instructions.

11. A semantic understanding device, characterized in that: Applied to a smart home system, the smart home system deploys a hierarchical attention network model, and the device includes: A voice command response module, used to respond to a user's current voice command and obtain context information of the current voice command; A semantic representation vector generation module, used for encoding the current voice instruction and the context information through the hierarchical attention network model to obtain a semantic representation vector containing the context; A semantic parsing module, used for performing semantic parsing based on the semantic representation vector containing the context to obtain a semantic parsing result; The device control module is used to execute corresponding home device control operations according to the semantic parsing results.

12. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the semantic understanding method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the semantic understanding method according to any one of claims 1 to 10 is implemented.

Images