Semantic recognition method, apparatus, device, storage medium, and vehicle

Abstract

The disclosure provides a semantic recognition method, device, equipment, storage medium and vehicle. It relates to the technical field of data processing, and particularly relates to the technical fields of natural language processing, deep learning and human-computer interaction. The specific implementation is: obtaining a target voice instruction; matching the target voice instruction with a plurality of semantic templates established based on rules to obtain a first confidence degree of matching the target voice instruction with an optimal semantic template of the target voice instruction; in a case where the first confidence degree is higher than a confidence threshold, obtaining a semantic recognition result of the target voice instruction based on the optimal semantic template; and in a case where the first confidence degree is less than or equal to the confidence threshold, inputting the target voice instruction into a natural language understanding model to obtain the semantic recognition result of the target voice instruction. The combination of the semantic template and the natural language understanding model can quickly and accurately understand the semantics of the voice instruction.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and more particularly to the fields of natural language processing, deep learning, and human-computer interaction technology. Background Technology

[0002] The rapid development of AI (Artificial Intelligence) technology has made it possible to achieve rapid human-computer interaction for diverse needs on-device platforms, thus overcoming the shortcomings of cloud-based semantic understanding, such as slow response speed and reliance on network smoothness. However, in the case of voice-based human-computer interaction, improving the accuracy of semantic understanding has become a challenge and a key focus. Summary of the Invention

[0003] This disclosure provides a semantic recognition method, apparatus, device, storage medium, and vehicle.

[0004] According to one aspect of this disclosure, a semantic recognition method is provided, comprising:

[0005] Obtain the target voice command;

[0006] The target speech command is matched with multiple semantic templates based on rules to obtain the first confidence score of the optimal semantic template match between the target speech command and the target speech command.

[0007] When the first confidence level is higher than the confidence threshold, the semantic recognition result of the target speech instruction is obtained based on the optimal semantic template;

[0008] If the first confidence level is less than or equal to the confidence threshold, the target speech command is input into the natural language understanding model to obtain the semantic recognition result of the target speech command.

[0009] According to another aspect of this disclosure, a method for training a natural language understanding model is provided, comprising:

[0010] Obtain sample speech commands and their annotation results;

[0011] Input the sample speech command into the model to be trained to obtain the semantic recognition result of the sample speech command;

[0012] The model loss is determined based on the semantic recognition and annotation results;

[0013] The parameters of the model to be trained are adjusted based on the model loss. When the model to be trained meets the convergence condition, a natural language understanding model is obtained.

[0014] Among them, the natural language understanding model is used to perform semantic recognition of the target speech instruction when the first confidence level of the target speech instruction matching the optimal semantic template of the target speech instruction is less than the confidence level threshold.

[0015] According to another aspect of this disclosure, a semantic recognition device is provided, comprising:

[0016] The first acquisition module is used to acquire the target voice command;

[0017] The matching module is used to match the target speech command with multiple semantic templates based on rules to obtain the first confidence score of the optimal semantic template matching between the target speech command and the target speech command.

[0018] The rule-based semantic recognition module is used to obtain the semantic recognition result of the target speech instruction based on the optimal semantic template when the first confidence level is higher than the confidence level threshold.

[0019] The model semantic recognition module is used to input the target speech command into the natural language understanding model and obtain the semantic recognition result of the target speech command when the first confidence level is less than or equal to the confidence level threshold.

[0020] According to another aspect of this disclosure, an apparatus for training a natural language understanding model is provided, comprising:

[0021] The second acquisition module is used to acquire sample speech commands and the annotation results of the sample speech commands;

[0022] The semantic recognition module is used to input sample speech commands into the model to be trained and obtain the semantic recognition results of the sample speech commands.

[0023] The loss determination module is used to determine the model loss based on the semantic recognition results and annotation results;

[0024] The adjustment module is used to adjust the parameters of the model to be trained based on the model loss, so as to obtain a natural language understanding model when the model to be trained meets the convergence condition.

[0025] Among them, the natural language understanding model is used to perform semantic recognition of the target speech instruction when the first confidence level of the target speech instruction matching the optimal semantic template of the target speech instruction is less than the confidence level threshold.

[0026] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0027] At least one processor; and

[0028] The memory is communicatively connected to the at least one processor; wherein,

[0029] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0030] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of this disclosure.

[0031] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of this disclosure.

[0032] According to another aspect of this disclosure, a vehicle is provided carrying electronic devices as described in the embodiments of this disclosure.

[0033] By combining semantic templates and natural language understanding models, the semantics of voice commands can be understood quickly and accurately.

[0034] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0035] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0036] Figure 1 This is a schematic flowchart of a semantic recognition method according to an embodiment of the present disclosure;

[0037] Figure 2 This is a schematic diagram of a natural language understanding model according to an embodiment of the present disclosure;

[0038] Figure 3 This is a flowchart of a semantic recognition scenario according to an embodiment of the present disclosure;

[0039] Figure 4 This is a flowchart illustrating a natural language understanding model training method according to an embodiment of the present disclosure;

[0040] Figure 5 This is a schematic diagram of a natural language understanding model according to an embodiment of the present disclosure;

[0041] Figure 6 This is a schematic diagram of the structure of a semantic recognition device according to an embodiment of the present disclosure;

[0042] Figure 7 This is a schematic diagram of the structure of a natural language understanding model training device according to an embodiment of the present disclosure;

[0043] Figure 8 This is a block diagram of an electronic device used to implement the semantic recognition method of the embodiments of this disclosure. Detailed Implementation

[0044] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0045] With the development of related technologies, human-computer interaction has been widely applied in various fields. For example, in the driving environment, manual operation is less convenient for users, while users have many diverse needs. These include closing windows, controlling air conditioning, providing navigation services, and even entertainment. Users can address these diverse needs by issuing voice commands.

[0046] The core challenge in achieving human-computer interaction is semantic recognition of user voice commands to determine their intent. Semantic recognition requires identifying the domain, intent, and slot of the voice command. For example, in the command "Query flights from Beijing to Shanghai today," the domain is "flights," the intent is "buy a flight," the time slot is "today," the origin slot is "Beijing," and the destination slot is "Shanghai."

[0047] Currently, there are two main methods for semantic recognition: one is to pre-set a template and then match the user's voice command with the template to achieve semantic recognition; the other is to achieve semantic recognition through a trained neural network model.

[0048] However, semantic template configuration is cumbersome and generalization ability is poor when semantic recognition is based on semantic templates. Multi-intent voice commands are easily misrepresented by single-intent templates. When semantic recognition is achieved by simply using a trained neural network model, the recognition speed is relatively slower than that of semantic templates.

[0049] Therefore, this disclosure provides a semantic recognition method that combines semantic recognition based on semantic templates with semantic recognition achieved through a trained neural network model, thereby overcoming the shortcomings of the two methods mentioned above. The semantic recognition method provided in this disclosure is not limited to end-side voice interaction implemented in vehicle terminals, but is also applicable to other scenarios requiring voice interaction. For example... Figure 1 As shown, the method includes:

[0050] S101, Obtain the target voice command.

[0051] In this embodiment of the disclosure, after receiving the voice signal, voice recognition is performed on the voice signal to obtain the target voice command.

[0052] S102. Match the target speech instruction with multiple semantic templates established based on rules to obtain the first confidence level of the optimal semantic template matching between the target speech instruction and the target speech instruction.

[0053] That is, in this embodiment of the disclosure, semantic recognition is first performed based on a semantic template. The first confidence level can be determined by two aspects: the first aspect is the degree of matching between the target voice command and the optimal semantic template, and the second aspect is whether the target voice command includes the keywords of the semantic template. For example, if the keywords of the semantic template "open the car window" are "open" and "car window", then if the target voice command does not contain "open" or "car window", the first confidence level is 0. The optimal semantic template refers to the semantic template with the highest first confidence level.

[0054] S103. When the first confidence level is higher than the confidence level threshold, the semantic recognition result of the target speech instruction is obtained based on the optimal semantic template; wherein, the confidence level threshold can be determined according to the rules for establishing the template and the applicable scenario.

[0055] S104. If the first confidence level is less than or equal to the confidence level threshold, input the target speech command into the natural language understanding model to obtain the semantic recognition result of the target speech command.

[0056] This disclosure first determines a first confidence level between the target speech command and the semantic template. When the first confidence level is high, the semantic recognition result of the target speech command is obtained based on the optimal semantic template, enabling fast and accurate semantic recognition of the target speech command. When the first confidence level is low, a natural language understanding model is used to obtain the semantic recognition result, improving the applicability and accuracy of semantic recognition. In summary, by combining semantic templates and a natural language understanding model, the semantic recognition method provided by this disclosure does not require a large number of semantic templates, solving the problem of cumbersome configuration when using only semantic templates. Furthermore, when semantic templates cannot achieve high confidence in semantic recognition, the use of a natural language understanding model can improve the generalization ability of semantic recognition. Moreover, performing semantic recognition using a speech template before the natural language understanding model can significantly improve the recognition speed for certain speech commands compared to using a purely neural network model.

[0057] In some embodiments, for frequently occurring voice commands, corresponding semantic templates can be established to improve semantic recognition speed. Therefore, the multiple semantic templates established based on rules can include: semantic templates corresponding to voice commands with frequencies higher than a frequency threshold (hereinafter referred to as templates for high-frequency voice commands). Thus, templates for high-frequency voice commands allow high-frequency voice commands to be semantically recognized without going through a neural network model, reducing the consumption of system resources and enabling rapid semantic recognition.

[0058] In some embodiments, natural language understanding models require training with a large number of training samples. These training samples include samples from different vertical categories. The number of training samples in different vertical categories is often unbalanced, and model training tends to focus more on learning from vertical categories with a larger number of samples, while learning from smaller samples is often insufficient. Therefore, the multiple semantic templates established based on rules can include semantic templates corresponding to speech instructions in target categories with a sample size below a threshold (hereinafter referred to as small sample templates). Thus, using small sample templates can address the problem of low recall rates in some imbalanced vertical categories of natural language understanding models, improving the semantic recognition ability of speech instructions.

[0059] In some embodiments, the natural language understanding model may include two network structures: single-intent recognition and multi-intent recognition. However, the multi-intent recognition network structure, which relies on joint learning of multi-intent recognition and slot filling, struggles to fully model each intent and its corresponding slot. Furthermore, the multi-intent recognition network structure is prone to confusion when performing multi-intent semantic recognition on certain multi-intent speech commands. For example, the phrase "open the sunroof and turn the sunshade up a bit" may not be recognized as both "open the sunroof" and "turn the sunshade up a bit" during semantic recognition; it may only be recognized as "turn the sunroof and sunshade up a bit." Therefore, to address the low accuracy of natural language understanding models in recognizing some multi-intent speech commands, multiple semantic templates built based on rules may include semantic templates corresponding to speech commands containing multiple intents (hereinafter referred to as multi-intent templates). In implementation, based on the training status of the natural language understanding model, corresponding multi-intent templates can be established for speech commands with multiple intents that the natural language model cannot accurately recognize. Using multi-intent templates can improve the semantic recognition capability of multi-intent speech commands and increase the accuracy of semantic recognition.

[0060] In some embodiments, when matching a target speech command with multiple rule-based semantic templates, the matching method used can be determined based on the storage method of the semantic templates. For example, when the storage method is a Trie (prefix) tree storage method, a prefix matching method can be used to select the optimal semantic template. If the first confidence level between the target speech command and the optimal semantic template is higher than a confidence threshold, the semantic recognition result identified by the optimal semantic template is output. Taking "open the car window" as an example, during prefix matching, subsequences such as "open," "open," "open the car window," "open the car window," and "car window" are matched with multiple slot lists to select the optimal semantic template, i.e., the slot combination with the first confidence level. If the first confidence level obtained from the matching is higher than the confidence threshold, the optimal semantic template is selected and the semantic recognition result is obtained; if the first confidence level obtained from the matching is lower than the confidence threshold, a natural language understanding model is used for semantic recognition. Of course, this disclosure does not specifically limit the storage method of the semantic templates and the corresponding matching method.

[0061] In some embodiments, to improve the accuracy of the natural language understanding model in recognizing speech instructions containing multiple intentions, the natural language understanding model in this disclosure includes a clause segmentation network and a semantic understanding network. The clause segmentation network is used to segment clauses from the target speech instruction, and the semantic understanding network can perform semantic recognition on the segmented clauses. The clause segmentation network and the semantic understanding network will be described below.

[0062] 1) Clause splitting network

[0063] In some embodiments, the neural network model may not meet the required accuracy for recognizing certain multi-intent speech commands. Therefore, a clause splitting network can be used to break down the target speech command containing multiple clauses into multiple single-intent clauses before semantic recognition, thereby improving the accuracy of the semantic recognition results. Specifically, this can be implemented as follows:

[0064] Step A1: The target speech instruction is split into clauses based on the clause splitting network in the natural language understanding model.

[0065] In some embodiments, to improve the accuracy of clause segmentation, the natural language understanding model may include an encoder for extracting features from the target speech instruction. Based on this, when identifying clauses in the target speech instruction, the process can be implemented as follows:

[0066] Step A11: Obtain the second vector representation of each word in the target speech instruction output by the encoder of the natural language understanding model.

[0067] For example, when the target speech command is segmented into words, resulting in word segment 1, word segment 2, and word segment 3, the second vector representation of each word segment includes the second vector representation of word segment 1, the second vector representation of word segment 2, and the second vector representation of word segment 3.

[0068] In implementation, the target speech command can be segmented first to obtain the segmentation results, and then the second vector representation of each segmented word can be obtained based on the encoder. This disclosure does not specifically limit the type of encoder or segmentation method used; for example, Ernie-nano (text-center model nano-encoder) can be used for encoding, and ErnieTokenizer (text-center model tokenizer) can be used for segmentation.

[0069] Step A12: Split the network by representing the second vector as the input clause to obtain the clause probability distribution.

[0070] Clause splitting networks are used to predict how many clauses a target speech instruction includes and predict the probability that each word belongs to each clause. Therefore, the clause probability distribution output by the clause splitting network can include the probability distribution of the number of clauses in the target speech instruction, as well as the probability distribution of each word belonging to each clause.

[0071] Step A13: Decompose the target speech instruction into clauses based on the clause probability distribution.

[0072] An encoder can extract features from each word segment, obtaining a second vector representation for each word. The second vector representation obtained through feature extraction provides features that facilitate clause segmentation, thus improving the accuracy of clause segmentation. By using a clause segmentation network to break down target speech instructions into clauses, multi-intent target speech instructions can be divided into single-intent clauses, thereby improving the recognition capability of multi-intent speech instructions.

[0073] When the target speech instruction is split into multiple clauses, step A2 is performed to obtain the semantic recognition result of the target speech instruction. Step A2 includes steps A21-A22.

[0074] Step A21: For each clause, treat the clause as a new target speech instruction and return to execute S102-S104 until the semantic recognition result of the clause is obtained.

[0075] That is, execute for each clause:

[0076] The clause is matched with multiple semantic templates based on rules to obtain the first confidence score of the clause matching the optimal semantic template. If the first confidence score is higher than the confidence score threshold, the semantic recognition result of the clause is obtained based on the optimal semantic template. If the first confidence score is less than or equal to the confidence score threshold, the clause is input into a natural language understanding model and the semantic recognition result of the clause is obtained through the semantic understanding network.

[0077] Step A22: Based on the semantic recognition results of each of the multiple clauses, determine the semantic recognition result of the target speech command.

[0078] In cases where no clause has a identifiable actual intent, the set of semantic recognition results for each clause is used as the semantic recognition result for the target speech command. If some clauses cannot be identified as having an actual intent, those clauses are removed, and the set of semantic recognition results for the remaining clauses is used as the semantic recognition result for the target speech command.

[0079] In some embodiments, a response is given for each clause as soon as its semantic recognition result is obtained, without waiting for the semantic recognition results of all clauses to be obtained. This can improve the speed of human-computer interaction.

[0080] Using a clause splitting model to break down the target speech command into multiple clauses can improve the recognition capability of multi-intent speech commands. By performing steps S102-S104 on each clause, we can ensure that the semantic recognition of each clause is fast and accurate, thereby achieving fast and accurate semantic recognition of the target speech command.

[0081] Step A3: When the target speech instruction is split into a clause, obtain the semantic recognition result of the target speech instruction output by the semantic understanding network in the natural language understanding model.

[0082] When the target speech instruction is split into a clause, using a model to implement the target speech instruction can avoid repeated steps such as semantic template matching, reduce resource consumption, and improve the speed of semantic recognition of the target speech instruction.

[0083] 2) Semantic Understanding Network

[0084] Semantic understanding networks can perform semantic recognition on target speech commands. In this embodiment of the disclosure, although the semantic recognition result of the target speech command can be obtained quickly based on the semantic template, the configuration of the semantic template is cumbersome, and it needs to be configured separately for different domains. Furthermore, it may not be able to accurately recognize some semantics. Therefore, semantic understanding networks can be used as a supplement.

[0085] The semantic understanding network includes a domain recognition layer, an intent recognition layer, and a slot recognition layer, which can respectively identify the domain, intent, and slot of the target speech command to obtain the semantic recognition result of the target speech command. Determining the semantic recognition result of the target speech command based on the semantic understanding network can be implemented as follows:

[0086] Step B1: The semantic understanding network based on the natural language understanding model determines the domain probability distribution, intent probability distribution, and slot probability distribution of each word segment in the target speech instruction; Step B1 specifically includes steps B11-B14:

[0087] Step B11: Input the target speech command into the encoder of the natural language understanding model to obtain the first vector representation of the target speech command and the second vector representation of each word in the target speech command.

[0088] Step B12: Input the first vector into the intent recognition layer to obtain the intent probability distribution; and input the first vector into the domain recognition layer to obtain the domain probability distribution.

[0089] Step B13: Concatenate the second vector representation of each word segment with the domain probability distribution to obtain the third vector representation.

[0090] Step B14: Input the third vector representation into the slot recognition layer to obtain the slot probability distribution of each word segment.

[0091] Since the slots for different word segments may differ across different domains, this embodiment uses both the second vector representation of each word and the domain probability distribution to determine the slot probability distribution. Taking "turn on the air conditioner" as an example, assuming the domain probability distribution has a dimension of 6 and the second vector representation of each word has a dimension of 3, the domain probability distribution and the second vector representation of each word are concatenated to obtain a 9-dimensional representation. Similarly, assuming there are n words, the dimension of the third vector representation is n*9, and the slot recognition layer uses this third vector representation to obtain the slot probability distribution for each word.

[0092] In the above method, the domain probability distribution and intent probability distribution of the target voice command can be obtained through the first vector representation of the target voice command; the domain probability distribution is concatenated with the second vector representation to improve the accuracy of slot recognition.

[0093] Step B2: Determine the semantic recognition result of the target speech command based on the domain probability distribution, intent probability distribution, and slot probability distribution of each word segmentation.

[0094] For example, taking "searching for flight tickets from Beijing to Shanghai today" as an example, the probability of the domain being "buying flight tickets" is 0.7, and the probability of the domain being "travel" is 0.3; the probability of the intent being "searching for flight tickets" is 0.8, and the probability of the intent being "other" is 0.2; the probability of the time slot being "today" is 1, the probability of the origin slot being "Beijing" is 0.9, the probability of the origin slot being "Shanghai" is 0.1, and the probability of the destination slot being "Shanghai" is 0.9, and the probability of the destination slot being "Beijing" is 0.1. Therefore, in the semantic recognition result, the domain is "buying flight tickets," the intent is "searching for flight tickets," the time slot is "today," the origin slot is "Beijing," and the destination slot is "Shanghai."

[0095] By obtaining the domain probability distribution, intent probability distribution, and slot probability distribution of each word segment of the target speech command, the semantic recognition result of the target speech command can be obtained. This ensures the accuracy of the semantic recognition result, its domain, the intent it contains, and the slot of each word segment.

[0096] like Figure 2 The diagram illustrates the processing of target speech commands by the natural language understanding model in this embodiment of the present disclosure.

[0097] First, an encoder obtains the first vector representation of the target speech command and the second vector representation of each word segment within the command. The second vector representation is then input into a clause splitting network to divide the command into clauses. When split into a single clause, the semantic understanding network directly performs semantic recognition. However, when split into multiple clauses, each clause is matched against a semantic template. If the confidence level is low, the clause is input into the semantic understanding network of a natural language understanding model for semantic recognition. Within the semantic understanding network, the first vector representation of the clause is input into the intent recognition layer to obtain the intent probability distribution; the first vector representation is also input into the domain recognition layer to obtain the domain probability distribution; the second vector representation of each word segment is concatenated with the domain probability distribution to obtain the third vector representation; and the third vector representation is input into the slot recognition layer to obtain the slot probability distribution for each word segment. After obtaining the intent probability distribution, domain probability distribution, and slot probability distribution, the semantic recognition result of the target speech command can be determined.

[0098] To facilitate understanding of the semantic recognition method provided in the embodiments of this disclosure, such as Figure 3 The semantic recognition method is illustrated below.

[0099] Upon obtaining the target speech instruction, step S301 is executed first to match the target speech instruction with a semantic template. If the first confidence level of the match between the target speech instruction and the optimal semantic template is greater than a confidence threshold, a semantic recognition result is obtained based on the optimal semantic template. This semantic recognition result may be a single-intent semantic recognition result or a multi-intent semantic recognition result. If the first confidence level of the match between the target speech instruction and the optimal semantic template is less than or equal to the confidence threshold, step S302 is executed to input the target speech instruction into the natural language understanding model, and step S303 is executed to determine whether the target speech instruction includes multiple intents (i.e., multiple clauses). If the target speech instruction is determined to contain multiple intents, clauses are split, and each clause is matched with the semantic template using step S301, followed by subsequent processing steps. If the target speech instruction is determined to contain a single intent, the semantic understanding network of the natural language understanding model is used to obtain the semantic recognition result.

[0100] For example, in autonomous driving, if the user's target voice command is "There is a school ahead, please slow down," it is first matched against multiple semantic templates built based on rules. The first confidence score of the match between the target voice command and the optimal semantic template is obtained. The first confidence score for the match with the optimal semantic template "reduce speed" is 0.3, while the confidence threshold is 0.5. This is then input into a natural language understanding model. A clause splitting network is used to determine that the target voice command consists of two clauses: "There is a school ahead" and "please slow down." Semantic recognition is then performed on each of these clauses separately. Specifically, these two clauses are matched against multiple semantic templates built based on rules. The first confidence score of the match between "please slow down" and the optimal semantic template "reduce speed" is 0.7, which is greater than the confidence threshold. Therefore, the semantic recognition result is obtained based on "reduce speed." The first confidence score of "there is a school ahead" matching the best semantic template is 0.1, which is less than the confidence threshold. Therefore, it is input into the natural language understanding model. Since it only has one clause, the semantic recognition result obtained by the semantic understanding network is an unintentional statement. Thus, the semantic recognition result of the target voice command is "reduce speed".

[0101] At the same time, such as Figure 2 As shown, the natural language understanding model in this embodiment has a relatively simple structure. Since the results of the intent recognition layer, domain recognition layer, slot recognition layer, and clause splitting network are all probability distributions, each can be implemented using a fully connected layer, thus reducing the model size. Furthermore, model compression techniques can be used to further reduce the model size. With a small model size, limited resources on the device can support multiple models. For example, the user's target voice command "There is a school ahead, please slow down" can be split into two clauses. Two identical natural language understanding models can then be used in parallel to perform semantic recognition for "There is a school ahead" and "please slow down," further improving the speed of semantic recognition.

[0102] Based on the same technical concept, this disclosure also provides a method for training a natural language understanding model, which is used to perform semantic recognition on a target speech instruction when the first confidence level of the optimal semantic template matching between the target speech instruction and the target speech instruction is less than a confidence threshold. Figure 4 As shown, the method includes:

[0103] S401. Obtain the sample speech commands and the annotation results of the sample speech commands.

[0104] The annotation results include the semantics of the sample speech commands.

[0105] S402. Input the sample voice command into the model to be trained to obtain the semantic recognition result of the sample voice command.

[0106] S403. Determine the model loss based on the semantic recognition results and annotation results.

[0107] S404. Adjust the parameters of the model to be trained based on the model loss. If the model to be trained meets the convergence condition, a natural language understanding model is obtained.

[0108] By combining semantic templates and natural language understanding models, the semantic recognition method provided in this disclosure does not require a large number of semantic templates, thus solving the problem of cumbersome configuration when using semantic templates alone. Furthermore, since semantic templates cannot achieve high confidence in semantic recognition, the use of natural language understanding models can improve the generalization ability of semantic recognition. In addition, performing semantic recognition using speech templates before the natural language understanding model can significantly improve the recognition speed for certain speech commands compared to using a neural network model alone.

[0109] This disclosure does not specifically limit the sample speech commands. However, in some embodiments, the sample speech commands can be speech commands with a first confidence level of less than a confidence threshold that match the optimal semantic template. This can make the natural language understanding model obtained after training more suitable for the combined use of semantic templates and natural language understanding models, thereby improving the speed and accuracy of semantic recognition.

[0110] In some embodiments, the training task of the model to be trained includes, in addition to, the task of segmenting sample speech instructions into clauses. Correspondingly, the annotation result also includes clauses representing the intentions of each sample speech instruction. The model to be trained in this embodiment includes a clause segmentation network. Accordingly, inputting sample speech instructions into the model to be trained to obtain the semantic recognition result of the sample speech instructions can be implemented as follows:

[0111] Step C1: Use the clause splitting network in the model to be trained to identify clauses in the sample speech instructions and obtain the clause splitting results of the sample speech instructions.

[0112] Step C2: When the sample speech command is split into multiple clauses, determine the semantic recognition result of the sample speech command based on the semantic recognition results of the multiple clauses.

[0113] By breaking down sample speech commands into clauses, the semantic recognition results of each clause can improve the model's ability to recognize multi-intent speech commands, thereby enhancing its semantic recognition capabilities for speech commands.

[0114] In performing semantic recognition of multiple clauses, the semantic understanding network in the model to be trained can be used to perform semantic recognition on multiple clauses. Of course, to improve the versatility of the semantic recognition method provided in this embodiment of the natural language understanding model, it can also be matched with a semantic template and semantic recognition performed. Specifically, for each clause, the implementation can be as follows:

[0115] Step C21: Match the clause with multiple semantic templates established based on rules to obtain the second confidence score of the clause matching the optimal semantic template of the clause.

[0116] Step C22: If the second confidence level is higher than the confidence level threshold, obtain the semantic recognition result of the clause based on the optimal semantic template.

[0117] Step C23: If the second confidence level is less than or equal to the confidence threshold, input the clause into the semantic understanding network of the model to be trained to obtain the semantic recognition result of the clause.

[0118] Using joint semantic templates during training makes the resulting natural language understanding model more suitable for situations where semantic templates and natural language understanding models are used together, thereby improving the speed and accuracy of semantic recognition. At the same time, targeted training also improves the training speed of the model.

[0119] Step C3: When the sample speech instruction is split into a clause, obtain the semantic recognition result of the sample speech instruction output by the semantic understanding network in the model to be trained.

[0120] Of course, in order to make the natural language understanding model obtained after training more suitable for the combined use of semantic templates and natural language understanding models, a clause split from the sample speech instruction can also be matched with the semantic template and semantic recognition can be performed.

[0121] When a sample speech instruction is split into a clause, using a model to perform semantic recognition of the sample speech instruction can avoid repeated steps such as semantic template matching, reduce resource consumption, and improve the speed of semantic recognition of sample speech instructions as well as the training speed of the model.

[0122] In some embodiments, the training task of the model to be trained includes a semantic recognition task. For the semantic recognition task, the model to be trained includes a semantic understanding network, which includes an intent recognition layer, a domain recognition layer, and a slot recognition layer. Inputting sample speech commands into the model to be trained yields the semantic recognition results of the sample speech commands, which can be implemented as follows:

[0123] Step D1: Input the sample speech command into the encoder of the natural language understanding model to obtain the fourth vector representation of the sample speech command and the fifth vector representation of each word segment in the sample speech command.

[0124] Step D2: Input the fourth vector into the intent recognition layer to obtain the intent probability distribution of the sample voice command; and input the fourth vector into the domain recognition layer to obtain the domain probability distribution of the sample voice command.

[0125] Step D3: Concatenate the fifth vector representation of each word segment with the domain probability distribution to obtain the sixth vector representation.

[0126] Step D4: Input the sixth vector representation into the slot recognition layer to obtain the slot probability distribution of each word segment.

[0127] For the splicing process, please refer to the introduction in step B1, which will not be repeated here.

[0128] In the above method, the domain probability distribution and intent probability distribution of the sample voice command can be obtained through the fourth vector representation of the sample voice command; concatenating the domain probability distribution with the fifth vector representation can improve the accuracy of slot recognition.

[0129] In some embodiments, the training task of the model to be trained further includes a mapping task, which is used to map the intent recognition result in the semantic recognition result of the sample speech command to the domain recognition result in the semantic recognition result.

[0130] In semantic recognition, a domain generally corresponds to an intent. For example, the intent "open the car window" corresponds to the domain of vehicle control. Therefore, mapping tasks can improve semantic recognition capabilities in business scenarios. Taking one domain corresponding to one intent as an example, a linear mapping method can be used to map the intent probability distribution to the domain probability distribution, as shown in Equation 1:

[0131] p ′ (x D)=W*p(x i )+b(Formula 1)

[0132] Where W is the weight, b is the bias, and p ′ (x D Let p(x) be the neighborhood probability distribution obtained by mapping. i ) represents the probability distribution of intent.

[0133] Of course, when there are multiple domains or multiple intentions, a non-linear mapping method can also be used to map the intention probability distribution to the domain probability distribution.

[0134] In this embodiment, multi-task joint learning training is employed to enable the resulting natural language understanding model to accurately understand the semantics of voice commands. For example, the clause segmentation task helps improve the model's ability to understand multi-intent voice commands by breaking them down into simple clauses. The mapping task can enhance semantic recognition capabilities in relevant business scenarios.

[0135] In this embodiment of the disclosure, the model loss corresponding to the semantic recognition task includes:

[0136] The first recognition loss for the domain recognition results of the sample voice commands;

[0137] The second recognition loss for the intent recognition result of the sample voice commands;

[0138] The third recognition loss of the slot recognition results for sample voice commands.

[0139] To facilitate understanding of the training of natural language understanding models, the following will combine... Figure 5 The training process for a natural language understanding model is explained.

[0140] First, the encoder obtains the fourth vector representation of the sample speech command and the fifth vector representation of each word segment within the sample speech command. A clause splitting network is used to split the sample speech command into clauses, and the splitting loss is determined. When split into a single clause, a semantic understanding network is used directly for semantic recognition. When split into multiple clauses, each clause is matched with a semantic template for semantic recognition. In the semantic understanding network, the fourth vector representation is input to the domain recognition layer to obtain the domain recognition result, and the first recognition loss is determined. The fourth vector representation is input to the intent recognition layer to obtain the intent recognition result, and the second recognition loss is determined. The fifth vector representation of each word segment is concatenated with the domain recognition result to obtain the sixth vector representation. The sixth vector representation is input to the slot recognition layer to obtain the slot probability distribution of each word segment, and the third recognition loss is determined. Simultaneously, the intent recognition result is mapped to the domain recognition result, and the similarity loss between the domain recognition result obtained from the mapping task and the domain recognition result obtained from the semantic recognition task is determined. After obtaining the splitting loss, similarity loss, first recognition loss, second recognition loss, and third recognition loss, the model loss is calculated and the parameters of the model to be trained are adjusted based on the model loss. When the model to be trained meets the convergence condition, the natural language understanding model is obtained.

[0141] Meanwhile, in the natural language understanding model of this disclosure, the clause splitting network and the semantic understanding network share the output of the same encoder. When performing mapping tasks, the domain recognition result and the intent recognition result also use the same fourth vector representation. Therefore, parameter sharing is achieved in the natural language understanding model, reducing resource consumption and improving the training speed of the model.

[0142] In some embodiments, where the training task includes a mapping task, the model loss includes a similarity loss between the domain recognition result obtained from the mapping task and the domain recognition result obtained from the semantic recognition task, as shown in Equation 2:

[0143] Loss Simularity =1-cos(p(x) D ),p′(x D ))(Formula 2),

[0144] Loss Simularity For similarity loss, p(x) D ) represents the probability distribution of the neighborhood, p ′ (x D ) represents the domain probability distribution obtained through mapping. Of course, this disclosure does not limit the method for determining the similarity loss.

[0145] By comparing the domain recognition results obtained from the mapping task with the domain recognition results obtained from the semantic recognition task, it can be determined whether the domain recognition results and intent recognition results of the semantic recognition results of the model to be trained correspond to those of the sample voice commands, thereby improving the semantic recognition capability in business scenarios.

[0146] In some embodiments, where the training task includes a clause splitting task, the model loss includes the splitting loss for the clause splitting task of the sample speech instructions. The splitting loss can be calculated using cross-entropy loss, as shown in Equation 3:

[0147] Loss SubQuery =-∑ i ∑ k p(x ki logargmax k∈K q(x ki )(Formula 3)

[0148] Loss SubQuery For the splitting loss, k is the clause label, i is the i-th word segmentation, and x ki This indicates that i belongs to clause k, p(x) ki ) represents x ki The probability distribution is given by argmax, where argmax represents finding the probability distribution p(x). ki Let K be the set of clause tags, and q(x) be the tag with the highest probability of each segmentation. ki The ) represents the annotation result of each intention clause of the sample speech command. Of course, the embodiments of this disclosure do not limit the method for determining the splitting loss.

[0149] Using clause splitting loss to determine model loss enables the trained natural language understanding model to accurately split sample speech commands into clauses, thereby improving the model's ability to recognize multi-intent speech commands.

[0150] In some embodiments, when identifying slots, step D4 is used to input the sixth vector representation into the slot identification layer to obtain the slot probability distribution of each word segment. At this time, when calculating the third identification loss, the loss can be determined using CRF (Conditional Random Field).

[0151] CRF is a scoring method that can score labeled sequences. For example, for two semantic recognition results, "I play basketball today" and "I play today's basketball", although the word segmentation is the same, the first semantic result has a higher CRF score than the second semantic recognition result because the sequence of the first one is more reasonable.

[0152] In this embodiment of the disclosure, Formula 4 can be used to calculate the first recognition loss, Formula 5 to calculate the second recognition loss, and Formula 6 to calculate the third recognition loss, as detailed below:

[0153]

[0154] Loss Domain For the first recognition loss, q(x) D p(x) represents the neighborhood in the annotation results. D The domain is the domain obtained through domain identification.

[0155]

[0156] Loss Intent For the second recognition loss, q(x) I p(x) represents the intent in the annotation results. I The intent is obtained from intent recognition.

[0157]

[0158] Loss Slot For the third recognition loss, Score forward This is the output of CRF, Score. gold For the CRF score of the slot in the labeling results, num tokens This represents the number of word segments.

[0159] By obtaining the first recognition loss, the second recognition loss, and the third recognition loss, the semantic recognition ability of the natural language understanding model for speech commands can be improved, enabling the natural language understanding model to perform semantic recognition of speech commands more accurately. By mapping the domain recognition results and concatenating them with the word segmentation vector representation to obtain the slot recognition results, the obtained slot recognition results can be made more accurate.

[0160] Since manually adjusting the weights of multiple tasks is difficult and costly, the homoscedastic uncertainty method is used to determine the loss weights for different training tasks in the model loss. Homoscedastic uncertainty, also known as task-dependent uncertainty, does not depend on the input data, nor is it the model output; instead, it is a constant that is the same for all input data, but a variable that differs for different tasks. The model loss is calculated as shown in Equation 7:

[0161]

[0162] σ1, σ2, σ3, and σ4 are the weights of the four training tasks of the natural language understanding model. These four weights need to be learned and adjusted so that the training does not favor any one task.

[0163] By determining weights using homoscedasticity uncertainty, it can be ensured that model training does not favor any particular task, and at the same time, there is no need to manually adjust the weights, thus reducing the resource consumption of model training.

[0164] In some embodiments, after the natural language understanding model is trained, its size can be reduced using model compression, enabling it to perform semantic recognition of speech commands on the edge and process speech commands in parallel. Model compression can employ optimization of the model structure, such as using matrix factorization to compress the model or performing grouped convolutions on convolutional layers. Model compression can also be achieved by quantizing or pruning the model. This disclosure does not limit the method of model compression.

[0165] Based on the same technical concept, this disclosure also provides a semantic recognition device, such as... Figure 6 As shown, it includes:

[0166] The first acquisition module 601 is used to acquire the target voice command;

[0167] The matching module 602 is used to match the target speech instruction with multiple semantic templates established based on rules, and obtain the first confidence score of the optimal semantic template matching between the target speech instruction and the target speech instruction.

[0168] The rule-based semantic recognition module 603 is used to obtain the semantic recognition result of the target speech instruction based on the optimal semantic template when the first confidence level is higher than the confidence level threshold.

[0169] The model semantic recognition module 604 is used to input the target speech command into the natural language understanding model and obtain the semantic recognition result of the target speech command when the first confidence level is less than or equal to the confidence level threshold.

[0170] In some embodiments, the model semantic recognition module includes:

[0171] The first splitting submodule is used to split the target speech instruction into clauses based on the clause splitting network in the natural language understanding model;

[0172] The clause processing submodule is used to perform the following steps for each clause when the target speech instruction is split into multiple clauses: treat the clause as a new target speech instruction and return to perform the step of matching the target speech instruction with the semantic template established based on rules, until the semantic recognition result of the clause is obtained;

[0173] The first result determination submodule is used to determine the semantic recognition result of the target speech command based on the semantic recognition results of multiple clauses.

[0174] In some embodiments, the first result determining submodule is further configured to:

[0175] When the target speech instruction is split into a clause, the semantic recognition result of the target speech instruction output by the semantic understanding network in the natural language understanding model is obtained.

[0176] In some embodiments, the model semantic recognition module includes:

[0177] The probability distribution determination submodule is used by the semantic understanding network based on the natural language understanding model to determine the domain probability distribution, intent probability distribution, and slot probability distribution of each word segment in the target speech instruction.

[0178] The second result determination submodule is used to determine the semantic recognition result of the target speech command based on the domain probability distribution, intent probability distribution, and slot probability distribution of each word segmentation.

[0179] In some embodiments, the semantic understanding network includes an intent recognition layer, a domain recognition layer, and a slot recognition layer, and a probability distribution determination submodule, for:

[0180] The target speech command is input into the encoder of the natural language understanding model to obtain the first vector representation of the target speech command and the second vector representation of each word segment in the target speech command;

[0181] The first vector is used to represent the input intent recognition layer, resulting in the intent probability distribution; and...

[0182] The first vector is used to represent the input to the domain recognition layer, thus obtaining the domain probability distribution;

[0183] The second vector representation of each word segment is concatenated with the domain probability distribution to obtain the third vector representation;

[0184] The third vector representation is input into the slot recognition layer to obtain the slot probability distribution of each word segment.

[0185] In some embodiments, the first splitting submodule is used for:

[0186] Obtain the second vector representation of each word segment in the target speech instruction output by the encoder of the natural language understanding model;

[0187] The second vector represents the input clause, which is then split into a network to obtain the clause probability distribution.

[0188] The target speech instruction is split into clauses based on the clause probability distribution.

[0189] In some embodiments, the rule-based semantic templates include at least one of the following:

[0190] Use semantic templates corresponding to voice commands whose frequency is higher than the frequency threshold;

[0191] Semantic templates corresponding to speech instructions of the target category whose sample size is below a certain threshold; samples of the target category are used to train the natural language understanding model.

[0192] A semantic template containing multiple intents for voice commands.

[0193] Based on the same technical concept, embodiments of this disclosure also provide a training device for a natural language understanding model, such as... Figure 7 As shown, it includes:

[0194] The second acquisition module 701 is used to acquire sample speech commands and the annotation results of sample speech commands;

[0195] The semantic recognition module 702 is used to input sample speech commands into the model to be trained and obtain the semantic recognition results of the sample speech commands.

[0196] The loss determination module 703 is used to determine the model loss based on the semantic recognition results and annotation results;

[0197] The adjustment module 704 is used to adjust the parameters of the model to be trained based on the model loss, so as to obtain a natural language understanding model when the model to be trained meets the convergence condition.

[0198] Among them, the natural language understanding model is used to perform semantic recognition of the target speech instruction when the first confidence level of the target speech instruction matching the optimal semantic template of the target speech instruction is less than the confidence level threshold.

[0199] In some embodiments, the training task of the model to be trained includes a semantic recognition task, and at least one of the following tasks: a clause splitting task for sample speech instructions and a mapping task, wherein the mapping task is used to map the intent recognition result in the semantic recognition result of the sample speech instructions to the domain recognition result obtained by the semantic recognition task.

[0200] In some embodiments, where the training task includes a clause splitting task, the model loss includes a splitting loss for the clause splitting task of the sample speech instructions.

[0201] In some embodiments, where the training task includes a mapping task, the model loss includes the similarity loss between the domain recognition results obtained from the mapping task and the domain recognition results obtained from the semantic recognition task.

[0202] In some embodiments, the loss in the model loss corresponding to the semantic recognition task includes:

[0203] The first recognition loss of the domain recognition result of the sample voice command;

[0204] The second recognition loss of the intent recognition result of the sample voice command;

[0205] The third recognition loss of the slot recognition results of sample voice commands.

[0206] In some embodiments, the semantic recognition module includes:

[0207] The second splitting submodule is used to split the sample speech instruction into clauses based on the clause splitting network in the model to be trained, and obtain the clause splitting result of the sample speech instruction;

[0208] The first semantic recognition submodule is used to determine the semantic recognition result of the sample voice command based on the semantic recognition results of each of the multiple clauses when the sample voice command is split into multiple clauses.

[0209] In some embodiments, the first semantic recognition submodule is configured to:

[0210] The clause is matched with multiple semantic templates based on rules to obtain the second confidence score of the optimal semantic template match between clauses;

[0211] When the second confidence level is higher than the confidence threshold, the semantic recognition result of the clause is obtained based on the optimal semantic template;

[0212] If the second confidence level is less than or equal to the confidence threshold, the clause is input into the semantic understanding network of the model to be trained to obtain the semantic recognition result of the clause.

[0213] In some embodiments, the semantic recognition module further includes:

[0214] The second semantic recognition submodule is used to obtain the semantic recognition result of the sample speech instruction output by the semantic understanding network in the model to be trained when the target speech instruction is split into a clause.

[0215] In some embodiments, the model loss uses a homoscedasticity uncertainty device to determine the loss weights corresponding to different training tasks.

[0216] In some embodiments, for a semantic recognition task, the model to be trained includes a semantic understanding network, which includes an intent recognition layer, a domain recognition layer, and a slot recognition layer. The semantic recognition module is used for:

[0217] The sample speech command is input into the encoder of the natural language understanding model to obtain the fourth vector representation of the sample speech command and the fifth vector representation of each word segment in the sample speech command.

[0218] The fourth vector is used to represent the input intent recognition layer, thus obtaining the intent probability distribution of the sample voice commands; and,

[0219] The fourth vector is used to represent the input domain recognition layer to obtain the domain probability distribution of the sample speech commands;

[0220] The fifth vector representation of each word segment is concatenated with the domain probability distribution to obtain the sixth vector representation;

[0221] The sixth vector is input into the slot recognition layer to obtain the slot probability distribution of each word segment.

[0222] The specific functions and examples of each module and submodule of the apparatus in this disclosure can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0223] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0224] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0225] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0226] like Figure 8 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0227] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0228] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as semantic recognition methods. For example, in some embodiments, the semantic recognition method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the semantic recognition method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform semantic recognition methods by any other suitable means (e.g., by means of firmware).

[0229] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0230] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0231] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0232] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0233] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0234] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0235] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0236] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A semantic recognition method, comprising: Obtain the target voice command; The target speech instruction is matched with multiple semantic templates established based on rules to obtain the first confidence score of the optimal semantic template matching between the target speech instruction and the target speech instruction. If the first confidence level is higher than the confidence threshold, the semantic recognition result of the target speech instruction is obtained based on the optimal semantic template; If the first confidence level is less than or equal to the confidence threshold, the target speech instruction is input into a natural language understanding model to obtain the semantic recognition result of the target speech instruction, including: The target speech instruction is input into the encoder of the natural language understanding model to obtain a first vector representation of the target speech instruction and a second vector representation of each word segment in the target speech instruction; The first vector representation is input into the intent recognition layer of the semantic understanding network of the natural language understanding model to obtain the intent probability distribution; and the first vector representation is input into the domain recognition layer of the semantic understanding network to obtain the domain probability distribution. The domain probability distribution is concatenated with the second vector representation of each word segment to obtain a third vector representation; the third vector representation is then input into the slot recognition layer of the semantic understanding network to obtain the slot probability distribution of each word segment. The semantic recognition result of the target voice command is determined based on the domain probability distribution, the intent probability distribution, and the slot probability distribution of each word segmentation. The training tasks of the natural language understanding model include a semantic recognition task and a mapping task; the mapping task is used to map the intent recognition result in the semantic recognition result of the sample speech command to the domain recognition result obtained by the semantic recognition task, and the mapping method includes linear mapping and nonlinear mapping.

2. The method according to claim 1, wherein, The step of inputting the target speech command into a natural language understanding model to obtain the semantic recognition result of the target speech command includes: The target speech instruction is split into clauses based on the clause segmentation network in the natural language understanding model; When the target speech instruction is split into multiple clauses, for each clause, the following steps are performed: the clause is treated as a new target speech instruction, and the process of matching the target speech instruction with the semantic template established based on rules is returned until the semantic recognition result of the clause is obtained. Based on the semantic recognition results of each of the multiple clauses, the semantic recognition result of the target speech instruction is determined.

3. The method according to claim 2, further comprising: When the target speech instruction is split into a clause, the semantic recognition result of the target speech instruction output by the semantic understanding network in the natural language understanding model is obtained.

4. The method according to claim 2, wherein, The clause segmentation network based on the natural language understanding model segments the target speech instruction into clauses, including: Obtain the second vector representation of each word segment in the target speech instruction output by the encoder of the natural language understanding model; The second vector representation of each word segment is input into the clause splitting network to obtain the clause probability distribution; The target speech instruction is split into clauses based on the clause probability distribution.

5. The method according to any one of claims 1-4, wherein, The rule-based semantic templates include at least one of the following: Use semantic templates corresponding to voice commands whose frequency is higher than the frequency threshold; The semantic templates corresponding to the speech instructions of the target category whose sample size is below a certain threshold are used to train the natural language understanding model. A semantic template containing multiple intents for voice commands.

6. A training method for a natural language understanding model, comprising: Obtain sample speech commands and the annotation results of the sample speech commands; The sample speech command is input into the model to be trained to obtain the semantic recognition result of the sample speech command; The model loss is determined based on the semantic recognition results and the annotation results; The parameters of the model to be trained are adjusted based on the model loss, and a natural language understanding model is obtained when the model to be trained satisfies the convergence condition. The natural language understanding model is used to perform semantic recognition on the target speech instruction when the first confidence level of matching the target speech instruction with the optimal semantic template of the target speech instruction is less than a confidence threshold. The training task of the model to be trained includes a semantic recognition task and a mapping task. The mapping task is used to map the intent recognition result in the semantic recognition result of the sample speech command to the domain recognition result obtained by the semantic recognition task. The mapping method includes linear mapping and nonlinear mapping.

7. The method according to claim 6, wherein the training task further includes a clause splitting task for the sample speech instructions and the mapping task.

8. The method according to claim 7, wherein, When the training task includes the clause splitting task, the model loss includes the splitting loss for the clause splitting task of the sample speech instructions.

9. The method according to claim 7 or 8, wherein, When the training task includes the mapping task, the model loss includes the similarity loss between the domain recognition result obtained from the mapping task and the domain recognition result obtained from the semantic recognition task.

10. The method according to claim 7 or 8, wherein, The model loss includes the loss corresponding to the semantic recognition task, which includes: The first recognition loss of the domain recognition result of the sample voice command; The second recognition loss of the intent recognition result of the sample voice command; The third recognition loss of the slot recognition result of the sample voice command.

11. The method according to claim 7 or 8, wherein, The step of inputting the sample speech command into the model to be trained to obtain the semantic recognition result of the sample speech command includes: The sample speech instruction is split into clauses based on the clause splitting network in the model to be trained, and the clause splitting result of the sample speech instruction is obtained. When the sample voice instruction is split into multiple clauses, the semantic recognition result of the sample voice instruction is determined based on the semantic recognition results of each of the multiple clauses.

12. The method according to claim 11, wherein, For each clause, determine the semantic recognition result of the clause, including: The clause is matched with multiple semantic templates established based on rules to obtain the second confidence score of the optimal semantic template matching of the clause; If the second confidence level is higher than the confidence level threshold, the semantic recognition result of the clause is obtained based on the optimal semantic template; If the second confidence level is less than or equal to the confidence threshold, the clause is input into the semantic understanding network of the model to be trained to obtain the semantic recognition result of the clause.

13. The method of claim 11, further comprising: When the target speech instruction is split into a clause, the semantic recognition result of the sample speech instruction output by the semantic understanding network in the model to be trained is obtained.

14. The method according to any one of claims 7, 8, 12 or 13, wherein the model loss employs the homoscedasticity uncertainty method to determine the loss weights corresponding to different training tasks.

15. The method according to any one of claims 7, 8, 12 or 13, wherein, For the semantic recognition task, the model to be trained includes a semantic understanding network, which includes an intent recognition layer, a domain recognition layer, and a slot recognition layer. The step of inputting the sample speech command into the model to be trained to obtain the semantic recognition result of the sample speech command includes: The sample speech command is input into the encoder of the natural language understanding model to obtain the fourth vector representation of the sample speech command and the fifth vector representation of each word segment in the sample speech command; The fourth vector representation is input into the intent recognition layer to obtain the intent probability distribution of the sample voice command; and, The fourth vector representation is input into the domain recognition layer to obtain the domain probability distribution of the sample voice command; The fifth vector representation of each word segment is concatenated with the domain probability distribution to obtain the sixth vector representation; The sixth vector representation is input into the slot recognition layer to obtain the slot probability distribution of each word segmentation.

16. A semantic recognition device, comprising: The first acquisition module is used to acquire the target voice command; The matching module is used to match the target speech instruction with multiple semantic templates established based on rules, and obtain a first confidence score of the optimal semantic template matching between the target speech instruction and the target speech instruction. The rule-based semantic recognition module is used to obtain the semantic recognition result of the target speech instruction based on the optimal semantic template when the first confidence level is higher than the confidence level threshold. The model semantic recognition module is used to input the target speech instruction into the natural language understanding model when the first confidence level is less than or equal to the confidence level threshold, so as to obtain the semantic recognition result of the target speech instruction; The model semantic recognition module is specifically used for: The target speech instruction is input into the encoder of the natural language understanding model to obtain a first vector representation of the target speech instruction and a second vector representation of each word segment in the target speech instruction; The first vector representation is input into the intent recognition layer of the semantic understanding network of the natural language understanding model to obtain the intent probability distribution; and the first vector representation is input into the domain recognition layer of the semantic understanding network to obtain the domain probability distribution. The domain probability distribution is concatenated with the second vector representation of each word segment to obtain a third vector representation; the third vector representation is then input into the slot recognition layer of the semantic understanding network to obtain the slot probability distribution of each word segment. The semantic recognition result of the target voice command is determined based on the domain probability distribution, the intent probability distribution, and the slot probability distribution of each word segmentation. The training tasks of the natural language understanding model include a semantic recognition task and a mapping task; the mapping task is used to map the intent recognition result in the semantic recognition result of the sample speech command to the domain recognition result obtained by the semantic recognition task, and the mapping method includes linear mapping and nonlinear mapping.

17. The apparatus according to claim 16, wherein, The model semantic recognition module includes: The first splitting submodule is used to split the target speech instruction into clauses based on the clause splitting network in the natural language understanding model; The clause processing submodule is used to, when the target speech instruction is split into multiple clauses, perform the following for each clause: treat the clause as a new target speech instruction and return to perform the step of matching the target speech instruction with a rule-based semantic template until the semantic recognition result of the clause is obtained; The first result determination submodule is used to determine the semantic recognition result of the target speech instruction based on the semantic recognition results of each of the plurality of clauses.

18. The apparatus of claim 17, wherein the first result determining submodule is further configured to: When the target speech instruction is split into a clause, the semantic recognition result of the target speech instruction output by the semantic understanding network in the natural language understanding model is obtained.

19. The apparatus according to claim 17, wherein, The first splitting submodule is used for: Obtain the second vector representation of each word segment in the target speech instruction output by the encoder of the natural language understanding model; The second vector representation is input into the clause splitting network to obtain the clause probability distribution; The target speech instruction is split into clauses based on the clause probability distribution.

20. The apparatus according to any one of claims 16-19, wherein, The rule-based semantic templates include at least one of the following: Use semantic templates corresponding to voice commands whose frequency is higher than the frequency threshold; The semantic templates corresponding to the speech instructions of the target category whose sample size is below a certain threshold are used to train the natural language understanding model. A semantic template containing multiple intents for voice commands.

21. A training device for a natural language understanding model, comprising: The second acquisition module is used to acquire sample voice commands and the annotation results of the sample voice commands; A semantic recognition module is used to input the sample speech command into the model to be trained and obtain the semantic recognition result of the sample speech command. The loss determination module is used to determine the model loss based on the semantic recognition result and the annotation result; An adjustment module is used to adjust the parameters of the model to be trained based on the model loss, and to obtain a natural language understanding model when the model to be trained meets the convergence condition. The natural language understanding model is used to perform semantic recognition on the target speech instruction when the first confidence level of matching the target speech instruction with the optimal semantic template of the target speech instruction is less than a confidence threshold. The training task of the model to be trained includes a semantic recognition task and a mapping task. The mapping task is used to map the intent recognition result in the semantic recognition result of the sample speech command to the domain recognition result obtained by the semantic recognition task. The mapping method includes linear mapping and nonlinear mapping.

22. The apparatus of claim 21, wherein the training task further includes a clause splitting task for the sample speech instructions and the mapping task.

23. The apparatus according to claim 22, wherein, When the training task includes the clause splitting task, the model loss includes the splitting loss for the clause splitting task of the sample speech instructions.

24. The apparatus according to claim 22 or 23, wherein, When the training task includes the mapping task, the model loss includes the similarity loss between the domain recognition result obtained from the mapping task and the domain recognition result obtained from the semantic recognition task.

25. The apparatus according to claim 22 or 23, wherein, The model loss includes the loss corresponding to the semantic recognition task, which includes: The first recognition loss of the domain recognition result of the sample voice command; The second recognition loss of the intent recognition result of the sample voice command; The third recognition loss of the slot recognition result of the sample voice command.

26. The apparatus according to claim 22 or 23, wherein, The semantic recognition module includes: The second splitting submodule is used to split the sample speech instruction into clauses based on the clause splitting network in the model to be trained, so as to obtain the clause splitting result of the sample speech instruction; The first semantic recognition submodule is used to determine the semantic recognition result of the sample voice instruction based on the semantic recognition result of each of the multiple clauses when the sample voice instruction is split into multiple clauses.

27. The apparatus according to claim 26, wherein, The first semantic recognition submodule is used for: The clause is matched with multiple semantic templates established based on rules to obtain the second confidence score of the optimal semantic template matching of the clause; If the second confidence level is higher than the confidence level threshold, the semantic recognition result of the clause is obtained based on the optimal semantic template; If the second confidence level is less than or equal to the confidence threshold, the clause is input into the semantic understanding network of the model to be trained to obtain the semantic recognition result of the clause.

28. The apparatus according to claim 26, wherein the semantic recognition module further comprises: The second semantic recognition submodule is used to obtain the semantic recognition result of the sample speech instruction output by the semantic understanding network in the model to be trained when the target speech instruction is split into a clause.

29. The apparatus according to any one of claims 22, 23, 27 or 28, wherein the model loss employs a homoscedastic uncertainty device to determine the loss weights corresponding to different training tasks.

30. The apparatus according to any one of claims 22, 23, 27 or 28, wherein, For the semantic recognition task, the model to be trained includes a semantic understanding network, which includes an intent recognition layer, a domain recognition layer, and a slot recognition layer. The semantic recognition module is used for: The sample speech command is input into the encoder of the natural language understanding model to obtain the fourth vector representation of the sample speech command and the fifth vector representation of each word segment in the sample speech command; The fourth vector representation is input into the intent recognition layer to obtain the intent probability distribution of the sample voice command; and, The fourth vector representation is input into the domain recognition layer to obtain the domain probability distribution of the sample voice command; The fifth vector representation of each word segment is concatenated with the domain probability distribution to obtain the sixth vector representation; The sixth vector representation is input into the slot recognition layer to obtain the slot probability distribution of each word segmentation.

31. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-15.

32. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-15.

33. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-15.

34. A vehicle carrying an electronic device as claimed in claim 31.

Images