Method for constructing training corpus of dialogue system, training method and computing device

Abstract

The present disclosure relates to a method for constructing a training corpus of a dialogue system, a training method and a computing device. The method for constructing a training corpus of a dialogue system comprises: obtaining an intent system of a device to which the dialogue system is to be applied; obtaining a sample to be labeled; generating an intent of the sample to be labeled based on the intent system by using a language model; and labeling the sample to be labeled with the intent to generate a training corpus of the dialogue system. The method uses the semantic understanding capability of the language model to automatically generate the intent, and at the same time uses the intent system of the device to constrain the intent generation result of the language model, so as to easily and quickly construct the training corpus and improve the accuracy of labeling of the training corpus.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of artificial intelligence, and in particular, to a dialogue system. BACKGROUND

[0002] A dialogue system is an artificial intelligence system that simulates human and aims to form a coherent and smooth dialogue / interaction with human, for example, can understand the meaning of a user input voice and complete a corresponding task and / or output a corresponding answer voice, etc. In some dialogue systems, a large number of samples need to be annotated for the purpose of intent recognition and / or slot extraction, etc., so as to form a training corpus of the dialogue system, and the annotation of the training corpus of the dialogue system is the data basis for the construction of the system model. SUMMARY

[0003] The embodiments of the present disclosure provide a method for constructing a training corpus of a dialogue system, a training method, and corresponding computing devices and non-transitory machine-readable storage media for executing the methods.

[0004] According to a first aspect of the embodiments of the present disclosure, a method for constructing a training corpus of a dialogue system is provided, comprising: obtaining an intent system of a device to which the dialogue system is to be applied; obtaining a sample to be annotated; generating an intent of the sample to be annotated based on the intent system by using a language model; and annotating the sample to be annotated with the intent to generate a training corpus of the dialogue system.

[0005] Optionally, the intent system is coded in a prefix tree data structure, wherein each path of the prefix tree forms an intent in the intent system, and generating the intent of the sample to be annotated based on the intent system by using the language model comprises: generating the intent by the language model in stages, and at each stage, using the prefix tree to constrain the part of the intent to be generated to ensure that the part to be generated currently belongs to one of the parts stored on the path from the last prefix tree node to which the previously generated prefix part belongs to the next child node.

[0006] Optionally, one character in each of the intents in the intent system is stored on each path of the prefix tree leading to a child node, and the language model is used to generate the intent character by character, and at each character generation stage, the prefix tree of the intent system is used to constrain the character to be generated to ensure that the character to be generated currently belongs to one of the characters stored on the path from the last prefix tree node to which the previously generated prefix part belongs to the next child node.

[0007] Optionally, the generating the intent of the sample to be annotated comprises: concatenating the sample to be annotated with an intent prompt, inputting the concatenated sample to be annotated into the language model to generate the intent.

[0008] Optionally, after the generating the intent of the sample to be annotated, the method further comprises: obtaining an intent-annotated sample after the sample to be annotated is annotated with the intent; concatenating the intent-annotated sample with a slot prompt adapted to the intent; inputting the concatenated intent-annotated sample into the language model to generate a slot; and annotating the concatenated intent-annotated sample with the slot, to generate the training corpus of the dialog system.

[0009] Optionally, the slot is any segment in the sample to be annotated.

[0010] Optionally, the language model utilizes a beam search algorithm to generate the intent of the sample to be annotated.

[0011] According to a second aspect of the embodiments of the present disclosure, a method for constructing a training corpus of a dialog system is provided, comprising: obtaining an intent system of a plurality of devices to which the dialog system is to be applied; obtaining a sample to be annotated; generating, by using a language model, an intent in each of the plurality of devices for the sample to be annotated based on an intent system of each of the plurality of devices; and annotating the sample to be annotated with the intent in each of the plurality of devices respectively, to generate a training corpus of the dialog system.

[0012] According to a third aspect of the embodiments of the present disclosure, a training method of a dialog system is provided, comprising: determining a device to which the dialog system is to be applied; training the dialog system based on the training corpus of the dialog system generated according to any of the first aspect to the second aspect until the dialog system meets a stop condition of training.

[0013] According to a fourth aspect of the embodiments of the present disclosure, a computing device is provided, comprising: a processor; and a memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method according to any of the first aspect to the third aspect.

[0014] According to a fifth aspect of the embodiments of the present disclosure, a non-transitory machine-readable storage medium is provided, having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method according to any of the first aspect to the third aspect. BRIEF DESCRIPTION OF DRAWINGS

[0015] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which like reference characters refer to like parts throughout the figures, and in which:

[0016] Figure 1 A schematic flowchart of a method of constructing a training corpus for a dialog system is shown, in accordance with at least one embodiment of the present disclosure.

[0017] Figure 2A And Figure 2B Prefix tree data structure diagrams of an intent schema of a device applying a dialog system are exemplarily shown, in accordance with at least one embodiment of the present disclosure.

[0018] Figure 3 A schematic flowchart of a method of constructing a training corpus for a dialog system applicable to multiple devices is exemplarily shown, in accordance with at least one embodiment of the present disclosure.

[0019] Figure 4 A schematic flowchart of a training method of a dialog system is shown, in accordance with at least one embodiment of the present disclosure.

[0020] Figure 5 A structural schematic diagram of a computing device is shown, in accordance with at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] Preferred embodiments of the present disclosure will be described herein below with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0022] In some cases, a dialog system is to be applied to various devices such as smart devices, e.g., to provide the devices with support for user voice control and / or chat. Typically, a dialog system is to analyze and understand user input dialog, e.g., to perform intent recognition and / or slot extraction operations. Intent recognition refers to determining a type of intent of a user dialog in a dialog system, while slot extraction refers to extracting a piece of key information in an intent of a user dialog, which is typically an entity, e.g., a destination name in a navigation intent. Therefore, typically, a model of a dialog system is to be provided with a considerable amount of training samples that are annotated with intents and slots, which at least cover all intents in an intent schema of a single device to which the system is to be applied. In this context, the expression "intent schema of a device" refers to a set of all intents that the device can involve. However, currently, the annotation of samples with intents and slots is mostly performed in a manual or semi-automatic manner, which is very labor-intensive and, in some cases, difficult to annotate and the accuracy of the annotation cannot be guaranteed.

[0023] Thus, embodiments of the present application propose an improved method of constructing training corpus of a dialog system, which utilizes semantic understanding capability of a language model to automatically generate intents, and meanwhile, constrains the intent generation results of the language model with an intent schema of a device, so as to both easily and quickly construct training corpus and improve the accuracy of the annotation of training corpus.

[0024] At least some embodiments of the solutions of the present disclosure will be described in detail below with reference to the drawings. Figures 1-3

[0025] A schematic flowchart of a method of constructing training corpus of a dialog system according to at least one embodiment of the present disclosure is shown. Figure 1 As shown in

[0026] Step S110, an intent schema of a device to which a dialog system is to be applied is obtained. Figure 1 In some embodiments, the dialog system can be applied to one or more devices, which can include various smart devices such as a mobile phone, a tablet computer, a notebook computer, a smart watch, smart glasses (e.g., AR glasses, VR glasses and MR glasses), a car machine, etc. Each of the various devices to which the dialog system is to be applied can be provided with a respective corresponding intent schema, which covers all intents that the device can involve, e.g., a mobile phone is provided with one intent schema, smart glasses are provided with another intent schema, and a car machine is provided with a third intent schema. Alternatively, in some cases, all or part of the devices to which the dialog system is to be applied can be provided with one general intent schema, which covers all intents that the all or part of the devices can involve.

[0027]

[0028] ​The intent system of the device can be obtained in various ways. For example, in some embodiments, the various intents that the dialog system can implement on the device can be designed by business personnel as needed, and then all the intents can be organized into an intent system in various suitable ways. For example, all the intents of the device can be stored together in various forms of data structures (such as arrays, linked lists, stacks, queues, or prefix tree structures to be described later, etc.) to form an intent system, so as to facilitate searching, etc. The following Table 1 gives an example of 7 intents of an AR glasses device for reference. Those skilled in the art understand that the intents of the AR glasses device are not limited to this. Please note that the "intent prompt", "slot", "slot prompt", etc. corresponding to each intent in Table 1 will be discussed in detail later.

[0029]

[0030]

[0031] Table 1

[0032] Then, in step S120, the sample to be labeled is obtained. For example, the above-mentioned sample to be labeled can be a dialog input by the user before, and the sample to be labeled can be obtained in various ways such as collecting historical dialog information of the dialog system, etc.

[0033] Then, in step S130, the intent of the sample to be labeled is generated based on the intent system by using the language model.

[0034] For example, the language model described above can be obtained by fine-tuning a pre-trained language model, which can be obtained by self-supervised training using unlabeled corpus, or the language model can be a neural network model trained using labeled corpus, which can include various neural network models such as single-modal network models or cross-modal network models, for example, BERT model using encoding structure in transformer, GPT model using decoding structure, and various variants thereof. It can be understood that the structure and training method of the neural network model described above do not limit the disclosure. The language model can be used to construct corpus conforming to common sense specifications in a supervised and / or unsupervised manner, and the generation speed (including intent generation and subsequent slot generation to be described in detail) far exceeds the speed of manual annotation. Moreover, according to actual effect statistics, the annotation results generated by the model can basically meet the requirements. In the present application, in order to further improve the accuracy of the model-generated results, a restricted generation method of the model is proposed, that is, the intent generation results of the language model are restricted by the intent system of the corresponding device to ensure that the generated intent belongs to the intent system. Therefore, the method proposed in the disclosure can save labor costs, improve the construction efficiency of the training corpus of the dialogue system, ensure the accuracy of the annotation of the training corpus, and even in some cases, the accuracy of the annotation can be improved compared to the manual annotation method, as will be described in detail later. In addition, the method proposed in the disclosure supports flexible addition, deletion and modification of the intent system (for example, when the device intent is changed, the business personnel only need to change the intent system data accordingly and let the model generate the annotation accordingly, without manually modifying the annotation of the training corpus one by one), which reduces the maintenance and operation costs.

[0035] Various methods can be used to restrict the generation of the intent of the sample to be annotated.

[0036] For example, in some embodiments, the prompt learning capability of the language model can be used to generate the intent. In natural language processing technology, "prompt" refers to directly completing the downstream machine learning task by using the constructed prompt sentence to the language model. For example, in the example shown in Table 1 above, the intent prompt ( "The intent of this sentence belongs to ____ for [AR glasses]") can be spliced to the sample to be annotated (user input dialogue) for the AR glasses device as shown in Table 1, and the spliced sample to be annotated is input into the language model. Then the language model fills in the blank part of the prompt sentence, that is, the intent of the sample to be annotated is generated. The process will be described in more detail below in conjunction with specific sample to be annotated examples.

[0037] In some embodiments, the generated intent can be restricted to the intent schema provided in step S110 after the language model generates a complete intent result. For example, the generated intent is compared with all the intents in the intent schema, and the intent that is closest to the generated intent is found as the final output intent result.

[0038] Alternatively, in some embodiments, the generated result of the language model can be restricted during the process of generating the intent to ensure that the final generated intent is in the intent schema. For example, in some embodiments, the language model generates the intent in stages (e.g., timestep by timestep), such as generating a part of the intent, e.g., a character, a word, or several characters, etc., in each stage. In this case, in some embodiments, the intent schema can be encoded in a prefix tree data structure, in which each path of the prefix tree forms an intent in the intent schema. For example, at least a part of each intent in the intent schema is stored on each path leading to a child node of the prefix tree, and the prefix tree of the intent schema is used to constrain the part of the intent to be generated at each generation stage of the language model to ensure that the part to be generated belongs to one of the parts stored on the path leading from the last prefix tree node of the previously generated prefix part to the next child node. In the scheme of the present disclosure, the expression "child node" refers to the nodes of the direct descendants (i.e., the first generation of descendants) and does not include the nodes of the indirect descendants (i.e., the descendants of the descendants, the second generation or more generations after the first generation), and the expression "each path leading to a child node" refers to the path of a node leading to its next (i.e., each of the next generation) child node, but not to the next next (i.e., the second generation or more generations) child node. The part of the intent stored on the path of the prefix tree described above can be a character, a word, or several characters, etc. The case where a character is stored on the path will be described in detail below.

[0039] A prefix tree is a special form of N-ary tree. Generally, a prefix tree is used to store strings. Each node of the prefix tree represents a string (prefix). Each node has multiple child nodes, and the paths leading to different child nodes have different characters. The string represented by a child node is composed of the prefix string of the parent node itself and all the characters on the path leading to the child node. An important property of the prefix tree is that all the descendants of a node have a common prefix of the string associated with the node.

[0040] In embodiments of the present disclosure, all the intents (e.g., n, n > 1) of a device can be uniformly encoded into an intent schema PreTreeIntent of a prefix tree structure, which is a set of strings W 1～n representing each intent, and each intent W 1～nIt is a textual description of intent, and in this prefix tree, the intent string W 1～n The single character w in i This represents the i-th character in the intent string, where each character w i All of them are preceded by the character w i-1 Next(w) is the set of suffixes of the nodes that the path leads to. i-1 (that is, the set of characters on all paths leading to its child nodes), or in other words, each character w i All were preceded by the character w i-1 Specifically, refer to the PreTreeIntent intent system, which uses a simple programming language to describe the prefix tree structure as follows:

[0041] PreTreeIntent = Collection(W 1～n (w i ∈Next(w i-1 )))

[0042] Where Collect represents the string W of all intents. 1～n (That is, W1, W2, ..., W) n The set of ); Next represents the previous character w i-1 The set of suffixes of the nodes to which the path leads, i.e., the set of characters on all paths leading to its child nodes; the value of i ranges from 1 to the total character length of each intent string, which is relative to each W. 1～n The values ​​are not necessarily the same; when i is 1, w0 does not represent a character in the intended string, but rather the root node, which is usually empty. w1 is one of the characters in the set of all paths from the root node to its child nodes, which is the first character in each string.

[0043] To make it easier to understand, Figure 2A An example of an intent system prefix tree consisting of seven intent examples of the AR glasses device in Table 1 above is given.

[0044] Figure 2A The values ​​marked in each node shown are the strings (prefix or the entire string) that the node represents. For example, the root node represents an empty string, and the terminal leaf nodes 1-7 represent the complete strings of the seven intentions in Table 1. Note that for clarity, the strings represented by many nodes are not explicitly labeled. Figure 2A Instead of using ellipses, these nodes are represented by prefix trees. However, based on the aforementioned principles of prefix trees, those skilled in the art can understand the content of these nodes.

[0045] from Figure 2AThe root node starts with only one child node, and the path to this child node has "A" on it, so the first child node represents the string "A". Following this pattern, the fourth child node represents the string "AR glasses" when it starts with two child nodes. If the leftmost path is always chosen, then eventually the leaf node 1 is reached, which represents the intent "AR glasses - application control - open [music player]" from Table 1 (note that the symbols "-" and "[]" are added for ease of viewing and do not mean that these characters are actually present in the string). Thus, the corresponding strings represented by the other six leaf nodes 2-7 are obtained by following the paths to these nodes in order and concatenating the characters on these paths, which are the other six intents from Table 1. Of course, Figure 2A are merely exemplary and not limiting, for example Figure 2A The prefix content of "AR glasses" in the prefix tree can also be removed from the prefix tree and other ways can be used to limit or indicate that the prefix tree belongs to the intent system of AR glasses. Alternatively, the intent system of other devices can also be compiled into Figure 2A the same prefix tree as AR glasses.

[0046] Of course, as mentioned above, the present disclosure is not limited to the prefix tree structure shown in Figure 2A For example, the part of the intent stored on each path is not limited to one character, but can be any number of characters, as shown in Figure 2B Figure 2B The intent system represented by the prefix tree of Figure 2A is the same as that of Figure 2B For example, as shown, the path from the root node to its child node has "AR glasses" on it, so the first child node represents the string "AR glasses"; following this pattern, the seven leaf nodes 1-7 are reached, which respectively represent the seven intents from Table 1.

[0047] In some embodiments, the language model generates intents character by character, so the prefix tree as described above can be easily used to constrain the intent generation process of the language model to ensure that the final generated intent belongs to the intent system. For example, when the language model generates the current character, the part generated before it must be the prefix part represented by a node of the prefix tree, so the current character only needs to be searched in the suffix set of the node (i.e., the set of characters on the paths to its child nodes). For example, according to Figure 2A ​If the string "AR glasses - app control - sound" (the symbol "-" is just for easy viewing and does not represent a real character in the intent string) has been generated before, the current character to be generated can only be selected from the suffix set ["fun", "quantity"]. The present disclosure does not limit the way of selecting the current character to be generated from the suffix set.

[0048] For example, in some embodiments, the sample to be annotated Q (in some cases concatenated with the intent prompt sentence) can be input into a language model based on GPT3, for example, and then the above-mentioned restricted generation method based on the intent system prefix tree PreTreeIntent is used to generate the optimal reply A for Q max As follows:

[0049]

[0050] A Max = Max(P(A | PreTreeIntent, Q)), A Max ∈ PreTreeIntent

[0051] Where the language model generates one character A at a time t , A <t represents the part that has been generated before, and in the case of using the prefix tree PreTreeIntent to constrain the generation, that is, the A t is constrained in the suffix set Next(A t-1 ) of the node to which the path of the previous character A t-1 leads, A t can be obtained under the condition of A <t , PreTreeIntent and input Q, and as described in the above formula, the conditional probability P of the entire reply A is the product of the conditional probabilities of each character A t , and the optimal reply A max generated finally is A that makes the conditional probability P maximum, which must belong to an intent string in the prefix tree PreTreeIntent. In some embodiments, the language model can use a beam search algorithm to generate the optimal reply A max mentioned above, that is, to generate the intent of the sample to be annotated Q. In this paper, beam search refers to retaining the n sequences (n > 1) with the optimal conditional probability up to the current stage in each generation stage (for example, each time step) during intent generation, and then taking them as input to obtain the output of the next stage, until the model gives a sign of generation end. In some cases, using beam search to generate intent can improve the accuracy of the generation result.

[0052] As can be seen from the above, the method for intent generation with prefix tree according to the present disclosure can not only ensure that the generated intent is in the defined intent set (intent system), but also reduce the candidate search range when each part (e.g., each character) of the intent is generated. Figure 1

[0053] Returning to Figure 1 After the intent is generated in step S130, the training corpus of the dialogue system is generated by labeling the generated intent in the sample to be labeled in step S140.

[0054] In some cases, the slot has been included in the generated intent, or according to the generated intent, no additional slot information is needed. At this time, the sample to be labeled does not need to be subjected to a slot extraction operation to become a training corpus. For example, as can be seen from the 2nd, 6th and 8th rows in Table 1, for some intents, their slots are clear from the intent, and no separate extraction is needed. For some intents, the intent itself is a complete and clear instruction, and no slot extraction is needed. Figure 1

[0055] However, in other cases, as shown in the 3rd to 5th and 7th rows of Table 1, after the intent is generated, slot extraction is needed to obtain the slot of the sample to be labeled, so as to obtain a complete training corpus. In some embodiments, the intent-labeled sample after the sample to be labeled is labeled with the generated intent can be obtained, the intent-labeled sample after the slot prompt is spliced with the generated intent, and then the intent-labeled sample after the spliced slot prompt is input into the language model to generate the slot. The language model used here can be the same as or different from the language model used to generate the intent. After the slot is generated, the intent-labeled sample after the spliced slot prompt is labeled with the generated slot, thereby generating the training corpus of the dialogue system. In particular, the slot extraction is an open result, and the slot can be any segment in the sample to be labeled. Therefore, the slot generation result is not limited to the prefix tree of the intent system, i.e., the prefix tree constraint is removed during generation.

[0056] The specific method of prompting and splicing in the process of generating the intent and the slot is shown in Table 2 below (the user input dialogue in Table 2 is the sample to be labeled):

[0057]

[0058] Table 2

[0059] The entire process is described below using the 2nd sample to be labeled in Table 2 as an example:

[0060] The user input dialogue (sample to be labeled) is: ​​

[0061] Xiaomeng, navigate to the Neusoft Mansion by using the Gaode map.

[0062] The input to the language model after the intent prompt splicing is:

[0063] Xiaomeng, navigate to the Neusoft Mansion by using the Gaode map, the intent of this sentence in [AR glasses] belongs to

[0064] The result generated by the language model is:

[0065] Xiaomeng, navigate to the Neusoft Mansion by using the Gaode map, the intent of this sentence in [AR glasses] belongs to navigation-route navigation

[0066] The input to the language model after the slot prompt splicing is:

[0067] Xiaomeng, navigate to the Neusoft Mansion by using the Gaode map, the intent of this sentence in [AR glasses] belongs to navigation-route navigation, the destination of navigation is

[0068] The result generated by the language model is:

[0069] Xiaomeng, navigate to the Neusoft Mansion by using the Gaode map, the intent of this sentence in [AR glasses] belongs to navigation-route navigation, the destination of navigation is the Neusoft Mansion

[0070] In some embodiments, each intent and slot can be designed by business personnel as needed. In some embodiments, the prompt template for each intent and slot can be manually designed according to the experience of data annotators.

[0071] The foregoing describes the present disclosure by taking the case where the dialogue system is applied to a single device as an example, but the present disclosure is particularly advantageous in the application scenario where the dialogue system is applied to multiple devices.

[0072] With the development of multi-terminal interconnection technology, the landing mode of the same set of cloud dialogue system algorithm adapting to different terminals has become the mainstream. In the application landing scene of the dialogue system cloud algorithm of multi-terminal interconnection, the intent system of different devices needs to be differentiated according to the specific environment of the device. Therefore, in order to support the training of such dialogue system model, it is necessary to construct training corpus that adapts to the differentiated intent of multiple devices. However, the workload of constructing training corpus for these different intent systems is very large. If the intent systems of different devices are annotated separately, the workload will double with the increase of the number of devices, and the annotators need to accurately distinguish and correspond to the intent systems of different devices, which is easy to confuse and increase the difficulty of annotation. Since different intent systems have certain relevance, some semi-automatic methods can be used to convert the intent systems of different devices to realize the conversion of annotated corpus on each device. However, this hard conversion method is usually not flexible and may encounter situations that cannot be converted.

[0073] Therefore, the present disclosure proposes to apply the aforementioned improved method of constructing training corpus to the scenario of constructing training corpus that adapts to the differentiated intent of multiple devices when applying dialogue system to multiple devices. By utilizing the semantic understanding ability of the language model, annotated corpus with device differentiated response can be quickly constructed for multiple intent systems of different devices, so that the dialogue system has better multi-device expandability and adaptability, and as mentioned before, the automation degree of annotation can be improved, greatly saving the labor cost, improving the construction efficiency of training corpus, and ensuring the accuracy of training corpus annotation. Moreover, in some cases, the automatic construction method of the present disclosure can more accurately distinguish and correspond to the intent systems of different devices than annotators, improving the accuracy of annotation. In addition, in the case where the intent systems of different devices are all encoded in the form of prefix coding tree, the semantic similarity of the intent can be reflected through the prefix tree structure.

[0074] Table 3 below gives an example of the intent and slot corresponding to different terminal devices of the same user dialogue (to-be-annotated sample):

[0075]

[0076] Table 3

[0077] The method of constructing training corpus that adapts to the differentiated intent of multiple devices according to at least some embodiments of the present disclosure will be described in detail below in combination with the examples of Table 3 and Figure 3 Please note that each operation of the following method can be in the manner described above unless otherwise stated, and the details will be omitted.

[0078] As Figure 3As shown, in step S310, the intent system of various devices to which the dialogue system is to be applied is obtained for subsequent use by the language model. Figure 3 Examples of intent systems of three devices (a mobile phone, AR glasses, and a smart watch) are given in Table 1. As mentioned before, in some embodiments, the intent systems of the three devices can be encoded into one prefix tree respectively, or all into one prefix tree.

[0079] Then, in step S320, a sample to be labeled is obtained, for example Figure 3 and the user dialogue "Xiaomeng, help me open QQ music" shown in Table 3.

[0080] Then, in step S330, the current dialogue (the sample to be labeled) is respectively spliced with the intent prompt for each device. As shown Figure 3 The current dialogue shown in step S320 is spliced with the intent prompt for different devices respectively, thereby forming three inputs spliced with different intent prompts, which are input into the language model such as GPT3.

[0081] Then, in step S340, the language model is used to respectively generate the intent of the sample to be labeled in various devices, wherein the intent system of the current device provided in step S310 is used to limit the generation result of the language model when generating for various devices, to ensure that the generated intent belongs to the intent system of the current device.

[0082] Then, in step S350, the intent generated for various devices is obtained, and the sample to be labeled is labeled with these intents respectively, thereby forming training corpus for the corresponding device respectively.

[0083] Note that although steps S320 and S330 are described above as being performed after step S310, it should be understood that step S310 can also be performed after steps S320 and S330, or simultaneously. Step S310 has no mutual restrictive relationship with steps S320 and S330, as long as step S310 is completed before the language model performs the restricted generation operation in step S340.

[0084] In addition, although Figure 3 the examples given in Table 1 do not need to further extract and label slots, in the case where further extraction and labeling of slots are needed as in the second sample to be labeled in Table 3, an additional step can be added after step S350, i.e. the sample to be labeled for which the intent has been generated is further spliced with the corresponding slot according to the situation of each device, and the spliced sample to be labeled is input into the language model to generate the respective corresponding slot. The sample to be labeled is labeled with the intent and slot generated for various devices respectively, thereby forming training corpus for various devices.​

[0085] In addition, the exemplary description of obtaining the intention system, the to-be-labeled sample, performing intention prompt splicing, generating the intention under the constraint, and labeling the to-be-labeled sample has been disclosed in the foregoing content of the present document, and will not be repeated here.

[0086] After obtaining the training corpus as described above, the dialog system can be trained using the training corpus. The following describes at least some embodiments of the training method of the dialog system of the present disclosure in detail by taking Figure 4 as an example.

[0087] Figure 4 An illustrative flowchart of the training method of the dialog system according to at least one embodiment of the present disclosure is shown.

[0088] As shown in Figure 4 , in step S410, the device to which the dialog system is to be applied is determined. For example, it is determined to which device the dialog system is to be applied, so as to construct the corresponding training corpus for this device using the various schemes described above, or to select the training corpus constructed for this device from the training corpus constructed for multiple devices using the various schemes described above.

[0089] Then, in step S420, the dialog system is trained based on the training corpus of the dialog system generated according to the various schemes described above until the dialog system meets the stopping condition of the training. In some embodiments, the training stopping condition can include that the loss function no longer decreases or that the preset number of training iterations is reached.

[0090] The method of constructing the training corpus of the dialog system and the training method of the dialog system according to the present disclosure have been described in detail above with reference to the accompanying drawings, which automatically generates the intention using the semantic understanding ability of the language model, and at the same time, constrains the intention generation result of the language model with the intention system of the device, so as to easily and quickly construct the training corpus and improve the accuracy of labeling the training corpus.

[0091] Figure 5 An illustrative structural diagram of a computing device according to an embodiment of the present disclosure that can be used to implement the method of constructing the training corpus of the dialog system or the training method of the dialog system described above is shown.

[0092] Referring to Figure 5 , the computing device 500 includes a memory 510 and a processor 520.

[0093] The processor 520 can be a multi-core processor or include multiple processors. In some embodiments, the processor 520 can include a general-purpose main processor and one or more special-purpose co-processors, such as a graphics processor (GPU), a digital signal processor (DSP), and the like. In some embodiments, the processor 520 can be implemented using custom circuitry, such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0094] The memory 510 can include various types of storage units, such as a system memory, a read-only memory (ROM), and a permanent storage device. Among them, the ROM can store static data or instructions required by the processor 520 or other modules of the computer. The permanent storage device can be a read-write storage device. The permanent storage device can be a non-volatile storage device that does not lose stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a mass storage device (such as a magnetic or optical disk, a flash memory) as a permanent storage device. In some other embodiments, the permanent storage device can be a removable storage device (such as a floppy disk, an optical drive). The system memory can be a read-write storage device or a volatile read-write storage device, such as a dynamic random access memory. The system memory can store some or all of the instructions and data required by the processor during runtime. In addition, the memory 510 can include a combination of any computer readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), magnetic disks and / or optical disks. In some embodiments, the memory 510 can include a read and / or write removable storage device, such as a compact disc (CD), a read-only digital versatile disc (such as DVD-ROM, double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a min SD card, a Micro-SD card, and the like), a magnetic floppy disk, and the like. The computer readable storage medium does not include a carrier wave and a transient electronic signal transmitted through a wireless or wired transmission.

[0095] The memory 510 stores executable code, which, when processed by the processor 520, can cause the processor 520 to perform the method of constructing the training corpus of the dialog system or the training method of the dialog system described above.

[0096] Furthermore, the method according to the present disclosure can also be implemented as a computer program or a computer program product, which comprises computer program code instructions for executing the above-mentioned steps defined in the above-mentioned method of the present disclosure.

[0097] Alternatively, the present disclosure can also be implemented as a non-transitory machine readable storage medium (or computer readable storage medium, or machine readable storage medium) having stored thereon executable code (or computer program, or computer instruction code), which, when executed by a processor of an electronic device (or computing device, server, etc.), causes the processor to perform the steps of the above-mentioned method according to the present disclosure.

[0098] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein can be implemented as electronic hardware, computer software, or combinations of both.

[0099] The flow diagrams and block diagrams in the drawings are presented to illustrate possible architectures, functions, and operations for systems and methods in accordance with various embodiments of the present disclosure. In this regard, each block in the flow diagrams and block diagrams can represent a module, a segment, or a portion of code that comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0100] The embodiments of the present disclosure have been described above, the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes are obvious to those skilled in the art without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles, practical application, or improvement of the technology in the market of the embodiments, or to enable other ordinary skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for constructing a training corpus of a dialogue system, comprising: obtaining an intent schema of a device to which the dialogue system is to be applied, wherein the intent schema is encoded in a prefix tree data structure, wherein each path of the prefix tree forms an intent in the intent schema, and wherein one character in each intent in the intent schema is stored on each path of the prefix tree leading to a child node; obtaining a sample to be annotated; generating an intent of the sample to be annotated character by character using a language model, and constraining a character to be generated at each character generation stage using the prefix tree to ensure that the character to be generated belongs to one of the characters stored on a path leading from a last prefix tree node of a previously generated prefix portion to a next child node; and annotating the sample to be annotated with the intent to generate a training corpus of the dialogue system.

2. The method of claim 1, wherein, The generating of the intent of the sample to be annotated comprises: concatenating the sample to be annotated with an intent prompt, and inputting the concatenated sample to be annotated into the language model to generate the intent.

3. The method of claim 1, wherein, After the generating of the intent of the sample to be annotated, the method further comprises: obtaining an intent annotated sample after the sample to be annotated is annotated with the intent; concatenating the intent annotated sample with a slot prompt adapted to the intent; inputting the concatenated intent annotated sample into the language model to generate a slot; and annotating the concatenated intent annotated sample with the slot to generate a training corpus of the dialogue system.

4. The method of claim 3, wherein, The slot is any segment in the sample to be annotated.

5. The method of claim 1, wherein, The language model uses a beam search algorithm to generate the intent of the sample to be annotated. 6.A method for constructing a training corpus of a dialogue system, comprising: obtaining intent schemas of multiple devices to which the dialogue system is to be applied, wherein the intent schemas of the multiple devices are each encoded in a prefix tree data structure, wherein each path of the prefix tree forms an intent in the corresponding intent schema, and wherein one character in each intent in the corresponding intent schema is stored on each path of the prefix tree leading to a child node; obtaining a sample to be annotated; generating intents of the sample to be annotated in the multiple devices character by character using a language model, and constraining a character to be generated at each character generation stage using the prefix trees of the multiple devices to ensure that the character to be generated belongs to one of the characters stored on a path leading from a last prefix tree node of a previously generated prefix portion to a next child node; and annotating the sample to be annotated with the intents in the multiple devices respectively to generate a training corpus of the dialogue system. 7.A method for training a dialogue system, comprising: determining a device to which the dialogue system is to be applied; training the dialogue system based on the training corpus of the dialogue system generated by any one of claims 1-6 until the dialogue system meets a stopping condition of training. 8.A computing device, comprising: a processor; and ​ a memory having stored thereon executable code that, when executed by the processor, is to cause the processor to perform the method of any of claims 1-7.

9. A non-transitory machine-readable storage medium having stored thereon executable code that, when executed by a processor of an electronic device, is to cause the processor to perform the method of any of claims 1-7.

Images