Training Method and System for Intention Prediction Model

Abstract

An embodiment of the present invention provides a method for training an intention prediction model. The method includes: the intention prediction model receives training data and determines whether the training data meets a preset training standard; when the preset training standard is met, based on BERT, true samples of the training data and first false samples generated by a generator are used to train at least the parameters of a discriminator so that the discriminator can distinguish the intention categories of all samples; the parameters of the generator are trained using second false samples output by the generator to generate false samples that the discriminator cannot predict; the adversarial training of the discriminator and the generator is repeated until the intention prediction model converges. An embodiment of the present invention also provides a training system for an intention prediction model. The embodiments of the present invention can reduce the cost of manually processing a data set, and can better utilize the data set provided by a user when the user fails to provide null samples required for model training well, without the need for relatively complex processing of the data set.

Description

Technical Field

[0001] The present invention relates to the field of semantic understanding, and particularly to a method and system for training an intent prediction model. Background Art

[0002] In order to make intelligent voice assistants more user-friendly, it is very important to be able to understand the intent in the user's conversation. Only by understanding the user's intent can accurate feedback be provided. To understand the user's intent, the following methods are usually adopted:

[0003] 1. Pre-trained language models

[0004] A pre-trained language model refers to training using a large amount of text that has appeared in people's lives, enabling the model to learn the probability distribution of each word or character in these texts, and thus building a model that conforms to these text distributions. The label of the language model corpus is its context, which determines that people can almost infinitely use large-scale corpora to train language models. These large-scale corpora enable the model to obtain powerful capabilities and further demonstrate excellent effects in downstream related tasks. On this basis, for text classification tasks, a small number of classification annotation tasks are used, and fine-tuning learning is performed through the pre-trained language model to obtain better classification effects.

[0005] 2. Meta-learning

[0006] Also known as "learning to learn", that is, using past knowledge and experience to guide the learning of new tasks, enabling the network to have the ability to learn to learn. The essence of meta-learning is to increase the generalization ability of the learner in multiple tasks. Meta-learning requires sampling for both tasks and data, so the "formula" learned can quickly (depending on very few samples) establish a mapping in tasks that have not appeared. Therefore, meta-learning is mainly reflected in the network's learning of multiple tasks, and by continuously adapting to each specific task, the network is equipped with an abstract learning ability.

[0007] In the process of implementing the present invention, the inventor found that there are at least the following problems in the related technologies:

[0008] Algorithms belonging to deep learning often require a large amount of manual text category annotation, which will consume extremely large time and labor costs.

[0009] Although the method based on the pre-trained model can learn through a large amount of unlabeled natural text, thereby reducing the cost of manual annotation, the annotation of text classification generally only annotates the text of the predefined categories, but the text content outside the categories cannot be processed. On the other hand, the method based on the pre-trained model cannot solve the problem of insufficient training corpus for new tasks.

[0010] On the one hand, the meta - learning - based method also needs to rely on a large amount of corpus for training. It requires different task data as input, and these task data can be obtained through full annotation or through a small amount of annotation + sampling. In this way, the data required in the model training stage of meta - learning is actually more rather than less, so it indirectly increases the data annotation cost. On the other hand, the model training and parameter tuning process of meta - learning, such as MAML, is relatively difficult. Although theoretically a model that can achieve better results through rapid learning on all tasks can be obtained, in practice, it cannot be fully ensured that the training of the model in the case of using different types of data is carried out in the correct direction. Summary of the Invention

[0011] In order to at least solve the problems of labeled data in small - sample learning in the prior art: insufficient labeled sample data and insufficient unlabeled sample data. This will lead to poor recognition effects for in - distribution data, that is, data that should belong to a certain classification may be misrecognized, while the latter will lead to poor recognition effects for out - of - distribution data. First, an embodiment of the present invention provides a training method for an intention prediction model, including:

[0012] The intention prediction model receives training data and determines whether the training data meets a preset training standard. Among them, the intention prediction model includes: a generator, BERT, and a discriminator;

[0013] When the preset training standard is met, based on BERT, the true samples of the training data and the first false samples generated by the generator are used to train at least the parameters of the discriminator, so that the discriminator can distinguish the intention categories of all samples, and is used to reduce the loss of the discriminator;

[0014] The second false samples output by the generator are used to train the parameters of the generator to generate false samples that the discriminator cannot predict, and are used to increase the loss of the discriminator;

[0015] Repeat the adversarial training of the discriminator and the generator until the intention prediction model converges.

[0016] Second, an embodiment of the present invention provides a training system for an intention prediction model, including:

[0017] A training data receiving program module is used for the intention prediction model to receive training data and determine whether the training data meets a preset training standard. Among them, the intention prediction model includes: a generator, BERT, and a discriminator;

[0018] An intention category discrimination program module, when meeting a preset training standard, is used to determine true samples of the training data based on the BERT, and at least train parameters of the discriminator with first false samples generated by the generator, so that the discriminator can distinguish intention categories of all samples and reduce the loss of the discriminator;

[0019] A sample generation program module is used to train parameters of the generator with second false samples output by the generator to generate false samples that the discriminator cannot predict and increase the loss of the discriminator;

[0020] An adversarial training program module is used to repeatedly perform adversarial training on the discriminator and the generator until the intention prediction model converges.

[0021] In a third aspect, an electronic device is provided, which includes: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute steps of the training method of the intention prediction model according to any embodiment of the present invention.

[0022] In a fourth aspect, an embodiment of the present invention provides a storage medium, on which a computer program is stored, characterized in that when the program is executed by a processor, steps of the training method of the intention prediction model according to any embodiment of the present invention are implemented.

[0023] The beneficial effects of the embodiments of the present invention are as follows: The cost of manually processing a data set can be reduced, and the data set provided by a user can be better utilized. When the user fails to well provide empty samples required for model training, the data set provided by the user can be better utilized without performing relatively complex processing on the data set. Further, if the data set provided by the user can be processed and then sent into the model, the model can also obtain better effects than an ordinary pre-trained fine-tuning model. Description of the Drawings

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0025] Figure 1 It is a flowchart of a training method of an intention prediction model provided by an embodiment of the present invention;

[0026] Figure 2Schematic diagram of the GANBERT structure of a training method for an intent prediction model provided by an embodiment of the present invention;

[0027] Figure 3 Schematic diagram of the CGANBERT structure of a training method for an intent prediction model provided by an embodiment of the present invention;

[0028] Figure 4 Flowchart of the training of an intent prediction model for a training method for an intent prediction model provided by an embodiment of the present invention;

[0029] Figure 5 Schematic diagram of the structure of a training system for an intent prediction model provided by an embodiment of the present invention;

[0030] Figure 6 Schematic diagram of the structure of an embodiment of an electronic device for training an intent prediction model provided by an embodiment of the present invention. Detailed implementation manners

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0032] As Figure 1 shown in the flowchart of a training method for an intent prediction model provided by an embodiment of the present invention, the method includes the following steps:

[0033] S11: The intent prediction model receives training data and determines whether the training data meets a preset training standard. Among them, the intent prediction model includes: a generator, BERT, and a discriminator;

[0034] S12: When the preset training standard is met, based on BERT, true samples of the training data and first false samples generated by the generator are used to train at least the parameters of the discriminator, so that the discriminator can distinguish the intent categories of all samples, for reducing the loss of the discriminator;

[0035] S13: The second false samples output by the generator are used to train the parameters of the generator to generate false samples that the discriminator cannot predict, for increasing the loss of the discriminator;

[0036] S14: Repeat the adversarial training of the discriminator and the generator until the intent prediction model converges.

[0037] In this embodiment, during the process of productizing the intent classification model of the dialogue robot, it is often encountered that users cannot correctly provide training samples well when using the intent model training tool. For example, when users provide corpus, only a few pieces of data (such as 5 pieces, the number of pieces is not limited) are provided for each intent category. Moreover, users generally cannot define empty samples well because most of the time users only know what kind of data is a positive example, but they are not very clear about the boundary of empty samples. Therefore, most of the training data provided by users does not contain or only contains very few empty samples. For this reason, the data input by users is often small-sample.

[0038] For step S11, the intent prediction model of this method includes a generator, BERT (Bidirectional Encoder Representation from Transformers), and a discriminator. Based on the above, regarding the problem that users cannot correctly provide training samples well, after receiving the training data input by users, it will judge whether the training data meets the training criteria.

[0039] As an embodiment, the training data includes: empty samples that do not belong to any intent category and non-empty samples that belong to any intent category;

[0040] Judging whether the training data meets the preset training criteria includes: judging whether the proportion of the empty samples in the training data reaches the preset training criteria.

[0041] In this embodiment, users will provide intent training data, which may contain empty samples. It can be judged according to the quantity ratio of empty samples and non-empty samples (such as 1:1), or determined through experiments. Among them, it is possible that a piece of data that does not belong to any classification is identified within a certain specific classification (for such data, it is called an empty sample). For example, sentences such as "I want to buy medicine" and "Navigate to the park" have obvious intents (purchase, navigation), while the sentence "The water in this river is turbid" does not belong to any intent classification.

[0042] For step S12, if the proportion of empty samples reaches the preset training criteria (such as the set 1:1), the intent prediction model of this method can be used for training, such as Figure 2The intention prediction model of the GANBERT structure is shown, which consists of a generator, BERT, and a discriminator. By combining GAN (Generative Adversarial Networks) with the BERT model, the model can enhance the original training data through the generative network, thereby improving the out-of-sample accuracy and robustness of the model. Among them, BERT is a pre-trained model. The G part is a generator, whose role is to generate data similar to the real data. The D part is a discriminator, whose role is first to distinguish which category a real example belongs to, and second, it adds a category to judge whether a data is a real sample or a sample generated by the generator in the G part.

[0043] Based on BERT, determine the true samples of the training data, randomly initialize the parameters of the generator G, and use a noise data input to generate some fake samples. Then mix the true and false samples together and let the discriminator D learn.

[0044] Specifically, when the true and false samples are mixed and input into the discriminator D, the predicted intention can be determined. Among the training data, there will be a benchmark intention for which the user has prepared true samples. Use the error between the benchmark intention and the predicted intention to determine the loss of the discriminator. Use the loss of the discriminator to optimize the parameters of the D part of the discriminator, so that the predicted intention after training approaches the benchmark intention, improving the accuracy of the discriminator's predicted intention and thus reducing the loss of the discriminator's prediction.

[0045] As an implementation, based on the true samples and the first fake samples, jointly train the parameters of the discriminator and the parameters of BERT, so that BERT extracts the deep semantic representation of the training data to reduce the loss of the discriminator;

[0046] Based on the same method, use the loss of the discriminator to jointly train the parameters of the discriminator and the parameters of BERT. After such training, BERT can extract the deep semantic representation from the training data, and further enable the discriminator D to better distinguish the categories of all sample intentions.

[0047] For step S13, continue to use the generator G to regenerate a part of the data as additional fake samples. Since more fake samples are inserted, it will affect the intention prediction result of the discriminator D.

[0048] As a real-time method, training the parameters of the generator using the second fake sample output by the generator includes:

[0049] Fix the parameters of the BERT and the discriminator, and use the loss of the discriminator's prediction of the second fake sample to train the parameters of the generator. The goal of the training is to increase the loss of the prediction in order to generate fake samples that the discriminator cannot predict.

[0050] In this embodiment, the training is for the entire model, but the parameters of the discriminator D and the BERT part are fixed, and only the parameters in the generator G are adjusted. For example, the fake sample data generated by the generator G in one round (outside the training data provided by the user) is "I want to buy a mobile phone". At this time, the discriminator can predict the corresponding intention (purchase) more accurately. At this time, the discriminator has been trained in step S12 and has a certain discrimination ability for such sentences with obvious intentions. At this time, the loss of the discriminator is still relatively low. However, the purpose of this method is to use richer fake samples for adversarial training. Therefore, the parameters in the generator G are trained using the loss of the discriminator. For example, after training, the fake sample data generated by the generator G in the next round is "I want a mobile phone". At this time, the discriminator can estimate the possible relationships (such as purchase, pick up, etc.) between the user and the mobile phone. At this time, the loss of the discriminator's prediction will increase. Through the above training method step by step, the generator generates fake samples that the discriminator cannot predict, improving the breadth of the training data of the intention prediction model.

[0051] For step S14, repeat the above steps S12 and S13 for the adversarial training of the discriminator, BERT, and generator. After S12, the accuracy of the discriminator's prediction of intentions can be gradually improved in the existing mixed true and false samples. After S13, on the basis of the accurate prediction of the discriminator's intentions, the breadth of the prediction samples can be further improved, and more dialogue intentions can be estimated. Then, on the basis of improving the sample breadth, the accuracy of the discriminator's prediction of intentions is improved again. Through continuous adversarial training until the intention prediction model converges. The convergence condition of the model can be set as the prediction error being less than a certain preset value; or the weight change between two iterations is already very small, and a threshold can be set. When it is less than this threshold, the training stops; or the maximum number of iterations can be set, and when the iteration exceeds the maximum number, the training stops. The convergence condition is not restricted here. Generally speaking, if the number of empty samples provided by the user meets the standard, the intention prediction model with the GANBERT structure composed of the generator, BERT, and discriminator can be directly used for training.

[0052] As an embodiment, when the training data does not meet the preset training standard, the method further includes:

[0053] Based on the BERT, true samples of the training data, first false empty samples generated by the generator, and labels corresponding to the first false empty samples are determined, and the parameters of the discriminator and the parameters of the BERT are jointly trained to enable the BERT to extract deep semantic representations of the training data for reducing the loss of the discriminator;

[0054] Repeat the adversarial training of the discriminator, BERT, and the generator until the intent prediction model converges.

[0055] In this embodiment, if the training data input by the user does not meet the preset conditions, that is, there are too few empty samples. If such data is directly used for training, the trained model will not solve the problem of empty sample screening. The intent prediction model of this method can be used for training, such as Figure 3 shown in the intent prediction model with a CGANBERT structure composed of a generator, BERT, and a discriminator. Among them, the CGAN (Conditional Generative Adversarial Nets) is combined with the BERT model. Among them, the idea of CGANBERT is based on GANBERT, and only the generator G is allowed to generate empty samples to enhance the prediction accuracy of the model for the empty sample part and improve the prediction accuracy of the model for all intents.

[0056] The specific approach is that during the input of the generator, the corresponding labels are input into the network while the noise data is input. Additionally, during the input of the discriminator D, the corresponding labels are also input into the network. In this way, the discriminator can learn to judge the samples that correctly match the labels and the generated samples as correct samples, and the generator can also learn to generate corresponding samples according to the labels. (The training process is similar to the above GANBERT, with the difference that the generator G generates empty samples and the property of adding labels is used for training, and the training process will not be elaborated). Generally speaking, if the number of empty samples in the training data input by the user does not meet the requirements, then the intent prediction model with a CGANBERT structure can be used.

[0057] If it is determined that the CGANBERT model needs to be used, then some empty samples are selected from the built-in corpus and mixed into the corpus provided by the user, and at the same time, the model with a CGANBERT structure is used for training. The trained CGANBERT model can be directly deployed like other models and then provide intent prediction services. The overall process is as Figure 4 shown. After training the small-sample training data input by the user, BERT and the discriminator are used for intent prediction to obtain accurate prediction results.

[0058] The method is tested, and the direct effect that can be obtained is the improvement of the intention prediction accuracy. Compared with the traditional fine-tuning of the pre-trained model (using BERT), CGANBERT can achieve a maximum improvement of 2.05% on the dataset in the express delivery field and a maximum improvement of 6% in the financial field.

[0059] It can be seen from this implementation that this method can reduce the cost of manually processing the dataset and make better use of the dataset provided by the user. When the user fails to provide the empty samples required for model training well, the dataset provided by the user can be better utilized without the need to perform relatively complex processing on the dataset. Further, if the dataset provided by the user can be processed and then fed into this model, the model can also achieve better results than the ordinary pre-trained fine-tuning model.

[0060] As Figure 5 shown is a schematic structural diagram of a training system for an intention prediction model provided by an embodiment of the present invention. The system can execute the training method of the intention prediction model described in any of the above embodiments and is configured in a terminal.

[0061] A training system 10 for an intention prediction model provided in this embodiment includes: a training data receiving program module 11, an intention category distinguishing program module 12, a sample generating program module 13, and an adversarial training program module 14.

[0062] Among them, the training data receiving program module 11 is used to receive training data for the intention prediction model and determine whether the training data meets the preset training criteria. The intention prediction model includes: a generator, BERT, and a discriminator; the intention category distinguishing program module 12 is used to, when the preset training criteria are met, determine the true samples of the training data based on BERT and at least train the parameters of the discriminator with the first false samples generated by the generator, so that the discriminator can distinguish the intention categories of all samples and reduce the loss of the discriminator; the sample generating program module 13 is used to train the parameters of the generator with the second false samples output by the generator to generate false samples that the discriminator cannot predict and increase the loss of the discriminator; the adversarial training program module 14 is used to repeatedly perform adversarial training on the discriminator and the generator until the intention prediction model converges.

[0063] An embodiment of the present invention also provides a non-volatile computer storage medium. The computer storage medium stores computer-executable instructions, and the computer-executable instructions can execute the training method of the intention prediction model in any of the above method embodiments;

[0064] As an implementation, the non-volatile computer storage medium of the present invention stores computer-executable instructions, and the computer-executable instructions are set as follows:

[0065] The intention prediction model receives training data and determines whether the training data meets a preset training standard. Among them, the intention prediction model includes: a generator, BERT, and a discriminator;

[0066] When the preset training standard is met, based on BERT, the true samples of the training data and the first false samples generated by the generator are used to train at least the parameters of the discriminator, so that the discriminator can distinguish the intention categories of all samples, for reducing the loss of the discriminator;

[0067] The parameters of the generator are trained using the second false samples output by the generator to generate false samples that the discriminator cannot predict, for increasing the loss of the discriminator;

[0068] Repeat the adversarial training of the discriminator and the generator until the intention prediction model converges.

[0069] As a non-volatile computer-readable storage medium, it can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods in the embodiments of the present invention. One or more program instructions are stored in the non-volatile computer-readable storage medium and, when executed by a processor, execute the training method of the intention prediction model in any of the above method embodiments.

[0070] Figure 6 It is a schematic diagram of the hardware structure of an electronic device for the training method of the intention prediction model provided in another embodiment of the present application, as Figure 6 shown. The device includes:

[0071] One or more processors 610 and a memory 620, Figure 6 Taking one processor 610 as an example. The device for the training method of the intention prediction model may further include: an input device 630 and an output device 640.

[0072] The processor 610, the memory 620, the input device 630, and the output device 640 can be connected through a bus or other means, Figure 6 Taking connection through a bus as an example.

[0073] The memory 620, being a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the training method of the intent prediction model in the embodiments of the present application. The processor 610 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 620, that is, implements the training method of the intent prediction model in the above method embodiments.

[0074] The memory 620 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data, etc. In addition, the memory 620 may include high-speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 620 may optionally include a memory remotely disposed relative to the processor 610, and these remote memories can be connected to the mobile device through a network. Examples of the above networks include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0075] The input device 630 can receive input digital or character information. The output device 640 may include a display device such as a display screen.

[0076] The one or more modules are stored in the memory 620 and, when executed by the one or more processors 610, execute the training method of the intent prediction model in any of the above method embodiments.

[0077] The above product can execute the method provided in the embodiments of the present application, and has functional modules and beneficial effects corresponding to the execution of the method. For technical details not described in detail in this embodiment, reference can be made to the method provided in the embodiments of the present application.

[0078] A non-volatile computer-readable storage medium may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the device, etc. In addition, the non-volatile computer-readable storage medium may include high-speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the non-volatile computer-readable storage medium may optionally include a memory remotely disposed relative to the processor, and these remote memories can be connected to the device through a network. Examples of the above networks include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0079] An embodiment of the present invention further provides an electronic device, which includes: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the steps of the training method of the intention prediction model according to any embodiment of the present invention.

[0080] The electronic devices in the embodiments of the present application exist in various forms, including but not limited to:

[0081] (1) Mobile communication devices: These devices are characterized by having mobile communication functions and mainly aim to provide voice and data communication. Such terminals include: smart phones, multimedia phones, functional phones, and low-end phones, etc.

[0082] (2) Ultra-mobile personal computer devices: These devices belong to the category of personal computers, have computing and processing functions, and generally also have the characteristic of mobile Internet access. Such terminals include: PDAs, MIDs, and UMPC devices, etc., such as tablet computers.

[0083] (3) Portable entertainment devices: These devices can display and play multimedia content. Such devices include: audio and video players, handheld game consoles, e-books, and smart toys and portable in-vehicle navigation devices.

[0084] (4) Other electronic devices with data processing functions.

[0085] In this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include" and "comprise", not only include those elements, but also include other elements not expressly listed, or also include elements inherent to such a process, method, article or device. Without more limitations, the elements defined by the statement "including..." do not exclude the existence of additional identical elements in the process, method, article or device including the said elements.

[0086] The device embodiments described above are only illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0087] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A training method for an intention prediction model, comprising: The intention prediction model receives training data and determines whether the training data meets a preset training standard, wherein the intention prediction model includes: a generator, BERT, and a discriminator; When the number of empty samples meets the standard, based on BERT, true samples of the training data and first fake samples generated by the generator are used to at least train the parameters of the discriminator, so that the discriminator can distinguish the intention categories of all samples, for reducing the loss of the discriminator. The second fake samples output by the generator are used to train the parameters of the generator to generate fake samples that the discriminator cannot predict, for increasing the loss of the discriminator. The adversarial training of pre-training and fine-tuning the discriminator and the generator is repeated until the intention prediction model converges; When the number of empty samples does not meet the standard, a conditional generative adversarial network is combined with BERT, and only the generator is allowed to generate the empty samples and their corresponding labels. Based on the true samples, the empty samples, the corresponding labels, and the first fake samples generated by the generator, the parameters of the discriminator are at least trained, so that the discriminator can distinguish the intention categories of all samples, for reducing the loss of the discriminator. The second fake samples output by the generator are used to train the parameters of the generator to generate fake samples that the discriminator cannot predict, for increasing the loss of the discriminator. The adversarial training of the conditional generative adversarial network for the discriminator and the generator is repeated until the intention prediction model converges.

2. The method according to claim 1, wherein, When the preset training standard is met, the method further includes: Based on the true samples and the first fake samples, the parameters of the discriminator and the parameters of BERT are jointly trained, so that BERT extracts the deep semantic representation of the training data, for reducing the loss of the discriminator; The adversarial training of the discriminator, BERT, and the generator is repeated until the intention prediction model converges.

3. The method according to claim 1, wherein, The training data includes: empty samples that do not belong to any intention category and non-empty samples that belong to any intention category; Determining whether the training data meets the preset training standard includes: determining whether the proportion of the empty samples in the training data reaches the preset training standard.

4. The method according to claim 1, wherein, When the training data does not meet the preset training standard, the method further includes: Based on the true samples of the training data determined by BERT, the first fake empty samples generated by the generator, and the labels corresponding to the first fake empty samples, the parameters of the discriminator and the parameters of BERT are jointly trained, so that BERT extracts the deep semantic representation of the training data, for reducing the loss of the discriminator; The adversarial training of the discriminator, BERT, and the generator is repeated until the intention prediction model converges.

5. The method according to claim 1, wherein Training the parameters of the generator using the second fake samples output by the generator includes: Fix the parameters of the BERT and the discriminator, and use the loss of the discriminator's prediction of the second fake sample to train the parameters of the generator. The goal of the training is to increase the loss of the prediction to generate fake samples that the discriminator cannot predict.

6. The method according to any one of claims 1-5, wherein, After the intent prediction model converges, the method further includes: Use the BERT and the discriminator for intent prediction.

7. A training system for an intent prediction model, comprising: A training data receiving program module for receiving training data for the intent prediction model and determining whether the training data meets a preset training standard. Among them, the intent prediction model includes: a generator, a BERT, and a discriminator; A pre-training and fine-tuning module for, when the preset training standard is met, determining true samples of the training data based on the BERT, and at least training the parameters of the discriminator with the first fake samples generated by the generator, so that the discriminator can distinguish the intent categories of all samples and reduce the loss of the discriminator. A sample generation program module for training the parameters of the generator with the second fake samples output by the generator to generate fake samples that the discriminator cannot predict and increase the loss of the discriminator. An adversarial training program module for repeatedly performing adversarial training on the discriminator and the generator until the intent prediction model converges; A conditional generation adversarial module for, when the number of empty samples does not meet the standard, combining a conditional generation adversarial network with the BERT, and only allowing the generator to generate the empty samples and corresponding labels. Based on the true samples, the empty samples, the corresponding labels, and the first fake samples generated by the generator, at least train the parameters of the discriminator so that the discriminator can distinguish the intent categories of all samples, reduce the loss of the discriminator, use the second fake samples output by the generator to train the parameters of the generator to generate fake samples that the discriminator cannot predict, increase the loss of the discriminator, and repeatedly perform adversarial training of the conditional generation adversarial network on the discriminator and the generator until the intent prediction model converges.

8. The system according to claim 7, wherein The training data includes: empty samples that do not belong to any intent category and non-empty samples that belong to any intent category; The training data receiving program module is used to: determine whether the proportion of the empty samples in the training data reaches the preset training standard.

9. An electronic device, comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the steps of the method according to any one of claims 1-6.

10. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method according to any one of claims 1-6.

Images