Artificial intelligence-based multi-task model training method and device, equipment and medium

Abstract

The present application is suitable for the field of artificial intelligence, and particularly relates to a multi-task model training method and device based on artificial intelligence, equipment and medium. The present application improves the generation capability of the generation model in the case of unsupervised corpus by splitting the preset text into a first and a second subtext, taking the first subtext as a training sample and the second subtext as a training label to train a generation model. The present application solves the problem of large difference in effect when different subtasks are trained separately by unifying different subtasks of different task categories into a generation task. The present application improves the accuracy of the multi-task classification model by inputting the preset task text into the generation model to obtain a task answer, and retraining the pre-trained generation model according to the task answer and the real answer to obtain a trained multi-task classification model.

Description

Technical Field

[0001] This invention is applicable to the field of artificial intelligence, and in particular relates to a method, apparatus, device and medium for training a multi-task model based on artificial intelligence. Background Technology

[0002] In existing deep learning systems, when training multi-task models for specific domains such as semantic matching, intent recognition, and entity recognition, the multi-task model is generally divided into multiple sub-task models for separate modeling and training, and then the various sub-task models are integrated into a single multi-task model.

[0003] However, due to differences in the number and quality of samples among different sub-tasks, training multiple sub-task models individually may result in significant differences in performance between different sub-tasks. Furthermore, as business scenarios continue to expand, adding new sub-tasks requires retraining the sub-task models and integrating the multi-task models, which greatly reduces the accuracy of the multi-task model.

[0004] Therefore, in the field of artificial intelligence, how to improve the accuracy of multi-task models has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method, apparatus, device and medium for training a multi-task model based on artificial intelligence, in order to solve the problem of low accuracy of existing multi-task models.

[0006] In a first aspect, embodiments of the present invention provide a multi-task model training method based on artificial intelligence, the multi-task model training method comprising:

[0007] Obtain preset texts from a preset corpus, split each preset text into a first sub-text and a second sub-text, use the first sub-text as a training sample and the second sub-text as a training label to train a preset generation model, and obtain a pre-trained generation model.

[0008] Obtain a preset statement, M preset question templates under N task categories, and corresponding preset candidate answer templates from a preset database. Substitute the preset statement into the M preset question templates under the N task categories to obtain several preset task questions. Concatenate each preset task question with the corresponding preset candidate answer template to obtain several preset task texts, where N and M are positive integers, and M>N.

[0009] Each of the preset task texts is input into the pre-trained generative model to obtain the task answers for each of the preset task texts;

[0010] Obtain the true answers for each of the preset texts, and retrain the pre-trained generative model based on the task answers and the true answers to obtain a trained multi-task classification model.

[0011] Secondly, embodiments of the present invention provide a multi-task model training device based on artificial intelligence, the multi-task model training device comprising:

[0012] The model pre-training module is used to obtain preset texts in a preset corpus, split each preset text into a first sub-text and a second sub-text, use the first sub-text as a training sample and the second sub-text as a training label to train the preset generation model, and obtain the pre-trained generation model.

[0013] The task text composition module is used to obtain a preset statement, M preset question templates under N task categories and corresponding preset candidate answer templates from a preset database, substitute the preset statement into the M preset question templates under the N task categories to obtain a number of preset task questions, and concatenate each preset task question and the corresponding preset candidate answer template to obtain a number of preset task texts, where N and M are positive integers and M>N;

[0014] The task answer generation module is used to input each of the preset task texts into the pre-trained generation model to obtain the task answers for each of the preset task texts;

[0015] The model retraining module is used to obtain the true answers for each of the preset texts, and to retrain the pre-trained generative model based on the task answers and the true answers to obtain a trained multi-task classification model.

[0016] Thirdly, embodiments of the present invention provide a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-task model training method as described in the first aspect.

[0017] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the multi-task model training method as described in the first aspect.

[0018] The beneficial effects of this invention compared to existing technologies are as follows: By splitting each preset text into a first sub-text and a second sub-text, using the first sub-text as a training sample and the second sub-text as a training label, a preset generation model is trained to obtain a pre-trained generation model, effectively improving the generation capability of the trained generation model in the case of unsupervised corpus; by substituting preset sentences into M preset question templates under N task categories to obtain several preset task questions, and concatenating each preset task question with its corresponding preset candidate answer template to obtain several preset task texts, subtasks of different task categories are uniformly constructed into generation tasks, solving the problem of multi-task model This addresses the issue of significant differences in performance when each subtask is trained independently. By inputting the text of each preset task into a pre-trained generative model, the task answers for each preset task text are obtained. The pre-trained generative model is then retrained based on the task answers and the actual answers to obtain a trained multi-task classification model. Compared to existing technologies, this eliminates the cumbersome modeling process of first building multiple different sub-task models and then integrating them into a unified multi-task model. Instead, it directly trains a multi-task classification model that adapts to various task classifications through the aforementioned dual training process, simplifying the model complexity. This not only adapts to different task classifications in various business scenarios but also effectively improves the accuracy of the multi-task model. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of an application environment for a multi-task model training method based on artificial intelligence provided in Embodiment 1 of the present invention;

[0021] Figure 2 This is a flowchart illustrating a multi-task model training method based on artificial intelligence provided in Embodiment 1 of the present invention;

[0022] Figure 3 This is a schematic diagram of an existing multi-task classification model;

[0023] Figure 4 This is a schematic diagram of a multi-task model based on artificial intelligence provided in Embodiment 1 of the present invention;

[0024] Figure 5 This is a schematic diagram of the structure of a multi-task model training device based on artificial intelligence provided in Embodiment 2 of the present invention;

[0025] Figure 6 This is a schematic diagram of the structure of a computer device provided in Embodiment 3 of the present invention. Detailed Implementation

[0026] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0027] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0028] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0029] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0030] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0031] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0032] The embodiments of this invention can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0033] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0034] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0035] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0036] The first embodiment of this invention provides a multi-task model training method based on artificial intelligence, which can be applied to, for example... Figure 1 In this application environment, the client communicates with the server. Clients include, but are not limited to, handheld computers, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud computing devices, and personal digital assistants (PDAs). The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0037] See Figure 2This is a flowchart illustrating a multi-task model training method based on artificial intelligence provided in Embodiment 1 of the present invention. The aforementioned multi-task model training method can be applied to... Figure 1 In the client-side, the multi-task model training method may include the following steps:

[0038] Step S201: Obtain preset texts from the preset corpus, split each preset text into a first sub-text and a second sub-text, use the first sub-text as a training sample and the second sub-text as a training label to train the preset generation model, and obtain the pre-trained generation model.

[0039] The preset prediction library includes several preset texts in the same field. For example, in the field of insurance, the preset prediction library may include several texts related to insurance, such as "how to buy Class A insurance products" and "what is the term of insurance products".

[0040] To reduce the workload of manual annotation and enable the generative model to learn text information in the preset corpus more effectively in the absence of unsupervised corpus, as well as the correlation information between texts in the corresponding domain, thereby generating output text related to the input text, in this embodiment, each preset text in the preset corpus is split into two sub-texts, namely the first sub-text and the second sub-text. The first sub-text is used for feature extraction, so that the generative model can learn the text information in the preset corpus to generate and output text content related to the input text content. The second sub-text is used to supervise the correlation between the text content generated by the generative model and the input text content, so that the generative model can have a higher generation accuracy.

[0041] Specifically, each preset text is split into a first sub-text and a second sub-text. The first sub-text is used as the training sample and the second sub-text is used as the training label to train the preset generation model, thereby obtaining a pre-trained generation model. This allows for effective training of the generation model's generation ability under unsupervised corpus conditions, thereby improving the generation accuracy of the generation model.

[0042] Optionally, the preset text can be split into a first subtext and a second subtext, including:

[0043] Each preset text is segmented into sentences to obtain K preset text sentences;

[0044] According to a preset splitting ratio, K preset text statements are split into K1 first sub-texts and K2 second sub-texts, where K1, K2, and K are all positive integers, and K1 + K2 = K, K2 <K1。

[0045] Among them, during the pre-training process of the generation model, the preset text is split into two parts, namely the first sub-text and the second sub-text, so as to effectively train the generation ability of the generation model in the case of unsupervised corpus. In order to ensure that the generation model can effectively learn sentence information, the preset text is first segmented into sentences, so as to retain more complete sentence content during the splitting process of the preset text.

[0046] Then, according to the preset splitting ratio, K preset text sentences are split into K1 first sub-texts and K2 second sub-texts. Among them, in order to make more full use of all the text information of each preset text, the preset splitting ratio satisfies the condition K1 + K2 = K. At the same time, since the first sub-text is the training sample and the second sub-text is the training label during the pre-training process of the generation model, the preset splitting ratio satisfies the condition K2 < K1.

[0047] Under the conditions of satisfying K1 + K2 = K and K2 < K1, the specific ratio value of the preset splitting ratio can be set according to the actual situation. In this embodiment, the preset splitting ratio is set to 3:1 according to the actual situation. At the same time, during the actual training process, both K1 and K2 are positive integers. Therefore, K2 = K - K1, where, is the largest integer not greater than .

[0048] Optionally, using the first sub-text as the training sample and the second sub-text as the training label, the preset generation model is trained to obtain a pre-trained generation model, including:

[0049] The generation model includes an encoder and a decoder;

[0050] Input the first sub-text into the encoder for feature extraction to obtain text semantic features;

[0051] Input the text semantic features into the decoder to obtain a generated sub-text;

[0052] Calculate the first loss of the generation model according to the generated sub-text and the second sub-text;

[0053] According to the gradient descent method, reverse correct the parameters of the encoder and the decoder until the first loss converges to obtain a pre-trained generation model.

[0054] Among them, the generation model uses the first sub-text as the training sample, so that the generation model can learn the text information in the first sub-text to generate and output text content related to the input text content. Therefore, the generation model includes an encoder for extracting features from the first sub-text to obtain text semantic features.

[0055] Meanwhile, the generative model uses the second subtext as a training label to supervise the correlation between the generated text content and the input text content. Therefore, the generative model includes a decoder to decode the semantic features of the text in order to output the generated subtext.

[0056] Then, to measure the generation accuracy of the generative model, the second subtext is used as the basis for measurement. The first loss of the generative model is calculated based on the generated subtext and the second subtext to characterize the degree of difference between the generated subtext and the second subtext.

[0057] The smaller the first loss, the better the feature extraction ability of the encoder and the decoding ability of the decoder in the generative model, resulting in a higher generation accuracy. Conversely, the larger the first loss, the worse the feature extraction ability of the encoder and the decoding ability of the decoder in the generative model, resulting in a lower generation accuracy. It is necessary to correct the encoder and decoder parameters in reverse according to the gradient descent method until the first loss converges, so that the generated subtext is as close as possible to the second subtext, in order to obtain the pre-trained generative model.

[0058] Optionally, the first loss of the generation model is calculated based on the generated subtext and the second subtext, including:

[0059] The generated subtext is converted into a generated subtext vector using word vector technology, and the second subtext is converted into a second subtext vector using word vector technology.

[0060] Calculate the degree of difference between the generated subtext vector and the second subtext vector, and determine the first loss of the generation model based on the degree of difference.

[0061] The first loss is used to measure the degree of difference between the generated subtext and the second subtext. Since both the generated subtext and the second subtext are text content, it is not convenient to directly calculate the degree of difference between them. However, word vector technology data, as the basis for text data to be processed by computers and an important prerequisite for machine learning technology to be applied to text data processing, can convert text data into vector data of different dimensions, providing a numerical representation method for the generated subtext and the second subtext.

[0062] Therefore, in this embodiment, the generated subtext is first converted into a generated subtext vector according to word vector technology, and the second subtext is converted into a second subtext vector according to word vector technology, wherein the word vector technology is, for example, word2vec technology (a word vector technology) or one-hot encoding technology.

[0063] Then, the degree of difference between the generated sub-text vector and the second sub-text vector is calculated, and the first loss of the generation model is determined based on the degree of difference. This loss serves as the basis for parameter correction of the encoder and decoder, thereby improving the generation accuracy of the generation model.

[0064] For example, let X be the number of training samples, and P be the i-th (i = 1, 2, ..., X) generated sub-text vector. i Let Q be the i-th second sub-text vector. i Calculate the degree of difference between X generated sub-text vectors and X second sub-text vectors to obtain the first loss:

[0065]

[0066] In the formula, L1 is the first loss, X is the number of training samples, and P... i For the i-th generated subtext vector, Q i Let be the i-th second sub-text vector.

[0067] The steps described above—obtaining preset texts from a preset corpus, splitting each preset text into a first sub-text and a second sub-text, using the first sub-text as a training sample and the second sub-text as a training label to train a preset generative model, and obtaining a pre-trained generative model—use the first sub-text as a training sample so that the generative model can learn the text information in the first sub-text, and use the second sub-text as a training label to supervise the correlation between the generated text content and the input text content, effectively improving the generative ability of the trained generative model in the case of unsupervised corpus.

[0068] Step S202: Obtain the preset statement, M preset question templates under N task categories and corresponding preset candidate answer templates from the preset database. Substitute the preset statement into the M preset question templates under N task categories to obtain several preset task questions. Concatenate each preset task question and the corresponding preset candidate answer template to obtain several preset task texts.

[0069] The multi-task model includes sub-samples of multiple tasks of different categories. The task categories can be set according to actual needs; for example, task categories may include semantic matching tasks, intent recognition tasks, entity recognition tasks, and other categories of tasks. Existing multi-task classification models generally divide the multi-task model into multiple sub-task models, train them separately, and then integrate these sub-task models into a single multi-task classification model. (See [link to previous section]). Figure 3This is a schematic diagram of an existing multi-task classification model, in which semantic matching, intent recognition, and entity recognition models are trained separately and then integrated into a single multi-task model. Because of differences in task intent, sample quantity, and sample quality among different sub-tasks, training multiple sub-task models separately can lead to significant differences in performance between different sub-tasks. Furthermore, with the continuous expansion of business scenarios, adding new sub-tasks requires retraining the sub-task models and re-integrating the multi-task model, which greatly reduces the accuracy of the multi-task model.

[0070] To address the aforementioned issues, this embodiment uses a pre-trained generative model as a foundation, uniformly constructing subtasks of different task categories into a single generative task. This allows the pre-trained generative model to be used to uniformly generate answers for various tasks. See [link to previous document]. Figure 4 This is a schematic diagram of the multi-task classification model in this embodiment. In this model, semantic matching task, intent recognition task, and entity recognition task are uniformly constructed into corresponding generation tasks, and a pre-trained generation model is used to uniformly generate answers for various tasks.

[0071] Meanwhile, in order to improve the generation performance of the pre-trained generative model under different task categories, this embodiment constructs the input content for different task categories by using preset question templates and preset candidate answer templates.

[0072] Specifically, first, obtain the preset statements, M preset question templates under N task categories and corresponding preset candidate answer templates from the preset database, where N and M are both positive integers and M>N.

[0073] For each task category, one, two, or more preset question templates can be set. For example, for semantic matching tasks, this task category may include preset question templates such as "Do {sentence A} and {sentence B} express the same meaning?" and "Are {sentence A} and {sentence B} semantically the same?"; for intent recognition tasks, this task category may include preset question templates such as "What is the intent of {sentence A}?", "What purpose does {sentence A} express?", and "What operation does {sentence A} want to achieve?"; for entity recognition tasks, this task category may include preset question templates such as "In sentence {sentence A}, what are all the insurance name entities?".

[0074] Simultaneously, for each task category, one, two, or more preset candidate answer templates can be set, and each candidate answer template can contain two, three, or more preset candidate answers. For example, for semantic matching tasks, this task category may include preset candidate answer templates such as "similar / dissimilar," "consistent / inconsistent," and "same / different." For intent recognition tasks, this task category may include preset candidate answer templates such as "insurance / claims / modify information…." For entity recognition tasks, this task category may include preset candidate answer templates such as "insurance / disease / occupation… entity" and "insurance / age / job… entity."

[0075] Therefore, in this embodiment, firstly, preset question templates and preset candidate answer templates for each task category can be constructed according to the actual situation. Then, preset statements from the preset database are substituted into each preset question template under each task category to obtain several preset task questions. Further, based on the task category corresponding to each preset task question, several preset candidate answer templates corresponding to each preset task question are determined. Each preset task question and any corresponding preset candidate answer template are then concatenated to obtain several preset task texts, which are used to construct input content for different types of tasks. This unifies the sub-tasks of different task categories into a generation task, so that the pre-trained generation model can be used to generate answers for various tasks.

[0076] Optionally, the preset statements can be substituted into M preset question templates under N task categories to obtain several preset task questions, including:

[0077] The N task categories include at least one of the following: semantic matching task, intent recognition task, and named entity recognition task;

[0078] Determine the number of statements S that need to be filled in each preset question template under each task category, where S is a positive integer;

[0079] Based on different task categories, select S preset statements from the database corresponding to each preset question template of each task category, and substitute them into M preset question templates to obtain several preset task questions.

[0080] Because the preset question templates are different for different task categories, the number of statements that need to be filled in each preset question template is also different. For example, in the preset question template "Do {Sentence A} and {Sentence B} express the same meaning?", the number of statements that need to be filled in is 2, while in the preset question template "What is the intention of {Sentence A}?", the number of statements that need to be filled in is 1.

[0081] Therefore, when substituting preset statements into M preset question templates under N task categories, it is first necessary to determine the number S of statements to be filled in each preset question template under each task category. Then, for different task categories, select S preset statements corresponding to each preset question template of each task category from the database and substitute them into the M preset question templates to obtain several preset task questions, where S is a positive integer.

[0082] In one embodiment, since the differences between multiple preset question templates of the same task category are small and can be used to represent the same task intent, in order to reduce the data redundancy of the generated preset task text, for different task categories, S preset statements corresponding to any preset question template of each group of task categories can be selected from the database and substituted into M preset question templates respectively to obtain a number of preset task questions.

[0083] The above steps involve obtaining preset statements from a preset database, M preset question templates under N task categories, and corresponding preset candidate answer templates; substituting the preset statements into the M preset question templates under N task categories to obtain several preset task questions; and concatenating each preset task question with its corresponding preset candidate answer template to obtain several preset task texts. By using the M preset question templates under N task categories and their corresponding preset candidate answer templates, subtasks of different task categories are uniformly constructed into generation tasks. This allows the use of a pre-trained generation model to uniformly generate answers for various tasks, effectively solving the problem of significant differences in performance when each subtask is trained individually in a multi-task model.

[0084] Step S203: Input each preset task text into the pre-trained generative model to obtain the task answer for each preset task text.

[0085] Among them, the pre-trained generative model has text generation capabilities. It can be used to process various preset texts under all task categories to generate task answers for each preset text.

[0086] The above steps of inputting each preset task text into the pre-trained generative model to obtain the task answers for each preset task text fully utilize the generative capabilities of the pre-trained generative model to obtain the task answers for each preset text, thus achieving the goal of uniformly constructing subtasks of different task categories into a generative task.

[0087] Step S204: Obtain the real answers for each preset text, and retrain the pre-trained generative model based on the task answers and the real answers to obtain the trained multi-task classification model.

[0088] In this context, after the multi-task classification model is trained, the task answer generated by the model is a preset candidate answer from the preset candidate answer template corresponding to the input. The real answer of each preset text is also a preset candidate answer from the preset candidate answer template in the preset text. When the task answer is consistent with the real answer, it indicates that the multi-task classification model has a good task answer generation effect for sub-tasks of different task categories, and the accuracy of the multi-task classification model is high.

[0089] To improve the performance of the pre-trained generative model in generating answers for sub-tasks of different task categories, this embodiment obtains the real answers of each preset text and measures the similarity between the task answers and the real answers to evaluate the performance of the pre-trained generative model in generating answers for sub-tasks of different task categories.

[0090] When the similarity between the task answer and the real answer is poor, the parameters of the pre-trained generative model need to be optimized again to retrain the pre-trained generative model and obtain a trained multi-task classification model.

[0091] Optionally, the pre-trained generative model can be trained based on the task answers and the actual answers to obtain a trained multi-task model, including:

[0092] Calculate the second loss of the generative model based on the task answers and the actual answers;

[0093] The parameters of the encoder and decoder are corrected in reverse using gradient descent until the second loss of the generative model converges, resulting in a well-trained multi-task model.

[0094] Specifically, a second loss of the generative model can be calculated based on the task answer and the real answer to characterize the degree of difference between the task answer and the real answer, and then to characterize the answer generation effect of the generative model on the sub-tasks of each task category, so as to measure the accuracy of the generative model.

[0095] Since a larger second loss indicates a greater difference between the task answer and the true answer, the parameters of the encoder and decoder can be corrected in reverse using gradient descent until the second loss of the generative model converges, resulting in a well-trained multi-task model. This allows the multi-task model to uniformly construct sub-tasks of different task categories into a generative task and accurately generate task answers for each task category, thereby improving the accuracy of completing sub-tasks of different task categories.

[0096] Optionally, a second loss for the generative model is calculated based on the task answer and the true answer, including:

[0097] When the task answer of each preset text matches the actual answer, the loss value corresponding to the preset text is determined to be 0;

[0098] When the task answer for each preset text is inconsistent with the actual answer, the loss value corresponding to the preset text is determined to be 1;

[0099] Calculate the mean of the loss values ​​corresponding to all preset texts, and determine the mean as the second loss.

[0100] Since both the task answer and the real answer are one of the corresponding preset candidate answers, there are two kinds of relationships between the task answer and the real answer: consistent and inconsistent. When determining the degree of difference between the task answer and the real answer, we can divide them into two cases.

[0101] Specifically, when the task answer of each preset text matches the actual answer, the loss value corresponding to the preset text is determined to be 0; when the task answer of each preset text does not match the actual answer, the loss value corresponding to the preset text is determined to be 1.

[0102] The mean of the loss values ​​corresponding to all preset texts is calculated, which can be used to characterize the generation effect of the generation model in this training process. The mean is then determined as the second loss to guide the training process of the generation model.

[0103] The above steps involve obtaining the real answers to each preset text, retraining the pre-trained generative model based on the task answers and the real answers to obtain a trained multi-task classification model. The generation effect of the generative model was measured based on the real answers to each preset text, and the parameters of the generative model were further corrected. This resulted in a trained multi-task classification model, which effectively improved the answer generation effect of the multi-task model for different types of tasks.

[0104] This invention improves the generation capability of a pre-trained generative model by splitting each preset text into a first sub-text and a second sub-text, using the first sub-text as a training sample and the second sub-text as a training label, and training the preset generative model with unsupervised corpus. Furthermore, by substituting preset sentences into M preset question templates under N task categories, several preset task questions are obtained. Each preset task question and its corresponding preset candidate answer template are then concatenated to obtain several preset task texts. This unifies the sub-tasks of different task categories into a single generative task, solving the problem of individual sub-tasks in multi-task models. The problem of significant differences in performance during independent training is addressed by inputting various preset task texts into a pre-trained generative model to obtain the task answers for each preset task text. The pre-trained generative model is then retrained based on the task answers and the actual answers to obtain a trained multi-task classification model. Compared to existing technologies, this eliminates the cumbersome modeling process of first building multiple different sub-task models and then integrating them into a unified multi-task model. Instead, it directly trains a multi-task classification model that adapts to multiple task classifications through the above dual training process, simplifying the model complexity. This not only adapts to different task classifications in multiple business scenarios but also effectively improves the accuracy of the multi-task model.

[0105] Corresponding to the multi-task model training method in the above embodiments, Figure 5 A structural block diagram of the multi-task model training device based on artificial intelligence provided in Embodiment 2 of the present invention is given. For ease of explanation, only the parts related to the embodiments of the present invention are shown.

[0106] See Figure 5 The multi-task model training device includes:

[0107] The model pre-training module 51 is used to obtain preset texts in the preset corpus, split each preset text into a first sub-text and a second sub-text, use the first sub-text as a training sample and the second sub-text as a training label to train the preset generation model and obtain the pre-trained generation model.

[0108] The task text composition module 52 is used to obtain preset statements, M preset question templates under N task categories and corresponding preset candidate answer templates from the preset database, substitute the preset statements into the M preset question templates under N task categories to obtain several preset task questions, and concatenate each preset task question and the corresponding preset candidate answer template to obtain several preset task texts, where N and M are positive integers;

[0109] The task answer generation module 53 is used to input each preset task text into the pre-trained generation model to obtain the task answer for each preset task text;

[0110] The model retraining module 54 is used to obtain the real answers to each preset text, and to retrain the pre-trained generative model based on the task answers and the real answers to obtain the trained multi-task classification model.

[0111] Optionally, the above-mentioned model pre-training module 51 includes:

[0112] The text segmentation module is used to segment each preset text into sentences, resulting in K preset text sentences;

[0113] The text splitting submodule is used to split K preset text statements into K1 first sub-texts and K2 second sub-texts according to a preset splitting ratio, where K1 + K2 = K, K2 <K1。

[0114] Optionally, the above-mentioned model pre-training module 51 includes:

[0115] The model structure determination submodule is used to determine the generated model, including the encoder and decoder;

[0116] The feature extraction submodule is used to input the first subtext into the encoder for feature extraction to obtain text semantic features;

[0117] The feature decoding submodule is used to input the semantic features of the text into the decoder to obtain the generated subtext;

[0118] The first loss calculation submodule is used to calculate the first loss of the generation model based on the generated subtext and the second subtext.

[0119] The first parameter correction submodule is used to correct the parameters of the encoder and decoder in reverse according to the gradient descent method until the first loss converges, thus obtaining the pre-trained generative model.

[0120] Optionally, the aforementioned first loss calculation submodule includes:

[0121] The text vector conversion unit is used to convert the generated subtext into a generated subtext vector according to word vector technology, and to convert the second subtext into a second subtext vector according to word vector technology;

[0122] The first loss calculation unit is used to calculate the degree of difference between the generated sub-text vector and the second sub-text vector, and to determine the first loss of the generation model based on the degree of difference.

[0123] Optionally, the task text component module 52 mentioned above includes:

[0124] The task category determination submodule is used for N groups of task categories, including at least one of semantic matching tasks, intent recognition tasks, and named entity recognition tasks.

[0125] The statement quantity determination submodule is used to determine the number of statements S that need to be filled in each preset question template under each task category, where S is a positive integer;

[0126] The task text composition submodule is used to select S preset statements from the database corresponding to each preset question template of each task category, based on different task categories, and substitute them into M preset question templates to obtain several preset task questions.

[0127] Optionally, the above model retraining module 54 includes:

[0128] The second loss calculation submodule is used to calculate the second loss of the generative model based on the task answer and the true answer.

[0129] The second parameter correction submodule is used to correct the parameters of the encoder and decoder in reverse according to the gradient descent method until the second loss of the generation model converges, thus obtaining the trained multi-task model.

[0130] Optionally, the aforementioned second loss calculation submodule includes:

[0131] The first loss value calculation unit is used to determine that the loss value corresponding to each preset text is 0 when the task answer and the actual answer are consistent.

[0132] The second loss value calculation unit is used to determine the loss value of the preset text as 1 when the task answer and the actual answer of each preset text are inconsistent.

[0133] The second loss calculation unit is used to calculate the mean of the loss values ​​corresponding to all preset texts, and determine the mean as the second loss.

[0134] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0135] Figure 6 This is a schematic diagram of the structure of a computer device provided in Embodiment 3 of the present invention. Figure 6 As shown, the computer device of this embodiment includes: at least one processor ( Figure 6 Only one is shown in the diagram), a memory, and a computer program stored in the memory and capable of running on at least one processor, which, when executed by the processor, implements the steps in any of the above-described multi-task model training method embodiments.

[0136] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 6The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0137] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0138] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of a computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0139] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the functions described above can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0140] The present invention can implement all or part of the processes in the methods of the above embodiments, or it can be accomplished by a computer program product. When the computer program product is run on a computer device, the computer device executes the steps in the above method embodiments.

[0141] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0142] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0143] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0144] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0145] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A multi-task model training method based on artificial intelligence, characterized in that, The multi-task model training method includes: Obtain preset texts from a preset corpus, split each preset text into a first sub-text and a second sub-text, use the first sub-text as a training sample and the second sub-text as a training label to train a preset generation model, and obtain a pre-trained generation model. Obtain a preset statement, M preset question templates under N task categories, and corresponding preset candidate answer templates from a preset database. Substitute the preset statement into the M preset question templates under the N task categories to obtain several preset task questions. Concatenate each preset task question with the corresponding preset candidate answer template to obtain several preset task texts, where N and M are positive integers, and M>N. Each of the preset task texts is input into the pre-trained generative model to obtain the task answers for each of the preset task texts; Obtain the actual answers to each of the preset texts, and retrain the pre-trained generative model based on the task answers and the actual answers to obtain a trained multi-task classification model. The process of substituting the preset statement into the M preset question templates under the N groups of task categories yields several preset task questions, including: The N groups of task categories include at least one of the following: semantic matching task, intent recognition task, and named entity recognition task; Determine the number of statements S that need to be filled in each preset question template under each task category, where S is a positive integer; Based on different task categories, S preset statements corresponding to each preset question template of each task category are selected from the database, and then substituted into M preset question templates to obtain the plurality of preset task questions. The step of retraining the pre-trained generative model based on the task answer and the real answer to obtain a trained multi-task model includes: Calculate the second loss of the generative model based on the task answer and the actual answer; The parameters of the encoder and decoder are corrected in reverse using the gradient descent method until the second loss of the generative model converges, thus obtaining the trained multi-task model. The calculation of the second loss of the generative model based on the task answer and the true answer includes: When the task answer of each of the preset texts is consistent with the actual answer, the loss value corresponding to the preset text is determined to be 0; When the task answer of each of the preset texts is inconsistent with the actual answer, the loss value corresponding to the preset text is determined to be 1; Calculate the mean of the loss values ​​corresponding to all the preset texts, and determine the mean as the second loss.

2. The multi-task model training method according to claim 1, characterized in that, The step of splitting each of the preset texts into a first sub-text and a second sub-text includes: Each of the preset texts is segmented into sentences to obtain K preset text statements; According to the preset splitting ratio, the K preset text statements are split into The first subtext and A second subtext, in which... , and All are positive integers, and , .

3. The multi-task model training method according to claim 1, characterized in that, The step of training a pre-defined generation model using the first sub-text as a training sample and the second sub-text as a training label to obtain a pre-trained generation model includes: The generative model includes an encoder and a decoder; The first sub-text is input into the encoder for feature extraction to obtain text semantic features; The semantic features of the text are input into the decoder to obtain the generated subtext; Calculate the first loss of the generation model based on the generated subtext and the second subtext; The parameters of the encoder and decoder are corrected in reverse using the gradient descent method until the first loss converges, thus obtaining the pre-trained generative model.

4. The multi-task model training method according to claim 3, characterized in that, The step of calculating the first loss of the generation model based on the generated subtext and the second subtext includes: The generated subtext is converted into a generated subtext vector according to word vector technology, and the second subtext is converted into a second subtext vector according to word vector technology; Calculate the degree of difference between the generated subtext vector and the second subtext vector, and determine the degree of difference to determine the first loss of the generation model.

5. A multi-task model training device based on artificial intelligence, characterized in that, The multi-task model training device includes: The model pre-training module is used to obtain preset texts in a preset corpus, split each preset text into a first sub-text and a second sub-text, use the first sub-text as a training sample and the second sub-text as a training label to train the preset generation model, and obtain the pre-trained generation model. The task text composition module is used to obtain a preset statement, M preset question templates under N task categories and corresponding preset candidate answer templates from a preset database, substitute the preset statement into the M preset question templates under the N task categories to obtain a number of preset task questions, and concatenate each preset task question and the corresponding preset candidate answer template to obtain a number of preset task texts, where N and M are positive integers and M>N; The task answer generation module is used to input each of the preset task texts into the pre-trained generation model to obtain the task answers for each of the preset task texts; The model retraining module is used to obtain the real answers for each of the preset texts, and to retrain the pre-trained generative model based on the task answers and the real answers to obtain a trained multi-task classification model. The task text component module includes: The task category determination submodule is used to determine at least one of the N groups of task categories: semantic matching task, intent recognition task, and named entity recognition task. The statement quantity determination submodule is used to determine the number S of statements that need to be filled in each preset question template under each task category, where S is a positive integer; The task text composition submodule is used to select S preset statements corresponding to each preset question template of each task category in the database according to different task categories, and substitute them into M preset question templates to obtain the plurality of preset task questions. The model retraining module includes: The second loss calculation submodule is used to calculate the second loss of the generated model based on the task answer and the real answer; The second parameter correction submodule is used to correct the parameters of the encoder and decoder in reverse according to the gradient descent method until the second loss of the generative model converges, thus obtaining the trained multi-task model. The second loss calculation submodule includes: The first loss value calculation unit is used to determine that the loss value corresponding to the preset text is 0 when the task answer of each preset text is consistent with the actual answer; The second loss value calculation unit is used to determine the loss value corresponding to the preset text as 1 when the task answer and the actual answer of each preset text are inconsistent. The second loss calculation unit is used to calculate the mean of the loss values ​​corresponding to all the preset texts, and determine the mean as the second loss.

6. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-task model training method as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the multi-task model training method as described in any one of claims 1 to 4.

Images