Model training method and device, equipment and storage medium

Abstract

The invention discloses a model training method and device, equipment and a storage medium. The method comprises the steps of performing autoregression training on a first model based on a first dialogue text to obtain a second model; through a second model, based on the context of each word in the second dialogue text, predicting first word probability distribution corresponding to the position where each word is located; based on each word in the second dialogue text, first word probability distribution corresponding to the position of each word and a process node to which each word belongs in the preset dialogue process, determining a to-be-optimized first process node in the preset dialogue process; based on a flow node to which each word in the first dialogue text belongs in the first dialogue text, masking words which do not belong to the first flow node in the first dialogue text to obtain a third dialogue text; and performing autoregression training on the second model based on the third dialogue text to obtain a third model applied to the preset dialogue process.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model training method, apparatus, device and storage medium. Background Art

[0002] When training models for multi-turn, task-based conversation scenarios, the coverage of the training data for different nodes in the conversation process and the difficulty of model training vary, resulting in different performance of the trained model at different nodes. Therefore, improving the model's performance at weak nodes becomes a challenge. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to provide a model training method, apparatus, equipment and storage medium to specifically enhance the capabilities of the model on process nodes with weak performance, and avoid negative interference problems such as overtraining and overfitting on process nodes with strong performance.

[0004] In order to achieve the above objectives, the embodiments of the present application adopt the following technical solutions: In a first aspect, an embodiment of the present application provides a model training method, comprising: Performing autoregressive training on the first model based on the first conversation text to obtain a second model; Using the second model, based on the context of each word in the second conversation text, predict the probability distribution of the first word corresponding to the position of each word; Determining a first process node to be optimized in the preset dialogue process based on each word in the second dialogue text, the probability distribution of the first word corresponding to the position of each word, and the process node to which each word belongs in the preset dialogue process; Based on the process node to which each word in the first dialogue text belongs in the first dialogue text, masking the words in the first dialogue text that do not belong to the first process node to obtain a third dialogue text; The second model is subjected to autoregressive training based on the third dialogue text to obtain a third model applied to the preset dialogue process.

[0005] In a second aspect, an embodiment of the present application provides a model training device, comprising: A training module, configured to perform autoregressive training on the first model based on the first conversation text to obtain a second model; a prediction module, configured to predict, by using the second model and based on the context of each word in the second conversation text, a probability distribution of the first word corresponding to the position of each word; a determination module configured to determine a first process node to be optimized in the preset dialogue process based on each word in the second dialogue text, the probability distribution of the first word corresponding to the position of each word, and the process node to which each word belongs in the preset dialogue process; a masking module configured to mask, based on the process node to which each word in the first dialogue text belongs in the first dialogue text, words in the first dialogue text that do not belong to the first process node to obtain a third dialogue text; The training module is further used to perform autoregressive training on the second model based on the third dialogue text to obtain a third model applied to the preset dialogue process.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, including: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the model training method provided in the first aspect.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute the model training method provided in the first aspect.

[0008] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: Based on the first dialogue text, the first model is trained with autoregression to obtain the second model, and the second model is used to predict the first word probability distribution corresponding to the position of each word based on the context of each word in the second dialogue text; for each process node in the preset dialogue process, the difference between the words belonging to the process node in the second dialogue text and the first word probability distribution corresponding to the position of these words can reflect the performance of the second model on the process node. Therefore, based on each word in the second dialogue text, the first word probability distribution corresponding to the position of each word, and the process node to which each word belongs in the preset dialogue process, the process node with weak performance in the preset process node can be accurately located as The first process node to be optimized; further, based on the process node to which each word in the first dialogue text belongs in the first dialogue text, the words that do not belong to the first process node in the first dialogue text are masked to obtain a third dialogue text, and the second model is trained on autoregression based on the third dialogue text, so that the model focuses on the content of the process nodes with weak performance in the sample dialogue text, thereby specifically enhancing the ability of the model on the process nodes with weak performance, and avoiding negative interference problems such as overtraining and overfitting on the process nodes with strong performance. The third model thus obtained has good performance on all process nodes in the preset dialogue process and can be more accurately applied to various dialogue processing tasks under the preset dialogue process. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A flowchart of a model training method provided for one embodiment of the present application; Figure 2 A flowchart of a model training method provided in another embodiment of the present application; Figure 3 A schematic diagram of the structure of a model training device provided in one embodiment of the present application; Figure 4 A schematic structural diagram of an electronic device provided in accordance with an embodiment of the present application. DETAILED DESCRIPTION

[0010] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0011] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0012] As mentioned earlier, when training models for multi-round task-based dialogue scenarios, the performance of the trained model at different process nodes in the dialogue process varies due to factors such as data distribution and training difficulty. For example, the performance at some process nodes may be under-trained.

[0013] In related technologies, models are typically retrained by adding conversation data covering weaker process nodes. However, after extensive research, the inventors discovered that this approach is a global training method, retraining the model's capabilities across all process nodes. This can lead to negative interference issues such as overtraining and overfitting in previously strong process nodes.

[0014] In view of this, the present invention proposes a model training method that, based on feedback from different process nodes, conducts targeted enhancement training on weaker-performing process nodes. This method employs a two-stage training mechanism. In the first stage, the model is trained with autoregressive training using sample conversation text. The trained model's performance at different process nodes is then evaluated to accurately identify process nodes with weak performance for the model. In the second stage, the model is trained again with autoregressive training using the masked sample text, masking the words in the sample conversation text that belong to strong-performing process nodes. This allows the model to focus on the content of weak-performing process nodes in the sample conversation text, thereby specifically enhancing the model's capabilities on weak-performing process nodes and avoiding negative interference problems such as overtraining and overfitting on strong-performing process nodes.

[0015] It should be understood that the model training method proposed in the embodiments of the present application can be executed by an electronic device. As an example, it can be executed by software in the electronic device. The so-called electronic devices here can include terminal devices, such as smartphones, tablet computers, laptops, desktop computers, intelligent voice interaction devices, smart home appliances, smart watches, vehicle terminals, aircraft, etc.; or, the electronic device can also include a server, such as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0016] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0017] Please refer to Figure 1 , is a flow chart of a model training method provided in one embodiment of the present application, the method comprising the following steps: S102: Perform autoregressive training on the first model based on the first conversation text to obtain a second model.

[0018] The first conversation text may be a historical conversation text used as a training sample. There may be multiple first conversation texts, and the specific number can be set based on actual needs and is not limited in this embodiment of the present application. The first conversation text has corresponding label data, which may include the process node to which each sentence in the first conversation text belongs in a preset conversation process. Based on this, the process node to which each sentence belongs is the process node to which each word in the sentence belongs.

[0019] A conversation flow refers to the steps or operations followed during multiple rounds of conversation. This flow can be a standard operating procedure (SOP) set based on business requirements. A conversation flow consists of multiple process nodes, each representing a step or operation followed during the conversation. For example, in some task-based conversations, the conversation flow might include opening remarks, event introduction, store invitation, contact information addition, and closing remarks.

[0020] In practical applications, we can combine the preset dialogue process with a model with annotation capabilities, such as a large language model (LLM), to annotate the corresponding process nodes for each sentence in the first dialogue text according to dialogue turns and roles (such as agent and customer). The process nodes corresponding to these sentences serve as the label data of the first dialogue text.

[0021] The first model can be any model suitable for processing dialogue text, such as large language models such as Qwen2.5-7b, etc., and this embodiment of the present application is not limited to this.

[0022] Autoregressive training is a modeling method based on sequential data. Its core concept is to gradually predict future data by leveraging historical information. In S102 above, first, a first model is used to predict the word probability distribution corresponding to each word's position in the first conversation text based on the context of each word. Then, the loss of the first model is determined based on each word and the word probability distribution corresponding to each word's position. Finally, the parameters of the first model are adjusted based on this loss to obtain a second model. For the i-th word in the first conversation text, its context includes the preceding i-1 words in the first conversation text, and the word probability distribution corresponding to its position includes the probability of each word in the vocabulary appearing at the i-th word's position, where i is a positive integer.

[0023] For example, an input-output pair is constructed based on the first conversation text. The input may include a first prompt word and the complete first conversation text. The first prompt word indicates the format of the first conversation text and the task requirement for answering based on the first conversation text. This helps control the model's generation scope, especially when the task requirements are complex and require a specific output format, such as providing both intent and dialogue in JSON format. This effectively prevents the model's generation scope from becoming uncontrollable. The output may also include the first prompt word and the first conversation text. The output can serve as a reference, providing a supervisory signal for the autoregressive training of the first model.

[0024] In order to distinguish between input and output, special marks are used in the input.<im_start> As the starting mark, with a special mark<im_end> as the end marker; the start marker is not included in the output<im_start> An example of an input-output pair is shown below: data_i={ “input”:“<im_start> system<im_end> <im_start> You are an intelligent assistant. Please answer the questions based on the given conversation content.<im_end> <im_start> user<im_end> <im_start> Hello, who is this?<im_end> <im_start> assistant<im_end> <im_start> Hello, I'm a sales representative from Dezhu Decoration Company. I'm calling to invite you to participate in our latest promotion. Would you be able to come and see us tomorrow?<im_end> <im_start> user<im_end> <im_start> Can<im_end> <im_start> assistant<im_end> <im_start> OK, I'll add your WeChat.<im_end> ”, “output”:“system<im_end> <im_start> You are an intelligent assistant. Please answer the questions based on the given conversation content.<im_end> <im_start> user<im_end> <im_start> Hello, who is this?<im_end> <im_start> assistant<im_end> <im_start> Hello, I'm a sales representative from Dezhu Decoration Company. I'm calling to invite you to participate in our latest promotion. Would you be able to come and see us tomorrow?<im_end> <im_start> user<im_end> <im_start> Can<im_end> <im_start> assistant<im_end> <im_start> OK, I'll add your WeChat.<im_end> ” } Furthermore, an autoregressive training mechanism is used, where the input of each time step t is the first t-1 words (tokens), and the prediction target is the tth word, forming a sliding window. For example, at the first time step,<im_start> Input the first model, and the first model predicts the word probability distribution corresponding to the position of system; at the second time step,<im_start> The system inputs the first model and the first model predicts<im_end> The probability distribution of the word corresponding to the position; and so on, until the first model completes the prediction of all words in the above input.

[0025] Furthermore, for each word in the first conversation text, the cross-entropy loss is calculated based on the word probability distribution of the word and its position, representing the difference between the prediction results for the word and its position. The cross-entropy losses for all words in the first conversation text are averaged or summed to obtain the loss of the first model. Finally, minimizing the loss of the first model is optimized using gradient descent algorithms, backpropagation algorithms, and other algorithms to adjust the parameters of the first model. This completes supervised fine-tuning (SFT) of the first model, and the trained first model is then determined as the second model.

[0026] S104: Using the second model, based on the context of each word in the second dialogue text, predict the probability distribution of the first word corresponding to the position of each word.

[0027] The second conversation text may be a historical conversation text serving as a test sample. There may be multiple second conversation texts, the specific number of which can be set based on actual needs and is not limited in this embodiment of the present application. The second conversation text also has corresponding label data, which may include the process node to which each sentence in the second conversation text belongs in a preset conversation process. Based on this, the process node to which each sentence in the second conversation text belongs is the process node to which each word in the sentence belongs.

[0028] In actual applications, we can combine the preset dialogue process and use models with annotation capabilities, such as LLM, to annotate the corresponding process nodes for each sentence in the second dialogue text according to the dialogue turns, roles (such as agent, customer service), etc. The process nodes corresponding to these sentences serve as the label data of the second dialogue text.

[0029] In S104 above, the second model can also use an autoregressive training mechanism to predict the first word probability distribution corresponding to each word position in the second dialogue text. Assume that the second dialogue text includes n words, where n is an integer greater than 1. For the i-th word, its context includes the first i-1 words in the second dialogue text, and the first word probability distribution corresponding to its position includes the probability of each word in the vocabulary appearing at the position of the i-th word, where i is a positive integer and 1≤i≤n.

[0030] For example, an input-output pair is constructed based on the second dialogue text. The input may include a second prompt word and the complete second dialogue text. The second prompt word indicates the format of the second dialogue text and the task requirements to be answered based on the second dialogue text. This helps control the generation range of the model, especially when the task requirements are complex and require a specific output format, effectively preventing the model generation range from being uncontrollable. The output may also include the second prompt word and the second dialogue text. The output can provide a reference for the prediction results of the second model.

[0031] Furthermore, an autoregressive training mechanism is employed, where the input for each time step t is the previous t-1 tokens, and the prediction target is the tth token, forming a sliding window. For example, at the first time step, the first token in the input is fed into the second model, which then predicts the probability distribution of the token corresponding to the second token's position. At the second time step, the first two tokens in the input are fed into the second model, which then predicts the probability distribution of the token corresponding to the third token's position. This continues until the second model has completed its predictions for all tokens in the input.

[0032] S106: Determine a first process node to be optimized in the preset dialogue process based on each word in the second dialogue text, the first word probability distribution corresponding to the position of each word, and the process node to which each word belongs in the preset dialogue process.

[0033] In one implementation, the above S106 includes the following steps: S1062: For each word in the second dialogue text, determine a first loss corresponding to the word based on the word and the first word probability distribution corresponding to the position of the word.

[0034] For each word in the second conversation text, the cross entropy loss for that word is calculated based on the probability distribution of the word and the first word corresponding to its position. This cross entropy loss is determined as the first loss for that word. The first loss for a word represents the difference between the model's prediction and the actual result for that word.

[0035] S1064: Determine a second loss corresponding to each process node in a preset dialogue process based on the first loss corresponding to each word in the second dialogue text and the process node to which each word belongs.

[0036] For each process node, the second loss corresponding to the process node reflects the performance of the second model at that process node. The smaller the second loss corresponding to the process node, the better the performance of the second model at that process node; conversely, the larger the second loss corresponding to the process node, the worse the performance of the second model at that process node.

[0037] As an example, for each process node in a preset dialogue process, a word set in the second dialogue text belonging to the process node is determined; and a first operation is performed on the first loss corresponding to each word in the word set to obtain a second loss corresponding to the process node.

[0038] The first operation can be performed according to actual needs, such as taking the average or sum.

[0039] For example, for a certain process node, the word set belonging to the process node in the second dialogue text includes word 1 to word m, then the second loss corresponding to each of word 1 to word m is averaged to obtain the second loss corresponding to the process node.

[0040] S1066: Determine a first process node to be optimized in the preset dialogue process based on the second loss corresponding to each process node.

[0041] As an example, a first mean is determined based on the second loss corresponding to each process node and the number of process nodes included in the preset dialogue process; and the process node in the preset dialogue process whose corresponding second loss is greater than the first mean is determined as the first process node to be optimized.

[0042] For example, the first mean = the sum of the second losses corresponding to all process nodes / the number of process nodes included in the preset dialogue process.

[0043] Therefore, the first mean can reflect the average level of the second model in the entire dialogue process. If the second loss corresponding to a process node is greater than the first mean, it means that the performance of the second model on this process node is lower than the average level, and the process node can be determined as the first process node to be optimized; if the second loss corresponding to a process node is less than or equal to the first mean, it means that the performance of the second model on this process node is not lower than the average level, and the process node can be determined as the second process node that does not need to be optimized. Therefore, based on the size relationship between the second loss and the first mean corresponding to each process node, the process nodes to be optimized in the preset dialogue process can be more accurately located, thereby providing reliable data support for targeted enhancement of the model's capabilities at weak-performing process nodes.

[0044] In another implementation, since the second loss corresponding to each process reflects the performance of the second model at that process node, the process node in the preset dialogue process whose corresponding second loss is greater than a loss threshold can be determined as the first process node to be optimized. The loss threshold can be set according to actual needs. For example, to avoid omissions, the loss threshold can be set to a smaller value, or to improve training efficiency, the loss threshold can be set to a larger value, etc. This embodiment of the application is not limited to this.

[0045] The above describes some implementations of the above S106. Of course, it should be understood that the above S106 can also be implemented in other ways, which are not limited in the present embodiment.

[0046] S108: Based on the process node to which each word in the first dialogue text belongs in the first dialogue text, mask the words in the first dialogue text that do not belong to the first process node to obtain a third dialogue text.

[0047] The words in the first dialogue text that do not belong to the first process node may be replaced with special identifiers, or the index number of the word in the vocabulary may be replaced with a special identifier, such as -100.

[0048] For example, when training the second model using input-output pairs constructed based on the first conversation text, the first conversation text in the input can be left unprocessed. Instead, the words in the first conversation text in the output that belong to the first process node can be masked to generate new input-output pairs. These new input-output pairs can serve as new training samples for retraining the second model. This allows the model to focus on the content of weaker process nodes in the sample conversation text during training, thereby specifically strengthening the model's capabilities in these weaker process nodes and avoiding negative interference such as overtraining and overfitting on stronger process nodes.

[0049] S110, performing autoregressive training on the second model based on the third dialogue text to obtain a third model applied to a preset dialogue process.

[0050] The specific implementation method of performing autoregressive training on the second model based on the third dialogue text is similar to the specific implementation method of S102 described above, except that the words belonging to the first process node are masked, and thus these words will not affect the parameter update of the second model, thereby achieving targeted local parameter adjustment of the second model and avoiding over-training and over-fitting problems in other process nodes.

[0051] Specifically, in one implementation, the above S110 includes the following steps: S1102: Using a second model, based on the context of each word in the first conversation text, predict a second word probability distribution corresponding to the position of each word.

[0052] For each word in the first conversation text, its context includes the words preceding it, and the second word probability distribution corresponding to its position includes the probability of each word in the vocabulary appearing at that position. For example, assume the first conversation text includes n words, where n is an integer greater than 1. For the i-th word, its context includes the preceding i-1 words, and the second word probability distribution corresponding to its position includes the probability of each word in the vocabulary appearing at the i-th word's position, where i is a positive integer and 1≤i≤n.

[0053] Specifically, the second model uses an autoregressive training mechanism. The input for each time step t is the first t-1 tokens in the first conversation text, and the prediction target is the tth token, forming a sliding window. For example, at the first time step, the first token in the input is fed into the second model, which predicts the token probability distribution corresponding to the second token's position. At the second time step, the first two tokens in the input are fed into the second model, which predicts the token probability distribution corresponding to the third token's position. This continues until the second model has predicted all tokens in the input, yielding the second token probability distribution corresponding to each token's position.

[0054] S1104: Based on the second word probability distribution corresponding to the position of each word in the third dialogue text and the first dialogue text, adjust the parameters of the second model to obtain a third model applied to the preset dialogue process.

[0055] As an example, for each unmasked word in the third dialogue text, the third loss corresponding to the word is determined based on the second word probability distribution corresponding to the word and the position of the word in the first dialogue text; based on the third loss corresponding to each unmasked word in the third dialogue text, the parameters of the second model are adjusted to obtain a third model applied to the preset dialogue process.

[0056] Exemplarily, for each word in the third dialogue text, if the word is a special identifier or the index number of the word in the vocabulary is a special identifier, it is determined that the word is masked, and then the word is ignored; if the word is not a special word and the index number of the word in the vocabulary is not a special identifier, it is determined that the word is not masked, and then based on the word and the probability distribution of the second word corresponding to the position of the word, the cross entropy loss is determined, and the cross entropy loss is determined as the third loss corresponding to the word; further, the sum of the third losses corresponding to all unmasked words in the third dialogue text is determined as the total loss of the second model; finally, reducing the total loss of the second model is taken as the optimization goal, and the parameters of the second model are adjusted using a gradient descent algorithm, a back propagation algorithm, etc. to obtain the third model.

[0057] Through the above implementation method, it is equivalent to taking the first dialogue text as input and the third dialogue text as the expected output or reference, so that the words belonging to the first process node in the first dialogue text no longer participate in the parameter adjustment of the second model, so that the second model focuses on the performance on the first process node and ignores the performance on other process nodes, thereby achieving targeted local parameter adjustment of the second model and avoiding over-training and over-fitting problems in other process nodes.

[0058] In the application, during the autoregressive training of the second model based on the third conversation text, the parameters of the second model related to the first process node can also be frozen to control the parameter update range, avoid negative interference problems (such as overfitting) in global retraining, and reduce the model adjustment cost of business changes.

[0059] Steps S104 to S110 above only perform a single targeted reinforcement training process for the second model. In practice, after a single targeted reinforcement training, the second model may still be undertrained at some process nodes. To address this, the second model can be trained multiple times until its performance at all process nodes meets the requirements.

[0060] Specifically, in another embodiment, after the above S110, it also includes: using the third model as a new second model, repeatedly executing the above S104 to S110 multiple times, and stopping when there is no first process node to be optimized in the preset dialogue process.

[0061] For ease of understanding, the following Figure 2 For explanation. Figure 2 As shown, first, a large amount of historical conversation texts are collected, and based on the preset conversation process, each sentence in the historical conversation text is marked with the process node to which it belongs, to obtain a dataset. The dataset includes multiple historical conversation texts and the process node to which each sentence in the historical conversation text belongs.

[0062] Then, these historical conversation texts are divided into a training set and a test set according to a certain ratio (e.g., 8:2). The historical conversation texts in the training set are called the first conversation texts and are used for model training. The historical conversation texts in the test set are called the second conversation texts and are used to test the performance of the trained model at each process node in the preset conversation process to identify process nodes with weak performance.

[0063] Furthermore, the first model is trained with autoregression based on the first dialogue text in the training set to obtain a second model.

[0064] Furthermore, the second dialogue text in the test set is processed, such as by constructing input-output pairs and masking the prompt words in the output. The processed second dialogue text is then used to test the performance of the second model at each process node in the preset dialogue flow. Process nodes with poor performance are identified as first process nodes to be optimized, and these first process nodes are added to a list of nodes to be optimized. Specifically, the second model predicts the first word probability distribution corresponding to each word's position based on the context of each word in the second dialogue text. Based on each word in the second dialogue text, the first word probability distribution corresponding to each word's position, and the process node to which each word belongs in the preset dialogue flow, the first process node to be optimized in the preset dialogue flow is determined.

[0065] Furthermore, if the list to be optimized is empty, it means that the second model performs well on all process nodes, and there is no need to perform targeted enhancement training on the second model, and the operation is terminated.

[0066] If the list to be optimized is not empty, masking is performed on the words belonging to the first process node in the first dialogue text to obtain a third dialogue text, and autoregressive training is performed on the second model based on the third dialogue text to obtain a third model.

[0067] The third model is used as the new second model, and the above test process is repeated until the list to be optimized is empty.

[0068] The model training method provided in the embodiment of the present application performs autoregressive training on the first model based on the first dialogue text to obtain a second model, and uses the second model to predict the first word probability distribution corresponding to the position of each word based on the context of each word in the second dialogue text; for each process node in the preset dialogue process, the difference between the words belonging to the process node in the second dialogue text and the first word probability distribution corresponding to the position of these words can reflect the performance of the second model on the process node. Therefore, based on each word in the second dialogue text, the first word probability distribution corresponding to the position of each word, and the process node to which each word belongs in the preset dialogue process, the performance of the preset process node can be accurately located. The weak process nodes are used as the first process nodes to be optimized; further, based on the process nodes to which each word in the first dialogue text belongs in the first dialogue text, the words that do not belong to the first process nodes in the first dialogue text are masked to obtain a third dialogue text, and the second model is trained on autoregression based on the third dialogue text, so that the model focuses on the content of the weak process nodes in the sample dialogue text, thereby specifically enhancing the model's ability on the weak process nodes and avoiding negative interference problems such as overtraining and overfitting on the strong process nodes. The third model thus obtained has good performance on all process nodes in the preset dialogue process and can be more accurately applied to various dialogue processing tasks under the preset dialogue process.

[0069] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0070] Based on the same inventive concept, this embodiment of the application also provides a model training device. Figure 3 , is a structural diagram of a model training device 300 provided in an embodiment of the present application, wherein the device 300 includes: a training module 310, a prediction module 320, a determination module 330 and a mask module 340.

[0071] The training module 310 is used to perform autoregressive training on the first model based on the first conversation text to obtain a second model.

[0072] The prediction module 320 is configured to predict the first word probability distribution corresponding to the position of each word based on the context of each word in the second dialogue text using the second model.

[0073] The determination module 330 is used to determine the first process node to be optimized in the preset dialogue process based on each word in the second dialogue text, the first word probability distribution corresponding to the position of each word, and the process node to which each word belongs in the preset dialogue process.

[0074] The masking module 340 is configured to mask the words in the first dialogue text that do not belong to the first process node based on the process node to which each word in the first dialogue text belongs in the first dialogue text to obtain a third dialogue text.

[0075] The training module 350 is used to perform autoregressive training on the second model based on the third dialogue text to obtain a third model applied to the preset dialogue process.

[0076] In another embodiment, the determining module is configured to: For each word in the second dialogue text, determining a first loss corresponding to the word based on the word and a first word probability distribution corresponding to the position of the word; Determining a second loss corresponding to each process node in the preset dialogue process based on the first loss corresponding to each word in the second dialogue text and the process node to which each word belongs; Based on the second loss corresponding to each process node, a first process node to be optimized in the preset dialogue process is determined.

[0077] In another embodiment, the determining module performs the following steps when determining the second loss corresponding to each process node in the preset dialogue process based on the first loss corresponding to each word in the second dialogue text and the process node to which each word belongs: For each process node in the preset dialogue process, determining a word set in the second dialogue text belonging to the process node; Furthermore, a first operation is performed on the first loss corresponding to each word in the word set to obtain a second loss corresponding to the process node.

[0078] In another embodiment, when determining the first process node to be optimized in the preset dialogue process based on the second loss corresponding to each process node, the determining module performs the following steps: Determining a first mean based on a second loss corresponding to each process node and the number of process nodes included in the preset dialogue process; A process node in the preset dialogue process whose corresponding second loss is greater than the first average is determined as a first process node to be optimized.

[0079] In another embodiment, the training module is used to: Using the second model, based on the context of each word in the first conversation text, predict a probability distribution of a second word corresponding to the position of each word; Based on the second word probability distribution corresponding to the position of each word in the third dialogue text and the first dialogue text, the parameters of the second model are adjusted to obtain a third model applied to the preset dialogue process.

[0080] In another embodiment, the training module adjusts the parameters of the second model based on the second word probability distribution corresponding to the position of each word in the third dialogue text and the first dialogue text to obtain the third model applied to the preset dialogue process, performing the following steps: For each unmasked word in the third dialogue text, determining a third loss corresponding to the word based on the word and a second word probability distribution corresponding to the word's position in the first dialogue text; Based on the third loss corresponding to each unmasked word in the third dialogue text, the parameters of the second model are adjusted to obtain a third model applied to the preset dialogue process.

[0081] In another embodiment, the training module is further configured to use the third model as a new second model; A loop module is used to repeatedly call the training module, the determination module and the mask module to repeatedly execute the step of predicting the first word probability distribution corresponding to the position of each word based on the context of each word in the second dialogue text through the second model to the step of using the third model as the new second model until there is no first process node to be optimized in the preset dialogue process.

[0082] Obviously, the model training device provided in the embodiment of the present application can be used as Figure 1 The execution body of the model training method shown, for example Figure 1 In the model training method shown, step S102 can be performed by Figure 3 The training module 310 in the model training device shown in FIG. 1 is executed, and step S104 can be performed by Figure 3 The prediction module 320 in the model training device shown in FIG. 1 is executed, and step S106 can be performed by Figure 3 The determination module 330 in the model training device shown in FIG. 1 is executed, and step S108 can be performed by Figure 3 The mask module 340 in the model training apparatus shown in FIG. 1 is executed, and step S110 can be performed by Figure 3 The training module 310 in the model training device shown is executed.

[0083] According to another embodiment of the present application, Figure 3The various modules in the model training device shown can be individually or completely combined into one or several other modules to form a whole, or one (or more) of the modules can be further divided into multiple smaller modules to form a whole, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In actual applications, the functions of a module can also be implemented by multiple modules, or the functions of multiple modules can be implemented by one module. In an embodiment of the present application, the model training device may also include other modules. In actual applications, these modules can also be implemented with the assistance of other modules, and can be implemented by the collaboration of multiple modules.

[0084] According to another embodiment of the present application, a general computing device such as a computer including a central processing unit (CPU), a random access memory (RAM), a read-only memory (ROM) and other processing elements and storage elements can be run to execute the following operations: Figure 1 A computer program (including program code) for each step involved in the corresponding method shown in FIG. Figure 3 The model training device shown in the figure and the model training method of the embodiment of the present application are implemented. The computer program can be recorded on a computer-readable storage medium, for example, and transferred to an electronic device through the computer-readable storage medium and run therein.

[0085] Figure 4 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 4 At the hardware level, the electronic device includes a processor and, optionally, an internal bus, a network interface, and memory. The memory may include internal memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for its services.

[0086] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0087] The memory is used to store programs. Specifically, the program may include program code, which includes computer operating instructions. The memory may include internal memory and non-volatile memory, and provides instructions and data to the processor.

[0088] The processor reads the corresponding computer program from the non-volatile memory into the internal memory and then runs it, forming a model training device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations: Performing autoregressive training on the first model based on the first conversation text to obtain a second model; Using the second model, based on the context of each word in the second conversation text, predict the probability distribution of the first word corresponding to the position of each word; Determining a first process node to be optimized in the preset dialogue process based on each word in the second dialogue text, the probability distribution of the first word corresponding to the position of each word, and the process node to which each word belongs in the preset dialogue process; Based on the process node to which each word in the first dialogue text belongs in the first dialogue text, masking the words in the first dialogue text that do not belong to the first process node to obtain a third dialogue text; The second model is subjected to autoregressive training based on the third dialogue text to obtain a third model applied to the preset dialogue process.

[0089] The above application Figure 1The methods performed by the model training device disclosed in the illustrated embodiments can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be performed by hardware integrated logic circuits in the processor or by software instructions. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0090] The electronic device may also perform Figure 1 Method, and realize the model training device in Figure 1 、 Figure 2 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0091] Of course, in addition to software implementation, the electronic device of this application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0092] The embodiment of the present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by a portable electronic device including multiple application programs, can enable the portable electronic device to execute Figure 1 The method of the embodiment shown is specifically used to perform the following operations: Performing autoregressive training on the first model based on the first conversation text to obtain a second model; Using the second model, based on the context of each word in the second conversation text, predict the probability distribution of the first word corresponding to the position of each word; Determining a first process node to be optimized in the preset dialogue process based on each word in the second dialogue text, the probability distribution of the first word corresponding to the position of each word, and the process node to which each word belongs in the preset dialogue process; Based on the process node to which each word in the first dialogue text belongs in the first dialogue text, masking the words in the first dialogue text that do not belong to the first process node to obtain a third dialogue text; The second model is subjected to autoregressive training based on the third dialogue text to obtain a third model applied to the preset dialogue process.

[0093] In short, the above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

[0094] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0095] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0096] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0097] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

Claims

1. A model training method, characterized in that: include: Performing autoregressive training on the first model based on the first conversation text to obtain a second model; Using the second model, based on the context of each word in the second conversation text, predict the probability distribution of the first word corresponding to the position of each word; Determining a first process node to be optimized in the preset dialogue process based on each word in the second dialogue text, the probability distribution of the first word corresponding to the position of each word, and the process node to which each word belongs in the preset dialogue process; Based on the process node to which each word in the first dialogue text belongs in the first dialogue text, masking the words in the first dialogue text that do not belong to the first process node to obtain a third dialogue text; The second model is subjected to autoregressive training based on the third dialogue text to obtain a third model applied to the preset dialogue process.

2. The method according to claim 1, characterized in that The determining, based on each word in the second dialogue text, the probability distribution of the first word corresponding to the position of each word, and the process node to which each word belongs in the preset dialogue process, the first process node to be optimized in the preset dialogue process includes: For each word in the second dialogue text, determining a first loss corresponding to the word based on the word and a first word probability distribution corresponding to the position of the word; Determining a second loss corresponding to each process node in the preset dialogue process based on the first loss corresponding to each word in the second dialogue text and the process node to which each word belongs; Based on the second loss corresponding to each process node, a first process node to be optimized in the preset dialogue process is determined.

3. The method according to claim 2, characterized in that The determining, based on the first loss corresponding to each word in the second dialogue text and the process node to which each word belongs, the second loss corresponding to each process node in the preset dialogue process includes: For each process node in the preset dialogue process, determining a word set in the second dialogue text belonging to the process node; Furthermore, a first operation is performed on the first loss corresponding to each word in the word set to obtain a second loss corresponding to the process node.

4. The method according to claim 2, characterized in that The determining, based on the second loss corresponding to each process node, a first process node to be optimized in the preset dialogue process includes: Determining a first mean based on a second loss corresponding to each process node and the number of process nodes included in the preset dialogue process; A process node in the preset dialogue process whose corresponding second loss is greater than the first average is determined as a first process node to be optimized.

5. The method according to claim 1, wherein The performing autoregressive training on the second model based on the third dialogue text to obtain a third model applied to the preset dialogue process includes: Using the second model, based on the context of each word in the first conversation text, predict a probability distribution of a second word corresponding to the position of each word; Based on the second word probability distribution corresponding to the position of each word in the third dialogue text and the first dialogue text, the parameters of the second model are adjusted to obtain a third model applied to the preset dialogue process.

6. The method according to claim 5, characterized in that The adjusting the parameters of the second model based on the second word probability distribution corresponding to the position of each word in the third dialogue text and the first dialogue text to obtain the third model applied to the preset dialogue process includes: For each unmasked word in the third dialogue text, determining a third loss corresponding to the word based on the word and a second word probability distribution corresponding to the word's position in the first dialogue text; Based on the third loss corresponding to each unmasked word in the third dialogue text, the parameters of the second model are adjusted to obtain a third model applied to the preset dialogue process.

7. The method according to any one of claims 1 to 6, characterized in that After performing autoregressive training on the second model based on the third dialogue text to obtain a third model applied to the preset dialogue process, the method further includes: Using the third model as a new second model; Repeat the step of using the third model as a new second model to predict the first word probability distribution corresponding to the position of each word based on the context of each word in the second dialogue text through the second model until there is no first process node to be optimized in the preset dialogue process.

8. A model training method, characterized in that: include: A training module, configured to perform autoregressive training on the first model based on the first conversation text to obtain a second model; a prediction module, configured to predict, by using the second model and based on the context of each word in the second conversation text, a probability distribution of the first word corresponding to the position of each word; a determination module configured to determine a first process node to be optimized in the preset dialogue process based on each word in the second dialogue text, the probability distribution of the first word corresponding to the position of each word, and the process node to which each word belongs in the preset dialogue process; a masking module configured to mask, based on the process node to which each word in the first dialogue text belongs in the first dialogue text, words in the first dialogue text that do not belong to the first process node to obtain a third dialogue text; The training module is further used to perform autoregressive training on the second model based on the third dialogue text to obtain a third model applied to the preset dialogue process.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the model training method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is enabled to execute the model training method as described in any one of claims 1 to 7.

Images