Multi-round dialogue interaction engine training method and system based on PRE-AGI

Abstract

The present invention relates to a method and system for training a multi-round dialogue interaction engine based on Pre-AG I, and to the technical field of multi-round dialogue engines, and is applied to a multi-round dialogue interaction engine based on Pre-AG I. The method comprises: establishing a plurality of batches of training sample sets, wherein the training sample sets comprise a plurality of training samples, and the training samples at least comprise the contents of a plurality of dialogue rounds, user intentions of each dialogue round, dialogue role information, text answer information of each dialogue round, and voice answer information of each dialogue round; and using a plurality of computing nodes and a plurality of batches of training sample sets to train and optimize the multi-round dialogue interaction engine based on Pre-AG I, which has the advantage of improving the accuracy of the multi-round dialogue interaction engine.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-round dialogue engines, and in particular to a PRE-AGI-based multi-round dialogue interaction engine training method and system. Background Art

[0002] Pre-AGI (Pre-Artificial General Intelligence) is used to describe the stage or state before achieving comprehensive and general artificial intelligence (AGI). Artificial intelligence systems in the Pre-AGI stage have not yet reached the standards of AGI, but their emergence and development have had a profound impact on human society. These systems have been widely used in many fields, such as conversational interaction engines. The product forms of artificial intelligence conversation systems mainly include single-round conversations and multi-round conversations. The single-round conversation mode is that the user inputs a sentence, and the conversation robot gives an answer based on the user's input. The main scenarios are FAQs or help centers. In multi-round conversations, the conversation robot will remember the historical information of the conversation when answering, maintain a conversation state internally, and give an answer based on the conversation state and user input. The definition of multi-round conversation is to conduct continuous conversations based on the context content to achieve the purpose of solving a certain type of specific task.

[0003] At present, the related technologies often use the retrieval-based multi-round dialogue model for dialogue prediction, which selects appropriate reply sentences from the database based on the previous information in the multi-round dialogue. However, in the related technologies, when training the retrieval-based multi-round dialogue model, no connection is established between the multi-round corpus of the same dialogue, resulting in a low prediction accuracy of the trained multi-round dialogue model.

[0004] Therefore, it is necessary to provide a multi-round dialogue interaction engine training method and system based on Pre-AGI to improve the accuracy of the multi-round dialogue interaction engine. Summary of the invention

[0005] The present invention provides a PRE-AGI-based multi-round dialogue interaction engine training method, which is applied to a Pre-AGI-based multi-round dialogue interaction engine, including: establishing multiple batches of training sample sets, wherein the training sample sets include multiple training samples, and the training samples at least include multiple rounds of dialogue content, user intentions of each round of dialogue, dialogue role information, text answer information of each round of dialogue, and voice answer information of each round of dialogue; through multiple computing power nodes, using the multiple batches of training sample sets to train and optimize the Pre-AGI-based multi-round dialogue interaction engine.

[0006] Furthermore, the multi-round dialogue interaction engine based on Pre-AGI includes: an intention recognition unit, which is used to determine the user's current round intention based on the user's current input content, historical dialogue content and the user's historical round intentions; an answer generation unit, which is used to generate a current round text answer based on the user's current round intention through a large language model, a knowledge base and a fixed speech template library; a text adjustment unit, which is used to adjust the current round text answer based on role information to generate a current round text answer that conforms to the role information; and a speech synthesis unit, which is used to generate a speech answer that conforms to the role information corresponding to the current round text answer that conforms to the role information based on the role information.

[0007] Furthermore, the training and optimization of the Pre-AGI-based multi-round dialogue interaction engine using the multiple batches of training sample sets through multiple computing power nodes includes: based on the computing power configuration information and test status information of the multiple computing power nodes, determining multiple first target computing power nodes from the multiple computing power nodes, arranging the intention recognition unit in each of the first target computing power nodes, and synchronously training the intention recognition unit through the multiple first target computing power nodes to obtain a pre-trained intention recognition unit; based on the computing power configuration information and test status information of the multiple computing power nodes and the process status information of the first target computing power nodes in the process of training the intention recognition unit, determining multiple second target computing power nodes from the multiple computing power nodes, arranging the answer generation unit in each of the second target computing power nodes, and synchronously training the answer generation unit through the multiple second target computing power nodes to obtain a pre-trained answer generation unit; based on the computing power configuration information and test status information of the multiple computing power nodes, the process status information of the first target computing power nodes in the process of training the intention recognition unit and the process status information of the second target computing power nodes in the process of training the intention recognition unit The process state information of the answer generation unit in the process of the answer generation unit is determined, a plurality of third target computing power nodes are determined from the plurality of computing power nodes, the text adjustment unit is arranged in each of the third target computing power nodes, and the text adjustment unit is synchronously trained through the plurality of third target computing power nodes to obtain a pre-trained text adjustment unit; based on the computing power configuration information and test state information of the plurality of computing power nodes, the process state information of the first target computing power node in the process of training the intention recognition unit, the process state information of the second target computing power node in the process of training the answer generation unit, and the process state information of the third target computing power node in the process of training the text adjustment unit, a plurality of fourth target computing power nodes are determined from the plurality of computing power nodes, the speech synthesis unit is arranged in each of the fourth target computing power nodes, and the speech synthesis unit is synchronously trained through the plurality of fourth target computing power nodes to obtain a pre-trained speech synthesis unit; based on the pre-trained intention recognition unit, the pre-trained answer generation unit, the pre-trained text adjustment unit, and the pre-trained speech synthesis unit, a pre-trained multi-round dialogue interaction engine based on Pre-AGI is generated;Based on the computing power configuration information and test status information of the multiple computing power nodes, the process status information of the first target computing power node in the process of training the intention recognition unit, the process status information of the second target computing power node in the process of training the answer generation unit, the process status information of the third target computing power node in the process of training the text adjustment unit, and the process status information of the fourth target computing power node in the process of training the speech synthesis unit, multiple fifth target computing power nodes are determined from the multiple computing power nodes, the pre-trained Pre-AGI-based multi-round dialogue interaction engine is arranged in each of the fifth target computing power nodes, and the pre-trained Pre-AGI-based multi-round dialogue interaction engine is synchronously trained through the multiple fifth target computing power nodes to obtain a trained Pre-AGI-based multi-round dialogue interaction engine. ;

[0008] Furthermore, the intention recognition unit is synchronously trained by the multiple first target computing power nodes to obtain a pre-trained intention recognition unit, including: S1, based on the computing power configuration information and test status information of each first target computing power node, the training sample set of the current batch is divided into multiple training sample groups, each first target computing power node corresponds to a training sample group, and S2 is executed; S2, for each first target computing power node, the corresponding training sample group is input to the intention recognition unit deployed on the first target computing power node, and the intention recognition loss corresponding to the first target computing power node is calculated, and S3 is executed; S3. Based on the intention recognition loss corresponding to each of the first target computing power nodes, determine whether the preset training conditions are met. If so, the training is completed; if not, execute S4; S4. Determine whether all batches of training sample sets are used. If so, the training is completed; if not, execute S5; S5. For each of the first target computing power nodes, calculate the first gradient based on the intention recognition loss and execute S6; S6. Based on the first gradient corresponding to each of the first target computing power nodes, synchronously update the intention recognition unit deployed on the first target computing power node and execute S7; S7. Obtain the next batch of training sample sets and execute S1.

[0009] Furthermore, calculating the intent recognition loss corresponding to the first target computing power node includes: for each of the training samples included in the training sample group corresponding to the first target computing power node, calculating the single-round intent recognition loss of each round of dialogue corresponding to the training sample, and determining the multi-round intent recognition loss corresponding to the training sample based on the single-round intent recognition loss of each round of dialogue; and calculating the intent recognition loss corresponding to the first target computing power node based on the multi-round intent recognition loss corresponding to each of the training samples.

[0010] Furthermore, based on the intention recognition loss corresponding to each of the first target computing power nodes, it is judged whether the preset training conditions are met, including: based on the intention recognition loss corresponding to each of the first target computing power nodes, calculating the intention recognition loss discrete parameter; when the intention recognition loss discrete parameter is greater than the first preset intention recognition loss discrete parameter, determining that the preset training conditions are not met; when the intention recognition loss discrete parameter is less than or equal to the first preset intention recognition loss discrete parameter, based on the intention recognition loss corresponding to each of the first target computing power nodes, calculating the intention recognition loss mean; when the intention recognition loss mean is less than the preset intention recognition loss mean, determining that the preset training conditions are met, and when the intention recognition loss mean is greater than or equal to the preset intention recognition loss mean, determining that the preset training conditions are not met.

[0011] Furthermore, based on the first gradient corresponding to each of the first target computing power nodes, the intent recognition unit deployed on the first target computing power node is synchronously updated, including: when the intent recognition loss discrete parameter is greater than the second preset intent recognition loss discrete parameter, the model update parameter is adjusted according to the intent recognition loss discrete parameter; for each of the first target computing power nodes, based on the multiple rounds of intent recognition losses corresponding to each of the training samples included in the training sample group corresponding to the first target computing power node, the node intent recognition loss discrete parameter corresponding to the first target computing power node is calculated; based on the node intent recognition loss discrete parameter corresponding to the first target computing power node, a gradient weight is assigned to the first target computing power node; based on the gradient weight and the first gradient of each of the first target computing power nodes, a first gradient aggregation value is calculated; based on the adjusted model update parameter and the first gradient aggregation value, the intent recognition unit deployed on the first target computing power node is synchronously updated; when the intent recognition loss discrete parameter is less than or equal to the second preset intent recognition loss discrete parameter, based on the first gradient corresponding to each of the first target computing power nodes, a first gradient mean is calculated; and based on the first gradient mean, the intent recognition unit deployed on the first target computing power node is synchronously updated.

[0012] Furthermore, based on the computing power configuration information and test status information of the multiple computing power nodes and the process status information of the first target computing power node in the process of training the intention recognition unit, multiple second target computing power nodes are determined from the multiple computing power nodes, including: for each of the computing power nodes, based on the computing power configuration information and test status information of the computing power node, the basic matching value of the computing power node is calculated; for each of the first target computing power nodes, based on the discrete parameters and operating status information of the node intent recognition loss of each batch of the first target computing power nodes in the process of training the intention recognition unit, the first performance matching parameters of the first target computing power are calculated; based on the basic matching value of each of the computing power nodes and the first performance matching parameters of each of the first target computing power nodes, multiple second target computing power nodes are determined from the multiple computing power nodes.

[0013] Furthermore, the speech synthesis unit is synchronously trained through the multiple fourth target computing power nodes, including: for each of the training samples, the speech synthesis similarity is calculated based on the sound characteristics and emotional characteristics of the speech synthesized by the speech synthesis unit, the speech synthesis loss is calculated based on the speech synthesis similarity corresponding to each of the training samples, and the speech synthesis unit is synchronously updated based on the speech synthesis loss.

[0014] The present invention provides a multi-round dialogue interaction engine training system based on PRE-AGI, including: a sample establishment module, used to establish multiple batches of training sample sets, wherein the training sample sets include multiple training samples, and the training samples at least include multiple rounds of dialogue content, user intentions of each round of dialogue, dialogue role information, text answer information of each round of dialogue, and voice answer information of each round of dialogue; a model training module, used to use the multiple batches of training sample sets to train and optimize the multi-round dialogue interaction engine based on Pre-AGI through multiple computing nodes.

[0015] Compared with the prior art, the multi-round dialogue interaction engine training method and system based on PRE-AGI provided in this specification has at least the following beneficial effects:

[0016] 1. Using multiple computing nodes for distributed training can significantly accelerate the model training process. By processing multiple batches of data in parallel, each computing node can independently calculate and update part of the model parameters, and finally merge the results of each node to quickly complete the training of the entire model. This not only saves time costs, but also improves resource utilization efficiency;

[0017] 2. By selecting the target computing power node based on the computing power configuration information and test status information, it can be ensured that each unit is trained on the node that is most suitable for its training. This avoids the waste of computing power resources and improves training efficiency. Splitting a complex conversational interaction engine into multiple units for independent training allows each unit to focus on its specific task and be optimized through a dedicated computing power node. This modular training method helps to discover and solve problems in each unit more quickly and improve the overall training effect. Using multiple computing power nodes to synchronously train different units can significantly speed up the entire training process. Since the training of each unit is in parallel, the total training time is not limited by the length of time a single unit is trained. Each unit is integrated after pre-training and eventually forms a complete conversational interaction engine. This step-by-step integration method allows the system to be tested and verified at each stage to ensure that the performance of each unit meets expectations, thereby improving the stability and reliability of the final system. In the final stage, using multiple computing power nodes to synchronously train the entire pre-trained conversational interaction engine can further optimize the entire system. This global optimization helps to improve the performance of the system in different scenarios and make it more adaptable to the needs of actual applications. Since each unit is trained independently, it is easy to add, modify or replace a unit as needed to adapt to new application scenarios or technical requirements. This flexibility and scalability enables the conversational interaction engine to continue to evolve and maintain its competitiveness. By continuously optimizing key links such as intent recognition, answer generation, text adjustment and speech synthesis, the accuracy and naturalness of the conversational interaction engine can be significantly improved, providing users with a more smooth and natural interaction experience;

[0018] 3. When the discrete parameter of intent recognition loss is greater than the second preset discrete parameter of intent recognition loss, appropriately reducing the learning rate can slow down the step size of parameter update, thereby reducing parameter fluctuations caused by gradient noise. Doing so helps the intent recognition unit move forward in a more stable direction and avoid being misled by gradient noise and deviating from the optimal solution. At the same time, a smaller learning rate also helps the intent recognition unit better explore the parameter space in complex problems and find a better solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0020] Figure 1 is a flowchart of a method for training a multi-round dialogue interaction engine based on PRE-AGI in one embodiment of the present application;

[0021] Figure 2is a flow chart of a synchronous training intention recognition unit shown in one embodiment of the present application;

[0022] Figure 3 It is a module diagram of a multi-round dialogue interaction engine training system based on PRE-AGI shown in one embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following briefly introduces the drawings required for describing the embodiments.

[0024] The multi-round dialogue interaction engine training method based on PRE-AGI can be applied to the multi-round dialogue interaction engine based on Pre-AGI. The multi-round dialogue interaction engine based on Pre-AGI includes:

[0025] The intention recognition unit may be used to determine the user's current turn intention based on the user's current input content, historical conversation content, and the user's historical turn intentions;

[0026] The answer generation unit can be used to generate a text answer for the current round based on the user's current round intention through a large language model, a knowledge base, and a fixed speech template library;

[0027] The text adjustment unit may be used to adjust the current round text answer based on the role information to generate the current round text answer that conforms to the role information;

[0028] The speech synthesis unit can be used to generate a speech answer that conforms to the role information and corresponds to the current round text answer that conforms to the role information based on the role information.

[0029] Among them, users can select or personalize role information.

[0030] Figure 1 FIG. 1 is a flowchart of a method for training a multi-round dialogue interaction engine based on PRE-AGI in one embodiment of the present application. Figure 1 As shown, the multi-round dialogue interaction engine training method based on PRE-AGI may include the following steps.

[0031] Step 110, establishing multiple batches of training sample sets, wherein the training sample sets include multiple training samples, and the training samples include at least multiple rounds of dialogue content, user intentions of each round of dialogue, dialogue role information, text answer information of each round of dialogue, and voice answer information of each round of dialogue.

[0032] Step 120, using multiple computing nodes and multiple batches of training sample sets to train and optimize the Pre-AGI-based multi-round dialogue interaction engine.

[0033] In some embodiments, a Pre-AGI-based multi-round dialogue interaction engine is trained and optimized using multiple batches of training sample sets through multiple computing nodes, including:

[0034] Based on computing power configuration information and test status information of multiple computing power nodes, multiple first target computing power nodes are determined from the multiple computing power nodes, an intention recognition unit is arranged in each first target computing power node, and the intention recognition unit is synchronously trained through the multiple first target computing power nodes to obtain a pre-trained intention recognition unit;

[0035] Based on the computing power configuration information and test status information of the plurality of computing power nodes and the process status information of the first target computing power node in the process of training the intention recognition unit, a plurality of second target computing power nodes are determined from the plurality of computing power nodes, an answer generation unit is arranged in each second target computing power node, and the answer generation unit is synchronously trained through the plurality of second target computing power nodes to obtain a pre-trained answer generation unit;

[0036] Based on the computing power configuration information and test status information of the plurality of computing power nodes, the process status information of the first target computing power node in the process of training the intention recognition unit, and the process status information of the second target computing power node in the process of training the answer generation unit, a plurality of third target computing power nodes are determined from the plurality of computing power nodes, a text adjustment unit is arranged in each of the third target computing power nodes, and the text adjustment unit is synchronously trained through the plurality of third target computing power nodes to obtain a pre-trained text adjustment unit;

[0037] Based on the computing power configuration information and test status information of the multiple computing power nodes, the process status information of the first target computing power node in the process of training the intention recognition unit, the process status information of the second target computing power node in the process of training the answer generation unit, and the process status information of the third target computing power node in the process of training the text adjustment unit, a plurality of fourth target computing power nodes are determined from the multiple computing power nodes, a speech synthesis unit is arranged in each of the fourth target computing power nodes, and the speech synthesis units are synchronously trained through the plurality of fourth target computing power nodes to obtain pre-trained speech synthesis units;

[0038] Generate a pre-trained Pre-AGI-based multi-round dialogue interaction engine based on the pre-trained intention recognition unit, the pre-trained answer generation unit, the pre-trained text adjustment unit, and the pre-trained speech synthesis unit;

[0039] Based on the computing power configuration information and test status information of multiple computing power nodes, the process status information of the first target computing power node in the process of training the intention recognition unit, the process status information of the second target computing power node in the process of training the answer generation unit, the process status information of the third target computing power node in the process of training the text adjustment unit, and the process status information of the fourth target computing power node in the process of training the speech synthesis unit, multiple fifth target computing power nodes are determined from the multiple computing power nodes, a pre-trained Pre-AGI-based multi-round dialogue interaction engine is arranged on each fifth target computing power node, and the pre-trained Pre-AGI-based multi-round dialogue interaction engine is synchronously trained through the multiple fifth target computing power nodes to obtain a trained Pre-AGI-based multi-round dialogue interaction engine.

[0040] Specifically, the computing power configuration information of the computing power node may include processor configuration information (for example, CPU manufacturer (for example, Intel, AMD, etc.), model (for example, Intel Xeon E5, AMD EPYC, etc.) and specific specifications (for example, number of cores, number of threads, main frequency, etc.), memory configuration (for example, memory type (such as DDR3, DDR4, etc.) and speed (such as frequency), storage configuration (for example, solid-state drive (SSD) and mechanical hard disk (HDD) capacity and read and write speed, etc.) and network configuration (for example, network interface type (for example, Ethernet, InfiniBand, etc.) and speed (for example, 1Gbps, 10Gbps, 100Gbps, etc.)).

[0041] The test status information may be status information of the computing node during the execution of a preset test task, such as CPU load information, network status information, memory usage, disk usage, disk read and write speed, and physical status information (for example, chassis temperature, chassis vibration, and chassis sound, etc.).

[0042] The basic evaluation model can be used to calculate the basic matching value of the computing power node based on the computing power configuration information and test status information of the computing power node, wherein the larger the basic matching value, the better the computing power configuration of the computing power node and / or the more stable the operating status. The basic evaluation model can be a long short-term memory network (LSTM, Long Short-Term Memory) model.

[0043] A computing power node whose basic matching value is greater than a preset basic matching value threshold may be used as the first target computing power node.

[0044] In some embodiments, based on the computing power configuration information and test status information of the multiple computing power nodes and the process status information of the first target computing power node in the process of training the intention recognition unit, determining multiple second target computing power nodes from the multiple computing power nodes includes:

[0045] For each computing power node, the basic matching value of the computing power node is calculated based on the computing power configuration information and test status information of the computing power node;

[0046] For each first target computing power node, based on the node intention recognition loss discrete parameters and operation status information of each batch of the first target computing power node in the process of training the intention recognition unit, calculate the first performance matching parameter of the first target computing power;

[0047] Based on the basic matching value of each computing power node and the first performance matching parameter of each first target computing power, multiple second target computing power nodes are determined from multiple computing power nodes.

[0048] Specifically, the first performance score of the first target computing power can be calculated through a performance evaluation model based on the operating status information of the first target computing power node during the training of the intent recognition unit, wherein the performance evaluation model can be a long short-term memory network (LSTM, Long Short-Term Memory) model.

[0049] The second performance score of the first target computing power node may be calculated based on the discrete parameter of the node intent recognition loss of each batch in the process of training the intent recognition unit of the first target computing power node. For example, the second performance score is calculated based on the following formula:

[0050]

[0051] Among them, S (k,2) is the second performance score of the kth first target computing power node, σ (k,f) is the node intention identification loss discrete parameter corresponding to the kth first target computing power node in the fth batch, where F is the total number of batches.

[0052] A weighted sum is taken of the first performance score and the second performance score to calculate a first performance matching parameter of the first target computing power.

[0053] For each computing power node, a weighted sum is taken of the basic matching value of the computing power node and the first performance matching parameter of each first target computing power to obtain the current score of the computing power node. Based on the current score of each computing power node, multiple second target computing power nodes are determined.

[0054] Based on the computing power configuration information and test status information of the multiple computing power nodes and the process status information of the first target computing power node in the process of training the intention recognition unit, multiple second target computing power nodes are determined from the multiple computing power nodes. Based on the computing power configuration information and test status information of the multiple computing power nodes, the process status information of the first target computing power node in the process of training the intention recognition unit, the process status information of the second target computing power node in the process of training the answer generation unit, and the process status information of the third target computing power node in the process of training the text adjustment unit, multiple fourth target computing power nodes are determined from the multiple computing power nodes. Based on the computing power configuration information and test status information of the multiple computing power nodes , process status information of the first target computing power node in the process of training the intention recognition unit, process status information of the second target computing power node in the process of training the answer generation unit, process status information of the third target computing power node in the process of training the text adjustment unit, and process status information of the fourth target computing power node in the process of training the speech synthesis unit. The method of determining multiple fifth target computing power nodes from multiple computing power nodes is similar to the method of determining multiple second target computing power nodes from multiple computing power nodes based on the computing power configuration information and test status information of the multiple computing power nodes and the process status information of the first target computing power node in the process of training the intention recognition unit, and will not be repeated here.

[0055] Figure 2 is a flowchart of a synchronous training intention recognition unit shown in an embodiment of the present application, such as Figure 2 As shown, in some embodiments, the intention recognition unit is synchronously trained by multiple first target computing power nodes to obtain a pre-trained intention recognition unit, including:

[0056] S1, based on the computing power configuration information and test status information of each first target computing power node, divide the training sample set of the current batch into multiple training sample groups, each first target computing power node corresponds to a training sample group, and execute S2;

[0057] S2. For each first target computing power node, input the corresponding training sample group into the intent recognition unit deployed on the first target computing power node, calculate the intent recognition loss corresponding to the first target computing power node, and execute S3;

[0058] S3, based on the intention recognition loss corresponding to each first target computing power node, determine whether the preset training condition is met, if so, complete the training, if not, execute S4;

[0059] S4, determine whether all batches of training sample sets are used, if so, complete the training, if not, execute S5;

[0060] S5. For each first target computing power node, calculate a first gradient based on the intent recognition loss, and execute S6.

[0061] S6. Based on the first gradient corresponding to each first target computing power node, synchronously update the intention recognition unit deployed on the first target computing power node, and execute S7;

[0062] S7. Obtain the next batch of training sample sets and execute S1.

[0063] In some embodiments, calculating the intent recognition loss corresponding to the first target computing power node includes:

[0064] For each training sample included in the training sample group corresponding to the first target computing power node, calculate the single-round intent recognition loss of each round of dialogue corresponding to the training sample, and determine the multi-round intent recognition loss corresponding to the training sample based on the single-round intent recognition loss of each round of dialogue;

[0065] Based on the multiple rounds of intent recognition losses corresponding to each training sample, the intent recognition loss corresponding to the first target computing power node is calculated.

[0066] Specifically, for each round of dialogue, the cross-entropy loss can be used to evaluate the difference between the predicted probability of the intent category of each round of dialogue and the actual intent category of the intent recognition unit as the single-round intent recognition loss.

[0067] The multi-round intent recognition loss can be calculated based on the following formula:

[0068]

[0069] Among them, L (i,sample) is the multi-round intent recognition loss corresponding to the i-th training sample, L ((i,sample),j) is the single-round intention recognition loss of the j-th round of dialogue for the ith training sample, and J is the total number of dialogue rounds included in the ith training sample.

[0070] The intent recognition loss corresponding to the first target computing power node can be calculated based on the following formula:

[0071]

[0072] Among them, L (k,node) is the intention recognition loss corresponding to the k-th first target computing power node, and I is the total number of training samples included in the training sample group corresponding to the k-th first target computing power node.

[0073] In some embodiments, based on the intent recognition loss corresponding to each first target computing power node, determining whether a preset training condition is met includes:

[0074] Calculate the intention recognition loss discrete parameter based on the intention recognition loss corresponding to each first target computing power node;

[0075] When the intention recognition loss discrete parameter is greater than the first preset intention recognition loss discrete parameter, it is determined that the preset training condition is not met;

[0076] When the intention recognition loss discrete parameter is less than or equal to the first preset intention recognition loss discrete parameter, calculating the intention recognition loss mean based on the intention recognition loss corresponding to each first target computing power node;

[0077] When the mean value of the intention recognition loss is less than the preset mean value of the intention recognition loss, it is determined that the preset training conditions are met; when the mean value of the intention recognition loss is greater than or equal to the preset mean value of the intention recognition loss, it is determined that the preset training conditions are not met.

[0078] Specifically, the discrete parameter of intent recognition loss can be calculated based on the following formula:

[0079]

[0080] Among them, σ total For the purpose of identifying the loss discrete parameter, K is the total number of the first target computing power nodes.

[0081] In some embodiments, based on the first gradient corresponding to each first target computing power node, synchronously updating the intention recognition unit deployed on the first target computing power node includes:

[0082] When the intention recognition loss discrete parameter is greater than the second preset intention recognition loss discrete parameter, adjust the model update parameter according to the intention recognition loss discrete parameter, for each first target computing power node, calculate the node intention recognition loss discrete parameter corresponding to the first target computing power node based on the multiple rounds of intention recognition losses corresponding to each training sample included in the training sample group corresponding to the first target computing power node, assign a gradient weight to the first target computing power node based on the node intention recognition loss discrete parameter corresponding to the first target computing power node, calculate the first gradient aggregation value based on the gradient weight and the first gradient of each first target computing power node, and synchronously update the intention recognition unit deployed on the first target computing power node based on the adjusted model update parameter and the first gradient aggregation value;

[0083] When the intention recognition loss discrete parameter is less than or equal to the second preset intention recognition loss discrete parameter, the first gradient mean is calculated based on the first gradient corresponding to each first target computing power node, and the intention recognition unit deployed on the first target computing power node is synchronously updated based on the first gradient mean.

[0084] Specifically, the model update parameter may be a learning rate. The learning rate may be updated based on the following formula:

[0085]

[0086] Among them, ηt+1 is the updated learning rate, σ0 is the preset intention recognition loss discrete parameter, η t is the learning rate before updating.

[0087] The node intent recognition loss discrete parameter can be calculated based on the following formula:

[0088]

[0089] Among them, σ k The node intention identification loss discrete parameter corresponding to the k-th first target computing power node.

[0090] The gradient weight of the first target computing power node can be calculated based on the following formula:

[0091]

[0092] Among them, α k is the gradient weight of the kth first target computing power node, σ e The node intention identification loss discrete parameter for the e-th first target computing power node.

[0093] Based on the gradient weight of each first target computing power node, a weighted sum is performed on the first gradient of each first target computing power node as the first gradient aggregation value.

[0094] When the discrete parameter of intent recognition loss is greater than the second preset discrete parameter of intent recognition loss, appropriately reducing the learning rate can slow down the step size of parameter update, thereby reducing parameter fluctuations caused by gradient noise. Doing so helps the intent recognition unit move forward in a more stable direction and avoid being misled by gradient noise and deviating from the optimal solution. At the same time, a smaller learning rate also helps the intent recognition unit better explore the parameter space in complex problems and find a better solution.

[0095] In some embodiments, synchronously training the speech synthesis unit through multiple fourth target computing power nodes includes:

[0096] For each training sample, the speech synthesis similarity is calculated based on the sound features (e.g., spectral features, time domain features, prosodic features, etc.) and emotional features of the speech synthesized by the speech synthesis unit, the speech synthesis loss is calculated based on the speech synthesis similarity corresponding to each training sample, and the speech synthesis unit is synchronously updated based on the speech synthesis loss.

[0097] Figure 3 is a module diagram of a multi-round dialogue interaction engine training system based on PRE-AGI shown in an embodiment of the present application, such as Figure 3 As shown, the multi-round dialogue interaction engine training system based on PRE-AGI may include a sample establishment module and a model training module.

[0098] The sample establishment module can be used to establish multiple batches of training sample sets, wherein the training sample sets include multiple training samples, and the training samples include at least multiple rounds of dialogue content, user intentions of each round of dialogue, dialogue role information, text answer information of each round of dialogue, and voice answer information of each round of dialogue;

[0099] The model training module can be used to train and optimize the Pre-AGI-based multi-round dialogue interaction engine using multiple computing nodes and multiple batches of training sample sets.

[0100] The PRE-AGI-based multi-round dialogue interaction engine training system can be used to execute the PRE-AGI-based multi-round dialogue interaction engine training method. For more descriptions of the PRE-AGI-based multi-round dialogue interaction engine training system, please refer to the PRE-AGI-based multi-round dialogue interaction engine training method and related descriptions, which will not be repeated here.

[0101] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A multi-round dialogue interaction engine training method based on PRE-AGI, which is applied to a multi-round dialogue interaction engine based on Pre-AGI, is characterized in that: include: Establishing multiple batches of training sample sets, wherein the training sample sets include multiple training samples, and the training samples include at least the content of multiple rounds of dialogue, the user intention of each round of dialogue, dialogue role information, text answer information of each round of dialogue, and voice answer information of each round of dialogue; The Pre-AGI-based multi-round dialogue interaction engine is trained and optimized using the multiple batches of training sample sets through multiple computing nodes; the Pre-AGI-based multi-round dialogue interaction engine includes: An intention recognition unit, used to determine the user's current turn intention based on the user's current input content, historical conversation content and the user's historical turn intention; An answer generation unit, configured to generate a text answer for the current round based on the user's current round intention through a large language model, a knowledge base, and a fixed speech template library; A text adjustment unit, used to adjust the current round text answer based on the role information to generate a current round text answer that conforms to the role information; A speech synthesis unit is used to generate a speech answer that conforms to the role information and corresponds to the current round text answer that conforms to the role information based on the role information; the training and optimization of the Pre-AGI-based multi-round dialogue interaction engine using the multiple batches of training sample sets through multiple computing nodes includes: Based on the computing power configuration information and test status information of the multiple computing power nodes, determine multiple first target computing power nodes from the multiple computing power nodes, arrange the intention recognition unit in each of the first target computing power nodes, and synchronously train the intention recognition unit through the multiple first target computing power nodes to obtain a pre-trained intention recognition unit; Based on the computing power configuration information and test status information of the multiple computing power nodes and the process status information of the first target computing power node in the process of training the intention recognition unit, determine multiple second target computing power nodes from the multiple computing power nodes, arrange the answer generation unit in each of the second target computing power nodes, and synchronously train the answer generation unit through the multiple second target computing power nodes to obtain a pre-trained answer generation unit; Based on the computing power configuration information and test status information of the multiple computing power nodes, the process status information of the first target computing power node in the process of training the intention recognition unit, and the process status information of the second target computing power node in the process of training the answer generation unit, a plurality of third target computing power nodes are determined from the multiple computing power nodes, the text adjustment unit is arranged in each of the third target computing power nodes, and the text adjustment unit is synchronously trained through the multiple third target computing power nodes to obtain a pre-trained text adjustment unit; Based on the computing power configuration information and test status information of the multiple computing power nodes, the process status information of the first target computing power node in the process of training the intention recognition unit, the process status information of the second target computing power node in the process of training the answer generation unit, and the process status information of the third target computing power node in the process of training the text adjustment unit, a plurality of fourth target computing power nodes are determined from the multiple computing power nodes, the speech synthesis unit is arranged in each of the fourth target computing power nodes, and the speech synthesis unit is synchronously trained through the multiple fourth target computing power nodes to obtain a pre-trained speech synthesis unit; Generate a pre-trained Pre-AGI-based multi-round dialogue interaction engine based on the pre-trained intention recognition unit, the pre-trained answer generation unit, the pre-trained text adjustment unit, and the pre-trained speech synthesis unit; Based on the computing power configuration information and test status information of the multiple computing power nodes, the process status information of the first target computing power node in the process of training the intention recognition unit, the process status information of the second target computing power node in the process of training the answer generation unit, the process status information of the third target computing power node in the process of training the text adjustment unit, and the process status information of the fourth target computing power node in the process of training the speech synthesis unit, multiple fifth target computing power nodes are determined from the multiple computing power nodes, the pre-trained Pre-AGI-based multi-round dialogue interaction engine is arranged on each of the fifth target computing power nodes, and the pre-trained Pre-AGI-based multi-round dialogue interaction engine is synchronously trained through the multiple fifth target computing power nodes to obtain a trained Pre-AGI-based multi-round dialogue interaction engine.

2. The method for training a multi-round dialogue interaction engine based on PRE-AGI according to claim 1, characterized in that: The intention recognition unit is synchronously trained by the plurality of first target computing power nodes to obtain a pre-trained intention recognition unit, including: S1. Based on the computing power configuration information and test status information of each of the first target computing power nodes, divide the training sample set of the current batch into multiple training sample groups, each of the first target computing power nodes corresponds to one training sample group, and execute S2; S2. For each of the first target computing power nodes, input the corresponding training sample group into the intent recognition unit deployed on the first target computing power node, calculate the intent recognition loss corresponding to the first target computing power node, and execute S3; S3, based on the intent recognition loss corresponding to each of the first target computing power nodes, determine whether the preset training conditions are met, if the training is completed, if not, execute S4; S4, determine whether all batches of training sample sets are used, if so, complete the training, if not, execute S5; S5. For each of the first target computing power nodes, calculate a first gradient based on the intention recognition loss, and execute S6; S6. Based on the first gradient corresponding to each of the first target computing power nodes, synchronously update the intention recognition unit deployed on the first target computing power node, and execute S7; S7. Obtain the next batch of training sample sets and execute S1.

3. The method for training a multi-round dialogue interaction engine based on PRE-AGI according to claim 2, characterized in that: Calculating the intent recognition loss corresponding to the first target computing power node includes: For each of the training samples included in the training sample group corresponding to the first target computing power node, calculate the single-round intent recognition loss of each round of dialogue corresponding to the training sample, and determine the multi-round intent recognition loss corresponding to the training sample based on the single-round intent recognition loss of each round of dialogue; Based on the multiple rounds of intent recognition losses corresponding to each of the training samples, the intent recognition loss corresponding to the first target computing power node is calculated.

4. The method for training a multi-round dialogue interaction engine based on PRE-AGI according to claim 2 or 3, characterized in that: Based on the intent recognition loss corresponding to each of the first target computing power nodes, determining whether a preset training condition is met includes: Calculate the intention recognition loss discrete parameter based on the intention recognition loss corresponding to each of the first target computing power nodes; When the intention recognition loss discrete parameter is greater than a first preset intention recognition loss discrete parameter, determining that the preset training condition is not met; When the intention recognition loss discrete parameter is less than or equal to the first preset intention recognition loss discrete parameter, calculating the intention recognition loss mean based on the intention recognition loss corresponding to each of the first target computing power nodes; When the intention recognition loss mean is less than the preset intention recognition loss mean, it is determined that the preset training condition is met; when the intention recognition loss mean is greater than or equal to the preset intention recognition loss mean, it is determined that the preset training condition is not met.

5. The method for training a multi-round dialogue interaction engine based on PRE-AGI according to claim 4, characterized in that: Based on the first gradient corresponding to each of the first target computing power nodes, synchronously updating the intention recognition unit deployed on the first target computing power node includes: When the intention recognition loss discrete parameter is greater than the second preset intention recognition loss discrete parameter, the model update parameter is adjusted according to the intention recognition loss discrete parameter, for each of the first target computing power nodes, based on the multiple rounds of intention recognition losses corresponding to each of the training samples included in the training sample group corresponding to the first target computing power node, the node intention recognition loss discrete parameter corresponding to the first target computing power node is calculated, based on the node intention recognition loss discrete parameter corresponding to the first target computing power node, a gradient weight is assigned to the first target computing power node, based on the gradient weight and the first gradient of each of the first target computing power nodes, a first gradient aggregation value is calculated, and based on the adjusted model update parameter and the first gradient aggregation value, the intent recognition unit deployed on the first target computing power node is synchronously updated; When the intention recognition loss discrete parameter is less than or equal to the second preset intention recognition loss discrete parameter, the first gradient mean is calculated based on the first gradient corresponding to each of the first target computing power nodes, and the intention recognition unit deployed on the first target computing power node is synchronously updated based on the first gradient mean.

6. The method for training a multi-round dialogue interaction engine based on PRE-AGI according to claim 5, characterized in that: Based on the computing power configuration information and test status information of the multiple computing power nodes and the process status information of the first target computing power node in the process of training the intention recognition unit, determining multiple second target computing power nodes from the multiple computing power nodes includes: For each of the computing power nodes, based on the computing power configuration information and test status information of the computing power node, calculate the basic matching value of the computing power node; For each of the first target computing power nodes, based on the node intention recognition loss discrete parameters and operation status information of each batch of the first target computing power nodes in the process of training the intention recognition unit, calculate the first performance matching parameter of the first target computing power; Based on the basic matching value of each of the computing power nodes and the first performance matching parameter of each of the first target computing power, multiple second target computing power nodes are determined from the multiple computing power nodes.

7. The method for training a multi-round dialogue interaction engine based on PRE-AGI according to any one of claims 1 to 3, characterized in that: Synchronously training the speech synthesis unit through the plurality of fourth target computing power nodes includes: For each of the training samples, the speech synthesis similarity is calculated based on the sound features and emotional features of the speech synthesized by the speech synthesis unit, the speech synthesis loss is calculated based on the speech synthesis similarity corresponding to each of the training samples, and the speech synthesis unit is synchronously updated based on the speech synthesis loss.

8. The multi-round dialogue interaction engine training system based on PRE-AGI is characterized by: The method for training a multi-round dialogue interaction engine based on PRE-AGI according to any one of claims 1 to 7 comprises: A sample establishment module, used to establish multiple batches of training sample sets, wherein the training sample sets include multiple training samples, and the training samples include at least multiple rounds of dialogue content, user intentions of each round of dialogue, dialogue role information, text answer information of each round of dialogue, and voice answer information of each round of dialogue; The model training module is used to train and optimize the Pre-AGI-based multi-round dialogue interaction engine using the multiple batches of training sample sets through multiple computing nodes.

Images