Sentence rewriting method and device, electronic equipment and storage medium

Abstract

The application provides a sentence rewriting method and device, electronic equipment and storage medium, wherein the method comprises: determining a multi-round dialogue; based on a sentence rewriting model, extracting overall semantic features of the multi-round dialogue, and applying the overall semantic features for sentence rewriting to obtain a rewritten sentence; the sentence rewriting model is trained based on the similarity between sample overall semantic features of a sample multi-round dialogue and positive example semantic features of a positive example rewritten sentence corresponding to the sample multi-round dialogue, and the similarity between the sample overall semantic features and negative example semantic features of a negative example rewritten sentence corresponding to the sample multi-round dialogue; the model is trained by using the similarity between the semantic features of the sample multi-round dialogue and the sample rewritten sentence, which can optimize the training process, improve the model performance, overcome the defects that the model training is limited and the rewriting effect is poor due to the neglect of the correlation between the two, improve the sentence rewriting process, and realize the double improvement of the credibility and accuracy of the rewritten sentence.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a method, apparatus, electronic device, and storage medium for rewriting sentences. Background Technology

[0002] Incomplete dialogue rewriting, as an important research direction in the field of multi-turn interaction, has great potential value in chatbots. Since users often use incomplete utterances in multi-turn dialogues—that is, they tend to omit and / or corenote entities and concepts appearing in the dialogue context—it is necessary to rewrite incomplete statements to reproduce the omitted and / or corenote content in the dialogue in order to help human-computer interaction models understand the dialogue.

[0003] Currently, most methods for rewriting sentences in incomplete dialogues use the CopyNet model or image-based semantic segmentation methods. These methods expand all sentence tokens into a matrix and control the generation of rewritten sentences using category labels such as Insert, Substitute, and None. However, the training of these models relies solely on samples and labels, meaning that the preceding dialogue and the rewritten sentences are implicitly and forcibly linked. This not only limits the model's performance but also results in questionable sentence rewriting performance. Summary of the Invention

[0004] This invention provides a statement rewriting method, apparatus, electronic device, and storage medium to address the shortcomings of existing technologies that suffer from limited model training and poor rewriting results due to neglecting the intrinsic relationship between multi-turn dialogues and rewritten statements in samples.

[0005] This invention provides a statement rewriting method, comprising:

[0006] Establish multiple rounds of dialogue;

[0007] Based on the statement rewriting model, the overall semantic features of the multi-turn dialogue are extracted, and the overall semantic features are applied to rewrite the statements to obtain the rewritten statements.

[0008] The statement rewriting model is trained based on the similarity between the overall semantic features of a multi-turn dialogue sample and the positive semantic features of the rewritten positive statements corresponding to the multi-turn dialogue sample, as well as the similarity between the overall semantic features of the sample and the negative semantic features of the rewritten negative statements corresponding to the multi-turn dialogue sample.

[0009] According to a statement rewriting method provided by the present invention, the statement rewriting model is trained based on the following steps:

[0010] Based on the initial statement rewriting model, the overall semantic features of the sample multi-turn dialogue, the positive semantic features of the positive rewritten statement, and the negative semantic features of the negative rewritten statement are extracted respectively.

[0011] Based on the similarity between the overall semantic features of the sample and the semantic features of the positive example, and the similarity between the overall semantic features of the sample and the semantic features of the negative example, the contrast loss is determined;

[0012] Based on the contrast loss, the initial statement rewriting model is iterated to obtain the statement rewriting model.

[0013] According to a statement rewriting method provided by the present invention, the step of performing parameter iteration on the initial statement rewriting model based on the contrastive loss to obtain the statement rewriting model includes:

[0014] Based on the contrast loss and response consistency loss, the initial statement rewriting model and the initial discrimination model are iterated to obtain the statement rewriting model and the response consistency discrimination model.

[0015] The response consistency discrimination model is used to determine the response consistency between the predicted rewritten statement and the preceding dialogue of the sample multi-turn dialogue, and the response consistency loss is determined based on the predicted rewritten statement and the preceding dialogue of the sample.

[0016] The predicted rewritten statement is determined by the statement rewriting model based on the sample multi-turn dialogue, and the sample preceding dialogue is the dialogue in the sample multi-turn dialogue excluding the sample statement to be rewritten.

[0017] According to a statement rewriting method provided by the present invention, the response consistency loss is determined based on the following steps:

[0018] Based on the initial statement rewriting model and the overall semantic features of the sample multi-turn dialogue, the statement is rewritten to obtain the predicted rewritten statement.

[0019] Based on the initial statement rewriting model, the predicted semantic features of the predicted rewritten statement and the sample context semantic features of the sample context dialogue are extracted.

[0020] Based on the similarity between the predicted semantic features and the semantic features of the sample context, and the response consistency between the predicted rewritten statement and the sample context dialogue, the response consistency loss is determined.

[0021] According to a statement rewriting method provided by the present invention, the step of iterating the parameters of the initial statement rewriting model and the initial discriminant model based on the contrast loss and the response consistency loss to obtain the statement rewriting model and the response consistency discriminant model includes:

[0022] Based on the response consistency between the sample context dialogue and the positive example rewritten statement, as well as the response consistency between the sample context dialogue and the negative example rewritten statement, the parameters of the initial discriminant model are iterated to obtain the first discriminant model.

[0023] Based on the contrast loss and the response consistency loss, the initial statement rewriting model and the first discriminant model are iterated to obtain the statement rewriting model and the response consistency discriminant model.

[0024] According to a statement rewriting method provided by the present invention, the negative example rewriting statement is determined based on the following steps;

[0025] Obtain at least two sample single-turn dialogues;

[0026] Based on the initial semantic rewriting model, the single-turn semantic features of each sample's single-turn dialogue are determined.

[0027] Based on the similarity between the single-turn semantic features of each sample and the semantic features of the positive example, negative example rewrite statements are determined from the at least two single-turn dialogues of the samples.

[0028] According to a statement rewriting method provided by the present invention, the step of determining negative example rewritten statements from at least two sample single-turn dialogues based on the similarity between the single-turn semantic features of each sample and the semantic features of the positive example includes:

[0029] Select the target sample single-turn dialogue from the at least two sample single-turn dialogues;

[0030] From the single-turn dialogue of the target sample, identify negative examples and rewrite statements;

[0031] The target sample single-turn dialogue is the sample single-turn dialogue corresponding to the first preset number of similarities between the single-turn semantic features of each sample arranged in ascending order and the semantic features of the positive example.

[0032] The present invention also provides a statement rewriting apparatus, comprising:

[0033] Dialogue determination unit, used to determine multi-turn dialogues;

[0034] The statement rewriting unit is used to extract the overall semantic features of the multi-turn dialogue based on the statement rewriting model, and apply the overall semantic features to rewrite the statements to obtain rewritten statements.

[0035] The statement rewriting model is trained based on the similarity between the overall semantic features of the sample multi-turn dialogue and the positive semantic features of the positive rewritten statement corresponding to the sample multi-turn dialogue, as well as the similarity between the overall semantic features of the sample and the negative semantic features of the negative rewritten statement corresponding to the sample multi-turn dialogue.

[0036] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the statement rewriting method as described above.

[0037] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the statement rewriting method as described above.

[0038] The statement rewriting method, apparatus, electronic device, and storage medium provided by this invention use the similarity between the overall semantic features of sample multi-turn dialogues and the positive semantic features of the positive rewritten statements corresponding to the sample multi-turn dialogues, as well as the similarity between the overall semantic features of the samples and the negative semantic features of the negative rewritten statements corresponding to the sample multi-turn dialogues, as references for model training. This enables the model to fully learn the semantic relationship between sample multi-turn dialogues and sample rewritten statements under different sample combinations during training, thereby providing crucial assistance for accurate statement rewriting. It overcomes the shortcomings of traditional solutions that ignore the inherent connection between sample multi-turn dialogues and sample rewritten statements, resulting in limited model training and poor rewriting effects. It improves the semantic extraction and statement rewriting process, achieving a dual improvement in the credibility and accuracy of rewritten statements. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0040] Figure 1 This is a flowchart illustrating the statement rewriting method provided by the present invention;

[0041] Figure 2 This is a flowchart illustrating the comparative training process provided by the present invention;

[0042] Figure 3 This is an example diagram illustrating the framework of the comparative training process provided by the present invention;

[0043] Figure 4This is a schematic diagram of the process for determining response consistency loss provided by the present invention;

[0044] Figure 5 This is a flowchart illustrating the joint training process provided by the present invention;

[0045] Figure 6 This is a framework example diagram of the training process of the initial discrimination model provided by the present invention;

[0046] Figure 7 This is an example diagram illustrating the framework of the joint training process provided by the present invention;

[0047] Figure 8 This is a flowchart illustrating the process of determining the negative instance rewrite statement provided by the present invention;

[0048] Figure 9 This is a flowchart illustrating step 830 in the statement rewriting method provided by the present invention;

[0049] Figure 10 This is a general framework diagram of the training process of the statement rewriting model provided by the present invention;

[0050] Figure 11 This is a schematic diagram of the structure of the statement rewriting device provided by the present invention;

[0051] Figure 12 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0053] The rewriting of sentences in incomplete dialogues has attracted much attention due to its important applications in the field of human-computer interaction. This progress and technological breakthrough, which involves engaging in multi-turn dialogues with users to gain a deeper understanding of them and responding to their multi-turn requests, is of great significance to the development of artificial intelligence.

[0054] The goal of rewriting statements in incomplete dialogues is to solve the problems of referential elimination and omission completion in multi-turn dialogues, so that statements can express their meaning independently of the context. For example, in the multi-turn dialogue shown in the table below, the omitted and core references in the statement to be rewritten (Why is it always like this?) need to be reproduced and rewritten as the rewritten statement (Why is it always cloudy in City A?).

[0055]

[0056] Currently, the rewriting of sentences in incomplete dialogues is mostly done through the CopyNet model, or by drawing on semantic segmentation methods in the image domain, which expand all sentence tokens into a matrix and then control the generation of rewritten sentences through category labels of Insert, Substitute, and None.

[0057] However, the training of the model in the above scheme only relies on samples and labels. That is, the model is trained only with sample multi-turn dialogues and sample rewritten sentences, without considering the relationship between the two. In other words, the consistency of response between the preceding dialogue and the rewritten sentence is not considered. In other words, the model is forced to associate the preceding dialogue and the rewritten sentence in an implicit way, ignoring the inherent connection and deep semantic relationship between the two. This limits the training of the model, results in poor model performance, and ultimately leads to poor sentence rewriting results.

[0058] To address this issue, this invention provides a sentence rewriting method that explicitly labels the relationship between sample multi-turn dialogues and sample rewritten sentences, and integrates this explicit relationship into the model training process. This allows the model to fully learn the mapping relationship between the two during training, enabling it to apply this mapping relationship for sentence rewriting during application. This achieves optimization of model performance and improvement of sentence rewriting effectiveness. Figure 1 This is a flowchart illustrating the statement rewriting method provided by the present invention, as follows: Figure 1 As shown, the method includes:

[0059] Step 110: Determine the multi-turn dialogue;

[0060] Specifically, before rewriting a statement, it is necessary to first determine the statement to be rewritten and the multi-turn dialogue in which the statement to be rewritten is located. In addition to the statement to be rewritten, the multi-turn dialogue also contains the preceding statement of the statement to be rewritten. The preceding statements can be assembled to form the preceding dialogue. Since the statement to be rewritten is usually the last statement in the multi-turn dialogue, it can be determined that the multi-turn dialogue contains the statement to be rewritten and the preceding dialogue of the statement to be rewritten.

[0061] Here, the multi-turn dialogue can be dialogue text directly input by the user, or dialogue text obtained by transcribing the dialogue speech captured in real time by the audio acquisition module, or dialogue text obtained by transcribing a segment of dialogue speech extracted from historical dialogue speech, or dialogue text obtained by capturing dialogue images from textbooks, journals, lecture notes, etc. through image acquisition devices such as scanners, mobile phones, and cameras, and then obtaining dialogue text by performing text recognition on the dialogue images. This embodiment of the invention does not specifically limit this.

[0062] It should be noted that the multi-turn dialogue here can be a single segment or multiple segments. In the case of multiple segments, it is necessary to rewrite the semantically incomplete statements in each segment of the multi-turn dialogue. That is, it is necessary to fill in the missing semantic information in the incomplete statements in each segment of the multi-turn dialogue to obtain the rewritten statements. In other words, it is necessary to reproduce the common references and / or omitted content in the semantically incomplete statements using the information above.

[0063] Step 120: Based on the statement rewriting model, extract the overall semantic features of the multi-turn dialogue, and apply the overall semantic features to rewrite the statements to obtain the rewritten statements.

[0064] The sentence rewriting model is trained based on the similarity between the overall semantic features of the sample multi-turn dialogue and the positive semantic features of the positive rewritten sentences corresponding to the sample multi-turn dialogue, as well as the similarity between the overall semantic features of the sample and the negative semantic features of the negative rewritten sentences corresponding to the sample multi-turn dialogue.

[0065] Specifically, after step 110, once the multi-turn dialogue is determined, step 120 can be executed. The overall semantic features of the multi-turn dialogue are extracted using a statement rewriting model, and these overall semantic features are applied to rewrite the statements, thus obtaining the rewritten statements. The specific process includes the following steps:

[0066] Since sentences in a multi-turn dialogue are usually interconnected, meaning that there must be semantic coherence and semantic repetition between sentences, when rewriting sentences in a multi-turn dialogue, the omitted and / or co-referenced content can be reproduced by using the context information of the sentence to be rewritten. In view of this, in this embodiment of the invention, the sentence rewriting model can be used to extract the semantics of the multi-turn dialogue to obtain the overall semantic features of the multi-turn dialogue. The overall semantic features here can be understood as features that can cover the semantic information of the entire dialogue.

[0067] Subsequently, the statement rewriting model can apply this overall semantic feature to generate rewritten statements. That is, it can use the overall semantic feature to rewrite the statements to be rewritten in multi-turn dialogues, thereby obtaining rewritten statements. The rewritten statements are those that supplement the statements to be rewritten with omitted content and / or replace core words.

[0068] Specifically, in this embodiment of the invention, the above process may first involve inputting the multi-turn dialogue into the encoder of the statement rewriting model, whereby the encoder performs semantic encoding on the input multi-turn dialogue to obtain a semantic encoding vector. This semantic encoding vector is a vector that can represent the semantic information of the entire dialogue, and can also be called the overall semantic feature. After obtaining the overall semantic feature, it can be input into the decoder of the statement rewriting model. The decoder can apply this overall semantic feature to rewrite the statements to be rewritten in the multi-turn dialogue, and finally obtain the rewritten statements of the multi-turn dialogue output by the statement rewriting model.

[0069] Before inputting multi-turn dialogues into the sentence rewriting model, the sentence rewriting model can be pre-trained. Unlike traditional methods that train the model solely through sample multi-turn dialogues and sample rewritten sentences, this embodiment of the invention considers that traditional training methods ignore the inherent connections and deep semantic relationships between sample multi-turn dialogues and sample rewritten sentences, merely forcing the model to associate them implicitly, leading to limited training and poor performance. Therefore, this embodiment applies the explicit association between the two for model training; in other words, it uses the response consistency between sample multi-turn dialogues and sample rewritten sentences for model training to obtain the trained sentence rewriting model.

[0070] Specifically, during training, the initial sentence rewriting model is trained based on the similarity between the overall semantic features of the sample multi-turn dialogue and the positive semantic features of the corresponding positive example rewritten sentences, as well as the similarity between the overall semantic features of the sample and the negative example semantic features of the corresponding negative example rewritten sentences. This initial model learns the mapping relationship between the sample multi-turn dialogue and the positive and negative example rewritten sentences by referring to the similarity between semantic features. This allows the model to apply this mapping relationship in subsequent sentence rewriting processes, improving model performance, aiding the sentence rewriting process, and optimizing the sentence rewriting effect. The similarity here can be calculated using cosine similarity, Euclidean distance, Minkowski distance, etc., between semantic features.

[0071] Among them, the positive example rewritten statement is the statement that is semantically related to the sample multi-turn dialogue and has the same semantics as the statement to be rewritten in the sample multi-turn dialogue; conversely, the negative example rewritten statement is the statement that has a different semantics from the statement to be rewritten in the sample.

[0072] In this embodiment of the invention, the model is trained by applying the similarity between the semantic features of sample multi-turn dialogues and positive and negative example rewritten sentences. This not only ensures the performance of the model, but also enables the model to fully learn the semantic relationship between sample multi-turn dialogues and sample rewritten sentences under different sample combinations during the training process, providing crucial assistance for accurate sentence rewriting.

[0073] Compared to the tacit training method used in traditional solutions, this invention integrates the explicit association between sample multi-turn dialogues and positive and negative example rewritten statements into the model training process. By adopting an explicit training method, the model training process is more refined, the training effect is more significant, and the sentence rewriting model obtained from the training achieves better results in sentence rewriting.

[0074] Furthermore, because traditional solutions neglect the correlation between sample multi-turn dialogues and sample rewritten statements, model training is limited. In this embodiment of the invention, the correlation between the two is explicitly labeled, and the initial statement rewriting model is trained by the similarity between semantic features that reflect this correlation. This enables the initial statement rewriting model to judge the similarity between semantic features based on the semantic relationship between sample multi-turn dialogues and sample rewritten statements. When inputting sample multi-turn dialogues and positive example rewritten statements, i.e., when the semantic relationship between the two is similar, the training aims to make the overall semantic features output by the initial statement rewriting model as similar as possible to the semantic features of the positive examples. Conversely, when inputting sample multi-turn dialogues and negative example rewritten statements, i.e., when the semantic relationship between the two is distant, the training aims to make the overall semantic features output by the initial statement rewriting model as similar as possible to the features of the negative examples.

[0075] In this embodiment of the invention, the application of the explicit association between labeled sample multi-turn dialogues and sample rewritten sentences during model training not only greatly reduces the model training time, but also improves the model training effect and performance. The accuracy and precision of semantic extraction are also higher, resulting in better sentence rewriting based on the overall semantic features obtained from semantic extraction. This refines the model training process and promotes the overall progress of accurate sentence rewriting.

[0076] The statement rewriting method provided by this invention uses the similarity between the overall semantic features of sample multi-turn dialogues and the positive semantic features of the positive rewritten statements corresponding to the sample multi-turn dialogues, as well as the similarity between the overall semantic features of the samples and the negative semantic features of the negative rewritten statements corresponding to the sample multi-turn dialogues, as references for model training. This enables the model to fully learn the semantic relationship between sample multi-turn dialogues and sample rewritten statements under different sample combinations during training, thus providing crucial assistance for accurate statement rewriting. It overcomes the shortcomings of traditional solutions that ignore the inherent connection between sample multi-turn dialogues and sample rewritten statements, resulting in limited model training and poor rewriting effects. This method improves the semantic extraction and statement rewriting process, achieving a dual improvement in the credibility and accuracy of rewritten statements.

[0077] Based on the above embodiments, Figure 2This is a flowchart illustrating the comparative training process provided by the present invention, as shown below. Figure 2 As shown, the statement rewriting model is trained based on the following steps:

[0078] Step 210: Based on the initial statement rewriting model, extract the overall semantic features of the sample multi-turn dialogue, the positive semantic features of the positive rewritten statement, and the negative semantic features of the negative rewritten statement.

[0079] Step 220: Determine the contrast loss based on the similarity between the overall semantic features of the sample and the semantic features of positive examples, and the similarity between the overall semantic features of the sample and the semantic features of negative examples;

[0080] Step 230: Based on the contrastive loss, perform parameter iteration on the initial statement rewriting model to obtain the statement rewriting model.

[0081] Specifically, the training process of the statement rewriting model can include the following steps:

[0082] First, step 210 is executed to determine the initial statement rewriting model. This initial statement rewriting model can be built on the basis of an encoder capable of statement encoding and a decoder capable of semantic decoding. Then, the initial statement rewriting model can be applied to extract semantics from the sample multi-turn dialogue, positive example rewritten statements, and negative example rewritten statements to obtain the overall semantic features, positive example semantic features, and negative example semantic features of the sample. Specifically, the positive example rewritten statements and negative example rewritten statements from the sample multi-turn dialogue and sample rewritten statements can be input into the initial statement rewriting model. The encoder in the initial statement rewriting model performs semantic extraction on the input statements, extracting the features that can represent the semantic information of the entire sample dialogue, as well as the features that represent the semantic information of the positive example rewritten statements and negative example rewritten statements. Finally, the overall semantic features of the sample multi-turn dialogue, the positive example semantic features of the positive example rewritten statements, and the negative example semantic features of the negative example rewritten statements output by the encoder in the initial statement rewriting model are obtained.

[0083] Subsequently, step 220 is executed, using the similarity between the overall semantic features of the sample and the semantic features of positive examples, and the similarity between the overall semantic features of the sample and the semantic features of negative examples as benchmarks, to calculate the contrast loss of the initial sentence rewriting model. The contrast loss here can be understood as the loss of the initial sentence rewriting model in contrast training based on the similarity between semantic features.

[0084] The training objective of the initial statement rewriting model is to maximize the similarity between the overall semantic features of the sample output by the initial statement rewriting model and the semantic features of the positive example when the semantic relationship between the sample multi-turn dialogue and the sample rewritten statement is similar (i.e., when the sample rewritten statement is a positive example rewritten statement). Conversely, when the semantic relationship between the sample multi-turn dialogue and the sample rewritten statement is distant (i.e., when the sample rewritten statement is a negative example rewritten statement), the similarity between the overall semantic features of the sample and the semantic features of the positive example rewritten statement is minimized.

[0085] Therefore, when the similarity between the overall semantic features of the multi-turn dialogue sample and the positive semantic features of the positive rewritten statement is high, and the similarity between the overall semantic features of the multi-turn dialogue sample and the negative semantic features of the negative rewritten statement is low, the contrast loss can be determined to be small. Conversely, when the similarity between the overall semantic features of the sample and the positive semantic features is low, and / or the similarity between the overall semantic features of the sample and the negative semantic features is high, the contrast loss can be determined to be large.

[0086] Subsequently, step 230 is executed, and the parameters of the initial sentence rewriting model are iterated based on the contrastive loss to obtain the sentence rewriting model. Specifically, this process may involve adjusting the parameters of the initial sentence rewriting model according to the contrastive loss so that the adjusted initial sentence rewriting model can output semantic features according to the semantic relationship between the sample multi-turn dialogue and the sample rewritten sentences. This aims to make the similarity between the overall semantic features of the samples output by the initial sentence rewriting model and the semantic features of the positive examples as high as possible, and the similarity between the overall semantic features of the samples output and the semantic features of the positive examples as low as possible, ultimately obtaining the trained sentence rewriting model.

[0087] Based on the above embodiments, Figure 3 This is an example diagram of the framework for the comparative training process provided by the present invention, such as... Figure 3 As shown, the initial sentence rewriting model undergoes comparative training to obtain the sentence rewriting model. This process involves inputting multi-turn dialogue samples (What is the weather like in City A today?; It is cloudy in City A today; Why is it always like this?) as well as positive rewritten sentences (Why is it always cloudy in City A?) and negative rewritten sentences (Where is City A?) from the sample rewritten sentences into the initial sentence rewriting model. The encoder of the initial sentence rewriting model performs semantic encoding and outputs the overall semantic features of the samples, the semantic features of positive examples, and the semantic features of negative examples.

[0088] Subsequently, the similarity between the overall semantic features of the sample and the semantic features of positive examples, as well as the similarity between the overall semantic features of the sample and the semantic features of negative examples, can be determined. Based on these two similarities, the contrastive loss of the initial sentence rewriting model training process can be calculated.

[0089] The contrast loss can be calculated using the following formula:

[0090]

[0091] Where, loss CE c represents the comparative loss. i This represents the overall semantic features of a multi-turn dialogue sample. The positive semantic features representing positive example rewrite statements. Then, it represents the negative semantic features of the negative example rewritten statement, sim is the similarity calculation function, N is the number of samples, that is, the number of sample combinations formed by the sample dialogue with the positive example rewritten statement and the negative example rewritten statement respectively, and τ is the temperature coefficient.

[0092] Based on the above embodiments, step 230 includes:

[0093] Based on contrast loss and response consistency loss, the initial statement rewriting model and the initial discriminant model are iterated to obtain the statement rewriting model and the response consistency discriminant model.

[0094] The response consistency discriminant model is used to determine the response consistency between the predicted rewritten statements and the preceding dialogue of a multi-turn sample dialogue. The response consistency loss is determined based on the predicted rewritten statements and the preceding dialogue of the sample.

[0095] The predicted rewritten statement is determined by the statement rewriting model based on sample multi-turn dialogues. The preceding dialogue of the sample is the dialogue in the sample multi-turn dialogue excluding the statement to be rewritten.

[0096] Considering that traditional solutions do not consider the response consistency between sample multi-turn dialogues and sample rewritten statements, and only implicitly force the model to associate sample multi-turn dialogues and sample rewritten statements, resulting in limited statement rewriting performance in multi-turn dialogue scenarios, this invention proposes a training scheme based on response consistency. This scheme draws on the adversarial training idea, adds response consistency loss to the training process of the statement rewriting model, and applies this response consistency loss along with the contrast loss to jointly train the initial statement rewriting model and the initial discriminant model to obtain the trained statement rewriting model and response consistency discriminant model.

[0097] The initial discriminant model here is added along with the response consistency loss and is used to discriminate the response consistency between the sample context dialogue and the predicted rewritten statement in the sample multi-turn dialogue. The sample context dialogue is the dialogue consisting of the context statements in the sample multi-turn dialogue except for the sample statement to be rewritten, and the predicted rewritten statement is the rewritten statement output by the initial statement rewriting model based on the sample multi-turn dialogue.

[0098] The response consistency loss can be determined based on the sample context dialogue and the predicted rewritten statement. That is, it is the loss of the sample context dialogue and the predicted rewritten statement in response association. It includes not only the loss of the initial discrimination model's discrimination process, but also the loss of the initial statement rewriting model's prediction process in generating the predicted rewritten statement. Therefore, by leveraging its association with the discrimination and prediction processes, the initial statement rewriting model and the initial discrimination model can be jointly trained to obtain the trained statement rewriting model and response consistency discrimination model.

[0099] Specifically, step 230, which iterates the parameters of the initial statement rewriting model based on the contrastive loss to obtain the statement rewriting model, may include the following steps:

[0100] In this embodiment of the invention, response consistency loss is incorporated into the model training process. Therefore, based on the contrastive loss, the parameters of the initial sentence rewriting model and the initial discriminant model can be iterated by combining the response consistency loss. That is, the parameters of the initial discriminant model and the initial sentence rewriting model are adjusted with reference to the losses at both levels, so that the predicted rewritten sentences output by the adjusted initial sentence rewriting model are as consistent as possible with the responses of the sample dialogue, thereby obtaining the trained sentence rewriting model and response consistency discriminant model.

[0101] Here, response consistency can reflect the semantic coherence and semantic similarity between the two. Therefore, the above process can be understood as adjusting the parameters of the initial statement rewriting model and the initial discriminant model with the aim of predicting the response consistency between the rewritten statement and the sample dialogue. This improves the predictive ability of the initial statement rewriting model and the discriminative ability of the initial discriminant model, and finally, the statement rewriting model and the response consistency discriminant model can be obtained.

[0102] It is worth noting that the response consistency discrimination process of the initial discrimination model here can be understood as a binary classification task. The sample context semantic features of the sample dialogue and the predicted semantic features of the predicted rewritten statement are used as binary classification targets. The model outputs 0 and 1, where 0 indicates that the responses between the sample context dialogue and the predicted rewritten statement are inconsistent, and 1 indicates that the responses between the sample context dialogue and the predicted rewritten statement are consistent, that is, they have response consistency. Specifically, if the similarity between the sample context semantic features and the predicted semantic features is high, the initial discrimination model outputs 1, and correspondingly, if the similarity between the two is low, the initial discrimination model outputs 0.

[0103] Among them, the sample context semantic features and the predicted semantic features are determined by the initial sentence rewriting model based on the sample context dialogue and the predicted rewritten sentence, respectively. In other words, they are the semantic features output by the initial sentence rewriting model after semantically encoding the input sample context dialogue and the predicted rewritten sentence.

[0104] In this embodiment of the invention, a response consistency loss is added to the contrastive loss. These two losses are used to jointly train the initial sentence rewriting model and the initial discriminative model. The addition of response consistency between the sample context dialogue and the predicted rewritten sentence can greatly improve the training effect of the model. Furthermore, combining loss at different levels for model training enables the model to learn information at different levels, resulting in better model performance and more effective semantic rewriting. In addition, the model training scheme based on contrastive loss and response consistency loss is applicable to most architectures and has strong universality.

[0105] Based on the above embodiments, Figure 4 This is a schematic diagram of the process for determining response consistency loss provided by the present invention, as shown below. Figure 4 As shown, the response consistency loss is determined based on the following steps:

[0106] Step 410: Based on the initial statement rewriting model and the overall semantic features of the sample multi-turn dialogue, the statement is rewritten to obtain the predicted rewritten statement.

[0107] Step 420: Based on the initial statement rewriting model, extract the predicted semantic features of the predicted rewritten statement and the sample context semantic features of the sample context dialogue.

[0108] Step 430: Determine the response consistency loss based on the similarity between the predicted semantic features and the semantic features of the sample context, as well as the response consistency between the predicted rewritten statement and the sample context dialogue.

[0109] Specifically, the process of determining the consistency loss in response may include the following steps:

[0110] Step 410: First, the initial statement rewriting model can apply the overall semantic features of the sample multi-turn dialogue obtained through semantic extraction to rewrite the sample statements to be rewritten in the sample multi-turn dialogue, so as to obtain the predicted rewritten statements of the sample statements to be rewritten in the sample multi-turn dialogue. Specifically, the overall semantic features of the sample multi-turn dialogue can be input into the decoder of the initial statement rewriting model. The decoder can apply these overall semantic features to rewrite the sample statements to be rewritten, and finally the decoder outputs the predicted rewritten statements.

[0111] Step 420: Subsequently, the initial statement rewriting model can be used to extract the semantic features of the predicted rewritten statement and the sample preceding dialogue, respectively, thereby obtaining the predicted semantic features and the sample preceding dialogue semantic features. Specifically, the predicted rewritten statement and the sample preceding dialogue can be input into the initial statement rewriting model. The encoder in the initial statement rewriting model performs semantic extraction on the input predicted rewritten statement and the sample preceding dialogue, respectively, extracting the features that can represent the semantic information of the entire statement and the features that represent the semantic information of the entire sample preceding dialogue. Finally, the predicted semantic features of the predicted rewritten statement and the sample preceding dialogue semantic features output by the encoder are obtained.

[0112] Here, the predicted semantic features are those that can encompass the overall semantic information of the predicted rewritten statement; the sample context semantic features are those that can reflect the overall semantic information of the sample context dialogue.

[0113] Step 430: After that, the response consistency loss can be calculated based on the similarity between the sample context semantic features and the predicted semantic features, as well as the response consistency between the predicted rewritten statement and the sample context dialogue. The response consistency loss here can be understood as the loss of the sample context statement and the predicted rewritten statement in terms of user response. It can cover the loss of the initial statement rewriting model prediction process and the loss of the initial discrimination model discrimination process.

[0114] Furthermore, since the training objective of the initial sentence rewriting model and the initial discriminant model is to maximize the correlation between the predicted rewritten sentence output by the initial sentence rewriting model and the response of the sample context dialogue, that is, to maximize the similarity between the predicted semantic features output by the initial sentence rewriting model and the semantic features of the sample context, the initial discriminant model outputs 1 when the similarity between the predicted semantic features and the semantic features of the sample context is high, in order to determine that the responses of the two are consistent. Correspondingly, if the prediction of the initial sentence rewriting model is biased, that is, if the similarity between the predicted semantic features output by the initial sentence rewriting model and the semantic features of the sample context is low, the initial discriminant model outputs 0 to determine that the responses of the two are inconsistent.

[0115] Therefore, if the similarity between the predicted semantic features output by the initial statement rewriting model and the semantic features of the sample context is high, and the initial discrimination model determines that the responses between the predicted rewritten statement and the sample context dialogue are consistent, then the response consistency loss is small. Conversely, if the similarity between the predicted semantic features and the semantic features of the sample context is low, and / or if the responses between the predicted rewritten statement and the sample context dialogue are inconsistent, then the response consistency loss is large.

[0116] Based on the above embodiments, Figure 5 This is a flowchart illustrating the joint training process provided by the present invention, as shown below. Figure 5 As shown, in step 120, based on the contrastive loss and the response consistency loss, the initial statement rewriting model and the initial discriminant model are iterated to obtain the statement rewriting model and the response consistency discriminant model, including:

[0117] Step 510: Based on the response consistency between the sample context dialogue and the positive example rewritten statement, and the response consistency between the sample context dialogue and the negative example rewritten statement, perform parameter iteration on the initial discriminant model to obtain the first discriminant model;

[0118] Step 520: Based on the contrast loss and the response consistency loss, perform parameter iteration on the initial statement rewriting model and the first discriminant model to obtain the statement rewriting model and the response consistency discriminant model.

[0119] Specifically, the process of iterating the parameters of the initial statement rewriting model and the initial discriminant model based on the contrast loss and response consistency loss to obtain the statement rewriting model and the response consistency discriminant model may include the following steps:

[0120] Step 510: First, the initial discriminant model can be trained using sample context dialogues, positive example rewritten statements, and negative example rewritten statements to obtain the first discriminant model. That is, the initial discriminant model is iterated with parameters based on the response consistency between sample context dialogues and positive example rewritten statements, as well as the response consistency between sample context dialogues and negative example rewritten statements. Here, response consistency represents the similarity or difference in responses between sample context dialogues and sample rewritten statements, that is, whether the responses output by the initial discriminant model between sample context dialogues and positive example rewritten statements are consistent, and whether the responses between sample context dialogues and positive example rewritten statements are consistent.

[0121] Specifically, the above process can be as follows: First, the sample context dialogue and the positive example rewritten statement are input into the initial discriminant model. Simultaneously, the sample context dialogue and the negative example rewritten statement can also be input into the initial discriminant model. Then, the initial discriminant model performs response consistency discrimination, and finally obtains the response consistency between the sample context dialogue and the positive example rewritten statement, as well as the response consistency between the sample context dialogue and the negative example rewritten statement, as output by the initial discriminant model. After this, based on the response consistency between the sample context dialogue and the positive example rewritten statement, as well as the response consistency between the sample context dialogue and the negative example rewritten statement, the loss of the initial discriminant model in the response consistency discrimination process, i.e., the discrimination loss, can be determined. Based on this discrimination loss, the parameters of the initial discriminant model can be adjusted to finally obtain the first discriminant model.

[0122] The parameter adjustment process for the initial discriminant model is actually based on the discriminant loss. The initial discriminant model is trained so that it can fully learn the response consistency relationship between the sample context dialogue and the sample rewritten statement during the training process. That is, when the input is the sample context dialogue and the positive example rewritten statement, it can output 1 to determine that the two responses are consistent; correspondingly, when the input is the sample context dialogue and the negative example rewritten statement, it can output 0 to determine that the two responses are inconsistent.

[0123] Here, the training process of the initial discriminant model can help the subsequent joint training. Directly loading the pre-trained first discriminant model during the joint training process can greatly accelerate the joint training process and help improve the effect of joint training.

[0124] It should be noted that the initial discrimination model here is built on the Transformer Encoder, and the parameters of the initial discrimination model are shared when performing response consistency judgment on the sample context dialogue and positive example rewritten statement, and on the sample context dialogue and negative example rewritten statement respectively.

[0125] Step 520: Subsequently, based on the pre-trained first discriminant model, response consistency loss and contrast loss can be applied to co-train the first discriminant model and the initial statement rewriting model. In this process, the backpropagation of the discrimination results can assist model training and help improve the performance of the model during joint training.

[0126] In this embodiment of the invention, based on the pre-trained first discriminant model, joint training is performed by combining response consistency loss and contrast loss to optimize model performance. Collaborative training can be performed by backpropagating the gradient information of the first discriminant model and the initial sentence rewriting model. While ensuring model performance, the semantic extraction and sentence rewriting process are improved, and the sentence rewriting effect is optimized.

[0127] Based on the above embodiments, Figure 6 This is a framework example diagram illustrating the training process of the initial discrimination model provided by this invention, as shown below. Figure 6 As shown, the training process of the initial discrimination model includes the following steps:

[0128] First, the sample context dialogue (What's the weather like in City A today?; It's cloudy in City A today) and the positive example rewritten statement (Why is it always cloudy in City A?) can be input into the initial discriminant model. At the same time, the sample context dialogue (What's the weather like in City A today?; It's cloudy in City A today) and the negative example rewritten statement (Where is City A?) can also be input into the initial discriminant model. Then, the initial discriminant model performs response consistency judgment, and finally obtains the judgment result output by the initial discriminant model, that is, the response consistency between the sample context dialogue and the positive example rewritten statement, and the response consistency between the sample context dialogue and the negative example rewritten statement.

[0129] Subsequently, the discrimination loss of the initial discrimination model can be determined based on the response consistency between the sample context dialogue and the positive example rewritten statement, as well as the response consistency between the sample context dialogue and the negative example rewritten statement. Specifically, if the initial discrimination model determines that the responses between the sample context dialogue and the positive example rewritten statement are consistent, and determines that the responses between the sample context dialogue and the negative example rewritten statement are inconsistent, the discrimination loss can be determined to be small; conversely, if the initial discrimination model determines that the responses between the sample context dialogue and the positive example rewritten statement are inconsistent, and / or determines that the responses between the sample context dialogue and the negative example rewritten statement are consistent, the discrimination loss can be determined to be large.

[0130] Then, the parameters of the initial discrimination model can be adjusted based on the discrimination loss to obtain the first discrimination model.

[0131] In this embodiment of the invention, the response consistency between the sample context dialogue and the positive example rewritten statement, as well as the response consistency between the sample context dialogue and the negative example rewritten statement, are used to pre-train the first discriminant model, which can help with subsequent joint training. Directly loading the trained first discriminant model during joint training can significantly improve the joint training process and help improve the joint training effect.

[0132] It is worth noting that the initial discrimination model here is built on the Transformer Encoder, and the parameters of the initial discrimination model are shared when performing response consistency judgment on the sample context dialogue and positive example rewritten statement, and on the sample context dialogue and negative example rewritten statement respectively.

[0133] Based on the above embodiments, Figure 7 This is an example diagram of the framework for the joint training process provided by the present invention, such as... Figure 7 As shown, the joint training process for the initial statement rewriting model and the initial discrimination model can specifically include the following steps:

[0134] First, the sample multi-turn dialogue (What is the weather like in City A today?; It is cloudy in City A today; Why is it always like this?) can be input into the initial sentence rewriting model. The initial sentence rewriting model performs semantic extraction and applies the overall semantic features of the sample obtained from the semantic extraction to rewrite the sentence. Finally, the predicted rewritten sentence (Why is the weather always cloudy in City A?) is output by the initial sentence rewriting model for the sample sentence to be rewritten in the sample multi-turn dialogue.

[0135] Then, the predicted rewritten statement (Why is the weather always gloomy in City A?) and the sample dialogue (How is the weather in City A today?; It is cloudy in City A today) can be used as input to the first discriminant model. The first discriminant model can then determine the consistency of the responses between the predicted rewritten statement and the sample dialogue to obtain the discriminant result, i.e., the similarities and differences in the responses between the two.

[0136] In this process, the contrast loss can be determined based on the similarity between the overall semantic features of the sample and the positive example semantic features obtained by semantic extraction from the initial statement rewriting model, as well as the similarity between the overall semantic features of the sample and the negative example semantic features; and the response consistency loss can be determined based on the response consistency between the sample context dialogue and the predicted rewritten statement output by the first discriminant model, as well as the similarity between the sample context semantic features and the predicted semantic features.

[0137] Subsequently, the initial statement rewriting model and the first discriminant model can be trained by combining the contrastive loss and the response consistency loss, thereby obtaining the statement rewriting model and the response consistency discriminant model.

[0138] It is worth noting that during this process, the prediction loss of the initial sentence rewriting model can also be calculated and applied to model training. This prediction loss is determined based on the predicted rewritten sentences and positive rewritten sentences output by the initial sentence rewriting model. This means determining the similarity between the predicted semantic features and the semantic features of the sample context, and calculating the prediction loss based on this similarity. This similarity can be calculated using cosine similarity, Euclidean distance, Minkowski distance, etc., between semantic features.

[0139] Based on the above embodiments, Figure 8 This is a flowchart illustrating the process of determining the negative instance rewrite statement provided by the present invention, as shown below. Figure 8 As shown, the negative example rewrite statement is determined based on the following steps;

[0140] Step 810: Obtain at least two sample single-turn dialogues;

[0141] Step 820: Based on the initial semantic rewriting model, determine the sample single-turn semantic features of each sample single-turn dialogue;

[0142] Step 830: Based on the similarity between the single-turn semantic features of each sample and the semantic features of the positive example, determine the negative example rewrite statement from at least two single-turn dialogues.

[0143] Specifically, the negative example rewrite statements in the sample rewrite statements can be determined based on the following steps:

[0144] Step 810: First, at least two sample single-talk dialogues can be obtained from the pre-collected sample dialogues.

[0145] Step 820: Subsequently, the initial statement rewriting model can be used to extract semantics from the sample single-turn dialogue, extracting the features that can represent the semantic information of the entire dialogue, thereby obtaining the sample single-turn semantic features of each sample single-turn dialogue. The semantic extraction process based on the initial statement rewriting model has been explained in detail above and will not be repeated here.

[0146] Step 830: Subsequently, based on the similarity between the single-turn semantic features of each sample and the semantic features of the positive example, negative example rewritten statements can be selected from at least two single-turn dialogues in step 810. Specifically, single-turn dialogues with low similarity to the semantic features of the positive example can be identified, and then single-turn dialogue statements from these single-turn dialogues can be selected as negative example rewritten statements. This ensures the difference between the positive and negative example rewritten statements.

[0147] It should be noted that the similarity here can be measured by a preset similarity threshold. A similarity score below the preset threshold is considered low, and the single-turn dialogue samples corresponding to these similarities are identified. Negative examples are then selected from these samples to rewrite the statements. The preset similarity score can be set according to actual needs, for example, it could be 50%, 70%, 85%, etc.; preferably, in this embodiment of the invention, the preset similarity threshold is set to 85%.

[0148] Based on the above embodiments, Figure 9 This is a flowchart illustrating step 830 of the statement rewriting method provided by the present invention, as follows: Figure 9 As shown, step 830 includes:

[0149] Step 831: Select target sample single-turn dialogue from at least two sample single-turn dialogues; the target sample single-turn dialogue is the sample single-turn dialogue corresponding to the first preset number of similarities between the semantic features of each sample single-turn dialogue and the semantic features of the positive example, arranged in ascending order.

[0150] Step 832: Identify negative example rewrite statements from the single-turn dialogue of the target sample.

[0151] Specifically, in step 830, the process of determining the negative example rewritten statements from at least two sample single-turn dialogues based on the similarity between the single-turn semantic features of each sample and the semantic features of the positive example includes the following steps:

[0152] First, in step 831, the similarity between the single-round semantic features of each sample and the semantic features of the positive example can be used as a benchmark to select the target sample single-round dialogue from at least two sets of sample data. Here, the target sample single-round dialogue is the sample single-round dialogue corresponding to the first preset number of similarities in the similarity arranged in ascending order. The preset number here can be set according to actual needs.

[0153] In addition, the target sample single-turn dialogue can also be selected based on a preset similarity threshold. That is, from the similarity between the single-turn semantic features of each sample and the semantic features of the positive example, the similarity less than the preset similarity threshold can be selected, and the single-turn dialogue of the sample corresponding to this part of the similarity can be used as the target sample single-turn dialogue. Alternatively, it can be determined by combining a preset number and a preset similarity threshold. That is, from the similarity arranged in ascending order, the first preset number of sample single-turn dialogues corresponding to similarity less than the preset similarity threshold can be selected as the target sample single-turn dialogue.

[0154] The preset similarity threshold here is a pre-set threshold used to determine the closeness of semantic features between positive and negative example rewritten statements. It represents the degree tolerable that the semantic relationship between positive and negative example rewritten statements is the closest.

[0155] Subsequently, step 832 is executed, from which negative example rewriting statements can be selected from the target sample single-turn dialogue. This can be done by selecting sample single-turn dialogue statements from the target sample that have the same expression form as the positive example rewriting statements. Here, the expression form is a sentence form, such as interrogative sentences, declarative sentences, imperative sentences, etc. In other words, sample single-turn dialogue statements from the target sample that have the same sentence form as the positive example rewriting statements can be selected as negative example rewriting statements.

[0156] For example, when the positive example rewrite statement is "Why is it always cloudy in City A?", the negative example rewrite statement could be "Where is City A?", "How do I get to City A?", "Is City A in the south or the north?", etc.

[0157] Based on the above embodiments, Figure 10 This is a general framework diagram of the training process of the statement rewriting model provided by this invention, as shown below. Figure 10As shown in the embodiment of the present invention, two training tasks are added to the training process of the sentence rewriting model: a semantic similarity training task and a response consistency training task. The semantic similarity training task is a model training task that aims at the overall semantic features of the sample multi-turn dialogue and the positive semantic features of the positive rewritten sentence, as well as the overall semantic features of the sample multi-turn dialogue and the negative semantic features of the negative rewritten sentence. The response consistency training task is a model training task that aims at the response consistency between the sample preceding dialogue and the predicted rewritten sentence.

[0158] When training a model based on these two training tasks, the model parameters are shared. Furthermore, by training the model through different levels of training tasks, the model can learn information at different levels, resulting in better model performance and improved semantic rewriting. In addition, the model training scheme based on these two training tasks is applicable to most architectures and has strong universality.

[0159] Based on the above embodiments, the overall process of the statement rewriting method includes the following steps:

[0160] First, it is necessary to determine the multiple rounds of dialogue;

[0161] Subsequently, based on the statement rewriting model, the overall semantic features of the multi-turn dialogue can be extracted, and the overall semantic features can be applied to rewrite the statements to obtain the rewritten statements.

[0162] The sentence rewriting model is trained based on the similarity between the overall semantic features of the sample multi-turn dialogue and the positive semantic features of the positive rewritten sentences corresponding to the sample multi-turn dialogue, as well as the similarity between the overall semantic features of the sample and the negative semantic features of the negative rewritten sentences corresponding to the sample multi-turn dialogue.

[0163] The similarity here can be calculated using cosine similarity, Euclidean distance, Minkowski distance, etc. among semantic features. Preferably, in this embodiment of the invention, cosine similarity among semantic features is used as the required similarity.

[0164] Furthermore, the sentence rewriting model is trained based on the following steps: First, based on the initial sentence rewriting model, the overall semantic features of the sample multi-turn dialogue, the positive semantic features of the positive rewritten sentences, and the negative semantic features of the negative rewritten sentences are extracted respectively; then, based on the similarity between the overall semantic features of the samples and the positive semantic features, and the similarity between the overall semantic features of the samples and the negative semantic features, the contrast loss is determined; subsequently, based on the contrast loss, the parameters of the initial sentence rewriting model are iterated to obtain the sentence rewriting model.

[0165] The process of obtaining a sentence rewriting model by iterating the parameters of the initial sentence rewriting model based on contrastive loss includes the following steps: Based on contrastive loss and response consistency loss, the initial sentence rewriting model and the initial discriminant model are iterated to obtain a sentence rewriting model and a response consistency discriminant model; wherein the response consistency discriminant model is used to determine the response consistency between the predicted rewritten statement and the preceding dialogue of the sample multi-turn dialogue, and the response consistency loss is determined based on the predicted rewritten statement and the preceding dialogue of the sample.

[0166] Here, the predicted rewritten statement is determined by the statement rewriting model based on the sample multi-turn dialogue, and the preceding dialogue of the sample is the dialogue in the sample multi-turn dialogue excluding the statement to be rewritten.

[0167] The response consistency loss is determined based on the following steps: rewriting sentences based on the initial sentence rewriting model and the overall semantic features of the sample multi-turn dialogue to obtain the predicted rewritten sentences; extracting the predicted semantic features of the predicted rewritten sentences and the sample context semantic features of the sample preceding dialogue based on the initial sentence rewriting model; and determining the response consistency loss based on the similarity between the predicted semantic features and the sample context semantic features, as well as the response consistency between the predicted rewritten sentences and the sample preceding dialogue.

[0168] Furthermore, based on contrastive loss and response consistency loss, the process of iterating the parameters of the initial statement rewriting model and the initial discriminant model to obtain the statement rewriting model and the response consistency discriminant model can specifically include: iterating the parameters of the initial discriminant model based on the response consistency between the sample context dialogue and the positive example rewritten statement, and the response consistency between the sample context dialogue and the negative example rewritten statement, to obtain the first discriminant model; and iterating the parameters of the initial statement rewriting model and the first discriminant model based on contrastive loss and response consistency loss to obtain the statement rewriting model and the response consistency discriminant model.

[0169] The negative example rewriting statement can be determined based on the following steps: First, obtain at least two sample single-turn dialogues; then, based on the initial semantic rewriting model, determine the sample single-turn semantic features of each sample single-turn dialogue; subsequently, select target sample single-turn dialogues from the at least two sample single-turn dialogues. Here, the target sample single-turn dialogues are the sample single-turn dialogues corresponding to the first preset number of similarities between the sample single-turn semantic features and the positive example semantic features, arranged in ascending order. The negative example rewriting statement is then determined from the target sample single-turn dialogues.

[0170] The method provided in this invention uses the similarity between the overall semantic features of sample multi-turn dialogues and the positive semantic features of the corresponding positive example rewritten statements, as well as the similarity between the overall semantic features of the samples and the negative example semantic features of the corresponding negative example rewritten statements, as references for model training. This allows the model to fully learn the semantic relationship between sample multi-turn dialogues and sample rewritten statements under different sample combinations during training, thus providing crucial assistance for accurate statement rewriting. It overcomes the shortcomings of traditional solutions that ignore the inherent connection between sample multi-turn dialogues and sample rewritten statements, resulting in limited model training and poor rewriting effects. This method improves the semantic extraction and statement rewriting process, achieving a dual improvement in the credibility and accuracy of rewritten statements.

[0171] The statement rewriting apparatus provided by the present invention will be described below. The statement rewriting apparatus described below can be referred to in correspondence with the statement rewriting method described above.

[0172] Figure 11 This is a schematic diagram of the structure of the statement rewriting device provided by the present invention, as shown below. Figure 11 As shown, the device includes:

[0173] Dialogue determination unit 1110 is used to determine multi-turn dialogues;

[0174] The statement rewriting unit 1120 is used to extract the overall semantic features of the multi-turn dialogue based on the statement rewriting model, and apply the overall semantic features to rewrite the statements to obtain rewritten statements.

[0175] The statement rewriting model is trained based on the similarity between the overall semantic features of the sample multi-turn dialogue and the positive semantic features of the positive rewritten statement corresponding to the sample multi-turn dialogue, as well as the similarity between the overall semantic features of the sample and the negative semantic features of the negative rewritten statement corresponding to the sample multi-turn dialogue.

[0176] The statement rewriting device provided by this invention uses the similarity between the overall semantic features of sample multi-turn dialogues and the positive semantic features of the positive rewritten statements corresponding to the sample multi-turn dialogues, as well as the similarity between the overall semantic features of the samples and the negative semantic features of the negative rewritten statements corresponding to the sample multi-turn dialogues, as references for model training. This enables the model to fully learn the semantic relationship between sample multi-turn dialogues and sample rewritten statements under different sample combinations during the training process, thereby providing crucial assistance for accurate statement rewriting. It overcomes the shortcomings of traditional solutions that ignore the inherent connection between sample multi-turn dialogues and sample rewritten statements, resulting in limited model training and poor rewriting effects. It improves the semantic extraction and statement rewriting process, achieving a dual improvement in the credibility and accuracy of rewritten statements.

[0177] Based on the above embodiments, the device further includes a model training unit, used for:

[0178] Based on the initial statement rewriting model, the overall semantic features of the sample multi-turn dialogue, the positive semantic features of the positive rewritten statement, and the negative semantic features of the negative rewritten statement are extracted respectively.

[0179] Based on the similarity between the overall semantic features of the sample and the semantic features of the positive example, and the similarity between the overall semantic features of the sample and the semantic features of the negative example, the contrast loss is determined;

[0180] Based on the contrast loss, the initial statement rewriting model is iterated to obtain the statement rewriting model.

[0181] Based on the above embodiments, the model training unit is used for:

[0182] Based on the contrast loss and response consistency loss, the initial statement rewriting model and the initial discrimination model are iterated to obtain the statement rewriting model and the response consistency discrimination model.

[0183] The response consistency discrimination model is used to determine the response consistency between the predicted rewritten statement and the preceding dialogue of the sample multi-turn dialogue, and the response consistency loss is determined based on the predicted rewritten statement and the preceding dialogue of the sample.

[0184] The predicted rewritten statement is determined by the statement rewriting model based on the sample multi-turn dialogue, and the sample preceding dialogue is the dialogue in the sample multi-turn dialogue excluding the sample statement to be rewritten.

[0185] Based on the above embodiments, the apparatus further includes a consistency loss determination unit, used for:

[0186] Based on the initial statement rewriting model and the overall semantic features of the sample multi-turn dialogue, the statement is rewritten to obtain the predicted rewritten statement.

[0187] Based on the initial statement rewriting model, the predicted semantic features of the predicted rewritten statement and the sample context semantic features of the sample context dialogue are extracted.

[0188] Based on the similarity between the predicted semantic features and the semantic features of the sample context, and the response consistency between the predicted rewritten statement and the sample context dialogue, the response consistency loss is determined.

[0189] Based on the above embodiments, the model training unit is used for:

[0190] Based on the response consistency between the sample context dialogue and the positive example rewritten statement, as well as the response consistency between the sample context dialogue and the negative example rewritten statement, the parameters of the initial discriminant model are iterated to obtain the first discriminant model.

[0191] Based on the contrast loss and the response consistency loss, the initial statement rewriting model and the first discriminant model are iterated to obtain the statement rewriting model and the response consistency discriminant model.

[0192] Based on the above embodiments, the device further includes a negative example determination unit, used for:

[0193] Obtain at least two sample single-turn dialogues;

[0194] Based on the initial semantic rewriting model, the single-turn semantic features of each sample's single-turn dialogue are determined.

[0195] Based on the similarity between the single-turn semantic features of each sample and the semantic features of the positive example, negative example rewrite statements are determined from the at least two single-turn dialogues of the samples.

[0196] Based on the above embodiments, the negative example determination unit is used for:

[0197] Select the target sample single-turn dialogue from the at least two sample single-turn dialogues;

[0198] From the single-turn dialogue of the target sample, identify negative examples and rewrite statements;

[0199] The target sample single-turn dialogue is the sample single-turn dialogue corresponding to the first preset number of similarities between the single-turn semantic features of each sample arranged in ascending order and the semantic features of the positive example.

[0200] Figure 12 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 12As shown, the electronic device may include a processor 1210, a communications interface 1220, a memory 1230, and a communication bus 1240, wherein the processor 1210, the communications interface 1220, and the memory 1230 communicate with each other through the communication bus 1240. The processor 1210 can call logical instructions in the memory 1230 to execute a statement rewriting method. This method includes: determining a multi-turn dialogue; extracting the overall semantic features of the multi-turn dialogue based on a statement rewriting model, and applying the overall semantic features to rewrite the statements to obtain rewritten statements; the statement rewriting model is trained based on the similarity between the overall semantic features of a sample multi-turn dialogue and the positive semantic features of the positive rewritten statements corresponding to the sample multi-turn dialogue, and the similarity between the overall semantic features of the sample and the negative semantic features of the negative rewritten statements corresponding to the sample multi-turn dialogue.

[0201] Furthermore, the logical instructions in the aforementioned memory 1230 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0202] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the statement rewriting method provided by the above methods, the method comprising: determining a multi-turn dialogue; extracting the overall semantic features of the multi-turn dialogue based on a statement rewriting model, and applying the overall semantic features to rewrite the statement to obtain a rewritten statement; the statement rewriting model is trained based on the similarity between the overall semantic features of the sample multi-turn dialogue and the positive semantic features of the positive rewritten statement corresponding to the sample multi-turn dialogue, and the similarity between the overall semantic features of the sample and the negative semantic features of the negative rewritten statement corresponding to the sample multi-turn dialogue.

[0203] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the statement rewriting method provided by the above methods. The method includes: determining a multi-turn dialogue; extracting the overall semantic features of the multi-turn dialogue based on a statement rewriting model, and applying the overall semantic features to rewrite the statement to obtain a rewritten statement; the statement rewriting model is trained based on the similarity between the overall semantic features of the sample multi-turn dialogue and the positive semantic features of the positive rewritten statement corresponding to the sample multi-turn dialogue, and the similarity between the overall semantic features of the sample and the negative semantic features of the negative rewritten statement corresponding to the sample multi-turn dialogue.

[0204] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0205] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0206] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A statement rewriting method, characterized in that, include: Establish multiple rounds of dialogue; Based on the statement rewriting model, the overall semantic features of the multi-turn dialogue are extracted, and the overall semantic features are applied to rewrite the statements to obtain the rewritten statements. The statement rewriting model is trained based on the initial statement rewriting model, using response consistency loss, the similarity between the overall semantic features of the sample multi-turn dialogue and the positive semantic features of the positive rewritten statement corresponding to the sample multi-turn dialogue, and the similarity between the overall semantic features of the sample and the negative semantic features of the negative rewritten statement corresponding to the sample multi-turn dialogue. The response consistency loss is determined based on the following steps: Based on the initial statement rewriting model and the overall semantic features of the sample multi-turn dialogue, the statement is rewritten to obtain the predicted rewritten statement. Based on the initial statement rewriting model, the predicted semantic features of the predicted rewritten statement and the sample context semantic features of the sample context dialogue are extracted. The preceding dialogue in the sample refers to the dialogue in the multi-turn dialogue of the sample, excluding the statement to be rewritten in the sample; The response consistency loss is determined based on the similarity between the predicted semantic features and the semantic features of the sample context, as well as the response consistency between the predicted rewritten statement and the sample context dialogue.

2. The statement rewriting method according to claim 1, characterized in that, The statement rewriting model is trained based on the following steps: Based on the initial statement rewriting model, the overall semantic features of the sample multi-turn dialogue, the positive semantic features of the positive rewritten statement, and the negative semantic features of the negative rewritten statement are extracted respectively. Based on the similarity between the overall semantic features of the sample and the semantic features of the positive example, and the similarity between the overall semantic features of the sample and the semantic features of the negative example, the contrast loss is determined; Based on the contrast loss and the response consistency loss, the initial statement rewriting model is iterated to obtain the statement rewriting model.

3. The statement rewriting method according to claim 2, characterized in that, The step of iterating the parameters of the initial statement rewriting model based on the contrast loss and the response consistency loss to obtain the statement rewriting model includes: Based on the contrast loss and the response consistency loss, the initial statement rewriting model and the initial discrimination model are iterated to obtain the statement rewriting model and the response consistency discrimination model. The response consistency discrimination model is used to determine the response consistency between the predicted rewritten statement corresponding to the sample multi-turn dialogue and the preceding dialogue of the sample.

4. The statement rewriting method according to claim 3, characterized in that, The step of iterating the parameters of the initial statement rewriting model and the initial discriminant model based on the contrast loss and the response consistency loss to obtain the statement rewriting model and the response consistency discriminant model includes: Based on the response consistency between the sample context dialogue and the positive example rewritten statement, as well as the response consistency between the sample context dialogue and the negative example rewritten statement, the parameters of the initial discriminant model are iterated to obtain the first discriminant model; Based on the contrast loss and the response consistency loss, the initial statement rewriting model and the first discriminant model are iterated to obtain the statement rewriting model and the response consistency discriminant model.

5. The statement rewriting method according to any one of claims 1 to 4, characterized in that, The negative instance rewrite statement is determined based on the following steps; Obtain at least two sample single-turn dialogues; Based on the initial semantic rewriting model, the single-turn semantic features of each sample's single-turn dialogue are determined. Based on the similarity between the single-turn semantic features of each sample and the semantic features of the positive example, negative example rewrite statements are determined from the at least two single-turn dialogues of the samples.

6. The statement rewriting method according to claim 5, characterized in that, The step of determining negative example rewrite statements from the at least two sample single-turn dialogues based on the similarity between the single-turn semantic features of each sample and the semantic features of the positive example includes: Select the target sample single-turn dialogue from the at least two sample single-turn dialogues; From the single-turn dialogue of the target sample, identify negative examples and rewrite statements; The target sample single-turn dialogue is the sample single-turn dialogue corresponding to the first preset number of similarities between the single-turn semantic features of each sample arranged in ascending order and the semantic features of the positive example.

7. A statement rewriting device, characterized in that, include: Dialogue determination unit, used to determine multi-turn dialogues; The statement rewriting unit is used to extract the overall semantic features of the multi-turn dialogue based on the statement rewriting model, and apply the overall semantic features to rewrite the statements to obtain rewritten statements. The statement rewriting model is trained based on the initial statement rewriting model, using response consistency loss, the similarity between the overall semantic features of the sample multi-turn dialogue and the positive semantic features of the positive rewritten statement corresponding to the sample multi-turn dialogue, and the similarity between the overall semantic features of the sample and the negative semantic features of the negative rewritten statement corresponding to the sample multi-turn dialogue. The response consistency loss is determined based on the following steps: Based on the initial statement rewriting model and the overall semantic features of the sample multi-turn dialogue, the statement is rewritten to obtain the predicted rewritten statement. Based on the initial statement rewriting model, the predicted semantic features of the predicted rewritten statement and the sample context semantic features of the sample context dialogue are extracted. The preceding dialogue in the sample refers to the dialogue in the multi-turn dialogue of the sample, excluding the statement to be rewritten in the sample; The response consistency loss is determined based on the similarity between the predicted semantic features and the semantic features of the sample context, as well as the response consistency between the predicted rewritten statement and the sample context dialogue.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the statement rewriting method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the statement rewriting method as described in any one of claims 1 to 6.

Images