Multi-turn dialogue rewriting methods, equipment and media

Abstract

This application relates to a method, device, and medium for rewriting multi-turn dialogues, belonging to the field of deep learning technology. The method includes: acquiring historical dialogue statements between the user and an intelligent dialogue system and the statement to be responded to in the current turn; determining the rewriting type of the statement to be responded to in the current turn; identifying recognition words that match the rewriting type in the historical dialogue statements; identifying extraction words that match the rewriting type in the statement to be responded to in the current turn; rewriting the recognition words using the extraction words according to the rewriting method indicated by the rewriting type, thereby obtaining the rewritten statement to be responded to. This method can solve the problem that the process of generating a complete statement takes a long time because the dialogue data needs to be encoded into a sequence from the first sentence and a label corresponding to the sequence needs to be generated. Furthermore, the sequence to be rewritten needs to be found based on the label. This results in a slow system response statement generation process, which affects the user experience.

Description

Technical Field

[0001] This application relates to a method, device, and medium for rewriting multi-turn dialogues, belonging to the field of deep learning technology. Background Technology

[0002] Multi-turn dialogue rewriting refers to the technique of rewriting an incomplete statement into a semantically complete statement that can be understood independently of its context. A single-turn dialogue refers to a user's input of a single query and the corresponding system response. Multi-turn dialogue rewriting is commonly used in intelligent dialogue systems to enable the system to effectively respond to user queries. In multi-turn dialogues between users and intelligent dialogue systems, users often input abbreviated or referential queries. Since intelligent dialogue systems cannot understand semantically incomplete statements, multi-turn dialogue rewriting is necessary to rewrite the incomplete statements input by the user.

[0003] In traditional multi-turn dialogue rewriting methods, historical single-turn dialogues and the current incomplete sentence are usually used as dialogue data input to a pre-trained model. The dialogue data is encoded into different sequences according to words or phrases starting from the first sentence, and different labels are generated based on different sequences. The labels in the incomplete sentence are used to indicate words that need to be deleted or added, and the labels in the historical single-turn dialogue are used to indicate words that need to be added or replaced to the same label position in the incomplete sentence. Then, the corresponding sequences are decoded by a machine translation model based on the labels to obtain the rewritten complete sentence.

[0004] However, because the dialogue data needs to be encoded into a sequence starting from the first sentence and tags corresponding to the sequence need to be generated, and the sequence that needs to be rewritten needs to be found based on the tags, the process of generating a complete sentence takes a long time, resulting in the system generating response sentences slowly and affecting the user experience. Summary of the Invention

[0005] This application provides a method, device, and medium for rewriting multi-turn dialogues. It solves the problem that the process of generating complete sentences is time-consuming because dialogue data needs to be encoded into a sequence starting from the first sentence, and tags corresponding to the sequence need to be generated. Subsequent steps require finding the sequence to be rewritten based on the tags. This results in slow response generation and negatively impacts user experience. This application provides the following technical solution:

[0006] Firstly, a method for rewriting multi-turn dialogues is provided, applied to an intelligent dialogue system, the method comprising:

[0007] The system acquires the user's historical dialogue statements and the current pending response statement between the user and the intelligent dialogue system; the historical dialogue statements include at least one historical pending response statement and the system response statement corresponding to each historical pending response statement.

[0008] Determine the rewrite type of the statements to be responded to in this round;

[0009] Identify the rewriting type by identifying the corresponding words in the historical dialogue statements;

[0010] In the current round of responses, identify the extracted words that match the rewrite type;

[0011] The extracted words are used to rewrite the identified words according to the rewriting method indicated by the rewriting type, resulting in the rewritten response statement.

[0012] Optionally, determining the rewrite type of the statement to be responded to in this round includes:

[0013] Obtain preset type prompt information and a first recognition model, wherein the type prompt information is used to instruct the first recognition model to perform a task of recognizing and rewriting the type;

[0014] The historical dialogue statements, the statements to be answered in the current round, and the type hint information are merged to obtain a type dataset;

[0015] The type dataset is input into the first recognition model to obtain the rewritten type.

[0016] Optionally, before obtaining the preset first recognition model, the following steps are also included:

[0017] The first historical training statement, the first training statement to be responded to, and the type prompt information are combined to obtain type recognition training data;

[0018] Input the type recognition training data into the initial first recognition model to obtain the initial rewrite type corresponding to the first training statement to be responded to.

[0019] The first comparison result is obtained by comparing the actual rewritten type corresponding to the first training statement to be responded to with the initial rewritten type.

[0020] The first recognition model is trained using the first comparison result to obtain the first recognition model.

[0021] Optionally, before obtaining the preset type hint information, the following is also included:

[0022] Get the prompt message template;

[0023] The prompt information template is used to generate prompt information according to the recognition purpose of the first recognition model. The recognition purpose includes recognizing the rewrite type. Accordingly, the prompt information includes the type prompt information.

[0024] Optionally, determining the identification words in the historical dialogue statements that match the rewriting type includes:

[0025] Obtain preset extraction prompt information and a second recognition model, wherein the extraction prompt information is used to instruct the second recognition model to perform the task of extracting and recognizing words;

[0026] The historical dialogue statements, the statements to be answered in the current round, and the extracted prompt information are combined to obtain the extracted dataset.

[0027] The extracted dataset is input into the second recognition model to obtain the recognition words.

[0028] Optionally, before obtaining the preset second recognition model, the method further includes:

[0029] The second historical training statement, the second training statement to be answered, and the extraction prompt information are combined to obtain the extraction training data.

[0030] The extracted training data is input into the initial second recognition model to obtain the initial recognition word corresponding to the second training statement to be answered.

[0031] The real recognition word corresponding to the second training statement to be answered is compared with the initial recognition word to obtain a second comparison result;

[0032] The second recognition model is trained using the second comparison result to obtain the second recognition model.

[0033] Optionally, the rewrite types include: replacement and completion;

[0034] The step of rewriting the identified word using the extracted word according to the rewriting method indicated by the rewriting type to obtain the rewritten response statement includes:

[0035] When the rewrite type is replacement, the extracted word is replaced with the identified word;

[0036] When the rewrite type is completion, the identified word is added based on the position of the extracted word.

[0037] Optionally, before obtaining the user's historical dialogue statements and the statements to be answered in the current round, the method further includes:

[0038] The system acquires the user's initial historical dialogue statements and the initial current round of pending responses from the intelligent dialogue system; the initial historical dialogue statements include at least one initial historical pending response statement and the initial system response statement corresponding to each initial historical pending response statement.

[0039] Perform syntax correction and / or type transformation on the initial historical dialogue statements to obtain the historical dialogue statements and the statements to be responded to in the current round.

[0040] In a second aspect, an electronic device is provided, the electronic device including a processor and a memory connected to the processor, the memory storing a program, and the processor executing the program to implement the multi-turn dialogue rewriting method provided in the first aspect.

[0041] Thirdly, a computer-readable storage medium is provided, wherein a program is stored in the storage medium, and the program, when executed by a processor, is used to implement the multi-turn dialogue rewriting method provided in the first aspect.

[0042] The beneficial effects of this application include at least the following: by acquiring the historical dialogue statements between the user and the intelligent dialogue system and the current round of statements to be answered; the historical dialogue statements include at least one historical statement to be answered and the system response statement corresponding to each historical statement to be answered, determining the rewriting type of the current round of statements to be answered, identifying the recognition words that match the rewriting type in the historical dialogue statements, identifying the extraction words that match the rewriting type in the current round of statements to be answered, and rewriting the recognition words using the extraction words according to the rewriting method indicated by the rewriting type, the rewritten statement to be answered is obtained; this can solve the problem that because the dialogue data needs to be encoded into a sequence from the first sentence and labels corresponding to the sequence need to be generated, and the sequence to be rewritten needs to be found based on the labels, the generation process of the complete statement takes a long time, resulting in a slow system response statement generation and affecting the user experience; since the rewriting type corresponding to the current round of statements to be answered is obtained first, and then the recognition words to be rewritten are obtained through the rewriting type, instead of generating labels for all words based on the sequence of dialogue data, or finding the sequence to be rewritten based on the labels, the generation time of the complete statement can be shortened, thereby improving the speed of the system response statement generation.

[0043] In addition, by merging historical dialogue statements, current response statements, and type hints, a type dataset is obtained. Inputting the type dataset into the first recognition model yields the rewritten type. This allows the first recognition model to obtain the rewritten type corresponding to the current response statement based on the semantic relationship between historical dialogue statements and the current response statement, thereby improving the accuracy of the rewritten type output by the model.

[0044] Furthermore, type recognition training data is obtained by merging type hints with the first historical training statement and the first training statement to be responded to, and this data is used to train the initial first model. Since type hints can instruct the model to perform the task of recognizing and rewriting types, the prior knowledge advantage of the initial first recognition model can be fully utilized, while also reducing the amount of training data used during model training and reducing the consumption of training resources.

[0045] Furthermore, since different models have different task objectives, the corresponding prompts will also be different. Prompts that do not match the task objectives will lead to poor performance of the model. Therefore, by using prompt templates to generate prompts, prompts that match the task objectives of the model can be obtained, thus improving the model's performance. Moreover, since different prompts can instruct the model to perform different tasks to achieve different task objectives, the same model can be used to perform different tasks, which can also improve the flexibility of the model's output.

[0046] In addition, performing syntax correction and / or type transformation on the acquired initial historical dialogue statements and the initial current round of responses can avoid the problem that the intelligent dialogue system cannot handle different types of data. It can also avoid the problem that the intelligent dialogue system will make recognition errors due to syntax errors. Therefore, it can improve the recognition accuracy of the intelligent dialogue system.

[0047] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0048] Figure 1 This is a flowchart of a multi-turn dialogue rewriting method provided in one embodiment of this application;

[0049] Figure 2 This is a block diagram of an apparatus for a multi-turn dialogue rewriting method provided in one embodiment of this application;

[0050] Figure 3 This is a block diagram of an electronic device provided in one embodiment of this application. Detailed Implementation

[0051] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but are not intended to limit the scope of this application.

[0052] First, several terms used in the embodiments of this application will be introduced.

[0053] Intelligent dialogue system: refers to a system that uses various intelligent algorithms to enable machines to understand human language and engage in effective communication and dialogue with humans, and then respond based on the understanding of the intention in human language.

[0054] Pre-trained model: refers to a model that has been pre-trained through self-supervised learning using a large amount of unlabeled data.

[0055] Self-supervised learning (SSL) can learn a general feature representation for the task at hand. Its main approach is self-supervision; specifically, it uses unlabeled data to construct an auxiliary task. For this auxiliary task, labels are derived from the data itself, and the network is trained in a supervised manner. For example, removing a few words from a sentence and using its context to predict the missing words.

[0056] Prompt-learning is a mechanism that, without significantly altering the structure and parameters of a pre-trained model, transforms the actual task into a text generation task by adding "hints" to the input. The prompt, or hint, is input along with the data into the pre-trained model, informing and guiding it on what task to perform next.

[0057] Figure 1 This is a flowchart of a multi-turn dialogue rewriting method provided in one embodiment of this application. The method is applied to an intelligent dialogue system; schematically, the following embodiments illustrate this using an intelligent dialogue system installed in an electronic device as an example. The following embodiments illustrate the application of this method to an electronic device, specifically to a processor within the electronic device. The electronic device includes, but is not limited to, mobile devices, tablets, computers, etc. This embodiment does not limit the implementation method of the electronic device.

[0058] In practical implementation, this method can also be used in other devices that communicate with electronic devices, such as user terminals or servers. User terminals include, but are not limited to, mobile phones, computers, tablets, wearable devices, etc. This embodiment does not limit the implementation methods of other devices or user terminals.

[0059] This method includes at least the following steps:

[0060] Step 101: Obtain the user's historical dialogue statements with the intelligent dialogue system and the statements to be responded to in the current round. The historical dialogue statements include at least one historical statement to be responded to and the system response statement corresponding to each historical statement to be responded to.

[0061] When the intelligent dialogue system receives voice data input by the user, it performs intent recognition on the voice data. If the semantics of the voice data are recognized, a preset dialogue algorithm is used to generate a response statement for the voice data. If the semantics of the voice data are not recognized, the voice data is determined to be the statement to be responded to in the current round, and steps 102-105 are executed to rewrite the statement to be responded to in the current round, generating a response statement for the rewritten statement to be responded to using the preset dialogue algorithm. Optionally, after generating the response statement for the rewritten statement to be responded to, the statement to be responded to in the current round and the response statement for the rewritten statement to be responded to can be used as historical dialogue statements for subsequent dialogues to rewrite the statements to be responded to in subsequent dialogues.

[0062] Intent recognition refers to identifying the themes and intentions contained in text. Intent recognition algorithms include word vector-based intent recognition, CNN-based intent recognition, and RNN-based intent recognition, among others.

[0063] Therefore, in this embodiment, the statement to be answered in this round is a semantically unrecognizable incomplete statement that needs to be rewritten, while the historical dialogue statements are semantically complete statements that have already been rewritten or do not need to be rewritten.

[0064] Because users often omit some words from previous conversations or replace them with pronouns when engaging in multi-round dialogues with intelligent dialogue systems, the semantics of the current response statement may be incomplete.

[0065] For example:

[0066] The historical dialogue statements are:

[0067] User: "In which year was XX born?"

[0068] Intelligent dialogue system: "XX was born in 1980";

[0069] The statement to be responded to in this round is:

[0070] User: "Who is his wife?"

[0071] If the current round of responses contains the pronoun "he", which refers to "XX" in the previous round of responses, the intelligent dialogue system cannot recognize the meaning of the pronoun "he" if only the current round of responses is used, resulting in the intelligent dialogue system being unable to output a valid response. Therefore, the intelligent dialogue system needs to obtain at least one round of historical dialogue statements in order to determine the information omitted or indicated by pronouns in the current round of responses.

[0072] After the intelligent dialogue system receives the initial response statement input by the user, the data type of this initial response statement stored in the system may be inconsistent with the data types the system can process. In this case, the intelligent dialogue system may be unable to process the initial response statement. Therefore, when the data type of the initial response statement is inconsistent with the data types the intelligent dialogue system can process, the data type needs to be transformed. Furthermore, users often use colloquial language and make spelling mistakes when interacting with the intelligent dialogue system, and the system's recognition of colloquial language and misspelled initial response statements is poor. Therefore, when the initial response statement contains colloquial language and misspellings, grammatical correction is necessary.

[0073] Based on the above issues, before obtaining the historical dialogue statements between the user and the intelligent dialogue system and the statements to be responded to in the current round, the following steps are also included: obtaining the initial historical dialogue statements between the user and the intelligent dialogue system and the initial statements to be responded to in the current round, performing syntax correction and / or type transformation on the initial historical dialogue statements to obtain the historical dialogue statements and the statements to be responded to in the current round.

[0074] The initial history dialogue statement includes at least one initial history pending response statement and the initial system response statement corresponding to each initial history pending response statement.

[0075] Syntax correction refers to correcting grammatically incorrect input sentences and outputting grammatically correct sentences. Syntax errors include: misspellings, disordered word order, punctuation errors, and / or spelling errors.

[0076] Methods for syntax error correction include: Grammatical Error Correction (GECToR) algorithms, candidate recall algorithms, or the use of syntax error correction tools.

[0077] For example, if a user inputs the statement "XX was born in which year", a syntax correction tool can be used to correct the syntax of the statement, resulting in the corrected statement "XX was born in which year".

[0078] Data type conversion methods include: automatic type conversion, forced type conversion, etc.

[0079] Automatic type conversion occurs when the source and target types of the data to be converted are compatible, or when the target type is broader than the source type. The precision of the data after automatic type conversion is not reduced.

[0080] Forced type conversion occurs when the source type of the data to be converted is greater than the target type, thus forcing a conversion from the source type to the target type. Forced type conversion may result in data overflow and reduced data precision.

[0081] By transforming the data type to match the data types that the intelligent dialogue system can process, the system can then effectively process the acquired data. Furthermore, by performing syntactic error correction on the initial historical dialogue statements and the initial statements to be responded to in the current round, semantically clear statements can be obtained, thereby improving the recognition performance of the intelligent dialogue system.

[0082] Step 102: Determine the rewrite type of the statement to be responded to in this round.

[0083] The rewrite type indicates the rewrite method that should be used for the statements to be responded to in this round.

[0084] Rewrite types include: replacement and completion.

[0085] In one example, determining the rewrite type of the statement to be responded to in the current round includes: obtaining a third recognition model, merging historical dialogue statements and the statement to be responded to in the current round to obtain a customized type dataset, and inputting the customized type dataset into the third recognition model to obtain the rewrite type.

[0086] The third identification model is a pre-trained neural network model used to identify the rewrite type of the statement to be responded to in this round.

[0087] Because the semantics of the current response statement are incomplete, the rewrite type obtained by the third recognition model solely from the current response statement is not accurate enough. Therefore, it is necessary to input both the historical dialogue statements and the current response statement into the third recognition model simultaneously. The third recognition model obtains the accurate rewrite type based on the semantic connection features between the historical dialogue statements and the current response statement. However, since the model cannot obtain the related semantic connection features between the historical dialogue statements and the current response statement when inputting them separately, it is necessary to merge the historical dialogue statements and the current response statement so that they can be input into the third recognition model together.

[0088] To avoid confusing the current dialogue statements with the historical dialogue statements, the historical dialogue statements and the current dialogue statements are merged to obtain a customized type dataset, including: merging the historical dialogue statements and the current dialogue statements according to the first preset rule to obtain a customized type dataset.

[0089] The first preset rule can be pre-stored in the electronic device or obtained from other devices. This embodiment does not limit the method of obtaining the first preset rule.

[0090] For example, the first preset rule is: merge the historical dialogue statements and the statements to be responded to in the current round in the format of {historical statements to be responded to [sep]system response statements [sep]current statements to be responded to} to obtain a customized type dataset. Here, sep is a delimiter used to separate different statements.

[0091] Optionally, before obtaining the third recognition model, the initial third recognition model needs to be trained using training data to obtain the third recognition model. Training the initial third recognition model using training data to obtain the third recognition model includes: merging the first historical training statement and the first training statement to be answered to obtain customized type training data; inputting the customized type training data into the initial third recognition model to obtain the initial rewritten type corresponding to the first training statement to be answered; comparing the actual rewritten type corresponding to the first training statement to be answered with the initial rewritten type to obtain a third comparison result; and training the initial third recognition model using the third comparison result to obtain the third recognition model.

[0092] Indicatively, the methods for obtaining the first historical training statement and the first training statement to be responded to include: web crawling, and / or public datasets, etc. This embodiment does not limit the methods for obtaining the first historical training statement and the first training statement to be responded to.

[0093] Web crawlers, also known as web spiders, are programs or scripts that automatically retrieve information from the web according to certain rules. Examples of web crawler programs include Google crawlers and Baidu crawlers.

[0094] Since it is usually necessary to connect the first training statement to be responded to with the historical training statement, that is, there is a semantic relationship between the historical training statement and the first training statement to be responded to, the first historical training data and the first training statement to be responded to are merged based on the first preset rule to obtain customized type training data. The third recognition model is trained using the customized type training dataset, which can enable the model to discover the semantic relationship features between the historical training statement and the first training statement to be responded to, thereby improving the accuracy of the obtained rewrite type.

[0095] The format of the first custom type training data is the same as that of the custom type dataset.

[0096] Indicatively, the initial third recognition model is a pre-trained model. The pre-trained model includes: a Bidirectional Encoder Representation from Transformers (BERT) + Conditional Random Field (CRF) model, a lightweight BERT (A Lite BERT, ALBERT) model, a binary classification model based on BERT fine-tuning, etc. This embodiment does not limit the implementation method of the pre-trained model.

[0097] Since the pre-trained model has already undergone the pre-training process, it contains a large amount of prior knowledge. When training it based on the actual task to be achieved, the training time can be reduced.

[0098] However, although pre-trained models contain a large amount of prior knowledge, they cannot accurately utilize prior knowledge relevant to the actual task being performed, thus failing to leverage the advantages of prior knowledge. Therefore, a significant amount of training data is still required to train the pre-trained model.

[0099] Therefore, in another example, determining the rewritten type of the statement to be responded to in the current round includes: obtaining preset type hint information and a first recognition model; merging historical dialogue statements, the statement to be responded to in the current round, and the type hint information to obtain a type dataset; and inputting the type dataset into the first recognition model to obtain the rewritten type.

[0100] Among them, the type hint information belongs to the hint learning mechanism, which is used to instruct the first recognition model to perform the task of recognizing and rewriting the type.

[0101] Since the first recognition model may be used to perform different tasks, type hints can be used to instruct the first recognition model to call upon the knowledge related to the type of rewriting obtained during pre-training, thereby efficiently performing the task of recognizing and rewriting types.

[0102] Before obtaining the preset type prompt information, the method further includes: obtaining a prompt information template; using the prompt information template to generate prompt information according to the recognition purpose of the first recognition model, wherein the recognition purpose includes recognizing and rewriting types, and correspondingly, the prompt information includes type prompt information.

[0103] The prompt message template and the recognition purpose can be input by the user or obtained from other devices. This embodiment does not limit the method of obtaining the prompt message template and the recognition purpose.

[0104] For example, if the prompt message template is "What is CATE", and the recognition purpose is to identify "rewrite type", where "CATE" is used to characterize the recognition purpose of the first recognition model, then the type prompt message generated based on the prompt message template and the recognition purpose is "What is rewrite type".

[0105] In actual implementation, the prompt message template can also be "What is this CATE", "CATE", etc., where "CATE" is used to characterize the recognition purpose of the first recognition model, and different recognition purposes correspond to different prompt messages.

[0106] The historical dialogue statements, the statements to be responded to in the current round, and the type hint information are merged to obtain a type dataset, including: based on the first merging rule, the historical dialogue statements, the statements to be responded to in the current round, and the type hint information are merged to obtain a type dataset.

[0107] The first merging rule can be pre-stored in the electronic device or obtained from other devices. This embodiment does not limit the method of obtaining the first merging rule.

[0108] Specifically, based on the first merging rule: {historical pending response statement [sep]system response statement [sep]current pending response statement [sep]type hint information}, the historical dialogue statements, current pending response statements, and type hint information are merged to obtain a type dataset. Here, sep is a delimiter used to separate different statements.

[0109] For example, a set of data in a public dataset is {

[0110] User: "In which year was XX born?"

[0111] Intelligent dialogue system: "1980";

[0112] User: "Who is his wife?"

[0113] The default type hint message is: "What is the rewrite type?"

[0114] The concatenated dataset is of type {XX was born in which year[sep]1980[sep]who is his wife[sep]what is the rewritten type}.

[0115] Since the input data is the same when using and training the model, type hints are also needed when pre-training the neural network model before obtaining the first recognition model.

[0116] Therefore, before obtaining the preset first recognition model, the process includes: merging the first historical training statement, the first training statement to be answered, and the type prompt information to obtain type recognition training data; inputting the type recognition training data into the initial first recognition model to obtain the initial rewritten type corresponding to the first training statement to be answered; comparing the actual rewritten type corresponding to the first training statement to be answered with the initial rewritten type to obtain a first comparison result; and using the first comparison result to train the initial first recognition model to obtain the first recognition model.

[0117] At this point, the format of the type recognition training data is the same as the format of the type dataset.

[0118] Indicatively, the initial first recognition model can be a pre-trained model.

[0119] Since type hints can instruct the initial first recognition model to call prior knowledge related to the type of rewriting, combining the hint learning mechanism with the pre-trained model can fully leverage the advantage of the prior knowledge of the pre-trained model, thereby reducing the training data used in the model training process. This allows the initial first recognition model to obtain a first recognition model with a high recognition accuracy using a small amount of training data.

[0120] Step 103: Identify the recognition words that match the rewriting type in the historical dialogue statements.

[0121] Identification words refer to information that should exist in the current round of responses but has been replaced by pronouns or omitted.

[0122] In one example, identifying recognition words that match the rewriting type in historical dialogue statements includes: obtaining preset extraction prompts and a second recognition model, whereby the extraction prompts instruct the second recognition model to perform the task of extracting recognition words; merging historical dialogue statements, current response statements, and extraction prompts to obtain an extraction dataset; and inputting the extraction dataset into the second recognition model to obtain recognition words.

[0123] The methods for obtaining preset extraction prompts include, but are not limited to, the following:

[0124] The first method: Use the prompt message template to generate extraction prompt messages according to the recognition purpose of the second recognition model.

[0125] In the first implementation, the prompt template is a template that is applicable to both type-based prompts and extracted prompts.

[0126] For example, if the prompt template is "What is CATE?", then the type prompt will be "What is the rewrite type?", and the extracted prompt will be "What is the recognition word?".

[0127] Since the second recognition model performs the task according to the recognition purpose indicated by the extracted prompt information, when the recognition purpose indicated by the extracted prompt information does not match the actual recognition purpose that the user expects the second recognition model to recognize, the recognition result reached by the second recognition model will also not match the actual recognition purpose. Therefore, the closer the extracted prompt information matches the actual recognition purpose of the second recognition model, the better the effect of the second recognition model. In the first implementation, since the extracted prompt information only includes the recognition purpose of the recognition word and does not have the corresponding rewriting type of the recognition word, and different rewriting types may correspond to different recognition words, the second recognition model may not be able to determine which rewriting type of recognition word needs to be recognized based on the extracted prompt information, resulting in low accuracy in recognizing the recognition word. Based on this, this embodiment also provides the following second method.

[0128] The second method involves generating extraction prompts using prompt templates and rewrite types. Different rewrite types correspond to different extraction prompts.

[0129] In the second implementation, the prompt message template is also a template that is applicable to both type-based prompt messages and extraction prompt messages.

[0130] For example, when the rewrite type is "replacement type", the purpose of the second recognition model is to identify "recognition words of the replacement type", and the extracted prompt information is "what is the recognition word of the replacement type"; when the rewrite type is "complete type", the purpose of the second recognition model is to identify "recognition words of the complete type", and the extracted prompt information is "what is the recognition word of the complete type".

[0131] At this point, since the recognition purpose indicated by the extracted prompt information is what the user expects, the second recognition model performs the task according to the recognition purpose indicated by the extracted prompt information, and the obtained recognition result matches the recognition purpose expected by the user, resulting in better performance.

[0132] The third method involves replacing the recognition objective of the first recognition model in the type prompt information with the recognition objective of the second recognition model to obtain the extraction prompt information.

[0133] For example, if the type prompt is "What is the rewrite type?", the first recognition model aims to identify "rewrite type", while the second recognition model aims to identify "recognition word". In this case, using "What is the rewrite type?" as a template, the "recognition word" is used to replace "rewrite type" in the template to obtain the extracted prompt "What is the recognition word?".

[0134] Optionally, the first recognition model and the second recognition model may be the same or different. If the first and second recognition models are the same, different prompts can be used to instruct the models to perform different tasks.

[0135] The extracted dataset is built based on a second merging rule, which differs from the first merging rule. The extracted dataset also varies depending on the rewrite type.

[0136] Optionally, before obtaining the preset second recognition model, it is necessary to train the initial second recognition model using training data. Training the initial second recognition model using training data includes: merging the second historical training statement, the second training statement to be answered, and the extracted prompt information to obtain extracted training data; inputting the extracted training data into the initial second recognition model to obtain the initial recognition word corresponding to the second training statement to be answered; comparing the actual recognition word corresponding to the second training statement to be answered with the initial recognition word to obtain a second comparison result; and using the second comparison result to train the initial second recognition model to obtain the second recognition model.

[0137] In one example, the training data used by the second recognition model is different from that used by the first recognition model. In this case, the second historical training statement and the second training statement to be answered have corresponding real rewrite types. Since different rewrite types correspond to different extraction prompts, before merging the historical training statement, the training statement to be answered, and the extraction prompts to obtain the extraction training data, the process also includes: obtaining the real rewrite type of the training statement to be answered, and obtaining the extraction prompts corresponding to the training statement to be answered based on the real rewrite type.

[0138] In another example, the training data used by the second recognition model is the same as that used by the first recognition model. In this case, the rewritten type of the second historical training statement and the second training statement to be responded to can be identified by the first recognition model, or it can be set by the user. This embodiment does not limit the method of obtaining the rewritten type. Taking the rewritten type of the second historical training statement and the second training statement to be responded to as an example, in this case, before merging the historical training statement, the training statement to be responded to, and the extraction prompt information to obtain the extraction training data, the method further includes: inputting the historical training statement and the training statement to be responded to into the first recognition model to obtain the rewritten type; and obtaining the extraction prompt information corresponding to the training statement to be responded to based on the identified rewritten type.

[0139] In this example, the methods for obtaining the second historical training statement and the second training statement to be responded to include: web crawling, and / or public datasets, etc. This embodiment does not limit the methods for obtaining the first historical training statement and the first training statement to be responded to.

[0140] In another example, extraction prompts can be omitted. In this case, the identification words that match the rewriting type are determined from the historical dialogue statements, including: obtaining the fourth identification model; merging the historical dialogue statements and the statements to be answered in the current round to obtain a customized extraction dataset; and inputting the customized extraction dataset into the fourth identification model to obtain the identification words.

[0141] The customized extraction dataset is built based on the second preset rule, which may be the same as or different from the first preset rule.

[0142] At this point, before obtaining the fourth recognition model, the training process for training the initial fourth recognition model is as follows: merge the second historical training statement and the second training statement to be answered to obtain customized extracted training data; input the customized extracted training data into the initial fourth recognition model to obtain the initial recognition word corresponding to the second training statement to be answered; compare the actual recognition word corresponding to the second training statement to be answered with the initial recognition word to obtain the fourth comparison result; use the fourth comparison result to train the initial fourth recognition model to obtain the fourth recognition model.

[0143] Step 104: Identify the extractable words that match the rewrite type from the statements to be responded to in this round.

[0144] The words extracted will vary depending on the type of rewriting.

[0145] Specifically, when the rewrite type is replacement, the extracted words are the replacement words corresponding to the replacement type.

[0146] When the rewrite type is completion, the extracted word is the insertion word corresponding to the completion type.

[0147] Since the principle of obtaining the extracted words is the same as that of obtaining the identified words after replacing the identified words with the extracted words and replacing the historical dialogue statements with the statements to be answered in the current round in step 103, this embodiment will not elaborate on them one by one here.

[0148] Optionally, step 104 can be executed before step 103, and steps 103 and 104 can be executed simultaneously using the second recognition model. This embodiment does not limit the execution order of steps 103 and 104.

[0149] Step 105: Rewrite the identified words using the extracted words according to the rewriting method indicated by the rewriting type to obtain the rewritten response statement.

[0150] Indicatively, the extracted words are used to rewrite the identified words according to the rewriting method indicated by the rewriting type to obtain the rewritten response statement, including: when the rewriting type is replacement, the extracted words are replaced with the identified words; when the rewriting type is completion, the identified words are added based on the position of the extracted words.

[0151] For example, the historical dialogue statement is "In which year was XX born?" "1980"; while the statement to be responded to in this round is "Who is his wife?" In this case, the intelligent rewriting system identifies the rewriting type as "replacement", the extracted word corresponding to the replacement type is "he", and the recognized word is "XX". The extracted word is then replaced with the recognized word to obtain the rewritten complete statement "Who is XX's wife?"

[0152] Specifically, when the rewrite type is completion, the identified word can be added before or after the extracted word, depending on different needs.

[0153] For example, the historical dialogue statement is "XX was born in which year" or "1980"; while the statement to be responded to in this round is "in which year did he retire". At this time, the intelligent rewriting system identifies the rewriting type as "complete", the extraction word corresponding to the complete type is "which year", and the recognition word is "XX". Then, the recognition word is added to the front of the extraction word to get the rewritten complete statement "XX retired in which year".

[0154] In summary, the multi-turn dialogue rewriting method provided in this embodiment obtains the historical dialogue statements between the user and the intelligent dialogue system, as well as the statement to be answered in the current turn. The historical dialogue statements include at least one historical statement to be answered and the system response statement corresponding to each historical statement to be answered. The rewriting type of the statement to be answered in the current turn is determined. Identification words matching the rewriting type are identified in the historical dialogue statements. Extraction words matching the rewriting type are identified in the statement to be answered in the current turn. The identification words are rewritten using the extraction words according to the rewriting method indicated by the rewriting type, resulting in the rewritten statement to be answered. This method can solve the problem of needing to start the dialogue data from the first sentence. The process of encoding the initial statement into a sequence and generating corresponding labels, followed by finding the sequence to be rewritten based on the labels, takes a long time to generate the complete statement, resulting in slow response generation and impacting user experience. However, by first obtaining the rewrite type of the statement to be responded to in the current round, and then obtaining the words to be rewritten based on the rewrite type, instead of generating labels for all words based on the dialogue data sequence or finding the sequence to be rewritten based on the labels, the time to generate the complete statement can be shortened, thus improving the speed of the system in generating response statements.

[0155] In addition, by merging historical dialogue statements, current response statements, and type hints, a type dataset is obtained. Inputting the type dataset into the first recognition model yields the rewritten type. This allows the first recognition model to obtain the rewritten type corresponding to the current response statement based on the semantic relationship between historical dialogue statements and the current response statement, thereby improving the accuracy of the rewritten type output by the model.

[0156] Furthermore, type recognition training data is obtained by merging type hints with the first historical training statement and the first training statement to be responded to, and this data is used to train the initial first model. Since type hints can instruct the model to perform the task of recognizing and rewriting types, the prior knowledge advantage of the initial first recognition model can be fully utilized, while also reducing the amount of training data used during model training and reducing the consumption of training resources.

[0157] Furthermore, since different models have different task objectives, the corresponding prompts will also be different. Prompts that do not match the task objectives will lead to poor performance of the model. Therefore, by using prompt templates to generate prompts, prompts that match the task objectives of the model can be obtained, thus improving the model's performance. Moreover, since different prompts can instruct the model to perform different tasks to achieve different task objectives, the same model can be used to perform different tasks, which can also improve the flexibility of the model's output.

[0158] In addition, performing syntax correction and / or type transformation on the acquired initial historical dialogue statements and the initial current round of responses can avoid the problem that the intelligent dialogue system cannot handle different types of data. It can also avoid the problem that the intelligent dialogue system will make recognition errors due to syntax errors. Therefore, it can improve the recognition accuracy of the intelligent dialogue system.

[0159] Figure 2 This is a block diagram of an apparatus for a multi-turn dialogue rewriting method according to an embodiment of this application. The apparatus includes at least the following modules: a data acquisition module 210, a type identification module 220, a first extraction module 230, a second extraction module 240, and a statement rewriting module 250.

[0160] The data acquisition module 240 is used to acquire the user's historical dialogue statements and the statements to be answered in the current round between the user and the intelligent dialogue system.

[0161] The type identification module 220 is used to determine the rewrite type of the statement to be responded to in this round.

[0162] The first extraction module 230 is used to determine the identification words that match the rewriting type in the historical dialogue statements.

[0163] The second extraction module 240 is used to determine the extraction words that match the rewrite type in the statements to be responded to in the current round.

[0164] The statement rewriting module 250 is used to rewrite the identified words using the extracted words according to the rewriting method indicated by the rewriting type, so as to obtain the rewritten statement to be answered.

[0165] For relevant details, please refer to the above method implementation examples.

[0166] It should be noted that the apparatus for the multi-turn dialogue rewriting method provided in the above embodiments is only illustrated by the division of the above functional modules when performing multi-turn dialogue rewriting. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the apparatus for the multi-turn dialogue rewriting method can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus for the multi-turn dialogue rewriting method provided in the above embodiments and the embodiments of the multi-turn dialogue rewriting method belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0167] Figure 3 This is a block diagram of an electronic device provided in one embodiment of this application. The device includes at least a processor 301 and a memory 302.

[0168] Processor 301 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 301 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 301 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 301 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 301 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0169] Memory 302 may include one or more computer-readable storage media, which may be non-transitory. Memory 302 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in memory 302 are used to store at least one instruction, which is executed by processor 301 to implement the multi-turn dialogue rewriting method provided in the method embodiments of this application.

[0170] In some embodiments, the electronic device may also optionally include: a peripheral device interface and at least one peripheral device. The processor 301, memory 302, and peripheral device interface can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface via a bus, signal line, or circuit board. Indicatively, peripheral devices include, but are not limited to: radio frequency circuitry, a touch display screen, audio circuitry, and a power supply.

[0171] Of course, electronic devices may also include fewer or more components, and this embodiment does not limit this.

[0172] Optionally, this application also provides a computer-readable storage medium storing a program that is loaded and executed by a processor to implement the multi-turn dialogue rewriting method of the above method embodiments.

[0173] Optionally, this application also provides a computer product including a computer-readable storage medium storing a program that is loaded and executed by a processor to implement the multi-turn dialogue rewriting method of the above method embodiments.

[0174] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0175] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for rewriting multi-turn dialogues, characterized in that, The method, applied to intelligent dialogue systems, includes: The system acquires the user's historical dialogue statements and the current pending response statement between the user and the intelligent dialogue system; the historical dialogue statements include at least one historical pending response statement and the system response statement corresponding to each historical pending response statement. Determining the rewrite type of the statement to be responded to in the current round includes: obtaining preset type prompt information and a first recognition model, wherein the type prompt information is used to instruct the first recognition model to perform the task of recognizing the rewrite type; merging the historical dialogue statements, the statement to be responded to in the current round, and the type prompt information to obtain a type dataset; and inputting the type dataset into the first recognition model to obtain the rewrite type. Before obtaining the preset first recognition model, the method further includes: merging the first historical training statement, the first training statement to be answered, and the type prompt information to obtain type recognition training data; inputting the type recognition training data into the initial first recognition model to obtain the initial rewritten type corresponding to the first training statement to be answered; comparing the actual rewritten type corresponding to the first training statement to be answered with the initial rewritten type to obtain a first comparison result; and training the initial first recognition model using the first comparison result to obtain the first recognition model. Determining the identification words that match the rewriting type in the historical dialogue statements includes: obtaining preset extraction prompt information and a second recognition model, wherein the extraction prompt information is used to instruct the second recognition model to perform the task of extracting identification words; merging the historical dialogue statements, the statements to be answered in the current round, and the extraction prompt information to obtain an extraction dataset; and inputting the extraction dataset into the second recognition model to obtain the identification words. Before obtaining the preset second recognition model, the method further includes: merging the second historical training statement, the second training statement to be answered, and the extracted prompt information to obtain extracted training data; inputting the extracted training data into the initial second recognition model to obtain the initial recognition word corresponding to the second training statement to be answered; comparing the actual recognition word corresponding to the second training statement to be answered with the initial recognition word to obtain a second comparison result; and training the initial second recognition model using the second comparison result to obtain the second recognition model. In the current round of responses, identify the extracted words that match the rewrite type; The extracted words are used to rewrite the identified words according to the rewriting method indicated by the rewriting type, resulting in the rewritten response statement.

2. The method according to claim 1, characterized in that, Before retrieving the preset type hint information, it also includes: Get the prompt message template; The prompt information template is used to generate prompt information according to the recognition purpose of the first recognition model. The recognition purpose includes recognizing the rewrite type. Accordingly, the prompt information includes the type prompt information.

3. The method according to claim 1, characterized in that, The rewriting types include: replacement and completion; The step of rewriting the identified word using the extracted word according to the rewriting method indicated by the rewriting type to obtain the rewritten response statement includes: When the rewrite type is replacement, the extracted word is replaced with the identified word; When the rewrite type is completion, the identified word is added based on the position of the extracted word.

4. The method according to claim 1, characterized in that, Before acquiring the user's historical dialogue statements and the statements to be answered in the current round of the intelligent dialogue system, the method further includes: The system acquires the user's initial historical dialogue statements and the initial current round of pending responses from the intelligent dialogue system; the initial historical dialogue statements include at least one initial historical pending response statement and the initial system response statement corresponding to each initial historical pending response statement. Perform syntax correction and / or type transformation on the initial historical dialogue statements to obtain the historical dialogue statements and the statements to be responded to in the current round.

5. An electronic device, characterized in that, The electronic device includes a processor and a memory connected to the processor, the memory storing a program, which the processor executes to implement the multi-turn dialogue rewriting method as described in any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that, The storage medium stores a program that, when executed by a processor, is used to implement the multi-turn dialogue rewriting method as described in any one of claims 1 to 4.

Images