Text processing model training method and device, text processing method and device and electronic equipment

Abstract

The invention discloses a text processing model training method and device, a text processing method and device and electronic equipment, and relates to the technical field of natural language processing. Inputting preset rewriting instruction data, preset rewriting example data and each piece of sample session data into a first text processing model, and performing question rewriting analysis on multiple rounds of sample question and answer texts in each piece of sample session data based on the preset rewriting instruction data and the preset rewriting example data, generating annotation rewriting associated data of each piece of sample session data; generating a rewriting training set based on the multiple pieces of sample session data and the annotation rewriting associated data; performing question rewriting training on the second text processing model based on the rewriting training set to obtain a target text processing model; the model data volume of the second text processing model is smaller than that of the first text processing model. By utilizing the scheme of the invention, the model rewriting capability can be improved on the basis of improving the rewriting training set construction efficiency and quality.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a text processing model training, a text processing method, a device and an electronic device. Background Art

[0002] In human-computer conversation scenarios, users often interact with conversation systems in the form of multiple rounds of questions and answers. There is often redundant content in the content of historical multiple rounds of questions and answers. The current round of questions and answers may omit or use other content to refer to information that has already appeared in the historical conversation. Therefore, it is necessary to rewrite the content of the current round of questions based on the content of historical multiple rounds of questions and answers to supplement the omitted information and eliminate the reference information, so as to obtain the rewritten question content that can fully reflect the intention of the user in this round.

[0003] In the relevant existing technologies, the rewriting task is usually modeled as a generative task based on a generative language model. By constructing a relevant rewriting dataset, the generative language model is fine-tuned to obtain a rewriting model. However, the scale of the relevant training dataset is often small, resulting in poor training effect of the model and weak rewriting ability. Summary of the invention

[0004] The present application provides a text processing model training, a text processing method, a device and an electronic device, which can improve the conversation rewriting ability of the model on the basis of improving the efficiency and quality of rewriting training set construction, and make the model lightweight. The technical solution of the present application is as follows:

[0005] On the one hand, a text processing model training method is provided, the method comprising:

[0006] Get multiple sample session data;

[0007] Inputting preset rewriting instruction data, preset rewriting example data and each sample conversation data into a first text processing model, and performing question rewriting analysis on multiple rounds of sample question and answer texts in each sample conversation data based on the preset rewriting instruction data and the preset rewriting example data, to generate annotated rewriting associated data corresponding to each sample conversation data;

[0008] generating a rewriting training set based on the plurality of sample conversation data and the annotated rewriting associated data corresponding to each of the plurality of sample conversation data;

[0009] Based on the rewriting training set, question rewriting training is performed on the second text processing model to obtain a target text processing model; the model data volume of the second text processing model is smaller than the model data volume of the first text processing model.

[0010] In another aspect, a text processing method is provided, the method comprising:

[0011] Obtaining a current question text and at least one round of historical question and answer text before the current question text;

[0012] Inputting preset rewriting instruction data, preset rewriting example data, the at least one round of historical question and answer text, and the current question text into a target text processing model, performing question rewriting analysis on the current question text based on the preset rewriting instruction data, the preset rewriting example data, and the at least one round of historical question and answer text, and generating rewriting associated data corresponding to the current question text;

[0013] Based on the rewriting associated data, generating a current answer text corresponding to the current question text;

[0014] Wherein, the target text processing model is obtained after training based on the text processing model training method as described above.

[0015] On the other hand, a text processing model training device is provided, the device comprising:

[0016] A sample session data acquisition module, used to acquire multiple sample session data;

[0017] A sample annotation module, used to input preset rewriting instruction data, preset rewriting example data and each sample conversation data into a first text processing model, and based on the preset rewriting instruction data and the preset rewriting example data, perform question rewriting analysis on multiple rounds of sample question and answer texts in each sample conversation data, and generate annotation rewriting associated data corresponding to each sample conversation data;

[0018] A rewriting training set generation module, used for generating a rewriting training set based on the plurality of sample conversation data and the annotated rewriting associated data corresponding to each of the plurality of sample conversation data;

[0019] A model training module is used to perform question rewriting training on a second text processing model based on the rewriting training set to obtain a target text processing model; the model data volume of the second text processing model is smaller than the model data volume of the first text processing model.

[0020] In another aspect, a text processing device is provided, the device comprising:

[0021] A text acquisition module, used to acquire a current question text and at least one round of historical question and answer text before the current question text;

[0022] A question rewriting module, used for inputting preset rewriting instruction data, preset rewriting example data, the at least one round of historical question and answer text and the current question text into a target text processing model, performing question rewriting analysis on the current question text based on the preset rewriting instruction data, the preset rewriting example data and the at least one round of historical question and answer text, and generating rewriting associated data corresponding to the current question text;

[0023] A current reply text generating module, used for generating a current reply text corresponding to the current question text based on the rewriting associated data;

[0024] Wherein, the target text processing model is obtained after training based on the text processing model training device as mentioned above.

[0025] On the other hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the text processing model training method or text processing method as described above.

[0026] On the other hand, a computer-readable storage medium is provided, in which at least one instruction or at least one program is stored, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the text processing model training method or text processing method as described above.

[0027] On the other hand, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-mentioned text processing model training method or text processing method.

[0028] The present application provides a text processing model training, text processing method, device and electronic device, which have the following technical effects:

[0029] The present application inputs preset rewriting instruction data, preset rewriting example data and each sample conversation data into a first text processing model, performs question rewriting analysis on multiple rounds of sample question and answer texts in each sample conversation data based on the preset rewriting instruction data and the preset rewriting example data, generates annotated rewriting associated data corresponding to each sample conversation data, and generates a rewriting training set based on multiple sample conversation data and the annotated rewriting associated data corresponding to each of the multiple sample conversation data. By introducing high-quality preset rewriting instruction data and preset rewriting example data, the first text processing model can be helped to perform analogical learning, and the first text processing model is guided to directly output the annotated rewriting associated data corresponding to each sample conversation data, thereby improving the construction efficiency and construction quality of the rewriting training set. Then, based on the rewriting training set, the second text processing model is rewritten and analyzed and trained to obtain a target text processing model, wherein the model data volume of the second text processing model is smaller than the model data volume of the first text processing model. The rewriting training set is used to perform knowledge distillation on the second text processing model with a smaller model data volume, which can make the model lightweight while improving the rewriting ability of the model, thereby saving computing resources of the deployed terminal and improving the terminal system operation performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 is a schematic diagram of an application environment provided by an embodiment of the present application;

[0032] Figure 2 It is a flowchart of a text processing model training method provided in an embodiment of the present application;

[0033] Figure 3 It is a flow chart of generating a rewriting training set based on a plurality of sample conversation data and annotated rewriting associated data corresponding to each of the plurality of sample conversation data, provided by an embodiment of the present application;

[0034] Figure 4 A flowchart of an embodiment of the present application provides a method of performing conversation reorganization on multiple single-round sample question and answer texts based on the question independence description information of each of the multiple single-round sample question and answer texts and the question instruction type information of each of the multiple single-round sample question and answer texts to obtain multiple reorganized conversation data;

[0035] Figure 5It is a flowchart of a model question rewriting training method provided in an embodiment of the present application;

[0036] Figure 6 It is a flowchart of another model question rewriting training method provided in an embodiment of the present application;

[0037] Figure 7 It is a flowchart of a text processing method provided in an embodiment of the present application;

[0038] Figure 8 It is a schematic diagram of a framework of a text processing system provided in an embodiment of the present application;

[0039] Fig. 9 It is a block diagram of a text processing model training device provided in an embodiment of the present application;

[0040] Fig.10 It is a block diagram of a text processing device provided in an embodiment of the present application;

[0041] Fig.11 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0043] It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0044] It is understandable that in the specific implementation of this application, related data such as user information is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0045] To facilitate understanding of the embodiments of the present application, several concepts are briefly introduced below:

[0046] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0047] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operation / interaction systems, mechatronics, etc. Among them, pre-trained models are also called large models and basic models. After fine-tuning, they can be widely used in downstream tasks in various major directions of artificial intelligence. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0048] Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can enable effective communication between people and computers using natural language. Natural language processing involves natural language, which is the language people use in daily life and is closely related to linguistic research; it also involves important technologies for model training in artificial intelligence fields such as computer science and mathematics. The pre-trained model is developed from the large language model (Large Language Model) in the field of NLP. After fine-tuning, the large language model can be widely used in downstream tasks. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

[0049] Machine Learning (ML) is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching. The pre-trained model is the latest development of deep learning, which integrates the above technologies.

[0050] Pre-training model, also known as cornerstone model or big model, refers to a deep neural network (DNN) with large parameters. It is trained on massive unlabeled data. The function approximation ability of large-parameter DNN is used to enable PTM to extract common features from the data. After fine tuning, parameter efficient fine tuning (PEFT), prompt-tuning and other technologies, it is suitable for downstream tasks. Therefore, the pre-training model can achieve ideal results in few-shot or zero-shot scenarios. PTM can be divided into language models (ELMO, BERT, GPT), visual models (swin-transformer, ViT, V-MOE), speech models (VALL-E), multimodal models (ViBERT, CLIP, Flamingo, Gato), etc. according to the data modality processed. Among them, the multimodal model refers to a model that establishes two or more data modality feature representations. The pre-training model is an important tool for outputting artificial intelligence generated content (AIGC), and can also be used as a general interface to connect multiple specific task models.

[0051] Model compression and quantization: refers to the use of compression and quantization techniques to help reduce model size and accelerate model reasoning, thereby reducing the model's storage and computing costs. Model compression usually includes pruning, low-rank decomposition, knowledge distillation, etc. Model quantization refers to converting floating-point parameters in the model into fixed-point or integer parameters, thereby reducing model size and accelerating model reasoning.

[0052] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, driverless cars, autonomous driving, drones, digital twins, virtual humans, robots, artificial intelligence generated content (AIGC), conversational interaction, smart medical care, smart customer service, game AI, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0053] In addition, the technical terms involved in this application include:

[0054] In-Context Learning (ICL): No need to fine-tune model weights or parameters, just input several examples of downstream task question-answer pairs and new questions into the pre-trained model, and guide the pre-trained model to predict the answer to the new question based on the examples. For example, "Input: 'Example: apple-apple, pear-pear. Question: banana-', output: 'banana'".

[0055] Fine Tune: refers to making adjustments to an existing model. Fine tuning can save certain computing resources and time and improve computing efficiency.

[0056] The solution provided in the embodiments of the present application involves technologies such as natural language processing and pre-training models of artificial intelligence, which are specifically described by the following embodiments:

[0057] The text processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the application environment may include a client 10 and a server 20, and the client 10 and the server 20 may be indirectly connected via wireless communication. A related object (such as a user) may send a text reply request carrying a current question text and at least one round of historical question and answer text before the current question text to the server 20 via the client 10. In response to the text reply request, the server 20 inputs the preset rewriting instruction data, the preset rewriting example data, the at least one round of historical question and answer text, and the current question text into the target text processing model, performs question rewriting analysis on the current question text based on the preset rewriting instruction data, the preset rewriting example data, and the at least one round of historical question and answer text, generates rewriting associated data corresponding to the current question text, and then generates a current reply text corresponding to the current question text based on the rewriting associated data, and feeds back the current reply text to the client 10. The current reply text, wherein the target text processing model is obtained by training the second text processing model for question rewriting based on the rewriting training set, the rewriting training set is generated based on multiple sample conversation data and the annotated rewriting associated data corresponding to each of the multiple sample conversation data, the annotated rewriting associated data corresponding to each sample conversation data is generated by inputting preset rewriting instruction data, preset rewriting example data and each sample conversation data into the first text processing model, and performing question rewriting analysis on multiple rounds of sample question and answer texts in each sample conversation data based on the preset rewriting instruction data and the preset rewriting example data, wherein the model data volume of the second text processing model is smaller than the model data volume of the first text processing model. It should be noted that Figure 1 Just an example.

[0058] The client can be a physical device such as a smart phone, a computer (such as a desktop computer, a tablet computer, a laptop computer), a digital assistant, an intelligent voice interaction device (such as an intelligent speaker), an intelligent wearable device, a vehicle-mounted terminal, etc., or it can be software running in a physical device, such as a computer program. The operating system corresponding to the first client can be an Android system (Android system), an iOS system (a mobile operating system developed by Apple), a Linux system (an operating system), a Microsoft Windows system (Microsoft Windows operating system), etc.

[0059] The server side can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server can include a network communication unit, a processor, and a memory, etc. The server side can provide background services for the corresponding client.

[0060] The client 10 and the server 20 can be used to construct a system related to text processing, which can be a distributed system.

[0061] It should be noted that the text processing model training method and text processing method provided in the present application can be applied both on the client side and on the server side, and are not limited to the embodiments of the above-mentioned application environment.

[0062] The following is a specific embodiment of a text processing model training method provided by the present application. Figure 2 It is a flowchart of a text processing model training method provided in an embodiment of the present application. The present application provides method operation steps as described in the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or product is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment). Specifically, as Figure 2 As shown, the method may include:

[0063] S201, obtaining a plurality of sample session data.

[0064] In actual human-computer conversation scenarios, users often interact with the system in the form of multiple rounds of questions and answers. There is often redundant content in the historical multiple rounds of questions and answers. The current round of questions and answers may omit or use other content to refer to information that has appeared in the historical conversation. In this case, if the current round of questions input by the user is directly identified and processed, and the current round of replies is determined, it will inevitably cause the current round of replies to be inaccurate or even wrong. Therefore, it is necessary to rewrite the current round of questions (supplement the omitted information and / or eliminate the reference information) based on the historical multiple rounds of questions and answers to obtain the rewritten question content that can fully reflect the user's intention in this round.

[0065] In the embodiments of this specification, the sample conversation data may be sample human-computer conversation data, each of which may include multiple rounds of sample question-and-answer texts, each of which may include: a sample question text and a sample answer text. In an optional embodiment, the sample conversation data may be sampled from historical human-computer text conversations, or may be sampled from speech recognition data corresponding to historical human-computer voice conversations.

[0066] Illustratively, sample session data A may be:

[0067] [Q1]: Are there many people going to City S during the National Day holiday?

[0068] [A1]: Many people went to City S during the National Day holiday. City S received a total of 17.9453 million tourists during the 2021 National Day holiday.

[0069] [Q2]: What was the weather like at that time?

[0070] [A2]: Sunny, maximum temperature: 27℃, minimum temperature: 23℃, relative humidity: 91%

[0071] [Q3]: Then please help me find the air tickets”;

[0072] Among them, [Qi] represents the i-th round of question text, and [Ai] represents the i-th round of answer text for the i-th round of question text.

[0073] S202, input the preset rewriting instruction data, the preset rewriting example data and each sample conversation data into the first text processing model, and based on the preset rewriting instruction data and the preset rewriting example data, perform question rewriting analysis on multiple rounds of sample question and answer texts in each sample conversation data to generate annotated rewriting associated data corresponding to each sample conversation data.

[0074] In an embodiment of the present specification, the first text processing model may be a pre-trained general generative language model. Specifically, the first text processing model may be used to predict the next word or a text based on the current context, that is, the first text processing model has context prediction capability.

[0075] In an optional embodiment, the first text processing model may include large language models such as GPT-3, ChatGPT, self-developed large language models, etc. In practical applications, a large language model refers to a deep neural network model with billions or hundreds of billions of parameters.

[0076] In a specific embodiment, the first text processing model is trained in the following manner:

[0077] 1) Obtain training corpus in general fields;

[0078] 2) Inputting the training corpus into the generative language model to be trained for self-supervised context prediction training to obtain a first text processing model.

[0079] In a specific embodiment, the training corpus in a general field may include: any public corpus in the prior art.

[0080] Specifically, self-supervised context prediction training means automatically learning language rules and features from knowledge texts through the model's own learning ability in the absence of labeled data, and then applying them to downstream natural language processing tasks.

[0081] In practical applications, task description information and task example information related to downstream natural language processing tasks can be pre-set to guide the text processing model to learn by analogy with the task example information based on the task description information, thereby predicting the answer corresponding to the current input question. The task description information and task example information are usually written in natural language.

[0082] In an embodiment of the present specification, the preset rewriting instruction data may be pre-set instruction description information for a question rewriting task, and the preset rewriting instruction data may be used to instruct a text processing model to perform question rewriting analysis on multiple rounds of sample question and answer texts in each sample conversation data according to the preset rewriting example data.

[0083] In a specific embodiment, the preset rewriting instruction data may include: a task description for the question rewriting task, format requirements for the conversation data to be rewritten for the input model, and format requirements for the rewriting associated data output by the model. Specifically, the format requirements for the rewriting associated data may indicate the various rewriting associated fields that the rewriting associated data needs to include.

[0084] In a specific embodiment, the rewriting associated fields here may include, but are not limited to: question independence description information, question dependency description information, question rewriting text, and question instruction type information.

[0085] Specifically, the question independence description information can be used to describe the question independence situation of the question text to be rewritten. In an optional embodiment, the question independence situation can include three situations: "complete question", "reference question" and "omitted question". Among them, when the question independence description information of a question text is "complete question", it means that the information in the question text is complete, and there is no need to supplement the omitted information and resolve the reference information; when the question independence description information of a question text is "reference question", it means that there is reference information in the question text, and the reference information needs to be resolved; when the question independence description information of a question text is "omitted question", it means that there is omitted information in the question text, and the omitted information needs to be supplemented.

[0086] Specifically, the question dependency description information can be used to describe the historical question and answer dependency of the question text to be rewritten. When the question independence description information of a question text is "complete question", the question dependency description information of the question text can be "no dependency", indicating that the information of the question text is complete and does not depend on the historical question and answer text in the current session; when the question independence description information of a question text is "omitted question" or "reference question", it is necessary to supplement the information of the question text and resolve the reference from the above historical question and answer text. At this time, the information source can be indicated in the question dependency description information of the question text (for example, the text identifier and text content of the historical question and answer text on which the question text depends).

[0087] Specifically, the question rewriting text may be a text obtained by rewriting the question text to be rewritten, wherein the question independence description information and the question dependency description information may be used as a reference for rewriting the question text to be rewritten. When the question independence description information of a question text is "complete question", the question rewriting text of the question text is the question text itself; when the question independence description information of a question text is "omitted question" or "reference question", the question text is supplemented with omitted information and references are resolved to obtain a question rewriting text with complete information.

[0088] Specifically, the question instruction type information may represent the instruction topic type to which the text content of the question text to be rewritten belongs. For example, the question instruction type information may include but is not limited to instruction topic types such as weather, travel, shopping, etc., which helps to better understand the background and context of the question.

[0089] Indicatively, the preset rewrite instruction data may be:

[0090] Complete text comprehension tasks based on the context of multiple rounds of dialogue: pay special attention to "reference" and "omission" in questions.

[0091] Omission means that the sentence is incomplete and needs to be supplemented according to the context. For example, when asking about the weather, the question "What about City A?" omits "weather". Reference means that there are pronouns or demonstratives in the sentence, and the specific object they refer to needs to be clarified according to the context. For example, when asking about the representative TV series of actress Yao, "What is her representative TV series?", "she" needs to be resolved into "Yao". Pay attention to whether common pronouns and demonstratives need to be resolved: "I, you, he, she, it, this, that, these, those, here, there, who, what, which, which, where, how, how, how much".

[0092] Independence: Question independence includes three situations: complete / omitted / referenced.

[0093] A complete question means that the question itself contains enough information and does not need to rely on context; an omitted question means that part of the information is omitted in the question and needs to be supplemented according to the context; a reference question means that the question contains pronouns or demonstrative words, and the specific object it refers to needs to be clarified according to the context.

[0094] Rewrite: When the question contains omissions, add the omitted content; when the question contains references, resolve the references; when the question is complete, keep the original question. For example:

[0095] -Omit the question: "What about S City?" Rewrite it as "What's the weather like in S City?"

[0096] -Referential question: "What is her representative TV series?" is rewritten as "What is Yao's representative TV series?"

[0097] -Complete question: "What's the weather like in City G today?" Keep the original question.

[0098] Dependencies: Identify whether the question depends on several previous questions or answers. For example:

[0099] - Dependent question: "What about City S?" depends on the previous question "What's the weather like in City G today?"

[0100] - Non-dependent questions: "What's the weather like in City G today?" does not depend on other questions or answers.

[0101] Directive: Identify the directive topic the question belongs to, such as weather, travel, shopping, etc. This helps to better understand the background and context of the question.”

[0102] In the embodiments of the present specification, the preset rewriting example data may be a reference example for the question rewriting task set based on the preset rewriting instruction data. Specifically, the preset rewriting example data may include: at least one rewriting example data, each rewriting example data may include: example conversation data for model input and example rewriting associated data for model output (i.e., rewriting associated data corresponding to multiple rounds of example question texts in the example conversation data)

[0103] In a specific embodiment, the rewriting example data can be set in combination with the topic type of the actual conversation scenario and the format requirements in the preset rewriting instruction data. Indicatively, the topic types here can include but are not limited to: query (e.g., time query, weather query, ticket query, encyclopedia query), art appreciation (e.g., poetry appreciation, film and television appreciation), artistic creation (e.g., picture creation, music creation), mathematical calculation, programming, etc.

[0104] Indicatively, rewriting example data 1 may be:

[0105] enter:

[0106] [Q1]: When was your most recent holiday?

[0107] [A2]: The nearest holidays are the Mid-Autumn Festival and National Day in 2023, which will be 8 days from September 29 to October 6.

[0108] [Q2]: Do you know the recently released movie A?

[0109] [A2]: Of course, I know "Movie A". This movie is directed by director Zhang. This movie was released in Country C on July 21, 2023.

[0110] [Q3]: I want to see it. Can you help me see if there are any tickets during the holidays?

[0111] Output:

[0112] [Q1_]: Independence: [Complete], Rewrite: [When is the nearest holiday? ], Dependency: [No dependency], Instruction: [Holiday query]

[0113] [Q2_]: Independence: [Complete], Rewrite: [Introduce the recently released movie A], Dependency: [No dependency], Instruction: [Movie query]

[0114] [Q3_]: Independence: [Referential], Rewrite: [Confirm whether there are tickets for Movie A during the Mid-Autumn Festival and National Day holidays], Dependency: [Q1: Recent holidays, Q2: Movie A], Instruction: [Movie ticketing]";

[0115] Rewriting example data 2 can be: "

[0116] enter:

[0117] [Q1]: Recite "A Night Mooring at Maple Bridge"

[0118] [A2]: Zhang Ji (Tang Dynasty) \nThe moon sets, crows cry, and the sky is full of frost. The maples on the river bank and the fishing fires are opposite my sad sleep. \nThe bell of Hanshan Temple outside Gusu City reaches the passenger ship at midnight.

[0119] [Q2]: Briefly describe the scene of the first two lines of the poem

[0120] [A2]: This poem describes a late autumn season, when the moon is setting, crows are crowing, and the sky is covered with frost. The maple trees by the river are dyed red by the frost, and the fishing fires flicker in the river, creating a beautiful night scene.

[0121] [Q3]: Help me draw this scene

[0122] [A3]: The map has been drawn.

[0123] [Q4]: If I want to convert this picture into anime style, how should I design the prompt?

[0124] Output:

[0125] [Q1_]: Independence: [Complete], Rewrite: [Recite "Night Mooring at Maple Bridge"], Dependency: [No Dependency], Instruction: [Recite]

[0126] [Q3_]: Independence: [Referential], Rewrite: [Describe the characteristics of the scene "The moon sets, the crows cry, and the sky is full of frost. The maples on the river bank and the fishing fires are opposite me, and I am worried about sleeping."], Dependence: [Q1: "Night Mooring at Maple Bridge"], Instruction: [Poetry Appreciation]

[0127] [Q3_]: Independence: [Referential], Rewrite: [Draw a picture of the scene of "The moon sets, the crows cry, and the sky is full of frost. The maples on the river bank and the fishing fires are opposite to my sad sleep."], Dependence: [Q1: "A Night Mooring at Maple Bridge", Q2: The scene of "The moon sets, the crows cry, and the sky is full of frost. The maples on the river bank and the fishing fires are opposite to my sad sleep."], Instruction: [Artistic Creation]

[0128] [Q4_]: Independence: [Referential], Rewrite: [How to design a prompt for the scene of "The moon sets, the crows cry, and the sky is full of frost. The maples on the river bank and the fishing fires are opposite my sad sleep." in anime style], Dependence: [Q2: Draw the scene of these two lines of poetry], Instruction: [Prompt design]";

[0129] Rewriting example data 3 can be:

[0130] enter:

[0131] [Q1]: How is the weather today?

[0132] [A1]: I have checked the weather in City G for you today:

[0133] Today (2023-09-15, Friday) weather: showers, maximum temperature: 30℃, minimum temperature: 25℃, relative humidity: 84%, breeze, wind force: Level 2 to Level 1

[0134] [Q2]: What about City S?

[0135] [A2]: I have checked the weather in S City for you today:

[0136] Today (2023-09-15, Friday) weather: moderate rain, maximum temperature: 27℃, minimum temperature: 23℃, relative humidity: 91%, northeast wind turning to north wind, wind force: Level 2

[0137] [Q3]: I want to travel there, can you recommend me some delicious food?

[0138] Output:

[0139] [Q1_]: Independence: [Complete], Rewrite: [Today's weather in G city], Dependency: [No dependency], Command: [Weather query]

[0140] [Q2_]: Independence: [Omitted], Rewrite: [Today's weather in S city], Dependency: [Q1: What's the weather like today], Instruction: [Weather query]

[0141] [Q3_]: Independence: [reference], rewrite: [I want to travel to S City, recommend me some delicious food], dependence: [Q2: S City], instruction: [food recommendation]".

[0142] In an embodiment of the present specification, the annotated rewriting associated data corresponding to each sample conversation data may be generated by the first text processing model after performing question rewriting analysis on multiple rounds of sample question and answer texts in each sample conversation data based on preset rewriting instruction data and preset rewriting example data. The annotated rewriting associated data corresponding to each sample conversation data may be annotated data related to the question rewriting content corresponding to each sample conversation data.

[0143] In a specific embodiment, the annotated rewriting associated data corresponding to each sample conversation data may include: the question rewriting associated data corresponding to each round of sample question and answer text in each sample conversation data, and the question rewriting associated data corresponding to each round of sample question and answer text may include: the question independence description information corresponding to the sample question and answer text, the question dependency description information corresponding to the sample question and answer text, the question rewriting text corresponding to the sample question and answer text, and the question instruction type information corresponding to the sample question and answer text. Specifically, the question independence description information of the sample question and answer text may refer to the question independence description information of the sample question text in the sample question and answer text; the question dependency description information of the sample question and answer text may refer to the question dependency description information of the sample question text in the sample question and answer text; the question rewriting text of the sample question and answer text may refer to the question rewriting text of the sample question text in the sample question and answer text; and the question instruction type information of the sample question and answer text may refer to the question instruction type information of the sample question text in the sample question and answer text.

[0144] In a specific embodiment, the preset rewriting instruction data, the preset rewriting example data and each sample conversation data are input into the first text processing model, and based on the preset rewriting instruction data and the preset rewriting example data, multiple rounds of sample question and answer texts in each sample conversation data are subjected to question rewriting analysis, and the generation of the annotated rewriting associated data corresponding to each sample conversation data may include:

[0145] S2021, concatenating the preset rewriting instruction data, the preset rewriting example data and each sample session data respectively to obtain sample rewriting indication data corresponding to each sample session data;

[0146] S2022, input the sample rewriting instruction data into the first text processing model, and based on the preset rewriting instruction data and the preset rewriting example data, perform question rewriting analysis on multiple rounds of sample question and answer texts in each sample conversation data, and generate annotated rewriting associated data corresponding to each sample conversation data.

[0147] Specifically, the sample rewriting indication data corresponding to each sample session data may be obtained by splicing the preset rewriting instruction data, the preset rewriting example data and the corresponding sample session data based on a preset splicing format.

[0148] Illustratively, the sample rewriting indication data corresponding to the sample session data may include:

[0149] [Preset rewrite command data]

[0150] [Preset rewrite sample data]

[0151] enter:

[0152] Sample session data

[0153] Output: ";

[0154] Taking the above sample conversation data A as an example, the annotation rewriting associated data corresponding to the sample conversation data A may include:

[0155] [Q1_]: Independence: [Complete], Rewrite: [Are there many people going to City S during the National Day? ], Dependency: [No dependency], Command: [Number of tourists]

[0156] [Q2_]: Independence: [reference], rewrite: [What is the weather like when I go to S city during the National Day holiday? ], dependency: [Q1: Go to S city during the National Day holiday], instruction: [Weather query]

[0157] [Q3_]: Independence: [Omitted], Rewrite: [Help me check the air tickets to S City during the National Day], Dependency: [Q1: Go to S City during the National Day], Instruction: [Transportation ticketing]".

[0158] S203 , generating a rewriting training set based on the plurality of sample conversation data and the annotated rewriting associated data corresponding to each of the plurality of sample conversation data.

[0159] In the embodiment of the present specification, the rewriting training set may be a supervised data set used to perform question rewriting training on a model.

[0160] In practical applications, topic switching often occurs in multi-round conversation scenarios, that is, the user first asks questions on a certain topic, and after several rounds, changes the topic to continue asking questions. Therefore, it is necessary to construct sample conversation data with topic switching in the corresponding text content and corresponding annotated rewriting related data, and use them for model training to reduce the recognition error rate of the model for question-dependent description information in such cases, thereby improving the model's rewriting accuracy for such conversation data.

[0161] In a specific embodiment, the annotation rewriting associated data corresponding to each sample conversation data may include: question rewriting associated data corresponding to each round of sample question and answer text in each sample conversation data, and the question rewriting associated data corresponding to each round of sample question and answer text may include: question independence description information corresponding to the sample question and answer text and question instruction type information corresponding to the sample question and answer text, such as Figure 3 As shown, the above-mentioned generation of the rewriting training set based on the plurality of sample conversation data and the annotated rewriting associated data corresponding to each of the plurality of sample conversation data may include:

[0162] S2031, decomposing the multiple rounds of sample question-answer texts of the multiple sample conversation data to obtain multiple single-round sample question-answer texts.

[0163] Schematically, by decomposing the multi-round sample question-answer texts in the sample conversation data A, we can obtain the single-round sample question-answer text 1 and the single-round sample question-answer text 2 shown below. Specifically:

[0164] Single round sample question and answer text 1 is:

[0165] [Q1]: Are there many people going to City S during the National Day holiday?

[0166] [A1]: Many people went to City S during the National Day holiday. City S received a total of 17.9453 million tourists during the 2021 National Day holiday";

[0167] Correspondingly, the question-related data corresponding to the single-round sample question-answering text 1 is rewritten as:

[0168] [Q1_]: Independence: [Complete], Rewrite: [Are there many people going to City S during the National Day? ], Dependency: [No dependency], Instruction: [Number of tourists]";

[0169] Single round sample question and answer text 2 is:

[0170] [Q2]: What was the weather like at that time?

[0171] [A2]: It is expected to be sunny, with a maximum temperature of 27°C, a minimum temperature of 23°C, and a relative humidity of 91%”; accordingly, the question-related data corresponding to the single-round sample question-answering text 2 is rewritten as follows:

[0172] [Q2_]: Independence: [reference], rewrite: [What is the weather like when going to S City during the National Day? ], dependency: [Q1: Go to S City during the National Day], instruction: [Weather query]”;.

[0173] S2032, based on the question independence description information of each of the multiple single-round sample question and answer texts and the question instruction type information of each of the multiple single-round sample question and answer texts, perform conversation reorganization on the multiple single-round sample question and answer texts to obtain multiple reorganized conversation data; each reorganized conversation data in the multiple reorganized conversation data is conversation data in which the corresponding text content has a topic switch.

[0174] Specifically, the question independence description information and the question instruction type information can be combined to perform topic association analysis on multiple single-round sample question and answer texts, thereby reorganizing multiple single-round sample question and answer texts that are unrelated to topics to construct conversation data with topic switching.

[0175] In a specific embodiment, Figure 4 As shown, based on the question independence description information of each of the multiple single-round sample question and answer texts and the question instruction type information of each of the multiple single-round sample question and answer texts, the multiple single-round sample question and answer texts are conversationally reorganized to obtain multiple reorganized conversation data, which may include:

[0176] S401, screening out multiple question-answering texts to be reorganized whose corresponding question independence description information is complete questions from multiple single-round sample question-answering texts.

[0177] Indicatively, the question independence description information corresponding to the single-round sample question and answer text 1 is a complete question, so the single-round sample question and answer text 1 can be used as the question and answer text to be reorganized; and the question independence description information corresponding to the single-round sample question and answer text 2 is a reference question, so the single-round sample question and answer text 2 is discarded.

[0178] S402, filtering out a first number of question and answer texts to be reorganized that correspond to question instruction type information and non-target type information from a plurality of question and answer texts to be reorganized, where the target type information is the question instruction type information corresponding to the target question and answer text; and the target question and answer text is one of the plurality of question and answer texts to be reorganized.

[0179] Illustratively, taking the question instruction type information corresponding to the target question and answer text as "number of tourists" as an example, the question instruction type information of the first number of question and answer texts to be reorganized can be other instruction types besides "number of tourists", for example, "weather inquiry", "artistic creation" and other instruction types.

[0180] In a specific embodiment, the above-mentioned filtering out a first number of question and answer texts to be reorganized corresponding to question instruction type information and non-target type information from a plurality of question and answer texts to be reorganized may include:

[0181] S4021, screening out a second number of question and answer texts to be reorganized corresponding to question instruction type information and non-target type information from a plurality of question and answer texts to be reorganized; the second number is greater than the first number.

[0182] Specifically, the first number and the second number here can be preset in combination with the construction requirements of the reorganized session data in actual applications.

[0183] S4022, performing question text similarity analysis on the second number of question and answer texts to be reorganized and the target question and answer text, respectively, to obtain question text similarity indicators corresponding to each of the second number of question and answer texts to be reorganized.

[0184] Specifically, the question text similarity index corresponding to each question and answer text to be reorganized in the second number of question and answer texts to be reorganized can represent the question text similarity between the corresponding question and answer text to be reorganized and the target question and answer text.

[0185] In a specific embodiment, the second number of question and answer texts to be reorganized are respectively subjected to question text similarity analysis with the target question and answer text to obtain question text similarity indicators corresponding to the second number of question and answer texts to be reorganized, which may include:

[0186] 1) extracting semantic features of the question text for each of the second number of question and answer texts to be reorganized, to obtain first semantic feature information corresponding to each of the second number of question and answer texts to be reorganized;

[0187] 2) Extracting semantic features of the target question and answer text to obtain second semantic feature information corresponding to the target question and answer text;

[0188] 3) Perform feature similarity analysis on each first semantic feature information and the second semantic feature information respectively to obtain a question text similarity index.

[0189] Specifically, a text semantic extraction model may be used to extract semantic features of the question text. Optionally, the text semantic extraction model may include but is not limited to: a BERT model, an LSTM model, and other neural network models.

[0190] Specifically, the first semantic feature information corresponding to the question and answer text to be reorganized may represent the semantic features of the question text in the question and answer text to be reorganized. Exemplarily, the first semantic feature information may be expressed in the form of a semantic feature vector.

[0191] Specifically, the second semantic feature information may represent the semantic features of the question text in the target question and answer text. Exemplarily, the second semantic feature information may be expressed in the form of a semantic feature vector.

[0192] Optionally, the distance between the first semantic feature information corresponding to the question and answer text to be reorganized and the second semantic feature information corresponding to the target question and answer text can be determined and used as a similarity index of the question text corresponding to the question and answer text to be reorganized.

[0193] In a specific embodiment, the distance between the first semantic feature information corresponding to the question and answer text to be reorganized and the second semantic feature information corresponding to the target question and answer text may include but is not limited to cosine distance (i.e., cosine similarity), Euclidean distance, Manhattan distance, etc.

[0194] S4023: Determine, from the second number of question and answer texts to be reorganized, a first number of question and answer texts to be reorganized whose corresponding question text similarity indicators meet a preset text difference condition.

[0195] In a specific embodiment, the preset text difference condition can be preset in combination with the screening requirements of the question and answer texts to be reorganized in actual applications. Exemplarily, the preset text difference condition can be the first number of question and answer texts to be reorganized with the smallest similarity index of the corresponding question texts among the second number of question and answer texts to be reorganized.

[0196] S403: Combine the first number of question and answer texts to be reorganized with the target question and answer text to obtain target reorganized conversation data.

[0197] In a specific embodiment, the first number of question and answer texts to be reorganized are concatenated with the question text in the target question and answer text to obtain target reorganized conversation data.

[0198] Illustratively, the target reorganized session data may be:

[0199] [Q1]: Do you know the weather in K City today?

[0200] [A1]: Today in K city, it will be cloudy, with a temperature of 25-28℃

[0201] [Q2]: Tell me the hot domestic news within 24 hours

[0202] [A2]:1. The Minister of Commerce of Country H visits China.2. The typhoon may land in City D.

[0203] [Q3]: Does Director Zhang have any new movies coming out? ".

[0204] S404, updating the target question and answer text to obtain an updated target question and answer text.

[0205] S405, based on the updated target question and answer text, jump to filter out a first number of question and answer texts to be reorganized corresponding to question instruction type information and non-target type information from multiple question and answer texts to be reorganized, until a preset reorganization end condition is reached.

[0206] Specifically, the preset reorganization end condition may be that the amount of target reorganized session data meets a preset amount threshold, which may be preset in combination with the actual application of the construction scale requirement of the reorganized session data.

[0207] S406, taking the multiple target reorganization session data obtained when the preset reorganization end condition is reached as the multiple reorganization session data.

[0208] It can be seen from the above embodiments that by determining the question text similarity between the question and answer text to be reorganized and the target question and answer text by the distance between the first semantic feature information corresponding to the question and answer text to be reorganized and the second semantic feature information corresponding to the target question and answer text, the screening accuracy of single-round sample question and answer texts that are irrelevant to the topic can be improved, thereby improving the construction quality of the reorganized conversation data.

[0209] S2033, combining the question rewriting associated data corresponding to each of the multiple rounds of sample question and answer texts in each reorganized conversation data to obtain the annotation rewriting associated data corresponding to each reorganized conversation data.

[0210] Specifically, the question rewriting associated data corresponding to each of the multiple rounds of sample question and answer texts in each reorganized conversation data are concatenated to obtain the annotated rewriting associated data corresponding to each reorganized conversation data.

[0211] S2034, obtaining multiple training session data based on the multiple sample session data and the multiple reorganized session data.

[0212] In a specific embodiment, in combination with the model training requirements in actual applications, a first sampling weight for sample session data and a second sampling weight for reorganized session data can be pre-set, and multiple sample session data and multiple reorganized session data can be sampled and processed based on the first sampling weight and the second sampling weight, respectively, to obtain multiple training session data.

[0213] S2035 , rewriting the associated data based on the plurality of training session data and the annotations corresponding to the plurality of training session data to obtain a rewritten training set.

[0214] It can be seen from the above embodiments that by constructing sample conversation data with topic switching in the corresponding text content and corresponding annotated rewriting related data for model training, the recognition error rate of the model for question-dependent description information in such cases can be reduced, thereby improving the model's rewriting accuracy for conversation data with topic switching.

[0215] S204, based on the rewriting training set, performing question rewriting training on the second text processing model to obtain a target text processing model; the model data volume of the second text processing model is smaller than the model data volume of the first text processing model.

[0216] In the embodiment of the present specification, the second text processing model may be a generative language model to be trained for a conversation question rewriting task. Specifically, the model data volume of the second text processing model is smaller than the model data volume of the first text processing model.

[0217] In the embodiment of the present specification, the target text processing model may be a text processing model obtained by performing question rewriting training on the second text processing model based on a rewriting training set, and the target text processing model has the ability to rewrite questions for conversation data.

[0218] As can be seen from the above embodiments, by inputting preset rewriting instruction data, preset rewriting example data and each sample session data into the first text processing model, based on the preset rewriting instruction data and the preset rewriting example data, question rewriting analysis is performed on multiple rounds of sample question and answer texts in each sample session data, and annotated rewriting associated data corresponding to each sample session data is generated, and a rewriting training set is generated based on multiple sample session data and the annotated rewriting associated data corresponding to each of the multiple sample session data. By introducing high-quality preset rewriting instruction data and preset rewriting example data, the first text processing model can be helped to perform analogical learning, and the first text processing model is guided to directly output the annotated rewriting associated data corresponding to each sample session data, thereby improving the construction efficiency and construction quality of the rewriting training set. Then, based on the rewriting training set, the second text processing model is rewritten and analyzed and trained to obtain a target text processing model, wherein the model data volume of the second text processing model is smaller than the model data volume of the first text processing model, and the rewriting training set is used to perform knowledge distillation on the second text processing model with a smaller model data volume, so that the rewriting ability of the model can be improved while the model can be made lightweight, thereby saving computing resources of the deployed terminal and improving the terminal system operation performance.

[0219] In an optional embodiment, the rewriting training set may include: a plurality of sample conversation data and annotated rewriting associated data corresponding to each of the plurality of sample conversation data, such as Figure 5 As shown, the above-mentioned question rewriting training is performed on the second text processing model based on the rewriting training set to obtain the target text processing model, which may include:

[0220] S2041, input the preset rewriting instruction data, the preset rewriting example data and each sample conversation data into the second text processing model, and based on the preset rewriting instruction data and the preset rewriting example data, perform question rewriting analysis on multiple rounds of sample question and answer texts in each sample conversation data to generate predicted rewriting associated data corresponding to each sample conversation data.

[0221] In an embodiment of the present specification, the predicted rewriting associated data corresponding to each sample conversation data may be generated by the second text processing model after performing question rewriting analysis on multiple rounds of sample question and answer texts in each sample conversation data based on preset rewriting instruction data and preset rewriting example data. The predicted rewriting associated data corresponding to each sample conversation data may be predicted data related to the question rewriting content corresponding to each sample conversation data.

[0222] In a specific embodiment, the prediction rewriting associated data corresponding to each sample conversation data may include: the question rewriting prediction data corresponding to each round of sample question and answer text in each sample conversation data, and the question rewriting prediction data corresponding to each round of sample question and answer text may include: the question independence prediction information corresponding to the sample question and answer text, the question dependency prediction information corresponding to the sample question and answer text, the question rewriting prediction text corresponding to the sample question and answer text, and the question instruction prediction information corresponding to the sample question and answer text. Specifically, the question independence prediction information of the sample question and answer text may refer to the question independence prediction information of the sample question text in the sample question and answer text; the question dependency prediction information of the sample question and answer text may refer to the question dependency prediction information of the sample question text in the sample question and answer text; the question rewriting text of the sample question and answer text may refer to the question rewriting prediction text of the sample question text in the sample question and answer text; and the question instruction prediction information of the sample question and answer text may refer to the question instruction prediction information of the sample question text in the sample question and answer text.

[0223] S2042 : Determine first rewriting loss information based on the annotated rewriting associated data corresponding to each of the plurality of sample session data and the predicted rewriting associated data corresponding to each of the plurality of sample session data.

[0224] Specifically, the first rewriting loss information may represent the difference between the annotated rewriting associated data corresponding to each of the plurality of sample conversation data and the predicted rewriting associated data corresponding to each of the plurality of sample conversation data. In a specific embodiment, the first rewriting loss information may represent the difference between the question rewriting associated data corresponding to each round of sample question and answer text in each sample conversation data and the question rewriting predicted data.

[0225] In a specific embodiment, determining the first rewriting loss information based on the annotated rewriting association data corresponding to each of the multiple sample session data and the predicted rewriting association data corresponding to each of the multiple sample session data may include: determining the first rewriting loss information between the annotated rewriting association data corresponding to each of the multiple sample session data and the predicted rewriting association data corresponding to each of the multiple sample session data based on a preset rewriting loss function.

[0226] In a specific embodiment, the preset rewriting loss function may include: cross entropy loss information.

[0227] S2043: Based on the first rewriting loss information, fine-tune the second text processing model to obtain a target text processing model.

[0228] In practical applications, when the training set for the generative model includes long conversations, the generative model may learn and imitate the structure of these conversations, model the distribution of long conversations, and thus tend to produce longer outputs. Accordingly, in the conversation rewriting scenario of the present application, when the fine-tuned text processing model inputs the long conversation data to be rewritten, it may not output the question rewriting prediction data corresponding to each round of question and answer text in the long conversation data as expected, but instead continue or answer the long conversation. On the one hand, the continuation content is longer, which will bring unnecessary reasoning overhead; on the other hand, the continuation produces unexpected results and requires additional strategies to cover the bottom, which increases the burden on system developers; therefore, only calculating the rewriting loss on the rewriting-related data output by the model can effectively avoid the continuation problem in the generative model.

[0229] It can be seen from the above embodiments that only calculating the rewriting loss information on the rewriting associated data output by the model can effectively avoid the continuation problem in the generative model and improve the model's session rewriting capability.

[0230] In an optional embodiment, the rewriting training set may further include: a plurality of reorganized session data and annotated rewriting associated data corresponding to each of the plurality of reorganized session data, such as Figure 6 As shown, the above method may also include:

[0231] S2044, input the preset rewriting instruction data, the preset rewriting example data and each reorganized conversation data into the second text processing model, and based on the preset rewriting instruction data and the preset rewriting example data, perform question rewriting analysis on multiple rounds of sample question and answer texts in each reorganized conversation data to generate predicted rewriting associated data corresponding to each reorganized conversation data.

[0232] In an embodiment of the present specification, the predicted rewriting associated data corresponding to each reorganized conversation data may be generated by the second text processing model after performing question rewriting analysis on multiple rounds of sample question and answer texts in each reorganized conversation data based on preset rewriting instruction data and preset rewriting example data. The predicted rewriting associated data corresponding to each reorganized conversation data may be predicted data related to the question rewriting content corresponding to each reorganized conversation data.

[0233] In a specific embodiment, the prediction rewriting associated data corresponding to each reorganized conversation data may include: the question rewriting prediction data corresponding to each round of sample question and answer text in each reorganized conversation data, and the question rewriting prediction data corresponding to each round of sample question and answer text may include: the question independence prediction information corresponding to the sample question and answer text, the question dependency prediction information corresponding to the sample question and answer text, the question rewriting prediction text corresponding to the sample question and answer text, and the question instruction prediction information corresponding to the sample question and answer text. Specifically, the question independence prediction information of the sample question and answer text may refer to the question independence prediction information of the sample question text in the sample question and answer text; the question dependency prediction information of the sample question and answer text may refer to the question dependency prediction information of the sample question text in the sample question and answer text; the question rewriting text of the sample question and answer text may refer to the question rewriting prediction text of the sample question text in the sample question and answer text; and the question instruction prediction information of the sample question and answer text may refer to the question instruction prediction information of the sample question text in the sample question and answer text.

[0234] S2045 , determining second rewriting loss information based on the annotated rewriting associated data corresponding to each of the plurality of reorganized session data and the predicted rewriting associated data corresponding to each of the plurality of reorganized session data.

[0235] Specifically, the second rewriting loss information may represent the difference between the annotated rewriting association data corresponding to each of the plurality of reorganized conversation data and the predicted rewriting association data corresponding to each of the plurality of reorganized conversation data. In a specific embodiment, the second rewriting loss information may represent the difference between the question rewriting association data corresponding to each round of sample question and answer text in each reorganized conversation data and the question rewriting prediction data.

[0236] In a specific embodiment, second rewriting loss information between the annotated rewriting associated data corresponding to each of the plurality of reorganized session data and the predicted rewriting associated data corresponding to each of the plurality of reorganized session data is determined based on a preset rewriting loss function.

[0237] In a specific embodiment, the preset rewriting loss function may include: cross entropy loss information.

[0238] S2046, fusing the first rewriting loss information and the second rewriting loss information to obtain target rewriting loss information.

[0239] In an optional embodiment, the sum of the first rewriting loss information and the second rewriting loss information can be used as the target rewriting loss information; the first rewriting loss information and the second rewriting loss information can also be weighted by combining the first sampling weight of the sample session data and the second sampling weight of the reorganized session data to obtain the target rewriting loss information.

[0240] S2047, based on the target rewriting loss information, fine-tune the second text processing model to obtain a target text processing model.

[0241] It can be seen from the above embodiments that combining the first rewriting loss information corresponding to the sample conversation data and the second rewriting loss information corresponding to the reorganized conversation data for model training can further improve the model's question rewriting ability, especially the rewriting ability for conversation data with topic switching.

[0242] It can be seen from the technical solution provided by the above embodiments of the present application that by inputting preset rewriting instruction data, preset rewriting example data and each sample conversation data into the first text processing model, based on the preset rewriting instruction data and the preset rewriting example data, question rewriting analysis is performed on multiple rounds of sample question and answer texts in each sample conversation data, annotated rewriting associated data corresponding to each sample conversation data is generated, and a rewriting training set is generated based on multiple sample conversation data and the annotated rewriting associated data corresponding to each of the multiple sample conversation data. By introducing high-quality preset rewriting instruction data and preset rewriting example data, it can help the first text processing model to Analogy learning guides the first text processing model to directly output the annotated rewritten associated data corresponding to each sample conversation data, thereby improving the construction efficiency and quality of the rewriting training set. Then, based on the rewriting training set, the second text processing model is trained for rewriting analysis to obtain the target text processing model, wherein the model data volume of the second text processing model is smaller than the model data volume of the first text processing model. The rewriting training set is used to perform knowledge distillation on the second text processing model with a smaller model data volume. While improving the rewriting ability of the model, the model can be made lightweight, thereby saving computing resources of the deployed terminal and improving the terminal system operation performance.

[0243] The present application also provides a text processing method, such as Figure 7 As shown, the text processing method may include:

[0244] S701, obtaining a current question text and at least one round of historical question and answer text before the current question text.

[0245] In the embodiment of the present specification, the current question text and at least one round of historical question and answer text belong to the same conversation.

[0246] S702, input preset rewriting instruction data, preset rewriting example data, at least one round of historical question and answer text, and current question text into the target text processing model, perform question rewriting analysis on the current question text based on the preset rewriting instruction data, preset rewriting example data, and at least one round of historical question and answer text, and generate rewriting related data corresponding to the current question text.

[0247] S703, generating a current answer text corresponding to the current question text based on the rewriting associated data.

[0248] In an embodiment of the present specification, the rewriting associated data corresponding to the current question text may be generated by the target text processing model after performing a question rewriting analysis on the current question text based on preset rewriting instruction data, preset rewriting example data and at least one round of historical question and answer text. The rewriting associated data corresponding to the current question text may be data related to the question rewriting content corresponding to the current question text.

[0249] In a specific embodiment, the rewrite associated data corresponding to the current question text may include: question independence description information corresponding to the current question text, question dependency description information corresponding to the current question text, question rewrite text corresponding to the current question text, and question instruction type information corresponding to the current question text.

[0250] In a specific embodiment, the above-mentioned generating the current answer text corresponding to the current question text based on rewriting the associated data may include:

[0251] S7031, based on the question rewriting text and the question instruction type information, perform intent recognition on the current question text to obtain intent recognition information.

[0252] Specifically, the intent recognition information can be used to characterize the instruction intent in the current question text.

[0253] S7032, generating a current answer text corresponding to the current question text based on the intention recognition information.

[0254] Among them, the above-mentioned target text processing model is obtained after training based on the above-mentioned text processing model training method.

[0255] See also Figure 8 , Figure 8 : is a schematic diagram of the architecture of a text processing system provided by an embodiment of the present application. Specifically, the text processing system may include: a rewriting module, an intent recognition module, and plug-in modules corresponding to various instruction types; wherein:

[0256] The rewriting module can be used as the entrance to the multi-round conversation system. Its main function is to combine historical questions and answers, supplement the omitted content and resolve the reference of the user's current question, and obtain the rewritten question.

[0257] The intent recognition module can recognize the intent of the rewritten question, thereby providing more abundant instruction intent information to the plug-in of the corresponding instruction type;

[0258] The plug-in module can combine the rewritten question and instruction intent information to generate an answer to the current question.

[0259] It can be seen from the above embodiments that, based on the text processing model obtained after training according to the above-mentioned text processing model training method, question rewriting analysis is performed on the current question text, rewriting related data corresponding to the current question text is generated, and based on the rewriting related data, the current reply text corresponding to the current question text is generated. This can improve the accuracy of the current reply text on the basis of improving the accuracy of the rewriting related data.

[0260] The present application also provides a text processing model training device, such as Fig. 9 As shown, the text processing model training device may include:

[0261] A sample session data acquisition module 910 is used to acquire a plurality of sample session data;

[0262] The sample annotation module 920 is used to input the preset rewriting instruction data, the preset rewriting example data and each sample conversation data into the first text processing model, and based on the preset rewriting instruction data and the preset rewriting example data, perform question rewriting analysis on multiple rounds of sample question and answer texts in each sample conversation data, and generate annotation rewriting associated data corresponding to each sample conversation data;

[0263] A rewriting training set generating module 930, configured to generate a rewriting training set based on a plurality of sample conversation data and the annotated rewriting associated data corresponding to each of the plurality of sample conversation data;

[0264] The model training module 940 is used to perform question rewriting training on the second text processing model based on the rewriting training set to obtain a target text processing model; the model data volume of the second text processing model is smaller than the model data volume of the first text processing model.

[0265] In a specific embodiment, the annotated rewriting associated data corresponding to each sample conversation data may include: question rewriting associated data corresponding to each round of sample question and answer text in each sample conversation data, the question rewriting associated data corresponding to each round of sample question and answer text may include: question independence description information corresponding to the sample question and answer text and question instruction type information corresponding to the sample question and answer text, and the rewriting training set generation module 930 may include:

[0266] A conversation decomposition unit, used to decompose the multiple rounds of sample question-answer texts of the multiple sample conversation data to obtain multiple single-round sample question-answer texts;

[0267] A conversation reorganization unit is used to perform conversation reorganization on multiple single-round sample question and answer texts based on the question independence description information of each of the multiple single-round sample question and answer texts and the question instruction type information of each of the multiple single-round sample question and answer texts to obtain multiple reorganized conversation data; each of the multiple reorganized conversation data is conversation data in which the corresponding text content has topic switching;

[0268] The rewritten data combination unit is used to combine the question rewritten associated data corresponding to each of the multiple rounds of sample question and answer texts in each reorganized conversation data to obtain the annotated rewritten associated data corresponding to each reorganized conversation data;

[0269] A training session data unit, used to obtain a plurality of training session data based on a plurality of sample session data and a plurality of reorganized session data;

[0270] The rewriting training set unit is used to rewrite the associated data based on the multiple training session data and the annotations corresponding to the multiple training session data to obtain the rewriting training set.

[0271] In a specific embodiment, the above session reorganization unit may include:

[0272] The first text screening unit is used to screen out multiple question-and-answer texts to be reorganized whose corresponding question independence description information is complete questions from multiple single-round sample question-and-answer texts;

[0273] The second text screening unit is used to screen out a first number of question and answer texts to be reorganized corresponding to question instruction type information and non-target type information from a plurality of question and answer texts to be reorganized, where the target type information is the question instruction type information corresponding to the target question and answer text; the target question and answer text is one of the plurality of question and answer texts to be reorganized;

[0274] A text combining unit, used for combining the first number of question and answer texts to be reorganized with the target question and answer text to obtain target reorganized conversation data;

[0275] A target question and answer text updating unit, used to update the target question and answer text to obtain an updated target question and answer text;

[0276] A jump unit, for jumping to, based on the updated target question and answer text, a first number of question and answer texts to be reorganized, which are selected from a plurality of question and answer texts to be reorganized and have corresponding question instruction type information and non-target type information, until a preset reorganization end condition is reached;

[0277] The reorganization session data unit is used to use the multiple target reorganization session data obtained when the preset reorganization end condition is reached as multiple reorganization session data.

[0278] In a specific embodiment, the second text screening unit may include:

[0279] The third text screening unit is used to screen out a second number of question and answer texts to be reorganized corresponding to the question instruction type information and non-target type information from the multiple question and answer texts to be reorganized; the second number is greater than the first number;

[0280] A question text similarity analysis unit is used to perform question text similarity analysis on the second number of question and answer texts to be reorganized and the target question and answer text, respectively, to obtain question text similarity indicators corresponding to the second number of question and answer texts to be reorganized;

[0281] The fourth text screening unit is used to determine, from the second number of question and answer texts to be reorganized, a first number of question and answer texts to be reorganized whose corresponding question text similarity indicators meet a preset text difference condition.

[0282] In an optional embodiment, the rewriting training set may include: a plurality of sample session data and annotated rewriting associated data corresponding to each of the plurality of sample session data, and the model training module 940 may include:

[0283] A first rewriting prediction unit is used to input preset rewriting instruction data, preset rewriting example data and each sample conversation data into a second text processing model, and based on the preset rewriting instruction data and the preset rewriting example data, perform question rewriting analysis on multiple rounds of sample question and answer texts in each sample conversation data to generate predicted rewriting associated data corresponding to each sample conversation data;

[0284] A first rewriting loss information determining unit, configured to determine first rewriting loss information based on the annotated rewriting associated data corresponding to each of the plurality of sample session data and the predicted rewriting associated data corresponding to each of the plurality of sample session data;

[0285] The first model fine-tuning unit is used to fine-tune the second text processing model based on the first rewriting loss information to obtain a target text processing model.

[0286] In an optional embodiment, the rewriting training set may further include: a plurality of reorganized session data and annotated rewriting associated data corresponding to each of the plurality of reorganized session data. The method may further include:

[0287] A second rewriting prediction unit is used to input preset rewriting instruction data, preset rewriting example data and each reorganized conversation data into a second text processing model, and based on the preset rewriting instruction data and the preset rewriting example data, perform question rewriting analysis on multiple rounds of sample question and answer texts in each reorganized conversation data to generate predicted rewriting associated data corresponding to each reorganized conversation data;

[0288] A second rewriting loss information determining unit, configured to determine second rewriting loss information based on the annotated rewriting associated data corresponding to each of the plurality of reorganized session data and the predicted rewriting associated data corresponding to each of the plurality of reorganized session data;

[0289] A target rewriting loss information unit, used for fusing the first rewriting loss information and the second rewriting loss information to obtain target rewriting loss information;

[0290] The second model fine-tuning unit is used to fine-tune the second text processing model based on the target rewriting loss information to obtain a target text processing model.

[0291] It should be noted that the device and method embodiments in the device embodiment are based on the same inventive concept.

[0292] The present application also provides a text processing device, such as Fig.10 As shown, the text processing device may include:

[0293] A text acquisition module 1010 is used to acquire a current question text and at least one round of historical question and answer text before the current question text;

[0294] The question rewriting module 1020 is used to input preset rewriting instruction data, preset rewriting example data, at least one round of historical question and answer texts, and the current question text into the target text processing model, and based on the preset rewriting instruction data, the preset rewriting example data, and at least one round of historical question and answer texts, perform question rewriting analysis on the current question text to generate rewriting associated data corresponding to the current question text;

[0295] A current reply text generating module 1030, for generating a current reply text corresponding to the current question text based on the rewritten associated data;

[0296] Among them, the above-mentioned target text processing model is obtained after training based on the above-mentioned text processing model training device.

[0297] In a specific embodiment, the rewriting associated data may include: question rewriting text corresponding to the current question text and question instruction type information corresponding to the current question text. The current answer text generating module 1030 may include:

[0298] An intention recognition unit, used to perform intention recognition on the current question text based on the question rewriting text and the question instruction type information to obtain intention recognition information;

[0299] The reply unit is used to generate a current reply text corresponding to the current question text according to the intention recognition information.

[0300] It should be noted that the device and method embodiments in the device embodiment are based on the same inventive concept.

[0301] An embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement a text processing model training method or a text processing method provided in the above method embodiment.

[0302] Further, Fig.11 A hardware structure diagram of an electronic device for implementing the text processing model training method or text processing method provided in the embodiment of the present application is shown. The electronic device may participate in or include the text processing model training device or text processing device provided in the embodiment of the present application. Fig.11 As shown, the electronic device 110 may include one or more (1102a, 1102b, ..., 1102n are used to illustrate) processors 1102 (the processor 1102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 1104 for storing data, and a transmission device 1106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Fig.11 The structure shown is only for illustration and does not limit the structure of the above electronic device. Fig.11 More or fewer components as shown, or with Fig.11 Different configurations are shown.

[0303] It should be noted that the one or more processors 1102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the electronic device 110 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0304] The memory 1104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the text processing model training method described in the embodiment of the present application or the program instructions / data storage device corresponding to the text processing method. The processor 1102 executes various functional applications and data processing by running the software programs and modules stored in the memory 1104, that is, realizing the above-mentioned text processing model training method or text processing method. The memory 1104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1104 may further include a memory remotely arranged relative to the processor 1102, and these remote memories may be connected to the electronic device 110 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0305] The transmission device 1106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the electronic device 110. In one example, the transmission device 1106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one embodiment, the transmission device 1106 can be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet wirelessly.

[0306] The display may be, for example, a touch screen liquid crystal display (LCD) that may enable a user to interact with a user interface of the electronic device 110 (or mobile device).

[0307] An embodiment of the present application also provides a computer-readable storage medium, which can be set in an electronic device to store at least one instruction or at least one program for implementing at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the text processing model training method or text processing method provided in the above method embodiment.

[0308] Optionally, in this embodiment, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0309] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the text processing model training method or text processing method provided in the method embodiment.

[0310] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0311] It should be noted that the above-mentioned sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above-mentioned specific embodiments of the present application are described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0312] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and equipment embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0313] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0314] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for training a text processing model, characterized in that, the method includes: obtaining a plurality of sample session data; inputting preset rewriting instruction data, preset rewriting example data, and each sample session data into a first text processing model, and based on the preset rewriting instruction data and the preset rewriting example data, performing question rewriting analysis on the multi-round sample Q&A text in each sample session data to generate annotation rewriting association data corresponding to each sample session data; generating a rewriting training set based on the plurality of sample session data and the annotation rewriting association data corresponding to each of the plurality of sample session data; performing question rewriting training on a second text processing model based on the rewriting training set to obtain a target text processing model; the model data volume of the second text processing model is smaller than the model data volume of the first text processing model.

2. The method according to claim 1, characterized in that, the annotation rewriting association data corresponding to each sample session data includes: the question rewriting association data corresponding to each round of sample Q&A text in each sample session data, and the question rewriting association data corresponding to each round of sample Q&A text includes: the question independence description information corresponding to the sample Q&A text and the question instruction type information corresponding to the sample Q&A text. The generating of the rewriting training set based on the plurality of sample session data and the annotation rewriting association data corresponding to each of the plurality of sample session data includes: decomposing the multi-round sample Q&A text of each of the plurality of sample session data to obtain a plurality of single-round sample Q&A texts; performing session recombination on the plurality of single-round sample Q&A texts based on the question independence description information of each of the plurality of single-round sample Q&A texts and the question instruction type information of each of the plurality of single-round sample Q&A texts to obtain a plurality of recombined session data; each recombined session data in the plurality of recombined session data is session data with topic switching in the corresponding text content; combining the question rewriting association data corresponding to each round of sample Q&A text in each recombined session data to obtain the annotation rewriting association data corresponding to each recombined session data; obtaining a plurality of training session data based on the plurality of sample session data and the plurality of recombined session data; obtaining the rewriting training set based on the plurality of training session data and the annotation rewriting association data corresponding to each of the plurality of training session data.

3. The method according to claim 2, characterized in that, the performing session recombination on the plurality of single-round sample Q&A texts based on the question independence description information of each of the plurality of single-round sample Q&A texts and the question instruction type information of each of the plurality of single-round sample Q&A texts to obtain a plurality of recombined session data includes: screening out a plurality of Q&A texts to be recombined with complete question independence description information from the plurality of single-round sample Q&A texts; From the multiple question-and-answer texts to be recombined, screen out the first number of question-and-answer texts to be recombined whose corresponding question instruction type information is not the target type information, where the target type information is the question instruction type information corresponding to the target question-and-answer text; the target question-and-answer text is one of the multiple question-and-answer texts to be recombined. Combine the first number of question-and-answer texts to be recombined with the target question-and-answer text to obtain target recombined session data. Update the target question-and-answer text to obtain an updated target question-and-answer text. Based on the updated target question-and-answer text, jump to the step of screening out the first number of question-and-answer texts to be recombined whose corresponding question instruction type information is not the target type information from the multiple question-and-answer texts to be recombined until a preset recombination end condition is reached. Use the multiple target recombined session data obtained when the preset recombination end condition is reached as the multiple recombined session data.

4. The method according to claim 3, wherein, the step of screening out the first number of question-and-answer texts to be recombined whose corresponding question instruction type information is not the target type information from the multiple question-and-answer texts to be recombined includes: Screen out the second number of question-and-answer texts to be recombined whose corresponding question instruction type information is not the target type information from the multiple question-and-answer texts to be recombined; the second number is greater than the first number. Perform question text similarity analysis on each of the second number of question-and-answer texts to be recombined and the target question-and-answer text to obtain the question text similarity indicators corresponding to each of the second number of question-and-answer texts to be recombined. Determine the first number of question-and-answer texts to be recombined whose corresponding question text similarity indicators meet the preset text difference condition from the second number of question-and-answer texts to be recombined.

5. The method according to claim 1, wherein, the rewritten training set includes: the multiple sample session data and the labeled rewritten association data corresponding to each of the multiple sample session data, and the step of performing question rewriting training on the second text processing model based on the rewritten training set to obtain the target text processing model includes: Input the preset rewriting instruction data, the preset rewriting example data, and each sample session data into the second text processing model, and perform question rewriting analysis on the multi-round sample question-and-answer texts in each sample session data based on the preset rewriting instruction data and the preset rewriting example data to generate the predicted rewritten association data corresponding to each sample session data. Determine the first rewriting loss information based on the labeled rewritten association data corresponding to each of the multiple sample session data and the predicted rewritten association data corresponding to each of the multiple sample session data. Perform fine-tuning processing on the second text processing model based on the first rewriting loss information to obtain the target text processing model.

6. The method according to claim 5, wherein, the rewritten training set further includes: multiple recombined session data and the labeled rewritten association data corresponding to each of the multiple recombined session data, and the method further includes: Input the preset rewriting instruction data, the preset rewriting example data, and each recombination session data into the second text processing model. Based on the preset rewriting instruction data and the preset rewriting example data, perform question rewriting analysis on the multi-round sample Q&A text in each recombination session data to generate prediction rewriting correlation data corresponding to each recombination session data; Based on the labeled rewriting correlation data corresponding to each of the multiple recombination session data and the prediction rewriting correlation data corresponding to each of the multiple recombination session data, determine the second rewriting loss information; Perform a fusion process on the first rewriting loss information and the second rewriting loss information to obtain the target rewriting loss information; Based on the target rewriting loss information, perform fine-tuning on the second text processing model to obtain the target text processing model.

7. A text processing method, characterized in that, the method includes: Obtain the current question text and at least one round of historical Q&A text before the current question text; Input the preset rewriting instruction data, the preset rewriting example data, the at least one round of historical Q&A text, and the current question text into the target text processing model. Based on the preset rewriting instruction data, the preset rewriting example data, and the at least one round of historical Q&A text, perform question rewriting analysis on the current question text to generate rewriting correlation data corresponding to the current question text; Generate the current answer text corresponding to the current question text based on the rewriting correlation data; wherein, the target text processing model is obtained by training according to the text processing model training method described in any one of claims 1-6.

8. The method according to claim 7, characterized in that, the rewriting correlation data includes: the question rewriting text corresponding to the current question text and the question instruction type information corresponding to the current question text. Generating the current answer text corresponding to the current question text based on the rewriting correlation data includes: Perform intent recognition on the current question text based on the question rewriting text and the question instruction type information to obtain intent recognition information; Generate the current answer text corresponding to the current question text according to the intent recognition information.

9. A text processing model training device, characterized in that, the device includes: A sample session data acquisition module for acquiring a plurality of sample session data; A sample annotation module for inputting the preset rewriting instruction data, the preset rewriting example data, and each sample session data into the first text processing model. Based on the preset rewriting instruction data and the preset rewriting example data, perform question rewriting analysis on the multi-round sample Q&A text in each sample session data to generate labeled rewriting correlation data corresponding to each sample session data; A rewriting training set generation module for generating a rewriting training set based on the plurality of sample session data and the labeled rewriting correlation data corresponding to each of the plurality of sample session data; A model training module, configured to perform question rewriting training on a second text processing model based on the rewritten training set to obtain a target text processing model; the model data volume of the second text processing model is smaller than that of the first text processing model.

10. A text processing device, characterized in that the device comprises: a text acquisition module, configured to acquire a current question text and at least one round of historical question-and-answer texts before the current question text; a question rewriting module, configured to input preset rewriting instruction data, preset rewriting example data, the at least one round of historical question-and-answer texts, and the current question text into the target text processing model, and perform question rewriting analysis on the current question text based on the preset rewriting instruction data, the preset rewriting example data, and the at least one round of historical question-and-answer texts to generate rewritten association data corresponding to the current question text; a current answer text generation module, configured to generate a current answer text corresponding to the current question text based on the rewritten association data; wherein, the target text processing model is obtained after being trained based on the text processing model training device described in claim 9.

11. An electronic device, characterized in that the device comprises a processor and a memory, and at least one instruction or at least one segment of program is stored in the memory, and the at least one instruction or the at least one segment of program is loaded and executed by the processor to implement the text processing model training method according to any one of claims 1 to 6 or the text processing method according to any one of claims 7 to 8.

12. A computer-readable storage medium, characterized in that at least one instruction or at least one segment of program is stored in the storage medium, and the at least one instruction or the at least one segment of program is loaded and executed by a processor to implement the text processing model training method according to any one of claims 1 to 6 or the text processing method according to any one of claims 7 to 8.

13. A computer program product, characterized in that the computer program product comprises at least one instruction or at least one segment of program, and the at least one instruction or the at least one segment of program is loaded and executed by a processor to implement the text processing model training method according to any one of claims 1 to 6 or the text processing method according to any one of claims 7 to 8.

Images