Text processing method, large language model training method and text processing equipment

Abstract

The invention discloses a text processing method, a large language model training method and text processing equipment, and relates to the technical field of natural language processing. The to-be-processed text is processed through a preset first big language model to obtain a first dialogue text, the first big language model is obtained through pre-training of training data, and the training data comprises at least one training text and a training label corresponding to the training text; the training label is a second dialogue text containing all text contents in the training text, and the second dialogue text contains at least one round of dialogue contents. The dialogue text construction scheme is simple in processing flow and low in time cost.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a text processing method, a large language model training method, and a text processing device. Background Technology

[0002] With the rapid development of natural language processing (NLP) technology, the field has seen a demand for high-quality dialogue texts. For example, high-quality multi-turn dialogue corpora are needed to train large language models in order to build intelligent question-answering systems. Currently, methods for building multi-turn dialogues suffer from complex processing procedures and high time costs.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a text processing method, a large language model training method, and a text processing device, aiming to propose a dialogue text construction scheme with a simple processing flow and low time cost.

[0005] To achieve the above objectives, this application proposes a text processing method, which includes:

[0006] Get the text to be processed;

[0007] The text to be processed is processed by a preset first language model to obtain a first dialogue text. The first language model is pre-trained using training data, which includes at least one training text and training labels corresponding to the training text. The training labels are a second dialogue text containing all the text content in the training text, and the second dialogue text contains at least one round of dialogue content.

[0008] Optionally, the step of processing the text to be processed through a preset first large language model to obtain the first dialogue text includes:

[0009] Obtain prompt words, wherein the prompt words include first prompt content, the first prompt content is used to prompt the first large language model to satisfy at least a first condition when processing the text to be processed, the first condition being that the first dialogue text contains at least all the text content in the text to be processed;

[0010] The first dialogue text is obtained by processing the text to be processed and the prompt words through the first large language model.

[0011] Optionally, the prompt word further includes a second prompt content corresponding to the first task. The second prompt content is used to prompt the first large language model to satisfy a second condition when processing the text to be processed. The second condition is the type or number of language phenomena contained in the first dialogue text. The first task is to train the dialogue model using the first dialogue text as training corpus.

[0012] Alternatively, the prompt word may also include a third prompt content corresponding to the second task. The third prompt content is used to prompt the first large language model to satisfy a third condition when processing the text to be processed. The third condition is used to limit the degree of intent clarity and / or dialogue completeness of the first dialogue text. The second task is to use the first dialogue text for the retrieval enhancement generation task.

[0013] Optionally, the text processing method further includes:

[0014] A knowledge base is constructed based on the first dialogue text;

[0015] Obtain the user query text, match the user query text with the knowledge base, and obtain the matching result;

[0016] The matching results and the user query text are processed by a preset second language model to obtain an answer corresponding to the user query text.

[0017] Optionally, constructing a knowledge base based on the first dialogue text includes:

[0018] Use at least one of the first dialogue texts corresponding to the text to be processed as a knowledge block in the knowledge base, or use at least one of the texts to be processed and the first dialogue texts corresponding to each of the texts to be processed as a knowledge block in the knowledge base;

[0019] Extract the questions from the question-answer pairs contained in the first dialogue text, and add the extracted questions to the knowledge base;

[0020] The step of obtaining the user query text and matching the user query text with the knowledge base to obtain the matching result includes:

[0021] Matching questions are obtained from the knowledge base based on the user query text, wherein the matching questions are the N questions in the knowledge base that have a matching degree higher than a first preset threshold or the highest matching degree, where N>0 and N is an integer;

[0022] The matching result is determined based on the first dialogue text corresponding to the matching question in the knowledge base, or based on the text to be processed corresponding to the matching question in the knowledge base and the first dialogue text.

[0023] Optionally, determining the matching result based on the text to be processed corresponding to the matching question in the knowledge base and the first dialogue text includes:

[0024] The text to be processed and the first dialogue text in the knowledge base corresponding to the matching question are used as the matching results;

[0025] Alternatively, the user query text can be matched with each target dialogue text to obtain matched dialogue texts. The matched dialogue texts are the M dialogue texts among the target dialogue texts whose matching degree with the user query text is higher than a second preset threshold or whose matching degree is the highest, where N>M>0, and M is an integer. Each target dialogue text is the first dialogue text in the knowledge base corresponding to the matching question. The text to be processed in the knowledge base corresponding to the matched dialogue text is used as the matching result, or the matched dialogue text and the text to be processed in the knowledge base corresponding to the matched dialogue text are used as the matching result.

[0026] Furthermore, to achieve the above objectives, this application also proposes a large language model training method, which includes:

[0027] Obtain training text and training labels corresponding to the training text, wherein the training labels are second dialogue texts containing all text content in the training texts, and the second dialogue texts contain at least one round of dialogue content;

[0028] The training text is processed by a pre-set first language model to be trained to obtain the third dialogue text;

[0029] The first large language model is trained based on the third dialogue text and the training labels.

[0030] Optionally, obtaining the training labels corresponding to the training text includes:

[0031] The training text is processed by a pre-defined third language model to obtain multiple fourth dialogue texts;

[0032] From multiple fourth dialogue texts, at least one second dialogue text that meets preset conditions is selected, and the second dialogue text is used as the training label corresponding to the training text. The preset conditions include a fourth condition, which is that the selected second dialogue text contains at least all the text content of the training text.

[0033] Optionally, the preset conditions also include a fifth condition corresponding to the first task. The fifth condition is the type or number of language phenomena at least contained in the selected second dialogue text. The first task is to train the dialogue model using the first dialogue text as training corpus. The first dialogue text is the dialogue text obtained by processing the text to be processed using the trained first large language model.

[0034] Alternatively, the preset conditions may include a sixth condition corresponding to the second task, which is used to limit the degree of intent clarity and / or dialogue completeness of the selected second dialogue text, and the second task is to use the first dialogue text for the retrieval enhancement generation task.

[0035] In addition, to achieve the above objectives, this application also proposes a text processing device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the text processing method as described above.

[0036] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the text processing method described above.

[0037] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the text processing method described above.

[0038] In this application, a first dialogue text is obtained by acquiring the text to be processed and processing it using a pre-defined first language model. The first language model is pre-trained using training data, which includes at least one training text and corresponding training labels. The training labels are second dialogue texts containing all the text content of the training text, and the second dialogue texts contain at least one round of dialogue. In this application, the second dialogue text corresponding to the training text is used as the corresponding training label to supervise the results obtained by the first language model processing the training text. Since the second dialogue text contains all the text content of the training text, the training labels can supervise the first language model to continuously move closer to the training objective. That is, the trained first language model can more accurately convert ordinary text into dialogue text containing all the text content of the ordinary text. Therefore, the first dialogue text obtained by processing the text using the first language model can more accurately cover all the text content of the text to be processed. In other words, only a piece of ordinary text needs to be provided to obtain a high-quality dialogue text containing all the content of that ordinary text, thus meeting the demand for high-quality dialogue text in specific application scenarios. Furthermore, this application uses a large language model for text conversion, which is simple, has a relatively low time cost, and produces more natural dialogues with more coherent question-answer pairs. Attached Figure Description

[0039] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart illustrating the first embodiment of the text processing method of this application;

[0042] Figure 2 This is one of the flowcharts involved in an embodiment of this application;

[0043] Figure 3 This is the second flowchart involved in an implementation example of this application;

[0044] Figure 4 This is a flowchart illustrating the fourth embodiment of the large language model training method of this application.

[0045] Figure 5This is the third flowchart involved in an implementation example of this application;

[0046] Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the text processing method in the embodiments of this application.

[0047] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0048] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0049] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0050] Currently, with the rapid development of natural language processing (NLP) technology, there is a demand for high-quality dialogue texts in the NLP field. For example, high-quality multi-turn dialogue corpora are needed to train large language models to build intelligent question-answering systems. Current methods for building multi-turn dialogues require multiple sequential steps: topic extraction, question generation, and answer generation, and multiple models need to work together. This generation method is complex and time-consuming.

[0051] In this embodiment, a first dialogue text is obtained by acquiring the text to be processed and processing it using a preset first language model. The first language model is pre-trained using training data, which includes at least one training text and corresponding training labels. The training labels are second dialogue texts containing all the text content of the training text, and the second dialogue texts contain at least one round of dialogue content. In this application, the second dialogue text corresponding to the training text is used as the corresponding training label to supervise the results obtained by the first language model processing the training text. Since the second dialogue text contains all the text content of the training text, the training labels can supervise the first language model to continuously move closer to the training target. That is, the trained first language model can more accurately convert ordinary text into dialogue text containing all the text content of the ordinary text. Therefore, the first dialogue text obtained by processing the text to be processed by the first language model can more accurately cover all the text content of the text to be processed. In other words, only a piece of ordinary text needs to be provided to obtain a high-quality dialogue text containing all the content of that ordinary text, thus meeting the demand for high-quality dialogue text in specific application scenarios. Furthermore, this application uses a large language model for text conversion, which is simple, has a relatively low time cost, and produces more natural dialogues with more coherent question-answer pairs.

[0052] The following presents a first embodiment of the text processing method of this application. (Refer to...) Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the text processing method of this application. The executing entity of the text processing method in this embodiment can be a device with program execution capabilities, specifically a server, personal computer, mobile phone, or other general computing service device. This device is used to execute the text processing method in this embodiment and is therefore referred to as a text processing device. For ease of description, the executing entity is omitted from the description of this embodiment and the following embodiments. In this embodiment, the text processing method includes steps S10 to S20:

[0053] Step S10: Obtain the text to be processed.

[0054] In this embodiment, a text-to-question-answer pair function is proposed. This involves converting non-dialogue text (hereinafter referred to as plain text for distinction) into question-and-answer pair text (hereinafter referred to as dialogue text for distinction) to meet the demand for high-quality dialogue text in some application scenarios. For example, when disseminating knowledge in plain text form, the converted dialogue text can replace the original plain text, which is more conducive to knowledge dissemination and understanding. Furthermore, the converted dialogue text can be used as training data for dialogue models to improve their dialogue performance. Converting plain text to dialogue text means presenting the content of plain text in a dialogue format. A dialogue model is a model that can generate corresponding following text (e.g., answer) based on the preceding context (e.g., a question), such as a large language model or other AI-based models with dialogue functionality.

[0055] It should be noted that, unless otherwise specified, the term "text" in this embodiment and the following embodiments can refer to formatted text or plain text without formatting. For example, the text to be processed can be plain text without formatting or formatted text, such as a document. This embodiment does not limit the form of the text to be processed. The text to be processed can be any ordinary text that needs to be processed and converted into dialogue text. This embodiment also does not limit the text content contained in the text to be processed or the purpose of the text to be processed; for example, the original training corpus of the dialogue model can be used as the text to be processed, and the dialogue model can be trained by converting the text to be processed into dialogue text to improve the dialogue performance of the dialogue model; or, for example, text blocks used to build the knowledge base in the Retrieval-Augmented Generation (RAG) task can be used as the text to be processed, and the knowledge base can be built by converting the text to be processed into dialogue text to improve the retrieval accuracy of the Retrieval-Augmented Generation task. RAG is an artificial intelligence framework that combines information retrieval technology with large-scale language models, aiming to improve the accuracy, relevance, and timeliness of the content generated by the model.

[0056] In this embodiment, the method of obtaining the text to be processed is not limited. For example, it can be obtained directly from local storage or other devices, or it can be obtained by receiving plain text uploaded or input by the user as the text to be processed. Alternatively, it can be obtained by preprocessing plain text uploaded or input by the user, plain text stored locally, or plain text obtained from other devices. The specific preprocessing method is not limited in this embodiment. For example, it can include segmentation processing, such as segmenting the text content in the document to obtain at least one piece of text to be processed.

[0057] Step S20: Process the text to be processed using a preset first large language model to obtain a first dialogue text. The first large language model is pre-trained using training data. The training data includes at least one training text and training labels corresponding to the training text. The training labels are a second dialogue text containing all the text content in the training text. The second dialogue text contains at least one round of dialogue content.

[0058] In this embodiment, the advantages of Large Language Models (LLMs) in natural language processing are leveraged to implement text-to-dialogue conversion. The LLM used for this function is referred to as the first LLM to distinguish it from other LLMs. The first LLM can be fine-tuned and trained on an open-source LLM model before being deployed in a text processing device. After acquiring the text to be processed, the text processing device can use the first LLM to process the text and obtain dialogue text (hereinafter referred to as the first dialogue text for distinction). The goal of pre-training the first LLM is to enable it to accurately convert ordinary text into dialogue text containing all the text content of the ordinary text. Therefore, when using the trained first LLM to process the text to be processed, a first dialogue text that accurately covers all the text content of the text to be processed can be obtained.

[0059] To achieve the aforementioned training objectives, specific training data can be used to train the first major language model. Specifically, the ordinary text used to train the first major language model is referred to as the training text (for distinction). The training data may include at least one training text and corresponding dialogue texts (hereinafter referred to as the second dialogue texts for distinction). The second dialogue text may contain all the text content of the corresponding training text, and it includes at least one round of dialogue. One round of dialogue consists of two sentences, one question and one answer. The dialogue text containing all the text content of the corresponding ordinary text means that it covers all the content of the original text in the form of dialogue; from the reader's perspective, reading the dialogue text allows them to fully understand all the content of the original ordinary text. The training text serves as the processing object of the first language model during the training process, while the corresponding second dialogue text serves as the training label, supervising the results obtained by the first language model in processing the training text. Since the second dialogue text contains all the text content of the training text, it is possible to supervise the first language model to continuously move closer to the training target through the training label. In other words, the trained first language model can be made to have the performance to more accurately convert ordinary text into dialogue text containing all the text content of that ordinary text.

[0060] Therefore, it is understandable that since the first large language model is trained and deployed on the text processing device, and during its training process, it achieves its training objectives using the aforementioned specific training data, the resulting dialogue text obtained by processing the text using the first large language model can more accurately cover all the text content in the text to be processed. This allows for obtaining high-quality dialogue text containing all the content of a given ordinary text, thus meeting the demand for high-quality dialogue text in specific application scenarios. Furthermore, in this embodiment, text conversion using a large language model is simple, has relatively low time costs, and produces more natural dialogue with greater coherence between question-and-answer pairs.

[0061] It should be noted that there are many training methods that can achieve the above training objectives. In this embodiment, the specific training process of the first language model is not limited.

[0062] In this embodiment, there are no restrictions on the source of the training text and the corresponding second dialogue text; for example, they can be collected manually.

[0063] In a specific implementation, the device for training the first language model can be a text processing device or other devices. That is, after training on other devices, the first language model can be deployed to the text processing device to support the text-to-speech function in the text processing device.

[0064] In a specific implementation, processing the text to be processed through the first large language model can involve directly inputting the text into the first large language model, or it can involve preprocessing the text before inputting it into the first large language model. The first large language model then processes the input data layer by layer through its internal structure, outputting the first dialogue text. The specific preprocessing methods are not limited in this embodiment; for example, they could be processing to conform to the format requirements of the large language model for the input data. It is understood that the process of processing training text through the first large language model during the training phase can also refer to the process of processing text to be processed through the first large language model described above, and will not be elaborated upon here.

[0065] In specific implementations, when there are specific formatting requirements for the dialogue text (e.g., JSON format), prompt words can be designed to instruct the first language model to directly output the formatted first dialogue text. Alternatively, the unformatted first dialogue text output by the first language model can be further converted into a specific format. In one feasible implementation, the specific format can be: (text to be processed, first dialogue text). After obtaining the text in this format through the first language model, the required text can be extracted from it as needed.

[0066] In one feasible implementation, multiple texts to be processed can be batch-converted to obtain the first dialogue text corresponding to each text.

[0067] In a specific implementation, the converted first dialogue text can be stored locally, or it can be output, for example, sent to other devices or displayed through a display device.

[0068] In one feasible implementation, to further improve the accuracy of the first dialogue text obtained by the first language model processing the text to be processed, and to ensure that the first dialogue text more accurately covers all the text content in the text to be processed, certain prompt words can be given to the first language model when processing the text to be processed, so as to prompt the first language model with the corresponding text processing target. Optionally, step S20 includes S201 to S202:

[0069] Step S201: Obtain prompt words, wherein the prompt words include first prompt content, the first prompt content is used to prompt the first large language model to at least satisfy a first condition when processing the text to be processed, the first condition being that the first dialogue text contains at least all the text content in the text to be processed.

[0070] It should be noted that, for the sake of distinction, the prompt words involved in steps S201 to S202 are called the first prompt words, the condition used to limit the first dialogue text to include all the text content in the text to be processed is called the first condition, and the prompt content used to prompt the first language model to satisfy the first condition when processing the text to be processed is called the first prompt content.

[0071] The first prompt word may include a first prompt content to prompt the first language model to at least satisfy a first condition when processing the text to be processed. This allows the first language model, guided by the first prompt content, to process a first dialogue text that more accurately covers all text content in the text to be processed. It should be noted that there are many types of first prompt content that can achieve the above-mentioned prompting purpose, and this embodiment does not impose any limitations on them.

[0072] For example, the first prompt could be: "Please help me convert the following text into dialogue text. The converted dialogue text needs to contain all the text content of the original text." Alternatively, it could include an example, which could be a piece of plain text and a dialogue text containing all the dialogue content of that plain text. The example helps the first language model better "understand" the meaning of the converted dialogue text containing all the text content of the original plain text, thus enabling it to process a first dialogue text that more accurately covers all the text content of the text to be processed.

[0073] The method of obtaining the first prompt word is not limited in this embodiment. For example, it can be preset in the text processing device. When the text processing device needs to process the text to be processed, it can obtain the first prompt word from the local storage. Alternatively, the text processing device can receive the first prompt word uploaded by the user.

[0074] Step S202: Process the text to be processed and the prompt words through the first large language model to obtain the first dialogue text.

[0075] Processing the text to be processed and the first prompt word through the first large language model can be achieved by directly inputting the text to the first large language model, or by preprocessing the text and the first prompt word before inputting them into the first large language model. The internal structure of the first large language model processes the input data layer by layer to output the first dialogue text. The specific preprocessing methods are not limited in this embodiment. For example, they could be processing to conform to the format requirements of the large language model for the input data, or the text to be processed could be embedded into a reserved embedding position in the first prompt word before being input into the large language model.

[0076] In one feasible implementation, to further improve the accuracy of the first dialogue text obtained by the first language model processing the text to be processed, the training data of the first language model may also include prompt words (hereinafter referred to as second prompt words for distinction). The second prompt words may contain prompting content to prompt the first language model to process the dialogue text obtained by processing the training text to include all the content in the training text. This allows the first language model to learn how to process accurate dialogue text based on this type of prompt content during the training phase. Consequently, when the trained first language model processes the text to be processed and the first prompt words, it can more accurately parse the meaning of the first prompt content in the first prompt words, thus making the first dialogue text more accurately cover all the text content in the text to be processed.

[0077] Based on the first embodiment described above, a second embodiment of the text processing method of this application is proposed. In this embodiment, content that is the same as or similar to that in the first embodiment can be referred to the above description and will not be repeated hereafter. In different application scenarios, the requirements for dialogue text may differ. For example, some application scenarios require dialogue content with a high degree of clarity of intent and completeness of dialogue, while others may require dialogue content with specific linguistic phenomena. In this embodiment, considering the above situations, it is proposed that the first prompt word, in addition to the first prompt content, may also include prompt content corresponding to the task. The task is the task applied to the converted first dialogue text. The task applied to the first dialogue text can be preset, and prompt content corresponding to that task can be set. The prompt content corresponding to the task can be used to prompt that the first dialogue text obtained by the first language model processing the text to be processed must meet the specific requirements of the task for the dialogue text. In this embodiment, there are no restrictions on the task applied to the first dialogue text or the specific requirements of the task for the dialogue text.

[0078] In one feasible implementation, when the training data of the first language model also includes the second prompt word, the second prompt word may also include prompt content corresponding to the task. This prompt content can be used to prompt the first language model to process the training text to obtain dialogue text that meets the specific requirements of the task for dialogue text. This allows the first language model to learn to process accurate dialogue text based on this type of prompt content during the training phase. Consequently, when the trained first language model processes the text to be processed and the first prompt word, it can more accurately parse the meaning of the prompt content corresponding to the task in the first prompt word, thereby ensuring that the processed first dialogue text meets the specific requirements of the task for dialogue text.

[0079] In one feasible implementation, the second dialogue text in the training data of the first language model can be dialogue text that meets the specific requirements of the task. Thus, when the first language model is supervised by the second dialogue text as training labels during the training phase, it can accurately learn how to process a piece of ordinary text to obtain dialogue text that meets the specific requirements. This enables the trained first language model to process the text to be processed to obtain the first dialogue text that meets the specific requirements of the task, thereby making it more suitable for the task.

[0080] In one feasible implementation, the task applied to the first dialogue text can be a dialogue model training task; that is, the first dialogue text can be used as the training corpus for the dialogue model. This task will be referred to as the first task. Alternatively, the first task can be understood as training the dialogue model using the first dialogue text as training corpus. The condition used to limit the types or number of linguistic phenomena contained in the first dialogue text is called the second condition, and the prompt content used to indicate that the first language model, when processing the text to be processed, still satisfies the second condition is called the second prompt content. The second prompt content is used as the prompt content corresponding to the first task, and the first prompt word may also include the second prompt content.

[0081] Among them, linguistic phenomena refer to some phenomena that may occur in the dialogue language of natural persons during the dialogue process, such as linguistic phenomena such as omission of intent and dissolution of reference.

[0082] For example, the following dialogue text (1) is a dialogue text that intends to omit this linguistic phenomenon, and dialogue text (2) is a dialogue text that uses reference to dissolve this linguistic phenomenon. A and B are two speakers.

[0083] Dialogue text (1):

[0084] A: Did you go to the movies yesterday?

[0085] B: Yeah, I went.

[0086] A: What do you think?

[0087] B: It was alright, but the ending felt a bit rushed.

[0088] The linguistic phenomena in the dialogue text (1) are analyzed as follows: the predicate "watching the movie" is omitted in the second round, the subject and object "this movie" are omitted in the third round, and the attributive "of the movie" is omitted in the fourth round.

[0089] Dialogue text (2):

[0090] A: Xiaoming recently bought a new mobile phone, and he really likes it.

[0091] B: What brand is it?

[0092] A: It's an Apple product, but it's too expensive.

[0093] B: That's definitely not cheap.

[0094] The linguistic phenomena in the dialogue text (2) are analyzed as follows: "he" refers to "Xiaoming", "it" refers to the "mobile phone" mentioned earlier, and "that" refers to the entire concept of "Apple mobile phone".

[0095] The type of language phenomena that the first dialogue text must contain refers to which type or several types of language phenomena must be included in the first dialogue text. The number of language phenomena that the first dialogue text must contain refers to how many language phenomena, or at least how many language phenomena, must be included in the first dialogue text.

[0096] For example, the second prompt could be: "The generated dialogue text needs to contain at least one of the two linguistic phenomena of intention omission and reference resolution," or it could be: "The generated dialogue text needs to contain at least two linguistic phenomena." Based on this, the second prompt could also include an example, which could be a dialogue text containing at least one of the two linguistic phenomena of intention omission and reference resolution, or it could be a dialogue text containing at least two linguistic phenomena. Through the example, the first language model can better "understand" the meaning of the linguistic phenomena that the converted dialogue text needs to contain, and thus be able to process a first dialogue text that more accurately contains a specific type of linguistic phenomenon or a specific number of linguistic phenomena.

[0097] In this embodiment, by designing a second prompt content corresponding to the dialogue model training task within the first prompt word, the second prompt content indicates that the first language model, when processing the text to be processed, also meets the second condition. That is, it indicates the type or number of linguistic phenomena at least contained in the first dialogue text processed by the first language model. This allows the first language model to automatically generate a first dialogue text containing a specific type or number of linguistic phenomena. Thus, by providing only a plain text, one can obtain a dialogue text containing all the content of that plain text and including a specific type or number of linguistic phenomena, which can be used for the dialogue model training task. When the training data requirement for the dialogue model is large, this significantly reduces the cost of constructing training data and improves the efficiency of training data construction.

[0098] In one feasible implementation, the second dialogue text in the training data of the first large language model can be dialogue text containing specific types or a specific number of linguistic phenomena. The linguistic phenomena contained in the second dialogue text in different training data can be different, making the variety of linguistic phenomena covered by all training data richer. By training the first large language model with training data containing a rich variety of linguistic phenomena, the first large language model can accurately learn how to process a piece of ordinary text to obtain dialogue text containing specific types or a specific number of linguistic phenomena when supervised by the second dialogue text used as training labels during the training phase. This enables the trained first large language model to process the text to be processed to obtain the first dialogue text that meets the specific requirements of the dialogue model training task, thus making it more suitable for the dialogue model training task.

[0099] In one feasible implementation, when the training data of the first language model also includes the second prompt word, the second prompt word may further include prompting content that prompts the first language model to process the training text and obtain dialogue text that also contains a specific type of language phenomenon or a specific number of language phenomena. This allows the first language model to learn during the training phase how to process accurate dialogue text based on this type of prompt content. Consequently, when the trained first language model processes the text to be processed and the first prompt word, it can more accurately parse the meaning of the second prompt content in the first prompt word. This ensures that the first dialogue text not only contains all the content of the text to be processed but also contains a specific type of language phenomenon or a specific number of language phenomena, making it suitable for the training task of the dialogue model.

[0100] In one feasible implementation, the task of applying the first dialogue text can be a retrieval enhancement generation task (hereinafter referred to as the second task for distinction). This can be understood as the second task being the application of the first dialogue text to the retrieval enhancement generation task. The condition used to limit the explicitness of intent and / or the completeness of dialogue in the first dialogue text is called the third condition, and the prompt content used to indicate that the first language model still satisfies the third condition when processing the text is called the third prompt content. The third prompt content is used as the prompt content corresponding to the second task, and the first prompt word may also include the third prompt content.

[0101] The clarity of intent in the dialogue text refers to the degree of explicitness of the intent within the dialogue text. This includes, for example, the clarity of the purpose and needs, subject, and professional names within the dialogue. Limiting the clarity of intent in the dialogue text can mean, for example, requiring the intent to be completely explicit, or requiring the dialogue text to reach a certain level of intent clarity, such as 90%. The specific setting can be determined according to the actual needs of the second task and is not limited in this embodiment. The completeness of the dialogue text includes, for example, the completeness of the dialogue background information, key details, etc. Limiting the completeness of the dialogue text can mean, for example, requiring the dialogue text to be complete, or requiring the dialogue text to reach a certain level of completeness, such as 90% completeness. The specific setting can be determined according to the actual needs of the second task and is not limited in this embodiment.

[0102] In specific implementations, a third condition can be set as needed to limit the clarity of intent of the first dialogue text, or to limit the completeness of the dialogue in the first dialogue text, or to limit both the clarity of intent and the completeness of the dialogue in the first dialogue text.

[0103] For example, the third prompt could be: "The generated dialogue text needs to be fully explicit in intent and complete in dialogue." Furthermore, the third prompt could include an example, which could be a dialogue text that is fully explicit in intent and complete in dialogue. This example helps the first language model better "understand" the meaning of fully explicit intent and complete dialogue, thereby enabling it to process the first dialogue text that meets both the requirements for explicit intent and complete dialogue.

[0104] In this embodiment, by designing a third prompt corresponding to the retrieval enhancement generation task within the first prompt word, the third prompt content instructs the large language model to meet a third condition when processing the task. Specifically, it prompts the large language model that the first dialogue text processed by the first large language model needs to achieve a specific level of intent clarity and / or dialogue completeness. This allows the large language model to automatically generate a first dialogue text that meets the requirements for intent clarity and / or dialogue completeness. Thus, by providing only a plain text, a dialogue text containing all the content of that plain text and meeting the requirements for intent clarity and / or dialogue completeness can be obtained for use in the retrieval enhancement generation task. When the number of dialogue texts required in the retrieval enhancement generation task is large, this significantly reduces the construction cost and improves the construction efficiency of the dialogue text. Furthermore, by using the processed dialogue text that meets the requirements for intent clarity and / or dialogue completeness in the retrieval enhancement generation task, the recall accuracy and retrieval efficiency of the retrieval enhancement generation task can be improved.

[0105] In one feasible implementation, the second dialogue text in the training data of the first large language model can be dialogue text whose intent clarity and / or dialogue completeness meet the requirements of the retrieval enhancement generation task. This allows the first large language model to accurately learn how to process dialogue text based on a piece of ordinary text to obtain dialogue text whose intent clarity and / or dialogue completeness meet the requirements of the retrieval enhancement generation task when supervised by the second dialogue text used as training labels during the training phase. Consequently, when the trained first large language model processes the text to be processed, it can obtain the first dialogue text that meets the specific needs of the retrieval enhancement generation task, thus making it more suitable for the retrieval enhancement generation task.

[0106] In one feasible implementation, when the training data of the first language model also includes the second prompt word, the second prompt word may further include prompt content used to prompt the first language model to meet specific requirements regarding the clarity of intent and / or the completeness of dialogue obtained by processing the training text. This allows the first language model to learn to process accurate dialogue text based on this type of prompt content during the training phase. Consequently, when the trained first language model processes the text to be processed and the first prompt word, it can more accurately parse the meaning of the third prompt content in the first prompt word. This ensures that the first dialogue text not only contains all the content of the text to be processed, but also meets the specific requirements of the retrieval enhancement generation task for dialogue text, making it suitable for the retrieval enhancement generation task.

[0107] It should be noted that, in specific implementations, the dialogue text used for the first and second tasks can be trained based on the same first large language model or on two different first large language models; this embodiment does not impose any restrictions on this. In a feasible implementation, when the dialogue text used for the first and second tasks is trained based on the same first large language model, during the training phase of the first large language model, a portion of the training data can be designed specifically for the first task. Specifically, the second dialogue text and / or second prompt words in this portion of the training data can be designed specifically for the first task. That is, the second dialogue text can contain all the text content of the training text and contain a specific type of language phenomenon or a specific number of language phenomena, and / or, the second prompt words can include prompts indicating that the dialogue text obtained by the first large language model after processing the training text must contain all the content of the training text, and also prompts indicating that the dialogue text obtained by the first large language model after processing the training text must also contain all the content of the training text. The training data consists of two parts: one part contains prompts for specific types or a specific number of language phenomena; the other part is designed for the second task. Specifically, the second dialogue text and / or second prompts in this part of the training data can be designed for the second task. That is, the second dialogue text can contain all the text content in the training text and the clarity of intent and / or the completeness of the dialogue also meet the requirements of the retrieval enhancement generation task. And / or, the second prompts can include prompts to indicate that the dialogue text obtained by the first language model from the training text must contain all the content of the training text, and also prompts to indicate that the clarity of intent and / or the completeness of the dialogue text obtained by the first language model from the training text must meet specific requirements.

[0108] In one feasible implementation, when the dialogue texts used for the first and second tasks are trained based on the same primary language model, the same text to be processed can be processed using the primary language model with different primary prompt words to obtain two or more primary dialogue texts. Each primary dialogue text can then be used for a different task. For example, the primary language model can be used to process the text to be processed with primary prompt words containing primary and secondary prompt content to obtain one primary dialogue text for the first task; and the primary language model can be used to process the text to be processed with primary prompt words containing primary and secondary prompt content to obtain another primary dialogue text for the second task.

[0109] Based on the first and / or second embodiments of the text processing method described above, a third embodiment of the text processing method of this application is proposed. In this embodiment, content that is the same as or similar to the first and second embodiments described above can be referred to the above description and will not be repeated hereafter. The text processing method further includes S30 to S50:

[0110] Step S30: Construct a knowledge base based on the first dialogue text.

[0111] In this embodiment, it is proposed to combine the text-to-dialogue function with RAG to improve the retrieval efficiency and accuracy of RAG.

[0112] Currently, documents or document fragments obtained by segmenting documents (all of which are plain text, referred to as text blocks for distinction) are generally used as knowledge blocks to construct the knowledge base in the retrieval enhancement generation task. In this embodiment, all or part of the text blocks originally used to construct the knowledge base can be used as text to be processed and converted into first dialogue text, and the knowledge base can be constructed based on each first dialogue text.

[0113] In this embodiment, the specific implementation method for constructing a knowledge base based on the first dialogue text is not limited. For example, constructing a knowledge base based on the first dialogue text could be using the first dialogue text as a knowledge block in the knowledge base to replace the original text block used as a knowledge block in the knowledge base; alternatively, the first dialogue text and the original text block could be used as a whole as a knowledge block in the knowledge base; or, the first dialogue text and the original text block could be used as different knowledge blocks in the knowledge base. Other feasible implementation methods are not listed here.

[0114] Step S40: Obtain the user query text, match the user query text with the knowledge base, and obtain the matching result.

[0115] The user query text obtained in this step can be the original query text submitted by the user to the large language model (hereinafter referred to as the second large language model for distinction); or, the user query text obtained can be obtained by processing the original query text, such as reconstructing the original query text to supplement the missing content to make it more complete; or, the user query text obtained can be text obtained by converting other forms of query information submitted by the user to the large language model, for example, converting the image submitted by the user into text as the user query text.

[0116] The purpose of matching user query text with the knowledge base is to find content related to the user query text, recall this content, and enable the second language model to answer the questions contained in the user query text based on the content related to the user query text.

[0117] Depending on the specific method used to construct the knowledge base using the first dialogue text, the specific method for matching the user query text with the knowledge base will also differ, resulting in different matching results. In this embodiment, no restrictions are placed on the matching method or the matching results. For example, in one feasible implementation, when the first dialogue text replaces the original text block as a knowledge block in the knowledge base, the user query text can be converted into a text vector and matched with the text vectors converted from each of the first dialogue texts used as knowledge blocks in the knowledge base. The first dialogue text corresponding to the preset number of text vectors with the highest matching degree is then taken as the matching result.

[0118] Step S50: Process the matching results and the user query text using a preset second language model to obtain an answer corresponding to the user query text.

[0119] The second major language model is a pre-defined major language model used to answer user queries, and it can be an open-source major language model. The second major language model and the first major language model can be the same open-source major language model, or they can be different open-source major language models; this embodiment does not impose any restrictions.

[0120] Depending on the matching results obtained, the specific implementation method for processing the matching results and user queries using the second language model to obtain answers in this step will also vary, and this embodiment does not impose any restrictions on this. For example, in one feasible implementation, when the matching results are a preset number of first dialogue texts, the user query text and the recalled preset number of first dialogue texts can be input into the second language model, and the second language model can generate an answer corresponding to the user query text.

[0121] In this embodiment, on the one hand, by converting the text blocks originally used to build the knowledge base into dialogue text containing all the text content within the text blocks, and constructing the knowledge base based on the dialogue text, the knowledge base can be retrieved more accurately because the dialogue text has a stronger language structure than the original plain text. This allows the large language model to make more accurate answers based on the more accurate retrieval results. On the other hand, traditional RAG systems directly use the original documents or segment the original documents according to characters as knowledge blocks in the knowledge base. The document formats are diverse, including PDF, Word, HTML, etc., and the retrieval efficiency is significantly affected by the document format. Compared with this traditional approach, this embodiment converts the text blocks originally used to build the knowledge base into dialogue text containing all the text content within the text blocks, and constructs the knowledge base based on the dialogue text with a unified format. The retrieval efficiency is not limited by the document format, thereby improving the retrieval efficiency.

[0122] In one feasible embodiment, step S30 includes S301 to S302:

[0123] Step S301: Use at least one of the texts to be processed as a knowledge block in the knowledge base, or use at least one of the first dialogue texts corresponding to the texts to be processed as a knowledge block in the knowledge base, or use at least one of the texts to be processed and the first dialogue texts corresponding to each text to be processed as a knowledge block in the knowledge base.

[0124] For example, if there are X texts to be processed, the X texts to be processed can be used as knowledge blocks in the knowledge base, that is, a total of X knowledge blocks; or, the first dialogue text corresponding to the X texts to be processed can be used as knowledge blocks in the knowledge base, that is, a total of X knowledge blocks; or, the X texts to be processed and their corresponding X first dialogue texts can both be used as knowledge blocks in the knowledge base, that is, a total of 2*X knowledge blocks.

[0125] Step S302: Extract the questions from the question-answer pairs contained in the first dialogue text and add the extracted questions to the knowledge base.

[0126] The first dialogue text includes multiple rounds of dialogue, resulting in multiple question-and-answer pairs. Each question-and-answer pair includes one question and one answer. The questions from the question-and-answer pairs contained in the first dialogue text can be extracted and added to the knowledge base. In specific implementations, when only the text to be processed is used as a knowledge block, the questions extracted from the corresponding first dialogue text can be bound to the text to be processed, serving as an index for the text to be processed in the knowledge base. Similarly, when only the corresponding first dialogue text is used as a knowledge block, the questions extracted from the first dialogue text can be bound to the first dialogue text, serving as an index for the first dialogue text in the knowledge base. When both the text to be processed and its corresponding first dialogue text are used as knowledge blocks, the questions extracted from the corresponding first dialogue text can be bound to both the text to be processed and the first dialogue text, respectively, serving as indexes for both texts in the knowledge base.

[0127] In one feasible embodiment, step S40 includes S401 to S402:

[0128] Step S401: Based on the user query text, obtain matching questions from the knowledge base, wherein the matching questions are the N questions in the knowledge base that have a matching degree higher than a first preset threshold or the highest matching degree, where N>0 and N is an integer.

[0129] In a specific implementation, the user's query text can be converted into a text vector, and then matched with the text vectors converted from various questions indexed in the knowledge base. The N questions with a matching degree higher than a first preset threshold or the highest matching degree are selected as the matching questions. The first preset threshold can be set as needed, for example, to 90%. N can be set as needed, for example, to 10.

[0130] Step S402: Determine the matching result based on the text to be processed corresponding to the matching question in the knowledge base, or determine the matching result based on the first dialogue text corresponding to the matching question in the knowledge base, or determine the matching result based on the text to be processed corresponding to the matching question and the first dialogue text in the knowledge base.

[0131] In specific implementations, when only the text to be processed is used as a knowledge block, the matching result can be determined based on the text to be processed corresponding to the matching question in the knowledge base; when only the first dialogue text corresponding to the text to be processed is used as a knowledge block, the matching result can be determined based on the first dialogue text corresponding to the matching question in the knowledge base; when both the text to be processed and its corresponding first dialogue text are used as knowledge blocks, the matching result can be determined based on the text to be processed corresponding to the matching question and the first dialogue text in the knowledge base.

[0132] In one feasible implementation, determining the matching result based on the text to be processed corresponding to the matching question in the knowledge base can be as follows: taking the text to be processed corresponding to the matching question as the matching result, inputting the user query text and the matching result into the second large language model, and generating an answer corresponding to the user query text through the second large language model.

[0133] In one feasible implementation, determining the matching result based on the first dialogue text corresponding to the matching question in the knowledge base can be as follows: taking the first dialogue text corresponding to the matching question as the matching result, inputting the user query text and the matching result into the second language model, and generating an answer corresponding to the user query text through the second language model.

[0134] In one feasible implementation, determining the matching result based on the text to be processed corresponding to the matching question and the first dialogue text in the knowledge base can be as follows: taking the text to be processed corresponding to the matching question and the first dialogue text as the matching result, inputting the user query text and the matching result into the second large language model, and generating an answer corresponding to the user query text through the second large language model.

[0135] In one feasible implementation, if a question in the knowledge base is found that is completely identical to the user's query text or has a matching degree greater than a third preset threshold, then the answer corresponding to the question in the first dialogue text can be directly fed back to the user without needing to process and generate an answer through a second language model. The third preset threshold can be set to be greater than the first preset threshold.

[0136] By using the questions extracted from the first dialogue text as indexes in the knowledge base for the corresponding first dialogue text or text to be processed, and by matching the user query text with the indexed questions, the first dialogue text or text to be processed corresponding to the questions with high matching degrees can be recalled as matching results, thereby improving recall accuracy and retrieval efficiency. Specifically, on the one hand, if a text to be processed is related to the user query text, then for the first dialogue text corresponding to the text to be processed, the similarity between the text vector converted from the question extracted from the first dialogue text and the text vector converted from the user query text is higher than the similarity between the text vector converted from the text to be processed itself and the text vector converted from the user query text. This is because both the questions extracted from the first dialogue text and the user query text are questions, and compared to the text to be processed, they are both shorter texts, so the similarity between the vectors will be higher. This makes it easier to retrieve knowledge blocks more relevant to the user query text from the knowledge base, thus improving recall accuracy. Consequently, when the second language model answers the questions contained in the user query text based on the recalled knowledge blocks, it can obtain more accurate answers. On the other hand, if the text vector obtained from the user query text is directly matched with the text vector obtained from the original text block (i.e., the text to be processed), the length of the text block cannot be too long, otherwise it will affect the accuracy of the matching. Because the length of the text block is limited, longer documents need to be segmented to ensure that the length of each segmented text block meets the requirements, such as 512 or 1024 characters per block. This will lead to incomplete knowledge in document segmentation, significant loss of contextual information, and difficulties in updating and maintaining the knowledge base. However, if the user query text is matched with the question used as an index, since the question itself is shorter, there is no length limit. There is also no length limit when converting the text block into the first dialogue text. Therefore, the document content can be directly treated as a single text block, or the document can be coarsely segmented into multiple text blocks, for example... Figure 2As shown, each text block should not exceed 8k in size, thus avoiding problems such as incomplete knowledge and loss of hierarchical information, reducing the difficulty of knowledge base maintenance, and improving recall accuracy. This, in turn, improves the accuracy of answers generated by the second language model based on the recalled knowledge blocks. In specific implementations, documents can be segmented using methods such as segmentation by specified length, by title, or by semantics.

[0137] For example, such as Figure 3 As shown, taking document 1 as an example, document 1 is converted into dialogue text 1, which contains n question-answer pairs. The n questions in the question-answer pairs are extracted, and the n questions are bound to the dialogue text 1 and / or document 1 containing the n question-answer pairs and added to the knowledge base. For user query text, it is first matched with the questions in the knowledge base. For example, if the most relevant question 1 is matched, it is indexed in the knowledge base based on question 1, and the dialogue text 1 and / or document 1 bound to it are retrieved for the second language model to generate answers to user queries.

[0138] In one feasible implementation, step S402, which involves determining the matching result based on the text to be processed corresponding to the matching question in the knowledge base and the first dialogue text, includes:

[0139] Step S4021: Match the user query text with each target dialogue text to obtain the matched dialogue text. The matched dialogue text is the M dialogue texts among the target dialogue texts that have a matching degree higher than the second preset threshold or the highest matching degree, where N>M>0 and M is an integer. Each target dialogue text is the first dialogue text in the knowledge base that corresponds to the matching question.

[0140] After obtaining N matching questions, the first dialogue text corresponding to each of the N matching questions can be found in the knowledge base (hereinafter, the found first dialogue text is referred to as the target dialogue text for distinction). The user query text can be further matched with each target dialogue text. That is, each target dialogue text is used as a secondary index of the text to be processed, and the M dialogue texts with a matching degree higher than the second preset threshold or the highest matching degree are selected. The selected dialogue texts are referred to as the matching dialogue texts for distinction.

[0141] The second preset threshold can be set as needed, for example, to 90%. M can be set as needed, for example, to 8; the relationship between M and N is N>M>0, in order to achieve further filtering.

[0142] In a specific implementation, the user query text can be converted into a text vector, and then matched with the text vector obtained by converting the target dialogue text to calculate the matching degree.

[0143] Step S4022: Take the text to be processed in the knowledge base that corresponds to the matching dialogue text as the matching result, or take the matching dialogue text and the text to be processed in the knowledge base that corresponds to the matching dialogue text as the matching result.

[0144] After obtaining M matching dialogue texts, the corresponding text to be processed in the knowledge base can be used as the matching result. The user query text and the matching result are then input into the second language model, which generates the answer corresponding to the user query text. Alternatively, both the matching dialogue text and the corresponding text to be processed in the knowledge base can be used as matching results. The user query text and the matching result are then input into the second language model, which generates the answer corresponding to the user query text.

[0145] Based on the first, second, and / or third embodiments of the text processing method described above, a fourth embodiment of the large language model training method of this application is proposed. In this embodiment, content that is the same as or similar to the first, second, and third embodiments described above can be referred to the above description and will not be repeated hereafter. In this embodiment, the executing entity of the large language model training method can be a training device. The training device can be a device with program execution capabilities, specifically a server, personal computer, mobile phone, or other general computing service device. The training device and the text processing device in the above embodiments can be the same device or different devices. For ease of description, the executing entity is omitted from the description of this embodiment below.

[0146] In this embodiment, as Figure 4 As shown, the large language model training method includes A10 to A30:

[0147] Step A10: Obtain training text and training labels corresponding to the training text, wherein the training labels are second dialogue texts containing all text content in the training texts, and the second dialogue texts contain at least one round of dialogue content.

[0148] The method of obtaining training text and training labels is not limited in this embodiment. For example, they can be obtained directly from local storage or other devices, or ordinary text uploaded or input by the user can be used as training text, and dialogue text uploaded or input by the user can be used as second dialogue text.

[0149] The amount of training data used to train the first language model is not limited in this embodiment. The required amount can be set according to actual needs, and the corresponding amount of training data can be obtained for training. That is, step A10 is not limited to obtaining only one training text and one corresponding training label.

[0150] In one feasible implementation, the first large language model can be trained in multiple rounds, with each round of training using at least one piece of training data.

[0151] Step A20: The training text is processed by a preset first language model to be trained to obtain the third dialogue text.

[0152] In a specific implementation, the first large language model to be trained can be an open-source large language model. Since the open-source large language model has already been pre-trained, the training process of the first large language model in this embodiment can also be understood as a fine-tuning process on the open-source large language model. During the fine-tuning process, for the obtained training samples, the text to be processed is processed by the first large language model to obtain the dialogue text (hereinafter referred to as the third dialogue text for distinction).

[0153] A30, the first large language model is trained based on the third dialogue text and the training labels.

[0154] There are many training methods that can be used to train the first language model. The specific process of training the first language model based on the third dialogue text and training labels will differ depending on the training method used. In this embodiment, the training method adopted is not limited, and therefore, the specific process of training the first language model based on the third dialogue text and training labels is also not limited. For example, in one embodiment, a loss function can be calculated based on the third dialogue text and training labels, and the parameters of the first language model can be updated to make the loss function converge, thereby achieving the purpose of training the first language model.

[0155] In this embodiment, by using the training text as the processing object of the first language model during the training process, and the corresponding second dialogue text as the training label, the results obtained by the first language model in processing the training text are supervised. Since the second dialogue text contains all the text content of the training text, it is possible to supervise the first language model to continuously move closer to the training target through the training label. That is, the trained first language model can be made to have the performance of converting ordinary text into dialogue text containing all the text content of the ordinary text more accurately.

[0156] In one feasible implementation, a dataset for training a large language model can be prepared in advance, and the dataset format is as follows: in Let tj be the i-th training text, and there are a total of N training texts. Let tj represent the task (t1 is the dialogue model training task, and t2 is the text augmentation generation task). It is the training label (second dialogue text) corresponding to the i-th training text. The first large language model is trained using this dataset, so that the trained first large language model can generate dialogue text that can be used for dialogue model training tasks and retrieval enhancement generation tasks.

[0157] In one feasible implementation, in the aforementioned dataset Based on this, a full-scale fine-tuning training method can be used to train the first major language model. The training process is as follows:

[0158] 1. You can first load all the parameters of a pre-trained open-source large language model.

[0159] θ model ={θ1,θ1,…,θ K}

[0160] Where θ k This represents the k-th parameter in the model, which has a total of K parameters.

[0161] 2. Fine-tuning the target: During the fine-tuning process, optimize θ. model To enable model processing The output result (That is, the third dialogue text) is close to the training label. Minimize This is achieved through a loss function between the label and the true label. The loss function can be either cross-entropy loss or mean squared error loss.

[0162] For example, the loss function can be set as: in Model processing The output, θ, represents the parameters of the model.

[0163] The model parameters can be updated using the backpropagation algorithm and the gradient descent method. By iterating repeatedly, the model parameters can be optimized until the loss function converges.

[0164] Alternatively, in one feasible implementation, in the aforementioned dataset Building upon this foundation, an efficient fine-tuning training method can be employed to train the first major language model. Efficient fine-tuning aims to reduce the number of parameters requiring tuning, thereby lowering computational and storage costs while preserving as much of the advantages of the pre-trained model as possible. In one feasible implementation, LoRA (Low-Rank Adaptation) can be used. The LoRA method reduces the number of parameters that need to be optimized during fine-tuning by performing a low-rank decomposition of the model's weight matrix. The training process is as follows:

[0165] 1. For any weight matrix θ in the first language model, it can be decomposed into two low-rank matrices θ = θ0 + Δθ = θ0 + AB using the LoRA method, where θ0 is the fixed weight during pre-training and Δθ = AB is the adjusted part obtained through training.

[0166] 2. Update and Optimization: During training, only the two low-rank matrices A and B are optimized, while θ0 remains unchanged. The loss function can be designed as follows:

[0167]

[0168] 3. Parameter update: The model parameters can be updated using the backpropagation algorithm and gradient descent method (only the parameters of A and B are updated). By iterating repeatedly, the model parameters are optimized until the loss function converges, and only the parameters of A and B are updated.

[0169] In one feasible implementation, when acquiring training text and training labels, a corresponding second prompt word can also be acquired. The second prompt word may contain prompting content that prompts the first language model to process the training text so that the resulting dialogue text contains all the content of the training text. This allows the first language model to learn how to process accurate dialogue text based on this type of prompt content during the training phase. Consequently, when the trained first language model processes the text to be processed and the first prompt word, it can more accurately parse the meaning of the first prompt content in the first prompt word, thus making the first dialogue text more accurately cover all the text content in the text to be processed.

[0170] In one feasible implementation, the step of obtaining the training label corresponding to the training text in step A10 includes A101 to A102:

[0171] Step A101: The training text is processed by a preset third language model to obtain multiple fourth dialogue texts.

[0172] In this embodiment, a large language model (hereinafter referred to as the third large language model for distinction) is used to process the training text to obtain dialogue text (hereinafter referred to as the fourth dialogue text for distinction). Then, by filtering the fourth dialogue text, a second dialogue text that can be used as training labels is obtained. In a specific embodiment, the third large language model can be an open-source large language model. The third large language model can be the first large language model to be trained. That is, the third large language model and the first large language model to be trained can be the same open-source large language model, or the third large language model and the first large language model to be trained can be implemented using different open-source large language models.

[0173] Understandably, the third language model was not specifically trained for text-to-dialogue conversion. Therefore, the fourth dialogue text obtained by the third language model from the training text may not meet the requirements. For example, it may not contain all the text content of the training text. Therefore, the fourth dialogue text can be further filtered to obtain the second dialogue text that can be used as training labels.

[0174] For a training text, a third-largest language model can be used to process the training text to obtain multiple fourth dialogue texts. For example, for document D, processing with the third-largest language model yields multiple fourth dialogue texts, each of which can be represented as {(Q1,A1),(Q2,A2),...,(Q...}}. n A n )}, where Q represents the question and A represents the answer.

[0175] In specific implementations, when processing training text using a third language model, certain prompt words can be provided to the third language model to indicate the corresponding text processing target. These prompt words will be referred to as "third prompt words" for distinction. For example, in one feasible implementation, the third prompt words may include prompts to indicate that the third language model processes one training text to obtain multiple dialogue texts, enabling the third language model to obtain multiple fourth dialogue texts with a single processing step, thus improving dialogue text generation efficiency; and / or, the third prompt words may include prompts to indicate that the third language model converts the training text into dialogue text containing all the text content of the training text, increasing the likelihood that the third language model will obtain a fourth dialogue text containing all the text content of the training text, thereby facilitating the accurate and rapid selection of second dialogue texts that can be used as training labels from multiple fourth dialogue texts.

[0176] Step A102: Select at least one second dialogue text that meets the preset conditions from multiple fourth dialogue texts, and use the second dialogue text as the training label corresponding to the training text. The preset conditions include a fourth condition, which is that the selected second dialogue text contains at least all the text content in the training text.

[0177] The condition used to limit the selected second dialogue text to contain all the text content in the training text is called the fourth condition for distinction. In specific implementations, the preset conditions may include only the fourth condition, or they may include other conditions; this implementation does not impose any restrictions.

[0178] The method for selecting the second dialogue text that meets the fourth condition from the fourth dialogue text is not limited in this embodiment. In one feasible embodiment, multiple fourth dialogue texts can be output for the user to select, and the dialogue text uploaded by the user can be received as the second dialogue text that meets the fourth condition, or the user's selection instruction can be received, and the fourth dialogue text pointed to by the selection instruction can be used as the second dialogue text that meets the fourth condition. In this embodiment, the user can filter the multiple output fourth dialogue texts, and can modify the fourth dialogue text to meet the preset conditions and upload it. In another feasible implementation, an evaluation model can be set up to evaluate whether each fourth dialogue text satisfies the fourth condition. There are many ways to implement the evaluation model, and no limitation is made here. For example, it can be implemented using an open-source large language model or a regular text classification model. If an open-source large language model is used, prompt words can be designed to prompt whether the fourth dialogue text contains all the text content of the training text. If a text classification model is used, the training text and the fourth dialogue text can be input into the text classification model, and the model outputs a binary classification result, that is, a result representing whether the fourth dialogue text contains all the text content of the training text. This text classification model can be pre-trained using training corpus. The training corpus can include multiple training data points. Each training data point can include a piece of ordinary text and the corresponding dialogue text, as well as training labels indicating whether the dialogue text contains all the text content of the ordinary text.

[0179] In one feasible implementation, if multiple second dialogue texts that meet preset conditions are selected from multiple fourth dialogue texts, then the multiple second dialogue texts can be combined with training texts to obtain multiple training data.

[0180] In one feasible implementation, based on the task applied to the first dialogue text obtained by processing the text to be processed using the first large language model trained, preset conditions can be set, including conditions related to the task. This can facilitate the selection of a second dialogue text that meets the specific requirements of the task. Then, the first large language model can be trained using the second dialogue text that meets the specific requirements of the task as training labels, which can make the first dialogue text obtained by processing the text to be processed by the trained first large language model more suitable for the task.

[0181] In one feasible implementation, when the third prompt word prompts the corresponding text processing target of the third language model, the third prompt word may also include prompt content corresponding to the task, so that the probability of the multiple fourth dialogue texts obtained by the third language model from processing the training text is higher, which is conducive to accurately and quickly selecting the second dialogue texts that can be used as training labels from the multiple fourth dialogue texts.

[0182] In one feasible implementation, the condition used to limit the types or number of linguistic phenomena that the selected second dialogue text must contain is referred to as the fifth condition for distinction. When the task applied to the first dialogue text includes the first task, i.e., the task of training the dialogue model using the first dialogue text as training corpus (dialogue model training task), the preset conditions may also include this fifth condition. Limiting the types of linguistic phenomena that the second dialogue text must contain refers to limiting which types or several types of linguistic phenomena should be included in the second dialogue text; limiting the number of linguistic phenomena that the second dialogue text must contain refers to limiting how many types of linguistic phenomena, or how many at least, or how many at most, should be included in the second dialogue text. In a specific implementation, when filtering fourth dialogue texts corresponding to different training texts, the types of linguistic phenomena limited to be included in the fifth condition may be different, or the number of linguistic phenomena limited to be included may be different, thereby making the types of linguistic phenomena covered by all training data more diverse. For example, for training text 1 and training text 2, dialogue texts containing the linguistic phenomenon of referential dissolution are selected from multiple fourth dialogue texts corresponding to training text 1 as the second dialogue text; dialogue texts containing the linguistic phenomenon of intentional omission are selected from multiple fourth dialogue texts corresponding to training text 1 as the second dialogue text; or, dialogue texts containing three linguistic phenomena are selected from multiple fourth dialogue texts corresponding to training text 1 as the second dialogue text; or dialogue texts containing two linguistic phenomena are selected from multiple fourth dialogue texts corresponding to training text 1 as the second dialogue text. Training the first language model with training data containing a rich variety of linguistic phenomena allows the first language model, when supervised by the second dialogue texts used as training labels during the training phase, to accurately learn how to process a piece of ordinary text to obtain dialogue texts containing specific types or a specific number of linguistic phenomena. This enables the trained first language model to process texts that meet the specific requirements of the dialogue model training task, thus making it more suitable for the dialogue model training task.

[0183] In this embodiment, the method for selecting the second dialogue text that meets the fifth condition from the fourth dialogue text is not limited. In one feasible embodiment, multiple fourth dialogue texts can be output for the user to select, and the dialogue text uploaded by the user can be received as the second dialogue text that meets the fifth condition, or the user's selection instruction can be received, and the fourth dialogue text pointed to by the selection instruction can be used as the second dialogue text that meets the fifth condition. In another feasible implementation, an evaluation model can be set up to evaluate whether each fourth dialogue text meets the fifth condition. There are many ways to implement the evaluation model, and no limitation is made here. For example, it can be implemented using an open-source large language model or a regular text classification model. If an open-source large language model is used, prompt words can be designed to prompt the large language model to evaluate whether the fourth dialogue text contains a specific type or number of language phenomena. If a text classification model is used, the training text and the fourth dialogue text can be input into the text classification model, and the model outputs a binary classification result, that is, a result representing whether the fourth dialogue text contains a specific type or number of language phenomena. This text classification model can be pre-trained using training corpus. The training corpus can include multiple training data points, each of which can include a piece of ordinary text and the corresponding dialogue text, as well as training labels indicating whether the dialogue text contains a specific type or number of language phenomena.

[0184] In one feasible implementation, when the third prompt word prompts the corresponding text processing target of the third language model, the third prompt word may also include prompt content corresponding to the dialogue model training task. Specifically, it may include prompt content for prompting the type or number of language phenomena to be included in the fourth dialogue text obtained by the third language model from processing the training text, so that the probability of including the fourth dialogue text that meets the specific requirements of the dialogue model training task among the multiple fourth dialogue texts obtained by the third language model from processing the training text is higher, which is conducive to accurately and quickly selecting the second dialogue text that can be used as training labels from the multiple fourth dialogue texts.

[0185] In one feasible implementation, the condition used to limit the intent clarity and / or dialogue completeness of the selected second dialogue text is referred to as the sixth condition for distinction. When the task applied to the first dialogue text includes a second task, i.e., when the first dialogue text is used for a retrieval enhancement generation task, the preset conditions may also include this sixth condition. Regarding the meaning of "limiting the intent clarity and / or dialogue completeness of the second dialogue text" in the sixth condition, refer to the meaning of "limiting the intent clarity and / or dialogue completeness of the first dialogue text" in the third condition of the above embodiments, which will not be repeated here. By selecting second dialogue texts whose intent clarity and / or dialogue completeness meet the requirements of the retrieval enhancement generation task, the first language model, when supervised by the second dialogue text used as training labels during the training phase, can accurately learn how to process dialogue texts based on ordinary text to obtain dialogue texts whose intent clarity and / or dialogue completeness meet the requirements of the retrieval enhancement generation task. This enables the trained first language model to process texts that meet the specific needs of the retrieval enhancement generation task, thus making it more suitable for the task.

[0186] In this embodiment, the method for selecting the second dialogue text that meets the sixth condition from the fourth dialogue text is not limited. In one feasible embodiment, multiple fourth dialogue texts can be output for the user to select, and the dialogue text uploaded by the user can be received as the second dialogue text that meets the sixth condition, or the user's selection instruction can be received, and the fourth dialogue text pointed to by the selection instruction can be used as the second dialogue text that meets the sixth condition. In another feasible implementation, an evaluation model can be set up to assess whether each fourth dialogue text meets the sixth condition. There are many ways to implement the evaluation model, and no limitation is made here. For example, it can be implemented using an open-source large language model or a regular text classification model. If an open-source large language model is used, prompt words can be designed to evaluate whether the explicitness of intent and / or the completeness of dialogue in the fourth dialogue text meets the requirements of the retrieval enhancement generation task. If a text classification model is used, the training text and the fourth dialogue text can be input into the text classification model, and the model outputs a binary classification result, that is, a result representing whether the explicitness of intent and / or the completeness of dialogue in the fourth dialogue text meets the requirements of the retrieval enhancement generation task. This text classification model can be pre-trained using training corpus. The training corpus can include multiple training data points, each of which can include a piece of ordinary text and its corresponding dialogue text, as well as training labels to indicate whether the explicitness of intent and / or the completeness of dialogue in the fourth dialogue text meets the requirements of the retrieval enhancement generation task.

[0187] In one feasible implementation, when the third prompt word prompts the corresponding text processing target of the third language model, the third prompt word may also include prompt content corresponding to the retrieval enhancement generation task. Specifically, it may include prompt content to prompt that the intent clarity and / or dialogue completeness of the fourth dialogue text obtained by the third language model from the training text must meet specific requirements. This makes it more likely that the fourth dialogue text obtained by the third language model from the training text includes a fourth dialogue text that meets specific requirements in terms of intent clarity and / or dialogue completeness, thereby facilitating the accurate and rapid selection of second dialogue texts that can be used as training labels from multiple fourth dialogue texts.

[0188] For example, multiple generated fourth-order dialogue texts are filtered to obtain two types of corpora suitable for dialogue model training tasks and retrieval enhancement generation tasks:

[0189] 1) Dialogue model training task: {(Q′1,A′1),(Q′2,A′2),...,(Q′...} n ,A′ n )}.

[0190] 2) Retrieval Enhancement Generation Task: {(Q″1,A″1),(Q″2,A″2),...,(Q″1,A″2)} n ,A″ n )}.

[0191] To aid in understanding the implementation process of the text processing methods and large language model training methods in the above embodiments, an implementation example is provided. For example... Figure 5 As shown in this example, the training text is processed by a pre-defined third language model, converting it into multiple fourth dialogue texts. At least one second dialogue text that meets pre-defined conditions is selected from the multiple fourth dialogue texts and used as the training label corresponding to the training text. The first language model is trained using the training text and training labels. The trained first language model processes at least one text to be processed, converting it into the corresponding first dialogue text. A knowledge base is constructed based on the first dialogue text corresponding to at least one text to be processed. After obtaining the user's query text, the user's query text is matched with the knowledge base to obtain the matching result. The matching result and the user's query text are processed by the second language model to obtain the answer corresponding to the user's query text.

[0192] This application provides a text processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the text processing method described in the above embodiments.

[0193] The following is for reference. Figure 6 It shows a schematic diagram of a text processing device suitable for implementing the embodiments of this application. Figure 6 The text processing device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0194] like Figure 6 As shown, a text processing device may include a processing unit 1001 (e.g., a DSP processor), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the text processing device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: an input device 1007 including, for example, a microphone, an accelerometer, etc.; an output device 1008 including, for example, a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 allows the text processing device to communicate wirelessly or wiredly with other devices to exchange data. Although text processing devices with various systems are shown in the figures, it should be understood that it is not required to implement or possess all the systems shown. It can be implemented alternatively or with more or fewer systems.

[0195] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0196] Compared with the prior art, the beneficial effects of the text processing device provided in this application embodiment are the same as the beneficial effects of the text processing method provided in the above embodiments, and will not be repeated here.

[0197] It should be understood that the various parts disclosed in the embodiments of this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0198] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the text processing method in the above embodiments.

[0199] The computer-readable storage medium provided in this application embodiment may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0200] The aforementioned computer-readable storage medium may be included in a text processing device; or it may exist independently and not be assembled into a text processing device.

[0201] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by a text processing device, cause the text processing device to perform the functions defined in the methods of the embodiments disclosed in this application.

[0202] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0203] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0204] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0205] The readable storage medium provided in this application embodiment is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for performing the above-described text processing method. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application embodiment are the same as the beneficial effects of the text processing method provided in the above-described embodiments, and will not be repeated here.

[0206] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the text processing method described above.

[0207] Compared with the prior art, the beneficial effects of the computer program product provided in this application embodiment are the same as the beneficial effects of the text processing method provided in the above embodiments, and will not be repeated here.

[0208] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A text processing method characterized by, The text processing method comprises: acquiring a text to be processed; processing the text to be processed by a preset first large language model to obtain a first dialogue text, wherein the first large language model is obtained by pre-training using training data, the training data comprises at least one training text and a training label corresponding to the training text, the training label is a second dialogue text containing all text contents in the training text, and the second dialogue text contains at least one round of dialogue content.

2. The text processing method of claim 1, wherein, The processing of the text to be processed by the preset first large language model to obtain the first dialogue text comprises: acquiring a prompt word, wherein the prompt word contains first prompt content, the first prompt content is used to prompt the first large language model to at least meet a first condition when processing the text to be processed, and the first condition is that the first dialogue text at least contains all text contents in the text to be processed; processing the text to be processed and the prompt word by the first large language model to obtain the first dialogue text.

3. The text processing method of claim 2, wherein, The prompt word further contains second prompt content corresponding to a first task, the second prompt content is used to prompt the first large language model to further meet a second condition when processing the text to be processed, the second condition is that the first dialogue text at least contains a type or quantity of language phenomenon, and the first task is a task of training a dialogue model by using the first dialogue text as training corpus; Or, the prompt word further contains third prompt content corresponding to a second task, the third prompt content is used to prompt the first large language model to further meet a third condition when processing the text to be processed, the third condition is used to limit an intention explicitness degree and / or dialogue completeness degree of the first dialogue text, and the second task is a task of using the first dialogue text for retrieval and enhancement generation.

4. The text processing method of claim 1, wherein, The text processing method further comprises: constructing a knowledge base according to the first dialogue text; acquiring a user query text, matching the user query text with the knowledge base to obtain a matching result; processing the matching result and the user query text by a preset second large language model to obtain an answer corresponding to the user query text.

5. The text processing method of claim 4, wherein, The construction of the knowledge base according to the first dialogue text comprises: taking the first dialogue text corresponding to at least one of the texts to be processed as a knowledge block in the knowledge base, or taking at least one of the texts to be processed and the first dialogue text corresponding to each of the texts to be processed as a knowledge block in the knowledge base; extracting a question in a question and answer pair contained in the first dialogue text, and adding the extracted question to the knowledge base; The acquisition of the user query text, the matching of the user query text with the knowledge base, and the obtaining of the matching result comprise: matching a matching question from the knowledge base according to the user query text, wherein the matching question is a question in the knowledge base with a matching degree higher than a first preset threshold or a highest matching degree N questions, N>0, and N is an integer. determine a matching result according to the first dialogue text corresponding to the matching question in the knowledge base, or determine a matching result according to the to-be-processed text and the first dialogue text corresponding to the matching question in the knowledge base.

6. The text processing method of claim 5, wherein, The determining a matching result according to the to-be-processed text and the first dialogue text corresponding to the matching question in the knowledge base comprises: taking the to-be-processed text and the first dialogue text corresponding to the matching question in the knowledge base as the matching result; or, matching the user query text with each target dialogue text to obtain a matching dialogue text, wherein the matching dialogue text is M dialogue texts in the target dialogue texts that have a matching degree higher than a second preset threshold or have the highest matching degree, N > M > 0, and M is an integer; taking the to-be-processed text corresponding to the matching dialogue text in the knowledge base as the matching result, or taking the matching dialogue text and the to-be-processed text corresponding to the matching dialogue text in the knowledge base as the matching result.

7. A large language model training method, characterized in that, The large language model training method comprises: obtaining training text and a training label corresponding to the training text, wherein the training label is a second dialogue text containing all text contents in the training text, and the second dialogue text contains at least one round of dialogue content; processing the training text by using a preset first large language model to be trained to obtain a third dialogue text; training the first large language model according to the third dialogue text and the training label.

8. The large language model training method of claim 7, wherein, The obtaining of the training label corresponding to the training text comprises: processing the training text by using a preset third large language model to obtain multiple fourth dialogue texts; selecting at least one second dialogue text meeting a preset condition from the multiple fourth dialogue texts, and taking the second dialogue text as the training label corresponding to the training text, wherein the preset condition comprises a fourth condition, and the fourth condition is that the selected second dialogue text contains all text contents in the training text.

9. The large language model training method of claim 8, wherein, The preset condition further comprises a fifth condition corresponding to a first task, and the fifth condition is a type or quantity of language phenomena contained in the selected second dialogue text, and the first task is a task of training a dialogue model by using a first dialogue text as training corpus, and the first dialogue text is a dialogue text obtained by processing to-be-processed text by using the first large language model after training. Or, the preset condition further comprises a sixth condition corresponding to a second task, and the sixth condition is used to limit an intention explicitness degree and / or dialogue completeness degree of the selected second dialogue text, and the second task is a task of using the first dialogue text for retrieval enhancement generation.

10. A text processing device, characterized by, The device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the text processing method according to any one of claims 1 to 6.

Images