Language processing model training method and language processing method

Abstract

Embodiments of the present disclosure provide a language processing model training method and a language processing method. The language processing model training method comprises: determining a plurality of text processing tasks and initial text training data; on the basis of each text processing task among the plurality of text processing tasks, performing data processing on the initial text training data, and obtaining task text training data corresponding to each text processing task; and training an initial language processing model on the basis of the task text training data corresponding to each text processing task, and obtaining a target language processing model, wherein when a target text is received, the target language processing model is used for performing initial text processing on the target text on the basis of an initial text processing task among the plurality of text processing tasks, obtaining an initial text processing result, performing target text processing on the initial text processing result on the basis of a target text processing task among the plurality of text processing tasks, and obtaining a target text processing result. Thus, the accuracy of the target text processing result is improved.

Description

Language processing model training methods, language processing methods

[0001] This disclosure claims priority to Chinese Patent Application No. 202411613928.7, filed with the China Patent Office on November 12, 2024, entitled “Language Processing Model Training Method, Language Processing Method”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of computer technology, and in particular to a language processing model training method and a language processing method. Background Technology

[0003] Utilizing Large Language Models (LLMs) for text-to-SQL (Text2SQL, where SQL stands for Structured Query Language) is a current technical solution for question answering based on large-scale databases and has significant commercial value. Most existing methods rely on general-purpose large language models for context learning to achieve text-to-SQL generation, but this can lead to substantial inference overhead during deployment in real-world scenarios.

[0004] Meanwhile, when migrating to smaller LLMs, the effectiveness of text-to-SQL generation decreases, exhibiting lower versatility and limiting its practical application. Furthermore, existing supervised fine-tuning (SFT) methods for Text2SQL mostly focus on fine-tuning a single task and a single model, which carries the risk of overfitting. Summary of the Invention

[0005] In view of this, embodiments of this disclosure provide a language processing model training method and a language processing method. One or more embodiments of this disclosure also relate to a language processing model training apparatus, a language processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, to solve the technical defects in the prior art where, when using a large language model to generate text to SQL, the general large language model generates a large amount of inference overhead, while using a single task to perform supervised fine-tuning of the large language model leads to inaccurate generated SQL.

[0006] According to a first aspect of the present disclosure, a language processing model training method is provided, comprising:

[0007] Define multiple text processing tasks and initial text training data;

[0008] Based on each of the multiple text processing tasks, the initial text training data is processed to obtain the task text training data corresponding to each text processing task.

[0009] Based on the task text training data corresponding to each text processing task, the initial language processing model is trained to obtain the target language processing model.

[0010] The target language processing model is used to, upon receiving target text, perform initial text processing on the target text according to the initial text processing task among the plurality of text processing tasks to obtain an initial text processing result, and then perform target text processing on the initial text processing result according to the target text processing task among the plurality of text processing tasks to obtain a target text processing result.

[0011] According to a second aspect of the present disclosure, a language processing model training apparatus is provided, comprising:

[0012] The determination module is configured to determine multiple text processing tasks and initial text training data;

[0013] The acquisition module is configured to perform data processing on the initial text training data according to each text processing task in the plurality of text processing tasks, and obtain the task text training data corresponding to each text processing task.

[0014] The training module is configured to train an initial language processing model based on the task text training data corresponding to each text processing task to obtain a target language processing model. The target language processing model is used to perform initial text processing on the target text according to the initial text processing task among the multiple text processing tasks when receiving target text, to obtain an initial text processing result, and to perform target text processing on the initial text processing result according to the target text processing task among the multiple text processing tasks to obtain a target text processing result.

[0015] According to a third aspect of the present disclosure, a language processing method is provided, comprising:

[0016] Identify the target text;

[0017] Based on the initial text processing task of the target language processing model, the target text is subjected to initial text processing to obtain the initial text processing result;

[0018] Based on the target text processing task of the target language processing model, the initial text processing result is subjected to target text processing to obtain the target text processing result.

[0019] According to a fourth aspect of the present disclosure, a language processing apparatus is provided, comprising:

[0020] The text determination module is configured to determine the target text;

[0021] The initial acquisition module is configured to perform initial text processing on the target text according to the initial text processing task of the target language processing model, and obtain the initial text processing result;

[0022] The target acquisition module is configured to perform target text processing on the initial text processing result according to the target text processing task of the target language processing model, and obtain the target text processing result.

[0023] According to a fifth aspect of the present disclosure, a computing device is provided, comprising:

[0024] Memory and processor;

[0025] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, they implement the steps of the above-mentioned language processing model training method and language processing method.

[0026] According to a sixth aspect of the present disclosure, a computer-readable storage medium is provided that stores a computer program / instructions, which, when executed by a processor, implement the steps of the above-described language processing model training method and language processing method.

[0027] According to a seventh aspect of the present disclosure, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described language processing model training method and language processing method.

[0028] This disclosure provides a language processing model training method according to one embodiment. It identifies multiple text processing tasks and initial text training data. Since each text processing task has a different task, the initial text training data is processed accordingly using each of the multiple text processing tasks to obtain task-specific text training data that meets the requirements of each task. The initial language processing model is then trained using this task-specific text training data, enabling the trained target language processing model to perform multiple text processing tasks. Therefore, this target language processing model can be used to process target text collaboratively using multiple text processing tasks upon receiving target text. Specifically… By using the initial text processing task from multiple text processing tasks, the initial text processing result corresponding to the target text is obtained. Then, by using the target text processing task from multiple text processing tasks, the obtained initial text processing result can be processed to obtain the accurate target text processing result corresponding to the target text. That is, by using the task text training data of multiple text processing tasks to train the initial language processing model, multi-task training is achieved on the initial language processing model, reducing the risk of overfitting and enabling the trained target language processing model to have the ability to perform multiple text processing tasks. When using the target language processing model to process the target text, the accuracy of the target text processing result can be improved through the collaboration of multiple text processing tasks. Attached Figure Description

[0029] Figure 1 is a schematic diagram of a scenario of a language processing method provided in an embodiment of this disclosure;

[0030] Figure 2 is a flowchart of a language processing model training method provided in an embodiment of this disclosure;

[0031] Figure 3 is a flowchart of a language processing method provided in an embodiment of this disclosure;

[0032] Figure 4 is a flowchart of the processing procedure of a language processing model training method provided in an embodiment of this disclosure;

[0033] Figure 5 is a flowchart of a language processing method provided in an embodiment of this disclosure;

[0034] Figure 6 is a schematic diagram of the structure of a language processing model training device provided in an embodiment of this disclosure;

[0035] Figure 7 is a schematic diagram of the structure of a language processing device provided in an embodiment of this disclosure;

[0036] Figure 8 is a structural block diagram of a computing device provided in an embodiment of this disclosure. Detailed Implementation

[0037] Numerous specific details are set forth in the following description to provide a full understanding of this disclosure. However, this disclosure can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this disclosure. Therefore, this disclosure is not limited to the specific implementations disclosed below.

[0038] The terminology used in one or more embodiments of this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this disclosure. The singular forms “a,” “the,” and “the” as used in one or more embodiments of this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this disclosure refers to and includes any or all possible combinations of one or more associated listed items.

[0039] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this disclosure, and similarly, second may also be referred to as first. Depending on the context, the word “if” as used herein may be interpreted as “when”, “in response to a determination”, or “when…”.

[0040] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this disclosure are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0041] First, the terms and concepts involved in one or more embodiments of this disclosure will be explained.

[0042] LLM: Large Language Model. A type of language model, represented by Transformer, has a very large number of parameters and powerful natural language generation capabilities.

[0043] Text2SQL: Text-to-SQL (Structured Query Language) generation, which means generating SQL queries using a large language model given a question and relevant database information.

[0044] SFT: Supervised Fine-Tuning refers to the process of further training / fine-tuning a pre-trained model to adapt it to a specific task or dataset.

[0045] In-Context Learning: This paradigm allows models to learn and reason using contextual information without explicit training. Unlike traditional supervised learning, in-context learning does not require a large amount of labeled data, but instead relies on examples and cues in the context to guide the model's reasoning process.

[0046] Schema linking: This involves identifying and extracting instances in the database that are relevant to the problem, such as tables and columns.

[0047] This disclosure provides a language processing model training method, a language processing method, a language processing model training device, a language processing device, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0048] Referring to Figure 1, Figure 1 shows a scenario diagram of a language processing method provided according to an embodiment of the present disclosure.

[0049] Specifically, this language processing method is implemented using a terminal device 102 and a server 104. The terminal device 102 is used to send target text to the server 104, such as the target text being "query the best-selling products so far". In practical applications, users can input target text into the terminal device 102 via text or voice. If voice is used, the terminal device 102 will also include a corresponding voice processing component, such as a voice parsing, voice-to-text, and voice synthesis module, to convert the user's voice into text. This disclosure does not impose any limitations on this.

[0050] A target language processing model is trained in server 104. The target language processing model is trained in the following way: multiple text processing tasks and initial text training data are determined; according to each of the multiple text processing tasks, the initial text training data is processed to obtain task text training data corresponding to each text processing task; according to the task text training data corresponding to each text processing task, the initial language processing model is trained to obtain the target language processing model.

[0051] When server 104 receives the target text sent by end device 102, the target language processing model performs initial text processing on the target text according to the initial text processing task among multiple text processing tasks, obtains the initial text processing result, and performs target text processing on the initial text processing result according to the target text processing task among multiple text processing tasks, obtains the target text processing result, and can obtain the corresponding text execution result based on the target text processing result, such as the text execution result being "Product A, Product B". In practical applications, taking the text-to-SQL scenario as an example, the target text can be natural language text, and the target text processing result is the SQL query statement corresponding to the target text. Therefore, by executing the SQL query statement corresponding to the target text, the target text processing result can be obtained to obtain the corresponding text execution result, and the text execution result is returned to end device 102.

[0052] The edge device 102 may include a browser, an app (application), or a web application such as an H5 (Hypertext Markup Language 5) application, a lightweight application (also known as a mini-program), or a cloud application. The edge device may be developed based on a software development kit (SDK) provided by the server, such as a real-time communication (RTC) SDK. The edge device can be deployed in an electronic device and depends on the device's operation or certain apps within the device to run. The electronic device may have a display screen and support information browsing, such as a personal mobile terminal like a mobile phone, tablet, or personal computer. Various other types of applications can also be configured in the electronic device, such as human-computer interaction applications, model training applications, language processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, and social media platform software.

[0053] Server 104 can be understood as a server providing various services, including physical servers and cloud servers. Examples include servers providing communication services to multiple clients, servers supporting backend training of models used on clients, and servers processing data sent by clients. It's important to note that Server 104 can be implemented as a distributed server cluster composed of multiple servers, or as a single server. Server 104 can also be a server in a distributed system, or a server integrated with blockchain. Server 104 can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0054] It is worth noting that the language processing method provided in this embodiment can be executed by the server 104. In other embodiments of this disclosure, the target language processing model can be deployed on the edge device 102, so that the edge device 102 can also have similar functions to the server 104, thereby executing the language processing method provided in this embodiment. In other embodiments, the language processing method provided in this embodiment can also be jointly executed by the edge device 102 and the server 104.

[0055] The language processing method provided in one embodiment of this disclosure, in which the target language processing model is obtained by training an initial language processing model with training data of task texts of multiple text processing tasks, enables the target language processing model to perform multiple text processing tasks. When the target language processing model is used to process the target text, the accuracy of the target text processing result can be improved through the collaboration of multiple text processing tasks.

[0056] Referring to Figure 2, which shows a flowchart of a language processing model training method provided in an embodiment of the present disclosure, the method includes the following steps.

[0057] Step 202: Identify multiple text processing tasks and initial text training data.

[0058] Text processing tasks can be understood as tasks that manipulate, analyze, and transform text data. For example, text processing tasks may include text generation tasks (tasks that use text data to generate data of a target language or target format), data filtering tasks (tasks that use text data to search for and filter data that meets specific conditions from databases, knowledge bases, or other data sources), noise correction tasks (tasks that correct text processing results that do not meet requirements or are incorrect), and data continuation tasks (tasks that continue writing relatively simple text processing results corresponding to text data to increase complexity and improve accuracy).

[0059] Initial text training data can be understood as data obtained from different data sources for training models to learn specific tasks or functions. In this embodiment of the disclosure, taking the text-to-SQL scenario as an example, when the initial text training data is used to train the model so that the model can generate SQL from text, the initial text training data may include text samples (natural language text), SQL samples corresponding to the text samples, and database samples.

[0060] The initial language processing model is identified as having multiple text processing tasks to learn (where the initial language processing model can be understood as a pre-trained machine learning model with text generation capabilities; to adapt the initial language processing model to specific tasks, it is fine-tuned using training data from specific tasks), and initial text training data is used to train the initial language processing model. Task text training data for each text processing task is constructed using the initial text training data, and the initial language processing model is trained using the task text training data for each text processing task to obtain a target language processing model capable of performing multiple text processing tasks.

[0061] Step 204: Based on each of the multiple text processing tasks, process the initial text training data to obtain the task text training data corresponding to each text processing task.

[0062] The task text training data is used to enable the model to learn the corresponding specific task.

[0063] Specifically, by processing the initial text training data, task text training data corresponding to each text processing task is constructed, and the initial language processing model is trained using the task text training data corresponding to the text processing tasks.

[0064] In one or more embodiments of this disclosure, noise filtering is performed on the initial text training data to avoid using noisy initial text training data to construct task text training data for each text processing task, which would lead to poor training results. Specific implementation methods are as follows:

[0065] The step of processing the initial text training data according to each of the plurality of text processing tasks to obtain task text training data corresponding to each text processing task includes:

[0066] The initial text training data is filtered for noise using a discriminator to obtain filtered text training data, wherein the initial text training data includes initial text samples, target language data samples corresponding to the initial text samples, and data source samples.

[0067] Based on each of the multiple text processing tasks, the filtered text training data is processed to obtain the task text training data corresponding to each text processing task.

[0068] The discriminator can be understood as a classification model. For example, the discriminator is a classification model implemented using classification algorithms such as logistic regression, Naive Bayes, decision trees, support vector machines, random forests, and gradient boosting trees. Classification models can perform not only binary classification but also multi-class classification. In this embodiment, the discriminator is used to perform binary classification on the initial text training data, dividing the initial text training data into positive examples (positive samples) or negative examples (negative samples). Noise filtering is achieved by filtering out the initial text training data containing negative examples. Filtered text training data can be understood as the initial text training data containing positive examples; the filtered text training data does not contain noise, meaning that all data in the filtered text training data is matched and correct.

[0069] In the context of text-to-SQL generation, the initial text sample in the initial text training data can be understood as a natural language text sample. This initial text sample is a query text sample, meaning it contains the intent to retrieve query information from the database. For example, the initial text sample might be "Find employees older than 30". The target language data sample corresponding to the initial text sample can be understood as the SQL query sample corresponding to the natural language text sample. For example, continuing the previous example, the target language data sample corresponding to this initial text sample could be "SELECT * FROM Employees WHERE Age>30". The data source sample can be understood as a database sample containing the database information needed when generating the SQL query sample from the natural language text sample. The database sample provides detailed information about the database, including but not limited to tables, columns, and the values ​​of specific fields. For example, in the previous example, the database sample provides information such as employee names and ages, and the data source sample could be an employee information database.

[0070] Specifically, the initial text training data is input into the discriminator, which can classify the initial text training data, such as dividing it into positive examples or negative examples. The initial text training data with a positive classification result is determined as the filtered text training data, while the initial text training data with a negative classification result indicates that the initial text training data contains noise, i.e., noisy data. Therefore, by filtering out the noise data, the filtered text training data after noise filtering of the initial text training data is obtained, realizing the identification and removal of noise data in the initial text training data.

[0071] Based on the noise-filtered text training data, task text training data corresponding to each text processing task is constructed according to each text processing task. This ensures that the task text training data corresponding to each text processing task does not contain noisy data, thus avoiding affecting the initial language processing model's learning of correct SQL knowledge when using the task text training data corresponding to each text processing task to train the initial language processing model in the future.

[0072] The speech processing model training method provided in this embodiment uses a discriminator to filter noise data in the initial text training data to obtain purified filtered text training data. This enables the initial language processing model to be trained using the correct task text training data when constructing task text training data for each text processing task using the filtered text training data, thus allowing the initial language processing model to learn the correct knowledge for each text processing task.

[0073] In one or more embodiments of this disclosure, the discriminator can be a classification model. By inputting the initial text training data into the discriminator, the classification result corresponding to the initial text training data can be quickly obtained. Based on the classification result, it can be determined whether the input initial text training data is noisy data, thus quickly achieving noise filtering. Specific implementation methods are as follows:

[0074] The step of using a discriminator to filter noise from the initial text training data to obtain filtered text training data includes:

[0075] The initial text training data is input into the discriminator to obtain the classification result of the initial text training data, wherein the discriminator is trained using labeled text training data of positive and negative samples;

[0076] The initial text training data whose classification result is a positive sample is determined as the filtered text training data.

[0077] Specifically, labeled text training data for positive samples and labeled text training data for negative samples are obtained. The labeled text training data for positive samples includes initial text training data samples and positive labels indicating that the initial text training data samples are positive samples. The labeled text training data for negative samples includes initial text training data samples and negative labels indicating that the initial text training data samples are negative samples.

[0078] A discriminator is trained using labeled text training data of positive and negative samples. Specifically, the initial text training data samples from the labeled text training data of positive (negative) samples are input into the initial discriminator. The initial discriminator is used to obtain the corresponding predicted classification result. A loss function is used to calculate the difference between the predicted classification result and the positive (negative) label. The loss function is minimized through an optimization algorithm to train the initial discriminator and obtain the discriminator. This enables the discriminator to distinguish whether the initial text training data is a positive sample (i.e., a positive example in the above embodiment) or a negative sample (i.e., a negative example in the above embodiment).

[0079] When the initial text training data is input into the discriminator, the discriminator outputs the classification result of the initial text training data. Based on the classification result, it is determined whether the initial text training data is noisy data. That is, if the classification result is a negative sample, the initial text training data is noisy data; if the classification result is a positive sample, the initial text training data is determined to be filtered text training data.

[0080] The language processing model training method provided in this disclosure obtains a discriminator through training. When the initial text training data is input into the discriminator, the classification result corresponding to the initial text training data can be quickly obtained. Based on the classification result, the noisy data in the initial text training data can be filtered, thereby obtaining the filtered text training data efficiently and accurately.

[0081] Step 206: Train the initial language processing model based on the task text training data corresponding to each text processing task to obtain the target language processing model. The target language processing model is used to perform initial text processing on the target text according to the initial text processing task among the multiple text processing tasks when receiving target text, to obtain the initial text processing result, and to perform target text processing on the initial text processing result according to the target text processing task among the multiple text processing tasks to obtain the target text processing result.

[0082] Specifically, when the initial language processing model is trained using training data from task texts corresponding to different text processing tasks, the resulting target language processing models will have different capabilities.

[0083] In one or more embodiments of this disclosure, multiple text processing tasks include an initial text processing task and a target text processing task. Initial task text training data corresponding to the initial text processing task and target task text training data corresponding to the target text processing task can be constructed using filtered text training data, respectively. Specific implementation methods are described below:

[0084] The multiple text processing tasks include initial text processing tasks and target text processing tasks;

[0085] The step of processing the initial text training data according to each of the plurality of text processing tasks to obtain task text training data corresponding to each text processing task includes:

[0086] The initial text training data is subjected to noise filtering to obtain filtered text training data;

[0087] Based on the initial text processing task, the filtered text training data is processed to obtain the initial task text training data corresponding to the initial text processing task.

[0088] Based on the target text processing task, the filtered text training data is processed to obtain the target task text training data corresponding to the target text processing task.

[0089] The initial text processing task can be understood as a task that helps the model obtain corresponding text processing results based on the text. For example, the initial text processing task may include the text generation task and the data filtering task in the above embodiments. The text generation task can directly help the initial language processing model generate corresponding text processing results using text data, while the data filtering task can help the initial language processing model efficiently generate text processing results using the filtered data by searching and filtering data that meets specific conditions when the initial language processing model generates corresponding text processing results using text data.

[0090] The target text processing task can be understood as a task that enables the model to adjust and optimize the obtained text processing results. For example, the target text processing task may include the noise correction task and the data continuation task in the above embodiments. The noise correction task enables the initial language processing model to correct the initial text processing results that do not meet the requirements or expectations, thereby optimizing the initial text processing results. The data continuation task, on the other hand, enables the initial language processing model to continue writing the more complex results after obtaining the initial text processing results or the results after correction of the initial text processing results, in order to avoid the problems of incomplete or inaccurate final results.

[0091] In practical applications, the step of training the initial language processing model based on the task text training data corresponding to each text processing task to obtain the target language processing model includes:

[0092] The initial language processing model is trained based on the initial task text training data corresponding to the initial text processing task and the target task text training data corresponding to the target text processing task to obtain the target language processing model.

[0093] Specifically, noise filtering is performed on the initial text training data, and then the filtered text training data is processed to construct task text training data for each text processing task. When constructing the initial task text training data corresponding to the initial text processing task, the filtered text training data is processed to enable the initial language processing model to efficiently and accurately generate text processing results based on the constructed initial task text training data. When constructing the target task text training data corresponding to the target text processing task, the filtered text training data is processed to enable the initial language processing model to improve the accuracy of the text processing results based on the constructed target task text training data. In other words, the target language processing model obtained by training the initial language processing model using the initial task text training data and the target task text training data has the ability to efficiently and accurately generate text processing results corresponding to the text data, as well as to correct and optimize the generated text processing results.

[0094] In one or more embodiments of this disclosure, the plurality of text processing tasks include text generation tasks and data filtering tasks; by reasonably constructing the task text training data corresponding to the text generation task, the target language processing model is enabled to generate SQL from text; by reasonably constructing the task text training data corresponding to the data filtering task, the target language processing model is enabled to simplify data source information. Specific implementation methods are as follows:

[0095] The step of processing the filtered text training data according to each of the plurality of text processing tasks to obtain task text training data corresponding to each text processing task includes:

[0096] According to the text generation task, example information is extracted from the data source samples in the filtered text training data to obtain the example information of the data source samples. The filtered text training data is obtained by noise filtering of the initial text training data. The filtered text training data includes the initial text sample, the target language data sample corresponding to the initial text sample, and the data source sample.

[0097] Based on the initial text sample, the target language data sample corresponding to the initial text sample, and the example information, the first target text training data corresponding to the text generation task is obtained;

[0098] According to the data filtering task, the data source sample information of the target language data sample in the filtered text training data is extracted to obtain the associated data source sample information corresponding to the initial text sample.

[0099] Based on the initial text sample, the associated data source sample information, and the example information, the second target text training data corresponding to the data filtering task is obtained.

[0100] In the context of generating SQL from text, the data source sample can be understood as a database sample. In this case, the example information includes the table information and field value information contained in the database sample.

[0101] Specifically, taking the text-to-SQL scenario as an example, we extract table information and field value information from the database sample. Using this extracted information, we determine the corresponding example information for the database sample. This example information is used to provide contextual information in the task's text training data.

[0102] For the text generation task, the first target text training data corresponding to the text generation task is constructed by using the initial text sample, the target language data sample corresponding to the initial text sample, and the example information.

[0103] Taking the SQL text generation scenario as an example, we will explain the language processing model training method in detail. In practical applications, we extract information from each table in the database sample and determine some example rows (example rows refer to some value examples of the database table, and each row includes the values ​​of all fields in the table). Combined with user question samples (i.e., the initial text samples in the filtered text training data) and the correct SQL query samples corresponding to the user question samples (the target language data samples corresponding to the initial text samples), we construct the first target text training data for the text generation task. This first target text training data is used to enable the initial language processing model to learn the text generation task.

[0104] For the data filtering task, table information and field value information of the database samples are extracted. Based on the extracted table and field value information, the corresponding example information is determined. Combined with the user question samples, the correct SQL query sample corresponding to the user question sample is determined. The table and column information of the database samples involved in the SQL query sample (i.e., the related database sample information) are extracted from the SQL query sample. Using the example information, the user question samples, and the table and column information of the corresponding correct SQL query samples, a second target text training data is constructed for the data filtering task. This second target text training data is used to enable the initial language processing model to learn the data filtering task.

[0105] The speech processing model training method provided in this disclosure processes the initial text training data through different text processing tasks, and can reasonably obtain the task text training data corresponding to each text processing task. When the target language processing model is trained by the first target text training data corresponding to the text generation task, the target language processing model can have the ability to generate SQL from text. When the target language processing model is trained by the second target text training data corresponding to the data filtering task, the target language processing model can have the ability to filter out relevant database information involved in the current input text from the database, thereby simplifying the database and improving inference efficiency.

[0106] In one or more embodiments described herein, the initial language processing model is a pre-trained model. Supervised fine-tuning is performed on the initial language processing model using training data corresponding to each text processing task. Specifically, for the first target text training data of the text generation task and the second target text training data of the data filtering task, prompt text and response tags are determined, and the initial language processing model is then fine-tuned using the prompt text and response tags. Specific implementation methods are as follows:

[0107] The step of training the initial language processing model based on the task text training data corresponding to each text processing task to obtain the target language processing model includes:

[0108] The initial text sample and the example information in the first target text training data are determined as the first prompt text, and the target language data sample in the first target text training data is determined as the first response tag;

[0109] The initial text sample, the example information, and the target request in the second target text training data are determined as the second prompt text, and the associated data source sample information in the second target text training data are determined as the second response tag;

[0110] The initial language processing model is trained based on the first prompt text, the first reply tag, the second prompt text, and the second reply tag to obtain the target language processing model.

[0111] Specifically, for the first target text training data of the text generation task, table information and example rows are extracted from the database (i.e., the extracted table information and example rows can be used as example information). The first prompt text is constructed by combining the user question sample (i.e. the initial text sample), and the correct SQL query sample corresponding to the user question sample (i.e. the target language data sample) is determined as the first response label.

[0112] For the training data of the second target text in the data filtering task, table information and example rows are extracted from the database. The second prompt text is constructed by combining the user question sample and the target request (which can be understood as a request to extract related data information). The table information and column information of the correct SQL query sample corresponding to the user question sample are determined as the second reply label.

[0113] The first prompt text is input into the initial language processing model to obtain the first predicted response. The initial language processing model is then tuned using the loss function of the first predicted response and the first response label. The second prompt text is input into the initial language processing model to obtain the second predicted response. The initial language processing model is then tuned using the loss function of the second predicted response and the second response label to obtain the target language processing model.

[0114] The speech processing model training method provided in this embodiment determines a specific first prompt text and a first reply label for the first target text training data of the text generation task, and determines a specific second prompt text and a second reply label for the second target text training data of the data filtering task. In this way, the initial language processing model is trained accurately and reasonably based on the first prompt text and the first reply label, the second prompt text and the second reply label, to obtain a target language processing model with text generation and data filtering capabilities.

[0115] In one or more embodiments of this disclosure, the plurality of text processing tasks further includes a target text processing task, which includes a noise correction task. The noise correction task processes the filtered text training data to obtain incorrectly matched initial text samples and replacement language data samples. Based on the incorrectly matched initial text samples, the replacement language data samples, the target language data samples that correctly match the initial text samples, and example information, a third target text training data for the noise correction task is obtained. Specific implementation methods are as follows:

[0116] The step of processing the filtered text training data according to each of the plurality of text processing tasks to obtain task text training data corresponding to each text processing task includes:

[0117] According to the noise correction task, the target language data samples in the filtered text training data are replaced to obtain the replaced language data samples.

[0118] Based on the initial text sample, the replacement language data sample, the target language data sample, and the example information, the third target text training data corresponding to the noise correction task is obtained.

[0119] Taking the SQL text generation scenario as an example, we will explain the language processing model training method in detail. Specifically, in the case where the noise correction task is used to correct erroneous or unacceptable results, we can replace the correct target language data sample with the erroneous or unacceptable replacement language data sample.

[0120] In practice, when the target language data sample is an SQL statement sample, some common SQL errors can be introduced into the SQL statement sample, such as schema linking errors, JOIN (an operation used to merge rows from two or more tables) errors, nesting errors, symbol errors, etc., thereby replacing the target language data sample with an incorrect alternative language data sample.

[0121] Using the example information, initial text samples, target language data samples that correctly match the initial text samples, and replacement language data samples that incorrectly match the initial text samples, the third target text training data corresponding to the noise correction task is constructed.

[0122] In practical applications, when multiple text processing tasks are involved, including text generation, data filtering, and noise correction, the initial language processing model is trained using the first target text training data corresponding to the text generation task, the second target text training data corresponding to the data filtering task, and the third target text training data corresponding to the noise correction task, to obtain the target language processing model.

[0123] The step of training the initial language processing model based on the task text training data corresponding to each text processing task to obtain the target language processing model includes:

[0124] The initial language processing model is trained based on the first target text training data corresponding to the text generation task, the second target text training data corresponding to the data filtering task, and the third target text training data corresponding to the noise correction task to obtain the target language processing model.

[0125] According to the above embodiments, when training the initial language processing model using task text training data corresponding to the text processing task, the prompt text and response tags in the target text training data are determined, thereby accurately training the initial language processing model based on the prompt text and response tags in the target text training data. Specific implementation methods are as follows:

[0126] The step of training the initial language processing model based on the task text training data corresponding to each text processing task to obtain the target language processing model includes:

[0127] The initial text sample and the example information in the first target text training data are determined as the first prompt text, and the target language data sample in the first target text training data is determined as the first response tag;

[0128] The initial text sample, the example information, and the target request in the second target text training data are determined as the second prompt text, and the associated data source sample information in the second target text training data are determined as the second response tag;

[0129] The initial text sample, the replacement language data sample, and the example information in the third target text training data are determined as the third prompt text, and the target language data sample in the third target text training data is determined as the third response tag;

[0130] The initial language processing model is trained based on the first prompt text, the first reply tag, the second prompt text, the second reply tag, the third prompt text, and the third reply tag to obtain the target language processing model.

[0131] Specifically, for the first target text training data corresponding to the text generation task, the first prompt text and the first reply label are determined. For the second target text training data corresponding to the data filtering task, the second prompt text and the second reply label are determined. The implementation can be found in the above embodiments, and will not be repeated here.

[0132] Given the training data of the third target text corresponding to the noise correction task, the initial text sample, the replacement language data sample, and the example information in the training data of the third target text are determined as the third prompt text, and the target language data sample is determined as the third response label. Thus, the initial language processing model is trained based on the third prompt text and the third response label. The third prompt text is input into the initial language processing model to obtain the third predicted response corresponding to the third prompt text. The parameters of the initial language processing model are tuned based on the loss function of the third predicted response and the third response label.

[0133] That is, based on the first prompt text and first reply label corresponding to the text generation task, the second prompt text and second reply label corresponding to the data filtering task, and the third prompt text and third reply label corresponding to the noise correction task, the initial language processing model is trained to obtain a target language processing model with text generation, data filtering and noise correction capabilities.

[0134] The speech processing model training method provided in this embodiment includes a noise correction task in the text processing task. By constructing third target training data corresponding to the noise correction task, the initial language processing model is trained. Based on text generation and data filtering, a target language processing model with noise correction is obtained so that when the target language processing model is applied to process the input query text, the obtained erroneous results can be corrected, thereby improving the accuracy of the text processing results.

[0135] In one or more embodiments of this disclosure, the plurality of text processing tasks further include a target text processing task, which includes a data continuation task. The data continuation task processes the filtered text training data to obtain incomplete truncated language data samples corresponding to the target language data samples. This allows the generation of fourth target text training data for the data continuation task based on the initial text sample, the target language data sample, the truncated language data samples corresponding to the target text samples, and example information. Specific implementation methods are as follows:

[0136] The step of processing the initial text training data according to each of the plurality of text processing tasks to obtain task text training data corresponding to each text processing task includes:

[0137] According to the data continuation task, the target language data samples in the filtered text training data are truncated to obtain truncated language data.

[0138] Based on the initial text sample, the truncated language data, the target language data sample, and the example information, the fourth target text training data corresponding to the data continuation task is obtained.

[0139] Taking the SQL text generation scenario as an example, we will explain the language processing model training method in detail. Specifically, in the case of a data continuation task used to continue writing the generated results, the correct target language data sample can be truncated to obtain an incomplete truncated language data sample.

[0140] Specifically, when the target language data sample is an SQL statement sample, it is possible to truncate the SQL statement sample at random positions, thereby obtaining an incomplete truncated language data sample based on the target language data sample in the filtered text training data.

[0141] Using the example information, initial text samples, target language data samples that correctly match the initial text samples, and truncated language data samples that correspond to the target language data samples, the fourth target text training data corresponding to the data continuation task is constructed.

[0142] In practical applications, when multiple text processing tasks are involved, including text generation, data filtering, and data continuation tasks, the initial language processing model is trained using the first target text training data corresponding to the text generation task, the second target text training data corresponding to the data filtering task, and the fourth target text training data corresponding to the data continuation task, to obtain the target language processing model.

[0143] The step of training the initial language processing model based on the task text training data corresponding to each text processing task to obtain the target language processing model includes:

[0144] The initial language processing model is trained based on the first target text training data corresponding to the text generation task, the second target text training data corresponding to the data filtering task, and the fourth target text training data corresponding to the data continuation task to obtain the target language processing model.

[0145] Similarly, according to the above embodiments, when training the initial language processing model using task text training data corresponding to the text processing task, the prompt text and response tags in the task text training data are determined, thereby accurately training the initial language processing model based on the prompt text and response tags in the task text training data. Specific implementation methods are as follows:

[0146] The step of training the initial language processing model based on the task text training data corresponding to each text processing task to obtain the target language processing model includes:

[0147] The initial text sample and the example information in the first target text training data are determined as the first prompt text, and the target language data sample in the first target text training data is determined as the first response tag;

[0148] The initial text sample, the example information, and the target request in the second target text training data are determined as the second prompt text, and the associated data source sample information in the second target text training data are determined as the second response tag;

[0149] The initial text sample, the truncated language data, and the example information in the fourth target text training data are determined as the fourth prompt text, and the target language data sample in the fourth target text training data is determined as the fourth response tag;

[0150] The initial language processing model is trained based on the first prompt text, the first reply tag, the second prompt text, the second reply tag, the fourth prompt text, and the fourth reply tag to obtain the target language processing model.

[0151] Following the example above, for the first target text training data corresponding to the text generation task, the first prompt text and the first reply label are determined. For the second target text training data corresponding to the data filtering task, the second prompt text and the second reply label are determined. The implementation can be found in the above embodiment, and will not be repeated here.

[0152] Given the training data for the fourth target text corresponding to the data continuation task, the initial text sample, truncated language data sample, and example information in the fourth target text training data are determined as the fourth prompt text, and the target language data sample is determined as the fourth response label. Thus, the initial language processing model is trained based on the fourth prompt text and the fourth response label. The fourth prompt text is then input into the initial language processing model to obtain the fourth predicted response corresponding to the fourth prompt text. The parameters of the initial language processing model are tuned based on the loss function of the fourth predicted response and the fourth response label.

[0153] That is, based on the first prompt text and first reply label corresponding to the text generation task, the second prompt text and second reply label corresponding to the data filtering task, and the fourth prompt text and fourth reply label corresponding to the data continuation task, the initial language processing model is trained to obtain a target language processing model with text generation, data filtering and data continuation capabilities.

[0154] The speech processing model training method provided in this embodiment includes a data continuation task in the text processing task. By constructing a fourth target training data corresponding to the data continuation task, the initial language processing model is trained. Based on text generation and data filtering, a target language processing model with data continuation capability is obtained. This enables the continuation of the obtained results to obtain refined text processing results when the target language processing model is subsequently applied to process the input query text, thereby improving the accuracy of the text processing results.

[0155] In one or more embodiments of this disclosure, the plurality of text processing tasks includes a target text processing task, which includes a noise correction task and a data continuation task.

[0156] The step of processing the initial text training data according to each of the plurality of text processing tasks to obtain task text training data corresponding to each text processing task includes:

[0157] Example information is extracted from the data source samples in the filtered text training data to obtain the example information of the data source samples. The filtered text training data is obtained by noise filtering the initial text training data. The filtered text training data includes the initial text samples, the target language data samples corresponding to the initial text samples, and the data source samples.

[0158] According to the noise correction task, the target language data samples in the filtered text training data are replaced to obtain the replaced language data samples.

[0159] Based on the initial text sample, the replacement language data sample, the target language data sample, and the example information, obtain the third target text training data corresponding to the noise correction task;

[0160] According to the data continuation task, the target language data samples in the filtered text training data are truncated to obtain truncated language data.

[0161] Based on the initial text sample, the truncated language data, the target language data sample, and the example information, the fourth target text training data corresponding to the data continuation task is obtained.

[0162] Specifically, based on the initial language processing model being a pre-trained model with a certain text generation capability, multiple text processing models can include a target text processing task. The generated text processing results are corrected and optimized using the task text training data corresponding to the target text processing task. For specific implementation details, please refer to the above embodiments, which will not be repeated here.

[0163] The step of training the initial language processing model based on the task text training data corresponding to each text processing task to obtain the target language processing model includes:

[0164] The initial language processing model is trained based on the third target text training data corresponding to the noise correction task and the fourth target text training data corresponding to the data continuation task to obtain the target language processing model.

[0165] For specific implementation details, please refer to the above embodiments, which will not be repeated here.

[0166] In one or more embodiments of this disclosure, the plurality of text processing tasks further includes a target text processing task, which includes a noise correction task and a data continuation task. In this case, the text processing tasks include a text generation task, a data filtering task, a noise correction task, and a data continuation task. An initial language processing model is trained using the task text training data from the above four text processing tasks to obtain a target language processing model capable of text generation, data filtering, noise correction, and data continuation. Specific implementation methods are as follows:

[0167] The step of processing the filtered text training data according to each of the plurality of text processing tasks to obtain task text training data corresponding to each text processing task includes:

[0168] According to the noise correction task, the target language data samples in the filtered text training data are replaced to obtain the replaced language data samples.

[0169] Based on the initial text sample, the replacement language data sample, the target language data sample, and the example information, obtain the third target text training data corresponding to the noise correction task;

[0170] According to the data continuation task, the target language data samples in the filtered text training data are truncated to obtain truncated language data.

[0171] Based on the initial text sample, the truncated language data, the target language data sample, and the example information, the fourth target text training data corresponding to the data continuation task is obtained.

[0172] The step of training the initial language processing model based on the task text training data corresponding to each text processing task to obtain the target language processing model includes:

[0173] The initial language processing model is trained using the first target text training data corresponding to the text generation task, the second target text training data corresponding to the data filtering task, the third target text training data corresponding to the noise correction task, and the fourth target text training data corresponding to the data continuation task, to obtain the target language processing model.

[0174] The specific process of constructing the first target text training data, the second target text training data, the third target text training data, and the fourth target text training data can be found in the above embodiments. The specific implementation of training the initial language processing model using the first target text training data, the second target text training data, the third target text training data, and the fourth target text training data can also be found in the above embodiments, and will not be repeated here.

[0175] The speech processing model training method provided in this disclosure constructs task text training data corresponding to multiple text processing tasks using initial text training data. By using different target text training data corresponding to different text processing tasks to perform supervised fine-tuning of the initial language processing model, the trained target language processing model acquires knowledge related to multiple text processing tasks. This avoids the inference overhead and deployment cost caused by using multiple different language processing models to implement different tasks.

[0176] Referring to Figure 3, Figure 3 shows a flowchart of a language processing method provided in an embodiment of the present disclosure, which specifically includes the following steps.

[0177] Step 302: Determine the target text.

[0178] The target text can be understood as natural language text or text of a target format and type in a professional field, which is determined by the client in response to user input, and there are no restrictions on this.

[0179] Specifically, in response to user input, the client sends the target text to the server for processing by the target language processing model deployed on the server.

[0180] Step 304: Perform initial text processing on the target text according to the initial text processing task of the target language processing model to obtain the initial text processing result.

[0181] The target language processing model is obtained by training the language processing model training method described in the above embodiments.

[0182] Specifically, depending on the text processing task, the processing performed on the target text varies. In one or more embodiments of this disclosure, the target text can be processed through multiple text processing tasks within the target language processing model. When these multiple text processing tasks include text generation tasks and data filtering tasks, the data filtering task can simplify the data source information to reduce redundant information. Text generation is then performed using the simplified obtained associated data source information to obtain the initial text processing result corresponding to the target text. Specific implementation methods are described below:

[0183] The step of performing initial text processing on the target text according to the initial text processing task of the target language processing model to obtain the initial text processing result includes:

[0184] According to the data filtering task of the target language processing model, the associated data source information of the target text is extracted from the target data source corresponding to the target language processing model;

[0185] Based on the text generation task of the target language processing model, the target text is predicted based on the associated data source information to obtain the initial text processing result.

[0186] Specifically, the target language processing model has associated target data sources. When the target language processing model receives target text, it retrieves the associated information or detailed information corresponding to the target text from the target data sources, i.e., associated data source information.

[0187] In practice, the data filtering task of the target language processing model can extract the related data source information of the target text from the target data source. That is, in the scenario of text to SQL generation, the target text can be the query text and the target data source can be the target database. The tables and columns related to the query text can be extracted from the target database, thereby simplifying the target database and reducing redundant information.

[0188] Using the previous example, with the target text being natural language text and the text processing result being an SQL statement, we will explain the language processing method in detail.

[0189] In practical applications, to avoid errors or omissions in the extracted relational database information and to improve the comprehensiveness of the relational database information, when the target language processing model receives the query text, it obtains the pseudo-text generation result (pseudo-SQL, a SQL directly generated by the target language processing model using the unsimplified target database and the query text) based on the basic prediction capabilities of the target language processing model. Based on the pseudo-text generation result, the first relational database information involved in the pseudo-text generation result is extracted from the target database corresponding to the target language processing model; and according to the data filtering task of the target language processing model, the second relational database information related to the target text is directly extracted from the target database; the relational database information is obtained by merging the first and second relational database information.

[0190] After obtaining the simplified relational database information, the text generation task of the target language processing model will predict and generate the query text using the relational database information to obtain the text processing result, which is the final SQL statement.

[0191] The language processing method provided in this disclosure obtains simplified relational database information through a data filtering task of the target language processing model, reducing redundant information and enabling the text generation task to efficiently generate text processing results corresponding to the query text based on the simplified relational database information.

[0192] Step 306: Based on the target text processing task of the target language processing model, perform target text processing on the initial text processing result to obtain the target text processing result.

[0193] The target text processing task can optimize and adjust the generated initial text processing results, thereby improving the accuracy of the target text processing results.

[0194] In one or more embodiments of this disclosure, the target text processing task includes a noise correction task. When an initial text processing result is obtained based on a text generation task, the initial text processing result can be judged. If the initial text processing result is determined to be incorrect based on the judgment result, the initial text processing result can be corrected based on the noise correction task to obtain a corrected text processing result. Specific implementation methods are as follows:

[0195] The step of performing target text processing on the initial text processing result according to the target text processing task of the target language processing model to obtain the target text processing result includes:

[0196] Based on the noise correction task of the target language processing model, the correctness of the initial text processing result is judged to obtain the judgment result;

[0197] If the judgment result is correct, the initial text processing result is determined as the target text processing result. If the judgment result is incorrect, the initial text processing result is corrected according to the noise correction task to obtain the corrected text processing result, and the target text processing result is determined according to the corrected text processing result.

[0198] Specifically, when the target language processing model obtains the target text through supervised fine-tuning based on the third target text training data of the noise correction task, the text processing result corresponding to the target text is obtained as the initial text processing result using the text generation task of the target language processing model. The correctness of the initial text processing result can be judged. If it is determined that the initial text processing result is incorrect, the initial text processing result is corrected according to the noise correction task of the target language processing model to obtain the corrected text processing result.

[0199] Following the previous example, in practical applications, after obtaining the initial text processing result (initial SQL statement), an SQL execution tool (such as the SQLite library in Python) can be used to execute the initial SQL statement once to determine the correctness of the initial text processing result. If the SQL execution tool does not throw an exception, the initial text processing result is considered to have been executed correctly, that is, the judgment result of the initial text processing result is correct, and the initial text processing result is determined to be the target text processing result. If the SQL execution tool throws an exception, the judgment result of the initial text processing result is determined to be incorrect. The noise correction task of the target language processing model is used to correct the incorrect initial text processing result, obtaining the corrected text processing result. The corrected text processing result still needs to be judged for correctness. If the SQL execution tool executes the corrected text processing result and the SQL execution tool does not throw an exception, the corrected text processing result is determined to be the text processing result.

[0200] The language processing method provided in this disclosure, when obtaining the initial text processing result corresponding to the target text, can judge the initial text processing result, and when it is determined that the initial text processing result is incorrect, can use the noise correction task of the target language processing model to correct the incorrect initial text processing result, thereby obtaining the corrected text processing result and improving the accuracy of the text processing result.

[0201] In one or more embodiments of this disclosure, the target text processing task includes a data continuation task;

[0202] The step of performing target text processing on the initial text processing result according to the target text processing task of the target language processing model to obtain the target text processing result includes:

[0203] Based on the data continuation task of the target language processing model, determine whether the initial text processing result meets the initial data continuation conditions;

[0204] If the initial text processing result does not meet the initial data continuation conditions, the initial text processing result is determined as the target text processing result. If the initial text processing result meets the initial data continuation conditions, the initial text processing result is continued according to the data continuation task to obtain the target text processing result.

[0205] The initial data continuation condition can be understood as the initial text processing result being a challenging initial text processing result, or the initial text processing result being unable to be executed correctly; when the initial text processing result involves two (which can be set according to the actual situation) or more database tables, the initial text processing result is considered to be a challenging initial text processing result.

[0206] Specifically, when the initial text processing result is obtained using the text generation task, it is determined whether the initial text processing result is challenging or whether it cannot be executed correctly. If so (i.e., the initial text processing result meets the initial data continuation conditions), the initial text processing result is continuated using the data continuation task of the target language processing model to refine it and obtain the continuation target text processing result, thereby improving the accuracy of the target text processing result.

[0207] If the initial text processing result does not meet the conditions for continuing the initial data, the initial text processing result can be directly determined as the target text processing result.

[0208] In practical applications, when the initial SQL statement generated by a text generation task involves multiple tables, in order to avoid the initial SQL statement from missing details and failing to execute correctly, it is necessary to refine and extend the initial SQL statement. This can be done by using the data extension task of the target language processing model to obtain the complete SQL statement, or by adjusting the initial SQL statement to improve its accuracy if it fails to execute correctly.

[0209] In one or more embodiments of this disclosure, the target text processing task further includes a data continuation task. That is, when the text processing task includes a text generation task, a data filtering task, a noise correction task, and a data continuation task, the corrected text processing result is used to determine whether it meets the target data continuation condition. If so, the corrected text processing result is continued according to the data continuation task. The specific implementation is as follows:

[0210] Determining the target text processing result based on the corrected text processing result includes:

[0211] Based on the data continuation task of the target language processing model, determine whether the corrected text processing result meets the target data continuation conditions;

[0212] If the corrected text processing result does not meet the target data continuation conditions, the corrected text processing result is determined as the target text processing result. If the corrected text processing result meets the target data continuation conditions, the corrected text processing result is continued according to the data continuation task to obtain the target text processing result.

[0213] Among them, the target data continuation condition can be understood as either the corrected text processing result being a challenging corrected text processing result or the corrected text processing result being unable to be executed correctly; when the corrected text processing result involves two (which can be set according to the actual situation) or more database tables, the corrected text processing result is considered to be a challenging corrected text processing result; the target execution tool can be understood as the above-mentioned SQL execution tool, which is used to execute SQL statements and obtain SQL execution results.

[0214] In practical applications, when the text processing result is executed through an SQL execution tool and the SQL execution tool does not throw an exception (the execution result is normal), it is determined whether the text processing result involves two or more database tables. If so, the text processing result is continued according to the data continuation task to obtain the target text processing result. If the SQL execution tool throws an exception, the text processing result is continued directly according to the data continuation task to obtain the target text processing result.

[0215] If the corrected text processing result does not meet the conditions for continuing the target data (i.e., the SQL execution tool does not throw an exception and the corrected text processing result does not involve two or more database tables), the corrected text processing result will be determined as the target text processing result.

[0216] The language processing method provided in this embodiment can, when the target text processing task includes a noise correction task and a data continuation task, use the data continuation task to continue writing the corrected text processing result obtained by the noise correction task, thereby refining the corrected text processing result and improving the accuracy of the target text processing result.

[0217] The language processing method provided in this disclosure employs a multi-task collaborative prompting strategy. This strategy decomposes the Text2SQL task into several simpler sub-tasks (multiple text processing tasks including an initial text processing task and a target text processing task). By leveraging collaboration across multiple SQL-related tasks, it reduces the illusion in the SQL generation process, thereby maximizing the potential of enhancing Text2SQL performance through explicit multi-task functionality. Furthermore, these multiple text processing tasks can collaborate using simple instructions, making it applicable to models of various sizes and possessing strong versatility. Multi-task collaboration can be completed through a single target language processing model, which also greatly enhances the possibility of applying the language processing method in real-world scenarios.

[0218] Referring to Figure 4, Figure 4 shows a flowchart of the processing procedure of a language processing model training method provided in an embodiment of the present disclosure.

[0219] During the model training phase, existing technologies perform single-task supervised fine-tuning on text-to-SQL generation and pattern linking tasks based on initial text training data, i.e., training a dedicated language model separately; while in the embodiments of this disclosure, a multi-task supervised fine-tuning paradigm is adopted, which endows the target language processing model with the functions of text-to-SQL generation, pattern linking, noise correction, and SQL continuation.

[0220] Specifically, initial text training data is obtained from different data sources, discriminative SFT data is manually constructed, and a discriminator is trained to filter noise from the initial text training data, thereby achieving the effect of purifying the initial text training data and obtaining filtered text training data.

[0221] Based on the noise-filtered text training data, multi-task SFT data synthesis is performed. The multi-tasks include text-to-SQL generation task (i.e., text generation task in the above embodiment), pattern linking task (i.e., data filtering task in the above embodiment), noise correction task, and SQL continuation task (i.e., data continuation task in the above embodiment). Then, the synthesized multi-task SFT data is used for supervised fine-tuning to obtain SQL-related multi-task LLM (i.e., target language processing model in the above embodiment).

[0222] In practical applications, to enable the LLM to determine whether an SQL query can answer a user's question, SFT data for discrimination is constructed. For each Text-SQL pair (initial text training data sample) in the initial text training data, positive and negative examples are constructed. The positive examples help the LLM learn which queries can answer questions, while the negative examples help the LLM learn which queries cannot answer questions and correct them, thus achieving discrimination by filtering out noise. To construct negative examples, the LLM can answer each SQL query in the initial text training data and determine whether the execution result matches. Result that does not match is used as an incorrect SQL query to construct a negative example. Simultaneously, to introduce diversity, some common SQL errors (such as schema linking errors, JOIN errors, nesting errors, symbol errors, etc.) are artificially constructed to create some incorrect SQL queries to build negative examples.

[0223] The positive example can be understood as the labeled text training data of the positive samples in the above embodiments, whose positive label can be "The execution result of the SQL query can correctly answer the question."; the negative example can be understood as the labeled text training data of the negative samples in the above embodiments, whose negative label can be "The execution result of the SQL query cannot correctly answer the question. The correct SQL query should be: {SQL*} (SQL* refers to the corrected SQL)."

[0224] By constructing SFT data for discrimination, a discriminator is trained, thereby quickly and efficiently filtering noise from the initial text training data. Based on the noise-filtered text training data, SFT data is constructed for the four tasks mentioned above:

[0225] For text-to-SQL generation tasks, information from each table and some example rows are extracted from the database. This information, combined with the user's question, is used to construct a prompt text, and the correct SQL statement (Ground-truth SQL) corresponding to the user's question is used as the response. For schema linking tasks, information from each table and some example rows are extracted from the database. This information, combined with the user's question, is used to construct a prompt text that requires the LLM to identify the tables and columns in the database relevant to the problem. The correct SQL statement corresponding to the user's question is used to extract the relevant tables and columns as the response. If COUNT(*) appears in the SQL, the primary key information is retained. COUNT(*) is an aggregate function used to count the number of rows in the table.

[0226] For the noise correction task, the noise correction SFT data structure is consistent with the aforementioned discrimination SFT data composition and construction. Information from each table and sample rows are extracted from the database, combined with user questions and correct SQL (or manually constructed incorrect SQL) to construct prompt text. Positive labels (or negative labels, where, in the case of manually constructed incorrect SQL, the original SQL statement corresponding to the user question is identified as the corrected SQL in the negative label) are used as the response. The continuation task aims to have the LLM rewrite a truncated SQL into a complete SQL. Information from each table and sample rows are extracted from the database, combined with ground-truth SQL truncated from random positions to construct prompt text, while the complete ground-truth SQL serves as the response.

[0227] This allows for supervised fine-tuning using synthesized multi-task SFT data to obtain SQL-related multi-task LLMs.

[0228] The language processing model training method provided in this disclosure introduces a multi-task SFT paradigm, which equips the LLM with various SQL-related professional functions, including pattern linking, noise correction, and continuation writing. By participating in various SQL generation tasks, the multi-task LLM can enhance its understanding of SQL syntax and improve its ability to generate high-quality SQL queries.

[0229] Referring to Figure 5, Figure 5 shows a process flowchart of a language processing method provided in an embodiment of the present disclosure.

[0230] During the model inference phase, a multi-task collaborative hint method is used to generate SQL queries and reduce the risk of illusions in the generation of complex SQL.

[0231] Step 1: Schema linking and text-to-SQL generation.

[0232] By using multi-task LLM to predict problem-related tables and columns and combining them with pseudo-SQL to implement schema linking, the database is simplified to reduce redundant information. Based on the simplified database, multi-task LLM is used to generate SQL for user problems.

[0233] Specifically, determine the problem q (i.e., query text) received by the multi-task LLM and the corresponding database d (i.e., the target database) of the multi-task LLM.

[0234] Multi-task LLM is used to predict problem-related tables and columns, combined with pseudo-SQL to implement schema linking, represented as:

[0235] in, For LLM, σ s and σt Functions for constructing tooltip text using pattern linking and Text2SQL respectively, d i and q i Let be the i-th database and its corresponding question, and f be the extraction function for retrieving relevant tables and columns from the SQL. For the corresponding simplified database (i.e., the associated data source information), This is used for merge operations to merge tables and columns.

[0236] Based on the simplified database, the text-to-SQL generation task generates the corresponding initial SQL, i.e., SQL(s*).

[0237] Step 2: Noise correction.

[0238] The generated SQL is reviewed and self-corrected using a multi-tasking LLM.

[0239] Specifically, an LLM noise correction task is used to determine if the generated initial SQL is correct. If correct, no action is taken. If incorrect, the noise correction task corrects the generated initial SQL, outputs the corrected SQL, and executes it using an SQL execution tool (e.g., the SQLite library in Python). If the execution succeeds without throwing an exception, the next step is performed on the corrected SQL.

[0240] Step 3: Continue writing.

[0241] By using a multi-task LLM, challenging SQL queries involving more than two tables and SQL queries that cannot be executed correctly are refined and the final SQL is obtained.

[0242] If the generated initial SQL or the corrected SQL involves multiple tables (two or more), the initial SQL or the corrected SQL is continued or rewritten. If the corrected SQL still cannot be executed correctly, the rewritten SQL is continued or rewritten. The SQL that can be executed correctly is taken as the final SQL.

[0243] In practical applications, multi-tasking LLM is used to refine the SQL by continuing from "SELECT" to obtain the refined SQL. If the refined SQL can be executed correctly, it is used as the final SQL. Furthermore, the text execution result can be obtained by executing the final SQL, which is the answer to the input question.

[0244] In this embodiment of the disclosure, the multi-task LLM can be used in competition question-and-answer scenarios. For example, the multi-task LLM can store a large amount of historical competition data in an SQL database. By using the multi-task LLM, it can accurately generate SQL query statements that can meet the user's intent. The relevant information can be obtained through the SQL query statements and combined with a large language model to achieve accurate responses.

[0245] The language processing method provided in this disclosure employs a multi-task collaboration prompting strategy that decomposes a Text2SQL task into several simpler subtasks. This strategy leverages collaboration across multiple SQL-related tasks to reduce illusions in the SQL generation process, thereby maximizing the potential of enhancing Text2SQL performance through explicit multitasking capabilities.

[0246] Corresponding to the above method embodiments, this disclosure also provides an embodiment of a language processing model training device. Figure 6 shows a schematic diagram of the structure of a language processing model training device provided in one embodiment of this disclosure. As shown in Figure 6, the device includes:

[0247] The determination module 602 is configured to determine multiple text processing tasks and initial text training data;

[0248] The obtaining module 604 is configured to perform data processing on the initial text training data according to each text processing task in the plurality of text processing tasks, and obtain the task text training data corresponding to each text processing task.

[0249] The training module 606 is configured to train an initial language processing model based on the task text training data corresponding to each text processing task to obtain a target language processing model. The target language processing model is used to perform initial text processing on the target text according to the initial text processing task among the multiple text processing tasks when receiving target text, to obtain an initial text processing result, and to perform target text processing on the initial text processing result according to the target text processing task among the multiple text processing tasks to obtain a target text processing result.

[0250] Optionally, the obtaining module 604 is further configured to:

[0251] The initial text training data is subjected to noise filtering to obtain filtered text training data;

[0252] Based on the initial text processing task, the filtered text training data is processed to obtain the initial task text training data corresponding to the initial text processing task.

[0253] Based on the target text processing task, the filtered text training data is processed to obtain the target task text training data corresponding to the target text processing task.

[0254] Optionally, the training module 606 is further configured to:

[0255] The initial language processing model is trained based on the initial task text training data corresponding to the initial text processing task and the target task text training data corresponding to the target text processing task to obtain the target language processing model.

[0256] Optionally, the obtaining module 604 is further configured to:

[0257] According to the text generation task, example information is extracted from the data source samples in the filtered text training data to obtain the example information of the data source samples. The filtered text training data is obtained by noise filtering of the initial text training data. The filtered text training data includes the initial text sample, the target language data sample corresponding to the initial text sample, and the data source sample.

[0258] Based on the initial text sample, the target language data sample corresponding to the initial text sample, and the example information, the first target text training data corresponding to the text generation task is obtained;

[0259] According to the data filtering task, the data source sample information of the target language data sample in the filtered text training data is extracted to obtain the associated data source sample information corresponding to the initial text sample.

[0260] Based on the initial text sample, the associated data source sample information, and the example information, the second target text training data corresponding to the data filtering task is obtained.

[0261] Optionally, the training module 606 is further configured to:

[0262] The initial text sample and the example information in the first target text training data are determined as the first prompt text, and the target language data sample in the first target text training data is determined as the first response tag;

[0263] The initial text sample, the example information, and the target request in the second target text training data are determined as the second prompt text, and the associated data source sample information in the second target text training data are determined as the second response tag;

[0264] The initial language processing model is trained based on the first prompt text, the first reply tag, the second prompt text, and the second reply tag to obtain the target language processing model.

[0265] Optionally, the obtaining module 604 is further configured to:

[0266] According to the noise correction task, the target language data samples in the filtered text training data are replaced to obtain the replaced language data samples.

[0267] Based on the initial text sample, the replacement language data sample, the target language data sample, and the example information, the third target text training data corresponding to the noise correction task is obtained.

[0268] Optionally, the training module 606 is further configured to:

[0269] The initial text sample and the example information in the first target text training data are determined as the first prompt text, and the target language data sample in the first target text training data is determined as the first response tag;

[0270] The initial text sample, the example information, and the target request in the second target text training data are determined as the second prompt text, and the associated data source sample information in the second target text training data are determined as the second response tag;

[0271] The initial text sample, the replacement language data sample, and the example information in the third target text training data are determined as the third prompt text, and the target language data sample in the third target text training data is determined as the third response tag;

[0272] The initial language processing model is trained based on the first prompt text, the first reply tag, the second prompt text, the second reply tag, the third prompt text, and the third reply tag to obtain the target language processing model.

[0273] Optionally, the obtaining module 604 is further configured to:

[0274] According to the data continuation task, the target language data samples in the filtered text training data are truncated to obtain truncated language data.

[0275] Based on the initial text sample, the truncated language data, the target language data sample, and the example information, the fourth target text training data corresponding to the data continuation task is obtained.

[0276] Optionally, the training module 606 is further configured to:

[0277] The initial text sample and the example information in the first target text training data are determined as the first prompt text, and the target language data sample in the first target text training data is determined as the first response tag;

[0278] The initial text sample, the example information, and the target request in the second target text training data are determined as the second prompt text, and the associated data source sample information in the second target text training data are determined as the second response tag;

[0279] The initial text sample, the truncated language data, and the example information in the fourth target text training data are determined as the fourth prompt text, and the target language data sample in the fourth target text training data is determined as the fourth response tag;

[0280] The initial language processing model is trained based on the first prompt text, the first reply tag, the second prompt text, the second reply tag, the fourth prompt text, and the fourth reply tag to obtain the target language processing model.

[0281] Optionally, the obtaining module 604 is further configured to:

[0282] According to the noise correction task, the target language data samples in the filtered text training data are replaced to obtain the replaced language data samples.

[0283] Based on the initial text sample, the replacement language data sample, the target language data sample, and the example information, obtain the third target text training data corresponding to the noise correction task;

[0284] According to the data continuation task, the target language data samples in the filtered text training data are truncated to obtain truncated language data.

[0285] Based on the initial text sample, the truncated language data, the target language data sample, and the example information, the fourth target text training data corresponding to the data continuation task is obtained.

[0286] Optionally, the training module 606 is further configured to:

[0287] The initial language processing model is trained using the first target text training data corresponding to the text generation task, the second target text training data corresponding to the data filtering task, the third target text training data corresponding to the noise correction task, and the fourth target text training data corresponding to the data continuation task, to obtain the target language processing model.

[0288] Optionally, the obtaining module 604 is further configured to:

[0289] Example information is extracted from the data source samples in the filtered text training data to obtain the example information of the data source samples. The filtered text training data is obtained by filtering noise from the initial text training data. The filtered text training data includes initial text samples, target language data samples corresponding to the initial text samples, and data source samples.

[0290] According to the noise correction task, the target language data samples in the filtered text training data are replaced to obtain the replaced language data samples.

[0291] Based on the initial text sample, the replacement language data sample, the target language data sample, and the example information, obtain the third target text training data corresponding to the noise correction task;

[0292] According to the data continuation task, the target language data samples in the filtered text training data are truncated to obtain truncated language data.

[0293] Based on the initial text sample, the truncated language data, the target language data sample, and the example information, the fourth target text training data corresponding to the data continuation task is obtained.

[0294] Optionally, the training module 606 is further configured to:

[0295] The initial language processing model is trained based on the third target text training data corresponding to the noise correction task and the fourth target text training data corresponding to the data continuation task to obtain the target language processing model.

[0296] Optionally, the obtaining module 604 is further configured to:

[0297] The initial text training data is filtered for noise using a discriminator to obtain filtered text training data, wherein the initial text training data includes initial text samples, target language data samples corresponding to the initial text samples, and data source samples.

[0298] Based on each of the multiple text processing tasks, the filtered text training data is processed to obtain the task text training data corresponding to each text processing task.

[0299] Optionally, the obtaining module 604 is further configured to:

[0300] The initial text training data is input into the discriminator to obtain the classification result of the initial text training data, wherein the discriminator is trained using labeled text training data of positive and negative samples;

[0301] The initial text training data whose classification result is a positive sample is determined as the filtered text training data.

[0302] The language processing model training apparatus provided in this embodiment determines multiple text processing tasks and initial text training data. Each text processing task has a different task; therefore, the initial text training data is processed according to the corresponding text processing tasks to obtain task text training data that meets the requirements of each text processing task. The initial language processing model is trained using the task text training data corresponding to each text processing task, enabling the trained target language processing model to possess the ability to perform multiple text processing tasks. Therefore, this target language processing model can be used to process query text collaboratively using multiple text processing tasks when target text is received, obtaining accurate text processing results corresponding to the target text. That is, by using the task text training data of multiple text processing tasks to train the initial language processing model, multi-task training is achieved on the initial language processing model, reducing the risk of overfitting and enabling the trained target language processing model to possess the ability to perform multiple text processing tasks. When processing target text using the target language processing model, the collaboration of multiple text processing tasks can improve the accuracy of the target text processing results.

[0303] The above is an illustrative scheme of a language processing model training device according to this embodiment. It should be noted that the technical solution of this language processing model training device and the technical solution of the language processing model training method described above belong to the same concept. For details not described in detail in the technical solution of the language processing model training device, please refer to the description of the technical solution of the language processing model training method described above.

[0304] Corresponding to the above method embodiments, this disclosure also provides a language processing device embodiment. Figure 7 shows a schematic diagram of the structure of a language processing device provided in one embodiment of this disclosure. As shown in Figure 7, the device includes:

[0305] Text determination module 702 is configured to determine target text;

[0306] The initial acquisition module 704 is configured to perform initial text processing on the target text according to the initial text processing task of the target language processing model, and obtain the initial text processing result;

[0307] The target acquisition module 706 is configured to perform target text processing on the initial text processing result according to the target text processing task of the target language processing model, and obtain the target text processing result.

[0308] Optionally, the initial acquisition module 704 is further configured to:

[0309] According to the data filtering task of the target language processing model, the associated data source information of the target text is extracted from the target data source corresponding to the target language processing model;

[0310] Based on the text generation task of the target language processing model, the target text is predicted based on the associated data source information to obtain the initial text processing result.

[0311] Optionally, the target acquisition module 706 is further configured to:

[0312] Based on the noise correction task of the target language processing model, the correctness of the initial text processing result is judged to obtain the judgment result;

[0313] If the judgment result is correct, the initial text processing result will be determined as the target text processing result.

[0314] If the judgment result is incorrect, the initial text processing result is corrected according to the noise correction task to obtain the corrected text processing result, and the target text processing result is determined according to the corrected text processing result.

[0315] Optionally, the target acquisition module 706 is further configured to:

[0316] Based on the data continuation task of the target language processing model, determine whether the initial text processing result meets the initial data continuation conditions;

[0317] If the initial text processing result does not meet the initial data continuation conditions, the initial text processing result will be determined as the target text processing result.

[0318] If the initial text processing result satisfies the initial data continuation condition, the initial text processing result is continued according to the data continuation task to obtain the target text processing result.

[0319] Optionally, the target acquisition module 706 is further configured to:

[0320] Based on the data continuation task of the target language processing model, determine whether the corrected text processing result meets the target data continuation conditions.

[0321] If the corrected text processing result does not meet the target data continuation conditions, the corrected text processing result will be determined as the target text processing result.

[0322] If the corrected text processing result meets the target data continuation conditions, the corrected text processing result is continued according to the data continuation task to obtain the target text processing result.

[0323] The language processing apparatus provided in this disclosure, when the target language processing model is trained on an initial language processing model using training data from task texts of multiple text processing tasks, enables the target language processing model to perform multiple text processing tasks. When the target language processing model is used to process target text, the accuracy of the target text processing result can be improved through the collaboration of multiple text processing tasks.

[0324] The above is an illustrative scheme of a language processing device according to this embodiment. It should be noted that the technical solution of this language processing device and the technical solution of the language processing method described above belong to the same concept. For details not described in detail in the technical solution of the language processing device, please refer to the description of the technical solution of the language processing method described above.

[0325] Figure 8 shows a structural block diagram of a computing device 800 provided according to an embodiment of the present disclosure.

[0326] The computing device 800 includes:

[0327] Memory 810 and processor 820;

[0328] The memory 810 is used to store computer programs / instructions, and the processor 820 is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor 820, they implement the steps of the language processing model training method and the language processing method.

[0329] In one or more embodiments of this disclosure, the computing device can be understood as an integrated smart terminal, including but not limited to a server, desktop computer, PC (Personal Computer), all-in-one model machine, mobile phone, tablet computer or other portable smart terminal, etc., and the computing device may have the model described in the above embodiments of this disclosure pre-installed.

[0330] Specifically, this computing device can pre-install various types of models, including but not limited to models in natural language processing, visual processing, speech processing, code processing, and multimodal task processing, thus providing diverse model selection. In different product forms, this computing device can support one or more model usage methods, including but not limited to model training, model invocation, model fine-tuning, model deployment, model inference, and application. In some product forms, this computing device also supports model management, including but not limited to multi-type model management (supporting the management of discriminative, generative, and other types of models), model version control (supporting the control of different model versions), and model evaluation (evaluating model performance and effectiveness based on model evaluation tools). In other product forms, this computing device can also create applications based on models, providing API (Application Programming Interface) calling capabilities. Users can call models into created applications through the API interface, and application management tools are also provided to manage and monitor the applications.

[0331] Furthermore, the computing device may also include data management (supporting the creation and management of model tuning datasets), a training center (providing abundant training resources to help users learn and master AI (Artificial Intelligence) technology), and basic control capabilities (providing enterprise-level basic control capabilities to ensure the security and efficient operation of the system). Through the above functions, it provides a comprehensive and integrated device for AI development, training, deployment, and application.

[0332] Furthermore, the components of the computing device 800 include, but are not limited to, a memory 810 and a processor 820. The processor 820 and the memory 810 can be connected via a bus.

[0333] The computing device 800 may also include an access device that enables the computing device 800 to communicate with a database storing data via one or more networks. Examples of such networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device may include one or more of any type of wired or wireless network interface (e.g., a network interface controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0334] In one embodiment of this disclosure, the aforementioned components of the computing device 800, as well as other components not shown in FIG. 8, may be interconnected, for example, via a bus. It should be understood that the computing device block diagram shown in FIG. 8 is merely for illustrative purposes and is not intended to limit the scope of this disclosure. Those skilled in the art can add or replace other components as needed.

[0335] The computing device 800 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 800 can also be a mobile or stationary server.

[0336] The processor 820 is used to execute the following computer program / instruction, which, when executed by the processor, implements the steps of the above-mentioned language processing model training method and language processing method.

[0337] The various embodiments in this disclosure are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the computing device embodiments are basically similar to the language processing model training method and language processing method embodiments, so the description is relatively simple. Relevant parts can be referred to the descriptions of the language processing model training method and language processing method embodiments.

[0338] An embodiment of this disclosure also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described language processing model training method and language processing method.

[0339] The various embodiments in this disclosure are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the computer-readable storage medium embodiments are relatively simple in description because they are fundamentally similar to the language processing model training method and language processing method embodiments. Relevant parts can be referred to the descriptions of the language processing model training method and language processing method embodiments.

[0340] An embodiment of this disclosure also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the above-described language processing model training method and language processing method.

[0341] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product belongs to the same concept as the technical solutions of the language processing model training method and the language processing method described above. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solutions of the language processing model training method and the language processing method described above.

[0342] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0343] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0344] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this disclosure are not limited to the described order of actions, because according to the embodiments of this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this disclosure.

[0345] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0346] The preferred embodiments disclosed above are merely illustrative of this disclosure. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments of this disclosure. These embodiments are selected and specifically described in this disclosure to better explain the principles and practical applications of the embodiments of this disclosure, thereby enabling those skilled in the art to better understand and utilize this disclosure. This disclosure is limited only by the claims and their full scope and equivalents.

Claims

1. A method for training a language processing model, comprising: Define multiple text processing tasks and initial text training data; Based on each of the multiple text processing tasks, the initial text training data is processed to obtain the task text training data corresponding to each text processing task. Based on the task text training data corresponding to each text processing task, the initial language processing model is trained to obtain the target language processing model. The target language processing model is used to, upon receiving target text, perform initial text processing on the target text according to the initial text processing task among the plurality of text processing tasks to obtain an initial text processing result, and then perform target text processing on the initial text processing result according to the target text processing task among the plurality of text processing tasks to obtain a target text processing result.

2. The language processing model training method according to claim 1, wherein the plurality of text processing tasks includes an initial text processing task and a target text processing task; The step of processing the initial text training data according to each of the plurality of text processing tasks to obtain task text training data corresponding to each text processing task includes: The initial text training data is subjected to noise filtering to obtain filtered text training data; Based on the initial text processing task, the filtered text training data is processed to obtain the initial task text training data corresponding to the initial text processing task. Based on the target text processing task, the filtered text training data is processed to obtain the target task text training data corresponding to the target text processing task.

3. The language processing model training method according to claim 2, wherein training the initial language processing model based on the task text training data corresponding to each text processing task to obtain the target language processing model includes: The initial language processing model is trained based on the initial task text training data corresponding to the initial text processing task and the target task text training data corresponding to the target text processing task to obtain the target language processing model.

4. The language processing model training method according to claim 1 or 2, wherein the plurality of text processing tasks includes an initial text processing task, and the initial text processing task includes a text generation task and a data filtering task. The step of processing the initial text training data according to each of the plurality of text processing tasks to obtain task text training data corresponding to each text processing task includes: According to the text generation task, example information is extracted from the data source samples in the filtered text training data to obtain the example information of the data source samples. The filtered text training data is obtained by noise filtering of the initial text training data. The filtered text training data includes the initial text sample, the target language data sample corresponding to the initial text sample, and the data source sample. Based on the initial text sample, the target language data sample corresponding to the initial text sample, and the example information, the first target text training data corresponding to the text generation task is obtained; According to the data filtering task, the data source sample information of the target language data sample in the filtered text training data is extracted to obtain the associated data source sample information corresponding to the initial text sample. Based on the initial text sample, the associated data source sample information, and the example information, the second target text training data corresponding to the data filtering task is obtained.

5. The language processing model training method according to claim 4, wherein training the initial language processing model based on the task text training data corresponding to each text processing task to obtain the target language processing model includes: The initial text sample and the example information in the first target text training data are determined as the first prompt text, and the target language data sample in the first target text training data is determined as the first response tag; The initial text sample, the example information, and the target request in the second target text training data are determined as the second prompt text, and the associated data source sample information in the second target text training data are determined as the second response tag; The initial language processing model is trained based on the first prompt text, the first reply tag, the second prompt text, and the second reply tag to obtain the target language processing model.

6. The language processing model training method according to claim 4, wherein the plurality of text processing tasks further includes a target text processing task, wherein the target text processing task includes a noise correction task; The step of processing the initial text training data according to each of the plurality of text processing tasks to obtain task text training data corresponding to each text processing task further includes: According to the noise correction task, the target language data samples in the filtered text training data are replaced to obtain the replaced language data samples. Based on the initial text sample, the replacement language data sample, the target language data sample, and the example information, the third target text training data corresponding to the noise correction task is obtained.

7. The language processing model training method according to claim 6, wherein training the initial language processing model based on the task text training data corresponding to each text processing task to obtain the target language processing model includes: The initial text sample and the example information in the first target text training data are determined as the first prompt text, and the target language data sample in the first target text training data is determined as the first response tag; The initial text sample, the example information, and the target request in the second target text training data are determined as the second prompt text, and the associated data source sample information in the second target text training data are determined as the second response tag; The initial text sample, the replacement language data sample, and the example information in the third target text training data are determined as the third prompt text, and the target language data sample in the third target text training data is determined as the third response tag; The initial language processing model is trained based on the first prompt text, the first reply tag, the second prompt text, the second reply tag, the third prompt text, and the third reply tag to obtain the target language processing model.

8. The language processing model training method according to claim 4, wherein the plurality of text processing tasks further includes a target text processing task, wherein the target text processing task includes a data continuation task; The step of processing the initial text training data according to each of the plurality of text processing tasks to obtain task text training data corresponding to each text processing task further includes: According to the data continuation task, the target language data samples in the filtered text training data are truncated to obtain truncated language data. Based on the initial text sample, the truncated language data, the target language data sample, and the example information, the fourth target text training data corresponding to the data continuation task is obtained.

9. The language processing model training method according to claim 8, wherein training the initial language processing model based on the task text training data corresponding to each text processing task to obtain the target language processing model includes: The initial text sample and the example information in the first target text training data are determined as the first prompt text, and the target language data sample in the first target text training data is determined as the first response tag; The initial text sample, the example information, and the target request in the second target text training data are determined as the second prompt text, and the associated data source sample information in the second target text training data are determined as the second response tag; The initial text sample, the truncated language data, and the example information in the fourth target text training data are determined as the fourth prompt text, and the target language data sample in the fourth target text training data is determined as the fourth response tag; The initial language processing model is trained based on the first prompt text, the first reply tag, the second prompt text, the second reply tag, the fourth prompt text, and the fourth reply tag to obtain the target language processing model.

10. The language processing model training method according to claim 4, wherein the plurality of text processing tasks further includes a target text processing task, wherein the target text processing task includes a noise correction task and a data continuation task; The step of processing the initial text training data according to each of the plurality of text processing tasks to obtain task text training data corresponding to each text processing task further includes: According to the noise correction task, the target language data samples in the filtered text training data are replaced to obtain the replaced language data samples. Based on the initial text sample, the replacement language data sample, the target language data sample, and the example information, obtain the third target text training data corresponding to the noise correction task; According to the data continuation task, the target language data samples in the filtered text training data are truncated to obtain truncated language data. Based on the initial text sample, the truncated language data, the target language data sample, and the example information, the fourth target text training data corresponding to the data continuation task is obtained.

11. The language processing model training method according to claim 10, wherein training the initial language processing model based on the task text training data corresponding to each text processing task to obtain the target language processing model includes: The initial language processing model is trained using the first target text training data corresponding to the text generation task, the second target text training data corresponding to the data filtering task, the third target text training data corresponding to the noise correction task, and the fourth target text training data corresponding to the data continuation task, to obtain the target language processing model.

12. The language processing model training method according to claim 1 or 2, wherein the plurality of text processing tasks includes a target text processing task, and the target text processing task includes a noise correction task and a data continuation task; The step of processing the initial text training data according to each of the plurality of text processing tasks to obtain task text training data corresponding to each text processing task includes: Example information is extracted from the data source samples in the filtered text training data to obtain the example information of the data source samples. The filtered text training data is obtained by filtering noise from the initial text training data. The filtered text training data includes initial text samples, target language data samples corresponding to the initial text samples, and data source samples. According to the noise correction task, the target language data samples in the filtered text training data are replaced to obtain the replaced language data samples. Based on the initial text sample, the replacement language data sample, the target language data sample, and the example information, obtain the third target text training data corresponding to the noise correction task; According to the data continuation task, the target language data samples in the filtered text training data are truncated to obtain truncated language data. Based on the initial text sample, the truncated language data, the target language data sample, and the example information, the fourth target text training data corresponding to the data continuation task is obtained.

13. The language processing model training method according to claim 12, wherein training the initial language processing model based on the task text training data corresponding to each text processing task to obtain the target language processing model includes: The initial language processing model is trained based on the third target text training data corresponding to the noise correction task and the fourth target text training data corresponding to the data continuation task to obtain the target language processing model.

14. The language processing model training method according to claim 1, wherein the step of processing the initial text training data according to each of the plurality of text processing tasks to obtain task text training data corresponding to each text processing task includes: The initial text training data is filtered for noise using a discriminator to obtain filtered text training data, wherein the initial text training data includes initial text samples, target language data samples corresponding to the initial text samples, and data source samples. Based on each of the multiple text processing tasks, the filtered text training data is processed to obtain the task text training data corresponding to each text processing task.

15. The language processing model training method according to claim 14, wherein the step of using a discriminator to filter noise from the initial text training data to obtain filtered text training data includes: The initial text training data is input into the discriminator to obtain the classification result of the initial text training data, wherein the discriminator is trained using labeled text training data of positive and negative samples; The initial text training data whose classification result is a positive sample is determined as the filtered text training data.

16. A language processing method, comprising: Identify the target text; Based on the initial text processing task of the target language processing model, the target text is subjected to initial text processing to obtain the initial text processing result; Based on the target text processing task of the target language processing model, the initial text processing result is subjected to target text processing to obtain the target text processing result.

17. The language processing method according to claim 16, wherein the initial text processing task includes a text generation task and a data filtering task; The step of performing initial text processing on the target text according to the initial text processing task of the target language processing model to obtain the initial text processing result includes: According to the data filtering task of the target language processing model, the associated data source information of the target text is extracted from the target data source corresponding to the target language processing model; Based on the text generation task of the target language processing model, the target text is predicted based on the associated data source information to obtain the initial text processing result.

18. The language processing method according to claim 16 or 17, wherein the target text processing task includes a noise correction task; The step of performing target text processing on the initial text processing result according to the target text processing task of the target language processing model to obtain the target text processing result includes: Based on the noise correction task of the target language processing model, the correctness of the initial text processing result is judged to obtain the judgment result; If the judgment result is correct, the initial text processing result will be determined as the target text processing result. If the judgment result is incorrect, the initial text processing result is corrected according to the noise correction task to obtain the corrected text processing result, and the target text processing result is determined according to the corrected text processing result.

19. The language processing method according to claim 16 or 17, wherein the target text processing task includes a data continuation task; The step of performing target text processing on the initial text processing result according to the target text processing task of the target language processing model to obtain the target text processing result includes: Based on the data continuation task of the target language processing model, determine whether the initial text processing result meets the initial data continuation conditions; If the initial text processing result does not meet the initial data continuation conditions, the initial text processing result will be determined as the target text processing result. If the initial text processing result satisfies the initial data continuation condition, the initial text processing result is continued according to the data continuation task to obtain the target text processing result.

20. The language processing method according to claim 18, wherein the target text processing task further includes a data continuation task; Determining the target text processing result based on the corrected text processing result includes: Based on the data continuation task of the target language processing model, determine whether the corrected text processing result meets the target data continuation conditions. If the corrected text processing result does not meet the target data continuation conditions, the corrected text processing result will be determined as the target text processing result. If the corrected text processing result meets the target data continuation conditions, the corrected text processing result is continued according to the data continuation task to obtain the target text processing result.

21. A computing device, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, they implement the steps of the language processing model training method according to any one of claims 1 to 15 and the language processing method according to any one of claims 16 to 20.

22. A computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the language processing model training method of any one of claims 1 to 15 and the language processing method of any one of claims 16 to 20.

23. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the language processing model training method of any one of claims 1 to 15 and the language processing method of any one of claims 16 to 20.

Images