Data processing method and apparatus, and target question-answering model training method and apparatus

Abstract

Provided in the embodiments of the present disclosure are a data processing method and apparatus, and a target question-answering model training method and apparatus. The data processing method comprises: adjusting an initial text processing result corresponding to initial text in an initial text pair, in order to obtain an updated text processing result, and on the basis of the initial text and the updated text processing result, constructing an updated text pair; on the basis of the updated text pair, updating an initial text processing model, in order to obtain a reference text processing model; on the basis of the initial text pair, using the initial text processing model and the reference text processing model to respectively obtain predicted loss results and reference loss results that correspond to a plurality of tokens in the initial text processing result; on the basis of the predicted loss results and the reference loss results, determining a loss change result for each token; and on the basis of the loss change result for each token, executing a data processing task. Thus, token-level distinction is realized, and reasoning-focused supervised fine-tuning can be performed on the basis of the token-level distinction, thereby improving the reasoning capability of models.

Description

Data processing method and device, target question and answer model training method and device

[0001] The present disclosure claims priority to Chinese patent application No. 202411120848.8, filed on August 14, 2024 with the Chinese Patent Office, entitled "Data processing method and device, target question and answer model training method and device", the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0002] Embodiments of the present disclosure relate to the technical field of computer, in particular to a data processing method and device, a target question and answer model training method and device. BACKGROUND

[0003] Large language models pre-trained by a large amount of data have strong potential, but the large language models obtained by pre-training cannot be directly applied to dialogues and solving various tasks such as question and answer, mathematics, tool calling, etc., so supervised fine-tuning is needed to enable the pre-trained large language model to learn to use pre-trained knowledge to solve various tasks.

[0004] Dialog data used for supervised fine-tuning training usually contains multiple tasks, and the data in each task usually has a large number of common and easy-to-learn patterns, and there is also a reasoning part unique to each data. Direct training on dialog data may cause overfitting on the easy-to-learn patterns of the data and underfitting on the reasoning part.

[0005] Therefore, there is an urgent need for a data processing method to distinguish the common and easy-to-learn patterns from the unique reasoning part in each data, so as to solve the problem of model performance decline caused by equal treatment of different parts of the supervised fine-tuning data when the pre-trained large language model is supervised fine-tuned, and to solve the problem of low data selection accuracy caused by irrelevant information interference when data selection is performed from the supervised fine-tuning data to obtain training data for a specific task. SUMMARY

[0006] Therefore, the embodiments of the present disclosure provide a data processing method. One or more embodiments of the present disclosure also relate to a data processing device, a target question and answer model training method, a target question and answer model training device, a model training platform, a computing device, a computer-readable storage medium, and a computer program product, to solve the technical defects of the model performance decline caused by equal treatment of different parts of the supervised fine-tuning data and the low data selection accuracy in the prior art.

[0007] According to a first aspect of the embodiments of the present disclosure, a data processing method is provided, comprising:

[0008] adjusting an initial text processing result corresponding to an initial text in an initial text pair, obtaining an updated text processing result, and constructing an updated text pair according to the initial text and the updated text processing result;

[0009] updating an initial text processing model according to the updated text pair, and obtaining a reference text processing model;

[0010] obtaining, according to the initial text pair, a predicted loss result and a reference loss result corresponding to each token in the initial text processing result by using the initial text processing model and the reference text processing model;

[0011] determining a loss change result of each token according to the predicted loss result and the reference loss result;

[0012] performing a data processing task according to the loss change result of each token.

[0013] According to a second aspect of the embodiments of the present disclosure, a data processing apparatus is provided, comprising:

[0014] a construction module configured to adjust an initial text processing result corresponding to an initial text in an initial text pair, obtain an updated text processing result, and construct an updated text pair according to the initial text and the updated text processing result;

[0015] an updating module configured to update an initial text processing model according to the updated text pair, and obtain a reference text processing model;

[0016] an obtaining module configured to obtain, according to the initial text pair, a predicted loss result and a reference loss result corresponding to each token in the initial text processing result by using the initial text processing model and the reference text processing model;

[0017] a determination module configured to determine a loss change result of each token according to the predicted loss result and the reference loss result;

[0018] an execution module configured to perform a data processing task according to the loss change result of each token.

[0019] According to a third aspect of the embodiments of the present disclosure, a target question and answer model training method is provided, comprising:

[0020] adjusting an initial answer text corresponding to an initial question text in an initial question and answer pair, obtaining an updated answer text, and constructing an updated question and answer pair according to the initial question text and the updated answer text;

[0021] updating an initial question and answer model according to the updated question and answer pair, and obtaining a reference question and answer model;

[0022] According to the initial question and answer pair, the initial question and answer model and the reference question and answer model are used to obtain prediction loss results and reference loss results respectively corresponding to a plurality of word units in the initial answer text;

[0023] According to the prediction loss results and the reference loss results, loss change results of each word unit are determined, and target loss weights corresponding to the word units are determined according to the loss change results of each word unit;

[0024] According to the target loss weights corresponding to each word unit and the prediction loss results of each word unit, a target loss result corresponding to the initial answer text is determined.

[0025] According to the target loss result, the initial question and answer model is trained to obtain a target question and answer model.

[0026] According to a fourth aspect of the embodiments of the present disclosure, a target question and answer model training device is provided, which comprises:

[0027] A data construction module is configured to adjust an initial answer text corresponding to an initial question text in an initial question and answer pair to obtain an updated answer text, and construct an updated question and answer pair according to the initial question text and the updated answer text.

[0028] A model updating module is configured to update an initial question and answer model according to the updated question and answer pair to obtain a reference question and answer model.

[0029] A result obtaining module is configured to use the initial question and answer model and the reference question and answer model to obtain prediction loss results and reference loss results respectively corresponding to a plurality of word units in the initial answer text according to the initial question and answer pair.

[0030] A weight determining module is configured to determine loss change results of each word unit according to the prediction loss results and the reference loss results, and determine target loss weights corresponding to the word units according to the loss change results of each word unit.

[0031] A result determining module is configured to determine a target loss result corresponding to the initial answer text according to the target loss weights corresponding to each word unit and the prediction loss results of each word unit.

[0032] A model training module is configured to train the initial question and answer model according to the target loss result to obtain a target question and answer model.

[0033] According to a fifth aspect of the embodiments of the present disclosure, a model training platform is provided, which comprises a request interface unit, a model training unit and a response unit.

[0034] The request interface unit is configured to receive a model training request, wherein the model training request comprises model information of an initial question and answer model to be trained.

[0035] The model training unit is configured to determine the initial question and answer model to be trained according to the model information, and perform model training on the initial question and answer model to be trained to obtain a target question and answer model, wherein the target question and answer model is obtained by using the target question and answer model training method.

[0036] The response unit is configured to output the target question and answer model.

[0037] According to a sixth aspect of the embodiments of the present disclosure, a computing device is provided, comprising:

[0038] a memory and a processor;

[0039] The memory is configured to store computer programs / instructions, and the processor is configured to execute the computer programs / instructions, so as to realize the steps of the above data processing method and the target question and answer model training method.

[0040] According to a seventh aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, which stores computer programs / instructions, and the computer programs / instructions are executed by a processor to realize the steps of the above data processing method and the target question and answer model training method.

[0041] According to an eighth aspect of the embodiments of the present disclosure, a computer program product is provided, comprising computer programs / instructions, and the computer programs / instructions are executed by a processor to realize the steps of the above data processing method and the target question and answer model training method.

[0042] The data processing method provided by one embodiment of the present disclosure adjusts the initial text processing result corresponding to the initial text pair to obtain an updated text processing result, constructs an updated text pair using the updated initial text and the updated text processing result, and in this case, updates the initial text processing model using the updated text pair to obtain a reference text processing model. The reference text processing model can learn the common easy-to-learn part in the initial text processing result and the updated text processing result, while the prediction performance of the inference part deteriorates and the loss increases. These common easy-to-learn parts and inference parts are represented as word units in the text, that is, the common easy-to-learn part is a common easy-to-learn word unit in the initial text processing result, and the inference part is an inference word unit in the initial text processing result. Thus, using the initial text processing model and the reference text processing model, the prediction loss result and the reference loss result corresponding to each word unit in the initial text processing result can be obtained respectively. The loss of the reference text processing model for the common easy-to-learn word unit is reduced, so the reference loss result of the reference text processing model for the common easy-to-learn word unit is smaller than the prediction loss result of the initial text processing model. However, the loss of the reference text processing model for the inference word unit is increased, so the reference loss result of the reference text processing model for the inference word unit is greater than the prediction loss result of the initial text processing model. On the basis of determining the loss change result of each word unit by the prediction loss result and the reference loss result, the common easy-to-learn word unit and the inference word unit in the initial text processing result are distinguished according to the loss change result of each word unit. In the case of performing a data processing task based on the loss change result, the accuracy and efficiency of data processing can be improved by flexibly setting the data processing task and using the distinguished common easy-to-learn word unit and inference word unit in the process of performing the data processing task. BRIEF DESCRIPTION OF DRAWINGS

[0043] FIG. 1 is a scene schematic diagram of a data processing method according to one embodiment of the present disclosure;

[0044] FIG. 2 is a flowchart of a data processing method according to one embodiment of the present disclosure;

[0045] FIG. 3 is a flowchart of a target question and answer model training method according to one embodiment of the present disclosure;

[0046] FIG. 4 is a process flowchart of a data processing method according to one embodiment of the present disclosure;

[0047] FIG. 5 is a framework diagram of a model training platform according to one embodiment of the present disclosure;

[0048] FIG. 6 is a structural schematic diagram of a data processing apparatus according to one embodiment of the present disclosure;

[0049] FIG. 7 is a structural schematic diagram of a target question answering model device according to an embodiment of the present disclosure;

[0050] FIG. 8 is a structural block diagram of a computing device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0051] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, the present disclosure can be practiced without the specific details, which are not described in the present disclosure, and it is understood that the scope of the present disclosure is not limited to the details of the embodiments described herein. In other instances, well-known methods, procedures, components, and networks have not been described in detail as not to unnecessarily obscure aspects of the present disclosure.

[0052] The terminology used in one or more embodiments of the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of one or more embodiments of the present disclosure. As used in one or more embodiments of the present disclosure and the accompanying claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be understood that the terms "and / or," "comprises" and / or "comprising" when used in one or more embodiments of the present disclosure are open-ended terms that specify the presence of any item, number, operational scenario, component, element, or combination of items, numbers, operational scenarios, components, elements, or the like, but do not exclude the presence of one or more other items, numbers, operational scenarios, components, elements, or combination of items, numbers, operational scenarios, components, elements.

[0053] It will be understood that, although the terms first, second, etc. can be used herein to describe various information, these terms are not intended to denote a temporal or chronological order. Rather, these terms are used only to distinguish one from another. For example, without departing from the scope of one or more embodiments of the present disclosure, first can be termed second, and similarly, second can be termed first. The term "if' as used herein can be interpreted as meaning "when" or "in response to determining" depending on the context.

[0054] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in one or more embodiments of the present disclosure are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0055] First, the nomenclature involved in one or more embodiments of the present disclosure is explained.

[0056] Token: Token, a large language model divides natural language text into tokens as processing units through a tokenizer.

[0057] Large Language Model, a super-large neural network model pre-trained with a large amount of data, has strong natural language understanding and generation capabilities.

[0058] Supervised Fine-tunning, the training of a neural network model based on labeled data, in the supervised fine-tuning of a large language model, dialogue data is often used.

[0059] Data selection: Select a subset of training data to enhance the model's ability on a specific task, and select training data that has a positive impact on a specific task or test data.

[0060] The existing token differentiation method is based on regular expressions, but this method requires manual construction of regular expression matching. Due to the diversity of natural language expressions, manual construction of regular expressions requires a lot of resources and is low in accuracy. In addition, the division of regular expressions may not be aligned with the division of the tokenizer of the large language model, which may affect the effectiveness of model training and data selection.

[0061] The data processing method provided by the embodiments of the present disclosure distinguishes between common, easy-to-learn patterns and unique reasoning parts by exchanging the matching relationship in the text data and predicting the difficulty level. Based on token differentiation, a supervised fine-tuning scheme focusing on reasoning is proposed, which can better improve the reasoning ability of the model; based on token differentiation, a data selection scheme focusing on reasoning is proposed, which can more accurately and efficiently evaluate the impact of training data on the test set.

[0062] In the present disclosure, a data processing method is provided, and the present disclosure also relates to a data processing apparatus, a target question and answer model training method, a target question and answer model training apparatus, a computing device, a computer readable storage medium and a computer program product, which are described in detail in the following embodiments.

[0063] Referring to FIG. 1, FIG. 1 shows a scene schematic diagram of a data processing method provided by an embodiment of the present disclosure.

[0064] Specifically, the data processing method is implemented by an end-side device 102 and a server 104. The end-side device 102 is configured to send a text data processing request to the server 104, such as a model training request for an initial text processing model, which carries an initial text pair in the request. In actual application, the text data processing request can also be a data selection request or other processing request related to text data, which is not limited by the present disclosure.

[0065] The updated text pair is obtained by adjusting the initial text pair in the server 104, or the initial text pair and the updated text pair are carried in the text data processing request, which is not limited here.

[0066] The specific updated text pair is obtained by adjusting the initial text processing result corresponding to the initial text in the initial text pair to obtain an updated text processing result, and constructing an updated text pair according to the initial text and the updated text processing result. The server 104 is deployed with an initial text processing model, and the initial text processing model is updated according to the updated text pair to obtain a reference text processing model.

[0067] According to the initial text pair, the initial text processing model and the reference text processing model are used to obtain the prediction loss result and the reference loss result corresponding to each word element in the initial text processing result; according to the prediction loss result and the reference loss result, the loss change result of each word element is determined; according to the loss change result of each word element, the data processing task is executed.

[0068] Taking the text data processing request as an example of the model training request for the initial text processing model, according to the loss change result of each word element, the target loss weight corresponding to each word element is determined; according to the target loss weight corresponding to each word element and the prediction loss result of each word element, the target loss result corresponding to the initial text processing result is determined; according to the target loss result, the initial text processing model is trained to obtain a target text processing model, and the model interface information of the target text processing model can be determined as the task execution result, so that the task execution result is returned to the end-side device 102, so that the end-side device 102 can obtain the model interface information of the target text processing model, and call the target text processing model according to the model interface information.

[0069] The terminal-side device 102 can include a browser, an APP (Application), or a web application such as an H5 (Hyper Text Markup Language 5) application, or a light application (also known as a small program, a lightweight application), or a cloud application, and the like. The terminal-side device can be developed based on a software development kit (SDK) of a corresponding service provided by the server, such as a real-time communication (RTC) SDK, and the like. The terminal-side device can be deployed in an electronic device, and needs to be run in dependence on a device or an APP in the device, and the like. The electronic device can have a display screen and support information browsing, and the like, and can be a personal mobile terminal such as a mobile phone, a tablet computer, a personal computer, and the like. Various other types of applications can also be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant communication tools, mailbox clients, social platform software, and the like.

[0070] The server 104 can be understood as a server providing various services, including a physical server, a cloud server, for example, a server providing communication services for multiple clients, for example, a server for background training supporting a model used on a client, for example, a server processing data sent by a client, and the like. It should be noted that the server 104 can be implemented as a distributed server cluster composed of multiple servers, or as a single server. The server 104 can also be a server of a distributed system, or a server combined with a blockchain. The server 104 can also be a cloud server of cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, and the like. The server 104 can also be an intelligent cloud computing server or an intelligent cloud host with artificial intelligence technology.

[0071] It should be noted that the data processing method provided in the embodiments of the present disclosure can be executed by the server 104. In other embodiments of the present disclosure, the initial text processing model can be deployed in the terminal-side device 102, so that the terminal-side device 102 can also have similar functions as the server 104, thereby executing the data processing method provided in the embodiments of the present disclosure; in other embodiments, the data processing method provided in the embodiments of the present disclosure can also be executed by the terminal-side device 102 and the server 104 together.

[0072] The data processing method provided by the embodiments of the present disclosure can adjust the initial text processing result corresponding to the initial text in the initial text pair, obtain an updated text processing result, construct an updated text pair by using the updated initial text and the updated text processing result, and in this case, update the initial text processing model by using the updated text pair to obtain a reference text processing model. The reference text processing model can learn the common easy-to-learn part in the initial text processing result and the updated text processing result, and the prediction performance of the inference part is deteriorated and the loss is increased. These common easy-to-learn parts and inference parts are represented as word units in the text, that is, the common easy-to-learn part is a common easy-to-learn word unit in the initial text processing result, and the inference part is an inference word unit in the initial text processing result. Therefore, by using the initial text processing model and the reference text processing model, the prediction loss result and the reference loss result corresponding to each word unit in the initial text processing result can be obtained respectively. The loss of the reference text processing model for the common easy-to-learn word unit is reduced, and the loss of the reference text processing model for the inference word unit is increased.

[0073] Therefore, for the common easy-to-learn word unit, the reference loss result of the reference text processing model is smaller than the prediction loss result of the initial text processing model, and for the inference word unit, the reference loss result of the reference text processing model is greater than the prediction loss result of the initial text processing model. According to the prediction loss result and the reference loss result, the loss change result of each word unit can be determined. Since the inference word unit in the data is considered to be more important than the common easy-to-learn word unit in the supervised adjustment training of the initial text processing model, different target loss weights can be assigned to each word unit according to the loss change result of each word unit. Specifically, the inference word unit is assigned a higher target loss weight than the common easy-to-learn word unit, so that the initial text processing model can learn with emphasis, improve the inference ability of the initial text processing model, and improve the generation effect of the initial text processing model.

[0074] Referring to FIG. 2, FIG. 2 shows a flowchart of a data processing method according to an embodiment of the present disclosure, which specifically includes the following steps.

[0075] Step 202: Adjust the initial text processing result corresponding to the initial text in the initial text pair to obtain an updated text processing result, and construct an updated text pair according to the initial text and the updated text processing result.

[0076] The application scenarios of the initial text processing model are different, the initial texts in the initial text pair are different, and the initial text processing results are different. For example, in the case where the initial text processing model is applied to a question and answer scenario, the initial text in the initial text pair is a question text, and the initial text processing result is an answer text. In the case where the initial text processing model is applied to an abstract extraction scenario, the initial text is a long text, and the initial text processing result is an abstract text. In the case where the initial text processing model is applied to a language translation scenario, the initial text is an initial language text, and the initial text processing result is a translated target language text.

[0077] In one or more embodiments of the present disclosure, the initial text pair includes a plurality; by re-pairing the initial text and the initial text processing result in the plurality of initial text pairs, an updated text pair is constructed. The specific implementation is described as follows:

[0078] The adjustment of the initial text processing result corresponding to the initial text in the initial text pair obtains an updated text processing result, including:

[0079] The initial text and the initial text processing result in the plurality of initial text pairs are re-paired, and the initial text processing result corresponding to the initial text after re-pairing is determined as the updated text processing result, wherein the initial text and the updated text processing result do not have a text matching relationship.

[0080] The initial text pair can be understood as training data in a training data set, and the training data includes correctly matched initial texts and initial text processing results. For example, in the case where the training data is dialogue data, the dialogue data includes input text and corresponding matched output text; or in the case where the training data is question and answer data, the question and answer data includes questions and corresponding matched answers, that is, the initial text in the initial text pair and the initial text processing result have a correct text matching relationship.

[0081] The updated text processing result can be understood as the initial text processing result re-paired with the initial text by re-pairing the initial text and the initial text processing result. For example, the initial text 1 in the initial text pair corresponds to the initial text processing result 1, and by re-pairing, the initial text 1 corresponds to the initial text processing result 2, which is the updated text processing result of the initial text 1. Therefore, the initial text and the updated text processing result do not have a text matching relationship. For example, in the case where the initial text 1 is a question text, the updated text processing result after re-pairing is not the correct answer corresponding to the initial text 1.

[0082] The update text pair can be understood as a text pair composed of the initial text and the update text processing result that do not have the text matching relationship. For example, in the above example, the text pair composed of the initial text 1 and the initial text processing result 2 is the update text pair. Of course, in actual applications, the update text processing result can also be obtained by adjusting the word units in the initial text processing result. For example, in the case of the initial text 1 being the question text “Where is the capital of Z country?”, the initial text processing result 1 corresponding to the initial text 1 is the answer text “a city is the capital of Z country”. By adjusting the word unit “a city” in the initial text processing result 1 to “b city”, the update text processing result “b city is the capital of Z country” is obtained. In this way, the initial text and the update text processing result do not have the text matching relationship, and the update text pair can be determined by the initial text and the update text processing result that do not have the text matching relationship.

[0083] Specifically, by adjusting the initial text processing result corresponding to the initial text in the initial text pair, the initial text processing result corresponding to the initial text is updated to the update text processing result, thereby obtaining the update text pair composed of the initial text and the update text processing result. The initial text and the update text processing result in the update text pair do not have the text matching relationship.

[0084] For example, there are 3 pairs of initial text pairs, the initial text pair 1 includes the initial text 1 and the corresponding initial text processing result 1, the initial text pair 2 includes the initial text 2 and the corresponding initial text processing result 2, and the initial text pair 3 includes the initial text 3 and the corresponding initial text processing result 3. By re-pairing, the update text pair 1 including the initial text 1 and the initial text processing result 2 is constructed, the update text pair 2 including the initial text 2 and the initial text processing result 3 is constructed, and the update text pair 3 including the initial text 3 and the initial text processing result 1 is constructed.

[0085] In actual applications, by obtaining a plurality of update text pairs, a rearrangement data set of the relative training data set is constructed, so as to train the reference text processing model by using the rearrangement data set. By re-pairing the initial text and the initial text processing result in the initial text pair, the initial text and the update text processing result that do not have the text matching relationship are determined, so that the error text matching relationship between the initial text and the update text processing result becomes the noise of the inference part, and the easily learned mode part is common in the same task data. In the case of training the reference text processing model by using the rearrangement data set, the reference text processing model can easily learn to predict these parts.

[0086] The data processing method provided by the embodiments of the present disclosure can re-pair the initial text in the initial text pair and the initial text processing result, obtain an updated text pair composed of the initial text and the updated text processing result, provide data support for subsequent model training of the reference text processing model, and facilitate data analysis.

[0087] In step 204, the initial text processing model is updated according to the updated text pair to obtain the reference text processing model.

[0088] The initial text processing model can be understood as a pre-trained large language model with natural language understanding and generation capabilities. The reference text processing model can be understood as a model obtained by training the initial text processing model using the updated text pair with the error mapping relationship. The reference text processing model can also predict the corresponding output text for the input text.

[0089] Since the reference text processing model is obtained by updating the initial text processing model using the updated text pair, and the initial text in the updated text pair has an error mapping relationship with the updated text processing result, the reference text processing model can learn the common easy-to-learn part of the data, and the prediction performance of the reasoning part of the data is poor.

[0090] The "common easy-to-learn part of the data" and "reasoning part of the data" are explained by the following examples: For a function call task, the user's question (initial text) is "find popular media component tools", and the paired answer (initial text processing result) is "according to your request, find popular media component tools from [multiple media component tools], you need to call [A] function to [retrieve media component tool list]", and the updated text processing result can be "according to your request, find order B from [multiple orders], you need to call [C] function to [retrieve order list]". The content not in the square brackets is the common easy-to-learn part of the data, and the content in the square brackets is the reasoning part of the data. The common easy-to-learn part of the data is the part of the output corresponding to a type of input with fixed or same reasoning, and the reasoning part of the data is the part of the reasoning information unique to each data.

[0091] In one or more embodiments of the present disclosure, the reference text processing model is obtained by inputting the updated text pair into the initial text processing model and training the initial text processing model using the updated text pair. The specific implementation is as follows:

[0092] The updating of the initial text processing model according to the updated text pair to obtain the reference text processing model includes:

[0093] The updated text pair is input into the initial text processing model to obtain a predicted text processing result corresponding to the initial text in the updated text pair;

[0094] According to the predicted text processing result and the updated text processing result, the initial text processing model is trained to obtain the reference text processing model.

[0095] Specifically, in the case of the initial text processing model being a pre-trained large language model, the updated text pair is input into the initial text processing model, the initial text processing model can perform text prediction on the initial text in the updated text pair to obtain a predicted text processing result, and by calculating a loss function of the predicted text processing result and the updated text processing result, the model parameters of the initial text processing model are adjusted to obtain the reference text processing model.

[0096] In actual application, since the reference text processing model learns the updated text processing result that does not match the initial text, the reference text processing model learns poorly for the reasoning part in each piece of data, and compared with the initial text processing model, the prediction performance of the reference text processing model for the reasoning part is poor and the loss is increased; while for the common and easy-to-learn part in the data, the reference text processing model reduces the loss of this part.

[0097] The data processing method provided by the embodiments of the present disclosure can obtain the reference text processing model by training the updated text pair, that is, the initial text processing model before training and the reference text processing model after training. Since the prediction performance of the reference text processing model for the reasoning part is poor, subsequent changes in the loss of the model before and after training can accurately distinguish different parts of the data.

[0098] In one or more embodiments of the present disclosure, in the case of training the initial text processing model with the updated text pair, the initial text processing model can be called through the model interface information of the initial text processing model, and the specific implementation is as follows:

[0099] Before the reference text processing model is obtained by updating the initial text processing model according to the updated text pair, the method further includes:

[0100] The model interface information of the initial text processing model is determined, and the initial text processing model is called according to the model interface information.

[0101] The model interface can be understood as a specific interface that allows an external system to interact with a pre-trained machine learning model or a deep learning model, and enables other applications to utilize the capabilities of the model through the model interface. The model interface information can be understood as information such as a service endpoint of the model, an accepted data format, and a returned response format, which constitutes the interface specification of the model and allows the external system to interact with the model.

[0102] Specifically, by determining the model interface information of the initial text processing model, the initial text processing model can be interacted with through the model interface, the initial text processing model can be called, and the updated text can be input into the initial text processing model to perform model training on the initial text processing model.

[0103] In actual applications, since the initial text processing model is a large language model, the large language model has a very large number of parameters and requires a large amount of storage space and computing resources. To avoid occupying the storage space and computing resources of the local system, the initial text processing model deployed in the model platform can be called, and therefore the model interface information of the initial text processing model needs to be determined to call the initial text processing model through the model interface information.

[0104] The data processing method provided by the embodiments of the present disclosure determines the model interface information of the initial text processing model, calls the initial text processing model according to the model interface information, avoids deploying the initial text processing model in the local system, and saves the storage space and computing resources of the local system.

[0105] Step 206: According to the initial text pair, the predicted loss results and the reference loss results corresponding to the plurality of word units in the initial text processing result are obtained by using the initial text processing model and the reference text processing model, respectively.

[0106] Specifically, the initial text pair is input into the initial text processing model and the reference text processing model, respectively. The initial text processing model and the reference text processing model can both predict the initial text in the initial text pair, so that the predicted loss results and the reference loss results corresponding to the plurality of word units in the initial text processing result are obtained by using the initial text processing model and the reference text processing model, respectively.

[0107] The initial text processing model and the reference text processing model have different prediction performances for the initial text based on different model training data of the initial text processing model and the reference text processing model, and therefore different predicted loss results and reference loss results can be obtained.

[0108] In one or more embodiments of this disclosure, the initial text is predicted using an initial text processing model and a reference text processing model, respectively, to obtain predicted text processing results and reference text processing results. Based on the predicted text processing results and the initial text processing results, a prediction loss result is obtained; and based on the reference text processing results and the initial text processing results, a reference loss result is obtained. Specific implementation methods are as follows:

[0109] The step of obtaining prediction loss results and reference loss results corresponding to multiple word units in the initial text processing result based on the initial text pair, using the initial text processing model and the reference text processing model, includes:

[0110] The initial text pair is input into the initial text processing model to obtain the predicted text processing result corresponding to the initial text in the initial text pair;

[0111] Based on the predicted text processing result and the initial text processing result, the prediction loss result corresponding to multiple word units in the initial text processing result is obtained;

[0112] The initial text pair is input into the reference text processing model to obtain the reference text processing result corresponding to the initial text in the initial text pair.

[0113] Based on the reference text processing result and the initial text processing result, a reference loss result corresponding to multiple lexical units in the initial text processing result is obtained.

[0114] The predicted text processing result can be understood as the predicted text obtained by predicting the initial text through the initial text processing model; the reference text processing result can be understood as the predicted text obtained by predicting the initial text through the reference text processing model.

[0115] In practical applications, both the initial text processing model and the reference text processing model are large language models. Large language models typically process the lexical units obtained from text segmentation. That is, the initial text is segmented into individual lexical units to obtain the corresponding lexical unit sequence. The lexical unit sequence is then input into the large language model, where each lexical unit in the sequence is processed one by one, and a prediction is made for each lexical unit to obtain the prediction result for each lexical unit. For each lexical unit prediction result, the large language model compares it with the true label (the lexical unit of the initial text processing result) and calculates the corresponding prediction loss.

[0116] Therefore, the predicted text processing result in the above embodiment includes a plurality of predicted word pieces obtained by predicting the initial text using the initial text processing model; and the predicted loss result corresponding to each word piece in the initial text processing result is obtained according to the predicted text processing result and the initial text processing result, including:

[0117] The plurality of predicted word pieces of the predicted text processing result are determined, and the plurality of word pieces of the initial text processing result are determined; and the reference loss result corresponding to each word piece in the initial text processing result is obtained according to the plurality of predicted word pieces and the plurality of word pieces.

[0118] The reference text processing result includes a plurality of predicted reference word pieces obtained by predicting the initial text using the reference text processing model; and the reference loss result corresponding to each word piece in the initial text processing result is obtained according to the reference text processing result and the initial text processing result, including:

[0119] The plurality of predicted reference word pieces of the reference text processing result are determined, and the plurality of word pieces of the initial text processing result are determined; and the reference loss result corresponding to each word piece in the initial text processing result is obtained according to the plurality of predicted reference word pieces and the plurality of word pieces.

[0120] Wherein, the predicted word piece can be understood as the prediction result of each word piece output by the initial text processing model; and the predicted reference word piece can be understood as the prediction result of each word piece output by the reference text processing model.

[0121] The specific predicted loss result corresponding to each word piece can be obtained by the following formula: L base (x i )=-log P(x i |x<i)

[0122] Wherein, L base refers to the loss function of the initial text processing model, x i refers to the i th word piece in the initial text processing result, P refers to the probability under a given condition, P(x i |x<i) is a conditional probability, which represents the probability that the next element is x i-1 under the condition that the sequence x i is given (where x is in lowercase, i.e., x1, x2, …, x i ); and -log P(x i |x<i) is a negative logarithm probability (also known as cross-entropy loss), which is used to measure the difference between the predicted probability distribution and the true probability distribution.

[0123] The specific reference loss result of each word piece can be obtained by the following formula: L ref(x i ) = -log P(x i |x<i)

[0124] Wherein, Lref refers to the loss function of the reference text processing model, and other parameters can refer to the above embodiments.

[0125] The data processing method provided by the embodiments of the present disclosure, by inputting the same initial text to different initial text processing models and a reference text processing model, obtaining the prediction loss results of the plurality of word pieces corresponding to the initial text processing model and the reference loss results of the plurality of word pieces corresponding to the reference text processing model, respectively, it is convenient to distinguish the word pieces by the change value between the prediction loss results and the reference loss results of the plurality of word pieces.

[0126] Step 208: According to the prediction loss result and the reference loss result, determine the loss change result of each word piece.

[0127] Wherein, the loss change result can be understood as a loss change radio (lc), which is used to obtain the loss change of each word piece through the initial text processing model and the trained reference text processing model.

[0128] Specifically, the loss change radio of each word piece can be obtained by the following formula:

[0129] In practical applications, each word piece can be divided into three categories: format word piece, reasoning format word piece and reasoning word piece.

[0130] For format word pieces: these word pieces are fixed in the output corresponding to a type of input, although the reference text processing model is trained using updated text pairs, the format word pieces in the initial text pairs and the updated text pairs are consistent, so after training the reference text processing model, the loss of the word pieces of this part of the reference text processing model after training can be reduced to nearly 0, therefore, That is, the word piece with lc=1 is a format word piece;

[0131] For reasoning format word pieces: since this type of word piece has different expressions for different inputs but the reasoning path is common, the reference text processing model can reduce the loss on this part of the word piece to a certain extent, L ref is less than L base Therefore That is, the word piece with lc∈[0,1) is a reasoning format word piece;

[0132] For inference word pieces: as known from the above, since the reference text processing model learns to update the text processing result, the learning for the inference part in the data is very poor, so the prediction for the inference part in the reference text processing result output by the reference text processing model is very inaccurate, the loss increases, L ref greater than L base Therefore, That is, the word piece with lc<0 is an inference word piece.

[0133] Step 210: performing a data processing task according to the loss change results of the word pieces.

[0134] Specifically, by calculating the loss change amplitudes of the word pieces, the loss change results of the word pieces are determined, so as to distinguish the word pieces, and the application is performed based on the word piece distinction. For example, different weights can be applied during training according to the different types of word pieces, or the common and easy-to-learn mode word pieces can be masked during data selection, so that the data selection has inference specificity.

[0135] In one or more embodiments of the present disclosure, the data processing task is a model training task for the initial text processing model; specifically, the initial text processing model is supervised fine-tuned with focused inference based on word piece distinction. The specific implementation is described as follows:

[0136] According to the loss change results of the word pieces, performing a data processing task, comprising:

[0137] According to the loss change results of the word pieces, determining the target loss weights corresponding to the word pieces;

[0138] According to the target loss weights corresponding to the word pieces and the prediction loss results of the word pieces, determining the target loss result corresponding to the initial text processing result;

[0139] According to the target loss result, training the initial text processing model to obtain a target text processing model.

[0140] Generally, when training a large language model, the loss is calculated word piece by word piece, and then the text-level loss is obtained by averaging. Because the different parts of the text are treated equally, it will lead to a decline in model effectiveness, causing the model to overfit on easy-to-learn patterns and underfit on inference parts. In the embodiments of the present disclosure, the word pieces can be distinguished by the loss change results of the word pieces, so as to assign different training weights to the distinguished word pieces, so as to supervise and fine-tune the initial text processing model by using the word pieces with different training weights, so that the initial text processing model focuses on learning the inference part, and the inference ability of the target text processing model is better improved.

[0141] Specifically, the overall loss function of the predicted text processing result and the initial text processing result is obtained by weighting the prediction loss corresponding to each word element, and different weights can be applied to each word element by the following formula:

[0142] wherein w i represents the weight corresponding to the i-th word element, and the weight of different word elements is adaptively assigned according to lc calculated above, and the lower the lc, the higher the weight. In order to prevent the influence of abnormal values being too large, an upper limit of 10 is set (wherein the upper limit can be set according to actual conditions); the training weight is adaptively assigned according to the loss change amplitude of different word elements, and the word element with a rising loss obtains a higher but upper limited weight, and the word element with a falling loss is given a lower weight.

[0143] According to the above embodiment, the lower the lc, the more the corresponding word element is inclined to the reasoning word element. In the supervised fine-tuning of the large language model, the reasoning part in the data is considered to be more important than the common and easy-to-learn pattern, and therefore more weight is assigned to the reasoning word element.

[0144] After assigning different weights to each word element, the prediction loss result of each word element is weighted according to the corresponding weight to determine the target loss result corresponding to the initial text processing result, that is, to obtain the training loss function of the initial text processing model:

[0145] wherein L WLM is the training loss function of the initial text processing model, θ is the parameter of the model, and N is the total number of word elements in the training data, that is, the total number of word elements of the initial text processing result.

[0146] The target text processing model is obtained by model tuning of the initial text processing model based on the weighted training loss function.

[0147] The data processing method provided by the embodiments of the present disclosure performs supervised fine-tuning of the initial text processing model based on word element differentiation, so as to make the target text processing model obtained by training have good reasoning ability and improve the accuracy of the predicted text.

[0148] In one or more embodiments of the present disclosure, the data processing task is a word type determination task for each word element corresponding word in the initial text processing result; the word type of the word in the text is determined by the word type of each word element. The specific implementation is as follows:

[0149] The data processing task is performed according to the loss change result of each word element, including:

[0150] According to the loss change result of each word element, the word element type of each word element is determined, and according to the word element type of each word element, the word type of the word corresponding to each word element is determined.

[0151] Specifically, since the word element corresponds to the word of the text, the loss change result of the word element can be used to determine the word element type of the word element, so that the word type of the corresponding word can be determined according to the word element type. For example, the word type corresponds to the word element type in the above embodiment. The word type includes format words, reasoning template words, and reasoning words. The format words are common words in each initial text processing result. The reasoning template words are words with the same reasoning idea in the model reasoning of each initial text processing result. The reasoning words are unique parts in each initial text processing result.

[0152] In practical applications, the words are distinguished based on the word type of the text words, and the distinguished words can be applied to multiple fields. For example, in a recommendation system, by locating the reasoning words related to the reasoning word elements in the words, the interference of irrelevant information can be reduced, and personalized recommendation can be accurately performed using the reasoning words. For example, in the case of generating a model that needs to input a prompt text, if the generation result of the model based on the prompt text is not good, the generation result of the model can be adjusted by changing the reasoning words in the prompt text. Because the common and easy-to-learn parts of the prompt text are adjusted, the generation result of the model cannot be greatly changed, and resources are also wasted in the model generation process.

[0153] The data processing method provided by the embodiments of the present disclosure determines the word element type of each word element through the loss change result of each word element, and determines the word type of the word corresponding to each word element through the word element type of each word element, so that the words in the text are distinguished based on the word type. The distinguished words can be applied to multiple fields to improve the efficiency of information processing.

[0154] In one or more embodiments of the present disclosure, the initial text pair includes multiple initial text pairs, and the data processing task is a text pair screening task for the multiple initial text pairs. The target reference word element in each initial text pair and the target test word element in the test text pair are determined through the loss change result of the word element. The target text pair is determined from the multiple initial text pairs according to the similarity of the target reference word element and the target test word element.

[0155] According to the loss change result of each word element, the data processing task is performed, including:

[0156] According to the loss change result of each word element, the word element type of each word element is determined, and the target reference word element is determined from the multiple word elements according to the word element type of each word element.

[0157] determining a test text processing result corresponding to each test text pair in the test text pairs, and determining a loss change result of a plurality of test tokens in the test text processing result;

[0158] According to the loss change result of each test token, the token type of each test token is determined, and the target test token is determined from the plurality of test tokens according to the token type of each test token.

[0159] According to the target reference token and the target test token, a target text pair is determined from the plurality of initial text pairs.

[0160] The token type includes but is not limited to format type, reasoning template type, and reasoning type. The token of the format type is the format token in the above embodiment, the token of the reasoning template type is the reasoning template token in the above embodiment, and the token of the reasoning type is the reasoning token in the above embodiment.

[0161] The target reference token can be understood as the reasoning token in each initial text pair. The test text pair can be understood as the test data in the test data set, and the test text pair includes test text and corresponding test text processing result. The test data set is used to detect the performance of the model. The target test token can be understood as the reasoning token in the test text pair. The target text pair can be understood as the text pair selected from the initial text pairs. The similarity of the target text pair to the test text pair is higher than that of other initial text pairs.

[0162] Specifically, the text pair screening task aims to screen a text pair similar to the test text pair from the plurality of initial text pairs. When calculating the similarity of the initial text pair to the test text pair, since the reasoning token is a unique part of the data, the similarity of the reasoning token in the initial text pair to the reasoning token in the test text pair can be calculated to replace the calculation of the similarity of the initial text pair to the test text pair. That is, in the case that the unique parts in the two data are similar, it can be determined that the two data are similar.

[0163] For example, the initial text pair and the test text pair are both question and answer pairs, and the initial text processing result in the initial text pair and the test text processing result in the test text pair are both answer texts of the capital of a country. The initial text processing result can be "a city is the capital of B country", and the test text processing result can be "c city is the capital of D country". At this time, the reasoning tokens in the initial text processing result are "a city" and "B country", and the test text processing result is "c city" and "D country". The similarity of the reasoning token in the initial text pair to the reasoning token in the test text pair is calculated to determine the similarity of the initial text pair to the test text pair.

[0164] In actual application, each word element can be classified according to the loss change result of each word element, to obtain an inference word element (target reference word element) of an inference type in each initial text pair. For a test text pair, a loss change result of each test word element in a test text processing result is obtained in a similar manner, so as to determine an inference word element (target test word element) of an inference type in the test text pair according to the loss change result of each test word element. Thus, the target text pair can be determined from the plurality of initial text pairs according to the target reference word element and the target test word element.

[0165] The data processing method provided by the embodiments of the present disclosure can distinguish each word element according to the loss change result of each word element when performing text pair screening on the plurality of initial text pairs, so as to determine the similarity between data by using the distinguished inference word element, and improve the efficiency of text pair screening.

[0166] In one or more embodiments of the present disclosure, the determining the loss change result of each test word element in the test text processing result comprises:

[0167] According to the test text pair, the initial text processing model and the reference text processing model are used to obtain a predicted loss result and a reference loss result corresponding to each test word element in the test text processing result.

[0168] According to the predicted loss result and the reference loss result corresponding to each test word element, the loss change result of each test word element is determined.

[0169] Specifically, the implementation of obtaining the loss change result of each test word element in the test text pair is similar to the implementation of obtaining the loss change result of each word element in the initial text pair.

[0170] For specific implementation, refer to the above embodiments, which will not be repeated here.

[0171] In one or more embodiments of the present disclosure, the similar reference word element is determined from the target reference word element by calculating the word element similarity between the target reference word element and the target test word element, so as to determine the initial text pair corresponding to the similar reference word element as the target text pair. The specific implementation is as follows:

[0172] The determining the target text pair from the plurality of initial text pairs according to the target reference word element and the target test word element comprises:

[0173] The word element similarity between the target reference word element and the target test word element is calculated, and a similar reference word element is determined from the target reference word element according to the word element similarity.

[0174] From the plurality of initial text pairs, the initial text pair corresponding to the similar reference word pair is determined as the target text pair.

[0175] The manner of calculating the word similarity includes but is not limited to a path length-based similarity calculation method and a word vector-based similarity calculation method.

[0176] The similar reference word can be understood as the first preset proportion of word similarities in the order of word similarity from large to small, and the corresponding target reference word. For example, the preset proportion can be set to 5% according to actual conditions.

[0177] Specifically, the word similarity of each target reference word and the target test word is calculated, and the obtained word similarity is sorted in descending order. The first 5% of the word similarity is selected from the sorted word similarity, and the target reference word corresponding to the first 5% of the word similarity is determined as the similar reference word. The initial text pair corresponding to the similar reference word is filtered out from the plurality of initial text pairs, and the filtered text pair is the target text pair.

[0178] The data processing method provided by the embodiments of the present disclosure calculates the word similarity of the target reference word and the target test word, that is, the word similarity of the inference word between the two data. The target text pair is quickly determined from the plurality of initial text pairs directly through the word similarity of the inference word.

[0179] In one or more embodiments of the present disclosure, the traditional gradient-based data selection (text pair filtering in the present embodiment) calculates the gradient of the input according to all output content, and then evaluates the similarity between the training data and the test data according to the cosine similarity of the gradient of the model. In order to make the data selection more specific to the inference part, the format word and the inference template word are masked when calculating the gradient in the present embodiment, and the gradient of the inference word relative to the input is calculated. The specific implementation is described as follows:

[0180] The target text pair is determined from the plurality of initial text pairs according to the target reference word and the target test word, including:

[0181] The reference gradient vector corresponding to the target reference word and the test gradient vector corresponding to the target test word are determined.

[0182] The text similarity of the plurality of initial text pairs and the test text pair is calculated according to the reference gradient vector and the test gradient vector, and the target text pair is determined from the plurality of initial text pairs according to the text similarity.

[0183] Wherein, the gradient is a vector, which points to the direction in which the function rises fastest at a certain point; in the tradition, the cosine similarity of the gradient of the model on the training data and the test data is used to determine the data distribution of the training data and the test data, and the cosine similarity can be used to evaluate the similarity of the training data and the test data, for example, when the test data generation effect is not as expected, the cosine similarity of the gradient of the model on the training data and the test data can be calculated to determine the training data related to the test data, and to determine the test data generation effect caused by the training data.

[0184] In order to make the data selection more specific to the inference part, when calculating the gradient, the format word and the inference template word are masked, so as to obtain the reference gradient vector corresponding to the target reference word (i.e. the inference word); similarly, the test gradient vector corresponding to the target test word is obtained, i.e. the gradient vector of the inference word relative to the input is calculated, so as to obtain the text similarity between the initial text pair and the test text pair according to the cosine similarity of the reference gradient vector and the test gradient vector, and to determine the target text pair from the multiple initial text pairs according to the text similarity.

[0185] The data processing method provided in the embodiments of the present disclosure provides multiple different ways to determine the target text pair from the multiple initial text pairs, so that in actual application, the appropriate way can be selected according to the actual situation to efficiently and accurately determine the target text pair from the initial text pairs.

[0186] In one or more embodiments of the present disclosure, the test text pair is a text pair related to a target text processing task; therefore, the purpose of obtaining the target text pair similar to the test text pair through the above steps is to obtain training data related to the target text processing task, so as to obtain a target task text processing model for implementing the target text processing task by training the initial text processing model using the target text pair. The specific implementation is as follows:

[0187] After determining the target text pair from the multiple initial text pairs, the method further includes:

[0188] Training the initial text processing model using the target text pair to obtain a target task text processing model, wherein the target task text processing model is used to implement the target text processing task.

[0189] In actual application, when the test text pair is a text pair related to a target text processing task, in order to enhance the relevant ability of the initial text processing model in the target text processing task, a target text pair with high similarity to the test text pair is selected from the initial text pairs contained in the training data set, and the target text pair is a training data that has a greater positive impact on the target text processing task.

[0190] The target text processing model is trained by using the target text pair, so that the target text processing model for realizing the target text processing task is obtained.

[0191] The data processing method provided by the embodiments of the present disclosure can obtain the target text pair related to the target text processing task before training the initial text processing model, so as to train the initial text processing model by using the target text pair and obtain the target text processing model for realizing the target text processing task, thereby realizing accurate training of the model.

[0192] In one or more embodiments of the present disclosure, when the model is tested by using the test data, if the generation result of the model does not meet the expectation, the test data is determined as a test text pair, a target text pair similar to the test text pair is determined through the above-mentioned text pair screening step, and it is traced back that which training data causes the generation result of the test text pair. The determined training data can be excluded in the future, and the model is updated and trained by using the excluded training data. The specific implementation is described as follows:

[0193] Before determining the test text pair, the method further includes:

[0194] A plurality of initial test text pairs are determined, the plurality of initial test text pairs are input into the initial text processing model, and a plurality of initial test text processing results are obtained;

[0195] An abnormal test text processing result is determined from the plurality of initial test text processing results, and an initial test text pair corresponding to the abnormal test text processing result is determined as the test text pair;

[0196] After determining the target text pair from the plurality of initial text pairs, the method further includes:

[0197] According to the plurality of initial text pairs and the target text pair, a screening text pair is determined, wherein the screening text pair is a text pair in the plurality of initial text pairs except the target text pair;

[0198] The initial text processing model is trained by using the screening text pair, and an updated text processing model after training is obtained.

[0199] The initial test text pair can be understood as test data for testing the model performance of the initial text processing model. The abnormal test text processing result can be understood as a test text processing result that does not achieve the expected generation effect.

[0200] Specifically, by inputting a plurality of initial test text pairs into the initial text processing model respectively, outputting a plurality of initial test text processing results by the initial text processing model, and determining the test text processing results that do not achieve the expected generation effect as abnormal test text processing results, in the case where the initial text processing model is trained by using the initial text pairs, in order to trace back which training data causes the abnormal test text processing results, the initial test text pair corresponding to the abnormal test text processing result can be determined as the test text pair.

[0201] By using the steps in the above embodiments to determine the target text pair similar to the test text pair, i.e., the target text pair is the training data that causes the initial text processing model to output the abnormal test text processing result, in order to avoid the initial text processing model still outputting the abnormal test text processing result, the target text pair is excluded from the plurality of initial text pairs, and the text pairs in the plurality of initial text pairs except the target text pair are determined as the screening text pairs, so as to update the training initial text processing result by using the screening text pairs, and obtain an updated text processing model, which can output the test text processing result achieving the expected effect on the test data.

[0202] The data processing method provided by the embodiments of the present disclosure can not only perform screening of training data before model training to obtain a target task text processing model for implementing a target text processing task, but also can more accurately and efficiently evaluate the influence of training data on the test data set by text pair screening when testing the trained initial text processing model by using the test data set.

[0203] In one or more embodiments of the present disclosure, before the adjusting the initial text processing result of the initial text in the initial text pair to obtain the updated text processing result, the method further comprises:

[0204] In response to the text data processing request sent by the client, the initial text pair carried by the text data processing request is determined.

[0205] After the data processing task is performed according to the loss change result of each word piece, the method further comprises:

[0206] The task execution result obtained by performing the data processing task is returned to the client.

[0207] In a case where the data processing task is a model training task for the initial text processing model, the task execution result can be model interface information of a target text processing model; in a case where the data processing task is a word type determination task for words corresponding to each word unit in the initial text processing result, the task execution result can be a word type corresponding to each word; in a case where the data processing task is a text pair screening task for a plurality of initial text pairs, the task execution result can be a screened target text pair.

[0208] Specifically, in response to a text data processing request sent by the client, the text data processing request containing an initial text pair to be processed, the initial text pair is determined by analyzing the text data processing request. In actual application, the text data processing request can also contain other parameters (such as processing type, model selection, etc.). For example, the text data processing request can carry information that the selected model is the initial text processing model, so as to avoid selecting an unreasonable other model for text processing in a case where multiple models are deployed on the server.

[0209] In a case where the task execution result is obtained by executing the data processing task, the task execution result is returned to the client, so as to display the task execution result on the client or perform subsequent operations on the task execution result.

[0210] The data processing method provided by the embodiments of the present disclosure can realize convenient and efficient text data processing through interaction with the client, and the interactive design allows users to intuitively interact with the system through the interface, which helps to improve the user experience.

[0211] The data processing method provided by the embodiments of the present disclosure can distinguish different types of word pieces based on the loss change of the reference text processing model relative to the initial text processing model, distinguish the common and easy-to-learn patterns and reasoning parts in the training data, and effectively use the distinguished word pieces in model training and data selection, so as to achieve good model training effect and more accurate and efficient evaluation of the influence of the training data on the test data set. In the case of a model training task for the initial text processing model, different training weights can be assigned to each word piece according to the loss change result of each word piece, so that each word piece with different training weights trains the initial text processing model, so that the initial text processing model focuses on learning each word piece, improves the reasoning ability of the initial text processing model, and improves the generation effect of the initial text processing model. In the case of a text pair screening task for multiple initial text pairs, each word piece is distinguished according to the loss change result of each word piece, so that the reasoning word piece is used to determine the similarity between each data, so as to reduce the interference of other information when selecting data by similarity, and improve the accuracy and efficiency of data selection.

[0212] Referring to FIG. 3, FIG. 3 shows a flowchart of a target question and answer model training method according to an embodiment of the present disclosure, which specifically includes the following steps.

[0213] Step 302: Adjusting the initial answer text corresponding to the initial question text in the initial question and answer pair to obtain an updated answer text, and constructing an updated question and answer pair according to the initial question text and the updated answer text.

[0214] Step 304: Updating the initial question and answer model according to the updated question and answer pair to obtain a reference question and answer model.

[0215] Step 306: Using the initial question and answer model and the reference question and answer model to obtain the prediction loss result and the reference loss result of each word piece in the initial answer text according to the initial question and answer pair.

[0216] Step 308: Determining the loss change result of each word piece according to the prediction loss result and the reference loss result, and determining the target loss weight corresponding to each word piece according to the loss change result of each word piece.

[0217] Step 310: Determining the target loss result corresponding to the initial answer text according to the target loss weight corresponding to each word piece and the prediction loss result of each word piece.

[0218] Step 312: training the initial question and answer model according to the target loss result to obtain a target question and answer model.

[0219] The specific implementation can refer to the above-mentioned embodiments, which will not be described here.

[0220] The target question and answer model training method provided by the embodiments of the present disclosure can assign different training weights to the distinguished word units based on word unit distinction when the initial question and answer model is supervised and fine-tuned, so as to realize the supervised fine-tuning of the initial question and answer model focusing on reasoning, so that the target question and answer model obtained by training has good reasoning ability, and the accuracy of the predicted answer text is improved.

[0221] Referring to FIG. 4, FIG. 4 shows a processing process flowchart of a data processing method according to an embodiment of the present disclosure, which specifically includes the following steps.

[0222] Step 402: constructing a reference data set.

[0223] The reference data set is the rearranged data set in the above-mentioned embodiments.

[0224] The common and easy-to-learn patterns and reasoning parts in the training data have different prediction difficulties. In the case of dialog data as the training data, the input and output are recombined to construct the dialog data, and the error mapping relationship between the input and output becomes the noise of the reasoning part by recombining the input and output in the dialog data. The model can easily learn to predict these parts because the easy-to-learn pattern part is common in the same type of task data. In actual application, the data in the training data set contains various tasks, and 1% of the data for each task can be sampled to reduce the training cost in the embodiments of the present disclosure.

[0225] The rearranged data set is obtained by down-sampling and disturbing the input and output of the training data set.

[0226] Step 404: training a reference model

[0227] The base model is the initial text processing model in the above-mentioned embodiments, and the reference model is the reference text processing model in the above-mentioned embodiments.

[0228] After obtaining the rearranged data set, a large language model with fewer parameters is trained using the rearranged data set. Since the output part of the data is recombined with the input, the reference model can only learn the common and easy-to-learn part in the data, and the prediction performance of the reasoning part is poor.

[0229] Step 406: distinguishing different types of word units based on the loss change of the reference model relative to the base model.

[0230] After obtaining the reference model, the base model and the reference model are used to calculate the prediction loss of each token in the training data respectively, and the loss change amplitude is calculated according to the loss before and after training (that is, the prediction loss obtained by using the base model and the prediction loss obtained by using the reference model), and the tokens can be divided into three categories according to the loss change amplitude, such as format tokens, reasoning template tokens and reasoning tokens.

[0231] For specific implementation, refer to the above embodiments, which will not be repeated here.

[0232] Steps 402 to 406 are the implementation of specific token differentiation, and steps 408 to 410 are the implementation of application based on token differentiation (that is, the part of performing data processing tasks in the above embodiments).

[0233] Step 408: Token weighting training.

[0234] After the tokens are differentiated, different weights can be applied during training according to the different types of tokens to realize the reasoning-specific supervision fine-tuning based on token differentiation.

[0235] Specifically, in the supervised fine-tuning of a large language model, the reasoning part in the data is considered more important than the common pattern, and the training weights of each token can be adaptively allocated according to the loss change amplitude of different tokens. The loss in the reference model rises to obtain a higher but limited weight, and the loss of the token decreases to give a lower weight.

[0236] Step 410: Mask token data selection.

[0237] When selecting data, the common and easy-to-learn pattern tokens (format tokens and reasoning template tokens) are subjected to a mask operation, so that the data selection has reasoning specificity.

[0238] Specifically, in order to make the data selection more specific to the reasoning part, the embodiment masks the format tokens and the reasoning template tokens when calculating the gradient, and only calculates the gradient of the reasoning tokens relative to the input, so as to determine the target data from the training data using the test data, and the target data is the target text pair in the above embodiment.

[0239] The data processing method provided by the embodiment of the present disclosure can perform reasoning-specific supervised fine-tuning based on token differentiation, thereby better improving the reasoning ability of the model, and can perform reasoning-specific data selection based on token differentiation, thereby more accurately and efficiently evaluating the influence of the training data on the test set.

[0240] Corresponding to the method embodiments described above, the present disclosure also provides model training platform embodiments. FIG. 5 shows a framework diagram of a model training platform according to an embodiment of the present disclosure. As shown in FIG. 5, the model training platform includes:

[0241] The request interface unit 502 is configured to receive a model training request, wherein the model training request includes model information of an initial question and answer model to be trained.

[0242] The model training unit 504 is configured to determine the initial question and answer model to be trained according to the model information, and perform model training on the initial question and answer model to be trained to obtain a target question and answer model, wherein the target question and answer model is obtained by using the target question and answer model training method described above.

[0243] The response unit 506 is configured to output the target question and answer model.

[0244] The model training platform provided by the embodiments of the present disclosure can determine an initial question and answer model to be trained and perform model training on the initial question and answer model to be trained to obtain a target question and answer model. The model training platform can provide powerful computing power and storage resources, thereby greatly improving the training efficiency of the model.

[0245] Corresponding to the method embodiments described above, the present disclosure also provides data processing device embodiments. FIG. 6 shows a structural schematic diagram of a data processing device according to an embodiment of the present disclosure. As shown in FIG. 6, the device includes:

[0246] The construction module 602 is configured to adjust an initial text processing result corresponding to an initial text in an initial text pair to obtain an updated text processing result, and construct an updated text pair according to the initial text and the updated text processing result.

[0247] The update module 604 is configured to update an initial text processing model according to the updated text pair to obtain a reference text processing model.

[0248] The obtaining module 606 is configured to obtain, according to the initial text pair, a prediction loss result and a reference loss result corresponding to each word piece in the initial text processing result by using the initial text processing model and the reference text processing model.

[0249] The determination module 608 is configured to determine a loss change result of each word piece according to the prediction loss result and the reference loss result.

[0250] The execution module 610 is configured to perform a data processing task according to the loss change result of each word piece.

[0251] Optionally, the execution module 610 is further configured to:

[0252] determine the target loss weight corresponding to each word element according to the loss change result of the each word element;

[0253] determine the target loss result corresponding to the initial text processing result according to the target loss weight corresponding to each word element and the prediction loss result of the each word element;

[0254] train the initial text processing model according to the target loss result to obtain a target text processing model.

[0255] Optionally, the obtaining module 606 is further configured to:

[0256] input the initial text pair into the initial text processing model to obtain a prediction text processing result corresponding to an initial text in the initial text pair;

[0257] obtain a prediction loss result corresponding to a plurality of word elements in the initial text processing result according to the prediction text processing result and the initial text processing result;

[0258] input the initial text pair into the reference text processing model to obtain a reference text processing result corresponding to an initial text in the initial text pair;

[0259] obtain a reference loss result corresponding to a plurality of word elements in the initial text processing result according to the reference text processing result and the initial text processing result.

[0260] Optionally, the execution module 610 is further configured to:

[0261] determine the word element type of each word element according to the loss change result of the each word element, and determine the word type of the word corresponding to the each word element according to the word element type of the each word element.

[0262] Optionally, the execution module 610 is further configured to:

[0263] determine the word element type of each word element according to the loss change result of the each word element, and determine a target reference word element from the plurality of word elements according to the word element type of the each word element.

[0264] determine a test text pair and a test text processing result corresponding to a test text in the test text pair, and determine a loss change result of a plurality of test word elements in the test text processing result;

[0265] According to the loss change result of each test word item, the word item type of each test word item is determined, and a target test word item is determined from the plurality of test word items according to the word item type of each test word item.

[0266] According to the target reference word item and the target test word item, a target text pair is determined from the plurality of initial text pairs.

[0267] Optionally, the execution module 610 is further configured to:

[0268] According to the test text pair, the initial text processing model and the reference text processing model are used to obtain the predicted loss result and the reference loss result corresponding to the plurality of test word items in the test text processing result;

[0269] According to the predicted loss result and the reference loss result corresponding to each test word item, the loss change result of each test word item is determined.

[0270] Optionally, the construction module 602 is further configured to:

[0271] The initial text in the plurality of initial text pairs is re-matched with the initial text processing result, and the initial text processing result corresponding to the initial text after re-matching is determined as the update text processing result.

[0272] Optionally, the update module 604 is further configured to:

[0273] The update text pair is input into the initial text processing model to obtain a predicted text processing result corresponding to the initial text in the update text pair;

[0274] According to the predicted text processing result and the update text processing result, the initial text processing model is trained to obtain the reference text processing model.

[0275] The device further comprises:

[0276] The calling module is configured to determine the model interface information of the initial text processing model, and call the initial text processing model according to the model interface information.

[0277] The device further comprises:

[0278] The interaction module is configured to determine the initial text pair carried by the text data processing request sent by the client in response to the text data processing request, and return the task execution result obtained by executing the data processing task to the client.

[0279] The data processing apparatus provided by the embodiment of the present disclosure can perform supervised fine-tuning of focused reasoning based on wordpiece differentiation, thereby improving the reasoning capability of the model, and can perform data selection based on wordpiece differentiation for focused reasoning, thereby more accurately and efficiently evaluating the influence of training data on the test set.

[0280] The above is a schematic scheme of the data processing apparatus of the embodiment. It should be noted that the technical scheme of the data processing apparatus belongs to the same concept as the technical scheme of the data processing method described above, and the details of the technical scheme of the data processing apparatus that are not described in detail can be referred to the description of the technical scheme of the data processing method.

[0281] Corresponding to the method embodiments described above, the present disclosure also provides a target question and answer model apparatus embodiment. FIG. 7 shows a structural schematic diagram of a target question and answer model apparatus according to an embodiment of the present disclosure. As shown in FIG. 7, the apparatus includes:

[0282] The data construction module 702 is configured to adjust the initial answer text corresponding to the initial question text in the initial question and answer pair, obtain an updated answer text, and construct an updated question and answer pair according to the initial question text and the updated answer text.

[0283] The model updating module 704 is configured to update the initial question and answer model according to the updated question and answer pair, and obtain a reference question and answer model.

[0284] The result obtaining module 706 is configured to obtain a prediction loss result and a reference loss result corresponding to each wordpiece in the initial answer text by using the initial question and answer model and the reference question and answer model according to the initial question and answer pair.

[0285] The weight determination module 708 is configured to determine a loss change result of each wordpiece according to the prediction loss result and the reference loss result, and determine a target loss weight corresponding to each wordpiece according to the loss change result of each wordpiece.

[0286] The result determination module 710 is configured to determine a target loss result corresponding to the initial answer text according to the target loss weight corresponding to each wordpiece and the prediction loss result of each wordpiece.

[0287] The model training module 712 is configured to train the initial question and answer model according to the target loss result, and obtain a target question and answer model.

[0288] The target question answering model training apparatus provided by the embodiments of the present disclosure can assign different training weights to the distinguished word units based on word unit distinction when performing supervised fine-tuning on the initial question answering model, implement supervised fine-tuning of the initial question answering model focusing on reasoning, and thus make the target question answering model obtained through training have better reasoning capability and improve the accuracy of the predicted answer text.

[0289] The above is a schematic scheme of the target question answering model apparatus of the present embodiment. It should be noted that the technical scheme of the target question answering model apparatus belongs to the same concept as the technical scheme of the target question answering model method described above, and the details of the technical scheme of the target question answering model apparatus that are not described in detail can be referred to the description of the technical scheme of the target question answering model method.

[0290] FIG. 8 shows a structural block diagram of a computing device 800 according to an embodiment of the present disclosure. The components of the computing device 800 include, but are not limited to, a memory 810 and a processor 820. The processor 820 is connected to the memory 810 through a bus 830, and a database 850 is used to save data.

[0291] The computing device 800 further includes an access device 840, which enables the computing device 800 to communicate via one or more networks 860. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 840 can include one or more of any type of network interface (e.g., a network interface card (NIC)) such as a IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, etc.

[0292] In one embodiment of the present disclosure, the above-mentioned components of the computing device 800 and other components not shown in FIG. 8 can also be connected to each other, for example, through a bus. It should be understood that the computing device structure block diagram shown in FIG. 8 is merely for the purpose of example, and is not a limitation on the scope of the present disclosure. Those skilled in the art can add or replace other components as needed.

[0293] The computing device 800 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 800 can also be a mobile or stationary server.

[0294] The processor 820 is configured to execute computer programs / instructions that implement the steps of the above-mentioned data processing method and target question answering model training method when executed by the processor.

[0295] Each of the embodiments in the present disclosure is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, the computing device embodiment is described simply because it is basically similar to the data processing method and target question answering model training method embodiments, and the relevant parts can be referred to the descriptions of the data processing method and target question answering model training method embodiments.

[0296] One embodiment of the present disclosure also provides a computer-readable storage medium storing computer programs / instructions that implement the steps of the above-mentioned data processing method and target question answering model training method when executed by the processor.

[0297] Each of the embodiments in the present disclosure is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, the computer-readable storage medium embodiment is described simply because it is basically similar to the data processing method and target question answering model training method embodiments, and the relevant parts can be referred to the descriptions of the data processing method and target question answering model training method embodiments.

[0298] One embodiment of the present disclosure also provides a computer program product including computer programs / instructions that implement the steps of the above-mentioned data processing method and target question answering model training method when executed by the processor.

[0299] The above is a schematic scheme of a computer program product of the embodiment. It should be noted that the technical scheme of the computer program product is the same as the technical scheme of the data processing method and the target question answering model training method described above, and the technical scheme of the computer program product which is not described in detail can be referred to the description of the technical scheme of the data processing method and the target question answering model training method.

[0300] The above describes specific embodiments of the disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order and still accomplish desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order to achieve desirable results. In certain implementations, multitasking and parallel processing can be advantageous.

[0301] The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer readable medium can be appropriately increased or decreased according to the requirements of patent practice, for example, in some regions, according to the patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0302] It should be noted that for the foregoing method embodiments, in order to facilitate description, they are all expressed as a combination of a series of actions, but those skilled in the art should know that the embodiments of the disclosure are not limited to the order of the actions described, because according to the embodiments of the disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the disclosure are all preferred embodiments, and the actions and modules involved are not necessarily necessary for the embodiments of the disclosure.

[0303] In the above embodiments, the description of each embodiment is focused on, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0304] The preferred embodiments of the present disclosure disclosed above are only used to help explain the present disclosure. The alternative embodiments do not describe all the details of the present disclosure, nor limit the present disclosure to the specific embodiments described. Obviously, according to the content of the embodiments of the present disclosure, many modifications and variations can be made. The present disclosure selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of the present disclosure, so that those skilled in the art can well understand and utilize the present disclosure. The present disclosure is limited only by the claims and their full scope and equivalents.

Claims

A data processing method comprises: adjusting an initial text processing result corresponding to an initial text in an initial text pair to obtain an updated text processing result, and constructing an updated text pair according to the initial text and the updated text processing result; updating an initial text processing model according to the updated text pair to obtain a reference text processing model; obtaining a predicted loss result and a reference loss result corresponding to each token in the initial text processing result respectively by using the initial text processing model and the reference text processing model according to the initial text pair; determining a loss change result of each token according to the predicted loss result and the reference loss result; performing a data processing task according to the loss change result of each token. The data processing task is a model training task for the initial text processing model according to claim 1; The performing of the data processing task according to the loss change result of each token comprises: determining a target loss weight corresponding to each token according to the loss change result of each token; determining a target loss result corresponding to the initial text processing result according to the target loss weight corresponding to each token and the predicted loss result of each token; training the initial text processing model to obtain a target text processing model according to the target loss result. The obtaining of the predicted loss result and the reference loss result corresponding to each token in the initial text processing result respectively by using the initial text processing model and the reference text processing model according to the initial text pair according to claim 1 or 2 comprises: inputting the initial text pair into the initial text processing model to obtain a predicted text processing result corresponding to the initial text in the initial text pair; obtaining the predicted loss result corresponding to each token in the initial text processing result according to the predicted text processing result and the initial text processing result; inputting the initial text pair into the reference text processing model to obtain a reference text processing result corresponding to the initial text in the initial text pair; obtaining the reference loss result corresponding to each token in the initial text processing result according to the reference text processing result and the initial text processing result. The data processing task is a token type determination task for a token corresponding to each token in the initial text processing result according to claim 1; The performing of the data processing task according to the loss change result of each token comprises: determining a token type of each token according to the loss change result of each token, and determining a token type of a token corresponding to each token according to the token type of each token. The initial text pair comprises a plurality of initial text pairs, and the data processing task is a text pair screening task for the plurality of initial text pairs according to claim 1; The performing of the data processing task according to the loss change result of each token comprises: determine wordpiece types of the wordpieces according to the loss change results of the wordpieces, and determine a target reference wordpiece from the plurality of wordpieces according to the wordpiece types of the wordpieces; determine a test text pair, determine test text processing results corresponding to the test text pair in the test text pair, and determine loss change results of a plurality of test wordpieces in the test text processing results; determine wordpiece types of the test wordpieces according to the loss change results of the test wordpieces, and determine a target test wordpiece from the plurality of test wordpieces according to the wordpiece types of the test wordpieces; determine a target text pair from the plurality of initial text pairs according to the target reference wordpiece and the target test wordpiece. The data processing method according to claim 5, wherein the determining the loss change results of the plurality of test wordpieces in the test text processing results comprises: obtaining, according to the test text pair, prediction loss results and reference loss results corresponding to a plurality of test wordpieces in the test text processing results by using the initial text processing model and the reference text processing model; determining the loss change results of the test wordpieces according to the prediction loss results and the reference loss results corresponding to the test wordpieces. The data processing method according to claim 5 or 6, wherein the determining the target text pair from the plurality of initial text pairs according to the target reference wordpiece and the target test wordpiece comprises: calculating wordpiece similarity between the target reference wordpiece and the target test wordpiece; determining a similar reference wordpiece from the target reference wordpiece according to the wordpiece similarity; determining, from the plurality of initial text pairs, an initial text pair corresponding to the similar reference wordpiece as the target text pair. The data processing method according to claim 5 or 6, wherein the determining the target text pair from the plurality of initial text pairs according to the target reference wordpiece and the target test wordpiece comprises: determining a reference gradient vector corresponding to the target reference wordpiece and a test gradient vector corresponding to the target test wordpiece; calculating text similarity between the plurality of initial text pairs and the test text pair according to the reference gradient vector and the test gradient vector; determining the target text pair from the plurality of initial text pairs according to the text similarity. The data processing method according to any one of claims 5-8, wherein the test text pair is a text pair related to a target text processing task; after the determining the target text pair from the plurality of initial text pairs, the method further comprises: training the initial text processing model by using the target text pair to obtain a target task text processing model, wherein the target task text processing model is used to implement the target text processing task. The data processing method according to any one of claims 5-9, before the determining the test text pair, the method further comprises: determining a plurality of initial test text pairs, inputting the plurality of initial test text pairs into the initial text processing model to obtain a plurality of initial test text processing results; determine an abnormal test text processing result from the plurality of initial test text processing results, and determine the initial test text pair corresponding to the abnormal test text processing result as the test text pair; after determining the target text pair from the plurality of initial text pairs, the method further includes: determining a screening text pair according to the plurality of initial text pairs and the target text pair, wherein the screening text pair is a text pair in the plurality of initial text pairs except the target text pair; training the initial text processing model by using the screening text pair to obtain an updated text processing model. The data processing method according to any one of claims 1-10, wherein the initial text pair includes a plurality of initial text pairs. The method further includes: re-pairing the initial text in the plurality of initial text pairs with the initial text processing result, and determining the initial text processing result corresponding to the initial text after re-pairing as the updated text processing result, wherein the initial text and the updated text processing result do not have a text matching relationship. The method further includes: inputting the updated text pair into the initial text processing model to obtain a predicted text processing result corresponding to an initial text in the updated text pair; training the initial text processing model according to the predicted text processing result and the updated text processing result to obtain the reference text processing model. The method further includes, before updating the initial text processing model according to the updated text pair to obtain the reference text processing model: determining model interface information of the initial text processing model, and calling the initial text processing model according to the model interface information. The method further includes, before adjusting the initial text processing result corresponding to the initial text in the initial text pair to obtain the updated text processing result: in response to a text data processing request sent by a client, determining the initial text pair carried by the text data processing request; after performing the data processing task according to the loss change result of each word piece, the method further includes: returning a task execution result obtained by performing the data processing task to the client. A target question and answer model training method includes: adjusting an initial answer text corresponding to an initial question text in an initial question and answer pair to obtain an updated answer text, and constructing an updated question and answer pair according to the initial question text and the updated answer text; updating an initial question and answer model according to the updated question and answer pair to obtain a reference question and answer model; according to the initial question and answer pair, using the initial question and answer model and the reference question and answer model to respectively obtain a predicted loss result and a reference loss result corresponding to a plurality of word pieces in the initial answer text; According to the prediction loss result and the reference loss result, a loss change result of each word element is determined, and according to the loss change result of each word element, a target loss weight corresponding to each word element is determined; According to the target loss weight corresponding to each word element and the prediction loss result of each word element, a target loss result corresponding to the initial answer text is determined; According to the target loss result, the initial question and answer model is trained to obtain a target question and answer model. A model training platform comprises a request interface unit, a model training unit and a response unit; The request interface unit is configured to receive a model training request, wherein the model training request comprises model information of an initial question and answer model to be trained; The model training unit is configured to determine the initial question and answer model to be trained according to the model information, and to train the initial question and answer model to obtain a target question and answer model, wherein the target question and answer model is obtained by using the target question and answer model training method of claim 15; The response unit is configured to output the target question and answer model. A data processing apparatus comprises: A construction module configured to adjust an initial text processing result corresponding to an initial text in an initial text pair to obtain an updated text processing result, and to construct an updated text pair according to the initial text and the updated text processing result; An updating module configured to update an initial text processing model according to the updated text pair to obtain a reference text processing model; An obtaining module configured to obtain, according to the initial text pair, prediction loss results and reference loss results corresponding to a plurality of word elements in the initial text processing result by using the initial text processing model and the reference text processing model; A determination module configured to determine a loss change result of each word element according to the prediction loss result and the reference loss result; An execution module configured to execute a data processing task according to the loss change result of each word element. A target question and answer model training apparatus comprises: A data construction module configured to adjust an initial answer text corresponding to an initial question text in an initial question and answer pair to obtain an updated answer text, and to construct an updated question and answer pair according to the initial question text and the updated answer text; A model updating module configured to update an initial question and answer model according to the updated question and answer pair to obtain a reference question and answer model; A result obtaining module configured to obtain, according to the initial question and answer pair, prediction loss results and reference loss results corresponding to a plurality of word elements in the initial answer text by using the initial question and answer model and the reference question and answer model; A weight determination module configured to determine a loss change result of each word element according to the prediction loss result and the reference loss result, and to determine a target loss weight corresponding to each word element according to the loss change result of each word element; A result determination module configured to determine a target loss result corresponding to the initial answer text according to the target loss weight corresponding to each word element and the prediction loss result of each word element. A model training module configured to train the initial question and answer model according to the target loss result to obtain a target question and answer model. A computing device comprising: a memory and a processor; the memory is configured to store computer programs / instructions, and the processor is configured to execute the computer programs / instructions, and the computer programs / instructions, when executed by the processor, implement the steps of the method according to any one of claims 1 to 15. A computer readable storage medium storing computer programs / instructions, and the computer programs / instructions, when executed by the processor, implement the steps of the method according to any one of claims 1 to 15. A computer program product comprising computer programs / instructions, and the computer programs / instructions, when executed by the processor, implement the steps of the method according to any one of claims 1 to 15.

Images