Data processing method, data processing model training method, and sample set construction method

Abstract

The embodiment of the present specification provides a data processing method, a data processing model training method and a sample set construction method, wherein the data processing method comprises: obtaining a to-be-processed text and a to-be-processed prompt text corresponding to the to-be-processed text; inputting the to-be-processed text and the to-be-processed prompt text into a data processing model to obtain reply data output by the data processing model, wherein the data processing model is obtained by training based on sample texts and target sample annotation data corresponding to the sample texts, the target sample annotation data is determined according to evaluation results corresponding to each group of sample annotation data in training sample data, the target sample annotation data comprises a plurality of sample prompt texts and sample reply data corresponding to each sample prompt text, the number of sample prompt texts in the target sample annotation data is less than the number of sample prompt texts in other sample annotation data, and the number of sample reply data in the target sample annotation data is more than the number of sample reply data in other sample annotation data.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computer technology, and in particular to data processing methods, data processing model training methods, and sample set construction methods. Background Technology

[0002] Currently, with the development of artificial intelligence, large models are increasingly being widely applied in various fields. Large models refer to machine learning models with a large number of parameters and complex computational structures. In practical applications, large models typically require a fine-tuning process of alignment with human feedback before being used, enabling them to learn content labeled by humans.

[0003] Although the fine-tuning process of human feedback alignment can enable large models to generate content that conforms to human preferences, the degree to which the output content of large models conforms to human preferences varies due to different methods of labeling training data. Therefore, how to improve the degree to which the output content of large models conforms to human preferences has become an urgent technical problem to be solved. Summary of the Invention

[0004] In view of the above, embodiments of this specification provide a data processing method, a data processing model training method, and a sample set construction method. One or more embodiments of this specification also relate to a data processing apparatus, a data processing model training apparatus, and a sample set construction apparatus, a computing device, a computer-readable storage medium, and a computer program, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a data processing method is provided, comprising:

[0006] Obtain the text to be processed and the corresponding prompt text to be processed;

[0007] The text to be processed and the prompt text to be processed are input into a data processing model to obtain the response data output by the data processing model. The data processing model is trained based on the sample text and the target sample annotation data corresponding to the sample text. The target sample annotation data is determined according to the evaluation results corresponding to each group of sample annotation data in the training sample data. The target sample annotation data includes multiple sample prompt texts and sample response data corresponding to each sample prompt text. The number of sample prompt texts in the target sample annotation data is less than the number of sample prompt texts in other sample annotation data, and the number of sample response data in the target sample annotation data is greater than the number of sample response data in other sample annotation data.

[0008] According to a second aspect of the embodiments of this specification, a data processing apparatus is provided, comprising:

[0009] The acquisition module is configured to acquire the text to be processed and the corresponding prompt text to be processed.

[0010] An input module is configured to input the text to be processed and the prompt text to be processed into a data processing model to obtain response data output by the data processing model. The data processing model is trained based on sample text and target sample annotation data corresponding to the sample text. The target sample annotation data is determined according to the evaluation results corresponding to each group of sample annotation data in the training sample data. The target sample annotation data includes multiple sample prompt texts and sample response data corresponding to each sample prompt text. The number of sample prompt texts in the target sample annotation data is less than the number of sample prompt texts in other sample annotation data, and the number of sample response data in the target sample annotation data is greater than the number of sample response data in other sample annotation data.

[0011] According to a third aspect of the embodiments of this specification, a data processing model training method is provided, comprising:

[0012] Obtain sample text and target sample annotation data corresponding to the sample text. The target sample annotation data is determined based on the evaluation results corresponding to each group of sample annotation data in the training sample data. The target sample annotation data includes multiple sample prompt texts and sample response data corresponding to each sample prompt text. The number of sample prompt texts in the target sample annotation data is less than the number of sample prompt texts in other sample annotation data, and the number of sample response data in the target sample annotation data is greater than the number of sample response data in other sample annotation data.

[0013] The sample text and the target sample annotation data are input into the data processing model to obtain the predicted data output by the data processing model;

[0014] The predicted data is evaluated to obtain the evaluation results of the predicted data;

[0015] Adjust the model parameters of the data processing model based on the evaluation results, and continue training the data processing model until the training stop condition is met.

[0016] According to a fourth aspect of the embodiments of this specification, a data processing model training apparatus is provided, comprising:

[0017] The data acquisition module is configured to acquire sample text and target sample annotation data corresponding to the sample text. The target sample annotation data is determined based on the evaluation results corresponding to each group of sample annotation data in the training sample data. The target sample annotation data includes multiple sample prompt texts and sample response data corresponding to each sample prompt text. The number of sample prompt texts in the target sample annotation data is less than the number of sample prompt texts in other sample annotation data, and the number of sample response data in the target sample annotation data is greater than the number of sample response data in other sample annotation data.

[0018] The data input module is configured to input the sample text and the target sample annotation data into the data processing model to obtain the prediction data output by the data processing model;

[0019] The data evaluation module is configured to evaluate the prediction data and obtain the evaluation result of the prediction data;

[0020] The parameter adjustment module is configured to adjust the model parameters of the data processing model according to the evaluation results, and continue training the data processing model until the training stop condition is met.

[0021] According to a fifth aspect of the embodiments of this specification, a sample set construction method is provided, applied to a server, comprising:

[0022] Receive a sample set construction request sent by the target terminal, wherein the sample set construction request carries the target text and the sample set to be constructed corresponding to the target text;

[0023] The target text and the sample set to be constructed are input into the data processing model to obtain the predicted response data corresponding to the sample set to be constructed.

[0024] The predicted response data is evaluated to obtain the evaluation results corresponding to the predicted response data;

[0025] Based on the evaluation results, a sample set is generated to construct feedback information;

[0026] The sample set construction feedback information is sent to the target terminal.

[0027] According to a sixth aspect of the embodiments of this specification, a sample set construction apparatus is provided, applied to a server, comprising:

[0028] The request receiving module is configured to receive a sample set construction request sent by the target terminal, wherein the sample set construction request carries target text and the sample set to be constructed corresponding to the target text;

[0029] The sample set input module is configured to input the target text and the sample set to be constructed into the data processing model to obtain the predicted response data corresponding to the sample set to be constructed.

[0030] The prediction data evaluation module is configured to evaluate the prediction response data and obtain the evaluation result corresponding to the prediction response data.

[0031] The information generation module is configured to generate a sample set to construct feedback information based on the evaluation results;

[0032] The information sending module is configured to send the sample set construction feedback information to the target terminal.

[0033] According to a seventh aspect of the embodiments of this specification, a computing device is provided, comprising:

[0034] Memory and processor;

[0035] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the above method.

[0036] According to an eighth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0037] According to a ninth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0038] One embodiment of this specification implements a method for further determining the annotation method of sample annotation data used to train the data processing model when the total number of annotation data in each group of sample annotation data is the same. This method compares the difference between the number of annotations in the sample prompt text and the number of annotations in the sample response data. Based on this annotation method, target sample annotation data is generated. The data processing model is then trained using the target sample annotation data, making the output of the data processing model more consistent with human preferences. Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating a data processing method provided in one embodiment of this specification;

[0040] Figure 2 This is a schematic diagram of sample annotation data provided in one embodiment of this specification;

[0041] Figure 3This is a flowchart illustrating a process for determining a target annotation method, provided in one embodiment of this specification.

[0042] Figure 4 This is a schematic diagram of a linear fitting provided in one embodiment of this specification;

[0043] Figure 5 This is another schematic diagram of linear fitting provided in one embodiment of this specification;

[0044] Figure 6 This is another linear fitting diagram provided in one embodiment of this specification;

[0045] Figure 7 This is a graph showing the relationship between a diversity index and the number of sample prompt texts, provided in one embodiment of this specification.

[0046] Figure 8 This is a schematic diagram of the structure of a data processing apparatus provided in one embodiment of this specification;

[0047] Figure 9 This is a flowchart illustrating a data processing model training method provided in one embodiment of this specification;

[0048] Figure 10 This is a schematic diagram of the structure of a data processing model training device provided in one embodiment of this specification;

[0049] Figure 11 This is a flowchart illustrating a sample set construction method provided in one embodiment of this specification;

[0050] Figure 12 This is a schematic diagram of a sample set construction device provided in one embodiment of this specification;

[0051] Figure 13 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0052] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0053] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0054] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0055] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0056] In one or more embodiments of this specification, a large model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. A large model can also be called a foundation model. It is pre-trained using large-scale unlabeled corpora to produce a pre-trained model with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and multi-modal pre-training models.

[0057] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as NLP tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios for large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0058] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0059] LLM: Large Language Models.

[0060] Textual diversity: The diversity of texts (corpora) includes the diversity of vocabulary, grammar, expression, etc.

[0061] Tip: Prompt means inputting the contents of the large model first.

[0062] Candidate response: Response, which is the content generated by the large model after receiving the prompt.

[0063] Currently, with the development of artificial intelligence, large models are increasingly being widely applied in various fields. Large models refer to machine learning models with a large number of parameters and complex computational structures. In practical applications, large models typically require a fine-tuning process of alignment with human feedback before being used, enabling them to learn content labeled by humans.

[0064] Although fine-tuning the alignment with human feedback can enable large models to generate content that conforms to human preferences, the degree to which the output of large models conforms to human preferences varies due to different methods of labeling training data.

[0065] This specification provides a data processing method, a data processing model training method, and a sample set construction method. One or more embodiments of this specification also relate to a data processing apparatus, a data processing model training apparatus, a sample set construction apparatus, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0066] See Figure 1 , Figure 1A flowchart of a data processing method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0067] Step 102: Obtain the text to be processed and the corresponding prompt text to be processed.

[0068] The text to be processed refers to the text that the data processing model needs to generate corresponding response content for. For example, in an intelligent question-answering scenario, the text to be processed can be the question asked by the user to the data processing model; in a video processing scenario, the text to be processed can be the content required for video processing; in an image generation scenario, the text to be processed can be the content required for generating images, and so on.

[0069] The pending prompt text refers to the text content used to assist the data processing model in processing the pending text. The data processing model refers to the large language model used to process text data and generate response data.

[0070] Step 104: Input the text to be processed and the prompt text to be processed into the data processing model to obtain the response data output by the data processing model.

[0071] The data processing model is trained based on sample text and its corresponding target sample annotation data. This model can be used to generate answers, videos, or images based on user needs. For example, in a question-answering scenario, the model can be a question-answering model; in a video processing scenario, it can be a video processing model; and in an image processing scenario, it can be an image generation model. The target sample annotation data is any one of the groups of sample annotation data in the training sample data, specifically determined based on the evaluation results of each group of sample annotation data in the training sample data. The target sample annotation data includes multiple sample prompt texts and corresponding sample response data. The number of sample prompt texts in the target sample annotation data is less than the number of sample prompt texts in other sample annotation data, while the number of sample response data in the target sample annotation data is greater than the number of sample response data in other sample annotation data. Other sample annotation data refers to the sample annotation data in the training sample data other than the target sample annotation data. Sample text refers to the sample requirement content data used to train the data processing model. The sample text corresponds to training sample data, which includes multiple sets of sample annotation data. Each sample annotation data includes multiple sample prompt texts corresponding to the sample text, as well as sample response data corresponding to each sample prompt text.

[0072] Response data refers to the content corresponding to the text to be processed output by the data processing model. Response data can be text data, video data, image data, audio data, etc. For example, in an intelligent question-answering scenario, response data is the answer generated based on the user's question; in a video processing scenario, response data is the video generated based on the user's needs; in an image scenario, response data is the generated image data, and so on.

[0073] Specifically, by inputting the text to be processed and the prompt text to be processed into the data processing model, the response data output by the data processing model can be obtained. By processing the text to be processed and the prompt text to be processed using a data processing model trained on labeled target sample data, the corresponding response data can be output, thus improving the accuracy of the data processing model's output. The specific implementation method for obtaining training sample data is as follows:

[0074] In one specific embodiment provided in this specification, the method further includes:

[0075] Obtain the sample text and the training sample data, wherein the training sample data includes at least two sets of sample annotation data, the sample annotation data includes sample prompt text and sample response data corresponding to the sample prompt text, the total number of annotation data in each set of sample annotation data is the same, but the number of sample prompt texts is different;

[0076] The sample text and the labeled data of each group of samples are input into the data processing model to obtain the predicted response data corresponding to each group of labeled data.

[0077] Each predicted response data point is evaluated to obtain the evaluation results corresponding to each predicted response data point;

[0078] Based on the evaluation results corresponding to each predicted response data, the target predicted response data is determined, and the data annotation method of the target sample annotation data corresponding to the target predicted response data is determined as the target annotation method.

[0079] In practical applications, training large models typically requires labeling the prompt text and response data. This labeled data is then used to train the large model. Since the large model generates response data corresponding to the input prompt text during application, a large amount of prompt text and response data is usually labeled before training the model.

[0080] Training sample data refers to labeled training data, including sample data labeled using at least two labeling methods. The training sample data includes at least two sets of labeled sample data, each set containing sample prompt text and corresponding sample response data. The sample prompt text is the labeled prompt text, used as input for training the large model; the sample response data is the labeled response data, generated by the large model based on the input prompt text. Training sample data is obtained from a training sample data set, which is a dataset created by collecting prompt text and response data and labeling the collected prompt text and response data.

[0081] In practical applications, sample response data can be text data, video data, audio data, etc., depending on the specific application.

[0082] See Figure 2 , Figure 2 A schematic diagram of sample annotation data provided according to one embodiment of this specification is shown. For example... Figure 2 As shown, a sample annotation data set consists of a sample prompt text and multiple corresponding sample response data sets. Specifically, sample prompt text x1 and its corresponding sample response data y11, y12, and y13 constitute a sample annotation data set; sample prompt text x2 and its corresponding sample response data y21, y22, and y23 constitute a sample annotation data set; and sample prompt text x3 and its corresponding sample response data y31, y32, and y33 constitute a sample annotation data set.

[0083] In one embodiment provided in this specification, a set of sample annotation data includes multiple sample prompt texts and sample response data corresponding to each sample prompt text. To determine a data annotation method that more effectively triggers the alignment effect of the large model with human preferences, given limited annotation resources (i.e., limited total data annotation resources), in practical applications, the total number of labeled data in each set of sample annotation data can be the same, but the number of sample prompt texts and sample response data in each set will differ. Therefore, when the total number of labeled data in each set of sample annotation data is the same, the final data annotation method is further determined by comparing the difference between the number of labeled sample prompt texts and the number of labeled sample response data. Specifically, the data annotation method refers to the annotation ratio between the number of labeled sample prompt texts and the number of labeled sample response data in the sample annotation data, not the method of annotating a specific sample prompt text or a specific sample response data.

[0084] In one specific embodiment provided in this specification, obtaining the training sample data includes:

[0085] Obtain the preset labeled data threshold;

[0086] Based on the preset annotation data threshold, at least two sets of sample prompt texts with different numbers are obtained, wherein the number of sample prompt texts in each set is less than or equal to the preset annotation data threshold;

[0087] Based on the preset annotation data threshold and the number of sample prompt texts in each group, obtain the sample response data corresponding to the sample prompt texts in each group;

[0088] Training sample data is obtained based on the prompt text of each group of samples and the corresponding sample response data of each group of samples.

[0089] The preset annotation data threshold refers to the total number of annotation data sets used to acquire annotation data for each group of samples. For example, if the preset annotation data threshold is 2N, then the total number of annotation data sets to be acquired for each group of samples is 2N, which means that the total number of annotation data sets for the sample prompt text and sample response data for each group is 2N.

[0090] In practical applications, since one sample prompt text may correspond to multiple sample response data, when obtaining sample annotation data according to the preset annotation data threshold, the sample prompt text can be obtained first, and then the sample response data corresponding to the remaining number of sample prompt texts can be obtained.

[0091] Specifically, a preset labeled data threshold is obtained. Based on this threshold, at least two sets of sample prompt texts with different numbers of texts are acquired. Then, based on the number of sample prompt texts acquired and the preset labeled data threshold, the remaining number of sample response data is obtained. This results in multiple sets of sample prompt texts with different numbers of texts, and the corresponding sample response data for each prompt text. In other words, multiple sets of sample labeled data with the same number of texts are obtained. These multiple sets of sample labeled data constitute the training sample data. Furthermore, after acquiring the sample prompt texts, a corresponding number of sample response data can be randomly selected from the multiple sample response data corresponding to each prompt text.

[0092] For example, with a preset annotation data threshold of 2N, N sample prompt texts, 2N / 3 sample prompt texts, and N / 2 sample prompt texts are obtained from the training sample dataset. For N sample prompt texts, two sample response data points need to be selected from the multiple sample response data points corresponding to each sample prompt text, so that the total number of annotation data points for this set of sample prompt texts is 2N. For 2N / 3 sample prompt texts, three sample response data points need to be selected from the multiple sample response data points corresponding to each sample prompt text, so that the total number of annotation data points for this set of sample prompt texts is 2N. For N / 2 sample prompt texts, four sample response data points need to be selected from the multiple sample response data points corresponding to each sample prompt text, so that the total number of annotation data points for this set of sample prompt texts is 2N.

[0093] By setting a preset threshold for labeled data, the number of labeled data in each group of sample labeled data is the same. Multiple groups of sample prompt text with different numbers of data are obtained. Then, for each group of sample prompt text, a different number of sample response data are selected from the corresponding sample response data, so that the number of sample prompt text and the number of sample response data are different in each group. Based on the obtained training sample data, subsequent processing is performed to determine the final data labeling method according to the different labeling methods corresponding to the labeled data of each group of sample in the training sample data.

[0094] Predicted response data refers to the response data output by the data processing model after processing the sample labeled data.

[0095] Specifically, by inputting the sample text and the corresponding sample annotation data into the data processing model, the predicted response data output by the data processing model can be obtained.

[0096] The evaluation result is the result obtained after evaluating the predicted response data output by the data processing model.

[0097] To determine the relationship between the model performance based on different sample labeled data and the data processing model, after processing the sample labeled data of each group based on the data processing model and obtaining the corresponding predicted response data, it is necessary to further evaluate the obtained predicted response data so that a better data labeling method can be determined in subsequent processes based on the evaluation results corresponding to the predicted response data.

[0098] In practical applications, manual evaluation can be used to assess the predicted response data and obtain corresponding evaluation results. However, due to the large amount of data processed by the data processing model, manual evaluation would incur high costs, low efficiency, and long processing time. Therefore, evaluation models can also be used to evaluate the predicted response data.

[0099] In one specific embodiment provided in this specification, each predicted response data is evaluated to obtain the evaluation result corresponding to each predicted response data, including:

[0100] Each predicted response data is input into the evaluation model to obtain the evaluation result corresponding to each predicted response data output by the evaluation model, wherein the evaluation model is trained based on the user feedback data corresponding to the sample response data.

[0101] The evaluation model refers to the model used to evaluate the output of the data processing model (i.e., the predicted response data). Specifically, the evaluation model can be a reward model or a scoring model. Correspondingly, when the evaluation model is a reward model, the evaluation result corresponding to the predicted response data is the reward value; when the evaluation model is a scoring model, the evaluation result corresponding to the predicted response data is the predicted score.

[0102] User feedback data refers to data obtained by users providing feedback on predicted response data, which is used to characterize the degree of user preference for predicted response data.

[0103] In practical applications, the predicted response data output by the data processing model is not just one response, but multiple responses; that is, for a single prompt text, the data processing model will output multiple responses. After obtaining the predicted response data output by the data processing model, user feedback data on these predicted responses can be obtained to understand user preferences for each predicted response. Furthermore, an evaluation model can be trained based on the user feedback data. After training, the trained evaluation model can be used to evaluate the predicted response data and obtain the corresponding evaluation results.

[0104] Specifically, by inputting each predicted response data into the trained evaluation model, the evaluation results corresponding to each predicted response data can be obtained.

[0105] By inputting the predicted response data corresponding to different sample labeled data into the evaluation model for evaluation, and obtaining the evaluation results corresponding to each predicted response data, it is helpful to further determine the better data labeling method in each group of sample labeled data.

[0106] Target predicted response data refers to the response data selected from all predicted response data to determine the data annotation method. Target predicted response data is the predicted response data with the best evaluation result among the corresponding evaluation results. Target annotation method refers to the data annotation method used for the target sample labeled data corresponding to the target predicted response data.

[0107] In practical applications, after obtaining the evaluation results corresponding to each predicted response data, the predicted response data with better evaluation results can be identified as the target predicted response data. Further, the target sample labeled data corresponding to the target predicted response data is determined, and the data labeling method of the target sample labeled data is defined as the target labeling method. It should be noted that the data labeling method of the target sample labeled data can specifically be the labeling ratio between the number of labeled sample prompt texts and the number of labeled sample response data, and not the method of labeling a specific sample prompt text or a specific sample response data.

[0108] Taking a reward-based evaluation model as an example, if the evaluation model is a reward model, then the evaluation result corresponding to each predicted response data is the reward value output by the reward model for each predicted response data. First, based on the reward value corresponding to each predicted response data, the predicted response data with the highest reward value is determined, and this predicted response data is designated as the target predicted response data. Further, the target sample annotation data corresponding to the target predicted response data is determined, and the data annotation method of the target sample annotation data is determined as the target annotation method. For example, sample annotation data 1 contains N sample prompt texts, and each sample prompt text corresponds to 2 sample response data; sample annotation data 2 contains 2N / 3 sample prompt texts, and each sample prompt text corresponds to 3 sample response data; sample annotation data 3 contains N / 2 sample prompt texts, and each sample prompt text corresponds to 4 sample response data. Among the predicted response data 1 corresponding to sample labeled data 1, the predicted response data 2 corresponding to sample labeled data 2, and the predicted response data 3 corresponding to sample labeled data 3, the reward value of predicted response data 3 is higher than that of predicted response data 1 and predicted response data 2. Therefore, predicted response data 3 is determined to be the target predicted response data, sample labeled data 3 is determined to be the target sample labeled data, and thus the data labeling method of sample labeled data 3 can be determined to be the target labeling method. That is, the data labeling method that labels N / 2 sample prompt texts, such that each sample prompt text corresponds to 4 sample response data, is the target labeling method.

[0109] The following is in conjunction with the appendix Figure 3 The process of determining the target annotation method will be further explained. Figure 3 This specification illustrates a flowchart of a process for determining a target annotation method according to an embodiment, as shown below. Figure 3As shown, the acquired sample text, sample prompt text, and sample response data are input into the data processing model to obtain the predicted response data output by the data processing model. The predicted response data is then input into the evaluation model to obtain the evaluation results output by the evaluation model for the predicted response data. If the evaluation result reaches the evaluation threshold, the target predicted response data is determined from the predicted response data, and the data annotation method corresponding to the target predicted response data is determined as the target annotation method. If the evaluation result does not reach the evaluation threshold, the data processing model is fine-tuned based on the fine-tuning algorithm, and the above process continues until the target annotation method is determined. It should be noted that the evaluation model is trained based on user feedback data regarding the predicted response data. During the process of determining the target annotation method, the evaluation model is a pre-trained model.

[0110] By acquiring sample labeled data with different annotation methods and processing each group of sample labeled data using a data processing model, the predicted response data corresponding to each group of sample labeled data is obtained. Then, each predicted response data is evaluated to obtain corresponding evaluation results. Based on the evaluation results of each predicted response data, target predicted response data is determined from among the predicted response data. Furthermore, based on the target sample labeled data corresponding to the target predicted response data, the target annotation method is determined. This allows for the evaluation and determination of the annotation method of training data in advance during model training, improving model training efficiency.

[0111] In practical applications, the diversity of training data can also affect model performance. Therefore, in one embodiment of this specification, the analysis is also conducted from the perspective of the diversity of sample prompt text to determine the relationship between sample prompt text and model performance.

[0112] In one specific embodiment provided in this specification, the method further includes:

[0113] Segment the prompt text of each group of samples to obtain the initial segmented prompt text corresponding to each group of samples.

[0114] The diversity index of the sample prompt texts for each group is calculated based on the number of initial prompt texts and the number of sample prompt texts for each group.

[0115] The initial prompt segmented text refers to the text obtained after segmenting each group of sample prompt texts. The diversity index is a numerical value of the sample prompt text diversity indicator, used to characterize the text diversity of the sample prompt texts.

[0116] Specifically, the obtained sample prompt texts for each group are segmented into words to obtain the initial segmented prompt texts corresponding to each group. Then, based on the number of initial segmented prompt texts and sample prompt texts for each group, the diversity index of each group of sample prompt texts is calculated.

[0117] Furthermore, in the process of segmenting the sample prompt texts for each group, N-grams can be used as the basic unit for text segmentation. For example, if the sample prompt text is "I am happy.I am a girl." and the N-grams are 2-grams, then segmenting the sample prompt text "I am happy.I am agirl." will yield "I am", "amhappy", "happy I", "I am", "am a", and "a girl". The initial segmented prompt text will then be "I am", "amhappy", "happyI", "I am", "am a", and "agirl".

[0118] In one specific embodiment provided in this specification, the diversity index of each group of sample prompt texts is calculated based on the number of initial prompt texts and the number of sample prompt texts in each group, including:

[0119] Identify the words to be filtered in the initial prompt word segmentation text to be processed, wherein the initial prompt word segmentation text to be processed is any one of the groups of initial prompt word segmentation texts;

[0120] Based on the segmented words to be filtered, the initial prompt segmented text to be processed is filtered to obtain the target prompt segmented text to be processed corresponding to the initial prompt segmented text to be processed;

[0121] Based on the number of each target word in the target prompt word segmentation text to be processed and the number of sample prompt texts to be processed, the diversity index of the sample prompt texts to be processed is calculated, wherein the sample prompt texts to be processed are the sample prompt texts corresponding to the initial prompt word segmentation text to be processed.

[0122] The initial prompt segmented text to be processed is any group of initial prompt segmented texts. The segment to be filtered refers to repeated segmented words present in the initial prompt segmented text. Continuing the example above, the initial prompt segmented text is “I am”, “amhappy”, “happy I”, “I am”, “am a”, and “a girl”, where “I am” is a repeated segment, i.e., the segment to be filtered. The target prompt segmented text to be processed refers to the segmented text after filtering the initial prompt segmented text. For example, if the initial prompt segmented text to be processed is “I am”, “am happy”, “happy I”, “I am”, “am a”, and “a girl”, and there are two segments to be filtered, “I am”, we remove one of them, resulting in “I am”, “am happy”, “happy I”, “am a”, and “a girl”, which is the target prompt segmented text to be processed.

[0123] Target word segmentation refers to the word segmentation in each target prompt word segmentation text. Sample prompt text to be processed refers to the sample prompt text corresponding to the initial prompt word segmentation text to be processed.

[0124] Specifically, for any one group of initial prompt segmented texts (the initial prompt segmented texts to be processed), the word to be filtered in the initial prompt segmented text to be processed is identified, and the initial prompt segmented text to be processed is filtered according to the word to be filtered, that is, only one word to be filtered is retained in the initial prompt segmented text to be processed, thus obtaining the target prompt segmented text to be processed corresponding to the initial prompt segmented text to be processed. Then, based on the number of target words in the target prompt segmented text to be processed and the number of sample prompt texts to be processed, the diversity index of the sample prompt texts to be processed is calculated.

[0125] Furthermore, in a specific embodiment provided in this specification, the diversity index of the sample prompt text to be processed is calculated based on the number of each target segmentation word in the target prompt text to be processed and the number of sample prompt texts to be processed, including:

[0126] Based on the number of each target word in the target prompt segmented text to be processed, determine the proportion of the target words to be processed corresponding to the target prompt segmented text to be processed;

[0127] Calculate the product of the percentage of words to be segmented and the number of sample prompt texts to be processed;

[0128] The product is determined as the diversity index of the prompt text of the sample to be processed.

[0129] The percentage of words to be processed refers to the ratio between the number of each target word in the target suggestion word segmentation text and the number of each initial word in the corresponding initial suggestion word segmentation text. In other words, it's the ratio between the number of each target word in the target suggestion word segmentation text and the number of each initial word in the initial suggestion word segmentation text. Continuing with the previous example, if the initial suggestion word segmentation text contains "I am", "am happy", "happy I", "I am", "am a", and "a girl", with a total of 6 initial words, and the corresponding target suggestion word segmentation text contains "I am", "amhappy", "happy I", "am a", and "a girl", with a total of 5 target words, then the percentage of words to be processed is 5 / 6.

[0130] Specifically, the proportion of unprocessed word segments corresponding to each group of sample prompt texts is determined, and the number of sample prompt texts in each group is calculated. The product of the unprocessed proportion and the number of sample prompt texts is then determined as the diversity index of the sample prompt texts.

[0131] To visually observe the relationship between the diversity of sample prompt text and model performance, linear fitting can be performed based on the data obtained above, and a corresponding linear fitting diagram can be generated, as shown below:

[0132] In one specific embodiment provided in this specification, after calculating the diversity index of the prompt text for each group of samples, the method further includes:

[0133] Based on the diversity index of the prompt texts in each group of samples and the evaluation results corresponding to the prompt texts in each group of samples, a two-dimensional dataset corresponding to the prompt texts in each group of samples is constructed.

[0134] Linear fitting is performed on the two-dimensional dataset corresponding to the prompt text of each group of samples to obtain the linear fitting results for each group of sample prompt text.

[0135] The two-dimensional data set refers to the set of two-dimensional data points used to store the prompt text for each group of samples.

[0136] Specifically, for any set of sample prompt texts, the diversity index of the sample prompt texts is determined as the x-axis of the two-dimensional data points, and the evaluation result corresponding to the sample prompt texts is determined as the y-axis of the two-dimensional data points, generating multiple two-dimensional data points for the sample prompt texts. Linear fitting is then performed on these multiple two-dimensional data points to obtain the corresponding linear fitting result.

[0137] See Figure 4 , Figure 4A schematic diagram of a linear fitting provided according to one embodiment of this specification is shown. Figure 4 The figure shows the linear fitting results obtained by fine-tuning the OPT (Open pre-trained transformer language models) model using the PRO (Preference Ranking Optimization) algorithm on two-dimensional data points, and performing linear fitting under different numbers of sample response data (2 sample response data, 3 sample response data, and 4 sample response data).

[0138] See Figure 5 , Figure 5 A schematic diagram of another linear fitting provided according to one embodiment of this specification is shown. Figure 5 As shown, the linear fitting results are obtained by using the SFT (Supervised Fine-tuning) algorithm to fine-tune the OPT model to form two-dimensional data points, under different numbers of sample response data (2 sample response data, 3 sample response data and 4 sample response data).

[0139] In practical applications, the number of prompt texts in each group of samples can also be used as an indicator to measure the diversity of prompt texts in each group of samples. Based on this, see [link to relevant documentation]. Figure 6 , Figure 6 A schematic diagram of yet another linear fitting method provided according to an embodiment of this specification is shown. For example... Figure 6 As shown, Figure 6 The horizontal axis represents the number of prompt texts in each sample group, and the vertical axis represents the evaluation result of the predicted response data obtained based on the prompt texts in each sample group. Using the same method as above, linear fitting can be performed to obtain the following... Figure 6 The linear fitting results are shown.

[0140] Furthermore, the accuracy of the linear fit can be determined by calculating the mean square error of the fit.

[0141] Regarding the above method for calculating the diversity index of sample prompt text, it can be seen that the diversity index of sample prompt text is also related to the number of sample prompt texts. Based on this, an embodiment of this specification also provides a method for data augmentation of sample prompt text.

[0142] In one specific embodiment provided in this specification, the method further includes:

[0143] Obtain the original sample prompt text;

[0144] Extract the subsequent prompt text corresponding to each group of sample prompt text from the original sample prompt text;

[0145] Based on the sample prompt text for each group and the subsequent prompt text for each group, expandable prompt text is generated.

[0146] Here, "original sample prompt text" refers to all sample prompt texts in the training sample dataset. "Following prompt text" refers to the text following the sample prompt text. "Expanded prompt text" refers to the prompt text after data expansion of the sample prompt text.

[0147] Specifically, the original sample prompt text is obtained, and the obtained sample prompt text is used as the preceding prompt text. Based on the preceding prompt text, the following prompt text corresponding to the sample prompt text is extracted from the original sample prompt text. The extracted following prompt text is merged with the obtained sample prompt text to obtain the expanded prompt text, thereby realizing the data expansion of the sample prompt text.

[0148] In one specific embodiment provided in this specification, expanded prompt text is generated based on each group of sample prompt text and each group of subsequent prompt text, including:

[0149] The prompt text of each group of samples is segmented into words to obtain the initial prompt segmented text corresponding to each group of sample prompt texts. The following prompt text of each group is segmented into words to obtain the initial following prompt text corresponding to each group of following prompt texts.

[0150] The initial prompt segmented text to be processed is segmented and filtered to obtain the target prompt segmented text to be processed corresponding to the initial prompt segmented text to be processed. The initial following text to be processed is segmented and filtered to obtain the target following text to be processed corresponding to the initial following text to be processed. The initial prompt segmented text to be processed is any one of the initial prompt segmented texts in each group, and the initial following text to be processed is any one of the initial following texts in each group.

[0151] Based on the target prompt segmented text and the target context segmented text, calculate the overlap coefficient between the target prompt segmented text and the target context segmented text;

[0152] Based on the overlap coefficients, select the text to be expanded from the following text in each group;

[0153] Based on the sample prompt texts for each group and the prompt texts to be expanded below, expandable prompt texts are generated.

[0154] The initial context segmented text refers to the text obtained after segmenting each group of context prompts. The initial context segmented text to be processed is any one of the initial context segmented texts. The target context segmented text to be processed refers to the segmented text after filtering the initial context segmented text to be processed.

[0155] The overlap coefficient characterizes the degree of overlap between the target prompt text and the target following text, and can specifically be the Jaccard coefficient. The following prompt text to be expanded refers to the prompt text used to expand the sample prompt text within each group of following texts. The following prompt text to be expanded is determined based on the overlap coefficient.

[0156] Specifically, based on the same method used to generate the initial prompt word segmentation texts for each group, initial subsequent word segmentation texts corresponding to each group's subsequent prompt texts are generated. Then, based on the same word segmentation filtering method described above, word segmentation filtering is performed on both the initial prompt word segmentation texts and the initial subsequent word segmentation texts to obtain the target subsequent word segmentation texts corresponding to each initial subsequent word segmentation text. Furthermore, based on the filtered target prompt word segmentation texts and target subsequent word segmentation texts, the overlap coefficient between the target prompt word segmentation texts and the target subsequent word segmentation texts is calculated. Then, based on the overlap coefficient, subsequent word prompt texts to be expanded are selected from each group's subsequent word prompt texts. Finally, the sample prompt texts from each group and the corresponding subsequent word prompt texts to be expanded are merged to generate the expanded prompt texts.

[0157] Furthermore, in the process of filtering the following prompt text to be expanded based on the overlap coefficient, the following prompt text with the smallest overlap coefficient can be directly determined as the following prompt text to be expanded, or a preset number of following prompt texts can be filtered as the following prompt text to be expanded based on the size of each overlap coefficient. The specific implementation method is not limited here.

[0158] Furthermore, in a specific embodiment provided in this specification, the overlap coefficient between the target prompt segmented text and the target context segmented text is calculated based on the target prompt segmented text and the target context segmented text, including:

[0159] Based on the number of each target word in the target prompt word segmentation text to be processed, determine the first unprocessed word segmentation ratio corresponding to the target prompt word segmentation text to be processed; based on the number of each target word in the target following word segmentation text to be processed, determine the second unprocessed word segmentation ratio corresponding to the target following word segmentation text to be processed.

[0160] Based on the first unprocessed word segmentation ratio and the second unprocessed word segmentation ratio, calculate the overlap coefficient between the first unprocessed word segmentation ratio and the second unprocessed word segmentation ratio.

[0161] Among them, the first unprocessed segmentation ratio refers to the unprocessed segmentation ratio corresponding to the target prompt segmented text; the second unprocessed segmentation ratio refers to the unprocessed segmentation ratio corresponding to the target following segmented text.

[0162] Specifically, the proportion of the first segmented word to be processed in the target prompt segmented text and the proportion of the second segmented word to be processed in the target following segmented text are determined respectively. Based on the proportion of the first segmented word to be processed and the proportion of the second segmented word to be processed, the overlap coefficient between the target prompt segmented text and the target following segmented text is calculated. Specifically, the Jaccard coefficient between the target prompt segmented text and the target following segmented text can be calculated based on the proportion of the first segmented word to be processed and the proportion of the second segmented word to be processed.

[0163] In addition, the large model can be used to predict the following text of each sample prompt text to obtain the following prompt text output by the large model.

[0164] By augmenting the sample prompt text with data, the textual diversity of the sample prompt text is improved.

[0165] See Figure 7 , Figure 7 A graph showing the relationship between a diversity index and the number of sample prompt texts, according to one embodiment of this specification, is shown. Figure 7 As shown, the diversity index of the sample prompt texts changes with the number of sample prompt texts; that is, the diversity index of the sample prompt texts increases significantly with the increase of the number of sample prompt texts. It can be concluded that the diversity index of the sample prompt texts is related to the number of sample prompt texts.

[0166] The data processing method provided in this specification includes: acquiring a text to be processed and a corresponding prompt text to be processed; inputting the text to be processed and the prompt text to be processed into a data processing model to obtain response data output by the data processing model, wherein the data processing model is trained based on sample text and target sample annotation data corresponding to the sample text, the target sample annotation data is determined according to the evaluation results corresponding to each group of sample annotation data in the training sample data, the target sample annotation data includes multiple sample prompt texts and sample response data corresponding to each sample prompt text, the number of sample prompt texts in the target sample annotation data is less than the number of sample prompt texts in other sample annotation data, and the number of sample response data in the target sample annotation data is greater than the number of sample response data in other sample annotation data.

[0167] This specification includes several embodiments that, when the total number of labeled data points in each group of sample labeled data is the same, further determine the final data labeling method by comparing the number of labeled points in the sample prompt text and the number of labeled points in the sample response data. It combines the word segmentation distribution and the distribution of different word segmentation combinations in the sample prompt text to determine the diversity of the sample prompt text; and based on the acquired sample prompt text, it expands the data to generate expanded prompt text, further improving the diversity of the sample prompt text. Finally, it uses a data processing model trained on the target sample labeled data to process the text to be processed and the prompt text to be processed, outputting corresponding response data, thus improving the accuracy of the output content of the data processing model.

[0168] Corresponding to the above method embodiments, this specification also provides data processing apparatus embodiments. Figure 8 A schematic diagram of the structure of a data processing apparatus according to one embodiment of this specification is shown. Figure 8 As shown, the device includes:

[0169] The acquisition module 802 is configured to acquire the text to be processed and the corresponding prompt text to be processed.

[0170] Input module 804 is configured to input the text to be processed and the prompt text to be processed into a data processing model to obtain response data output by the data processing model. The data processing model is trained based on sample text and target sample annotation data corresponding to the sample text. The target sample annotation data is determined according to the evaluation results corresponding to each group of sample annotation data in the training sample data. The target sample annotation data includes multiple sample prompt texts and sample response data corresponding to each sample prompt text. The number of sample prompt texts in the target sample annotation data is less than the number of sample prompt texts in other sample annotation data, and the number of sample response data in the target sample annotation data is greater than the number of sample response data in other sample annotation data.

[0171] Optionally, the device further includes:

[0172] The sample acquisition module is configured to acquire the sample text and the training sample data, wherein the training sample data includes at least two sets of sample annotation data, the sample annotation data includes sample prompt text and sample response data corresponding to the sample prompt text, the total number of annotation data in each set of sample annotation data is the same, and the number of sample prompt texts is different;

[0173] The sample input module is configured to input the sample text and each group of sample annotation data into the data processing model to obtain the predicted response data corresponding to each group of sample annotation data.

[0174] The response evaluation module is configured to evaluate each predicted response data and obtain the evaluation results corresponding to each predicted response data.

[0175] The annotation determination module is configured to determine the target predicted response data based on the evaluation results corresponding to each predicted response data, and to determine the data annotation method of the target sample annotation data corresponding to the target predicted response data as the target annotation method.

[0176] Optionally, the sample acquisition module is further configured to:

[0177] Obtain the preset labeled data threshold;

[0178] Based on the preset annotation data threshold, at least two sets of sample prompt texts with different numbers are obtained, wherein the number of sample prompt texts in each set is less than or equal to the preset annotation data threshold;

[0179] Based on the preset annotation data threshold and the number of sample prompt texts in each group, obtain the sample response data corresponding to the sample prompt texts in each group;

[0180] Training sample data is obtained based on the prompt text of each group of samples and the corresponding sample response data of each group of samples.

[0181] Optionally, the response evaluation module is further configured as follows:

[0182] Each predicted response data is input into the evaluation model to obtain the evaluation result corresponding to each predicted response data output by the evaluation model, wherein the evaluation model is trained based on the user feedback data corresponding to the sample response data.

[0183] Optionally, the device further includes:

[0184] The word segmentation module is configured to segment the prompt text of each group of samples to obtain the initial prompt word segmentation text corresponding to each group of sample prompt text.

[0185] The calculation module is configured to calculate the diversity index of each group of sample prompt texts based on the number of initial prompt texts and the number of sample prompt texts in each group.

[0186] Optionally, the computing module is further configured to:

[0187] Identify the words to be filtered in the initial prompt word segmentation text to be processed, wherein the initial prompt word segmentation text to be processed is any one of the groups of initial prompt word segmentation texts;

[0188] Based on the segmented words to be filtered, the initial prompt segmented text to be processed is filtered to obtain the target prompt segmented text to be processed corresponding to the initial prompt segmented text to be processed;

[0189] Based on the number of each target word in the target prompt word segmentation text to be processed and the number of sample prompt texts to be processed, the diversity index of the sample prompt texts to be processed is calculated, wherein the sample prompt texts to be processed are the sample prompt texts corresponding to the initial prompt word segmentation text to be processed.

[0190] Optionally, the computing module is further configured to:

[0191] Based on the number of each target word in the target prompt segmented text to be processed, determine the proportion of the target words to be processed corresponding to the target prompt segmented text to be processed;

[0192] Calculate the product of the percentage of words to be segmented and the number of sample prompt texts to be processed;

[0193] The product is determined as the diversity index of the prompt text of the sample to be processed.

[0194] Optionally, the device further includes:

[0195] The construction module is configured to construct a two-dimensional dataset corresponding to each set of sample prompt texts based on the diversity index of each set of sample prompt texts and the evaluation results corresponding to each set of sample prompt texts.

[0196] The fitting module is configured to perform linear fitting on the two-dimensional dataset corresponding to each group of sample prompt texts to obtain the linear fitting results for each group of sample prompt texts.

[0197] Optionally, the device further includes:

[0198] The text acquisition module is configured to acquire the original sample prompt text.

[0199] The text extraction module is configured to extract the following prompt text corresponding to each group of sample prompt text from the original sample prompt text;

[0200] The text generation module is configured to generate expanded prompt text based on the sample prompt text for each group and the subsequent prompt text for each group.

[0201] Optionally, the text generation module is further configured to:

[0202] The prompt text of each group of samples is segmented into words to obtain the initial prompt segmented text corresponding to each group of sample prompt texts. The following prompt text of each group is segmented into words to obtain the initial following prompt text corresponding to each group of following prompt texts.

[0203] The initial prompt segmented text to be processed is segmented and filtered to obtain the target prompt segmented text to be processed corresponding to the initial prompt segmented text to be processed. The initial following text to be processed is segmented and filtered to obtain the target following text to be processed corresponding to the initial following text to be processed. The initial prompt segmented text to be processed is any one of the initial prompt segmented texts in each group, and the initial following text to be processed is any one of the initial following texts in each group.

[0204] Based on the target prompt segmented text and the target context segmented text, calculate the overlap coefficient between the target prompt segmented text and the target context segmented text;

[0205] Based on the overlap coefficients, select the text to be expanded from the following text in each group;

[0206] Based on the sample prompt texts for each group and the prompt texts to be expanded below, expandable prompt texts are generated.

[0207] Optionally, the text generation module is further configured to:

[0208] Based on the number of each target word in the target prompt word segmentation text to be processed, determine the first unprocessed word segmentation ratio corresponding to the target prompt word segmentation text to be processed; based on the number of each target word in the target following word segmentation text to be processed, determine the second unprocessed word segmentation ratio corresponding to the target following word segmentation text to be processed.

[0209] Based on the first unprocessed word segmentation ratio and the second unprocessed word segmentation ratio, calculate the overlap coefficient between the first unprocessed word segmentation ratio and the second unprocessed word segmentation ratio.

[0210] The data processing apparatus provided in this specification includes: an acquisition module configured to acquire a text to be processed and a corresponding prompt text to be processed; and an input module configured to input the text to be processed and the prompt text to be processed into a data processing model to obtain response data output by the data processing model. The data processing model is trained based on sample text and target sample annotation data corresponding to the sample text. The target sample annotation data is determined based on the evaluation results corresponding to each group of sample annotation data in the training sample data. The target sample annotation data includes multiple sample prompt texts and sample response data corresponding to each sample prompt text. The number of sample prompt texts in the target sample annotation data is less than the number of sample prompt texts in other sample annotation data, and the number of sample response data in the target sample annotation data is greater than the number of sample response data in other sample annotation data.

[0211] This specification includes several embodiments that, when the total number of labeled data points in each group of sample labeled data is the same, further determine the final data labeling method by comparing the number of labeled points in the sample prompt text and the number of labeled points in the sample response data. It combines the word segmentation distribution and the distribution of different word segmentation combinations in the sample prompt text to determine the diversity of the sample prompt text; and based on the acquired sample prompt text, it expands the data to generate expanded prompt text, further improving the diversity of the sample prompt text. Finally, it uses a data processing model trained on the target sample labeled data to process the text to be processed and the prompt text to be processed, outputting corresponding response data, thus improving the accuracy of the output content of the data processing model.

[0212] The above is an illustrative scheme of a data processing apparatus according to this embodiment. It should be noted that the technical solution of this data processing apparatus and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the data processing apparatus, please refer to the description of the technical solution of the data processing method described above.

[0213] See Figure 9 , Figure 9 A flowchart of a data processing model training method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0214] Step 902: Obtain the sample text and the target sample annotation data corresponding to the sample text.

[0215] The target sample annotation data is determined based on the evaluation results corresponding to each group of sample annotation data in the training sample data. The target sample annotation data includes multiple sample prompt texts and sample response data corresponding to each sample prompt text. The number of sample prompt texts in the target sample annotation data is less than the number of sample prompt texts in other sample annotation data, and the number of sample response data in the target sample annotation data is greater than the number of sample response data in other sample annotation data.

[0216] Step 904: Input the sample text and the target sample annotation data into the data processing model to obtain the prediction data output by the data processing model.

[0217] Step 906: Evaluate the predicted data to obtain the evaluation results of the predicted data.

[0218] Step 908: Adjust the model parameters of the data processing model according to the evaluation results, and continue training the data processing model until the training stop condition is met.

[0219] Specifically, based on the evaluation results corresponding to the labeled data of each group of samples in the training sample data, the target labeled data corresponding to the sample text is determined. The sample text and target labeled data are input into the data processing model to obtain predicted data. The predicted data is then evaluated to obtain the corresponding evaluation results. Specifically, the predicted data can be evaluated using an evaluation model to obtain the evaluation results. The model parameters of the data processing model are then adjusted based on the evaluation results, and the data processing model continues to be trained until the training stopping condition is met.

[0220] The data processing model is used to generate predicted data (response data) for the sample text. At this point, the data processing model is not yet trained, and there will be a deviation between the generated predicted data and the actual sample response data. It is necessary to adjust the model parameters of the data processing model accordingly. Specifically, the evaluation results corresponding to the predicted data are determined, and the model parameters of the data processing model are adjusted based on the evaluation results. The adjusted model parameters are then used to continue training the data processing model for the next batch of sample text and target sample labeled data until the model training stops.

[0221] Specifically, the model training stopping conditions include the evaluation result corresponding to the predicted data reaching the evaluation threshold and / or the training rounds reaching a preset number of rounds.

[0222] In one specific embodiment provided in this specification, the training stopping condition is taken as the evaluation result corresponding to the predicted data reaching the evaluation threshold. The evaluation threshold is 0.6. When the evaluation result corresponding to the predicted data reaches 0.6, the data processing model is considered to have completed training.

[0223] In another specific embodiment provided in this specification, taking a preset number of training rounds as the training stopping condition as an example, the preset number of training rounds is 30 rounds. When the training rounds of the target sample labeled data reach 30 rounds, the data processing model is considered to have completed training.

[0224] In another specific embodiment provided in this specification, two training stopping conditions are set: an evaluation threshold and a preset training round. At the same time, the evaluation result and the training round are determined. When either the evaluation result or the training round meets the training stopping condition, the data processing model is considered to have completed training.

[0225] One embodiment of this specification implements training a data processing model based on target sample labeled data with a determined target labeling method, thereby improving the accuracy of training the data processing model.

[0226] Corresponding to the above method embodiments, this specification also provides embodiments of a data processing model training device. Figure 10 A schematic diagram of a data processing model training apparatus according to one embodiment of this specification is shown. Figure 10 As shown, the device includes:

[0227] The data acquisition module 1002 is configured to acquire sample text and target sample annotation data corresponding to the sample text. The target sample annotation data is determined based on the evaluation results corresponding to each group of sample annotation data in the training sample data. The target sample annotation data includes multiple sample prompt texts and sample response data corresponding to each sample prompt text. The number of sample prompt texts in the target sample annotation data is less than the number of sample prompt texts in other sample annotation data, and the number of sample response data in the target sample annotation data is greater than the number of sample response data in other sample annotation data.

[0228] The data input module 1004 is configured to input the sample text and the target sample annotation data into the data processing model to obtain the prediction data output by the data processing model;

[0229] The data evaluation module 1006 is configured to evaluate the prediction data and obtain the evaluation result of the prediction data.

[0230] The parameter adjustment module 1008 is configured to adjust the model parameters of the data processing model according to the evaluation results, and continue training the data processing model until the training stop condition is met.

[0231] One embodiment of this specification implements training a data processing model based on target sample labeled data with a determined target labeling method, thereby improving the accuracy of training the data processing model.

[0232] The above is an illustrative scheme of a data processing model training device according to this embodiment. It should be noted that the technical solution of this data processing model training device and the technical solution of the data processing model training method described above belong to the same concept. For details not described in detail in the technical solution of the data processing model training device, please refer to the description of the technical solution of the data processing model training method described above.

[0233] See Figure 11 , Figure 11 A flowchart of a sample set construction method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0234] Step 1102: Receive a sample set construction request sent by the target terminal, wherein the sample set construction request carries the target text and the sample set to be constructed corresponding to the target text.

[0235] The sample set construction method is applied to the server. A sample set construction request is a request sent by the target terminal to construct a sample set. Target text refers to the text data corresponding to the already annotated sample set (prompt text and response data). The sample set to be constructed refers to a sample set that has already been annotated but still needs further annotation adjustments.

[0236] Step 1104: Input the target text and the sample set to be constructed into the data processing model to obtain the predicted response data corresponding to the sample set to be constructed.

[0237] Step 1106: Evaluate the predicted response data to obtain the evaluation results corresponding to the predicted response data.

[0238] Step 1108: Based on the evaluation results, generate a sample set to construct feedback information.

[0239] Sample set construction feedback information refers to the feedback information used to adjust the annotations of the sample set to be constructed. For example, sample set construction feedback information could be to reduce the number of annotations for sample prompt text by 30%, or to increase the number of annotations for sample response data by 10%.

[0240] Step 1110: Send the sample set construction feedback information to the target terminal.

[0241] After generating the sample set construction feedback information, the sample set construction feedback information is sent to the target terminal so that the user can adjust the annotation method of the sample set to be constructed.

[0242] In one embodiment of this specification, a sample set construction request is received from a target terminal, and the annotation method of the sample set to be constructed is evaluated to generate sample set construction feedback information for the sample set to be constructed. This allows the user to adjust the annotation method of the sample set to be constructed based on the sample set construction feedback information, thereby improving the accuracy of determining the data annotation method.

[0243] Corresponding to the above method embodiments, this specification also provides embodiments of a sample set construction apparatus. Figure 12 A schematic diagram of a sample set construction apparatus provided in one embodiment of this specification is shown. Figure 12 As shown, this device is used in a server and includes:

[0244] The request receiving module 1202 is configured to receive a sample set construction request sent by the target terminal, wherein the sample set construction request carries target text and the sample set to be constructed corresponding to the target text;

[0245] The sample set input module 1204 is configured to input the target text and the sample set to be constructed into the data processing model to obtain the predicted response data corresponding to the sample set to be constructed.

[0246] The prediction data evaluation module 1206 is configured to evaluate the prediction response data and obtain the evaluation result corresponding to the prediction response data.

[0247] The information generation module 1208 is configured to generate a sample set to construct feedback information based on the evaluation results.

[0248] The information sending module 1210 is configured to send the sample set construction feedback information to the target terminal.

[0249] In one embodiment of this specification, a sample set construction request is received from a target terminal, and the annotation method of the sample set to be constructed is evaluated to generate sample set construction feedback information for the sample set to be constructed. This allows the user to adjust the annotation method of the sample set to be constructed based on the sample set construction feedback information, thereby improving the accuracy of determining the data annotation method.

[0250] The above is a schematic scheme of a sample set construction apparatus according to this embodiment. It should be noted that the technical solution of this sample set construction apparatus and the technical solution of the sample set construction method described above belong to the same concept. For details not described in detail in the technical solution of the sample set construction apparatus, please refer to the description of the technical solution of the sample set construction method described above.

[0251] Figure 13 A structural block diagram of a computing device 1300 according to one embodiment of this specification is shown. The components of the computing device 1300 include, but are not limited to, a memory 1310 and a processor 1320. The processor 1320 is connected to the memory 1310 via a bus 1330, and a database 1350 is used to store data.

[0252] The computing device 1300 also includes an access device 1340, which enables the computing device 1300 to communicate via one or more networks 1360. Examples of such networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 1340 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.13 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0253] In one embodiment of this specification, the above-described components of the computing device 1300 and Figure 13 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 13 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0254] The computing device 1300 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 1300 can also be a mobile or stationary server.

[0255] The processor 1320 is used to execute the following computer program / instructions, which, when executed by the processor, implement the steps of the above method.

[0256] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the above method belong to the same concept, and all details not described in detail in the technical solution of the computing device can be referred to the description of the technical solution of the above method.

[0257] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0258] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the method described above belong to the same concept, and all details not described in detail in the technical solution of the storage medium can be referred to the description of the technical solution of the method described above.

[0259] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0260] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the above method belong to the same concept, and all details not described in detail in the technical solution of the computer program product can be referred to the description of the technical solution of the above method.

[0261] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0262] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0263] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0264] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0265] The preferred embodiments disclosed above are merely illustrative of this specification. Optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data processing method, comprising: Obtain the text to be processed and the corresponding prompt text to be processed; The text to be processed and the prompt text to be processed are input into a data processing model to obtain the response data output by the data processing model. The data processing model is trained based on the sample text and the target sample annotation data corresponding to the sample text. The target sample annotation data is any one of the sample annotation data in each group of training sample data. The training sample data includes at least two groups of sample annotation data using different data annotation methods. The data annotation method is the annotation ratio between the number of annotations for the sample prompt text and the number of annotations for the sample response data. The target sample annotation data is determined based on the evaluation results obtained by evaluating the predicted response data corresponding to each group of sample annotation data in the training sample data. The target sample annotation data includes multiple sample prompt texts and sample response data corresponding to each sample prompt text. The number of sample prompt texts in the target sample annotation data is less than the number of sample prompt texts in other sample annotation data, and the number of sample response data in the target sample annotation data is greater than the number of sample response data in other sample annotation data.

2. The method of claim 1, further comprising: Obtain the sample text and the training sample data, wherein the training sample data includes at least two sets of sample annotation data, the sample annotation data includes sample prompt text and sample response data corresponding to the sample prompt text, the total number of annotation data in each set of sample annotation data is the same, and the number of sample prompt texts is different; The sample text and the labeled data of each group of samples are input into the data processing model to obtain the predicted response data corresponding to each group of labeled data. Each predicted response data point is evaluated to obtain the evaluation results corresponding to each predicted response data point; Based on the evaluation results corresponding to each predicted response data, the target predicted response data is determined, and the data annotation method of the target sample annotation data corresponding to the target predicted response data is determined as the target annotation method.

3. The method as described in claim 2, wherein obtaining the training sample data includes: Obtain the preset labeled data threshold; Based on the preset annotation data threshold, at least two sets of sample prompt texts with different numbers are obtained, wherein the number of sample prompt texts in each set is less than or equal to the preset annotation data threshold; Based on the preset annotation data threshold and the number of sample prompt texts in each group, obtain the sample response data corresponding to the sample prompt texts in each group; Training sample data is obtained based on the prompt text of each group of samples and the corresponding sample response data of each group of samples.

4. The method as described in claim 2, wherein evaluating each predicted response data and obtaining the evaluation result corresponding to each predicted response data includes: Each predicted response data is input into the evaluation model to obtain the evaluation result corresponding to each predicted response data output by the evaluation model, wherein the evaluation model is trained based on the user feedback data corresponding to the sample response data.

5. The method of claim 2, further comprising: Segment the prompt text of each group of samples to obtain the initial segmented prompt text corresponding to each group of samples. The diversity index of the sample prompt texts for each group is calculated based on the number of initial prompt texts and the number of sample prompt texts for each group.

6. The method as described in claim 5, wherein the diversity index of each group of sample prompt texts is calculated based on the number of initial prompt texts and the number of sample prompt texts in each group, including: Identify the words to be filtered in the initial prompt word segmentation text to be processed, wherein the initial prompt word segmentation text to be processed is any one of the groups of initial prompt word segmentation texts; Based on the segmented words to be filtered, the initial prompt segmented text to be processed is filtered to obtain the target prompt segmented text to be processed corresponding to the initial prompt segmented text to be processed; Based on the number of each target word in the target prompt word segmentation text to be processed and the number of sample prompt texts to be processed, the diversity index of the sample prompt texts to be processed is calculated, wherein the sample prompt texts to be processed are the sample prompt texts corresponding to the initial prompt word segmentation text to be processed.

7. The method as described in claim 6, wherein the diversity index of the sample prompt text to be processed is calculated based on the number of each target segmentation word in the target prompt text to be processed and the number of sample prompt texts to be processed, comprising: Based on the number of each target word in the target prompt segmented text to be processed, determine the proportion of the target words to be processed corresponding to the target prompt segmented text to be processed; Calculate the product of the percentage of words to be segmented and the number of sample prompt texts to be processed; The product is determined as the diversity index of the prompt text of the sample to be processed.

8. The method of claim 5, further comprising, after calculating the diversity index of the prompt text for each group of samples: Based on the diversity index of the prompt texts in each group of samples and the evaluation results corresponding to the prompt texts in each group of samples, a two-dimensional dataset corresponding to the prompt texts in each group of samples is constructed. Linear fitting is performed on the two-dimensional dataset corresponding to the prompt text of each group of samples to obtain the linear fitting results for each group of sample prompt text.

9. The method of claim 2, further comprising: Obtain the original sample prompt text; Extract the subsequent prompt text corresponding to each group of sample prompt text from the original sample prompt text; Based on the sample prompt text for each group and the subsequent prompt text for each group, expandable prompt text is generated.

10. The method of claim 9, wherein expanding prompt text is generated based on the sample prompt text of each group and the following prompt text of each group, comprising: The prompt text of each group of samples is segmented into words to obtain the initial prompt segmented text corresponding to each group of sample prompt texts. The following prompt text of each group is segmented into words to obtain the initial following prompt text corresponding to each group of following prompt texts. The initial prompt segmented text to be processed is segmented and filtered to obtain the target prompt segmented text to be processed corresponding to the initial prompt segmented text to be processed. The initial following text to be processed is segmented and filtered to obtain the target following text to be processed corresponding to the initial following text to be processed. The initial prompt segmented text to be processed is any one of the initial prompt segmented texts in each group, and the initial following text to be processed is any one of the initial following texts in each group. Based on the target prompt segmented text and the target context segmented text, calculate the overlap coefficient between the target prompt segmented text and the target context segmented text; Based on the overlap coefficients, select the text to be expanded from the following text in each group; Based on the sample prompt texts for each group and the prompt texts to be expanded below, expandable prompt texts are generated.

11. The method of claim 10, wherein calculating the overlap coefficient between the target prompt segmented text and the target context segmented text based on the target prompt segmented text and the target context segmented text, includes: Based on the number of each target word in the target prompt word segmentation text to be processed, determine the first unprocessed word segmentation ratio corresponding to the target prompt word segmentation text to be processed; based on the number of each target word in the target following word segmentation text to be processed, determine the second unprocessed word segmentation ratio corresponding to the target following word segmentation text to be processed. Based on the first unprocessed word segmentation ratio and the second unprocessed word segmentation ratio, calculate the overlap coefficient between the first unprocessed word segmentation ratio and the second unprocessed word segmentation ratio.

12. A data processing model training method, comprising: Obtain sample text and corresponding target sample annotation data, wherein the target sample annotation data is any one of the sample annotation data from each group of training sample data, the training sample data includes at least two groups of sample annotation data using different data annotation methods, the data annotation method being the annotation ratio between the number of annotations for sample prompt text and the number of annotations for sample response data, the target sample annotation data is determined based on the evaluation results obtained by evaluating the predicted response data corresponding to each group of sample annotation data in the training sample data, the target sample annotation data includes multiple sample prompt texts and sample response data corresponding to each sample prompt text, the number of sample prompt texts in the target sample annotation data is less than the number of sample prompt texts in other sample annotation data, and the number of sample response data in the target sample annotation data is greater than the number of sample response data in other sample annotation data; The sample text and the target sample annotation data are input into the data processing model to obtain the predicted data output by the data processing model; The predicted data is evaluated to obtain the evaluation results of the predicted data; Adjust the model parameters of the data processing model based on the evaluation results, and continue training the data processing model until the training stop condition is met.

13. A method for constructing a sample set, applied to a server, comprising: Receive a sample set construction request sent by the target terminal, wherein the sample set construction request carries the target text and the sample set to be constructed corresponding to the target text; The target text and the sample set to be constructed are input into the data processing model to obtain the predicted response data corresponding to the sample set to be constructed. The data processing model is trained based on the sample text and the target sample annotation data corresponding to the sample text. The target sample annotation data is any one of the sample annotation data in each group of training sample data. The training sample data includes at least two groups of sample annotation data using different data annotation methods. The data annotation method is the annotation ratio between the number of annotations in the sample prompt text and the number of annotations in the sample response data. The target sample annotation data is determined based on the evaluation results obtained by evaluating the predicted response data corresponding to each group of sample annotation data in the training sample data. The predicted response data is evaluated to obtain the evaluation results corresponding to the predicted response data; Based on the evaluation results, sample set construction feedback information is generated. The sample set construction feedback information refers to feedback information used to adjust the annotation of the sample set to be constructed. The annotation adjustment includes adjusting either the number of annotations for the target text prompt text or the number of annotations for the target text response data. The sample set construction feedback information is sent to the target terminal.

14. A computing device, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 13.

15. A computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 13.

16. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 13.

Images