Method, device, electronic equipment and medium for selecting training data

Abstract

Embodiments of the present disclosure relate to methods, apparatuses, electronic devices and media for selecting training data. The method includes determining a first set of loss values for a training data set based on an initial model, the training data set including a plurality of labeled data points, the initial model being a pre-trained or fine-tuned model. The method further includes determining a second set of loss values for the training data set based on a reference model, the reference model being determined based on fine-tuning of the initial model on the training data set. In addition, the method further includes selecting a plurality of target data points for fine-tuning from the training data set based on a plurality of loss differences between the first set of loss values and the second set of loss values. According to embodiments of the present disclosure, the learning ability of the model on the data points in the training data set can be determined, the accuracy of the training data set and the pertinence to the model are improved, and thus the accuracy of the model fine-tuning and the model performance are improved.

Description

Technical Field

[0001] This disclosure generally relates to the field of computers, and more specifically to methods, apparatus, electronic devices, and media for selecting training data. Background Technology

[0002] With the continuous development of technology, network models such as image processing models and language processing models are being applied more and more widely. Among them, language processing models can handle a variety of natural language tasks, including situational question answering, text generation, and language translation, and have important value in various fields.

[0003] High-quality training datasets play a crucial role in various processing models. During the training process, the richness of the training dataset and the accuracy of the annotation information lead to better training results. As processing models continue to improve, the matching degree between the training dataset and the processing model is receiving increasing attention. Summary of the Invention

[0004] Embodiments of this disclosure provide a method, apparatus, electronic device, and medium for selecting training data.

[0005] In a first aspect of this disclosure, a method for selecting training data is provided. The method includes determining a first set of loss values ​​for a training dataset based on an initial model, the training dataset comprising multiple labeled data points, the initial model being a pre-trained or fine-tuned model. The method also includes determining a second set of loss values ​​for the training dataset based on a reference model determined by fine-tuning the initial model based on the training dataset. Furthermore, the method includes selecting multiple target data points from the training dataset for fine-tuning based on multiple loss differences between the first set of loss values ​​and the second set of loss values.

[0006] In a second aspect of this disclosure, an apparatus for determining data pairs is provided. The apparatus includes a first set of loss value determination module configured to determine a first set of loss values ​​for a training dataset based on an initial model, the training dataset comprising multiple labeled data points, the initial model being a pre-trained or fine-tuned model. The apparatus also includes a second set of loss value determination module configured to determine a second set of loss values ​​for the training dataset based on a reference model determined by fine-tuning the initial model using the training dataset. Furthermore, the apparatus includes a target data point selection module configured to select multiple target data points from the training dataset for fine-tuning based on multiple loss differences between the first set of loss values ​​and the second set of loss values.

[0007] In a third aspect of this disclosure, an electronic device is provided. The electronic device includes a processor and a memory coupled to the processor, the memory having instructions stored therein, which, when executed by the processor, cause the electronic device to perform the method according to the first aspect.

[0008] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores computer-executable instructions, which are executed by a processor to implement the method according to the first aspect.

[0009] The summary section is intended to present the chosen concepts in a simplified form, which will be further described in the detailed description below. The summary section is not intended to identify key or principal features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0011] Figure 1 A schematic diagram of an example environment in which some embodiments of this disclosure may be implemented is shown;

[0012] Figure 2 Flowcharts illustrating methods for selecting training data according to some embodiments of this disclosure are shown;

[0013] Figure 3 A schematic diagram illustrating the process of selecting training data according to some embodiments of this disclosure is shown;

[0014] Figure 4 A schematic diagram of a method for labeling data points based on loss difference, according to some embodiments of the present disclosure, is shown;

[0015] Figure 5 A schematic diagram illustrating the sorting and distribution of loss differences in some embodiments of this disclosure is shown.

[0016] Figure 6 A schematic diagram of another method for labeling data points based on loss difference, according to some embodiments of this disclosure, is shown;

[0017] Figure 7 Block diagrams of apparatus for selecting training data according to some embodiments of the present disclosure are shown; and

[0018] Figure 8 Block diagrams of electronic devices according to some embodiments of the present disclosure are shown.

[0019] In all the accompanying figures, the same or similar reference numerals denote the same or similar elements. Detailed Implementation

[0020] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0021] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0022] For example, upon receiving a user's proactive request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0023] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0024] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0025] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0026] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects unless explicitly stated. Other explicit and implicit definitions may also be included below.

[0027] Supervised fine-tuning (SFT) of network models plays a crucial role in improving training speed and performance. Supervised fine-tuning involves training a network model on a source dataset as the source model. Then, a new network model is created, copying all parameters from the source model except for the output layer. Since these parameters incorporate the learned experience from the source dataset, the new network model can handle the same or similar tasks as the source model. When fine-tuning the new network model, the output layer parameters are set, the output layer parameters are trained, and the remaining copied parameters are fine-tuned.

[0028] Supervised fine-tuning is a crucial step in training network models, such as Large Language Models (LLMs), and models that have undergone supervised fine-tuning often exhibit better performance. The most important aspect of fine-tuning is the training data used for it. Traditionally, data selection for fine-tuning is based on characteristics such as data quality and distribution. High-quality data ensures the network model learns accurate and reliable information, while data diversity helps the model adapt to various tasks and scenarios. However, traditional approaches do not consider the network model when selecting data; different network models may require different training data, potentially leading to a disconnect between the training data and the network model.

[0029] In the embodiments of this disclosure, a pre-trained model or a fine-tuned model is first selected as the initial model. This initial model is then fine-tuned using a training dataset to obtain a reference model. Next, multiple initial loss values ​​(referred to as the first set of loss values) are calculated for multiple data points in the training dataset using the initial model, and multiple reference loss values ​​(referred to as the second set of loss values) are calculated for the same data point using the reference model. Further, the loss difference between the initial loss value and the reference loss value for the same data point is calculated to determine the initial model's learning ability for that data point. Based on these multiple loss differences, data points in the training dataset are filtered to obtain multiple data points with higher learning ability for the initial model. Finally, the initial model is fine-tuned using these filtered data points.

[0030] In this way, the learning ability of the initial model to the data points is used as a reference factor during the selection of data points in the training dataset. This ensures a good match between the selected data points and the initial model, improving the accuracy of the dataset during initial model training. Furthermore, by selecting data points suitable for the initial model, the amount of data can be reduced during fine-tuning. For initial models that need to handle multiple task types, selecting data points for a specific task type allows for adjusting the proportion of data points of different task types in the dataset while maintaining fine-tuning effectiveness. This improves adaptability to multi-task scenarios and ultimately enhances model performance.

[0031] Figure 1 A schematic diagram of an example environment 100 in which some embodiments of this disclosure may be implemented is shown. Reference Figure 1 Example environment 100 can be set as computing device 102, which can be set as computing system, single server, distributed server, or cloud-based server, etc. It can also be set as user terminal, mobile device, computer, etc., or a combination of the above devices. For example, computing device 102 can be a combination of server and user terminal, based on which interaction with user is realized.

[0032] refer to Figure 1 A training dataset 104 is obtained and input into an initial model 106 (e.g., a pre-trained model or a fine-tuned model) and a reference model 108. The reference model 108 is obtained by fine-tuning the initial model 106 using the training dataset 104. The initial model 106 and the reference model 108 process multiple data points in the training dataset to obtain a set of initial loss values ​​(referred to as the first set of loss values) and a set of reference loss values ​​(referred to as the second set of loss values).

[0033] Continue to refer to Figure 1 After obtaining a set of initial loss values ​​and a set of reference loss values, the loss difference between the initial loss value and the reference loss value for the same data point is calculated. This loss difference reflects the learning ability of the initial model for that data point. Then, loss difference sorting 110 is performed to determine the sorting result of multiple loss differences, and multiple selected data points 112 (which may be called target data points) are determined from multiple data points in the training dataset 104 according to the sorting result.

[0034] Continue to refer to Figure 1After determining multiple selection data points 112, the initial model 106 is fine-tuned based on these selection data points 112 to obtain the final fine-tuned model 114. The fine-tuned model 114 can be a model for different tasks or scenarios, such as an image processing model, video processing model, text processing model, or speech processing model. When the fine-tuned model 114 is set to a text processing model, it can obtain the user's question 116, generate an answer 118 based on the question 116, and provide feedback to the user.

[0035] In the embodiments of this disclosure, during the screening of data points in the training dataset 104, the learning ability of the initial model 106 on the data points is used as a reference factor, thereby ensuring the matching between the screened data points and the initial model 106, and improving the accuracy of the dataset during the training of the initial model 106. Furthermore, by screening out data points suitable for the initial model 106, the amount of data can be reduced during fine-tuning. For the initial model 106, which needs to handle multiple types of tasks, screening data points of a specific task type can adjust the proportion of data points of different task types in the training dataset 104 while ensuring the fine-tuning effect, thus improving its adaptability to multi-task scenarios.

[0036] It should be understood that the architecture and functionality in example environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure. Embodiments of this disclosure can also be applied to other environments with different structures and / or functionalities.

[0037] The following will combine Figures 2 to 8 The process according to embodiments of this disclosure is described in detail. For ease of understanding, the specific data mentioned in the following description are exemplary and not intended to limit the scope of this disclosure. It is understood that the embodiments described below may also include additional actions not shown and / or actions shown may be omitted, and the scope of this disclosure is not limited in this respect.

[0038] Figure 2 A flowchart of a method 200 for selecting training data, according to some embodiments of the present disclosure, is shown. In some embodiments, Figure 1 The computing device 102 in the diagram can serve as the execution entity of method 200. In block 202, a first set of loss values ​​is determined based on an initial model for a training dataset, which includes multiple labeled data points, and the initial model is a pre-trained or fine-tuned model. In some embodiments, in Figure 1 In the example environment 100 shown, a training dataset 104 is obtained and input into the initial model 106.

[0039] In some embodiments, the initial model 106 may be a pre-trained model, such as a pre-trained large language model, image processing model, etc. Alternatively or additionally, the initial model 106 may also be a fine-tuned model. In some embodiments, the initial model 106 is a local model. After the training dataset 104 is directly input into the computing device 102, the computing device 102 can fine-tune the initial model 106 without sending the training dataset 104 online.

[0040] In box 204, a second set of loss values ​​is determined for the training dataset based on a reference model, which is determined by fine-tuning the initial model based on the training dataset. In some embodiments, in Figure 1 In the example environment 100 shown, a training dataset 104 is obtained and input into a reference model 108. The reference model 108 is a model obtained by fine-tuning the initial model 106 using the training dataset 104. It can be understood that the initial model 106 is the model before learning from the training dataset 104, and the reference model 108 is the model after learning from the training dataset 104. Therefore, the parameters in the reference model 108 reflect the experience gained from learning from the training dataset 104.

[0041] In box 206, multiple target data points for fine-tuning are selected from the training dataset based on multiple loss differences between the first set of loss values ​​and the second set of loss values. It can be understood that... Figure 1 In the example environment shown, since the parameters in the reference model 108 reflect the experience learned from the training dataset 104, the magnitude of the loss difference between the initial loss value determined based on the initial model 106 and the reference loss value determined based on the reference model 108 reflects the amount of experience learned in the reference model 108, or in other words, how well the initial model 106 learns from the training dataset 104.

[0042] In some embodiments, Figure 1 In the example environment 100 shown, after obtaining a set of initial loss values ​​through the initial model 106 and a set of reference loss values ​​through the reference model 108, the difference between the initial loss value and the reference loss value for the same data point is calculated, thereby obtaining multiple loss differences corresponding to multiple data points. Then, the multiple loss differences are sorted, and the data point with the larger loss difference (which can be called the target data point) is selected for fine-tuning the initial model 106.

[0043] In the embodiments of this disclosure, during the screening of data points in the training dataset 104, the learning ability of the initial model 106 on the data points is used as a reference factor, thereby ensuring the matching between the screened data points and the initial model 106, and improving the accuracy of the dataset during the training of the initial model 106. Furthermore, by screening out data points suitable for the initial model 106, the amount of data can be reduced during fine-tuning. For the initial model 106, which needs to handle multiple types of tasks, screening data points of a specific task type can adjust the proportion of data points of different task types in the training dataset 104 while ensuring the fine-tuning effect, thus improving its adaptability to multi-task scenarios.

[0044] Figure 3 A schematic diagram of a process 300 for selecting training data according to some embodiments of this disclosure is shown. Process 300 is mainly divided into three stages. In the first stage, a training dataset 302 and an initial model 304 are prepared and initialized. In some embodiments, the initial model 304 selects a trained network model. In some embodiments, for scenarios where the initial model 304 is a text or language processing model, the input of data points in the training dataset 302 can be set as prompt words, and the output can be set as feedback from the text or language processing model to the prompt words. The prompt words can be presented according to a certain format and structure.

[0045] In some embodiments, the initial model 304 is a local model. During the processing of the training dataset 302, it is not necessary to call an external model or upload the training dataset 302 to other terminals or servers over the network. This ensures the security of the training dataset 302, avoids data leakage during the training and fine-tuning of the initial model 304, and also reduces device deployment costs.

[0046] In some embodiments, training dataset 302 can be generated using an existing network model (referred to as a pre-prepared network model). First, multiple custom inputs (referred to as initial inputs) are set and fed into the existing network model. The existing network model processes the custom inputs to obtain multiple outputs (referred to as initial outputs). Further, the custom inputs and the outputs of the existing network model are used as data points to generate training dataset 302. This approach reduces the cost of obtaining training dataset 302, and since training dataset 302 also contains prior knowledge from the existing network model, the quality of training dataset 302 is improved while reducing generation costs.

[0047] In some embodiments, manually annotated data points can also be combined by mixing data points generated based on existing network models with manually annotated data points in a certain proportion to generate training dataset 302. In this way, the richness of data points in training dataset 302 can be improved, and training dataset 302 can simultaneously contain prior knowledge of existing network models, as well as prior knowledge and human preferences, thereby further improving the accuracy of training dataset 302.

[0048] Continue to refer to Figure 3 In the second stage, the training dataset 302 is input into the initial model 304 (e.g., using a pre-trained model), and the initial model 304 is trained using the training dataset 302 to obtain the reference model 306. In some embodiments, the training dataset 302 includes multiple labeled data points, each data point including input and output (which can serve as the label for the data point). The initial model 304 processes the input of the data points to obtain the output data of the initial model 304, i.e., the training output, and the parameters of the initial model 304 are adjusted based on the loss between the output of the data points and the training output. When the loss between the output of the data points and the training output satisfies the convergence condition, the reference model 306 can be obtained.

[0049] In this way, during the training and fine-tuning of the initial model 304 using the training dataset 302, the initial model 304 can learn the prior knowledge of the training dataset 302. Therefore, by comparing the output difference between the initial model 304 and the reference model 306, the degree of learning of the reference model 306 on the training dataset 302 can be determined, thereby conveniently determining the learning ability of the initial model 304 on the training dataset 302 and improving the efficiency of selecting the training dataset 302.

[0050] Continue to refer to Figure 3 In the third stage, after obtaining the reference model 306, the training dataset 302 is input to the initial model 304 and the reference model 306. The initial model 304 calculates a set of initial loss values ​​308 for multiple data points in the training dataset 302, and the reference model 306 calculates a set of reference loss values ​​310 for the same multiple data points. Then, multiple scores 312 are determined based on the multiple loss differences between the initial loss values ​​308 and the reference loss values ​​310. In some embodiments, the loss difference between the initial loss values ​​308 and the reference loss values ​​310 can be directly used as the score 312. Alternatively or additionally, the loss difference can be processed, and the processing result can be used as the score 312.

[0051] In some embodiments, data points in the training dataset 302 are obtained, and the input of the data points is processed by the initial model 304 to obtain the output of the initial model 304 (which may be referred to as the first output). The output of the data point is compared with the output obtained by the initial model 304 in processing the data point to obtain the loss value for that data point (which may be referred to as the loss value in the first set of loss values).

[0052] In some embodiments, data points from training dataset 302 are obtained, and the input of the data points is processed by reference model 306 to obtain the output of reference model 306 (which may be referred to as the second output). The output of the data point is compared with the output obtained by the reference model 306 to obtain the loss value for that data point (which may be referred to as the loss value in the second set of loss values). In some embodiments, the formula for calculating the above loss value is as follows:

[0053]

[0054] Where x and y are the input and output of data points in training dataset 302, respectively, and p(y i |x) represents the predicted probability of input x under the i-th token of each category (i.e., output) y, and Len(y) is the length of the output.

[0055] Continue to refer to Figure 3 After determining multiple scores 312, the scores 312 are sorted to obtain a sorting result 314. Based on the sorting result 314, data points in the training dataset 302 are filtered, and the data points corresponding to the top-ranked scores are selected as selected data points 316. Then, the initial model 304 is fine-tuned by selecting data points 316 to obtain the fine-tuned model.

[0056] Figure 4 A schematic diagram of a method 400 for labeling data points based on loss difference, according to some embodiments of the present disclosure, is shown. In some embodiments, at box 402, data points are acquired. Multiple data points are acquired from a training dataset for processing. At box 404, an initial loss value is determined. The data points are processed using an initial model to obtain an initial loss value. At box 406, a reference loss value is determined. The data points are processed using a reference model to obtain a reference loss value.

[0057] Continue to refer to Figure 4In box 408, determine whether both the initial loss value and the reference loss value are higher than the lower limit (which can be used as the lower limit of the first or second loss range). It's understandable that if a data point is too simple for the initial model, it won't improve the initial model's learning ability during fine-tuning, and therefore it's unsuitable for fine-tuning the initial model. If both the initial loss value and the reference loss value are too small, it indicates that the data point is too simple for both the initial and reference models. Therefore, both the initial loss value and the reference loss value should be above a lower limit, which can be set based on the actual scenario.

[0058] Continue to refer to Figure 4 In box 410, determine whether both the initial loss value and the reference loss value are below an upper limit (which can be used as the upper limit of the first or second loss range). It's understandable that if a data point is too complex for the initial model, it won't improve the initial model's learning ability during fine-tuning, and therefore, it's unsuitable for fine-tuning the initial model. If both the initial loss value and the reference loss value are too large, it indicates that the data point is too complex for both the initial and reference models. Therefore, both the initial loss value and the reference loss value should be below an upper limit, which can be set based on the actual scenario.

[0059] Continue to refer to Figure 4 In box 412, it is determined whether the difference between the initial loss value and the reference loss value meets the condition. It can be understood that when the initial model can learn a data point, it indicates that the data point has high learning value. The initial model's learning ability for a data point can be reflected in the difference between the initial loss value and the reference loss value. Therefore, the difference between the initial loss value and the reference loss value should be higher than a threshold, or the difference of that data point should be ranked relatively high among all data points.

[0060] In some embodiments, if the data points are too simple, both the initial loss value and the reference loss value are too small, resulting in a small difference between them. Conversely, if the data points are too complex, both the initial loss value and the reference loss value are too large, again resulting in a small difference. That is, when the condition in box 412 is met, the conditions in boxes 408 and 410 will also be met. Therefore, when the difference between the initial loss value and the reference loss value should be higher than a threshold, or when the difference of the data point is ranked relatively high among all data points, the corresponding data point can be directly determined as the data point for fine-tuning the initial model.

[0061] Continue to refer to Figure 4In box 412, mark the data points. When both the initial loss value and the reference loss value meet the judgment conditions in boxes 408-412, obtain multiple data points that meet the conditions, sort the multiple loss differences between the multiple initial loss values ​​and the multiple reference loss values ​​of the multiple data points in descending order, select the multiple data points with the higher ranking and mark them as data points for fine-tuning the initial model.

[0062] This approach ensures that sorting and filtering are only performed when the initial and reference loss values ​​meet the criteria. This eliminates data points that are too simple or too complex for the initial model, while retaining data points with learning value. This makes the selected data points more targeted to the initial model, thereby improving the effectiveness of fine-tuning the initial model.

[0063] Figure 5 A schematic diagram illustrating the sorting of loss differences and the distribution of reference loss values ​​in some embodiments of this disclosure is provided. In some embodiments, in the above-described manner, if both the initial loss value and the reference loss value are relatively large, it will result in a similarly large loss difference between the initial loss value and the reference loss value, thus conforming to the above-described... Figure 4 Condition 412. For example, refer to Figure 5 The x-axis represents the sorting of loss differences, with larger loss differences appearing first. The y-axis represents the loss value. The distribution of loss values ​​based on the sorting of loss differences roughly falls within the closed range formed by curves 502 and 504. At point 506, the loss difference is relatively large, but the corresponding reference loss value is also relatively large.

[0064] It's understandable that although the difference between the initial loss value and the reference loss value at point 506 meets the condition, the initial and reference loss values ​​are too large, making it too difficult for the initial model to handle. Therefore, the corresponding data point also needs to be excluded. This phenomenon is understandable because during fine-tuning, the tokens corresponding to the input might be masked from the loss calculation, resulting in only the loss related to the tokens corresponding to the output being calculated—that is, only the output-related loss, such as the average cross-entropy loss, is calculated. Since the magnitude of the loss is related to the length of the output, the output length of the top-ranked data points may increase, leading to situations where the initial and reference loss values ​​are too large, and their difference meets the condition.

[0065] Figure 6A schematic diagram of another method 600 for labeling data points based on loss difference, according to some embodiments of this disclosure, is shown. In some embodiments, at box 602, data points are acquired. Multiple data points are acquired from a training dataset. At box 604, an initial loss value is determined. The data points are input into an initial model, and an initial loss value for each data point is calculated using the initial model. At box 606, a reference loss value is determined. The data points are input into a reference model, and a reference loss value for each data point is calculated using the reference model. At box 608, a loss difference is determined. The difference in loss values ​​for each data point is calculated using the initial loss value and the reference loss value.

[0066] Continue to refer to Figure 6 In box 610, the normalization result is determined. In some embodiments, an initial loss value or a reference loss value is selected as the normalization term, and the loss difference is normalized to calculate the normalization result, wherein the normalization result can be the ratio of the loss difference to the initial loss value (referred to as the ratio in the first set of ratios) or the ratio of the loss difference to the reference loss value (referred to as the ratio in the second set of ratios).

[0067] Continue to refer to Figure 6 In box 612, sorting. After obtaining the normalized results, sort the data points based on multiple ratios of the loss difference to the initial loss value, or multiple ratios of the loss difference to the reference loss value, to select the data points with larger ratios. In box 614, marking data points. After obtaining the data points with larger ratios, mark the data points for subsequent fine-tuning of the initial model.

[0068] In this way, when both the initial and reference loss values ​​are relatively large, and the loss difference between them meets certain conditions, the ratio of the loss difference to the initial or reference loss value is calculated. Since the loss difference is much smaller than the initial or reference loss value, the calculated ratio will also be smaller, resulting in a lower ranking during the sorting process, thus effectively reducing the likelihood of a significant loss. Figure 5 The deletion of data point 506 in the model improved the accuracy of the data points used for fine-tuning the initial model, thereby improving the fine-tuning effect of the initial model.

[0069] In some embodiments, since the quality of the training dataset has a significant impact on the fine-tuning effect of the initial model, both high-quality and low-quality datasets can be used to describe the effects of the embodiments described herein. For example, if a language processing model is chosen as the initial model, the annotations (i.e., outputs) of both the high-quality and low-quality datasets are identical, but the quality of the inputs constituting the data points differs. Furthermore, a mixed dataset can be selected as a test set to test the functionality of various aspects such as text generation, mathematics, and encoding.

[0070] Then, after filtering the high-quality and low-quality datasets, it was found that the high-quality dataset only requires a small number of data points to achieve better performance than the entire dataset. For example, the high-quality dataset has 52k data points, while the filtered dataset may only have 3k data points, and the fine-tuning effect of the filtered data points is better than that of the entire dataset. For the low-quality dataset, the number of filtered data points needs to be higher than that of the high-quality dataset to achieve similar results.

[0071] In some embodiments, there is an upper limit to the effectiveness of data point selection for the complete training dataset. For example, when the number of selected data points accounts for 6% of the entire training dataset, the performance of the selected data points and the entire training dataset is similar when fine-tuning the initial model. When the proportion of selected data points in the entire training dataset rises to 20%, the performance of fine-tuning the initial model decreases, indicating that the fine-tuning of the initial model has reached its limit. This is understandable because when the proportion of selected data points is 6%, the data points are highly matched with the learning ability of the initial model, while when the proportion of selected data points is 20%, some data points with low matching degree are used for fine-tuning, resulting in a decrease in the fine-tuning performance of the initial model. Therefore, when selecting data points for the training dataset, it is also necessary to determine an appropriate number of data points to ensure that the fine-tuned model achieves the best results.

[0072] Figure 7 A block diagram of an apparatus 700 for selecting training data, according to some embodiments of the present disclosure, is shown. Figure 7 As shown, the apparatus 700 includes a first set of loss value determination module 702, configured to determine a first set of loss values ​​for a training dataset based on an initial model, the training dataset including multiple labeled data points, and the initial model including a pre-trained or fine-tuned model. The apparatus also includes a second set of loss value determination module 704, configured to determine a second set of loss values ​​for the training dataset based on a reference model, the reference model being determined based on fine-tuning of the initial model using the training dataset. Furthermore, the apparatus includes a target data point selection module 706, configured to select multiple target data points from the training dataset for fine-tuning based on multiple loss differences between the first set of loss values ​​and the second set of loss values.

[0073] Figure 8 A block diagram of an electronic device 800 according to some embodiments of the present disclosure is shown. Device 800 may be the device or apparatus described in the embodiments of the present disclosure. Figure 8As shown, device 800 includes a central processing unit (CPU) and / or a graphics processing unit (GPU) 802, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 804 or loaded from storage unit 816 into random access memory (RAM) 806. Various programs and data required for the operation of device 800 can also be stored in RAM 806. The CPU / GPU 802, ROM 804, and RAM 806 are interconnected via bus 808. Input / output (I / O) interface 810 is also connected to bus 808. Although not shown in... Figure 8 As shown, device 800 may also include a coprocessor.

[0074] Multiple components in device 800 are connected to I / O interface 810, including: input unit 812, such as keyboard, mouse, etc.; output unit 814, such as various types of monitors, speakers, etc.; storage unit 816, such as disk, optical disk, etc.; and communication unit 818, such as network card, modem, wireless transceiver, etc. Communication unit 818 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0075] The various methods or processes described above can be executed by CPU / GPU 802. For example, in some embodiments, the methods may be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 816. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 804 and / or communication unit 818. When the computer program is loaded into RAM 806 and executed by CPU / GPU 802, one or more steps or actions in the methods or processes described above may be performed.

[0076] In some embodiments, the methods and processes described above can be implemented as a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.

[0077] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0078] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, a local area network (LAN), a wide area network (WAN), and / or a wireless network, to an external computer or external storage device. The network may include copper cables, fiber optic cables, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0079] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​and conventional procedural programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0080] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0081] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0082] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0083] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0084] The following are some example implementations of this disclosure.

[0085] Example 1. A method for selecting training data, comprising:

[0086] The first set of loss values ​​is determined based on the initial model for the training dataset, which includes multiple labeled data points, and the initial model is a pre-trained or fine-tuned model.

[0087] A second set of loss values ​​is determined for the training dataset based on a reference model, which is determined by fine-tuning the initial model based on the training dataset; and

[0088] Based on multiple loss differences between the first set of loss values ​​and the second set of loss values, multiple target data points are selected from the training dataset for fine-tuning.

[0089] Example 2. According to the method described in Example 1, wherein the plurality of labeled data points include a plurality of inputs and a plurality of outputs, and determining the reference model includes:

[0090] Obtain the initial model;

[0091] Based on the initial model, determine multiple training outputs corresponding to the multiple inputs of the multiple labeled data points; and

[0092] The parameters in the initial model are trained based on the multiple outputs of the multiple labeled data points and the multiple training outputs.

[0093] Example 3. The method according to Example 1 or 2, wherein determining the loss value in the first set of loss values ​​includes:

[0094] Based on the initial model, determine the first output corresponding to the input of the labeled data points; and

[0095] Based on the output of the labeled data points and the first output, the loss value in the first group of loss values ​​is determined.

[0096] Example 4. The method according to any one of Examples 1-3, wherein determining the loss value in the second set of loss values ​​includes:

[0097] Based on the reference model, determine the second output corresponding to the input of the labeled data points; and

[0098] Based on the output of the labeled data points and the second output, the loss value in the second set of loss values ​​is determined.

[0099] Example 5. The method according to any one of Examples 1-4, wherein selecting a plurality of target data points from the training dataset for fine-tuning includes:

[0100] Determine the plurality of loss differences between the first set of loss values ​​and the second set of loss values;

[0101] Sort the plurality of loss differences; and

[0102] Based on the sorting results of the multiple loss differences, multiple target data points are selected from the training dataset for fine-tuning.

[0103] Example 6. The method according to any one of Examples 1-5, wherein selecting multiple target data points for fine-tuning from the training dataset based on the ranking result of the multiple loss differences includes:

[0104] Based on the plurality of loss differences and the first set of loss values, a first set of ratios is determined, or based on the plurality of loss differences and the second set of loss values, a second set of ratios is determined; and

[0105] Based on the sorting results of multiple ratios in the first group of ratios or the sorting results of multiple ratios in the second group of ratios, multiple target data points for fine-tuning are selected from the training dataset.

[0106] Example 7. The method according to any one of Examples 1-6, wherein selecting multiple target data points for fine-tuning from the training dataset based on the ranking result of the multiple loss differences includes:

[0107] Determine whether the loss value in the first group of loss values ​​is within the first loss range and whether the corresponding loss value in the second group of loss values ​​is within the second loss range;

[0108] In response to a situation where the loss value in the first set of loss values ​​falls within the first loss range and the corresponding loss value in the second set of loss values ​​falls within the second loss range, the corresponding labeled data point is determined as a candidate data point; and

[0109] Based on the ranking results of multiple loss differences corresponding to multiple candidate data points in the training dataset, multiple target data points are selected from the training dataset for fine-tuning.

[0110] Example 8. The method according to any one of Examples 1-7 further includes:

[0111] Obtain multiple initial inputs;

[0112] Based on a pre-prepared network model, obtain multiple initial outputs for the multiple initial inputs; and

[0113] The training dataset is determined based on the plurality of initial inputs and the plurality of initial outputs.

[0114] Example 9. The method according to any one of Examples 1-8, wherein determining the training dataset based on the plurality of initial inputs and the plurality of initial outputs includes:

[0115] Based on the multiple initial inputs and the multiple initial outputs, multiple initial data points are determined;

[0116] Obtain multiple manually annotated data points; and

[0117] The training dataset is determined based on the multiple initial data points and the multiple manually annotated data points.

[0118] Example 10. The method according to any one of Examples 1-9 further includes:

[0119] Acquire the multiple target data points; and

[0120] The parameters in the initial model are fine-tuned based on the multiple target data points.

[0121] Example 11. The method according to any one of Examples 1-10, wherein the finely tuned initial model is used to process at least one of images, videos, speech, and text.

[0122] Example 12. The method according to any one of Examples 1-11, wherein the fine-tuned initial model is a local model.

[0123] Example 13. An apparatus for determining data pairs, comprising:

[0124] The first set of loss value determination module is configured to determine the first set of loss values ​​for the training dataset based on the initial model. The training dataset includes multiple labeled data points, and the initial model is a pre-trained or fine-tuned model.

[0125] The second set of loss value determination modules is configured to determine a second set of loss values ​​for the training dataset based on a reference model, wherein the reference model is determined based on fine-tuning of the initial model using the training dataset; and

[0126] The target data point selection module is configured to select multiple target data points for fine-tuning from the training dataset based on multiple loss differences between the first set of loss values ​​and the second set of loss values.

[0127] Example 14. The apparatus according to Example 13, wherein the plurality of labeled data points include a plurality of inputs and a plurality of outputs, and the apparatus further includes:

[0128] The initial model acquisition module is configured to acquire the initial model;

[0129] The training output determination module is configured to determine, based on the initial model, multiple training outputs corresponding to the multiple inputs of the multiple labeled data points; and

[0130] The parameter training module is configured to train the parameters in the initial model based on the multiple outputs of the multiple labeled data points and the multiple training outputs.

[0131] Example 15. The apparatus according to Example 13 or 14, wherein the first set of loss value determination modules is further configured to:

[0132] Based on the initial model, determine the first output corresponding to the input of the labeled data points; and

[0133] Based on the output of the labeled data points and the first output, the loss value in the first group of loss values ​​is determined.

[0134] Example 16. The apparatus according to any one of Examples 13-15, wherein the second set of loss value determination modules is further configured to:

[0135] Based on the reference model, determine the second output corresponding to the input of the labeled data points; and

[0136] Based on the output of the labeled data points and the second output, the loss value in the second set of loss values ​​is determined.

[0137] Example 17. The apparatus according to any one of Examples 13-16, wherein the target data point selection module is further configured as follows:

[0138] Determine the plurality of loss differences between the first set of loss values ​​and the second set of loss values;

[0139] Sort the plurality of loss differences; and

[0140] Based on the sorting results of the multiple loss differences, multiple target data points are selected from the training dataset for fine-tuning.

[0141] Example 18. The apparatus according to any one of Examples 13-17, wherein the target data point selection module is further configured to:

[0142] Based on the plurality of loss differences and the first set of loss values, a first set of ratios is determined, or based on the plurality of loss differences and the second set of loss values, a second set of ratios is determined; and

[0143] Based on the sorting results of multiple ratios in the first group of ratios or the sorting results of multiple ratios in the second group of ratios, multiple target data points for fine-tuning are selected from the training dataset.

[0144] Example 19. The apparatus according to any one of Examples 13-18, wherein the target data point selection module is further configured to:

[0145] Determine whether the loss value in the first group of loss values ​​is within the first loss range and whether the corresponding loss value in the second group of loss values ​​is within the second loss range;

[0146] In response to a situation where the loss value in the first set of loss values ​​falls within the first loss range and the corresponding loss value in the second set of loss values ​​falls within the second loss range, the corresponding labeled data point is determined as a candidate data point; and

[0147] Based on the ranking results of multiple loss differences corresponding to multiple candidate data points in the training dataset, multiple target data points are selected from the training dataset for fine-tuning.

[0148] Example 20. The apparatus according to any one of Examples 13-19 further includes:

[0149] The initial input acquisition module is configured to acquire multiple initial inputs.

[0150] The initial output acquisition module is configured to acquire multiple initial outputs for the multiple initial inputs based on a pre-prepared network model; and

[0151] The training dataset determination module is configured to determine the training dataset based on the plurality of initial inputs and the plurality of initial outputs.

[0152] Example 21. The apparatus according to any one of Examples 13-20, wherein

[153] the training dataset determination module is further configured to:

[0153] Based on the multiple initial inputs and the multiple initial outputs, multiple initial data points are determined;

[0154] Obtain multiple manually annotated data points; and

[0155] The training dataset is determined based on the multiple initial data points and the multiple manually annotated data points.

[0156] Example 22. The apparatus according to any one of Examples 13-21 further includes:

[0157] The target data point acquisition module is configured to acquire the plurality of target data points; and

[0158] The parameter fine-tuning module is configured to fine-tune the parameters in the initial model based on the multiple target data points.

[0159] Example 23. The apparatus according to any one of Examples 13-22, wherein the finely tuned initial model is used to process at least one of images, videos, speech, and text.

[0160] Example 24. The apparatus according to any one of Examples 13-23, wherein the fine-tuned initial model is a local model.

[0161] Example 25. An electronic device comprising:

[0162] Processor; and

[0163] A memory coupled to the processor, the memory having instructions stored therein, which, when executed by the processor, cause the electronic device to perform actions, the actions including:

[0164] The first set of loss values ​​is determined based on the initial model for the training dataset, which includes multiple labeled data points, and the initial model is a pre-trained or fine-tuned model.

[0165] A second set of loss values ​​is determined for the training dataset based on a reference model, which is determined by fine-tuning the initial model based on the training dataset; and

[0166] Based on multiple loss differences between the first set of loss values ​​and the second set of loss values, multiple target data points are selected from the training dataset for fine-tuning.

[0167] Example 26. The electronic device according to Example 25, wherein the plurality of labeled data points include a plurality of inputs and a plurality of outputs, and determining the reference model includes:

[0168] Obtain the initial model;

[0169] Based on the initial model, determine multiple training outputs corresponding to the multiple inputs of the multiple labeled data points; and

[0170] The parameters in the initial model are trained based on the multiple outputs of the multiple labeled data points and the multiple training outputs.

[0171] Example 27. The electronic device according to Example 25 or 26, wherein determining the loss value in the first set of loss values ​​includes:

[0172] Based on the initial model, determine the first output corresponding to the input of the labeled data points; and

[0173] Based on the output of the labeled data points and the first output, the loss value in the first group of loss values ​​is determined.

[0174] Example 28. An electronic device according to any one of Examples 25-27, wherein determining the loss value in the second set of loss values ​​includes:

[0175] Based on the reference model, determine the second output corresponding to the input of the labeled data points; and

[0176] Based on the output of the labeled data points and the second output, the loss value in the second set of loss values ​​is determined.

[0177] Example 29. An electronic device according to any one of Examples 25-28, wherein selecting a plurality of target data points from the training dataset for fine-tuning includes:

[0178] Determine the plurality of loss differences between the first set of loss values ​​and the second set of loss values;

[0179] Sort the plurality of loss differences; and

[0180] Based on the sorting results of the multiple loss differences, multiple target data points are selected from the training dataset for fine-tuning.

[0181] Example 30. An electronic device according to any one of Examples 25-29, wherein selecting multiple target data points for fine-tuning from the training dataset based on the ranking result of the multiple loss differences includes:

[0182] Based on the plurality of loss differences and the first set of loss values, a first set of ratios is determined, or based on the plurality of loss differences and the second set of loss values, a second set of ratios is determined; and

[0183] Based on the sorting results of multiple ratios in the first group of ratios or the sorting results of multiple ratios in the second group of ratios, multiple target data points for fine-tuning are selected from the training dataset.

[0184] Example 31. The apparatus according to any one of Examples 25-30, wherein selecting a plurality of target data points for fine-tuning from the training dataset based on the ranking result of the plurality of loss differences comprises:

[0185] Determine whether the loss value in the first group of loss values ​​is within the first loss range and whether the corresponding loss value in the second group of loss values ​​is within the second loss range;

[0186] In response to a situation where the loss value in the first set of loss values ​​falls within the first loss range and the corresponding loss value in the second set of loss values ​​falls within the second loss range, the corresponding labeled data point is determined as a candidate data point; and

[0187] Based on the ranking results of multiple loss differences corresponding to multiple candidate data points in the training dataset, multiple target data points are selected from the training dataset for fine-tuning.

[0188] Example 32. The electronic device according to any one of Examples 25-31, wherein the operation further includes:

[0189] Obtain multiple initial inputs;

[0190] Based on a pre-prepared network model, obtain multiple initial outputs for the multiple initial inputs; and

[0191] The training dataset is determined based on the plurality of initial inputs and the plurality of initial outputs.

[0192] Example 33. An electronic device according to any one of Examples 25-32, wherein determining the training dataset based on the plurality of initial inputs and the plurality of initial outputs includes:

[0193] Based on the multiple initial inputs and the multiple initial outputs, multiple initial data points are determined;

[0194] Obtain multiple manually annotated data points; and

[0195] The training dataset is determined based on the multiple initial data points and the multiple manually annotated data points.

[0196] Example 34. The electronic device according to any one of Examples 25-33, wherein the operation further includes:

[0197] Acquire the multiple target data points; and

[0198] The parameters in the initial model are fine-tuned based on the multiple target data points.

[0199] Example 35. An electronic device according to any one of Examples 25-34, wherein the finely tuned initial model is used to process at least one of images, videos, speech, and text.

[0200] Example 36. An electronic device according to any one of Examples 25-35, wherein the fine-tuned initial model is a local model.

[0201] Example 37. A computer-readable storage medium having stored thereon computer-executable instructions, wherein the computer-executable instructions are executed by a processor to implement the method according to any one of Examples 1 to 12.

[0202] Example 38. A computer program product tangibly stored on a computer-readable medium and comprising computer-executable instructions that, when executed by a device, cause the device to perform the method according to any one of Examples 1 to 12.

[0203] Although this disclosure has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for data processing, comprising: The first set of loss values ​​is determined based on the initial model for the training dataset, which includes multiple labeled data points, and the initial model is a pre-trained or fine-tuned model. A second set of loss values ​​is determined for the training dataset based on a reference model, which is determined by fine-tuning the initial model based on the training dataset; Based on multiple loss differences between the first set of loss values ​​and the second set of loss values, multiple target data points are selected from the training dataset for fine-tuning the initial model; In response to the finely tuned initial model used to process the text, the user's question is obtained; Generate an answer based on the question; as well as The answer will be fed back to the user.

2. The method according to claim 1, wherein the plurality of labeled data points includes a plurality of inputs and a plurality of outputs, and determining the reference model includes: Obtain the initial model; Based on the initial model, determine the multiple training outputs corresponding to the multiple inputs of the multiple labeled data points; as well as The parameters in the initial model are trained based on the multiple outputs of the multiple labeled data points and the multiple training outputs.

3. The method of claim 2, wherein determining the loss value in the first set of loss values ​​comprises: Based on the initial model, determine the first output corresponding to the input of the labeled data points; as well as Based on the output of the labeled data points and the first output, the loss value in the first group of loss values ​​is determined.

4. The method of claim 3, wherein determining the loss value in the second set of loss values ​​comprises: Based on the reference model, determine the second output corresponding to the input of the labeled data points; as well as Based on the output of the labeled data points and the second output, the loss value in the second set of loss values ​​is determined.

5. The method of claim 1, wherein selecting a plurality of target data points from the training dataset for fine-tuning comprises: Determine the plurality of loss differences between the first set of loss values ​​and the second set of loss values; Sort the multiple loss differences; as well as Based on the sorting results of the multiple loss differences, multiple target data points are selected from the training dataset for fine-tuning.

6. The method of claim 5, wherein selecting a plurality of target data points for fine-tuning from the training dataset based on the ranking result of the plurality of loss differences comprises: Based on the plurality of loss differences and the first group of loss values, a first group of ratios is determined, or based on the plurality of loss differences and the second group of loss values, a second group of ratios is determined; as well as Based on the sorting results of multiple ratios in the first group of ratios or the sorting results of multiple ratios in the second group of ratios, multiple target data points for fine-tuning are selected from the training dataset.

7. The method of claim 5, wherein selecting multiple target data points for fine-tuning from the training dataset based on the ranking result of the plurality of loss differences comprises: Determine whether the loss value in the first group of loss values ​​is within the first loss range and whether the corresponding loss value in the second group of loss values ​​is within the second loss range; In response to a situation where the loss value in the first set of loss values ​​is within the first loss range and the corresponding loss value in the second set of loss values ​​is within the second loss range, the corresponding labeled data point is determined as a candidate data point. as well as Based on the ranking results of multiple loss differences corresponding to multiple candidate data points in the training dataset, multiple target data points are selected from the training dataset for fine-tuning.

8. The method according to claim 1, further comprising: Obtain multiple initial inputs; Based on a pre-prepared network model, obtain multiple initial outputs for the multiple initial inputs; as well as The training dataset is determined based on the plurality of initial inputs and the plurality of initial outputs.

9. The method of claim 6, wherein determining the training dataset based on the plurality of initial inputs and the plurality of initial outputs comprises: Based on the multiple initial inputs and the multiple initial outputs, multiple initial data points are determined; Obtain multiple manually annotated data points; as well as The training dataset is determined based on the multiple initial data points and the multiple manually annotated data points.

10. The method of claim 1, further comprising: Acquire the multiple target data points; as well as The parameters in the initial model are fine-tuned based on the multiple target data points.

11. The method of claim 10, wherein the finely tuned initial model is further used to process at least one of images, videos, and speech.

12. The method of claim 10, wherein the fine-tuned initial model is a local model.

13. An apparatus for data processing, comprising: The first set of loss value determination module is configured to determine the first set of loss values ​​for the training dataset based on the initial model. The training dataset includes multiple labeled data points, and the initial model is a pre-trained or fine-tuned model. The second set of loss value determination module is configured to determine a second set of loss values ​​for the training dataset based on a reference model, which is determined based on fine-tuning of the initial model using the training dataset. as well as The target data point selection module is configured to select multiple target data points from the training dataset for fine-tuning the initial model based on multiple loss differences between the first set of loss values ​​and the second set of loss values. The question retrieval module is configured to retrieve the user's question in response to the finely tuned initial model for text processing; The answer generation module is configured to generate an answer based on the question. as well as The response feedback module is configured to provide the response to the user.

14. An electronic device, comprising: processor; as well as A memory coupled to the processor, the memory having instructions stored therein, which, when executed by the processor, cause the electronic device to perform the method according to any one of claims 1 to 12.

15. A computer-readable storage medium having stored thereon computer-executable instructions, wherein the computer-executable instructions are executed by a processor to implement the method according to any one of claims 1 to 12.

Images