Model fine-tuning data determination method and apparatus, electronic device, and storage medium

Abstract

The present application provides a model fine-tuning data determination method and device, electronic equipment and storage medium, relating to the field of artificial intelligence and machine learning. The method is to obtain an original data set for downstream task fine-tuning and fine-tuning resource constraint conditions; based on the fine-tuning resource constraint conditions, a data selection strategy is constructed; based on the data selection strategy, a reference data subset is determined from the original fine-tuning data set; the reference data subset and the fine-tuning resource constraint conditions are used to fine-tune the target model to obtain fine-tuned model parameters; the fine-tuned model parameters are used to update the data selection strategy, and in the case that the fine-tuned target model meets the preset convergence condition, the updated data selection strategy is used to determine a target fine-tuning data subset from the original data set, thereby dynamically balancing the quality, quantity and distribution characteristics of the data subset, selecting data beneficial to the improvement of model performance, reducing the fine-tuning cost and improving the model convergence speed and verification performance.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and machine learning, and more specifically, to a method, apparatus, electronic device, and storage medium for determining model fine-tuning data. Background Technology

[0002] With the rapid growth in the size of deep learning models and the amount of fine-tuning data, the enormous computational resources required for full-data fine-tuning have become a major bottleneck restricting their widespread application in real-world scenarios. To improve fine-tuning efficiency, researchers have proposed various data subset selection methods to reduce fine-tuning overhead by selecting representative samples. However, most existing techniques employ static or predefined data selection strategies, failing to directly incorporate computational resource constraints into the optimization process. This makes it difficult to adaptively balance the relationship between data quality, quantity, and model performance under different budget conditions. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method, apparatus, electronic device and storage medium for determining model fine-tuning data.

[0004] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows: In a first aspect, the present invention provides a method for determining model fine-tuning data, the method comprising: Obtain the original dataset and fine-tuning resource constraints for downstream task fine-tuning; Based on the aforementioned fine-tuning resource constraints, a data selection strategy is constructed; Based on the data selection strategy, a reference data subset is determined from the original dataset; The target model is fine-tuned using the reference data subset and the fine-tuning resource constraints to obtain the fine-tuned model parameters; The data selection strategy is updated using the model parameters, and if the fine-tuned target model meets the preset convergence condition, a target fine-tuned data subset is determined from the original fine-tuned dataset based on the updated data selection strategy.

[0005] Optionally, the step of determining a reference data subset from the original fine-tuning dataset based on the data selection strategy includes: Based on the data selection strategy, a candidate data subset distribution is generated; The original dataset is sampled according to the distribution of the candidate data subset to obtain the reference data subset.

[0006] Optionally, the data selection strategy corresponds to a sample granularity, and the step of generating a candidate data subset distribution according to the data selection strategy includes: Each sample data in the original dataset is modeled to obtain a first probability for each sample data. The first probability represents the probability that the sample data is used for model fine-tuning. The candidate data subset distribution includes the first probability for each sample data.

[0007] Optionally, the data selection strategy corresponds to a data granularity of data source granularity, and the step of generating a candidate data subset distribution according to the data selection strategy includes: All sample data in the original fine-tuning dataset are grouped to obtain multiple sample data groups corresponding to different data sources; Each of the sample data groups is modeled to obtain a second probability for each sample data group. The second probability represents the probability that the sample data group is used for model fine-tuning. The candidate data subset distribution includes the second probability for each of the sample data groups.

[0008] Optionally, the step of updating the data selection strategy using the fine-tuned model parameters includes: The performance of the fine-tuned model parameters was evaluated based on the validation dataset to obtain the validation loss. The data selection strategy is updated based on the verification loss and the pre-built loss proxy function.

[0009] Optionally, the step of updating the data selection strategy based on the verification loss and the pre-built loss surrogate function includes: Calculate the update gradient with respect to the data selection strategy based on the verification loss and the loss surrogate function; The data selection strategy is iteratively adjusted according to the update gradient to obtain the updated data selection strategy.

[0010] Optionally, the method further includes: If the fine-tuned target model does not meet the preset convergence condition, a new reference data subset is determined from the original dataset based on the updated data selection strategy, and the step of fine-tuning the target model using the reference data subset and the fine-tuning resource constraints is re-executed until the fine-tuned target model meets the preset convergence condition.

[0011] Secondly, the present invention provides a model fine-tuning data determination device, the device comprising: The acquisition module is used to acquire the original dataset and fine-tuning resource constraints for downstream task fine-tuning; The determination module is used to construct a data selection strategy based on the fine-tuning resource constraints; determine a reference data subset from the original dataset based on the data selection strategy; fine-tune the target model using the reference data subset and the fine-tuning resource constraints to obtain the fine-tuned model parameters; update the data selection strategy using the fine-tuned model parameters; and determine the target fine-tuning data subset from the original dataset based on the updated data selection strategy, provided that the fine-tuned target model meets the preset convergence conditions.

[0012] Thirdly, the present invention provides an electronic device including a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, the processor executing the machine-executable instructions to implement the model fine-tuning data determination method described in the first aspect above.

[0013] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the model fine-tuning data determination method as described in the first aspect above.

[0014] The model fine-tuning data determination method, apparatus, electronic device, and storage medium provided in this invention involve: acquiring the original dataset and fine-tuning resource constraints for downstream task fine-tuning; constructing a data selection strategy based on the fine-tuning resource constraints; determining a reference data subset from the original dataset based on the data selection strategy; fine-tuning the target model using the reference data subset and the fine-tuning resource constraints to obtain the fine-tuned model parameters; updating the data selection strategy using the fine-tuned model parameters; and determining the target fine-tuning data subset from the original dataset based on the updated data selection strategy, provided that the fine-tuned target model meets preset convergence conditions. Because this invention generates a data selection strategy based on fine-tuning resource constraints and forms a closed-loop learning mechanism by iteratively updating the data selection strategy and model parameters, it adaptively generates the optimal target fine-tuning data subset under preset convergence conditions. This dynamically balances the quality, quantity, and distribution characteristics of the data subset, selects the samples or data sources that contribute most to improving model performance, and significantly reduces fine-tuning costs while improving model convergence speed and final validation performance.

[0015] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This figure shows a schematic block diagram of an electronic device provided by an embodiment of the present invention; Figure 2 This illustration shows a flowchart of a model fine-tuning data determination method provided by an embodiment of the present invention. Figure 1 ; Figure 3 The diagram illustrates a fitting process for constructing a loss surrogate function according to an embodiment of the present invention. Figure 4 This illustration shows a flowchart of a model fine-tuning data determination method provided by an embodiment of the present invention. Figure 2 ; Figure 5 The diagram shows a functional block diagram of a model fine-tuning data determination device provided in an embodiment of the present invention.

[0018] Icons: 100 - Electronic device; 110 - Memory; 120 - Processor; 130 - Communication module; 200 - Model fine-tuning data determination device; 201 - Acquisition module; 202 - Determination module. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0020] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0021] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0022] With the widespread application of large pre-trained models (LPMs) represented by the Transformer architecture in downstream tasks such as visual understanding, instruction following, and multimodal inference, their parameter count has reached billions to hundreds of billions. Fine-tuning these models for downstream tasks has become the mainstream adaptation paradigm. However, their high computational cost is increasingly becoming a key bottleneck for practical implementation, especially in typical scenarios such as edge device fine-tuning, personalized adaptation in low-resource environments, and rapid iteration of enterprise-level private models. These scenarios often face strict and quantifiable computational budget constraints, such as: the maximum number of forward-backward propagation steps allowed under GPU memory limitations; the fixed FLOPs limit for a single fine-tuning task; and the total training time limited under cloud-based per-second billing models.

[0023] Maximizing validation performance within a given budget, rather than simply pursuing a reduction in training loss, has become a core challenge in improving the efficiency of fine-tuning large models and reducing deployment costs.

[0024] Therefore, in order to achieve adaptive allocation of fine-tuning data under strict budget, and to improve data utilization efficiency and fine-tuning speed while ensuring or even improving the final performance of the model, this invention provides a method, apparatus, electronic device and storage medium for determining model fine-tuning data, which will be described in detail below.

[0025] Please refer to Figure 1 This is a block diagram of electronic device 100. Electronic device 100 includes a memory 110, a processor 120, and a communication module 130. The memory 110, processor 120, and communication module 130 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0026] The memory 110 is used to store programs or data. The memory 110 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0027] The processor 120 is used to read / write data or programs stored in the memory 110 and to perform corresponding functions.

[0028] The communication module 130 is used to establish a communication connection between the electronic device 100 and other communication terminals via a network, and to send and receive data via the network.

[0029] It should be understood that, Figure 1 The structure shown is only a schematic diagram of the electronic device 100. The electronic device 100 may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0030] Please refer to Figure 2 The model fine-tuning data determination method provided in this embodiment of the invention includes steps S101 to S105.

[0031] S101, obtain the original dataset and fine-tuning resource constraints for downstream task fine-tuning.

[0032] In this embodiment of the invention, the original dataset D={x_i,y_i} to be used is first obtained, where x_i represents the i-th input sample and y_i is its corresponding label or target output.

[0033] The original dataset can come from a single data source or be aggregated from multiple heterogeneous data sources, such as text / image collections from different acquisition devices, corpora, or labeled quality.

[0034] Meanwhile, the currently available fine-tuning resource constraints, namely the total computational budget C, can be quantified in terms of "forward-backward propagation steps," representing the maximum number of gradient updates or equivalent computational cost allowed during the entire fine-tuning process.

[0035] The total computing budget C reflects the limitations of hardware computing power, time overhead, or energy consumption in the actual deployment environment, and is one of the core parameters driving subsequent data selection decisions.

[0036] S102, based on fine-tuning resource constraints, construct a data selection strategy.

[0037] After obtaining the computational budget C, a parameterizable data selection strategy is constructed to dynamically determine which data should be prioritized for model fine-tuning.

[0038] The data selection strategy aims to achieve joint control over data volume, data quality, and data distribution, thereby maximizing the performance of the final model on the validation set within the budget C.

[0039] The data selection strategy is constructed using a probabilistic modeling approach: a set of learnable selection parameters s are introduced, and the sampling distribution of the candidate subset is adjusted by optimizing s.

[0040] Depending on the specific application scenario, this embodiment of the invention provides two complementary data granularity levels, namely: (1) Data selection strategy at the sample granularity: Define the selection probability for each independent sample.

[0041] (2) Data selection strategy at the data source granularity: assign weights to sample groups with common source characteristics.

[0042] The two strategies described above can be flexibly switched or combined in data scenarios of different scales and structures, taking into account both fine-grained control and system scalability.

[0043] S103, Based on the data selection strategy, determine a subset of reference data from the original dataset.

[0044] In a possible implementation, step S103 can be implemented as follows: S103-1, Generate a candidate data subset distribution based on the data selection strategy.

[0045] As one possible implementation, the data selection strategy corresponds to the sample granularity. The implementation process of step S103-1 can be: modeling each sample data in the original dataset to obtain the first probability of each sample data.

[0046] Here, the first probability represents the probability that the sample data is used in the model, and the candidate data subset distribution includes the first probability of each sample data.

[0047] In other words, when the data selection strategy corresponds to the sample granularity, each sample in the original dataset is modeled, and its first probability of being selected for fine-tuning is estimated through a neural network or other differentiable function. The first probabilities of all samples constitute a binary mask distribution, resulting in the candidate data subset distribution, where each first probability is obtained by transforming the learnable parameter vector s through a sigmoid mapping or other normalization transformation.

[0048] In this case, the candidate data subset distribution is a random subset generation mechanism defined by the independent Bernoulli distribution. This mechanism supports differentiable approximations of the discrete sampling process through the policy gradient method, thereby supporting end-to-end fine-tuning.

[0049] As another possible implementation, the data selection strategy corresponds to the data granularity of the data source. The implementation process of step S103-1 can be: grouping all sample data in the original dataset to obtain multiple sample data groups corresponding to different data sources; modeling each sample data group to obtain the second probability of each sample data group.

[0050] The second probability represents the probability that the sample data set is used for model fine-tuning, and the candidate data subset distribution includes the second probability for each sample data set.

[0051] In other words, when the data selection strategy corresponds to the data source granularity, all samples in the original dataset are first grouped according to their source information to form multiple non-overlapping sample data groups. Each sample data group represents a specific data source (such as a certain type of sensor, a certain corpus, a certain domain, etc.).

[0052] Subsequently, each sample data group is modeled to learn its corresponding second probability, representing the likelihood of the group being sampled as a whole. All second probabilities form a weight vector, yielding the distribution of the candidate data subset, where each second profile is also derived from a learnable parameter s.

[0053] To enhance fine-tuning stability and avoid extreme weight allocation, embodiments of this invention employ a truncated Gaussian distribution or a softmax distribution with a temperature coefficient to model the sampling process of the candidate data subset distribution. This helps to suppress the influence of noisy data sources while maintaining diversity.

[0054] S103-2, the original dataset is sampled according to the distribution of candidate data subsets to obtain a reference data subset.

[0055] Based on the candidate data subset distribution generated above (whether at the sample granularity or the data source granularity), several candidate subsets m are extracted from the original dataset using the Monte Carlo sampling method. Each subset m satisfies the following condition: (1) Its size is indirectly controlled by budget C; (2) The learning outcome reflects the current chosen strategy.

[0056] In this embodiment of the invention, each outer layer update employs K independent samples (K being a small integer, 2~5) to estimate the desired gradient and reduce variance. The final selected subset (one or more) serves as the reference data subset m upon which this inner layer fine-tuning is based.

[0057] S104. The target model is fine-tuned using a subset of reference data and fine-tuning resource constraints to obtain the fine-tuned model parameters.

[0058] The target model is fine-tuned using the reference data subset m obtained in the previous step until the preset computational budget C is reached, and the model parameters at this point are output. .

[0059] The fine-tuning process is performed under a fixed budget C, allowing only a limited number of forward and backward propagation operations. For example, if a single round of complete traversal of subset m requires t steps, then at most C / t rounds of iteration will be run to ensure that resource constraints are not exceeded.

[0060] It is important to note that traditional two-layer optimization frameworks require solving for the inner layer's optimal solution, but this process is costly and difficult to converge in large-scale scenarios. Therefore, this invention transforms the inner layer's optimality constraint into a penalty term in the outer layer's objective function, and constructs a one-dimensional loss surrogate function l(m) to approximate the actual trend of the fine-tuning loss.

[0061] The loss surrogate function l(m) is defined as a differentiable function of the size |m| of the reference data subset or the number of effective samples. It can be a linear function, a log-linear function, or a piecewise fitted curve (such as cubic spline interpolation).

[0062] The parameters of the loss surrogate function l(m) are obtained by fitting historical data that has been finely tuned multiple times over short periods at different subset sizes. This simplifies the originally complex two-layer optimization problem into a single-layer differentiable optimization problem, significantly reducing the computational overhead of higher-order gradients.

[0063] Please see Figure 3 This illustrates a schematic diagram of the fitting process for constructing the loss surrogate function l(|m|) in an embodiment of the present invention. Figure 3 As shown, the horizontal axis represents the size of the selected reference data subset |m| (i.e., the number of effective samples participating in the fine-tuning), and the vertical axis represents the average fine-tuning loss value obtained after a finite number of rounds of fine-tuning of the subset under a fixed computational budget C.

[0064] By independently performing lightweight fine-tuning experiments on several typical subset-sized points (e.g., |m|=10k, 20k, 30k, 40k, 50k) in advance, and recording the corresponding fine-tuning loss observations for each point, a set of discrete data points is formed.

[0065] A one-dimensional continuous and differentiable surrogate function l(|m|) is constructed by fitting the trend of fine-tuning loss with subset size using a differentiable functional form.

[0066] Optionally, the original data can first be logarithmically transformed (i.e., the horizontal axis can be converted to log|m|), and then linear interpolation, least squares linear regression, or cubic spline interpolation can be used to fit the curve. When the data distribution exhibits nonlinear saturation characteristics (e.g., the loss decreases more slowly after a subset exceeds a certain threshold), piecewise fitting or spline smoothing strategies can be used to improve approximation accuracy and extrapolation stability.

[0067] The loss surrogate function l(|m|) is embedded in the outer optimization objective as a regularization term to estimate the expected convergence difficulty of fine-tuning under different selection strategies. Because it possesses an analytical gradient, the efficiency of a certain sampling distribution can be quickly evaluated without actually performing a full fine-tuning, thus achieving efficient gradient updates for the data selection strategy s.

[0068] Furthermore, during the model fine-tuning iteration, new fine-tuning loss observations can be continuously collected, and the fitting parameters of the surrogate function can be dynamically adjusted to achieve online adaptive correction. This mechanism effectively addresses the fluctuations in the loss surface caused by data distribution shifts, model stage changes, or noise interference, further enhancing the robustness of the embodiments of the present invention in complex task environments.

[0069] S105, update the data selection strategy using the fine-tuned model parameters, and determine the target fine-tuned data subset from the original dataset based on the updated data selection strategy, provided that the fine-tuned target model meets the preset convergence conditions.

[0070] In a possible implementation, the process of "updating the data selection strategy using the fine-tuned model parameters" could be as follows: evaluate the performance of the model parameters based on the validation dataset to obtain the validation loss; and update the data selection strategy based on the validation loss and a pre-built loss surrogate function.

[0071] Furthermore, in this embodiment of the invention, the process of "updating the data selection strategy according to the verification loss and the pre-built loss surrogate function" can be as follows: calculate the update gradient of the data selection strategy according to the verification loss and the loss surrogate function; iteratively adjust the data selection strategy according to the update gradient to obtain the updated data selection strategy.

[0072] Understandably, this is first based on the validation dataset. Calculate the validation loss of the current model. This metric directly reflects the generalization performance of the model and is the basis for the objective function of the outer layer optimization.

[0073] Next, combining the verification loss Using the pre-constructed loss surrogate function l(m), compute the update gradient with respect to the data selection parameters s.

[0074] Alternatively, the gradient can be calculated as follows: (1) Gradient expression in the θ direction:

[0075] (2) Gradient expression in the s direction:

[0076] In the formula, This represents the expectation of the reference data subset m; To verify the loss; This is the fine-tuning loss on the reference data subset m; for The proxy loss is a differentiable approximation with respect to the subset size |m|; α is a weight hyperparameter used to balance the effects of the term, and can be set to 100.

[0077] Finally, based on the calculated update gradient, the selection parameters s are iteratively adjusted using a standard first-order optimization algorithm (such as SGD or Adam) to obtain the updated data selection strategy.

[0078] When the fine-tuned target model meets the preset convergence conditions on the validation dataset (such as the loss no longer decreasing after several consecutive validation rounds, reaching the maximum number of iterations, or meeting the accuracy threshold), the adjustment of the data selection strategy can be stopped, and the final target fine-tuning data subset can be determined from the original dataset based on the current data selection strategy. This subset is the most representative and effective data set for fine-tuning under a given computational budget C, and can be used for subsequent model deployment or incremental learning.

[0079] Please refer to Figure 4 The method for determining the model fine-tuning data also includes step S106.

[0080] S106, if the fine-tuned target model does not meet the preset convergence condition, based on the updated data selection strategy, a new reference data subset is determined from the original dataset, and the step of fine-tuning the target model using the reference data subset and fine-tuning resource constraints is re-executed until the fine-tuned target model meets the preset convergence condition.

[0081] Understandably, if the target model after fine-tuning has not yet met the preset convergence condition, the process returns to step S103, resamples based on the updated data selection strategy to generate a new subset of reference data, and then performs the model fine-tuning process in step S104 again.

[0082] This closed-loop feedback mechanism enables the coordinated evolution of data selection and model fine-tuning. As the data selection strategy is continuously optimized, the selected subset of reference data gradually focuses on high-value, high signal-to-noise ratio data regions, thereby accelerating model convergence and improving final performance within a limited budget.

[0083] Throughout the iteration process, the loss surrogate function l(m) continuously receives feedback from new fine-tuning results, dynamically adjusting its fitting parameters to ensure that the approximation of the fine-tuning loss always maintains high accuracy and robustness. Furthermore, strategies such as temperature annealing and distribution regularization can be introduced to prevent policy collapse or premature convergence to local optima.

[0084] To more clearly demonstrate the practical application effect of the model fine-tuning data determination method provided in the embodiments of the present invention, the embodiments of the present invention provide examples in two artificial intelligence tasks: one is an image classification task (based on the MNIST dataset), used to verify the performance of the method provided in the embodiments of the present invention in small-scale fine control scenarios; the other is an instruction fine-tuning task (based on a mixed language data source), used to verify the scalability and robustness of the method provided in the embodiments of the present invention in a large-scale multi-source data environment.

[0085] Example 1: Image classification experiment based on the MNIST dataset Obtain the original MNIST dataset, which contains 60,000 images of handwritten digits across 10 categories. Divide the data into a fine-tuning set (50,000 samples) and a validation set (10,000 samples). All images are standardized (pixels normalized to the [0,1] interval), and basic data augmentation strategies (such as random translation and rotation) are applied to improve generalization ability.

[0086] Multiple fine-tuning resource constraints C are set, corresponding to budget levels with equivalent forward-backward propagation steps of 200, 400 and 600 respectively, to simulate fine-tuning scenarios under different computing power constraints.

[0087] The target model architecture is a standard convolutional neural network (e.g., a variant of LeNet-5), which is fine-tuned using the cross-entropy loss function and the Adam optimizer.

[0088] Set the following comparison method: Random selection method: In each iteration, a subset of the original dataset that meets the budget is randomly sampled; PBCS method: A baseline selection method based on gradient matching of data subsets.

[0089] Perform the steps S101 to S106 as described above: In S102, the selection probability of each sample is initialized to form the initial data selection strategy s; In S103, Monte Carlo sampling is performed based on the current s to generate a reference data subset m; In S104, the model is fine-tuned using subset m until the budget C is exhausted, and the final model parameters are recorded. In S105, based on the Top-1 accuracy feedback on the validation set, combined with the pre-constructed one-dimensional log-linear surrogate loss function l(|m|), the policy gradient with respect to s is calculated and the selection distribution is updated. If the convergence condition is not met (e.g., the accuracy change is less than 0.1% for three consecutive rounds), then proceed to S106 and repeat the iterative optimization process.

[0090] Throughout the fine-tuning process, the loss surrogate function l(m) is initially fitted by short-cycle fine-tuning of different subset sizes (such as 10k, 20k, and 30k samples), and then dynamically updated using linear interpolation after logarithmic transformation to ensure the stability of gradient estimation.

[0091] The table below shows the Top-1 accuracy of each method on the validation set under different budget conditions:

[0092] Experimental results show that, under the same or lower computational budget, the method provided by the embodiments of the present invention is superior to existing technologies such as random selection and PBCS, and has a faster convergence speed.

[0093] Furthermore, analysis of the selected target fine-tuning data subset revealed that the embodiments of the present invention tend to retain handwritten samples with clear edges and standard shapes, effectively avoiding low-quality data that is blurry or has high noise, and possessing adaptive recognition capabilities for data quality.

[0094] Example 2: Fine-tuning experiment based on mixed instruction data We construct a multi-source instruction fine-tuning task, integrating publicly available corpora from Alpaca and Alpaca-GPT4 as the original dataset.

[0095] The two types of data have significant differences: Alpaca-GPT4 data has higher quality and a more complete inference chain, but the acquisition cost is high; Alpaca data has a wide coverage but contains more noise.

[0096] The raw data was divided into two data source groups based on their source: D_1 (i.e., Alpaca) and D_2 (i.e., Alpaca-GPT4), with a total of approximately 50,000 samples.

[0097] Set a uniform calculation budget C=2 10 4 Step (forward-backward propagation), the validation set consists of high-quality question-answer pairs selected by humans.

[0098] The GPT-2 architecture is used as the basic language model, the task is instruction conformation generation, the evaluation metric is perplexity, and the optimizer is AdamW.

[0099] The comparison methods include: Uniform baseline: Fine-tunes by uniformly sampling all data sources; Full-dataset fine-tuning: Fine-tuning is done using all the data (beyond budget).

[0100] In this embodiment of the invention, weights r_1 and r_2 are learned for two data sources respectively, and sampling modeling is performed by truncating the normal distribution.

[0101] Perform the steps S101 to S106 as described above: In S102, define the data selection strategy at the data source granularity and initialize the sampling weights of the two sets of data. In S103, the data source is proportionally sampled according to the weight vector r to generate a subset of reference data. In S104, perform model fine-tuning within budget to obtain the current model parameters; In S105, based on the perplexity feedback of the validation dataset, the weight vector r is updated by combining the loss surrogate function of piecewise spline fitting. Repeat the process until the model perplexity converges.

[0102] The table below lists the final perplexity performance of each method on the test set:

[0103] The results show that, despite limitations in computational budget, the method provided by the embodiments of the present invention can automatically identify the importance of Alpaca-GPT4 data sources and gradually increase their sampling weights during the iteration process, thereby achieving lower perplexity than uniform sampling with limited resources, indicating that it has the ability to assess and dynamically allocate cross-source data value.

[0104] More importantly, the embodiments of the present invention achieve better language generation quality with less than 60% computational overhead of full fine-tuning, and are efficient and practical in real-world deployment scenarios.

[0105] To perform the corresponding steps in the above embodiments and various possible methods, an implementation of the model fine-tuning data determination device 200 is given below. Further, please refer to... Figure 5 , Figure 5 This is a functional block diagram of a model fine-tuning data determination device 200 provided in an embodiment of the present invention. It should be noted that the basic principle and technical effects of the model fine-tuning data determination device 200 provided in this embodiment are the same as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments. The model fine-tuning data determination device 200 includes: The acquisition module 201 is used to acquire the original fine-tuning dataset and fine-tuning resource constraints for downstream task fine-tuning.

[0106] The determination module 202 is used to construct a data selection strategy based on fine-tuning resource constraints; determine a reference data subset from the original dataset based on the data selection strategy; fine-tune the target model using the reference data subset and fine-tuning resource constraints to obtain the fine-tuned model parameters; update the data selection strategy using the fine-tuned model parameters; and determine the target fine-tuned data subset from the original fine-tuned dataset based on the updated data selection strategy, provided that the fine-tuned target model meets the preset convergence conditions.

[0107] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory 110 shown is either stored in or embedded in the operating system (OS) of the electronic device 100, and can be used by... Figure 1 The processor 120 executes the program. Meanwhile, the data and program code required to execute the above modules can be stored in the memory 110.

[0108] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0109] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0110] If the functionality is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0111] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for determining model fine-tuning data, characterized in that, The method includes: Obtain the original dataset and fine-tuning resource constraints for downstream task fine-tuning; Based on the aforementioned fine-tuning resource constraints, a data selection strategy is constructed; Based on the data selection strategy, a reference data subset is determined from the original dataset; The target model is fine-tuned using the reference data subset and the fine-tuning resource constraints to obtain the fine-tuned model parameters; The data selection strategy is updated using the fine-tuned model parameters, and a target data subset is determined from the original dataset based on the updated data selection strategy, provided that the fine-tuned target model meets the preset convergence conditions.

2. The method for determining model fine-tuning data as described in claim 1, characterized in that, The step of determining a reference data subset from the original dataset based on the data selection strategy includes: Based on the data selection strategy, a candidate data subset distribution is generated; The original dataset is sampled according to the distribution of the candidate data subset to obtain the reference data subset.

3. The method for determining model fine-tuning data as described in claim 2, characterized in that, The data selection strategy corresponds to a sample granularity, and the step of generating a candidate data subset distribution based on the data selection strategy includes: Each sample data in the original dataset is modeled to obtain a first probability for each sample data. The first probability represents the probability that the sample data is used for model fine-tuning. The candidate data subset distribution includes the first probability for each sample data.

4. The method for determining model fine-tuning data as described in claim 2, characterized in that, The data selection strategy corresponds to a data granularity of data source granularity. The step of generating a candidate data subset distribution based on the data selection strategy includes: All sample data in the original dataset are grouped to obtain multiple sample data groups corresponding to different data sources; Each of the sample data groups is modeled to obtain a second probability for each sample data group. The second probability represents the probability that the sample data group is used for model fine-tuning. The candidate data subset distribution includes the second probability for each of the sample data groups.

5. The method for determining model fine-tuning data as described in claim 1, characterized in that, The step of updating the data selection strategy using the fine-tuned model parameters includes: The performance of the fine-tuned model parameters was evaluated based on the validation dataset to obtain the validation loss. The data selection strategy is updated based on the verification loss and the pre-built loss proxy function.

6. The method for determining model fine-tuning data as described in claim 5, characterized in that, The step of updating the data selection strategy based on the verification loss and the pre-built loss proxy function includes: Calculate the update gradient with respect to the data selection strategy based on the verification loss and the loss surrogate function; The data selection strategy is iteratively adjusted according to the update gradient to obtain the updated data selection strategy.

7. The method for determining model fine-tuning data as described in claim 1, characterized in that, The method further includes: If the fine-tuned target model does not meet the preset convergence condition, a new reference data subset is determined from the original dataset based on the updated data selection strategy, and the step of fine-tuning the target model using the reference data subset and the fine-tuning resource constraints is re-executed until the fine-tuned target model meets the preset convergence condition.

8. A device for determining model fine-tuning data, characterized in that, The device includes: The acquisition module is used to acquire the original dataset and fine-tuning resource constraints for downstream task fine-tuning; The determination module is used to construct a data selection strategy based on the fine-tuning resource constraints; determine a reference data subset from the original dataset based on the data selection strategy; fine-tune the target model using the reference data subset and the fine-tuning resource constraints to obtain the fine-tuned model parameters; update the data selection strategy using the fine-tuned model parameters; and determine the target data subset from the original dataset based on the updated data selection strategy, provided that the fine-tuned target model meets the preset convergence conditions.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, the processor executing the machine-executable instructions to implement the model fine-tuning data determination method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the model fine-tuning data determination method as described in any one of claims 1-7.

Images