Model training method based on distribution calibration and computing device

Abstract

A model training method based on distribution calibration and a computing device, the method comprising: acquiring a training set, a test set and a verification set, the training set, the test set and the verification set respectively comprising sample data of different data distributions; obtaining a weight parameter corresponding to each piece of sample data in the verification set, wherein the weight parameter is in positive correlation with a first probability that the input data in the sample data belongs to the test set, and is in negative correlation with a second probability that the input data belongs to the training set; model parameters of the first model are adjusted based on the training set, in the adjusting process, weighting performance indexes of the first model are evaluated based on the verification set and the weight parameters, the size of the weight parameters corresponding to the sample data is in positive correlation with the influence degree of the sample data on the weighting performance indexes, and optimization of the weighting performance indexes is taken as a target. Searching is carried out in a hyper-parameter space, a plurality of target hyper-parameters are obtained, and the hyper-parameter space comprises value spaces of the plurality of hyper-parameters; and training a second model based on the target hyper-parameter and the training set.

Description

Technical Field

[0001] The embodiments in this specification belong to the field of machine learning technology, and in particular relate to a model training method and computing device based on distribution calibration. Background Technology

[0002] In today's digital age, machine learning technology has been widely applied to key business scenarios such as financial risk control, intelligent marketing, and personalized recommendations. In practical applications, the target customer group (hereinafter referred to as the customer base) faced by the model is usually not static. Under the influence of factors such as changes in the market environment, adjustments in operational strategies, and the evolution of user behavior, the characteristic distribution of the customer base will continuously and dynamically evolve over time. This phenomenon is called customer base distribution drift.

[0003] Current mainstream machine learning model training and hyperparameter optimization methods generally rely on pre-defined training and validation sets, implicitly assuming that the data in the training / validation sets follows the same distribution as the real online customer base. However, when the customer distribution shifts significantly, this assumption no longer holds, and the training and validation sets fail to accurately reflect the statistical characteristics of the actual online data. In this situation, if the automatic hyperparameter tuning framework (such as Bayesian optimization, grid search, etc.) still uses evaluation metrics (such as accuracy) on a fixed validation set as optimization feedback signals, it is highly likely to select hyperparameter combinations that perform well on historical validation sets but are unsuitable for the real online environment during the hyperparameter tuning process. Models trained with such hyperparameter combinations are prone to overfitting to historical data, exhibiting severely insufficient generalization ability, and ultimately failing to adapt to gradually shifting business data after deployment, resulting in a sharp performance decline.

[0004] Therefore, when customer distribution drifts, it is necessary to calibrate the data distribution in the training / validation sets with the real online customer base to achieve data alignment. Current research aligns the distributions of different datasets such as training, validation, and test sets by minimizing the KL (Kullback–Leibler) divergence or by using MMD (Maximum Mean Discrepancy)-based distribution alignment to mitigate the performance degradation caused by feature distribution drift. However, in high-dimensional, sparse feature scenarios, the KL divergence itself is difficult to estimate and optimize accurately and stably, resulting in limited distribution alignment effects and failing to fundamentally eliminate the performance degradation caused by distribution drift. Furthermore, these KL alignment methods are usually designed as independent preprocessing steps, first performing distribution correction on the data, and then separately training the model and searching for hyperparameters. The lack of end-to-end joint optimization between the two leads to a disconnect between distribution adaptation and model tuning, making it difficult for them to mutually promote each other and incurring additional computational overhead, resulting in low overall tuning efficiency. Summary of the Invention

[0005] The purpose of this invention is to provide a model training method and computing device based on distributed calibration, so as to integrate the hyperparameter tuning and distributed calibration of the model into the same end-to-end optimization process, thereby improving the tuning effect and computational efficiency.

[0006] The first aspect of this specification provides a model training method based on distribution calibration, the method comprising:

[0007] Obtain a training set, a test set, and a validation set. The training set, test set, and validation set each include sample data with different data distributions. The sample data includes input data for inputting into the model and label data as model training labels. The input data is user data, and the label data is user labels. The sample data in the test set was collected later than the training set and validation set.

[0008] Obtain the weight parameters corresponding to each sample data in the validation set. The weight parameters are positively correlated with the first probability that the input data in the sample data belongs to the test set and negatively correlated with the second probability that it belongs to the training set.

[0009] The model parameters of the first model are adjusted based on the training set. During the adjustment process, the weighted performance index of the first model is evaluated based on the validation set and the weight parameters. The magnitude of the weight parameters corresponding to the sample data is positively correlated with the degree of influence of the sample data on the weighted performance index. With the goal of optimizing the weighted performance index, a search is performed in the hyperparameter space to obtain several target hyperparameters. The hyperparameter space includes the value space of several hyperparameters.

[0010] The second model is trained based on the target hyperparameters and the training set.

[0011] A second aspect of this specification provides a computing device including a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method described in the first aspect.

[0012] The technical solution provided in this specification dynamically assigns different weight parameters to different sample data based on the probability that the sample data in the validation set belongs to the sample data in the training / test set. The closer the sample data in the validation set is to the test set collected later, the higher the weight parameter. Therefore, when calculating the weighted performance index during the hyperparameter search phase, more weight is assigned to sample data whose data distribution is closer to the test set. This allows the model trained with the target hyperparameters obtained through the search to focus more on samples similar to the test set. Since the samples in the test set are closer to the data distribution of the current real-world application scenario, the model trained in this way is better able to adapt to the current data distribution after deployment, thereby mitigating the performance degradation caused by time drift, improving the model's generalization ability, and extending the online service life. Furthermore, because the weight parameters used to calibrate the sample data distribution are applied to the calculation of performance indicators during the hyperparameter search phase, model tuning and distribution calibration are integrated into a single end-to-end optimization process, which can improve tuning effectiveness and computational efficiency. Attached Figure Description

[0013] To more clearly illustrate the technical solutions of the embodiments in this specification, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart illustrating a model training method based on distribution calibration in one embodiment;

[0015] Figure 2 This is a schematic diagram of each stage in a model training process based on distributed calibration in one embodiment;

[0016] Figure 3 This is a schematic diagram of a model training device based on distribution calibration in one embodiment. Detailed Implementation

[0017] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0018] To facilitate understanding of the solutions presented in this specification, the training set, validation set, and test set used for model training in current technologies are first explained below. The model in this specification can be a machine learning model, such as a tree model or a neural network model.

[0019] In supervised machine learning, the training set is the primary data source for model learning. The model continuously adjusts its parameters (such as weights and biases in a neural network) using data from the training set to minimize the loss function. The validation set is used to evaluate model performance during training, allowing for the adjustment of hyperparameters. Hyperparameters are parameters that cannot be automatically learned from training data and must be pre-set, such as the learning rate, batch size, number of layers, and dropout rate in neural network models, and the learning rate, maximum depth, and minimum sample split in tree models. By observing the model's performance on the validation set, it's possible to determine whether overfitting or underfitting has occurred, allowing for adjustments to the model structure or training strategy. For example, network depth, regularization strength, or optimization algorithms can be changed. A common practice is to evaluate the current model using samples from the validation set after every few epochs of training on the training set. For example, assuming the training set has 10,000 samples, and the batch size is set to 100 (meaning 100 samples are input to the model in each iteration), then each epoch will contain 10,000 / 100 = 100 mini-batches. During training, the model will process these 100 mini-batches sequentially. One epoch is completed when all 10,000 training samples have been used for forward propagation and parameter updates. The test set is the final evaluation standard for the model. It does not participate in the model's training or parameter tuning process; it is only used to verify the actual performance of the final model. A model's performance on the test set reflects its generalization ability on real-world data. Therefore, the test set should be as consistent as possible with the data distribution that the model may encounter after deployment.

[0020] Generally, training, validation, and test sets are used for the model training, validation, and testing phases, respectively. The division of these sets should avoid data leakage; that is, the same sample should not appear in multiple datasets simultaneously. Typically, a certain ratio (e.g., 6:2:2 or 8:1:1) is used, and the specific ratio can be adjusted according to the data volume and task requirements. Reasonable data partitioning is fundamental to ensuring the reliability of the evaluation results.

[0021] During the training phase, given the model structure and hyperparameters, the goal is to find parameters on the training set that minimize the loss function. Only the training set is used during training; validation and test sets are not used. Taking a neural network model as an example, a typical process includes first calculating the output and loss through forward propagation, then calculating the gradients of each parameter through backpropagation, and then iteratively updating the parameters using optimization algorithms such as SGD (Stochastic Gradient Descent) and Adam (Adaptive Moment Estimation). This process is usually performed epoch-wise and in mini-batch format to improve computational efficiency and stability. At this stage, data augmentation, regularization, and gradient pruning may also be combined to further enhance the model's robustness and convergence performance.

[0022] During the validation phase, such as after a complete training run or after several epochs during training, the currently trained model is used to make predictions on the validation set, and corresponding evaluation metrics are calculated. These metrics can include accuracy, F1 score, and AUC (Area Under the ROC Curve). Based on this, several candidate hyperparameter combinations are pre-defined or determined using automatic search methods such as random search or Bayesian optimization. For each candidate hyperparameter combination, the model is trained from scratch on the training set, and its performance is then evaluated on the validation set. The results of all candidate hyperparameter combinations on the validation set are compared, and the configuration that performs well on the validation set is selected. If necessary, strategies such as early stopping can be used to determine when to stop training based on the validation set performance to prevent overfitting. Understandably, once the final hyperparameter combination is selected, it can be used with the training set for final model training to obtain the trained model.

[0023] During the testing phase, the trained model is evaluated using a test set. In existing technologies, the test set should remain invisible throughout the development process and should not be used for training or tuning in model selection to ensure the objectivity and fairness of the evaluation. The model's metrics on the test set are considered an estimate of its ability to generalize to new data in real-world application environments. Test sets typically simulate real-world online data distributions as closely as possible; sometimes, multiple test sets are constructed based on business needs to evaluate the model's stability and robustness under different scenarios or time periods.

[0024] In this specification, the data sources for the training and validation sets can be past business data, log data, or public datasets from a specific business scenario. The test set is an out-of-time (OOT) dataset from the same business scenario, meaning data that is later than the training and validation sets in time. In actual modeling, the data collected from a specific business scenario can be segmented according to time. Data from earlier time periods can be used for the training and validation sets, while more recent data (later than the training / validation periods) can be used for the test set. This ensures that the data distribution in the test set closely approximates the actual situation faced after the model goes live. The time periods corresponding to the training and validation sets should generally be earlier than those of the test set; these can be adjacent consecutive time periods, or they can be randomly divided from earlier data according to a certain proportion.

[0025] The implementation process of the solution provided in this manual is described below.

[0026] Figure 1 This is a flowchart illustrating a method for model training based on distributed calibration in one embodiment of this specification. This method can be implemented by any device with computing and processing capabilities, and the entity executing the method can be any device, platform, server, or device cluster with computing and processing capabilities. Figure 1 As shown, the method includes the following steps:

[0027] In step 101, the training set, test set, and validation set are obtained.

[0028] The sample data in the test set was collected later than that in the training and validation sets. The sample data includes input data for the model and label data for model training. The input data is user data, and the label data is user labels. Different models may have different input and label data; this specification does not restrict the models used or the business scenarios in which they are applied. For example, in a financial risk control scenario, the financial risk control model can output a user's credit score based on relevant user characteristics. The higher the credit score, the higher the user's creditworthiness and the lower the risk. For instance, the model outputs a user default probability p∈[0,1], used to generate an anti-fraud score (1-p) and a credit score (binned and mapped to 300–850). The model's input data can include multi-dimensional features such as the user's education level, occupation type, number of credit accounts, credit inquiry records, and historical borrowing behavior. The label data can be the user's actual credit score. In a personalized recommendation scenario, the recommendation model can output a predicted click-through rate for a user clicking on a product based on the user's data. The input data for this model can include user click behavior records, browsing behavior records, purchase records, and collection records, while the tag data can be whether the user clicked on a certain product.

[0029] In this manual, due to different data collection times, the training set, test set, and validation set can each include sample data with different data distributions. This mainly refers to the different data distributions of the input data; that is, the input data in the training set, test set, and validation set are data from different time periods within the same feature space. The following explanation uses a financial risk control scenario. Suppose we need to train a consumer loan risk control scoring model to predict whether a user will default within the next 90 days. The data collection period for the training set could be the first half of 2022. At that time, the main customer base was white-collar workers in first-tier cities, with medium loan amounts, mostly using newly released mid-to-high-end mobile phone models, and timely repayments. The data collection period for the validation set was the second half of 2022. The platform began expanding to users in second-tier cities and some third-tier cities, with an increase in small and micro-enterprise owners, mostly using mid-range Android models, and more diverse loan purposes. The customer structure had changed somewhat compared to the training phase. The sample data in the test set was collected during the first half of 2023. During this period, the macroeconomic environment changed significantly, with increased unemployment and salary reductions. The platform further expanded into more third- and fourth-tier cities, resulting in a higher proportion of younger users among newly registered users, a higher percentage of customers opting for "buy now, pay later," and a greater prevalence of low-end mobile devices. It is evident that the passage of time and changes in business strategy led to changes in the data distribution of users' city tiers, occupational types, and behavioral characteristics such as loan amounts, loan purposes, and repayment methods. Furthermore, changes in the distribution of input data often accompany changes in the distribution of tag data; for example, the distribution of delinquency probability is also likely to change under the aforementioned circumstances.

[0030] In step 102, the weight parameters corresponding to each sample data in the validation set are obtained.

[0031] For each sample in the validation set, its weight parameter is a dynamic weight value that changes with the similarity between the input data in the sample and the data in the test set. The weight parameter is positively correlated with the first probability that the input data belongs to the test set and negatively correlated with the second probability that it belongs to the training set. A higher weight parameter indicates that the input data is closer to the data distribution of the test set, with a lower deviation from the test set and a higher deviation from the training set. The weight parameter is used to guide the model to focus more on samples that are closer to the future online distribution during subsequent hyperparameter optimization, avoiding overfitting of the model to the historical training set distribution during training, thereby improving the model's generalization ability and long-term stability.

[0032] This embodiment does not restrict the specific method for obtaining the weight parameters corresponding to each sample data in the validation set. It can be understood that the calculation process for obtaining the weight parameters is only related to the input data and not to the label data. For example, for each input data in the validation set, its M nearest neighbors can be found in the test set and its N nearest neighbors in the training set. Then, the average similarity between the input data and its M nearest neighbors is calculated and converted into a first weight, and the average similarity between the input data and its N nearest neighbors is calculated and converted into a second weight. Subtracting the second weight from the first weight yields the weight parameter. Another example is merging the N samples in the training set... train The input data and N in the test set test The input data is clustered into K clusters, and the number of training set samples in each cluster is calculated. and the number of samples in the test set Then, for each input data x in the validation set, its corresponding weight parameter can be defined as:

[0033]

[0034] In one alternative implementation, such as Figure 2 As shown, this step belongs to the weight allocation stage. First, a classification model can be trained based on the input data and source labels of the input data in the training set and the test set. The source labels of the input data in the training set and the validation set are used to indicate whether the input data belongs to the training set or the test set. Then, based on the classification model, the first probability that each input data in the validation set belongs to the test set and / or the second probability that it belongs to the training set is determined. Next, based on the first probability and / or the second probability corresponding to the input data, the weight parameters of the sample data corresponding to the input data are determined.

[0035] This embodiment does not limit the classification model used; it can be a generative model or a discriminative model, specifically a logistic regression model, a tree model, a discriminator, etc. For example, a discriminative model can be a fully connected neural network with two hidden layers, each with 128 neurons, using the ReLU activation function, and the output layer using sigmoid activation to output the probability of belonging to the target distribution. For example, a training set is defined... test set Constructing binary classification source labels z k When z k =1 indicates that the input data comes from the test set, z k =0 indicates that the input data comes from the training set. A binary classification discriminant model f can be constructed. disc(x) = p(z = 1|x), meaning that given an input data x, the model will predict the probability that it comes from the test set. This is a binary classification supervised learning task. It's understood that the discriminant model training only uses the input data and its source identifiers (training set / test set), without using business labels (such as whether it's overdue). For example, before each Bayesian optimization iteration, N positive samples (target distribution) are sampled from the most recent 7-day online logs, and N negative samples (source distribution) are sampled from the historical joint modeling data. The discriminant model is fine-tuned using the cross-entropy loss function with a learning rate of 0.001, and trained for 10 epochs. Specifically, during training, for the input data x in the training and test sets... k After inputting it into the discriminative model, the model will output a predicted probability value f that indicates it belongs to the test set. disc (x k ), by predicting the probability value f disc (x k ) and source tag z k The differences between them can be calculated using a loss function, such as the cross-entropy loss function, and then the parameters of the discriminant model can be adjusted to minimize this loss function. For example, the objective function θ can be optimized using maximum likelihood estimation. * To minimize the loss function:

[0036]

[0037] After training, the trained discriminant model is obtained, and it is used for the input data x in the validation set. v After inputting it into the discriminant model, the first probability p of its test set belonging to the test set is obtained. test =f disc (x v Furthermore, we can obtain the second probability P that it belongs to the training set. train =1-p test When determining the weight parameters of the sample data corresponding to the input data based on the first probability and / or the second probability, the weight parameters w can be obtained by dividing the first probability by the second probability. v =p test / p train Alternatively, considering the difference in the number of samples in the training and test sets, a ratio constant N can be added. train / N test To compensate for the imbalance in the number of samples, the weight parameters are used.

[0038] Considering that the value obtained by dividing the first probability by the second probability may be close to infinity or close to 0, to prevent extreme values ​​from affecting training stability, the weight parameters can be truncated and smoothed. Specifically, the base weight w can be obtained by dividing the first probability corresponding to the input data by the second probability. v =p test / p train Then, in response to the basic weight being less than the first threshold, the weight parameter of the sample data corresponding to the input data is determined to be the first threshold; or, in response to the basic weight being greater than the second threshold, the weight parameter of the sample data corresponding to the input data is determined to be the second threshold; or, in response to the basic weight being between the first and second thresholds, the weight parameter of the sample data corresponding to the input data is determined to be the basic weight.

[0039] This embodiment does not restrict the setting of the first and second thresholds, which can be set according to actual needs. For example, the first threshold can be 0.1 and the second threshold can be 100. For instance, a truncation threshold τ can be introduced for smoothing. The first threshold can be the truncation threshold τ, and the second threshold can be the reciprocal of the truncation threshold τ. The weight parameters obtained after smoothing are shown in the following formula:

[0040]

[0041] In other examples, the underlying weights w for all samples can also be applied. v Calculate its 5th and 95th percentiles, and use them as the first threshold w, respectively. min Second threshold w max Weights exceeding this range are truncated.

[0042] In step 103, the model parameters of the first model are adjusted based on the training set. During the adjustment process, the weighted performance index of the first model is evaluated based on the validation set and weight parameters. With the goal of optimizing the weighted performance index, a search is performed in the hyperparameter space to obtain several target hyperparameters.

[0043] The performance metrics are used to measure the ability of the first model to make accurate and reliable predictions. The better the performance metrics of the first model, the more accurately it can achieve the business objectives. The weighted performance metrics, based on the performance metrics, focus more on the prediction accuracy of input data that is "more like the test set" when measuring the ability of the first model. The magnitude of the weight parameters corresponding to the sample data is positively correlated with the degree of influence of the sample data on the weighted performance metrics. This embodiment does not limit the weighted performance metrics used to measure the first model. For example, it can use accuracy, precision, recall, or AUC, etc. The weighted performance metrics can specifically be one or more metrics.

[0044] The hyperparameter space includes a space of values ​​for several hyperparameters. The hyperparameter space can be a mathematical space consisting of all possible hyperparameter values; for example, it can be a range of values ​​or a set of parameters. This embodiment does not limit the specific values ​​covered by the hyperparameter space.

[0045] Understandably, the process of searching in the hyperparameter space corresponds to the hyperparameter tuning phase of the model training process. The purpose is to evaluate the model during training using a validation set to determine the final combination of hyperparameters used by the trained model from the hyperparameter space. For example... Figure 2 As shown, this step corresponds to Figure 2 The middle search phase.

[0046] In this embodiment, the first model and the subsequent second model can be the same model; that is, the model used in the hyperparameter tuning stage is completely consistent with the final training and deployment business model. To reduce computational resource consumption and accelerate hyperparameter search during the hyperparameter tuning stage, the number of parameters in the first model can be less than the number of parameters in the second model, meaning the first model is a lighter model than the second model. For example, if both the first and second models are tree models, the first model can use a lightweight classifier (Hist Gradient BoostingClassifier), while the second model can use a more complex tree model such as XGBoost or LightGBM. By using a lightweight model first to quickly complete the hyperparameter search, computation time can be significantly reduced. Furthermore, if both the first and second models are neural network models, the first model can be a simplified network structure with fewer layers than the second model to reduce parameter size and improve hyperparameter tuning efficiency.

[0047] In this step, the model parameters of the first model are adjusted round by round using sample data from the training set. During the adjustment process, after several rounds, the current model is evaluated using sample data from the validation set. The evaluation assesses the weighted performance index of the first model based on the validation set and weight parameters, and searches the hyperparameter space based on a hyperparameter optimization strategy to determine candidate hyperparameter combinations. Then, the first model is reconfigured using this hyperparameter combination, and the model parameters are adjusted round by round using sample data from the training set… After multiple rounds of trying different hyperparameter combinations, the hyperparameter combination that maximizes the weighted performance index (or satisfies the convergence condition) is selected as the target hyperparameter.

[0048] This embodiment does not restrict the method of adjusting the model parameters of the first model based on the training set.

[0049] In one example, training might involve multiple rounds of iterative training on sample data in the training set. In each round, the following process is executed sequentially: Inputting the training set's input data into the first model for inference, yielding a second inference result. This second inference result can be in the form of a class probability distribution, regression values, etc., depending on the task type. Based on several hyperparameters found in each round and the difference between the second inference result and the labeled data, the model parameters of the first model are adjusted. For example, based on the difference between the second inference result and the labeled data, a loss function is calculated. The choice of loss function depends on the nature of the task; for example, cross-entropy loss can be used for classification tasks, while mean squared error or mean absolute error loss can be used for regression tasks. In the current training round, a set of hyperparameters found in this hyperparameter search (such as the learning rate η, regularization coefficient λ, etc.) is used to configure the optimization algorithm and training strategy to determine how the model parameters are updated. Then, the configured optimization algorithm (such as SGD, Adam, etc.) is used in conjunction with the current hyperparameter configuration (especially the learning rate) to update the model parameters θ. Specifically, the gradient of the loss function with respect to the model parameters is calculated using the backpropagation algorithm. The model parameters are then adjusted based on the gradient and optimization algorithm rules to gradually optimize the model's performance.

[0050] In other examples, adversarial training mechanisms can be introduced into the supervised training process described above. Specifically, in each training iteration, based on the current model and label information, a small perturbation is applied to the input sample to generate adversarial examples. These adversarial examples are visually or semantically similar to the original samples, but are more easily misclassified by the model. Then, the original sample and its corresponding adversarial example are input into the first model, and the loss between their prediction results and the label data is calculated. The loss of the original sample and the loss of the adversarial example can be weighted and summed to obtain the total loss. The model parameters are then updated based on this total loss and a set of hyperparameters found in this hyperparameter search. By introducing adversarial training, the robustness of the first model to input perturbations can be improved.

[0051] This embodiment does not limit the specific method for evaluating the weighted performance index of the first model based on the validation set and weight parameters. It is understood that when different weighted performance metrics are used, such as weighted accuracy, weighted F1 score, or weighted AUC, the method for evaluating the weighted performance index based on the validation set and weight parameters will also differ. In one embodiment, the input data of the validation set can be input into the first model to obtain the first inference result of the first model. In the process of calculating the performance index of the first model based on the difference between the first inference result and the label data, the sample contribution of each sample data in the performance index is weighted based on the weight parameters corresponding to each sample data in the validation set, thus obtaining the weighted performance index of the first model.

[0052] For example, when the model is used for a binary classification task, the first inference result can be the probability value that the model considers the input data to be a positive sample, and the performance metric can be the AUC metric. Weighted AUC can be constructed based on the weighted TruePositive Rate (TPR) and False Positive Rate (FPR), or weight parameters can be introduced into the ranking loss (such as the Mann-Whitney U statistic). Weighted AUC is obtained by reweighting the validation set samples according to truncation weights and then using the weighted ROC curve area under the curve calculation method (refer to Hand & Till, 2001). Specifically, when calculating the ranking correctness of positive and negative sample pairs, for each pair of positive and negative samples (i, j) (where y... i =1, y j =0), its sample contribution can be calculated by the product of the corresponding weight parameters w. i ·w j After weighting, the final weighted AUC can be expressed as:

[0053]

[0054] in, Let s represent the positive sample set and the negative sample set in the validation set, respectively. i s represents the first output result of the first model on the input data of sample i. j This is the first output result of the first model on the input data of sample j. In s i >s j The value is 1 if the test set is positive and 0 otherwise. In weighted AUC, if a positive sample has a high weight, it means that it is closer to the test set, and its ranking position is more valuable, thus giving more sample contribution.

[0055] In yet another example, in a multi-class scenario, the performance metric could be the F1 score, with the weighted F1 score calculated separately for each class and then averaged. Assume the set of classes is... For each category First, the weight parameter w is based on the sample data i. i Calculate its precision and recall, and then obtain the F1 score for that category. c Then, a weighted average is calculated using the total weight of each category in the validation set (i.e., the sum of the weight parameters of the sample data belonging to that category) as the weight:

[0056]

[0057] This embodiment does not restrict the specific optimization method for searching in the hyperparameter space to obtain several target hyperparameters with the goal of optimizing the weighted performance index. For example, Bayesian optimization, grid search, or genetic algorithms can be used in the hyperparameter search process. By optimizing the weighted performance index, more attention can be paid to sample data that is more like the test set during hyperparameter tuning. The goal can be to improve the weighted performance index by iteratively searching for several hyperparameters in the hyperparameter space, and using the hyperparameters obtained from each search and the training set to continue adjusting the model parameters of the first model; the specific adjustment process can be found above. Then, in response to meeting the search stopping condition, the several hyperparameters corresponding to the largest weighted performance index during the search process are determined as several target hyperparameters. Specifically, this may include the following steps:

[0058] Step 1: Before starting the search, you can first initialize the hyperparameters of the first model, usually by random initialization or default initial values.

[0059] Step 2: Configure the first model using hyperparameters and iteratively optimize the model parameters of the first model on the training set round by round.

[0060] Step 3: After completing a certain number of training epochs (e.g., every 5 epochs), pause training and use the current model to perform forward inference on the validation set to calculate the weighted performance metrics.

[0061] In the above process, hyperparameter optimization strategies (such as grid search, random search, Bayesian optimization, etc.) are employed to explore different combinations of hyperparameters within a predefined hyperparameter space. For each set of candidate hyperparameters, the first model is configured using this set, and the training and validation processes in steps 2 and 3 are executed, recording the maximum (or final) weighted performance index achieved on the validation set. Then, the search process dynamically adjusts the candidate hyperparameter schemes with the objective function of maximizing the weighted performance index. After multiple rounds of attempts, in response to meeting the search stopping condition, several hyperparameters corresponding to the maximum weighted performance index are selected as several target hyperparameters. If the search stopping condition is not met, then... Figure 2 As shown, continue iteratively executing the above process.

[0062] The search stopping condition can include any of the following: the weighted performance metric does not improve during consecutive iterations; or the number of iterations reaches the tolerance limit. The tolerance limit refers to a pre-defined maximum number of allowed iterations (e.g., a maximum of 50 attempts). For example, if the model's weighted performance metric (such as weighted AUC, weighted accuracy, etc.) does not improve after several consecutive iterations (e.g., 3 or 5), this usually indicates that the search is close to convergence, and continuing iterations is unlikely to find a better solution. Stopping the iteration in this case conserves computational resources. Alternatively, when the number of iterations reaches the tolerance limit, even if the model performance has not fully converged, the algorithm will stop to avoid unlimited consumption of computational resources. The search process terminates as long as either of the above conditions is met, and the hyperparameter combination corresponding to the largest weighted performance metric found so far is used as the final target hyperparameter. This design ensures both algorithm efficiency and prevents unnecessary computational overhead.

[0063] In step 104, a second model is trained based on the target hyperparameters and the training set.

[0064] The second model is configured using target hyperparameters, and then trained using sample data from the training set. This embodiment does not limit the specific method of training the second model using the training set. For example, the input data can be input into the second model for inference to obtain prediction results. Then, the loss function corresponding to the sample data is calculated based on the difference between the prediction results and the label data. The model parameters of the second model are updated in each iteration using the backpropagation algorithm with the goal of minimizing the loss function, until the loss value of the loss function meets the preset loss value requirement, or the number of iterations is set to a preset number.

[0065] Because more weights are assigned to samples whose data distribution is closer to the test set when calculating weighted performance metrics during the hyperparameter search phase, the model trained with the target hyperparameters obtained through the search focuses more on samples similar to the test set. This mitigates performance degradation caused by time drift, improves model generalization ability, and extends the online service life. Furthermore, since the weight parameters used to calibrate the sample data distribution are applied to the calculation of performance metrics during the hyperparameter search phase, model tuning and distribution calibration are integrated into a single end-to-end optimization process, improving tuning effectiveness and computational efficiency. Finally, the fact that the above process does not use labeled data from the test set for training does not violate the principle of evaluation independence.

[0066] In real-world business operations, especially large-scale and wide-ranging ones, multiple scenarios are common. Different regions, customer groups, and clients can all be considered different scenarios. Sample data from different scenarios typically differ. When performing joint modeling across multiple scenarios, directly mixing sample data from different scenarios for training will prevent the model from distinguishing the characteristics of different scenarios, thus affecting the model's predictive performance.

[0067] Therefore, in order to solve the above problems, in another embodiment, while performing hyperparameter search, scene weight search can also be performed, so that different weights can be assigned to sample data in different scenes during model training, so as to balance the importance of each scene, help the model to correctly learn the features of different scenes, avoid the negative impact of differences between scenes on the model, and thus improve its accuracy, robustness and generalization ability in actual prediction.

[0068] In this embodiment, the training set, test set, and validation set each include sample data with different data distributions under multiple scenarios. These different scenarios can refer to different data sources (e.g., proprietary platforms and third-party platforms), different customer groups (new and existing customers), different regions, and different operational strategies, etc.

[0069] Based on the above embodiments, in step 103, the goal may be to optimize the weighted performance index by searching in the hyperparameter space and scene weight space to obtain several target hyperparameters and target scene weights corresponding to different scenes. The scene weight space includes the value space for each scene weight.

[0070] This embodiment does not restrict the specific values ​​covered by the scene weight space. For example, a hyperparameter space Λ = {λ1, ..., λ} can be constructed. d} and the scene weight space Ω={ω1,...,ω m For example, the scene weights {ω1, ..., ω} in the scene weight space. m} can be obtained through a K-dimensional learnable vector β via a Softmax transformation, i.e. Ensure ∑ω k =1 and ω k>0. The same optimization algorithm can be used to search for scene weights and hyperparameters, such as Bayesian optimization, grid search, and genetic algorithms. The calculation process of the weighted performance index parameters is described in the above embodiments and will not be repeated here. Bayesian optimization is preferred, as it reduces the number of evaluations by using a surrogate model. Bayesian optimization approximates the objective function (i.e., the weighted performance index) by establishing a surrogate model (usually a Gaussian process), thus avoiding extensive calculations directly on the objective function. In each iteration, the algorithm intelligently selects the combination of weight parameters most likely to improve prediction performance for evaluation based on the surrogate model and the acquisition function. For example, a surrogate model GP(μ(·), k(·,·)) can be defined, and the search in the hyperparameter space and scene weight space can be guided by the acquisition function EI(·).

[0071]

[0072] Among them, f AUC (·) is the evaluation function for the weighted AUC, which can be calculated using the above formula (4). AUC (·), where the hyperparameters obtained in each search are represented as λ. * The scene weight is represented by ω. * .

[0073] When adjusting the model parameters of the first model based on the training set, the target weights obtained through the search are used to adjust the weights of each scene in the training objective, thereby adjusting the contribution of sample data under different scenes to model training. In step 2 above, for each sample data in the training set, the following processing can be performed iteratively: input the input data into the first model for inference to obtain the prediction result, then calculate the loss term corresponding to the sample data based on the difference between the prediction result and the label data, then weight the loss term corresponding to the sample data under each scene using the scene weights obtained in this search to obtain the loss function, and adjust the model parameters of the first model based on the loss function. Then, in step 3, forward inference is performed on the validation set using the current model to calculate the weighted performance index. After multiple iterations, in response to meeting the search stopping condition, several hyperparameters and several scene weights corresponding to the largest weighted performance index are used as several target hyperparameters and several target scene weights. The specific search process can be found in the relevant descriptions in the above embodiments, and will not be repeated here.

[0074] In step 104, when training the second model based on the target hyperparameters and the training set, during the training process, the first loss term corresponding to the sample data in each scenario can be weighted based on the target scenario weights corresponding to each scenario to obtain the first loss function, and the model parameters of the second model can be adjusted based on the first loss function.

[0075] like Figure 2 As shown, the model structure and training strategy of the second model are configured using target hyperparameters. During training, the following processing can be performed iteratively for each sample data in the training set: input the input data into the second model for inference to obtain the prediction result, then calculate the first loss term corresponding to the sample data based on the difference between the prediction result and the label data, then weight the first loss term corresponding to the sample data in each scenario based on the target scenario weights to obtain the first loss function, and adjust the model parameters of the second model based on the first loss function.

[0076] This embodiment does not limit the calculation method of the first loss term; for example, cross-entropy loss, mean squared error loss, etc., can be used. For instance, ω can be used. A For each sample data in scenario A, the first loss term is weighted and ω is used. B For each sample data in scenario B, the first loss term is weighted and ω is used. C For each sample data in scenario C, the first loss term is weighted.

[0077] Then, the weighted summation of the first loss terms yields the first loss function. The second model's parameters are updated in each iteration using backpropagation, with the goal of minimizing this first loss function, until the first loss function meets a preset loss value requirement, or the number of iterations reaches a preset number. In this way, the trained second model can be applied to various business scenarios. Moreover, because the model training comprehensively learns the differences in sample data from different scenarios, it exhibits stable model performance across various scenarios.

[0078] The above process unifies scene weight search and model hyperparameter optimization under the same framework, enabling collaborative exploration of the parameter space, improving search efficiency, saving computational resources, and making the hyperparameters and scene weights obtained from the search more suitable for the model.

[0079] Figure 3 This is a schematic diagram of a model training device based on distributed calibration, as described in an embodiment of this specification. This device can be applied to any device, platform, or cluster of devices with computing and processing capabilities. The device includes:

[0080] The acquisition module 31 is used to acquire the training set, test set and validation set. The training set, test set and validation set include sample data with different data distributions. The sample data includes input data for inputting into the model and label data as model training labels. The input data is user data and the label data is user labels. The sample data in the test set is collected later than the training set and validation set.

[0081] The weight module 32 is used to obtain the weight parameters corresponding to each sample data in the validation set. The weight parameters are positively correlated with the first probability that the input data in the sample data belongs to the test set and negatively correlated with the second probability that it belongs to the training set.

[0082] Search module 33 is used to adjust the model parameters of the first model based on the training set. During the adjustment process, the weighted performance index of the first model is evaluated based on the validation set and weight parameters. The magnitude of the weight parameters corresponding to the sample data is positively correlated with the influence of the sample data on the weighted performance index. With the goal of optimizing the weighted performance index, a search is performed in the hyperparameter space to obtain several target hyperparameters. The hyperparameter space includes the value space of several hyperparameters.

[0083] Training module 34 is used to train a second model based on the target hyperparameters and the training set.

[0084] In one optional implementation, the training set, test set, and validation set each include sample data with different data distributions in multiple scenarios. The search module 33, when searching in the hyperparameter space with the goal of optimizing the weighted performance index to obtain the target hyperparameters, specifically searches in the hyperparameter space and scene weight space with the goal of optimizing the weighted performance index to obtain several target hyperparameters and target scene weights corresponding to different scenarios. The scene weight space includes the value space of each scene weight. The training module 34 is specifically used to train the second model based on the target hyperparameters and the training set. During the training process, the first loss term corresponding to the sample data in each scenario is weighted based on the target scene weights corresponding to each scenario to obtain the first loss function, and the model parameters of the second model are adjusted based on the first loss function.

[0085] In one optional implementation, the search module 33, when searching in the hyperparameter space with the goal of optimizing the weighted performance index to obtain target hyperparameters, is specifically used to iteratively search for several hyperparameters in the hyperparameter space with the goal of improving the weighted performance index, and use the hyperparameters obtained from each search and the training set to continue adjusting the model parameters of the first model; in response to satisfying the search stopping condition, several hyperparameters corresponding to the largest weighted performance index during the search process are determined as several target hyperparameters.

[0086] In one optional implementation, the weight module 32 is specifically used to train a classification model based on input data and source labels of the input data in the training set and test set, wherein the source labels are used to indicate whether the input data belongs to the training set or the test set, and based on the classification model, to determine a first probability that each input data in the validation set belongs to the test set and / or a second probability that belongs to the training set; and to determine the weight parameters of the sample data corresponding to the input data based on the first probability and / or the second probability corresponding to the input data.

[0087] In one optional implementation, the weighting module 32, when determining the weight parameters of the sample data corresponding to the input data based on the first probability and / or the second probability corresponding to the input data, specifically divides the first probability corresponding to the input data by the second probability to obtain a basic weight; in response to the basic weight being less than a first threshold, determines the weight parameters of the sample data corresponding to the input data as a first threshold; in response to the basic weight being greater than a second threshold, determines the weight parameters of the sample data corresponding to the input data as a second threshold; and in response to the basic weight being between the first threshold and the second threshold, determines the weight parameters of the sample data corresponding to the input data as a basic weight.

[0088] In one optional implementation, the search module 33, when evaluating the weighted performance index of the first model based on the validation set and weight parameters, specifically inputs the input data of the validation set into the first model to obtain the first inference result of the first model; in the process of calculating the performance index of the first model through the difference between the first inference result and the label data, the weighted contribution of each sample data in the performance index is weighted based on the weight parameters corresponding to each sample data in the validation set to obtain the weighted performance index of the first model.

[0089] In one optional implementation, the search module 33, when adjusting the model parameters of the first model based on the training set, performs the following processing on the sample data in the training set: inputting the input data in the training set into the first model for inference to obtain a second inference result; adjusting the model parameters of the first model based on several hyperparameters found each time, and the difference between the second inference result and the label data.

[0090] In one alternative implementation, the first model and the second model are the same model.

[0091] In one alternative implementation, the number of model parameters in the first model is less than the number of model parameters in the second model.

[0092] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0093] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0094] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a server system. Of course, this application does not exclude the possibility that, with the future development of computer technology, the computer implementing the functions of the above embodiments can be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0095] While one or more embodiments of this specification provide the operational steps of the methods described in the embodiments or flowcharts, more or fewer operational steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or end product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in the process, method, product, or apparatus that includes the elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any particular order.

[0096] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, when implementing one or more of these specifications, the functions of each module can be implemented in one or more software and / or hardware components, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0097] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0098] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0099] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0100] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0101] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0102] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage, graphene storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0103] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0104] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0105] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0106] The above description is merely an embodiment of one or more embodiments of this specification and is not intended to limit the scope of these embodiments. Various modifications and variations can be made to these embodiments by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims.

Claims

1. A model training method based on distribution calibration, the method comprising: Obtain a training set, a test set, and a validation set. The training set, test set, and validation set each include sample data with different data distributions. The sample data includes input data for inputting into the model and label data as model training labels. The input data is user data, and the label data is user labels. The sample data in the test set was collected later than the training set and validation set. Obtain the weight parameters corresponding to each sample data in the validation set. The weight parameters are positively correlated with the first probability that the input data in the sample data belongs to the test set and negatively correlated with the second probability that it belongs to the training set. The model parameters of the first model are adjusted based on the training set. During the adjustment process, the weighted performance index of the first model is evaluated based on the validation set and the weight parameters. With the goal of optimizing the weighted performance index, a search is performed in the hyperparameter space to obtain several target hyperparameters. The magnitude of the weight parameter corresponding to the sample data is positively correlated with the degree of influence of the sample data on the weighted performance index. The hyperparameter space includes the value space of several hyperparameters. The second model is trained based on the target hyperparameters and the training set.

2. The method according to claim 1, wherein, The training set, test set, and validation set each include sample data from different data distributions across multiple scenarios. The goal is to optimize the weighted performance metric by searching the hyperparameter space to obtain the target hyperparameters, including: With the goal of optimizing the weighted performance index, a search is performed in the hyperparameter space and the scene weight space to obtain several target hyperparameters and target scene weights corresponding to different scenarios. The scene weight space includes the value space of each scene weight. The training of the second model based on the target hyperparameters and the training set includes: The second model is trained based on the target hyperparameters and the training set. During the training process, the first loss term corresponding to the sample data in each scenario is weighted according to the target scenario weights for each scenario to obtain the first loss function, and the model parameters of the second model are adjusted based on the first loss function.

3. The method according to claim 1, wherein, The goal is to optimize the weighted performance index by searching the hyperparameter space to obtain several target hyperparameters, including: With the goal of improving the weighted performance index, several hyperparameters are iteratively searched in the hyperparameter space, and the model parameters of the first model are further adjusted using the hyperparameters obtained from each search and the training set. In response to the search stopping condition being met, several hyperparameters corresponding to the largest weighted performance metric during the search process are determined as several target hyperparameters.

4. The method according to claim 1, wherein, The step of obtaining the weight parameters corresponding to each sample data in the validation set includes: Based on the input data in the training set and the test set and the source labels of the input data, a classification model is trained. The source labels are used to indicate whether the input data belongs to the training set or the test set. Based on the classification model, a first probability that each input data in the validation set belongs to the test set and / or a second probability that belongs to the training set is determined. Based on the first probability and / or second probability corresponding to the input data, determine the weight parameters of the sample data corresponding to the input data.

5. The method according to claim 4, wherein, The step of determining the weight parameters of the sample data corresponding to the input data based on the first probability and / or the second probability corresponding to the input data includes: Divide the first probability corresponding to the input data by the second probability to obtain the basic weight; In response to the fact that the basic weight is less than the first threshold, the weight parameter of the sample data corresponding to the input data is determined to be the first threshold. In response to the fact that the basic weight is greater than the second threshold, the weight parameter of the sample data corresponding to the input data is determined to be the second threshold. In response to the basic weight being between the first threshold and the second threshold, the weight parameter of the sample data corresponding to the input data is determined as the basic weight.

6. The method according to claim 1, wherein, The evaluation of the weighted performance metric of the first model based on the validation set and the weight parameters includes: The input data of the validation set is input into the first model to obtain the first inference result of the first model; In the process of calculating the performance index of the first model based on the difference between the first inference result and the label data, the sample contribution of each sample data in the performance index is weighted based on the weight parameters corresponding to each sample data in the validation set, so as to obtain the weighted performance index of the first model.

7. The method according to claim 1, wherein, In adjusting the model parameters of the first model based on the training set, the following processing is performed on the sample data in the training set, including: The input data in the training set is input into the first model for inference to obtain the second inference result; Based on several hyperparameters found in each search, and the differences between the second inference result and the label data, the model parameters of the first model are adjusted.

8. The method according to claim 1, wherein, The first model and the second model are the same model.

9. The method according to claim 1, wherein, The first model has fewer model parameters than the second model.

10. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-9.

Images