A cross-task dataset fusion method based on meta-Bayesian optimization

Abstract

A cross-task data set fusion method based on meta-Bayesian optimization belongs to the field of electrical data processing technology. The method includes: S1: input a small amount of observation data, and sample a data subset from the observation data; S2: calculate the loss function according to the data subset; S3: update the parameters of the Bayesian learner according to the loss function; S4: determine whether the loss function is minimum, if the loss function does not reach the minimum, update the parameters of the Bayesian learner, and return to step S2, if the loss function reaches the minimum, then end the training and output the optimal Bayesian learner. The present invention applies the meta-learning idea to the combination optimization of data sets, and completes the learning of meta-knowledge of hyperparameter optimization through the proxy model in the meta-learning Bayesian optimization, so as to realize the hyperparameter optimization under a small number of optimization iterations.

Description

Technical Field

[0001] The invention relates to a cross-task data set fusion method based on meta-Bayesian optimization, and belongs to the technical field of data electrical data processing. Background Art

[0002] With the rapid development of deep learning technology, more and more methods can be used to improve a basic model architecture, such as regularization, data enhancement, loss function design, learning methods, etc. For a specific task, even if there is prior knowledge of the network infrastructure, there are still quite a lot of hyperparameters (HPs) that can be selected and configured. For tasks in different fields, the selection of hyperparameters has developed into professional knowledge in the field, which widely determines the performance of the model in terms of accuracy, complexity, speed, resource consumption, etc.

[0003] The Chinese invention patent application with publication number CN116561614A discloses a small sample data processing system based on meta-learning, which includes a regression modeling module, a data enhancement module, a task classification module, a data processing module, and a target prediction module; the regression modeling module is used to construct a non-parametric model for regression analysis of data using Gaussian regression, and the Gaussian process regression model is applied to the data by utilizing the characteristics of small sample scattered data; the data enhancement module is used to combine the meta-learning MAML algorithm and the generative adversarial network to obtain a fusion model; the task classification module adopts the classifier-meta-learner framework, and automatically learns the model parameters by measuring the performance of the classifier on the classification task through the meta-learner; the data processing module adopts an optimization strategy for parallel processing for data processing; the target prediction module is used to migrate the trained shallow network to the target domain data and use the classifier for classification. However, it does not publicly perform parameter super-optimization. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the present invention provides a cross-task dataset fusion method based on meta-Bayesian optimization, which applies the idea of ​​meta-learning to the combinatorial optimization of datasets, and completes the learning of meta-knowledge of hyperparameter optimization through the proxy model in meta-learning Bayesian optimization, so as to achieve hyperparameter optimization under a small number of optimization iterations.

[0005] To achieve the above-mentioned object of the invention, the present invention provides a cross-task data set fusion method based on meta-Bayesian optimization, which comprises:

[0006] S1: Input a small amount of observation data , from the observation data Sampling subset ;

[0007] S2: Calculate the loss function according to the following formula: ,

[0008] In the formula, is the i-th data set combination, Combination of the i-th data set Evaluation results of cross-task meta-learning; and Bayesian learner The current parameters of

[0009] S3: The current parameters of the Bayesian learner are given by the following formula Updated to :

[0010] ,

[0011] In the formula, is the learning coefficient; For The gradient of

[0012] S4: The current parameters of the Bayesian learner are given by the following formula Updated to :

[0013] ,

[0014] In the formula, is the learning coefficient; For The gradient of

[0015] S5: Determine whether the loss function L is the minimum. If the loss function L is not the minimum, Assign to ,Will Assign to , and return to step S2. If the loss function L reaches the minimum, the training ends and the Bayesian learner is output. .

[0016] Compared with the prior art, the cross-task dataset fusion method based on meta-Bayesian optimization provided by the present invention has the following beneficial effects:

[0017] The idea of ​​meta-learning is applied to the combinatorial optimization of data sets. Through the proxy model in meta-learning Bayesian optimization, the meta-knowledge of hyperparameter optimization is learned to achieve hyperparameter optimization with a small number of optimization iterations. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is the first flow chart of the cross-task data set fusion method based on meta-Bayesian optimization provided by the present invention.

[0019] Figure 2 This is the second flow chart of the cross-task data set fusion method based on meta-Bayesian optimization provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0021] In the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance.

[0022] Figure 1 is a first flow chart of the cross-task data set fusion method based on meta-Bayesian optimization provided by the present invention, such as Figure 1 As shown, the cross-task data set fusion method based on meta-Bayesian optimization provided by the present invention includes:

[0023] S1: Input a small amount of observation data , from the observation data Sampling subset ;

[0024] S2: Calculate the loss function according to the following formula: ,

[0025] In the formula, is the i-th data set combination, Combination of the i-th data set Evaluation results of cross-task meta-learning; and Bayesian learner The current parameters of

[0026] S3: The current parameters of the Bayesian learner are given by the following formula Updated to :

[0027] ,

[0028] In the formula, is the learning coefficient; For The gradient of

[0029] S4: The current parameters of the Bayesian learner are given by the following formula Updated to :

[0030] ,

[0031] In the formula, is the learning coefficient; For The gradient of

[0032] S5: Determine whether the loss function L is the minimum. If the loss function L is not the minimum, Assign to ,Will Assign to , and return to step S2. If the loss function L reaches the minimum, the training ends and the optimal learner is output. .

[0033] Figure 2 is a second flow chart of the cross-task data set fusion method based on meta-Bayesian optimization provided by the present invention, such as Figure 2 As shown, the cross-task data set fusion method based on meta-Bayesian optimization provided by the present invention also includes:

[0034] S6: Make ;

[0035] S7: From observational data Sampling subset ;

[0036] S8: Calculate the loss function according to the following formula: ;

[0037] S9: The parameters of the Bayesian learner are set by the following formula Updated to :

[0038] ,

[0039] In the formula, is the learning coefficient; For The gradient of

[0040] S10: The parameters of the Bayesian learner are set by the following formula Updated to :

[0041] ,

[0042] In the formula, is the learning coefficient; For The gradient of

[0043] S11: Determine the loss function Is it the minimum? If the loss function If the minimum value is not reached, Assign to ,Will Assign to , assign M+1 to M, and return to step S7. If the loss function L m When it reaches the minimum, the training ends and the hyperparameter Bayesian learner is output .

[0044] In the present invention, , where is the noise, and f is the proxy model.

[0045] Evaluation results The metric has the same scale on different tasks, is comparable in value, and can be ranked. The value includes the standardized accuracy.

[0046] In the present invention, the cross-task data set fusion method based on meta-Bayesian optimization also includes:

[0047] S12: Experts combine the tth group of data Fusion is performed to obtain the fusion result , the fusion result Conduct an evaluation and obtain the evaluation results , generate sequence evaluation: , N is a positive integer greater than or equal to 2;

[0048] S13: According to the following formula Rating value:

[0049] ,

[0050] In the formula, is the parameter of the evaluator D;

[0051] S14: According to the evaluation value Generate data fusion strategy ,in

[0052] ,

[0053] In the formula, is a hyperparameter; Data fusion strategy The optimization variables of the neural network, is the updated parameter of the data fusion strategy; are the current parameters of the data fusion strategy;

[0054] S15: Evaluate the result of data fusion on the data combination according to the data fusion strategy to obtain a sequence evaluation: ; For the mth data set c m The fusion result obtained by data fusion; For the fusion result The evaluation results;

[0055] S16: Update the evaluator D according to the following formula :

[0056] ,

[0057] In the formula, is the learning coefficient, For gradient.

[0058] In the present invention, the data set combination Have similarities.

[0059] The present invention also provides a system, which includes a storage medium and one or more processors, wherein the storage medium is used to store a computer program, and the computer program can be called by one or more processors to implement the above method.

[0060] The present invention also provides a computer program product, which is a program code compiled by a computer language according to the above method, and the program code can be called and executed by one or more processors.

[0061] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A cross-task dataset fusion method based on meta-Bayesian optimization, characterized in that: include: S1: Input a small amount of observation data , from the observation data Sampling subset ; S2: Calculate the loss function according to the following formula: , In the formula, is the i-th data set combination, Combination of the i-th data set Evaluation results of cross-task meta-learning; and Bayesian learner The current parameters of S3: The current parameters of the Bayesian learner are given by the following formula Updated to : ， In the formula, is the learning coefficient; For Gradient S4: The current parameters of the Bayesian learner are given by the following formula Updated to : ， In the formula, is the learning coefficient; For The gradient of S5: Determine whether the loss function L is the minimum. If the loss function L is not the minimum, Assign to ,Will Assign to , and return to step S2. If the loss function L reaches the minimum, the training ends and the hyperparameter Bayesian learner is output. .

2. The cross-task dataset fusion method based on meta-Bayesian optimization according to claim 1 is characterized in that: S6: Make ; S7: From observational data Sampling subset ; S8: Calculate the loss function according to the following formula: ; S9: The parameters of the Bayesian learner are set by the following formula Updated to : ， In the formula, is the learning coefficient; For The gradient of S10: The parameters of the Bayesian learner are adjusted by the following formula Updated to : ， In the formula, is the learning coefficient; For The gradient of S11: Determine the loss function Is it the minimum? If the loss function If the minimum value is not reached, Assign to ,Will Assign to , assign M+1 to M, and return to step S7. If the loss function reaches the minimum, the training ends and the hyperparameter Bayesian learner is output. .

3. The cross-task dataset fusion method based on meta-Bayesian optimization according to claim 2 is characterized in that: , where is the noise, and f is the proxy model.

4. The cross-task data set fusion method based on meta-Bayesian optimization according to claim 3 is characterized in that: Also includes: S12: Experts combine the tth group of data Fusion is performed to obtain the fusion result , the fusion result Conduct an evaluation and obtain the evaluation results , generate sequence evaluation: , N is a positive integer greater than or equal to 2; S13: According to the following formula Rating value: , In the formula, is the parameter of the evaluator D; S14: According to the evaluation value Generate data fusion strategy ,in, ， In the formula, is a hyperparameter; Data fusion strategy The optimization variables of the neural network, is the updated parameter of the data fusion strategy; is the current parameter of the data fusion strategy S15: Evaluate the result of data fusion on the data combination according to the data fusion strategy to obtain a sequence evaluation: ; For the mth data set c m The fusion result obtained by data fusion; For the fusion result The evaluation results; S16: Update the evaluator D according to the following formula : ， In the formula, is the learning coefficient, For gradient.

5. The cross-task dataset fusion method based on meta-Bayesian optimization according to claim 4 is characterized in that ,Evaluation results The metric has the same scale across different tasks.

6. The cross-task dataset fusion method based on meta-Bayesian optimization according to claim 5 is characterized in that ,Evaluation results They are numerically comparable and sortable.

7. The cross-task dataset fusion method based on meta-Bayesian optimization according to claim 6 is characterized in that , the values ​​include standardized accuracy.

Images