Multi-model fusion method and device based on model application portrait

Abstract

The present invention provides a multi-model fusion method and device based on model application portraits, the method comprising: extracting feature data of multiple basic models to generate application portraits corresponding to each of the multiple basic models; screening the multiple basic models according to the application data and multiple application portraits; and fusing the multiple basic models according to the distribution differences of the prediction results of the application data from the screened multiple basic models. The present invention performs attention-weighted integration and multi-level random sampling model fusion customization on the prediction results based on the basic model application portraits and in combination with the distribution characteristics of the application data. The present invention can perform personalized model fusion based on the distribution characteristics of the data and the prediction capabilities of the basic models in specific application data to meet actual application needs.

Description

Technical Field

[0001] The present invention relates to the field of model integration technology, and in particular to a multi-model fusion method and device based on model application portrait. Background Art

[0002] In recent years, model integration has been widely used in image segmentation, classification, detection and other fields. Model integration is to fuse all the prediction results of multiple basic models on the application data to form the final result. The goal is to make the final result "complement each other's strengths" and improve the generalization ability of the final model by fusing the learning ability of each model.

[0003] At present, model fusion mainly adopts three general frameworks: bagging, boosting and stacking. Bagging is a voting algorithm. First, Bootstrap is used to generate different training data sets, and then multiple basic classifiers are obtained based on these training data sets. Finally, a relatively better prediction model is obtained by combining the classification results of the basic classifiers.

[0004] Boosting is a machine learning algorithm that can be used to reduce the bias in supervised learning. It is a framework algorithm that mainly obtains sample subsets by operating on sample sets, and then uses weak classification algorithms to train and generate a series of base classifiers on the sample subsets. It can be used to improve the recognition rate of other weak classification algorithms, that is, other weak classification algorithms are placed in the Boosting framework as base classification algorithms. Through the operation of the training sample set by the Boosting framework, different training sample subsets are obtained, and the sample subsets are used to train and generate base classifiers; each time a sample set is obtained, a base classifier is generated on the sample set using the base classification algorithm. In this way, after a given number of training rounds n, n base classifiers can be generated. Then the Boosting framework algorithm weights and fuses these n base classifiers to generate a final result classifier. Among these n base classifiers, the recognition rate of each individual classifier may not be very high, but the result after their combination has a high recognition rate, which improves the recognition rate of the weak classification algorithm. When generating a single base classifier, the same classification algorithm or different classification algorithms can be used. These algorithms are generally unstable weak classification algorithms, such as neural network (BP), decision tree (C4.5), etc.

[0005] Stacking is a common integrated learning framework. Stacking means that after learning several base learners with the initial training data, the prediction results of these learners are used as the new training set to learn a new learner. The basic layer of stacking usually includes different learning algorithms, so the stacking ensemble is often heterogeneous. Generally speaking, it is to train a multi-layer (usually two-layer) learner structure. The first layer (also called the learning layer) uses n different classifiers (or models with different parameters) to merge the prediction results into a new feature set and use it as the input of the next layer of classifiers.

[0006] The model fusion method in the prior art does not consider the prediction ability of each basic model in the new data distribution and small sample data application when fusing the model, and cannot dynamically weight and customize the prediction results according to the characteristics of the model in different application data distributions. Summary of the invention

[0007] According to the multi-model fusion method and device based on model application portrait provided by the present invention, the prediction results can be weighted and integrated according to the basic model application portrait and the distribution characteristics of the application data, and the multi-level random sampling model fusion can be customized. This method can perform personalized model fusion based on the distribution characteristics of the data and the prediction ability of the basic model in specific application data to better meet the actual application needs.

[0008] In order to achieve the above object, the present invention provides a multi-model fusion method based on model application portrait, comprising:

[0009] Extracting feature data of multiple basic models to generate application portraits corresponding to each of the multiple basic models;

[0010] Filtering the multiple basic models according to the application data and multiple application portraits;

[0011] The multiple basic models are fused according to the distribution differences of the prediction results of the application data based on the screened multiple basic models.

[0012] In one embodiment, extracting the feature data of the multiple basic models to generate application portraits corresponding to the multiple basic models includes:

[0013] Extracting performance evaluation parameters, overall prediction accuracy, new data prediction accuracy, partial sample data prediction accuracy, and authoritative evaluation parameters of the multiple basic models;

[0014] Based on the performance evaluation parameters, overall prediction accuracy, new data prediction accuracy, partial sample data prediction accuracy and authoritative evaluation parameters, application portraits corresponding to each of the multiple basic models are generated.

[0015] In one embodiment, the screening of the multiple basic models according to the application data and the multiple application portraits includes:

[0016] determining distribution differences of the application data;

[0017] Determine the performance parameters and prediction accuracy corresponding to each of the multiple basic models according to the distribution difference and the multiple application portraits;

[0018] The multiple basic models are screened according to the performance parameter, the preset performance parameter threshold, the prediction accuracy and the preset prediction accuracy threshold.

[0019] In one embodiment, the fusing of the multiple basic models according to the selected multiple basic models based on the distribution difference of the prediction results of the application data includes:

[0020] Using a multi-level random sampling method, multiple fusion sub-models are generated based on the screened multiple basic models;

[0021] Clustering the prediction results of the multiple fusion sub-models;

[0022] The multiple basic models are fused according to the prediction results after clustering and the distribution difference of the prediction results of the multiple basic models for the application data.

[0023] In one embodiment, the multi-model fusion method based on model application portraits further includes:

[0024] Based on the multiple algorithms and the training data, basic models corresponding to the multiple algorithms are generated.

[0025] In a second aspect, the present invention provides a multi-model fusion device based on model application portrait, the device comprising:

[0026] A feature data extraction module, used to extract feature data of multiple basic models to generate application portraits corresponding to each of the multiple basic models;

[0027] A basic model screening module, used to screen the multiple basic models according to the application data and multiple application portraits;

[0028] The basic model fusion module is used to fuse multiple basic models according to the distribution differences of the prediction results of the application data from the multiple basic models after screening.

[0029] In one embodiment, the feature data extraction module includes:

[0030] A parameter extraction unit, used to extract the performance evaluation parameters, overall prediction accuracy, new data prediction accuracy, partial sample data prediction accuracy and authoritative evaluation parameters of the multiple basic models;

[0031] The portrait generation unit is used to generate application portraits corresponding to multiple basic models according to the performance evaluation parameters, overall prediction accuracy, new data prediction accuracy, partial sample data prediction accuracy and authoritative evaluation parameters.

[0032] In one embodiment, the basic model screening module includes:

[0033] a distribution difference determining unit, configured to determine a distribution difference of the application data;

[0034] A parameter determination unit, configured to determine the performance parameters and prediction accuracy corresponding to each of the plurality of basic models according to the distribution difference and the plurality of application portraits;

[0035] The basic model screening unit is used to screen the multiple basic models according to the performance parameter, the preset performance parameter threshold, the prediction accuracy and the preset prediction accuracy threshold.

[0036] In one embodiment, the basic model fusion module includes:

[0037] A sub-model generating unit, used for generating a plurality of fusion sub-models according to the screened plurality of basic models by using a multi-level random sampling method;

[0038] A result clustering unit, used for clustering the prediction results of the multiple fusion sub-models;

[0039] The basic model fusion unit is used to fuse multiple basic models according to the prediction results after clustering and the distribution difference of the prediction results of the multiple basic models for the application data.

[0040] In one embodiment, the multi-model fusion device based on model application portraits further includes:

[0041] The basic model generation module is used to generate basic models corresponding to the multiple algorithms according to the multiple algorithms and training data.

[0042] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of a multi-model fusion method based on model application portraits are implemented.

[0043] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a multi-model fusion method based on model application portraits.

[0044] From the above description, it can be seen that the multi-model fusion method and device based on model application portraits provided by the embodiments of the present invention first extract the feature data of multiple basic models to generate application portraits corresponding to each of the multiple basic models; then, the multiple basic models are screened according to the application data and the multiple application portraits; finally, the multiple basic models are fused according to the distribution differences of the prediction results of the application data based on the screened multiple basic models. According to the basic model application portraits and in combination with the distribution characteristics of the application data, the present invention performs attention-weighted integration and multi-level random sampling model fusion customization on the prediction results. The present invention can perform personalized model fusion according to the distribution characteristics of the data and the prediction ability of the basic model in specific application data to meet actual application needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 Schematic diagram of the process of the multi-model fusion method based on model application portrait provided in an embodiment of the present invention Figure 1 ;

[0047] Figure 2 1 is a flow chart of step 100 in an embodiment of the present invention;

[0048] Figure 3 200 is a flowchart of step 200 in an embodiment of the present invention;

[0049] Figure 4 300 is a flowchart of an embodiment of the present invention;

[0050] Figure 5 Schematic diagram of the process of the multi-model fusion method based on model application portrait provided in an embodiment of the present invention Figure 2 ;

[0051] Figure 6 A schematic diagram of a flow chart of a multi-model fusion method based on model application portrait provided in a specific application example of the present invention;

[0052] Figure 7 A mind map of a multi-model fusion method based on model application portrait provided in a specific application example of the present invention;

[0053] Figure 8The structure of the multi-model fusion device based on the model application portrait in the embodiment of the present invention is shown as follows Figure 1 ;

[0054] Fig. 9 1 is a schematic diagram of the structure of a feature data extraction module 10 in an embodiment of the present invention;

[0055] Fig.10 Schematic diagram of the structure of the basic model screening module 20 in an embodiment of the present invention;

[0056] Fig.11 Schematic diagram of the structure of the basic model fusion module 30 in an embodiment of the present invention;

[0057] Fig.12 The structure of the multi-model fusion device based on the model application portrait in the embodiment of the present invention is shown as follows Figure 2 ;

[0058] Fig.13 It is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0061] The embodiment of the present invention provides a specific implementation of a multi-model fusion method based on model application portrait, see Figure 1 , the method specifically includes the following contents:

[0062] Step 100: Extract feature data of multiple basic models to generate application portraits corresponding to each of the multiple basic models.

[0063] It can be understood that the application portrait in step 100 refers to a means of abstracting the overall picture of information from the basic model. When step 100 is implemented, the evaluation results of each basic model are deeply analyzed, and the application portrait of the basic model is dynamically generated from multiple dimensions such as performance, overall prediction accuracy, new data prediction accuracy, and small sample data prediction accuracy.

[0064] Step 200: Filter the multiple basic models according to the application data and multiple application portraits.

[0065] There are two principles for screening: one is to delete basic models whose performance does not meet the task requirements; the other is to delete models whose prediction accuracy is lower than a predefined threshold.

[0066] Step 300: Fusion the multiple basic models based on the distribution differences of the prediction results of the application data according to the multiple basic models that have been screened.

[0067] Model fusion refers to training multiple models and then integrating them in a certain way. If the models are homogeneous, such as linear regression or decision trees, they are called base learners. If the models used for fusion are heterogeneous, such as a decision tree and a neural network fusion, it is called a component learner. Neural network models are non-convex and have many local optimal points, so multiple models can be fused and multiple initial values ​​can be taken to obtain a solution that is closer to the global optimal solution.

[0068] There are four types of fusion models (including hybrid):

[0069] Bagging: Train multiple base models using different randomly selected data subsets with replacement. Let the base models vote on the final prediction. Commonly used in Random Forests;

[0070] Boosting: Iteratively trains the model and updates the importance of each training example after each iteration. Commonly used in Gradient Boosting algorithm;

[0071] Blending: Train many different types of base models and make predictions on a holdout set. Train a new model from their predictions and make predictions on the test set (stacked with a holdout set);

[0072] Stacking: Train multiple different types of base models and make predictions on k-folds of the dataset. Then train a new model from their predictions and make predictions on the test set.

[0073] From the above description, it can be seen that the multi-model fusion method based on model application portraits provided in the embodiment of the present invention first extracts the feature data of multiple basic models to generate application portraits corresponding to each of the multiple basic models; then, the multiple basic models are screened according to the application data and the multiple application portraits; finally, the multiple basic models are fused according to the distribution differences of the prediction results of the application data based on the screened multiple basic models. According to the basic model application portraits and in combination with the distribution characteristics of the application data, the present invention performs attention-weighted integration and multi-level random sampling model fusion customization on the prediction results. The present invention can perform personalized model fusion based on the distribution characteristics of the data and the prediction ability of the basic model in specific application data to meet actual application needs.

[0074] In one embodiment, see Figure 2 , step 100 comprises:

[0075] Step 101: extracting the performance evaluation parameters, overall prediction accuracy, new data prediction accuracy, partial sample data prediction accuracy and authoritative evaluation parameters of the multiple basic models;

[0076] Basic model application profile: = {performance score, authority score, overall prediction ability, generalization ability of new sample distribution, prediction ability of small sample data}

[0077] Performance evaluation parameters: Each basic model is evaluated and ranked in multiple dimensions such as training speed, inference speed, and convergence status, and performance scores (ranking) are given according to the rankings. Preferably, the training speed (train), inference speed (inference), and convergence status (loss) are also evaluated and ranked.

[0078] The model's performance score Score_Performance is defined as follows:

[0079] Score_Performance=α1×Score(train)+β1×Score(inference)+γ1×Score(loss);

[0080] Among them, α1, β1, and γ1 are predefined weights.

[0081] Authority evaluation: Evaluate the model’s overall predictive ability, new data distribution predictive ability, and small sample data predictive ability, and calculate the authority of the basic model on this basis.

[0082] Overall prediction capability (Acc_All): Taking text analysis as an example, evaluate the accuracy, precision, recall and other quality evaluation indicators of the prediction results

[0083] New sample distribution generalization ability (Acc_New): prediction accuracy of new distribution samples. The difference D between the test sample S and the training set distribution T KL＝ =(S||T) is greater than the predefined threshold k, it is considered to be a new distribution sample.

[0084] Acc_Few: The prediction accuracy of test data in a small sample data subset. For example, in text classification, if the number of training samples of category Ci is less than n (n<=20), then Ci belongs to the small sample category.

[0085] The authority score of the model (Authority):

[0086] Authority=α2×Score(Acc_All)+β2×Score(Acc_New)+γ2×Score(Acc_Few);

[0087] Among them, α2, β2, and γ2 are pre-defined weights.

[0088] Step 102: Generate application portraits corresponding to multiple basic models according to the performance evaluation parameters, overall prediction accuracy, new data prediction accuracy, partial sample data prediction accuracy and authoritative evaluation parameters.

[0089] Specifically, the evaluation results of each basic model are deeply analyzed, and the application portrait of the basic model is dynamically generated from several dimensions such as performance, overall prediction accuracy, new data prediction accuracy, and small sample data prediction accuracy.

[0090] In one embodiment, see Figure 3 , step 200 comprises:

[0091] Step 201: Determine the distribution difference of the application data;

[0092] It is necessary to determine whether the overall distribution of the target data segment is divergent or concentrated, in which frequency segment it is concentrated, in which interval segment the median is concentrated, and in which data interval segment 80% of the data is concentrated. Optional tools include: histogram, box plot, normal distribution, dot plot, Pareto chart.

[0093] Step 202: Determine the performance parameters and prediction accuracy corresponding to each of the multiple basic models according to the distribution difference and the multiple application portraits;

[0094] Step 203: Screening the multiple basic models according to the performance parameter, the preset performance parameter threshold, the prediction accuracy and the preset prediction accuracy threshold.

[0095] Specifically, according to the application portrait of the basic model on the current data distribution, the trained model is screened and the preferred basic model is output.

[0096] In one embodiment, see Figure 4 , step 300 comprises:

[0097] Step 301: Generate multiple fusion sub-models based on the screened multiple basic models using a multi-level random sampling method;

[0098] The random sampling method is that each part of the survey object population has an equal chance of being selected. It is a sampling survey conducted completely according to the principle of equal opportunity, which is called "equal probability". According to the principle of randomness, it is guaranteed that each object in the population has a known, non-zero probability of being selected as the object of research, ensuring the representativeness of the sample.

[0099] There are four basic forms of random sampling, namely simple random sampling, equidistant sampling, type sampling and cluster sampling. Its greatest advantage is that when inferring the population based on sample data, the reliability of the inference value can be objectively measured in a probability manner, so that the inference is based on a scientific basis. Because of this, random sampling is widely used in social surveys and social research. Commonly used random sampling methods include pure random sampling, stratified sampling, systematic sampling, cluster sampling, multi-stage sampling, etc.

[0100] When step 301 is implemented, the prediction result categories {Cd1, Cd2, ... Cdk} to be randomly sampled are first selected according to a predefined ratio, and the sample layer within each model is randomly sampled. In the selected category Cdi∈{Cd1, Cd2, ... Cdk}, samples are randomly sampled according to a predefined ratio within each category.

[0101] Step 302: clustering the prediction results of the multiple fusion sub-models;

[0102] According to the multi-dimensional portrait of the basic model, the basic model is clustered {C1, C2, ... Cn}, and the prediction results of each basic model are classified into the corresponding category.

[0103] Step 303: Fusion the multiple basic models according to the clustered prediction results and the distribution differences of the prediction results of the multiple basic models for the application data.

[0104] The objective function of model fusion customization is defined as follows: Given a prediction result data sample (x i ,y i ), minimize lossL i , where L i NLL is the loss of fusion sub-model i, L ikL is the mean difference in the distribution of prediction results between the fusion sub-models

[0105] L i =L i NLL +αL i kL

[0106] L i NLL = -logP1 w (y i |x i )-logP2 w (y i |x i ),

[0107]

[0108] Among them, y i is the correct answer.X i It is a sequence of prediction results based on the weighted integration of the attention of each basic model.

[0109] Xi={x 1 i :Att(M1,x 1 i ),x 2 i :Att(M2,x 2 i ),………x k i :Att(M k ,x k i )}},

[0110] Among them, Att(M j ,x j i ) is the basic model M j The predicted result x j i The attention received; D kL (P1 w (y i |x i )||P2 w (y i |x i )) is the distribution P1 w (y i |x i ) and P1 w (y i |x i ) is the KL divergence of .

[0111] After fusion, the fusion model needs to be evaluated. If it has converged, the fusion model is output; otherwise, iterative training continues.

[0112] In one embodiment, see Figure 5 ,The multi-model fusion method based on model application portrait also includes:

[0113] Step 400: Generate basic models corresponding to the multiple algorithms respectively according to the multiple algorithms and training data.

[0114] Specifically, multiple algorithms are selected from the algorithm library, and the multiple algorithms are learned on the original training data set to generate multiple basic models.

[0115] To further illustrate this solution, this application provides a specific application example of the multi-model fusion method based on model application portrait, see Figure 6 .

[0116] See also Figure 7 ,Overall, the multi-model fusion method based on model application ,profiling is mainly divided into two stages:

[0117] Phase 1: Basic model training and dynamic generation of model application portraits.

[0118] Basic model training: Select multiple algorithms from the algorithm library and learn multiple basic models on the original training data set.

[0119] Multi-dimensional basic model application portrait generation: In-depth analysis of the evaluation results of each basic model, dynamic generation of basic model application portraits from the dimensions of performance, overall prediction accuracy, new data prediction accuracy, and small sample data prediction accuracy

[0120] Phase 2: Model fusion customization phase based on basic model application portrait.

[0121] Integration of prediction results based on model attention: After the basic model generates a sequence of prediction results on the application data set, the basic model application portrait is dynamically adjusted according to the differences in application data distribution, and the attention weight is dynamically generated for each prediction result.

[0122] S1: Basic model training.

[0123] According to the application task requirements, select an available algorithm set from the algorithm library, perform model training and evaluation on the training data set, and output the test results.

[0124] S2: Multi-dimensional basic model application portrait generation.

[0125] Based on the model performance analysis and evaluation results analysis, the basic model application portrait is dynamically generated.

[0126] S3: Integration of prediction results based on model attention.

[0127] Given an application data set, run the optimal basic model {M1, M2, ...Mn} to obtain the prediction result set {x1, x2, ...x k}

[0128] Prediction result integration based on model attention: Given a prediction result set {x1, x2, ... k}, and weight the model attention for each prediction result according to the model application portrait.

[0129] Given the prediction result, the prediction result Xi = {x 1 i ,x 2 i ,…x s i …,x k i}, through the following attention weighted calculation, we can get

[0130] X'={x 1 i :Att(M1,x 1 i ),x 2 i :Att(M2,x 2 i ),……x s i: Att(M s ,x s i )…x k i :Att(M k ,x k i )}

[0131] Base Model M s Prediction result x s i The degree of attention is mainly calculated based on the authority of the basic model, the confidence of the prediction results, and the difference with the prediction results of other basic models.

[0132] Att(M s ,x s i )=α×Authority(M s )×Confidence(x s i )+β×(1-Divergence(x si ,M s ))

[0133] Among them, α, β are predefined weights, Authority (M s ) is the model M s Authority, Confidence(x s i ):M s Prediction result x s i Credibility, Divergence(x s i ,M s ): Model M s Prediction result x s i 's difference.

[0134] Model M s Prediction result x s i Divergence(x s i ,M s ): Model M s The average distribution difference between the prediction results of and the prediction results of other models:

[0135] Divergence(x s i ,M s )=L i KL =1 / (K-1)∑D kL (x s i ||x t i ),1<=t<=K, and s<>t,

[0136] In the above formula, D kL (x s i ||x t i ) is the predicted result x s i With x t i The KL divergence of .

[0137] Dynamic weighting of model authority based on differences in application data distribution: The authority of the model can be dynamically weighted based on the differences in the distribution of current application data and training data.

[0138] If the current application data and the small sample training data are distributed from the same source, the authority of the model with strong small sample weight prediction ability is weighted by δ1, that is, Authority(M s )=Authority0(M s )×(1+δ1);

[0139] If the current application data belongs to the new data distribution, the authority of the model with strong generalization ability of the new sample is weighted by δ2, that is, Authority(M s )=Authority0(M s )×(1+δ2).

[0140] S4: Model fusion customization based on basic model application profiling and multi-level random sampling.

[0141] Integration of prediction results based on model attention: After the basic model generates a sequence of prediction results on the application data set, the basic model application profile is dynamically adjusted according to the difference in application data distribution, and attention weights are dynamically generated for each prediction result. Given a set of basic model prediction results with weighted attention, different fusion sub-models are customized using multi-level random sampling to minimize the difference in the final output results of each fusion sub-model.

[0142] From the above description, it can be seen that the multi-model fusion method based on model application portraits provided in the embodiment of the present invention first extracts the feature data of multiple basic models to generate application portraits corresponding to each of the multiple basic models; then, the multiple basic models are screened according to the application data and the multiple application portraits; finally, the multiple basic models are fused according to the distribution differences of the prediction results of the application data based on the screened multiple basic models. According to the basic model application portraits and in combination with the distribution characteristics of the application data, the present invention performs attention-weighted integration and multi-level random sampling model fusion customization on the prediction results. The present invention can perform personalized model fusion based on the distribution characteristics of the data and the prediction ability of the basic model in specific application data to meet actual application needs.

[0143] Based on the same inventive concept, the embodiments of the present application also provide a multi-model fusion device based on model application portraits, which can be used to implement the methods described in the above embodiments, such as the following embodiments. Since the principle of solving the problem by the multi-model fusion device based on model application portraits is similar to that of the multi-model fusion method based on model application portraits, the implementation of the multi-model fusion device based on model application portraits can refer to the implementation of the multi-model fusion method based on model application portraits, and the repetitions will not be repeated. As used below, the terms "unit" or "module" can be a combination of software and / or hardware that implements predetermined functions. Although the system described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0144] The embodiment of the present invention provides a specific implementation of a multi-model fusion device based on model application portraits that can implement a multi-model fusion method based on model application portraits, see Figure 8 The multi-model fusion device based on model application portrait specifically includes the following contents:

[0145] A feature data extraction module 10 is used to extract feature data of multiple basic models to generate application portraits corresponding to each of the multiple basic models;

[0146] A basic model screening module 20, configured to screen the multiple basic models according to the application data and multiple application portraits;

[0147] The basic model fusion module 30 is used to fuse multiple basic models according to the distribution difference of the prediction results of the application data based on the multiple basic models after screening.

[0148] In one embodiment, see Fig. 9 , the feature data extraction module 10 comprises:

[0149] A parameter extraction unit 101 is used to extract the performance evaluation parameters, overall prediction accuracy, new data prediction accuracy, partial sample data prediction accuracy and authoritative evaluation parameters of the multiple basic models;

[0150] The portrait generation unit 102 is used to generate application portraits corresponding to multiple basic models according to the performance evaluation parameters, overall prediction accuracy, new data prediction accuracy, partial sample data prediction accuracy and authoritative evaluation parameters.

[0151] In one embodiment, see Fig.10 , the basic model screening module 20 includes:

[0152] A distribution difference determining unit 201, configured to determine a distribution difference of the application data;

[0153] A parameter determination unit 202, configured to determine the performance parameters and prediction accuracy corresponding to each of the multiple basic models according to the distribution difference and the multiple application portraits;

[0154] The basic model screening unit 203 is used to screen the multiple basic models according to the performance parameter, the preset performance parameter threshold, the prediction accuracy and the preset prediction accuracy threshold.

[0155] In one embodiment, see Fig.11 , the basic model fusion module 30 includes:

[0156] The sub-model generating unit 301 is used to generate a plurality of fusion sub-models according to the screened multiple basic models by using a multi-level random sampling method;

[0157] A result clustering unit 302, used for clustering the prediction results of the multiple fusion sub-models;

[0158] The basic model fusion unit 303 is used to fuse multiple basic models according to the prediction results after clustering and the distribution difference of the prediction results of the multiple basic models for the application data.

[0159] In one embodiment, see Fig.12 The multi-model fusion device based on the model application portrait also includes:

[0160] The basic model generation module 40 is used to generate basic models corresponding to the multiple algorithms according to the multiple algorithms and training data.

[0161] From the above description, it can be seen that the multi-model fusion device based on model application portrait provided by the embodiment of the present invention first extracts the feature data of multiple basic models to generate application portraits corresponding to each of the multiple basic models; then, the multiple basic models are screened according to the application data and the multiple application portraits; finally, the multiple basic models are fused according to the distribution differences of the prediction results of the application data based on the screened multiple basic models. According to the basic model application portrait and in combination with the distribution characteristics of the application data, the present invention performs attention-weighted integration and multi-level random sampling model fusion customization on the prediction results. The present invention can perform personalized model fusion according to the distribution characteristics of the data and the prediction ability of the basic model in specific application data to meet the actual application needs.

[0162] The embodiment of the present application also provides a specific implementation of an electronic device that can implement all steps in the multi-model fusion method based on model application portrait in the above embodiment, see Fig.13 , electronic equipment specifically includes the following:

[0163] Processor (processor) 1201, memory (memory) 1202, communication interface (CommunicationsInterface) 1203 and bus 1204;

[0164] The processor 1201 , the memory 1202 , and the communication interface 1203 communicate with each other via the bus 1204 ; the communication interface 1203 is used to realize information transmission between relevant devices such as the server-side device, the power measurement device, and the user-side device.

[0165] The processor 1201 is used to call the computer program in the memory 1202. When the processor executes the computer program, all the steps in the multi-model fusion method based on the model application portrait in the above embodiment are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0166] Step 100: extracting feature data of multiple basic models to generate application portraits corresponding to each of the multiple basic models;

[0167] Step 200: Filter the multiple basic models according to the application data and multiple application portraits;

[0168] Step 300: Fusion the multiple basic models based on the distribution differences of the prediction results of the application data according to the multiple basic models that have been screened.

[0169] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps in the multi-model fusion method based on model application portrait in the above embodiment. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, all the steps of the multi-model fusion method based on model application portrait in the above embodiment are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0170] Step 100: extracting feature data of multiple basic models to generate application portraits corresponding to each of the multiple basic models;

[0171] Step 200: Filter the multiple basic models according to the application data and multiple application portraits;

[0172] Step 300: Fusion the multiple basic models based on the distribution differences of the prediction results of the application data according to the multiple basic models that have been screened.

[0173] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the hardware + program embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0174] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0175] Although the present application provides method operation steps such as embodiments or flow charts, more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in the embodiments is only one way of executing the order of many steps and does not represent the only execution order. When the actual device or client product is executed, it can be executed in the order of the method shown in the embodiments or the drawings or in parallel (for example, in a parallel processor or multi-threaded processing environment).

[0176] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0177] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0178] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0179] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A multi-model fusion method based on model application portrait, which is applied to image processing and is characterized by: include: Extracting feature data of multiple basic models to generate application portraits corresponding to each of the multiple basic models; wherein the application portrait refers to a means of abstracting the overall picture of information from the basic model; Filtering the multiple basic models according to the application data and multiple application portraits; According to the selected multiple basic models, the multiple basic models are integrated according to the distribution difference of the prediction results of the application data; According to the selected multiple basic models, the multiple basic models are integrated according to the distribution difference of the prediction results of the application data, including: Using a multi-level random sampling method, multiple fusion sub-models are generated based on the screened multiple basic models; Clustering the prediction results of the multiple fusion sub-models; The multiple basic models are fused according to the prediction results after clustering and the distribution difference of the prediction results of the multiple basic models for the application data, specifically: The objective function of model fusion customization is defined as follows: Given a prediction result data sample (x i ,y i ), minimize lossL i , where L i NLL is the loss of fusion sub-model i, L i kL is the mean difference in the distribution of prediction results between the fusion sub-models L i =L i NLL +αL i kL L i NLL =-logP1 w (and i |x i )-logP2 w (and i |x i ), Among them, y i is the correct answer.X i It is a sequence of prediction results based on the weighted integration of the attention of each basic model; Xi={x 1 i :At(M1,x 1 i ),x 2 i :At(M2,x 2 i ),………x k i :That(M k ,x k i )}}, Among them, Att(M j ,x j i ) is the basic model M j The predicted result x j i The attention received; D kL (P1 w (y i |x i )||P2 w (y i |x i )) is the distribution P1 w (y i |x i ) and P2 w (y i |x i )’s KL divergence; After fusion, the fusion model is evaluated. If convergence has been achieved, the fusion model is output; otherwise, iterative training continues.

2. The multi-model fusion method based on model application portrait according to claim 1 is characterized in that: The extracting the feature data of the multiple basic models to generate application portraits corresponding to the multiple basic models respectively includes: Extracting performance evaluation parameters, overall prediction accuracy, new data prediction accuracy, partial sample data prediction accuracy, and authoritative evaluation parameters of the multiple basic models; Based on the performance evaluation parameters, overall prediction accuracy, new data prediction accuracy, partial sample data prediction accuracy and authoritative evaluation parameters, application portraits corresponding to each of the multiple basic models are generated.

3. The multi-model fusion method based on model application portrait according to claim 1 is characterized in that: The screening of the multiple basic models according to the application data and the multiple application portraits includes: determining distribution differences of the application data; Determine the performance parameters and prediction accuracy corresponding to each of the multiple basic models according to the distribution difference and the multiple application portraits; The multiple basic models are screened according to the performance parameter, the preset performance parameter threshold, the prediction accuracy and the preset prediction accuracy threshold.

4. The multi-model fusion method based on model application portrait according to claim 1 is characterized in that: Also includes: Based on the multiple algorithms and the training data, basic models corresponding to the multiple algorithms are generated.

5. A multi-model fusion device based on model application portrait, which is applied to image processing, characterized in that: include: A feature data extraction module, used to extract feature data of multiple basic models to generate application portraits corresponding to each of the multiple basic models; wherein the application portrait refers to a means of abstracting the overall picture of information from the basic model; A basic model screening module, used to screen the multiple basic models according to the application data and multiple application portraits; A basic model fusion module, used to fuse multiple basic models according to the distribution difference of prediction results of the application data from the screened multiple basic models; According to the selected multiple basic models, the multiple basic models are integrated according to the distribution difference of the prediction results of the application data, including: Using a multi-level random sampling method, multiple fusion sub-models are generated based on the screened multiple basic models; Clustering the prediction results of the multiple fusion sub-models; The multiple basic models are fused according to the prediction results after clustering and the distribution difference of the prediction results of the multiple basic models for the application data, specifically: The objective function of model fusion customization is defined as follows: Given a prediction result data sample (x i ,y i ), minimize lossL i , where L i NLL is the loss of fusion sub-model i, L i kL is the mean difference in the distribution of prediction results between the fusion sub-models L i =L i NLL +αL i kL L i NLL =-logP1 w (and i |x i )-logP2 w (and i |x i ), Among them, y i is the correct answer.X i It is a sequence of prediction results based on the weighted integration of the attention of each basic model; Xi={x 1 i :At(M1,x 1 i ),x 2 i :At(M2,x 2 i ),………x k i :That(M k ,x k i )}}, Among them, Att(M j ,x j i ) is the basic model M j The predicted result x j i The attention received; D kL (P1 w (y i |x i )||P2 w (y i |x i )) is the distribution P1 w (y i |x i ) and P2 w (y i |x i )’s KL divergence; After fusion, the fusion model is evaluated. If convergence has been achieved, the fusion model is output; otherwise, iterative training continues.

6. The multi-model fusion device based on model application portrait according to claim 5 is characterized in that: The feature data extraction module comprises: A parameter extraction unit, used to extract the performance evaluation parameters, overall prediction accuracy, new data prediction accuracy, partial sample data prediction accuracy and authoritative evaluation parameters of the multiple basic models; The portrait generation unit is used to generate application portraits corresponding to multiple basic models according to the performance evaluation parameters, overall prediction accuracy, new data prediction accuracy, partial sample data prediction accuracy and authoritative evaluation parameters.

7. The multi-model fusion device based on model application portrait according to claim 5 is characterized in that: The basic model screening module includes: a distribution difference determining unit, configured to determine a distribution difference of the application data; A parameter determination unit, configured to determine the performance parameters and prediction accuracy corresponding to each of the plurality of basic models according to the distribution difference and the plurality of application portraits; The basic model screening unit is used to screen the multiple basic models according to the performance parameter, the preset performance parameter threshold, the prediction accuracy and the preset prediction accuracy threshold.

8. The multi-model fusion device based on model application portrait according to claim 5, characterized in that: Also includes: The basic model generation module is used to generate basic models corresponding to the multiple algorithms according to the multiple algorithms and training data.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the multi-model fusion method based on model application portrait described in any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the multi-model fusion method based on model application portrait described in any one of claims 1 to 4 are implemented.

Images