Closed source model stable integration method and device based on voting weight learning

Abstract

The invention provides a closed source model stable integration method and device based on voting weight learning, and belongs to the field of model integration and distribution external generalization. The method comprises the steps that prediction results of a plurality of closed source models of the same type on a preset training data sample and the weight of the training data sample are acquired, then a voting weight distributor is trained, the input of the voting weight distributor is a sample input feature, and the output of the voting weight distributor is the voting weight of each closed source model; and during actual prediction, performing weighted summation on the prediction result of the to-be-predicted sample by using the closed source model by using the voting weight output by the voting weight distributor so as to obtain a final prediction result of the to-be-predicted sample after integration. According to the method, under the condition that a plurality of closed source models are given, the prediction performance of the uncertainty test data samples can be improved by utilizing the unique advantages of all the models, and the good prediction performance of the uncertainty test data samples is kept.

Description

Technical Field

[0001] The present invention belongs to the field of model integration and distribution generalization, and in particular relates to a closed-source model stable integration method and device based on voting weight learning. Background Art

[0002] In recent years, the emergence of advanced algorithms and increases in computing power have enabled the widespread deployment of machine learning models in commercial scenarios, with applications ranging from visual recognition software to credit risk management systems. In these scenarios, developers train their models using proprietary data and then provide these models as closed-source digital products and services. This means that the model's source code, parameters, and training data are not publicly available; users access query results by calling function interfaces.

[0003] Through extensive training, these closed-source models typically achieve minimal empirical prediction loss on training datasets, meaning they have low expected prediction loss on samples that are common in the training data distribution. However, due to objective constraints such as sampling cost, time, and location, training data is often subject to sample selection bias and cannot capture all heterogeneous sample types. As a result, the trained models tend to learn specific correlation patterns associated with samples that are common in the training data, leading to increased prediction risk for data samples that are uncommon in the training distribution. When there is a distribution shift between the actual test data and the training data, model performance degrades. However, during commercial deployment, closed-source models inevitably face diverse and unpredictable data environments in the real world. Their unstable performance in responding to distribution shifts can easily lead to serious security issues.

[0004] On the other hand, because developers often use unique strategies to obtain different private data, they often train different closed-source models under different data distributions. For a given test data distribution, when the training data distribution of a closed-source model aligns with the test data distribution, the model performance is generally good; conversely, if there is a significant distribution shift between the training data and test data of a closed-source model, the performance of these models will drop significantly. Therefore, different closed-source models will show their respective advantages in different data environments. So, a practical question is, when users can call multiple closed-source models, how to leverage the complementary advantages of these models to ensure good predictive performance for uncertain test samples.

[0005] For example, in the field of image recognition, companies such as SenseTime, Megvii, and Yitu Technology all offer their own image recognition services. Users call these services to identify images to be predicted. However, each company's image recognition service has its own strengths and weaknesses. For example, if the model training data contains a large number of scenes with abundant lighting, the image recognition service will perform well in brightly lit scenes, but will struggle with low-light scenes. In this case, accurate predictions will need to be provided by image recognition services provided by other models whose training data contains a richer variety of low-light scenes.

[0006] However, there is currently no method that can well integrate the advantages of multiple existing closed-source models and achieve complementary advantages. Summary of the Invention

[0007] The purpose of the present invention is to overcome the shortcomings of existing technologies by proposing a method and apparatus for the stable integration of closed-source models based on voting weight learning. Given multiple closed-source models, the present invention can leverage the unique advantages of each model to improve the predictive performance of uncertain test data samples and maintain good predictive performance for these uncertain test data samples. In the field of image recognition, by integrating image recognition services provided by different companies and leveraging the advantages of these different image recognition services, the present invention can effectively cope with diverse image data environments and maintain good predictive performance for uncertain image samples.

[0008] The first embodiment of the present invention proposes a closed-source model stable integration method based on voting weight learning, including:

[0009] Obtain prediction results of multiple closed-source models of the same type on preset training data samples and the weights of the training data samples;

[0010] Using the prediction results and the weights of the training data samples, a neural network-based voting weight allocator is trained; the input of the voting weight allocator is the sample input features, and the output is the voting weights of each closed-source model;

[0011] During actual prediction, the voting weights output by the voting weight allocator are used to perform weighted summation on the prediction results of the samples to be predicted by the closed-source model to obtain the final prediction results after integration of the samples to be predicted.

[0012] In a specific embodiment of the present invention, it also includes:

[0013] Get multiple closed-source models with the same input data type, the same output data type, and different parameters And training data samples Among them, h m (x) represents the mth closed-source model, M≥2 is the number of closed-source models, x iis the input feature of the i-th training data sample, y i is the true output result of the i-th training data sample, N ≥ 5 represents the number of training data samples;

[0014] After inputting the input features of the training data samples into M closed-source models, the prediction results of each closed-source model for each training data sample are obtained. where h m (x i ) is the prediction result of the mth closed-source model for the i-th training data sample.

[0015] In a specific embodiment of the present invention, it also includes:

[0016] Learning training data samples through stochastic gradient descent optimization Weight Make minimize where w i is the weight of the i-th training data sample, x i,j is the input feature x of the i-th training data sample i The value of the j-th dimension feature in x i,k is the input feature x of the i-th training data sample i The value of the k-th dimension feature in, p is x i The number of feature dimensions.

[0017] In a specific embodiment of the present invention, the voting weight allocator training process is as follows:

[0018] 1) Construct a voting weight allocator g(x), with the model parameter of the random initial g(x) being θ;

[0019] The input of g(x) is the sample input feature x, and the output is the voting weight of each closed-source model where v m is the weight of the prediction result of the mth closed-source model for the input feature x in the final prediction result;

[0020] 2) Using training data samples Train the current voting weight allocator;

[0021] During training, the x of each training data sample is i Input the current g(x), g(x) outputs the voting weight of each closed-source model for the training data sample Among them, v m,i is the voting weight of the mth closed-source model on the i-th training data sample;

[0022] Then, use the voting weight to predict the results of M closed-source models for each training data sample Weighted, to get the prediction results after voting by each closed-source model in The prediction result of the i-th training data sample after voting by the closed-source model;

[0023] 3) Update the parameter θ using stochastic gradient descent optimization; based on the weight of the training data samples Prediction results after reducing voting And the actual output The weighted prediction error Thus, the updated parameter θ is obtained;

[0024] 4) Repeat steps 2)-3) until the preset training end condition is reached, and the optimized model parameters θ of the voting weight distributor g(x) are obtained. * , the voting weight allocator is trained.

[0025] In a specific embodiment of the present invention, it also includes:

[0026] 1) In the actual scenario, obtain test data x with the same input feature type and feature dimension as the training data sample test As the sample to be predicted;

[0027] 2) The test data x test Input closed-source model Get each closed-source model for x test The prediction results where {h m (x test ) represents the mth closed-source model for x test The prediction results;

[0028] 3) x test Input the trained voting weight distributor g(x) to get the closed-source model for x test Voting weight where v m,test Represents the mth closed-source model for x test voting weight;

[0029] 4) Based on the results of steps 2) and 3), the prediction results of the closed-source model are weighted summed to obtain the closed-source model's prediction of x test The final prediction result:

[0030]

[0031] The second embodiment of the present invention provides a closed-source model stable integration device based on voting weight learning, comprising:

[0032] A training prediction result acquisition module is used to obtain the prediction results of multiple closed-source models of the same type on preset training data samples and the weights of the training data samples;

[0033] A voting weight allocator training module is used to train a neural network-based voting weight allocator using the prediction results and the weights of the training data samples; the input of the voting weight allocator is the sample input feature, and the output is the voting weight of each closed-source model; during actual prediction, the integrated prediction module output by the voting weight allocator is used to perform weighted summation on the prediction results of the samples to be predicted by the closed-source models using the voting weights output by the voting weight allocator, so as to obtain the final prediction result of the samples to be predicted after integration.

[0034] In a specific embodiment of the present invention, it also includes:

[0035] Get multiple closed-source models with the same input data type, the same output data type, and different parameters And training data samples Among them, h m (x) represents the mth closed-source model, M≥2 is the number of closed-source models, x i is the input feature of the i-th training data sample, y i is the true output result of the i-th training data sample, N ≥ 5 represents the number of training data samples;

[0036] After inputting the input features of the training data samples into M closed-source models, the prediction results of each closed-source model for each training data sample are obtained. where h m (x i ) is the prediction result of the mth closed-source model for the i-th training data sample.

[0037] In a specific embodiment of the present invention, it also includes:

[0038] Learning training data samples through stochastic gradient descent optimization Weight Make minimize where w i is the weight of the i-th training data sample, x i,j is the input feature x of the i-th training data sample i The value of the j-th dimension feature in x i,k is the input feature x of the i-th training data sample i The value of the k-th dimension feature in, p is x i The number of feature dimensions.

[0039] In a specific embodiment of the present invention, the voting weight allocator training process is as follows:

[0040] 1) Construct a voting weight allocator g(x), with the model parameter of the random initial g(x) being θ;

[0041] The input of g(x) is the sample input feature x, and the output is the voting weight of each closed-source model where v m is the weight of the prediction result of the mth closed-source model for the input feature x in the final prediction result;

[0042] 2) Using training data samples Train the current voting weight allocator;

[0043] During training, the x of each training data sample is i Input the current g(x), g(x) outputs the voting weight of each closed-source model for the training data sample Among them, v m,i is the voting weight of the mth closed-source model on the i-th training data sample;

[0044] Then, use the voting weight to predict the results of M closed-source models for each training data sample Weighted, to get the prediction results after voting by each closed-source model in The prediction result of the i-th training data sample after voting by the closed-source model;

[0045] 3) Update the parameter θ using stochastic gradient descent optimization; based on the weight of the training data samples Prediction results after reducing voting And the actual output The weighted prediction error Thus, the updated parameter θ is obtained;

[0046] 4) Repeat steps 2)-3) until the preset training end condition is reached, and the optimized model parameters θ of the voting weight distributor g(x) are obtained. * , the voting weight allocator is trained.

[0047] In a specific embodiment of the present invention, it also includes:

[0048] 1) In the actual scenario, obtain test data x with the same input feature type and feature dimension as the training data sample test As the sample to be predicted;

[0049] 2) The test data x testInput closed-source model Get each closed-source model for x test The prediction results where {h m (x test ) represents the mth closed-source model for x test The prediction results;

[0050] 3) x test Input the trained voting weight distributor g(x) to get the closed-source model for x test Voting weight where v m,test Represents the mth closed-source model for x test voting weight;

[0051] 4) Based on the results of steps 2) and 3), the prediction results of the closed-source model are weighted summed to obtain the closed-source model's prediction of x test The final prediction result:

[0052]

[0053] A third embodiment of the present invention provides an electronic device, including:

[0054] at least one processor; and a memory communicatively coupled to the at least one processor;

[0055] The memory stores instructions that can be executed by the at least one processor, and the instructions are configured to execute the above-mentioned closed-source model stable integration method based on voting weight learning.

[0056] A fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions for enabling the computer to execute the above-mentioned closed-source model stable integration method based on voting weight learning.

[0057] Features and beneficial effects of the present invention:

[0058] 1) Compared with previous model integration technologies, this invention focuses on a large number of trained closed-source models in actual scenarios. By leveraging the complementary advantages of different closed-source models, it amplifies the overall predictive performance of existing models, enhances the predictive performance of uncertain data samples in a cost-effective manner, and has high application value for the reliable deployment and value mining of a large number of closed-source models.

[0059] 2) Compared with previous out-of-distribution generalization techniques, this invention cleverly utilizes the combined effect of trained closed-source models to improve the generalization ability in the face of uncertain data samples. That is, it selects appropriate closed-source models to provide computing services for different data samples to maintain good performance for different samples and improve generalization ability.

[0060] 3) The present invention combines stable learning technology to train a highly reliable closed-source model voting weight allocator, which can accurately identify closed-source models that perform more robustly on test samples and automatically output the voting weights of different closed-source models (the more robust the closed-source model, the higher the voting weight). By weighting the prediction results of different closed-source models, a highly reliable integrated prediction result is obtained, ensuring good prediction performance.

[0061] 4) When applied to the field of image recognition, the present invention can integrate image recognition services provided by different companies, utilize the complementary advantages of different image recognition services, effectively cope with diverse image data environments, maintain good prediction performance for uncertain image samples, and have high application value for the reliable deployment and value mining of a large number of closed-source image recognition models and services. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is an overall flow chart of a closed-source model stable integration method based on voting weight learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The present invention proposes a closed-source model stable integration method and device based on voting weight learning. The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0064] The first embodiment of the present invention proposes a closed-source model stable integration method based on voting weight learning, including:

[0065] Obtain prediction results of multiple closed-source models of the same type on preset training data samples and the weights of the training data samples;

[0066] Using the prediction results and the weights of the training data samples, a neural network-based voting weight allocator is trained; the input of the voting weight allocator is the sample input features, and the output is the voting weights of each closed-source model;

[0067] During actual prediction, the voting weights output by the voting weight allocator are used to perform weighted summation on the prediction results of the samples to be predicted by the closed-source model to obtain the final prediction results after integration of the samples to be predicted.

[0068] In a specific embodiment of the present invention, the closed-source model stable integration method based on voting weight learning is divided into a training phase and a prediction phase. The overall process is as follows: Figure 1 As shown, the following steps are included:

[0069] 1) Training phase.

[0070] 1-1) Obtain prediction results for multiple closed-source models of the same type on a preset training data sample.

[0071] In this embodiment, multiple closed-source models with the same input data type, the same output data type, and different parameters are obtained. And training data samples Among them, h m (x) represents the mth closed-source model, M≥2 is the number of closed-source models, x i is the input feature of the i-th training data sample, y i is the true output result of the i-th training data sample, and N ≥ 5 represents the number of training data samples.

[0072] After inputting the input features of the training data samples into the M closed-source models respectively, the prediction results of each closed-source model for each training data sample are obtained. where h m (x i ) is the prediction result of the mth closed-source model for the i-th training data sample.

[0073] This embodiment has no special requirements on the source of training data samples; if Previously used to train any of M closed-source models, It is still applicable in the method described in this embodiment.

[0074] 1-2) Obtain the weights of the training data samples in step 1-1).

[0075] For training data samples This embodiment uses the stochastic gradient descent optimization method to learn the weights of the training data samples Make Can be minimized where w i is the weight of the i-th training data sample, x i,j is the input feature x of the i-th training data sample i The value of the j-th dimension feature in x i,k is the input feature x of the i-th training data sample i The value of the k-th dimension feature in, p is x i The number of feature dimensions.

[0076] In this embodiment, the weight of the training data sample This allows the input features of the training data samples to be independent of each other after sample weighting, thereby eliminating the impact of the correlation between the features of each dimension on the subsequent learning of the voting weight allocator. This will be used when training the voting weight allocation model in steps 1-5).

[0077] 1-3) Build and initialize the voting weight allocator.

[0078] In this embodiment, a voting weight allocator g(x) based on a neural network (specifically a convolutional neural network for image data, a multi-layer perceptron for structured table data, and a transformer architecture for text data) is learned. The model parameter of the random initial g(x) is θ. The input of the voting weight allocator g(x) is the sample input feature x, and the output is the voting weight of each closed-source model. where v m is the weight of the prediction result of the mth closed-source model for the input feature x in the final prediction result.

[0079] 1-4) Using the prediction results obtained in step 1-1), the current voting weight allocator is trained to obtain the voting weights of the closed-source model for the training data samples and the prediction results for the training data samples after voting.

[0080] In this embodiment, the training data sample of step 1-1) is used Train the current voting weight distributor. During training, the x of each training data sample is i Input the current g(x), g(x) outputs the voting weight of each closed-source model for the training data sample Among them, v m,i is the voting weight of the mth closed-source model for the i-th training data sample.

[0081] Then, use the voting weight to predict the results of M closed-source models for each training data sample Weighted, to get the prediction results after voting by each closed-source model in is the prediction result of the i-th training data sample after voting by the closed-source model.

[0082] 1-5) Based on the results of steps 1-2) and 1-4), optimize the model parameters of the voting weight allocator.

[0083] In this embodiment, the stochastic gradient descent optimization method is used to update the model parameter θ of the voting weight distributor. Prediction results after reducing voting And the actual output The weighted prediction error To learn a stable and reliable voting weight allocator, the updated model parameters θ of the voting weight allocator are obtained.

[0084] 1-6) Repeat steps 1-4)-1-5) until the model parameters of the voting weight allocator converge (i.e., the weighted prediction error no longer decreases) or the upper limit of the preset optimization round is reached, and the optimized model parameters θ of the voting weight allocator g(x) are obtained. * , the voting weight allocator is trained.

[0085] 2) Prediction stage.

[0086] 2-1) In the actual scenario, obtain test data x with the same input feature type and feature dimension as the training data sample test as the sample to be predicted.

[0087] 2-2) The test data x obtained in step 2-1) is test Input closed-source model Get each closed-source model for x test The prediction results where {h m (x test ) represents the mth closed-source model for x test prediction results.

[0088] 2-3) Substitute the x obtained in step 2-1) test Input the voting weight distributor g(x) trained in step 1) to obtain the closed-source model for x test Voting weight where v m,test Represents the mth closed-source model for xt est voting weight.

[0089] 2-4) Based on the results of steps 2-2) and 2-3), the prediction results of the closed-source model are weighted summed to obtain the closed-source model's prediction of x test The final prediction result:

[0090]

[0091] The method described in this embodiment is further described in detail below with reference to a specific embodiment.

[0092] A specific embodiment of the present invention is applied to image recognition tasks. The method described in this embodiment can be used to integrate image recognition services provided by different companies, and the advantages of closed-source visual models that support different image recognition services can be complementary. For example, a closed-source model with the same lighting conditions as the training image data and the test image data can be adaptively used to predict the test image. This can effectively cope with diverse image data environments and maintain good prediction performance for uncertain image samples.

[0093] In this embodiment, the closed-source model stable integration method based on voting weight learning is divided into a training phase and a prediction phase, including the following steps:

[0094] 1) Training phase.

[0095] 1-1) Obtain prediction results for multiple closed-source models of the same type on a preset training data sample.

[0096] In this embodiment, multiple closed-source visual models with different inputs, image data, and outputs of different image types are obtained. And available training image data samples Among them, h m (x) represents the mth closed-source visual model, M ≥ 2 is the number of closed-source visual models, x i is the visual input feature of the i-th training image data sample, y i is the category label of the i-th training image data sample, and N≥5 represents the number of training image data samples. i It is a vector of length C with only one element set to 1 and the rest to 0. The position where the element is 1 indicates the category of the image, and C is the total number of image categories.

[0097] After the visual input features of the image data samples are input into the M closed-source visual models respectively, the prediction results of each closed-source visual model for each training image data sample are obtained. where h m (x i ) is the predicted category of the mth closed-source visual model for the i-th training image data sample. It is also a vector of length C with only one element being 1 and the rest being 0. The position of the element being 1 indicates the category of the image.

[0098] In this embodiment, there is no special requirement for the source of the training image data samples; if Previously used to train any of M closed-source vision models, It is still applicable in the method described in this embodiment.

[0099] 1-2) Obtain the weights of the training data samples in step 1-1).

[0100] For training image data samples This embodiment uses the optimization method of stochastic gradient descent to learn the weights of the training image data samples Make Can be minimized where w i is the weight of the i-th training image data sample, x i,j is the visual input feature x of the i-th training image data sample i The value of the j-th dimension feature in x i,k is the input feature x of the i-th training image data sample i The value of the k-th dimension feature in , p is x i The number of feature dimensions.

[0101] In this embodiment, the data sample weight This allows the visual features of the training image data samples to be independent of each other after sample weighting, thereby eliminating the impact of the correlation between the visual features of each dimension on the subsequent learning of the voting weight allocator. This will be used when training the voting weight allocation model in steps 1-5).

[0102] 1-3) Build and initialize the voting weight allocator.

[0103] In this embodiment, a voting weight allocator g(x) based on a convolutional neural network is learned, and the model parameters of the random initial g(x) are θ. The input of the voting weight allocator g(x) is the image visual input feature x, and the output is the voting weight of each closed-source visual model. where v m is the weight of the predicted category of the image visual feature x by the mth closed-source visual model in the final predicted category.

[0104] 1-4) Using the prediction results obtained in step 1-1), the current voting weight allocator is trained to obtain the voting weight of the closed-source visual model for the training image data sample and the predicted category of the training image data sample after voting.

[0105] In this embodiment, the training image data of step 1-1) is used Train the current voting weight distributor. During training, the x of each training image data sample is i Input the current g(x), g(x) outputs the voting weight of each closed-source visual model for the training image data sample Among them, v m,i is the voting weight of the mth closed-source vision model for the i-th training image data sample.

[0106] Then, the voting weight is used to predict the category of each training image data sample of M closed-source visual models. Weighted, to get the prediction results after voting by each closed-source visual model in is the predicted category distribution of the i-th training image data sample after voting by the closed-source vision model. It is a continuous vector of length C, and each element indicates the probability that the image belongs to the corresponding category.

[0107] 1-5) Based on the results of steps 1-2) and 1-4), optimize the model parameters of the voting weight allocator.

[0108] In this embodiment, the stochastic gradient descent optimization method is used to update the model parameter θ of the voting weight distributor based on the training image data sample weights. Reduce the predicted category distribution of the image after voting and real image categories The weighted prediction error To learn a stable and reliable voting weight allocator, the updated model parameters θ of the voting weight allocator are obtained.

[0109] 1-6) Repeat steps 1-4)-1-5) until the model parameters of the voting weight allocator converge (i.e., the weighted prediction error no longer decreases) or the upper limit of the preset optimization round is reached, and the optimized model parameters θ of the voting weight allocator g(x) are obtained. * , the voting weight allocator is trained.

[0110] 2) Prediction stage.

[0111] 2-1) Obtain a test image x with the same visual input feature dimension as the training image data sample in the actual scene test as the sample to be predicted.

[0112] 2-2) The test image x obtained in step 2-1) is test Input closed-source vision model Get each closed-source visual model pair x test Prediction category where {h m (x test ) represents the mth closed-source visual model for image x test The predicted category.

[0113] 2-3) The test image data x obtained in step 2-1) is test Input the voting weight distributor g(x) trained in step 1) to obtain the closed-source visual model for image x test Voting weight where v m,test Represents the mth closed-source visual model for image xtest voting weight.

[0114] 2-4) Based on the results of steps 2-2) and 2-3), the predicted categories of the closed-source visual model are weighted summed, from

[0115] To implement the above embodiment, a second embodiment of the present invention proposes a closed-source model stable integration device based on voting weight learning, comprising:

[0116] A training prediction result acquisition module is used to obtain the prediction results of multiple closed-source models of the same type on preset training data samples and the weights of the training data samples;

[0117] A voting weight allocator training module is used to train a neural network-based voting weight allocator using the prediction results and the weights of the training data samples; the input of the voting weight allocator is the sample input feature, and the output is the voting weight of each closed-source model; during actual prediction, the integrated prediction module output by the voting weight allocator is used to perform weighted summation on the prediction results of the samples to be predicted by the closed-source models using the voting weights output by the voting weight allocator, so as to obtain the final prediction result of the samples to be predicted after integration.

[0118] In a specific embodiment of the present invention, it also includes:

[0119] Get multiple closed-source models with the same input data type, the same output data type, and different parameters And training data samples Among them, h m (x) represents the mth closed-source model, M≥2 is the number of closed-source models, x i is the input feature of the i-th training data sample, y i is the true output result of the i-th training data sample, N ≥ 5 represents the number of training data samples;

[0120] After inputting the input features of the training data samples into M closed-source models, the prediction results of each closed-source model for each training data sample are obtained. where h m (x i ) is the prediction result of the mth closed-source model for the i-th training data sample.

[0121] In a specific embodiment of the present invention, it also includes:

[0122] Learning training data samples through stochastic gradient descent optimization Weight Make minimize where w i is the weight of the i-th training data sample, x i,j is the input feature x of the i-th training data sample i The value of the j-th dimension feature in x i,k is the input feature x of the i-th training data sample i The value of the k-th dimension feature in, p is x i The number of feature dimensions.

[0123] In a specific embodiment of the present invention, the voting weight allocator training process is as follows:

[0124] 1) Construct a voting weight allocator g(x), with the model parameter of the random initial g(x) being θ;

[0125] The input of g(x) is the sample input feature x, and the output is the voting weight of each closed-source model where v m is the weight of the prediction result of the mth closed-source model for the input feature x in the final prediction result;

[0126] 2) Using training data samples Train the current voting weight allocator;

[0127] During training, the x of each training data sample is i Input the current g(x), g(x) outputs the voting weight of each closed-source model for the training data sample Among them, v m,i is the voting weight of the mth closed-source model on the i-th training data sample;

[0128] Then, use the voting weight to predict the results of M closed-source models for each training data sample Weighted, to get the prediction results after voting by each closed-source model in The prediction result of the i-th training data sample after voting by the closed-source model;

[0129] 3) Update the parameter θ using stochastic gradient descent optimization; based on the weight of the training data samples Prediction results after reducing voting And the actual output The weighted prediction error Thus, the updated parameter θ is obtained;

[0130] 4) Repeat steps 2)-3) until the preset training end condition is reached, and the optimized model parameters θ of the voting weight distributor g(x) are obtained. *, the voting weight allocator is trained.

[0131] In a specific embodiment of the present invention, it also includes:

[0132] 1) In the actual scenario, obtain test data x with the same input feature type and feature dimension as the training data sample test As the sample to be predicted;

[0133] 2) The test data x test Input closed-source model Get each closed-source model for x test The prediction results where {h m (x test ) represents the mth closed-source model for x test The prediction results;

[0134] 3) x test Input the trained voting weight distributor g(x) to get the closed-source model for x test Voting weight where v m,test Represents the mth closed-source model for x test voting weight;

[0135] 4) Based on the results of steps 2) and 3), the prediction results of the closed-source model are weighted summed to obtain the closed-source model's prediction of x test The final prediction result:

[0136]

[0137] This makes it possible to utilize the unique advantages of each model to improve the prediction performance of uncertainty test data samples and maintain good prediction performance for uncertainty test data samples when multiple closed-source models are given.

[0138] To implement the above embodiment, a third aspect of the present invention provides an electronic device, including:

[0139] at least one processor; and a memory communicatively coupled to the at least one processor;

[0140] The memory stores instructions that can be executed by the at least one processor, and the instructions are configured to execute the above-mentioned closed-source model stable integration method based on voting weight learning.

[0141] To implement the above-mentioned embodiment, the fourth aspect of the present invention proposes a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the above-mentioned closed-source model stable integration method based on voting weight learning.

[0142] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0143] The computer-readable medium may be included in the electronic device, or may exist independently and not incorporated into the electronic device. The computer-readable medium carries one or more programs. When executed by the electronic device, the one or more programs cause the electronic device to perform the closed-source model stable integration method based on voting weight learning described in the above embodiment.

[0144] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0145] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0146] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. Throughout the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0147] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0148] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing it in a suitable manner if necessary, and then storing it in a computer memory.

[0149] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0150] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0151] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0152] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A closed-source model stable integration method based on voting weight learning, characterized by: include: Obtain prediction results of multiple closed-source models of the same type on preset training data samples and the weights of the training data samples; Using the prediction results and the weights of the training data samples, training a voting weight allocator based on a neural network; The input of the voting weight allocator is the sample input features, and the output is the voting weight of each closed-source model; During actual prediction, the voting weights output by the voting weight allocator are used to perform weighted summation on the prediction results of the samples to be predicted by the closed-source model to obtain the final prediction results after integration of the samples to be predicted.

2. The method according to claim 1, characterized in that Also includes: Get multiple closed-source models with the same input data type, the same output data type, and different parameters And training data samples Among them, h m (x) represents the mth closed-source model, M≥2 is the number of closed-source models, x i is the input feature of the i-th training data sample, y i is the true output result of the i-th training data sample, N ≥ 5 represents the number of training data samples; After inputting the input features of the training data samples into M closed-source models, the prediction results of each closed-source model for each training data sample are obtained. where h m (x i ) is the prediction result of the mth closed-source model for the i-th training data sample.

3. The method according to claim 2, characterized in that Also includes: Learning training data samples through stochastic gradient descent optimization Weight Make minimize where w i is the weight of the i-th training data sample, x i,j is the input feature x of the i-th training data sample i The value of the j-th dimension feature in x i,k is the input feature x of the i-th training data sample i The value of the k-th dimension feature in, p is x i The number of feature dimensions.

4. The method according to claim 3, characterized in that The voting weight allocator training process is as follows: 1) Construct a voting weight allocator g(x), with the model parameter of the random initial g(x) being θ; The input of g(x) is the sample input feature x, and the output is the voting weight of each closed-source model where v m is the weight of the prediction result of the mth closed-source model for the input feature x in the final prediction result; 2) Using training data samples Train the current voting weight allocator; During training, the x of each training data sample is i Input the current g(x), g(x) outputs the voting weight of each closed-source model for the training data sample Among them, v m,i is the voting weight of the mth closed-source model on the i-th training data sample; Then, use the voting weight to predict the results of M closed-source models for each training data sample Weighted, to get the prediction results after voting by each closed-source model in The prediction result of the i-th training data sample after voting by the closed-source model; 3) Update the parameter θ using stochastic gradient descent optimization; based on the weight of the training data samples Prediction results after reducing voting And the actual output The weighted prediction error Thus, the updated parameter θ is obtained; 4) Repeat steps 2)-3) until the preset training end condition is reached, and the optimized model parameters θ of the voting weight distributor g(x) are obtained. * , the voting weight allocator is trained.

5. The method according to claim 4, characterized in that Also includes: 1) In the actual scenario, obtain test data x with the same input feature type and feature dimension as the training data sample test As the sample to be predicted; 2) Test data x test Input closed-source model Get each closed-source model for x test The prediction results where {h m (x test ) represents the mth closed-source model for x test The prediction results; 3) x test Input the trained voting weight distributor g(x) to get the closed-source model for x test Voting weight where v m,test Represents the mth closed-source model for x test voting weight; 4) Based on the results of steps 2) and 3), the prediction results of the closed-source model are weighted summed to obtain the closed-source model's prediction of x test The final prediction result:

6. A closed-source model stable integration device based on voting weight learning, characterized in that: include: A training prediction result acquisition module is used to obtain the prediction results of multiple closed-source models of the same type on preset training data samples and the weights of the training data samples; A voting weight allocator training module, configured to train a voting weight allocator based on a neural network using the prediction results and the weights of the training data samples; The input of the voting weight allocator is the sample input feature, and the output is the voting weight of each closed-source model; during actual prediction, the integrated prediction module output by the voting weight allocator is used. When used for actual prediction, the closed-source model's prediction results for the sample to be predicted are weighted and summed using the voting weight output by the voting weight allocator to obtain the final prediction result of the sample to be predicted after integration.

7. The device according to claim 6, characterized in that Also includes: Get multiple closed-source models with the same input data type, the same output data type, and different parameters And training data samples Among them, h m (x) represents the mth closed-source model, M≥2 is the number of closed-source models, x i is the input feature of the i-th training data sample, y i is the true output result of the i-th training data sample, N ≥ 5 represents the number of training data samples; After inputting the input features of the training data samples into M closed-source models, the prediction results of each closed-source model for each training data sample are obtained. where h m (x i ) is the prediction result of the mth closed-source model for the i-th training data sample.

8. The device according to claim 7, characterized in that Also includes: Learning training data samples through stochastic gradient descent optimization Weight Make minimize where w i is the weight of the i-th training data sample, x i,j is the input feature x of the i-th training data sample i The value of the j-th dimension feature in x i,k is the input feature x of the i-th training data sample i The value of the k-th dimension feature in, p is x i The number of feature dimensions.

9. The device according to claim 8, characterized in that The voting weight allocator training process is as follows: 1) Construct a voting weight allocator g(x), with the model parameter of the random initial g(x) being θ; The input of g(x) is the sample input feature x, and the output is the voting weight of each closed-source model where v m is the weight of the prediction result of the mth closed-source model for the input feature x in the final prediction result; 2) Using training data samples Train the current voting weight allocator; During training, the x of each training data sample is i Input the current g(x), g(x) outputs the voting weight of each closed-source model for the training data sample Among them, v m,i is the voting weight of the mth closed-source model on the i-th training data sample; Then, use the voting weight to predict the results of M closed-source models for each training data sample Weighted, to get the prediction results after voting by each closed-source model in The prediction result of the i-th training data sample after voting by the closed-source model; 3) Update the parameter θ using stochastic gradient descent optimization; based on the weight of the training data samples Prediction results after reducing voting And the actual output The weighted prediction error Thus, the updated parameter θ is obtained; 4) Repeat steps 2)-3) until the preset training end condition is reached, and the optimized model parameters θ of the voting weight distributor g(x) are obtained. * , the voting weight allocator is trained.

10. The device according to claim 9, characterized in that Also includes: 1) In the actual scenario, obtain test data x with the same input feature type and feature dimension as the training data sample test As the sample to be predicted; 2) Test data x test Input closed-source model Get each closed-source model for x test The prediction results where {h m (x test ) represents the mth closed-source model for x test The prediction results; 3) x test Input the trained voting weight distributor g(x) to get the closed-source model for x test Voting weight where v m,test Represents the mth closed-source model for x test voting weight; 4) Based on the results of steps 2) and 3), the prediction results of the closed-source model are weighted summed to obtain the closed-source model's prediction of x test The final prediction result:

Images