Prediction method and apparatus based on stacking and machine learning models

Abstract

The application provides a prediction method and device based on stacking and a machine learning model. The method comprises: obtaining service data and a plurality of base models, the service data being data to be predicted, and the base models being used for prediction according to the service data; inputting the service data into the plurality of base models respectively to obtain a plurality of prediction results corresponding to the outputs of the plurality of base models; performing statistical processing on the plurality of prediction results to obtain a statistical result; inputting the plurality of prediction results and the statistical result into a meta model to obtain a final prediction result output by the meta model, the meta model being used for prediction according to the outputs of the plurality of base models, and the meta model being obtained by training an initial meta model based on training data, wherein the training data comprises training outputs of the plurality of base models and a result obtained by performing statistical processing on the training outputs. The plurality of prediction results of the plurality of base models are input into the meta model at the same time, and the statistical result of the plurality of prediction results is also input, so that the accuracy of prediction by the machine learning model and the stacking technology is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning, and particularly relates to a prediction method based on stacking and a machine learning model, a prediction device based on stacking and a machine learning model, an electronic device, a computer readable storage medium and a computer program product. BACKGROUND

[0002] With the advent of the era of artificial intelligence (AI), many businesses cannot be carried out without machine learning models (ML). Through training of the ML model and using the trained model for business prediction and analysis, accurate prediction and analysis of the business can be achieved.

[0003] In order to further improve the accuracy of the ML model prediction, the stacking technology is usually used. Specifically, a plurality of trained ML models are prepared as base models, and a logistic regression algorithm is prepared as a meta model. When it is necessary to make a prediction based on certain business data, the business data is input into each base model as input. Each base model processes the business data and outputs the prediction result. The prediction results output by each base model are input into the meta model as input. The meta model processes the prediction results using the logistic regression algorithm to obtain and output the final prediction result.

[0004] However, the stacking technology is limited by the accuracy of each ML model and the limitations of the logistic regression algorithm, and the effect of improving the prediction accuracy is limited. It can be seen that how to significantly improve the prediction accuracy is a problem to be solved at present. SUMMARY

[0005] The embodiments of the present application provide a prediction method based on stacking and a machine learning model, a prediction device based on stacking and a machine learning model, an electronic device, a computer readable storage medium and a computer program product, to improve the accuracy of prediction based on the machine learning model and the stacking technology.

[0006] To solve the above technical problems, the embodiments of the present application provide the following technical solutions:

[0007] The first aspect of the present application provides a prediction method based on stacking and machine learning model, the method comprising: obtaining service data and a plurality of base models, the service data being data to be predicted, and the base models being used to predict according to the service data; inputting the service data into the plurality of base models respectively to obtain a plurality of prediction results corresponding to the outputs of the plurality of base models; performing statistical processing on the plurality of prediction results to obtain a statistical result; and inputting the plurality of prediction results and the statistical result into a meta model to obtain a final prediction result output by the meta model, the meta model being used to predict according to the outputs of the plurality of base models, and the meta model being obtained by training an initial meta model based on training data, the training data comprising training outputs of the plurality of base models and a result obtained by performing statistical processing on the training outputs.

[0008] Compared with the prior art, the prediction method based on stacking and machine learning model provided by the first aspect of the present application, after the plurality of base models output the prediction results, the prediction results output by each base model are statistically processed, and then the statistical result and the prediction result of each base model are input into the meta model, so that the meta model further combines the statistical result for analysis and processing on the basis of the prediction results of the plurality of base models. Since the meta model can only perform logistic regression calculation based on the prediction results of the plurality of base models before, the data dimension and processing capacity are limited, and this time the prediction results of the plurality of base models are input into the meta model, and the statistical result of the plurality of prediction results is also input, so that the data dimension and processing capacity processed by the meta model can be improved, thereby improving the accuracy of prediction by the machine learning model and the stacking technology.

[0009] In some modified embodiments of the first aspect of the present application, the statistical processing on the plurality of prediction results to obtain the statistical result comprises: determining a physical metric value of the base model corresponding to each prediction result in the plurality of prediction results respectively based on the each prediction result, obtaining a plurality of physical metric values, and performing optimization processing on the plurality of physical metric values to obtain an optimal metric value; performing nonlinear processing on at least two prediction results in the plurality of prediction results to obtain a nonlinear result; and determining the optimal metric value and the nonlinear result as the statistical result.

[0010] During specific statistical processing, each prediction result can be converted and optimized, and all or two of the plurality of prediction results can be nonlinearly processed, which can increase the dimension of the statistical result, so that the data input into the meta model is more abundant, and the accuracy of prediction is further improved.

[0011] In some modified embodiments of the first aspect of the present application, the physical metric value comprises a confidence degree, and the nonlinear processing comprises at least one of the following: calculating a mode, calculating a variance, calculating an average value, calculating a difference between two, calculating a sum of two, and an optimal model.

[0012] Parameters such as confidence level and mode can be used to perform simple and accurate statistical analysis on multiple prediction results, thereby improving the efficiency and accuracy of the final prediction.

[0013] In some modified embodiments of the first aspect of this application, the meta-model includes multiple preset algorithms; inputting multiple prediction results and statistical results into the meta-model includes: inputting multiple prediction results, statistical results and a specified algorithm name into the meta-model, so that the meta-model can find the target algorithm corresponding to the specified algorithm name among multiple preset algorithms, and then process the multiple prediction results and statistical results based on the target algorithm.

[0014] Algorithms can be specified in the metamodel, improving the flexibility of metamodel construction. Furthermore, specifying the algorithm name allows for easy algorithm specification within the metamodel, enhancing the convenience of algorithm specification within the metamodel.

[0015] In some modified embodiments of the first aspect of this application, before inputting business data into multiple base models, the method further includes: determining the prediction accuracy parameter of each base model and determining the prediction similarity between pairs of base models; based on the prediction accuracy parameter and the prediction similarity, deleting one or more base models from the multiple base models to obtain a preset number of target base models, so that the business data is input into the target base models respectively, wherein the base model with the lower prediction accuracy parameter and the higher prediction similarity is deleted with priority, and the priority of the prediction accuracy parameter is higher than the priority of the prediction similarity.

[0016] Before using multiple base models for prediction, these models are screened. Base models with low prediction accuracy and similarity to other base models are removed. This avoids the reverse interference of low-accuracy base models with high-accuracy predictions from similar base models, thus improving the accuracy of the final prediction.

[0017] In some modified embodiments of the first aspect of this application, based on the prediction accuracy parameter and similarity, one or more base models from multiple base models are deleted to obtain a preset number of target base models. This includes: taking the base model with the largest prediction accuracy parameter among the multiple base models as the current base model, and taking the base models other than the current base model among the multiple base models as other base models; among the other base models, deleting the base model with the highest prediction similarity to the current base model to obtain multiple remaining base models including the current base model; if the number of multiple remaining base models is greater than the preset number, then the base model with the largest prediction accuracy parameter among the multiple remaining base models is taken as the new current base model, and the base models other than the new current base model among the multiple remaining base models are taken as new other base models, and among the new other base models, the base model with the highest prediction similarity to the new current base model is deleted to obtain new remaining base models including the new current base model, until the number of new remaining base models is equal to the preset number.

[0018] In the selection of multiple base models, the current base model is selected sequentially based on its prediction accuracy parameter. Then, the base model most similar to the current base model is removed from the remaining base models until the required number of base models is reached. This method allows for the rapid and accurate removal of base models with low prediction accuracy that are also similar to other base models, improving the efficiency of base model optimization.

[0019] In some modified embodiments of the first aspect of this application, each base model can output multiple prediction results based on data corresponding to multiple known true results. The method further includes: if the prediction similarity between two base models is less than or equal to a similarity threshold, then determining the data quantity representation, category quantity representation, and / or data generation distance duration representation of the prediction differences between the multiple prediction results corresponding to the two base models respectively; based on the prediction accuracy parameter and the data quantity representation, category quantity representation, and / or data generation distance duration representation, deleting one or more base models from the multiple base models to obtain a preset number of target base models, so that business data is input into the target base models respectively. Among them, the base models with lower prediction accuracy parameters, smaller data quantity representation, smaller category quantity representation, and smaller data generation distance duration representation are deleted with priority. The priority of the prediction accuracy parameter is higher than the priority of the data quantity representation, category quantity representation, and data generation distance duration representation.

[0020] When the base models are not similar to each other, we can choose to delete the base models with fewer data points that predict differences, fewer categories, and shorter time intervals between data generation. This can ensure that the differences between the remaining base models are maximized, thereby improving the accuracy of subsequent predictions.

[0021] A second aspect of this application provides a prediction device based on stacking and machine learning models. The device includes: an acquisition module for acquiring business data and multiple base models, wherein the business data is data to be predicted, and the base models are used to make predictions based on the business data; a first processing module for inputting the business data into the multiple base models respectively to obtain multiple prediction results corresponding to the outputs of the multiple base models; a second processing module for performing statistical processing on the multiple prediction results to obtain statistical results; and a prediction module for inputting the multiple prediction results and statistical results into a meta-model to obtain the final prediction result output by the meta-model. The meta-model is used to make predictions based on the outputs of the multiple base models. The meta-model is trained based on an initial meta-model using training data, wherein the training data includes the training outputs of the multiple base models and the statistical results of the training outputs.

[0022] A third aspect of this application provides an electronic device, which includes a processor, a memory, and a bus. The processor and the memory communicate with each other via the bus. The processor is used to call program instructions in the memory to execute the method in the first aspect.

[0023] A fourth aspect of this application provides a computer-readable storage medium including a stored program, wherein the program, when executed, controls the device on which the computer-readable storage medium is located to perform the method of the first aspect.

[0024] The fifth aspect of this application provides a computer program product, which includes a computer program or instructions that, when executed by a device, implement the method of the first aspect.

[0025] The prediction device based on stacking and machine learning models provided in the second aspect of this application, the electronic device provided in the third aspect, the computer-readable storage medium provided in the fourth aspect, and the computer program product provided in the fifth aspect have the same or similar beneficial effects as the prediction method based on stacking and machine learning models provided in the first aspect. Attached Figure Description

[0026] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this application are illustrated by way of example and not limitation, with the same or corresponding reference numerals denoteing the same or corresponding parts, wherein:

[0027] Figure 1 This is a schematic diagram of the scenario architecture of the prediction method based on stacking and machine learning models in the embodiments of this application;

[0028] Figure 2This is a flowchart illustrating the prediction method based on stacking and machine learning models in the embodiments of this application. Figure One ;

[0029] Figure 3 This is a schematic diagram of the overall architecture of the prediction method based on stacking and machine learning models in the embodiments of this application;

[0030] Figure 4 This is a flowchart illustrating the prediction method based on stacking and machine learning models in the embodiments of this application. Figure Two ;

[0031] Figure 5 This is a flowchart illustrating the prediction method based on stacking and machine learning models in the embodiments of this application. Figure Three ;

[0032] Figure 6 This is a schematic diagram of the structure of the prediction device based on stacking and machine learning models in the embodiments of this application. Figure One ;

[0033] Figure 7 This is a schematic diagram of the structure of the prediction device based on stacking and machine learning models in the embodiments of this application. Figure Two ;

[0034] Figure 8 This is a schematic diagram of the structure of the electronic device in the embodiments of this application. Detailed Implementation

[0035] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0036] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0037] Currently, the accuracy of predictions using stacking techniques is limited by the accuracy of the predictions of each base model and the limitations of the logistic regression algorithm in the meta-model, which prevents a significant improvement in prediction accuracy.

[0038] In view of this, embodiments of this application provide a prediction method, a prediction device, an electronic device, a computer-readable storage medium, and a computer program product based on stacking and machine learning models. After each base model outputs prediction results, the multiple prediction results are first statistically processed, and then the statistical results and each prediction result are input into a meta-model for prediction. This increases the dimensionality of the input data in the meta-model, and inputting statistical results into the meta-model increases the diversity of data processing methods the meta-model can handle. Overall, this significantly improves the accuracy of prediction.

[0039] First, the application scenarios of the prediction method based on stacking and machine learning models provided in the embodiments of this application will be described.

[0040] Figure 1 This is a schematic diagram of the scenario architecture for the prediction method based on stacking and machine learning models in the embodiments of this application. See also Figure 1 As shown, the architecture can include: multiple base models and a meta-model.

[0041] The base model is capable of making predictions based on the input and outputting the prediction results.

[0042] Meta-models can make decisions based on the outputs of multiple base models, and then output the decision results.

[0043] In practical applications, both the base model and the metamodel are ML models, specifically AutoML models. The specific types of the base model and metamodel are not limited here.

[0044] When business data needs to be predicted, the business data is input into each base model. Each base model makes a prediction and outputs a prediction result. Then, the prediction results of multiple base models are statistically analyzed to obtain a statistical result. Next, the prediction results of each base model and the statistical result are input into the meta-model, so that the meta-model makes a decision based on multiple prediction results and statistical results. Finally, the output of the meta-model is the final prediction result of the business data.

[0045] It should be noted that all models, data, and processing procedures involved in the embodiments of this application have been authorized in advance and are legal and compliant.

[0046] Next, the prediction method based on stacking and machine learning models provided in the embodiments of this application will be described in detail.

[0047] Figure 2 This is a flowchart illustrating the prediction method based on stacking and machine learning models in the embodiments of this application. Figure One SeeFigure 2 As shown, the method may include:

[0048] S21: Obtain business data and multiple base models.

[0049] Business data refers to data that needs to be predicted. In other words, it requires reasoning and judgment based on the content of the business data. For example, if the business data is an image, the goal is to identify faces within the image.

[0050] The base model is used for prediction based on business data. Since the base model is generally an machine learning (ML) model, the base model here refers to the trained model.

[0051] The base model can be provided along with the user's business data, or a default configuration can be used. When using the default configuration, the base model is generally an untrained model. Users need to provide training data of the same type with known prediction results along with the business data. The default configuration model is first trained using the training data to obtain the base model before proceeding to subsequent steps.

[0052] S22: Input the business data into multiple base models respectively to obtain multiple prediction results corresponding to the output of multiple base models.

[0053] Since the base model is trained using training data similar to the business data, when the business data is input into the base model, the base model can make corresponding predictions on the business data and output prediction results.

[0054] Different base models, with varying network architectures or specific algorithms, may produce different prediction results for the same business data. Inputting business data into different base models will generate different prediction results from each model. The number of base models determines the number of prediction results obtained.

[0055] It should be noted that the base models used in the embodiments of this application are all known base models. The specific content and processing methods of the base models will not be elaborated here.

[0056] S23: Perform statistical processing on multiple prediction results to obtain statistical results.

[0057] The statistics mentioned here can refer to processing procedures not involved in the algorithm of the meta-model. For example, the meta-model may involve a logistic regression algorithm, which does not involve nonlinear calculations. Therefore, the statistics in this step can refer to various nonlinear calculations. As another example, the meta-model may involve a logistic regression algorithm, which is a regression calculation based on multiple results. Therefore, the statistics in this step can also refer to various non-regression calculations. Of course, any calculation that differs from logistic regression but is related to the accuracy of the prediction results can be used as the statistical method in this step; these will not be listed here.

[0058] S24: Input multiple prediction results and statistical results into the meta-model to obtain the final prediction result output by the meta-model.

[0059] Metamodels are used to make predictions based on the outputs of multiple base models. By inputting the prediction results from multiple base models and the statistical results of these predictions into the metamodel, it can make predictions based on these multiple preset results and statistical findings. Because the metamodel considers statistical results in its predictions, it not only increases the dimensionality of the input data but also enriches the metamodel's computational methods, thereby improving the accuracy of the final prediction results.

[0060] Previously, the meta-model only used simple algorithms such as logistic regression, which only processed the prediction results output by multiple base models. However, this time, the input of the meta-model has added statistical results, and the dimensionality of the data input has increased. Therefore, the meta-model needs to be trained in advance based on the original data and the new data before it can be put into use.

[0061] In the specific training process, an initial meta-model is first constructed. Then, the training data is input into multiple base models, and the prediction results output by the multiple base models are statistically analyzed to obtain statistical results. Next, the statistical results, multiple prediction results, and the known true results of the training data are input into the initial meta-model for training, resulting in a usable meta-model. During formal prediction, the outputs of the business data in the multiple base models, along with the statistical results of each output, are input into the meta-model. The output of the meta-model is the final prediction result for the business data.

[0062] As described above, the prediction method based on stacking and machine learning models provided in this application involves statistically analyzing the prediction results of each base model after multiple base models output their predictions. The statistical results, along with the prediction results of each base model, are then input into a meta-model. This allows the meta-model to further analyze and process the predictions from multiple base models by combining the statistical results. Previously, the meta-model could only perform logistic regression calculations based on the predictions from multiple base models, limiting the dimensionality of the input data and its processing capabilities. This new method inputs not only the predictions from multiple base models but also the statistical results of those predictions, thus enhancing the dimensionality of the data processed and the processing capabilities of the meta-model. This, in turn, improves the accuracy of predictions made using machine learning models and stacking techniques.

[0063] Furthermore, as a response to Figure 2 In a refinement and extension of the method shown, this application also provides a prediction method based on stacking and machine learning models.

[0064] Figure 3 This is a schematic diagram of the overall architecture of the prediction method based on stacking and machine learning models in the embodiments of this application. See also Figure 3 As shown, the architecture can include three parts: input, optimization, and output.

[0065] At the input stage, users need to provide multiple base models, datasets (training set, validation set, and test set), and evaluation metrics.

[0066] Multiple base models can be used, including historical training models from any AutoML platform, or historical training models that the developers have already tried.

[0067] In the dataset, the training and validation sets are used to train the meta-model. The test set is the actual set of data to be predicted.

[0068] Evaluation metrics, also known as evaluation functions used by the meta-model, such as the logloss function.

[0069] The optimization section mainly includes: a model selection module, a feature construction module, and an AutoML metamodel optimization module.

[0070] The model selection module is used to select a certain number of models from multiple provided base models that have the best performance and significant differences between them. This number can be determined based on the actual situation, for example, 5. In practical applications, model selection can be performed using a loss function.

[0071] The feature construction module can further process multiple prediction results based on the prediction results of multiple base models after screening, and construct more new feature values.

[0072] The metamodel optimization module of AutoML mainly has two functions: one is to optimize the metamodel based on the training set and validation set, and the other is to use the optimized metamodel to make predictions based on the test set.

[0073] Multiple base models are input from the training and validation sets after filtering, resulting in multiple prediction results. These prediction results are then input into the feature construction module to obtain statistical results. Finally, the multiple prediction results, statistical results, and known true results from the training and validation sets are input into the AutoML meta-model optimization module to complete the meta-model optimization.

[0074] Multiple base models are input into the test set, resulting in multiple predictions. These predictions are then input into the feature construction module to obtain statistical results. Finally, the multiple predictions based on the test set and the statistical results are input into the optimized meta-model to obtain the final prediction result.

[0075] The output can not only show the final prediction results, but also a set of models and functions (i.e., the optimal stacking model's pipeline) including multiple base models after filtering, feature construction methods, and parameters in the optimized meta-model. This allows users to use the pipeline set to obtain the final prediction results for the test set.

[0076] Figure 4 This is a flowchart illustrating the prediction method based on stacking and machine learning models in the embodiments of this application. Figure Two See Figure 4 As shown, the method may include:

[0077] S41: Obtain business data and multiple base models.

[0078] When a user needs to make predictions (e.g., classification) about content in business data, the user can input the business data and multiple base models for prediction.

[0079] The business data here can include the actual data to be predicted, as well as data with known prediction results used for model training. In other words, business data can include a training set, a validation set, and a test set. The training and validation sets are the data used for model training with known prediction results. The test set is the data to be predicted.

[0080] Different base models have varying accuracy in predicting business data. Furthermore, among two base models with similar prediction results, the one with lower accuracy can negatively impact the final prediction accuracy. For example, consider an image where the goal is to identify the face of person 'a'. Base model 1 identifies face 1, while base model 2 identifies face 2. In reality, face 'a' is face 1. Base model 2 can also identify a face, specifically face 2, which is quite similar to face 'a'. If base model 2 is still used in the final prediction, it may lower the probability of face 2 being used as the final prediction result, thus reducing prediction accuracy. In this case, it's necessary to remove the base model with low prediction accuracy that is also similar to other base models.

[0081] S42: Determine the prediction accuracy parameter for each base model, and determine the prediction similarity between any two base models.

[0082] To determine the prediction accuracy parameter, data from the training and validation sets can be input into the base model. The base model outputs prediction results based on the data. The similarity between these prediction results and the known true results for the same data in the training and validation sets is calculated. The prediction accuracy parameter of the base model is then determined based on the similarity calculation result. The higher the calculated similarity, the greater the prediction accuracy parameter of the base model.

[0083] To improve the accuracy of the prediction accuracy parameters of the base model, the above similarity calculation can be performed on multiple data sets in the training and validation sets respectively, and the average value of the similarity calculation results can be taken to determine the prediction accuracy parameters of the corresponding base model.

[0084] Of course, to improve the efficiency of determining the prediction accuracy parameters of the base model, the above similarity calculation can be performed using a small amount of data from both the training and validation sets, and the average of the similarity calculation results can be taken to determine the prediction accuracy parameters of the corresponding base model. Alternatively, a specific set of data from both the training and validation sets can be used to perform the above similarity calculation, and the prediction accuracy parameters of the corresponding base model can be determined based on the similarity calculation results.

[0085] Determining the predictive similarity requires comparison between each pair of base models. First, one or more data sets can be selected from the training, validation, and test sets. These selected data are then input into each base model to obtain its prediction result. Next, the similarity between the corresponding prediction results of each pair of base models is calculated. This yields the predictive similarity between each pair of base models.

[0086] If you select one data point from the training, validation, and test sets, the prediction similarity between any two base models is the similarity between the predictions made by the base models based on that data. If you select multiple data points from the training, validation, and test sets, the prediction similarity between any two base models is the average of the similarities between the predictions made by one base model for each data point and the predictions made by another base model for each data point.

[0087] The number of data points to select from the training, validation, and test sets is similar to the determination of the prediction accuracy parameter; it needs to be determined comprehensively based on the required accuracy and efficiency.

[0088] S43: Based on the prediction accuracy parameter and prediction similarity, delete one or more base models from multiple base models to obtain a preset number of target base models.

[0089] In this selection process, base models with lower prediction accuracy parameters and higher prediction similarity are prioritized for deletion. Prediction accuracy has a higher priority than prediction similarity. In other words, given two similar base models with relatively high prediction accuracy parameters and one base model with a low prediction accuracy parameter, the base model with the lower prediction accuracy parameter will be deleted. This maximizes the retention of base models with significant prediction differences and reduces the undesirable interference from similar base models.

[0090] During the deletion process, base models can be sorted from highest to lowest prediction similarity, and the base model with the lower prediction accuracy parameter among two similar base models can be deleted sequentially. Alternatively, they can be sorted from highest to lowest prediction accuracy parameter, and the base models most similar to the current base model can be deleted sequentially. After deleting each base model, it can be checked whether the remaining base models meet the required number. If they do, deletion can stop; otherwise, deletion continues, achieving precise control over the number of base models selected. Alternatively, after deleting multiple base models, the remaining base models can be checked to improve the efficiency of model deletion. To maximize the retention of base models with significant prediction differences and reduce the adverse interference from similar base models, deletion can be performed by sorting by prediction accuracy parameter.

[0091] Specifically, step S43 above may include:

[0092] Step A1: Select the base model with the highest prediction accuracy parameter among multiple base models as the current base model, and select the base models other than the current base model among multiple base models as other base models.

[0093] Step A2: Among the other base models, delete the base model with the highest prediction similarity to the current base model, and obtain multiple remaining base models including the current base model.

[0094] Step A3: If the number of multiple remaining basis models is greater than the preset number, then the basis model with the highest prediction accuracy parameter among the multiple remaining basis models is taken as the new current basis model, and the basis models other than the new current basis model among the multiple remaining basis models are taken as new other basis models. Among the new other basis models, the basis model with the highest prediction similarity to the new current basis model is deleted, and new remaining basis models including the new current basis model are obtained, until the number of new remaining basis models is equal to the preset number.

[0095] For example, suppose there are *i* base models, and after filtering, the number of base models is 5. Each of these *i* base models has a prediction accuracy parameter. These *i* prediction accuracy parameters are sorted in descending order. The base model ranked first has a prediction similarity with the remaining *i-1* base models. Among these *i-1* prediction similarities, the base model with the highest prediction similarity is deleted. At this point, *i-1* base models remain. If the number of *i* is 10, there are still 9 base models remaining, which does not meet the standard of 5 remaining base models, so base models are deleted. Next, for the base model ranked second, it has a prediction similarity with the remaining *i-2* base models. Among these *i-2* prediction similarities, the base model with the highest prediction similarity is deleted. At this point, *i-2* base models remain. This process is repeated until the number of remaining *in* base models is 5. The number of base models after filtering, i.e., the preset number, can be determined according to the actual situation; no specific limit is made here.

[0096] If the predictions of pairwise base models are dissimilar (i.e., the similarity between the predictions of pairwise base models is less than or equal to the similarity threshold), deleting base models in the above manner would result in the deletion of base models with significant differences, thereby reducing the accuracy of subsequent meta-model integrated decisions. Therefore, alternative methods for deleting base models are needed.

[0097] Specifically, prior to step A2 above, the method may further include:

[0098] Step A01: If the prediction similarity between the two basis models is less than or equal to the similarity threshold, then determine the data quantity representation, category quantity representation, and / or data generation distance duration representation of the prediction differences between the multiple prediction results corresponding to the two basis models.

[0099] If the prediction similarity between any two base models is less than or equal to the similarity threshold, it means that the predictions of these models are not similar. Instead of deleting base models based on prediction accuracy, we can sort them by prediction similarity and delete the base models with the smallest prediction difference between the current base model and other base models in descending order.

[0100] If, during the process of deleting base models based on prediction accuracy, a current base model is found to have low prediction similarity to multiple other base models, with the highest similarity being below a similarity threshold, then the base model with the smallest prediction difference among these base models is deleted. Therefore, it is necessary to further determine the magnitude of the prediction difference between the multiple prediction results corresponding to each pair of base models, i.e., the representation of the number of data points with prediction differences, the number of categories, and / or the time elapsed since the data was generated.

[0101] The number of data points showing prediction discrepancies can refer to the number of predictions from one base model that differ from the predictions from another base model. This number can be represented either by the total number of data points showing prediction discrepancies or by the proportion of such data points among the total number of predictions.

[0102] The number of predicted discrepancies can refer to the number of categories in the multiple predictions of one base model that differ from the multiple predictions of another base model. The representation of the number of predicted discrepancies can be either the total number of predicted discrepancies or the percentage of such categories in the total number of predicted categories.

[0103] The so-called data generation distance duration of prediction discrepancies can refer to the maximum time difference among the generation times of samples (i.e., the data input to the base model) that differ from the prediction results of another base model among multiple prediction results of one base model. The data generation distance duration of prediction discrepancies can be represented as either the total data generation distance duration of prediction discrepancies or the proportion of the total data generation distance duration among all the maximum data generation distance durations.

[0104] For ease of calculation, the time elapsed since the data was generated can be a specific time. In the case where the data in the business data is sorted according to the time of generation, the data index can also be used instead of the data generation time.

[0105] The number of data points, the number of categories, and the time elapsed between the data generation and the predicted differences can be used individually or in combination; no specific restrictions are imposed here.

[0106] Step A02: Based on the prediction accuracy parameter and the data quantity representation, category quantity representation and / or data generation distance duration representation, delete one or more base models from multiple base models to obtain a preset number of target base models.

[0107] Among them, the lower the prediction accuracy parameter, the smaller the data quantity representation, the smaller the category quantity representation, and the smaller the data generation distance duration representation, the higher the priority of the base model to be deleted. The priority of the prediction accuracy parameter is higher than the priority of the data quantity representation, category quantity representation, and data generation distance duration representation.

[0108] In other words, multiple base models are sorted according to the prediction accuracy parameter. For the current base model, if the prediction similarity with other base models is less than the similarity threshold, then among the other base models, the base model with the smallest number of data with the smallest prediction difference, the smallest number of categories, or the shortest time between data generation and deletion is selected.

[0109] For example, suppose Table 1 below shows the classification prediction results of some data using base model 1 and base model 2 respectively.

[0110] Table 1 shows the classification results of some data using base model 1 and base model 2.

[0111]

[0112]

[0113] In Table 1, data indices 0-19 represent 20 data points. Base model 1 and base model 2 each perform classification predictions for each data point. 0, 1, 2, and 3 identify the four different predicted categories.

[0114] If base model 1 is used as the current base model, and the prediction similarity between base model 1 and base model 2 is not high, and the prediction similarity between base model 1 and base model 3 is also not high, then the base model to be deleted from base model 2 and base model 3 will no longer be based on prediction similarity. Instead, the base model to be deleted will be selected from base model 2 and base model 3 based on the number of data points with prediction differences, the number of categories, and / or the time elapsed since the data was generated.

[0115] Compared to base model 1, base model 2 has 4 data points with different class classifications. Compared to base model 1, base model 3 has 5 data points with different class classifications. Based on the number of data points with different predictions, base model 2, which has fewer data points with different predictions, is selected for deletion.

[0116] Compared to base model 1, base model 2 has four data points with different class classifications: class 1 and 3 are classified as class 2, and class 2 and 1 are classified as class 0, resulting in a total of two class differences. Compared to base model 1, base model 3 has five data points with different class classifications: class 1 is classified as class 2, class 3 is classified as class 1, class 2 is classified as class 0, and class 1 is classified as class 0, resulting in a total of three class differences. Based on the number of class differences, base model 2, with fewer class differences, is selected for deletion.

[0117] Compared to base model 1, base model 2 has four data points with different category classifications, and the largest time difference among these four data points is 10 days. Compared to base model 1, base model 3 has five data points with different category classifications, and the largest time difference among these five data points is 2 days. Based on the time interval between the occurrence of the predicted differences, base model 3, which has the shortest time interval between the occurrence of the predicted differences, is deleted.

[0118] If we select a base model to delete based on the number of data points predicting discrepancies, the number of categories, or the time elapsed since the data generation, we can directly choose one from the two base models. However, if we select different base models based on these factors, it becomes difficult to choose one to delete. In this case, we can express the number of data points predicting discrepancies, the number of categories, and the time elapsed since the data generation as a function, and then determine the base model to delete based on the function's calculation result.

[0119] The specific functions are as follows:

[0120] F(x) = Accuracy + Differential Sample Distribution Coefficient × Differential Category Coefficient (Formula 1)

[0121] Here, F(x) is used to characterize the similarity.

[0122] Accuracy is used to characterize the number of data points where predictions differ. Accuracy = Number of different predictions / Total number of predictions.

[0123] The variance distribution coefficient is used to characterize the time elapsed since the occurrence of data that caused the prediction discrepancies. Variance distribution coefficient = (Index of inconsistent samples / Total number of records)

[0124] The difference category coefficient is used to characterize the number of predicted difference categories. Difference category coefficient = Total number of predicted different categories / Total number of predicted categories.

[0125] The smaller the value of F(x), the higher the probability that the corresponding base model will be deleted. For the current base model, each other base model has a corresponding F(x). From these F(x), the base model corresponding to the F(x) with the smallest value is selected for deletion, thus completing the deletion of the most similar base model to the current base model. Deletion continues until the number of remaining base models reaches a preset number after multiple deletions, at which point the deletion stops, and the target base model is obtained.

[0126] S44: Input the business data into the target base model respectively to obtain multiple prediction results corresponding to the output of multiple base models.

[0127] Generally, there are multiple target base models. Business data is input into each base model, and each base model outputs a prediction result. This results in multiple prediction results for the business data under different base models.

[0128] S45: Based on each prediction result among multiple prediction results, determine the physical metric value of the base model corresponding to each prediction result, obtain multiple physical metric values, and select the best among the multiple physical metric values ​​to obtain the optimal metric value.

[0129] The prediction results can also be used to characterize the predictions of the base model. That is, the prediction results are converted into physical metrics of the base model, and then the best physical metric is selected from various physical metrics. The specific content of the physical metrics can be determined according to the actual prediction needs. Physical metrics may include, but are not limited to, confidence level, precision, etc.

[0130] The confidence level of the base model can be determined by utilizing its characteristic: the model outputs both the prediction result and the confidence level. Therefore, the confidence level of the corresponding base model can be found in the prediction result data packet, and the highest confidence level found can be determined as one of the statistical results.

[0131] The specific value of the confidence score directly reflects the degree of "lack of confidence" of the base models in this data. The specific value of the maximum confidence score reflects the degree of "lack of confidence" among all base models in this data. In multi-class classification scenarios, for hard examples, the maximum confidence score is often only slightly above 0.5, and cannot reach above 0.98.

[0132] S46: Perform nonlinear processing on at least two of the multiple prediction results to obtain a nonlinear result.

[0133] Any processing method other than linear processing can be considered nonlinear processing.

[0134] In nonlinear processing, all multiple prediction results can be processed, or some prediction results can be processed first, and then the processed prediction results can be combined.

[0135] Nonlinear processing may include at least one of the following: calculating the mode, calculating the variance, calculating the mean, calculating the difference between pairs of numbers, calculating the sum of pairs of numbers, and the optimal model.

[0136] The mode, in this context, can refer to the prediction with the highest number of identical predictions among multiple predictions, reflecting the greatest probability of judgments from multiple "differentiated" base models.

[0137] Variance, here, can refer to the variance of multiple prediction results. The larger the variance, the more difficult it is to judge the business data.

[0138] The average value here can refer to the average of multiple prediction results, or it can reflect the highest probability of judgments from multiple "differentiated" base models.

[0139] The difference between two base models can refer to the difference in prediction results between two base models.

[0140] The sum of pairs can refer to the sum of the predictions from two base models.

[0141] The optimal model, or pseudo-optimal model index, is the index of the model with the highest confidence among the correctly predicted models during the training phase. During the testing phase, the base model with the highest confidence among the base models whose predictions represent the mode is used.

[0142] It should be noted that steps S45 and S46 can be executed simultaneously to improve prediction efficiency. Alternatively, steps S45 and S46 can be executed asynchronously to avoid mutual interference between steps and improve prediction accuracy.

[0143] S47: Determine the physical metrics and nonlinear results of each base model as statistical results.

[0144] The above statistical results can, in addition to the prediction results of each base model, more fully characterize the relationships between the various base models and the relationships between the various prediction results, thereby improving the accuracy of prediction.

[0145] It should be noted that the business data processed in step S44 above can include training set, validation set, and test set. The results corresponding to the data in the training set and validation set are known, while the results corresponding to the data in the test set are unknown, and this is the data that actually needs to be predicted. In other words, step S44 above requires using the target base model to predict each data point in the training set, validation set, and test set, and each data point corresponds to multiple prediction results equal to the number of predictions from the target base model.

[0146] Correspondingly, in step S47 above, each data point in the training set, validation set, and test set has a corresponding statistical result.

[0147] Since the meta-model makes decisions based on multiple prediction results as before, the decision-making process now includes statistical results of multiple prediction results, increasing the data dimensions processed by the meta-model. In order to enable the meta-model to make accurate decisions, before inputting multiple prediction results and statistical results corresponding to the test set into the meta-model for prediction, the meta-model is first trained using multiple prediction results and statistical results of each data in the training set and validation set, as well as the known results of each data.

[0148] The metamodel can include multiple pre-defined algorithms. Users can choose these pre-defined algorithms according to their actual needs. Alternatively, if the user does not specify any, the metamodel will directly select the default algorithm from the pre-defined algorithms.

[0149] In practical applications, the pre-set algorithm can be a multi-classification algorithm such as logistic regression algorithm or Bayesian classification algorithm.

[0150] S48: Input multiple prediction results, statistical results and specified algorithm names into the meta-model so that the meta-model can find the target algorithm corresponding to the specified algorithm name among multiple preset algorithms, and then process multiple prediction results and statistical results based on the target algorithm.

[0151] After receiving multiple prediction results, statistical results, and a specified algorithm name, the meta-model first searches for the corresponding algorithm (the target algorithm) among several pre-set algorithms based on the specified algorithm name. If no unique algorithm is found precisely based on the specified algorithm name, then the model searches for the most similar algorithm among several pre-set algorithms using the specified algorithm name to ensure successful prediction.

[0152] After determining the target algorithm, the meta-model is first trained using multiple prediction results, statistical results, and known results from the training and validation sets of business data. This process optimizes the parameters of the target algorithm, resulting in a trained meta-model. Then, data from the test set of business data is input into the trained meta-model. Through processing by the optimized target algorithm, the prediction results output by the trained meta-model become the more accurate final prediction results.

[0153] Finally, a specific example will be used to further illustrate the prediction method based on stacking and machine learning models provided in the embodiments of this application.

[0154] Figure 5 This is a flowchart illustrating the prediction method based on stacking and machine learning models in the embodiments of this application. Figure Three See Figure 5 As shown, when a user needs to make predictions on certain data, that data is used as the test set, and data with known results in the same category are used as the training and validation sets. These training, validation, and test sets are then input into the system as business data. Simultaneously, multiple base models are also input into the system. These base models can include: Trained models of SVM, MLP, LightBGM, ..., KNN, LSTM, etc.

[0155] The business data and these base models first enter the system's model selection module. Based on the business data, the model selection module calculates the prediction accuracy parameters and prediction similarity of these base models, and selects the TOP5 best models from these base models based on the prediction accuracy parameters and prediction similarity parameters, namely, the trained model of SVM, MLP, LightBGM, KNN and LSTM.

[0156] Once the top 5 optimal models are determined, the business data can be input into the top 5 optimal models respectively. The predicted class output by the trained model of SVM is 1, ... the predicted class output by MLP is 2, ... the predicted class output by LightBGM is 2, ... the predicted class output by KNN is 0, ... the predicted class output by LSTM is 2. At this time, a 5-dimensional array is obtained, namely [1,2,2,0,2].

[0157] Next, the predicted categories corresponding to the five optimal models are input into the system's feature construction module. Based on these predicted categories, the feature construction module generates the maximum confidence score, mode, variance, mean, pseudo-optimal model index, pairwise difference, and pairwise sum of the predicted categories for each of the five optimal models. The predicted categories corresponding to the five optimal models are considered original features. The maximum confidence score, mode, variance, mean, pseudo-optimal model index, pairwise difference, and pairwise sum of the predicted categories for each of the five optimal models are considered newly added features.

[0158] The original and newly added features are input into the AutoML metamodel optimization module of the system. The AutoML metamodel optimization module includes a search space and a search process. The search space can include Naive Bayes search space and logistic regression search space, etc. The search process can include evolutionary algorithm search strategies and early stopping strategies, etc. The Naive Bayes search space involves Bayesian kernels. Bayesian kernels can include GaussianNB, MultinomialNB, BernouliNB, etc. The logistic regression search space involves logistic regression parameters. Logistic regression parameters can include regularization type, regularization strength, maximum number of iterations, number of iterations, classification method, etc. After optimizing the parameters involved in the AutoML metamodel using the original and newly added features corresponding to the training and validation sets, the optimized AutoML metamodel is then used to process the original and newly added features corresponding to the test set. The output of the AutoML metamodel optimization module is the final prediction result.

[0159] This concludes the description of the prediction methods based on stacking and machine learning models provided in the embodiments of this application.

[0160] Based on the same inventive concept, as an implementation of the above method, this application also provides a prediction device based on stacking and machine learning models.

[0161] Figure 6 This is a schematic diagram of the structure of the prediction device based on stacking and machine learning models in the embodiments of this application. Figure One See Figure 6 As shown, the device may include: an acquisition module 61, a first processing module 62, a second processing module 63, and a prediction module 64.

[0162] The acquisition module 61 is used to acquire business data and multiple base models. The business data is the data to be predicted, and the base models are used to make predictions based on the business data.

[0163] The first processing module 62 is used to input business data into multiple base models respectively, and obtain multiple prediction results corresponding to the output of multiple base models.

[0164] The second processing module 63 is used to perform statistical processing on multiple prediction results to obtain statistical results.

[0165] The prediction module 64 is used to input multiple prediction results and statistical results into the meta-model to obtain the final prediction result output by the meta-model. The meta-model is used to make predictions based on the outputs of multiple base models. The meta-model is trained on the initial meta-model using training data, which includes the training outputs of multiple base models and the statistical results of the training outputs.

[0166] Furthermore, as a response to Figure 6 In addition to the refinement and expansion of the illustrated device, this application embodiment also provides a prediction device based on stacking and machine learning models.

[0167] Figure 7 This is a schematic diagram of the structure of the prediction device based on stacking and machine learning models in the embodiments of this application. Figure Two See Figure 7 As shown, the device may include:

[0168] The acquisition module 71 is used to acquire business data and multiple base models. The business data is the data to be predicted, and the base models are used to make predictions based on the business data.

[0169] The filtering module 72 includes: a determining unit 721 and a deleting unit 722.

[0170] The determination unit 721 is used to determine the prediction accuracy parameter of each base model and the prediction similarity between pairs of base models.

[0171] The deletion unit 722 is used to delete one or more base models from multiple base models based on the prediction accuracy parameter and prediction similarity to obtain a preset number of target base models, so that business data can be input into the target base models respectively. Among them, the base models with lower prediction accuracy parameters and higher prediction similarity are deleted with priority. The priority of prediction accuracy parameters is higher than the priority of prediction similarity.

[0172] The deletion unit 722 is specifically used to take the base model with the largest prediction accuracy parameter among multiple base models as the current base model, and to take the base models other than the current base model among multiple base models as other base models; among the other base models, delete the base model with the highest prediction similarity to the current base model, and obtain multiple remaining base models including the current base model; if the number of multiple remaining base models is greater than a preset number, then continue to take the base model with the largest prediction accuracy parameter among multiple remaining base models as the new current base model, and take the base models other than the new current base model among multiple remaining base models as new other base models, and delete the base model with the highest prediction similarity to the new current base model among the new other base models, and obtain new remaining base models including the new current base model, until the number of new remaining base models is equal to the preset number.

[0173] The deletion unit 722 is further configured to, if the prediction similarity between two base models is less than or equal to a similarity threshold, determine the data quantity representation, category quantity representation, and / or data generation distance duration representation of the prediction differences between multiple prediction results corresponding to the two base models respectively; based on the prediction accuracy parameter and the data quantity representation, category quantity representation, and / or data generation distance duration representation, delete one or more base models from the multiple base models to obtain a preset number of target base models, so that business data can be input into the target base models respectively. Among them, the base models with lower prediction accuracy parameter, smaller data quantity representation, smaller category quantity representation, and smaller data generation distance duration representation are deleted with priority. The priority of the prediction accuracy parameter is higher than the priority of the data quantity representation, category quantity representation, and data generation distance duration representation.

[0174] The first processing module 73 is used to input business data into multiple base models respectively, and obtain multiple prediction results corresponding to the output of multiple base models.

[0175] The second processing module 74 includes: a first processing unit 741, a second processing unit 742, and a collection unit 743.

[0176] The first processing unit 741 is used to determine the physical metric value of the base model corresponding to each prediction result based on each prediction result among multiple prediction results, obtain multiple physical metric values, and perform optimization processing among multiple physical metric values ​​to obtain the optimal metric value.

[0177] The second processing unit 742 is used to perform nonlinear processing on at least two of the multiple prediction results to obtain a nonlinear result.

[0178] Set unit 743 is used to determine the optimal metric and nonlinear results as statistical results.

[0179] The physical metrics include confidence levels. Nonlinear processing includes at least one of the following: calculating the mode, calculating the variance, calculating the mean, calculating pairwise differences, calculating pairwise sums, and determining the optimal model.

[0180] When the meta-model includes multiple pre-set algorithms, the prediction module 75 is used to input multiple prediction results, statistical results and specified algorithm names into the meta-model so that the meta-model can find the target algorithm corresponding to the specified algorithm name among the multiple pre-set algorithms, and then process the multiple prediction results and statistical results based on the target algorithm.

[0181] It should be noted that the description of the above device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0182] Based on the same inventive concept, embodiments of this application also provide an electronic device.

[0183] Figure 8 This is a schematic diagram of the electronic device in an embodiment of this application. See also... Figure 8 As shown, the electronic device may include: a processor 81, a memory 82, and a bus 83. The processor 81 and the memory 82 communicate with each other through the bus 83. The processor 81 is used to call program instructions in the memory 82 to execute the methods in one or more of the above embodiments.

[0184] It should be noted that the descriptions of the above electronic device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the electronic device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0185] Based on the same inventive concept, embodiments of this application also provide a computer-readable storage medium, which may include: a stored program that, when the program is running, controls the device where the storage medium is located to execute the methods described in one or more of the above embodiments.

[0186] It should be noted that the description of the above computer-readable storage medium embodiments is similar to the description of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the computer-readable storage medium embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0187] Based on the same inventive concept, this application also provides a computer program product, which includes a computer program or instructions that, when executed by the device, implement the methods in one or more of the above embodiments.

[0188] It should be noted that the descriptions of the above computer program product embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the computer program product embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0189] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A prediction method based on stacking and machine learning models, characterized in that, The method includes: Acquire business data and multiple base models, wherein the business data is data to be predicted, and the base models are used to make predictions based on the business data, wherein the business data is an image, and the prediction is to identify a face in the image; The business data is input into the multiple base models respectively to obtain multiple prediction results output by the multiple base models; The multiple prediction results are statistically processed to obtain statistical results; The multiple prediction results and the statistical results are input into the meta-model to obtain the final prediction result output by the meta-model. The meta-model is used to make predictions based on the outputs of the multiple base models. The meta-model is trained based on the initial meta-model using training data, and the training data includes the training outputs of the multiple base models and the statistical results of the training outputs. The step of statistically processing the multiple prediction results to obtain statistical results includes: Based on each of the multiple prediction results, the physical metric value of the base model corresponding to each prediction result is determined to obtain multiple physical metric values. The optimal metric value is obtained by selecting the best among the multiple physical metric values. At least two of the multiple prediction results are subjected to nonlinear processing to obtain a nonlinear result; The optimal metric and the nonlinear result are determined as the statistical result; The physical metric includes confidence level, and the nonlinear processing includes at least one of the following: calculating the mode, calculating the variance, calculating the mean, calculating the difference between pairs of numbers, calculating the sum of pairs of numbers, and calculating the optimal model.

2. The method according to claim 1, characterized in that, The meta-model includes multiple pre-defined algorithms; inputting the multiple prediction results and the statistical results into the meta-model includes: The multiple prediction results, the statistical results, and the specified algorithm name are input into the meta-model so that the meta-model can find the target algorithm corresponding to the specified algorithm name among the multiple preset algorithms, and then process the multiple prediction results and the statistical results based on the target algorithm.

3. The method according to any one of claims 1 to 2, characterized in that, Before inputting the business data into the multiple base models respectively, the method further includes: Determine the prediction accuracy parameter for each base model, and determine the prediction similarity between any two base models; Based on the prediction accuracy parameter and the prediction similarity, one or more base models from the plurality of base models are deleted to obtain a preset number of target base models, so that the business data is input into the target base models respectively. Among them, the base models with lower prediction accuracy parameters and higher prediction similarity are deleted with priority. The priority of prediction accuracy parameters is higher than the priority of prediction similarity.

4. The method according to claim 3, characterized in that, The step of deleting one or more base models from the plurality of base models based on the prediction accuracy parameter and the similarity to obtain a preset number of target base models includes: The base model with the highest prediction accuracy parameter among the multiple base models is selected as the current base model, and the base models other than the current base model among the multiple base models are selected as other base models. Among the other base models, the base model with the highest prediction similarity to the current base model is deleted, resulting in multiple remaining base models including the current base model; If the number of the plurality of remaining basis models is greater than the preset number, then the basis model with the highest prediction accuracy parameter among the plurality of remaining basis models is taken as the new current basis model, and the basis models other than the new current basis model among the plurality of remaining basis models are taken as new other basis models. Among the new other basis models, the basis model with the highest prediction similarity to the new current basis model is deleted, and new remaining basis models including the new current basis model are obtained, until the number of new remaining basis models is equal to the preset number.

5. The method according to claim 3, characterized in that, The method further includes: If the prediction similarity between the two basis models is less than or equal to the similarity threshold, then the data quantity representation, category quantity representation, and / or data generation distance duration representation of the prediction differences between the multiple prediction results corresponding to the two basis models are determined. Based on the prediction accuracy parameter and the data quantity representation, category quantity representation, and / or data generation distance duration representation, one or more base models among the multiple base models are deleted to obtain a preset number of target base models, so that the business data is respectively input into the target base models. Among them, the base models with lower prediction accuracy parameter, smaller data quantity representation, smaller category quantity representation, and smaller data generation distance duration representation are deleted with priority. The priority of prediction accuracy parameter is higher than the priority of data quantity representation, category quantity representation, and data generation distance duration representation.

6. A prediction device based on stacking and machine learning models, characterized in that, The device includes: An acquisition module is used to acquire business data and multiple base models. The business data is data to be predicted, and the base models are used to make predictions based on the business data. The business data is an image, and the prediction is to identify a face in the image. The first processing module is used to input the business data into the multiple base models respectively, and obtain multiple prediction results corresponding to the output of the multiple base models; The second processing module is used to perform statistical processing on the multiple prediction results to obtain statistical results; The prediction module is used to input the multiple prediction results and the statistical results into the meta-model to obtain the final prediction result output by the meta-model. The meta-model is used to make predictions based on the outputs of the multiple base models. The meta-model is trained on the initial meta-model using training data. The training data includes the training outputs of the multiple base models and the statistical results of the training outputs. The second processing module includes: a first processing unit, a second processing unit, and a collection unit; The first processing unit is used to determine the physical metric value of the base model corresponding to each prediction result based on each prediction result among multiple prediction results, obtain multiple physical metric values, and perform optimization processing among multiple physical metric values ​​to obtain the optimal metric value; The second processing unit is used to perform nonlinear processing on at least two of the multiple prediction results to obtain a nonlinear result. A set of units is used to determine the optimal metric and nonlinear results as statistical results; The physical metrics include confidence levels. Nonlinear processing includes at least one of the following: calculating the mode, calculating the variance, calculating the mean, calculating pairwise differences, calculating pairwise sums, and determining the optimal model.

7. An electronic device, characterized in that, The electronic device includes a processor, a memory, and a bus. The processor and the memory communicate with each other via the bus. The processor is used to call program instructions in the memory to execute the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program that, when executed, controls the device on which the computer-readable storage medium is located to perform the method as described in any one of claims 1 to 5.

9. A computer program product, characterized in that, The computer program product includes a computer program or instructions that, when executed by the device, implement the method as described in any one of claims 1 to 5.

Images