Result prediction model training method and device, equipment, medium and program product

Abstract

The invention discloses a result prediction model training method and device, equipment, a medium and a program product, and relates to the field of artificial intelligence. The method comprises the following steps: acquiring first sample data; performing category prediction on the first sample data for many times based on the domain label to obtain a plurality of category prediction results corresponding to the first sample data; performing data integration on the plurality of category prediction results based on the prediction categories corresponding to the plurality of category prediction results and the confidence score to obtain a reliability score corresponding to the first sample data; and in response to the fact that the reliability score corresponding to the first sample data meets a reliability condition, training a sample prediction model through the first sample data to obtain the result prediction model. Namely, the first sample data is screened according to the data accuracy of the first sample data and then used for model training, and the prediction accuracy of the result prediction model is improved.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of artificial intelligence, and particularly to a training method, device, equipment, medium, and program product for a result prediction model. Background Art

[0002] In the process of training a result prediction model, it is necessary to obtain sample training data for training the result prediction model.

[0003] In the related art, data authorized on a public platform is usually collected as sample data to train the result prediction model.

[0004] However, in the related art, since the data on the public platform usually contains a large amount of noisy data, using it for model training will result in a low prediction accuracy of the trained result prediction model. Summary of the Invention

[0005] Embodiments of the present application provide a training method, device, equipment, medium, and program product for a result prediction model, which can improve the training efficiency and prediction accuracy of the result prediction model. The technical solution is as follows:

[0006] On the one hand, a training method for a result prediction model is provided. The method includes:

[0007] Obtain first sample data, where the first sample data corresponds to a domain label, and the domain label is used to indicate a first domain category corresponding to the first sample data;

[0008] Based on the domain label, perform multiple category predictions on the first sample data to obtain multiple category prediction results corresponding to the first sample data. The category prediction results include a predicted category and a confidence score, and the confidence score is used to indicate the probability value that the first sample data belongs to the predicted category;

[0009] Based on the predicted categories and the confidence scores respectively corresponding to the multiple category prediction results, perform data integration on the multiple category prediction results to obtain a reliability score corresponding to the first sample data. The reliability score is used to indicate the accuracy of the first sample data belonging to the predicted category;

[0010] In response to the reliability score corresponding to the first sample data meeting the reliability condition, train a sample prediction model with the first sample data to obtain the result prediction model. The result prediction model is used to predict input data corresponding to multiple domain categories respectively to obtain prediction results corresponding to the input data, and the multiple domain categories include the first domain category.

[0011] On the other hand, a training device for a result prediction model is provided. The device includes:

[0012] An acquisition module, configured to acquire first sample data, where the first sample data corresponds to a domain label, and the domain label is used to indicate a first domain category corresponding to the first sample data;

[0013] A prediction module, configured to perform multiple category predictions on the first sample data based on the domain label to obtain multiple category prediction results corresponding to the first sample data. The category prediction results include a predicted category and a confidence score, and the confidence score is used to indicate a probability value that the first sample data belongs to the predicted category;

[0014] An integration module, configured to perform data integration on the multiple category prediction results based on the predicted categories and the confidence scores respectively corresponding to the multiple category prediction results to obtain a reliability score corresponding to the first sample data. The reliability score is used to indicate an accuracy situation of the first sample data belonging to the predicted category;

[0015] A training module, configured to, in response to the reliability score corresponding to the first sample data meeting the reliability condition, train a sample prediction model with the first sample data to obtain the result prediction model. The result prediction model is used to perform predictions on input data corresponding to multiple domain categories respectively to obtain prediction results corresponding to the input data, and the multiple domain categories include the first domain category.

[0016] On the other hand, a computer device is provided. The computer device includes a processor and a memory. At least one instruction, at least one program, a code set, or an instruction set is stored in the memory. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the training method of the result prediction model according to any one of the embodiments of the present application as described above.

[0017] On the other hand, a computer-readable storage medium is provided. At least one instruction, at least one program, a code set, or an instruction set is stored in the storage medium. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the training method of the result prediction model according to any one of the embodiments of the present application as described above.

[0018] On the other hand, a computer program product or a computer program is provided. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the training method of the result prediction model described in any one of the above embodiments.

[0019] The beneficial effects brought by the technical solutions provided in the embodiments of the present application at least include:

[0020] After obtaining the first sample data labeled with domain labels, perform multiple category predictions on the first sample data according to the domain labels, so as to obtain multiple category prediction results corresponding to the first sample data, and perform data integration on the multiple category prediction results according to the predicted categories and confidence levels corresponding to the category prediction results, so as to obtain the accuracy of the first sample belonging to the predicted category, which is used as its reliability score. Finally, the first sample data whose reliability score meets the reliability conditions is used to train the sample prediction model, and a result prediction model capable of predicting input data of multiple domain categories is obtained. That is, after obtaining the first sample data, by performing category prediction on it, the first sample data is screened according to the data accuracy of the first sample data, so that the sample data corresponding to the predicted category is used for model training, which can not only reduce the number of training data and improve the accuracy of model training, but also remove noise data and improve the prediction accuracy of the result prediction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0022] Figure 1 is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application;

[0023] Figure 2 is a flowchart of a method for training a result prediction model provided by an exemplary embodiment of the present application;

[0024] Figure 3 is a flowchart of a method for training a result prediction model provided by an exemplary embodiment of the present application;

[0025] Figure 4 is a flowchart of a method for training a result prediction model provided by an exemplary embodiment of the present application;

[0026] Figure 5 It is a schematic diagram of a method for training a result prediction model provided by an exemplary embodiment of the present application;

[0027] Figure 6 It is a structural block diagram of a training device for a result prediction model provided by an exemplary embodiment of the present application;

[0028] Figure 7 It is a structural block diagram of a computer device provided by an exemplary embodiment of the present application. Detailed implementation manners

[0029] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0030] Here, the exemplary embodiments will be described in detail, and their examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0031] The terms used in the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The singular forms "a", "the", and "said" used in the present application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0032] It should be noted that the information and data involved in the present application (including but not limited to the first sample data, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0033] It should be understood that although the terms first, second, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first parameter may also be referred to as the second parameter, and similarly, the second parameter may also be referred to as the first parameter. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0034] First, a brief introduction to the nouns involved in the embodiments of the present application:

[0035] Artificial Intelligence (AI): It is a new technical science that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. Artificial intelligence is an important part of the intelligent discipline. It attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a way similar to human intelligence. The research in this field includes robots, speech recognition, image recognition, natural language processing, and expert systems, etc. Since its birth, the theory and technology of artificial intelligence have become increasingly mature, and the application fields have also been continuously expanding. It can be imagined that the future scientific and technological products brought by artificial intelligence will be the "containers" of human wisdom. Artificial intelligence can simulate the information process of human consciousness and thinking. Artificial intelligence is not human intelligence, but it can think like a human and may even exceed human intelligence.

[0036] Machine Learning (ML): It is an interdisciplinary subject that covers knowledge of probability theory, statistics, approximation theory, and complex algorithms. It uses a computer as a tool and is committed to simulating the human learning method in real time and effectively improving the learning efficiency by dividing the existing content into knowledge structures.

[0037] First, the implementation environment involved in the embodiments of this application is described. For illustration, please refer to Figure 1 , in this implementation environment, there are a server 120, a communication network 140, and a terminal 100. Among them, the terminal 100 and the server 120 are connected through the communication network 140.

[0038] The terminal 100 sends first sample data to the server 120 through the communication network 140. The first sample data corresponds to a domain label corresponding to a first domain category.

[0039] After receiving the first sample data, the server 120 performs multiple category predictions on the first sample data based on the domain label, and obtains multiple category prediction results corresponding to the first sample data. Among them, the category prediction results include a predicted category and a confidence score. The confidence score is used to indicate the probability value that the first sample data belongs to the predicted category.

[0040] The server 120 integrates the multiple category prediction results based on the predicted categories and confidence scores respectively corresponding to the multiple category prediction results, and obtains a reliability score corresponding to the first sample data. The reliability score represents the accuracy of the first sample data belonging to the predicted category. When the reliability score corresponding to the first sample data meets the reliability condition, the sample prediction model is trained with the first sample data, and finally a result prediction model that can be used to predict the input data corresponding to multiple domain categories is obtained.

[0041] Among them, after the terminal 100 sends input data to the server 120, the server 120 predicts the input data through the result prediction model, obtains the prediction result, and feeds back the prediction result to the terminal 100.

[0042] It should be noted that the above-mentioned terminal can be various forms of terminal devices such as mobile phones, tablet computers, desktop computers, portable laptops, smart TVs, vehicle-mounted terminals, and smart home devices, and the embodiments of the present application do not limit this.

[0043] It should be noted that the above-mentioned server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0044] Among them, cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to achieve data calculation, storage, processing, and sharing. Cloud technology is the general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, be used on demand, and is flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites, and more portal websites. With the high development and application of the Internet industry, in the future, each item may have its own identification mark and needs to be transmitted to the background system for logical processing. Data at different levels will be processed separately, and various industry data requires a powerful system background support, which can only be achieved through cloud computing.

[0045] In some embodiments, the above-mentioned server can also be implemented as a node in a blockchain system.

[0046] Combined with the above noun introduction and application scenarios, the training method of the result prediction model provided by the present application is described. This method can be executed by the server or the terminal, or jointly executed by the server and the terminal. In the embodiments of the present application, an example is given where this method is executed by the server, as Figure 2 shown, Figure 2 is a flowchart of the training method of the result prediction model provided by an exemplary embodiment of the present application. This method includes the following steps.

[0047] Step 210, obtain the first sample data.

[0048] Among them, the first sample data corresponds to a domain label, and the domain label is used to indicate the first domain category corresponding to the first sample data.

[0049] In some embodiments, the first sample data is sample data for model training.

[0050] Optionally, the acquisition method of the first sample data includes at least one of the following methods:

[0051] 1. Download the first sample data from a public platform under the condition of obtaining authorization;

[0052] 2. Obtain the data records within the historical time range as the first sample data.

[0053] It should be noted that the above acquisition methods of the first sample data are only illustrative examples, and the embodiments of the present application are not limited thereto.

[0054] Optionally, the data content in the first sample data includes at least one of the following contents:

[0055] 1. The first sample data includes text content;

[0056] 2. The first sample data includes sample images;

[0057] 3. The first sample data includes sample images and text content, and the sample images and text content correspond. For example: the sample image is a lung CT (Computed Tomography) picture, and there is a pneumonia condition in the lung, and the text content is "Excessive pleural effusion accumulates in the pleural cavity".

[0058] It should be noted that the above data content in the first sample data is only illustrative examples, and the embodiments of the present application are not limited thereto.

[0059] In some embodiments, the domain label is used to represent the domain category corresponding to the first sample data. For example: when the first sample data belongs to medical data of heart disease category, the domain label is represented as "heart disease", then the domain label is "heart disease".

[0060] Optionally, the domain label is represented by the name of the domain category. For example: when the domain category is "heart disease", the domain label is "heart disease"; or, multiple different candidate domain categories are obtained in advance, and corresponding numerical labels are set for different candidate domain categories. For example: "heart disease" corresponds to 1, "stomach disease" corresponds to 2. If the first domain category is "stomach disease", the domain label is "2".

[0061] Optionally, the acquisition method of the first domain category includes at least one of the following methods:

[0062] 1. When the first sample data includes text content, a keyword library is preset in advance. The keyword library includes multiple candidate domain categories. After obtaining the first sample data, the first text content in the first sample data is matched with the keyword library, and the target domain category that matches the first text content is determined as the first domain category.

[0063] 2. When the first sample data includes text content, a text classification model is pre-trained in advance to identify the domain category corresponding to the input text content. The first text content in the first sample data is input into the text classification model, and the first domain category corresponding to the first text content is output.

[0064] 3. When the first sample data only contains images, an image recognition model is pre-trained in advance to identify the domain category corresponding to the input image. The first sample image in the first sample data is input into the image recognition model, and the first domain category corresponding to the first sample image is output.

[0065] It should be noted that the above-mentioned method for obtaining the first domain category is only a schematic example, and the embodiments of the present application are not limited thereto.

[0066] Step 220: Perform multiple category predictions on the first sample data based on the domain label, and obtain multiple category prediction results corresponding to the first sample data.

[0067] Among them, the category prediction result includes a predicted category and a confidence score. The confidence score is used to indicate the probability value that the first sample data belongs to the predicted category.

[0068] Schematically, the category prediction is used to predict the predicted category corresponding to the first sample data, and the probability value of the first sample data corresponding to the predicted category, that is, the confidence score. For example: The first domain category corresponding to the first sample data is the heart disease category. Based on this heart disease category, two category predictions are performed on the first sample data, and the category prediction results a and b corresponding to the first sample data are obtained. Among them, the category prediction result a is: rheumatic heart disease, and its corresponding confidence score is 80%, indicating that the predicted category corresponding to the first sample data is rheumatic heart disease, and the probability value that the first sample data belongs to rheumatic heart disease is 80%. The category prediction result b is: hypertensive heart disease, and its corresponding confidence score is 75%.

[0069] In some embodiments, multiple classification prediction models corresponding to the first domain category are obtained in advance, so that the first sample data is input into the multiple classification prediction models, and the classification prediction results corresponding to the first sample data are respectively output by the multiple classification prediction models.

[0070] Among them, for a single classification prediction model, after inputting the first sample data into the classification prediction model, the classification prediction model will output a probability distribution value for multiple candidate prediction categories, and determine the prediction category corresponding to the first sample data and its corresponding confidence score. For example: after inputting the first sample data into the classification prediction model a, the output results include "rheumatic heart disease, 5%", "hypertensive heart disease, 85%", and "cor pulmonale, 10%". Among them, the highest confidence score is 85%, and the corresponding candidate prediction category is hypertensive heart disease. Therefore, the prediction category corresponding to the first sample data is hypertensive heart disease, and its corresponding confidence score is 85%.

[0071] In some embodiments, the classification prediction model is implemented as a binary classification model, including at least one of a Logistic Regression model, a Support Vector Machines model, a Decision Trees model, a Random Forests model, a K-Nearest Neighbors model, and a Neural Networks model.

[0072] Step 230, perform data integration on the multiple category prediction results based on the prediction categories and confidence scores respectively corresponding to the multiple category prediction results, to obtain the reliability score corresponding to the first sample data.

[0073] Among them, the reliability score is used to indicate the accuracy of the first sample data belonging to the prediction category.

[0074] Illustratively, since multiple category predictions are performed on the first sample data, multiple category prediction results are obtained. Therefore, by combining the prediction categories and confidence scores to perform data integration on the multiple category prediction results, the accuracy of the first sample data belonging to the prediction category is determined.

[0075] In some embodiments, the reliability score is used to indicate the prediction category with the highest probability value.

[0076] Illustratively, the higher the reliability score, the higher the accuracy of the first sample data belonging to the prediction category; on the contrary, the lower the reliability score, the lower the accuracy of the first sample data belonging to the prediction category.

[0077] Optionally, the reliability score is only used to represent the accuracy of the first sample data belonging to a single predicted category. For example, through multiple classification predictions, the predicted categories corresponding to the first sample data include category a, category b, and category c. By integrating the data of the predicted categories and confidence scores, it is finally obtained that the first sample data corresponds to category a, and category a corresponds to a reliability score; or, the reliability score represents the accuracy of the first sample data belonging to multiple predicted categories. For example, through multiple classification predictions, the predicted categories corresponding to the first sample data include category a, category b, and category c. By integrating the data of the predicted categories and confidence scores, it is finally obtained that the first sample data corresponds to category a with a reliability score of 1, category b with a reliability score of 2, and category c with a reliability score of 3.

[0078] Step 240, in response to the reliability score corresponding to the first sample data meeting the reliability condition, training the sample prediction model with the first sample data to obtain a result prediction model.

[0079] Among them, the result prediction model is used to predict the input data corresponding to multiple field categories respectively to obtain the prediction result corresponding to the input data, and the multiple field categories include the first field category.

[0080] Schematically, after obtaining the reliability score corresponding to the first sample data, the first sample data is screened according to the reliability score. When the reliability score corresponding to the first sample data reaches the pre-set reliability score, it is considered that the first sample data meets the condition of being a sample training data, so as to train the sample prediction model with the first sample data.

[0081] In some embodiments, the reliability condition is implemented as a reliability score threshold. When the reliability score corresponding to the first sample data reaches the reliability score threshold, the reliability score corresponding to the first sample data meets the reliability condition.

[0082] Schematically, when obtaining sample data of different field categories and meeting the reliability condition, training the sample prediction model with these sample data, and finally obtaining a result prediction model for multi-classification recognition. That is, the result prediction model can predict the input data corresponding to multiple field categories to obtain the prediction result corresponding to the input data.

[0083] In one example, the finally trained result prediction model can predict the input lung image to obtain the disease prediction result "advanced lung cancer" corresponding to the lung image, can also predict the input intestinal image to obtain the disease prediction result "early intestinal cancer" corresponding to the intestinal image, and can also predict the input heart image to obtain the disease prediction result "hypertensive heart disease" corresponding to the heart image.

[0084] In summary, for the method provided in this application, after obtaining the first sample data labeled with domain tags, multiple category predictions are made on the first sample data according to the domain tags, so as to obtain multiple category prediction results corresponding to the first sample data, and data integration is performed on the multiple category prediction results according to the predicted categories and confidence levels corresponding to the category prediction results, so as to obtain the accuracy of the first sample belonging to the predicted category, which is used as its reliability score. Finally, the first sample data whose reliability score meets the reliability conditions is used to train the sample prediction model, and a result prediction model capable of predicting input data of multiple domain categories is obtained. That is, after obtaining the first sample data, by performing category prediction on it, the first sample data is screened according to the data accuracy of the first sample data, so that the sample data corresponding to the predicted category is used for model training, which can not only reduce the number of training data and improve the accuracy of model training, but also remove noise data and improve the prediction accuracy of the result prediction model.

[0085] In some embodiments, there is an application program or model in the server that can perform category prediction. Figure 3 It is a flowchart of a method for training a result prediction model provided by an exemplary embodiment of this application. That is,

[0086] Step 220 further includes step 221 and step 222, and the method includes the following steps.

[0087] Step 221, obtaining multiple category prediction models corresponding to the first domain category based on the domain tag.

[0088] In some embodiments, after obtaining the first sample data corresponding to the first domain category, multiple category prediction models corresponding to the first domain category are determined according to the domain tag.

[0089] Schematically, the first sample data includes a first sample image, the first domain category is used to indicate the domain category corresponding to the first sample image, and the category prediction model is used to perform classification prediction on the first sample image.

[0090] In some embodiments, an open-source model library is obtained in advance. Among them, the open-source model library includes multiple candidate category prediction models corresponding to different domain categories. The candidate category prediction models are used to perform classification prediction on the input data within their corresponding domain categories. For example, the model library includes a lung cancer pathological image detection model. When the first sample image in the first sample data is a lung cancer pathological image, therefore, the lung cancer pathological image detection model is selected to perform lung cancer pathological recognition on the first sample image.

[0091] Schematically, multiple class prediction models all belong to the first domain category, but correspond to different model structures. For example, class prediction models a, b, and c all belong to the heart disease pathology detection model, but class prediction model a is a support vector machine model, class prediction model b is a random forest model, and class prediction model c is a K-nearest neighbor model.

[0092] Schematically, multiple class prediction models are pre-trained class prediction models.

[0093] In this embodiment, the class prediction model can be implemented as a binary classification model.

[0094] Step 222: Input the first sample data into multiple class prediction models, and respectively output the class prediction results corresponding to the first sample data.

[0095] Schematically, after obtaining multiple class prediction models, input the first sample image in the first sample data into multiple class prediction models. Each classification prediction model outputs the class prediction result corresponding to the first sample image. Among them, a single class prediction result includes the predicted class corresponding to the first sample image and the confidence score corresponding to the predicted class.

[0096] Optionally, the predicted classes respectively output by multiple prediction class models are the same, but the confidence scores are different. For example, the first sample data includes a pathological image and the corresponding text content "lung cancer". Therefore, input the pathological image into lung cancer pathological image detection model a and lung cancer pathological image detection model b. The predicted class output by lung cancer pathological image detection model a is small cell lung cancer, and the corresponding confidence score is 72%. The predicted class output by lung cancer pathological image detection model b is also small cell lung cancer, and the corresponding confidence score is 84%. Or, the predicted classes respectively output by multiple prediction class models are different and the confidence scores are different. For example, input the pathological image into lung cancer pathological image detection model c and lung cancer pathological image detection model d. The predicted class output by lung cancer pathological image detection model c is small cell lung cancer, and the corresponding confidence score is 86%. The predicted class output by lung cancer pathological image detection model d is non-small cell lung cancer, and the corresponding confidence score is 78%.

[0097] In some embodiments, the multiple class prediction models include the i-th class prediction model, where i is a positive integer. Input the first sample data into the i-th class prediction model, and output the confidence scores corresponding to multiple candidate classes. The confidence scores are used to indicate the probability values that the first sample data belongs to the candidate classes. Determine the predicted class corresponding to the first sample data based on the candidate classes corresponding to the multiple confidence scores. Determine the predicted class and the confidence score as the class prediction result corresponding to the i-th class prediction model.

[0098] In this embodiment, the case where the classification prediction result is obtained by the i-th classification prediction model is taken as an example for illustration.

[0099] When the first sample image in the first sample data is input into the i-th classification prediction model corresponding to the first domain category, a plurality of candidate categories and the confidence scores respectively corresponding to the plurality of candidate categories are output, and the prediction category corresponding to the first sample data and its corresponding confidence score are determined according to the plurality of confidence scores.

[0100] For example: when the first sample image is input into the classification prediction model A, the categories 1 and 2 are output, where the confidence score corresponding to the category 1 is 92%, and the confidence score corresponding to the category 2 is 8%. Among them, the category 1 corresponding to the highest confidence score (92%) is used as the prediction category corresponding to the first sample data, and the prediction category and its corresponding confidence score are used as the classification prediction result corresponding to the first sample data. That is, the candidate category corresponding to the target confidence score with the highest confidence score is determined as the prediction category from the plurality of confidence scores.

[0101] Step 230, perform data integration on the multiple classification prediction results based on the prediction categories and confidence scores respectively corresponding to the multiple classification prediction results to obtain the reliability score corresponding to the first sample data.

[0102] Next, several data integration methods will be described in detail.

[0103] The first method: weighted average method.

[0104] In some embodiments, based on the model prediction capabilities respectively corresponding to the multiple classification prediction models, the first weights respectively corresponding to the multiple classification prediction models are obtained, and the model prediction capabilities are used to indicate the prediction accuracy metrics corresponding to the classification prediction models; based on the product results between the confidence scores and the first weights in the multiple classification prediction results, the first prediction scores respectively corresponding to the multiple classification prediction results are obtained; the multiple first prediction scores are fused to obtain the reliability score corresponding to the first sample data.

[0105] In this embodiment, first, a first weight value is assigned to the classification prediction model according to the model prediction capability corresponding to the classification prediction model, where the model prediction capability includes multiple metrics for indicating the model prediction performance, such as at least one of the metric types of prediction accuracy, F1 score, etc.

[0106] Among them, the model prediction capability can be obtained in the test stage of the classification prediction model after the classification prediction model is trained. For example: the test data is input into the classification prediction model, the classification prediction result corresponding to the test data is output, and the model prediction capability corresponding to the classification prediction model is determined according to the classification prediction result.

[0107] In this embodiment, after obtaining the model prediction ability corresponding to the class prediction model, the first weight corresponding to the class prediction model is obtained according to the model prediction ability. For example, if the prediction accuracy of the class prediction model is 90%, the first weight corresponding to the class prediction model is 0.9.

[0108] Schematically, when the model prediction abilities corresponding to multiple class prediction models are different, the corresponding first weights are also different. Among them, when the model prediction ability is higher, the first weight corresponding to the class prediction model is also higher.

[0109] In this embodiment, after obtaining the first weights corresponding to multiple class prediction models, the first weight corresponding to the class prediction model is multiplied by the confidence score predicted by the class prediction model to obtain the product result corresponding to the class prediction model as the first prediction score.

[0110] In this embodiment, the first prediction scores corresponding to multiple class prediction models are added together to obtain the reliability score corresponding to the first sample data.

[0111] The second method: voting method.

[0112] In some embodiments, the quantity distribution of the predicted classes in multiple class prediction results is obtained. The quantity distribution includes the occurrence times corresponding to multiple predicted classes respectively. Based on the quantity distribution, the target quantity corresponding to the predicted class with the highest occurrence times among multiple predicted classes is determined. The reliability score corresponding to the first sample data is obtained based on the proportional relationship between the target quantity and the number of model of the class prediction model.

[0113] In this embodiment, since multiple class prediction results include multiple predicted classes, the occurrence times corresponding to each predicted class in multiple class prediction results are counted to obtain the quantity distribution, and then the predicted class with the highest occurrence times is determined as the target predicted class and its corresponding occurrence times are recorded.

[0114] The target occurrence times corresponding to the target predicted class are divided by the number of model of the class prediction model to obtain the reliability score corresponding to the first sample data.

[0115] The third method: weighted voting method.

[0116] In some embodiments, first weights corresponding to multiple category prediction models are obtained based on the model prediction capabilities respectively corresponding to the multiple category prediction models, where the model prediction capabilities are used to indicate the prediction accuracy metrics corresponding to the category prediction models; second prediction scores corresponding to the multiple category prediction results are obtained based on the product results between the confidence scores and the first weights in the multiple category prediction results; the prediction category corresponding to the second prediction score with the highest score among the multiple second prediction scores is used as the target prediction category; and a reliability score is obtained through performing data operations on the second prediction score corresponding to the target prediction category and the sum of the multiple first weights.

[0117] In this embodiment, the first weight corresponding to the category prediction model is obtained according to the model prediction capability corresponding to the category prediction model.

[0118] In this embodiment, the product between the first weight corresponding to the category prediction model and the confidence score is used to obtain the second prediction score corresponding to the category prediction model. The prediction category with the highest second prediction score is determined from the second prediction scores respectively corresponding to the multiple category prediction models and used as the target prediction category corresponding to the first sample data. The second prediction score corresponding to the target prediction category is divided by the sum of the multiple first weights to obtain the reliability score corresponding to the first sample data.

[0119] In summary, for the method provided in this application, after obtaining the first sample data labeled with domain labels, multiple category predictions are performed on the first sample data according to the domain labels, so as to obtain multiple category prediction results corresponding to the first sample data. Data integration is performed on the multiple category prediction results according to the prediction categories and confidence degrees corresponding to the category prediction results, so as to obtain the accuracy of the first sample belonging to the predicted category as its reliability score. Finally, the first sample data whose reliability score meets the reliability condition is used to train the sample prediction model, and a result prediction model capable of predicting input data of multiple domain categories is obtained. That is, after obtaining the first sample data, by performing category prediction on it, the first sample data is screened according to the data accuracy of the first sample data, so that the sample data corresponding to the predicted category is used for model training, which can not only reduce the number of training data and improve the accuracy of model training, but also remove noise data and improve the prediction accuracy of the result prediction model.

[0120] In this embodiment, by using different category prediction models to perform classification prediction on the first sample data, different category prediction results can be obtained, which is convenient for improving the data accuracy.

[0121] In this embodiment, by selecting the prediction category with the highest confidence score as the prediction category corresponding to the first sample data, the matching degree between the first sample data and the prediction category can be guaranteed, which is convenient for improving the accuracy of model training.

[0122] In this embodiment, data integration is performed on the confidence score and the predicted category through a variety of different methods, so as to obtain the reliability score corresponding to the first sample data, which can screen out the sample data with low reliability. On the one hand, the number of samples is reduced, and the model training efficiency is improved. On the other hand, using the sample data with high reliability to train the sample prediction model can improve the accuracy of the result prediction model.

[0123] In some embodiments, the sample data is screened by a reliability threshold. Schematically, please refer to Figure 4 , which shows the flowchart of the training method of the result prediction model provided by an exemplary embodiment of the present application. That is, step 210 further includes step 2101, and step 240 further includes steps 241 and 242, as Figure 4 shown, the method includes the following steps.

[0124] Step 2101, perform domain recognition on the first sample image based on the first text content to obtain the domain label corresponding to the first sample image.

[0125] Schematically, the first sample data includes a first sample image and the first text content corresponding to the first sample image.

[0126] In this embodiment, taking the form of image + text content in the first sample data as an example for illustration, wherein the first text content in the first sample data is used to describe the text information corresponding to the first sample image.

[0127] In an alternative case, a keyword library is preset in advance. The keyword library includes a plurality of candidate domain categories and candidate domain labels corresponding to the candidate domain categories. The first text content is matched with the keyword library to determine the first domain category corresponding to the first sample image and the domain label corresponding to the first domain category.

[0128] In another alternative case, a text recognition model is pre-trained, and the first text content in the first sample data is input into the text recognition model, and the output domain category prediction result is used as the first domain category corresponding to the first sample image, so as to determine the domain label corresponding to the first domain category.

[0129] Step 241, obtain a preset reliability threshold.

[0130] Obtain the preset reliability threshold. For example: 85 points, and this reliability threshold can be designed according to actual needs.

[0131] Step 242: In response to the reliability score corresponding to the first sample data reaching the reliability threshold, train the sample prediction model with the first sample data to obtain the result prediction model.

[0132] In some embodiments, in response to the reliability score corresponding to the first sample data meeting the reliability condition, train the sample prediction model with the first sample image in the first sample data to obtain the result prediction model.

[0133] In this embodiment, when the reliability score corresponding to the first sample data reaches the reliability threshold, use the first sample image in the first sample data as the training data to train the sample prediction model to obtain the result prediction model.

[0134] In one example, after training the result prediction model with the first sample data, the original first sample data can be re - input into the result prediction model, and the classification prediction result corresponding to the first sample data can be output. Then, according to the classification prediction result, a new reliability score corresponding to the first sample data can be obtained again. If the new reliability score meets the reliability condition, the first sample data is used to train the result prediction model again to form an iterative training process.

[0135] In summary, for the method provided in this application, after obtaining the first sample data labeled with domain labels, multiple category predictions are made on the first sample data according to the domain labels, so as to obtain multiple category prediction results corresponding to the first sample data. Then, data integration is performed on the multiple category prediction results according to the predicted category and confidence level corresponding to the category prediction results, so as to obtain the accuracy of the first sample belonging to the predicted category as its reliability score. Finally, the first sample data whose reliability score meets the reliability condition is used to train the sample prediction model to obtain a result prediction model that can predict input data of multiple domain categories. That is, after obtaining the first sample data, by making category predictions on it, the first sample data is screened according to the data accuracy of the first sample data, so that the sample data corresponding to the predicted category is used for model training, which can not only reduce the number of training data and improve the accuracy of model training, but also remove noise data and improve the prediction accuracy of the result prediction model.

[0136] In this embodiment, by presetting the reliability threshold as the reliability condition, sample data that does not meet the reliability condition can be quickly screened out, improving the model training efficiency and the prediction accuracy of the model.

[0137] In some embodiments, schematically, please refer to Figure 5 , which shows a schematic diagram of the training process of the result prediction model provided by an exemplary embodiment of this application, as Figure 5As shown, this method will be described by taking its application in a medical scenario as an example.

[0138] After obtaining the first sample data 501 including the first sample image and the first text content, text analysis is performed according to the first text content in the first sample data 501 to obtain the neighborhood label corresponding to the first sample data 501.

[0139] Optionally, the text analysis includes the following two methods:

[0140] (1) Based on keyword matching: By constructing a thesaurus containing various disease names and their synonyms, and then searching for keywords in the first text content that match the disease names in the thesaurus. If a certain disease name appears in the first text content, it is considered that the first sample data is related to the disease, and the disease is determined as the neighborhood label.

[0141] (2) Text classification based on machine learning: Train a text classification model, such as Support Vector Machine (SVM), Naive Bayes, or deep learning models, such as Bidirectional Encoder Representation from Transformers (Bert model), RoBERTa, etc., to identify the diseases contained in the first text content, and determine the diseases as the neighborhood labels. To train these models, a pre-labeled training set is required, which contains relevant texts of various diseases and their corresponding disease labels.

[0142] After obtaining the neighborhood label, the first sample image is subjected to class prediction by obtaining multiple class prediction models 502 corresponding to the neighborhood label to obtain multiple class prediction results 503. The specific prediction content is as follows.

[0143] (1) For the first sample data, according to the neighborhood label, select an appropriate open-source prediction model. For example: If the sample is labeled as related to heart disease, an open-source model for heart disease prediction will be selected.

[0144] (2) Input the first sample data into the selected open-source prediction model. To be compatible with the model, the sample may need to be further preprocessed, such as converting the text into the input format required by the model (such as word vectors, sentence vectors, etc.).

[0145] (3) Obtain the class prediction results of the model. Open-source prediction models usually output a probability value for each predicted class (i.e., disease type), indicating the possibility that the first sample data 501 belongs to the predicted class. These probability values can be regarded as prediction confidence, that is, confidence scores.

[0146] (4) Select the prediction class with the highest prediction confidence (confidence score) as the final predicted disease of the sample. Meanwhile, record the prediction confidence of this class to evaluate the reliability of the first sample data 501 in subsequent steps. Take the prediction confidence and the prediction class as the class prediction results corresponding to the first sample data 501.

[0147] After obtaining multiple class prediction results 503, perform data integration on the multiple class prediction results according to the confidence score and the prediction class to obtain the reliability score 504 of the first sample data 501. The specific methods are as follows.

[0148] (1) Weighted average method: Assign a weight to each open-source prediction model according to its performance (such as accuracy, F1 score, etc.). Then, multiply the prediction confidence of each model by its corresponding weight, and finally sum the weighted prediction confidences. In this way, we can obtain a comprehensive reliability score.

[0149] (2) Voting method: For each sample, count the number of times a certain disease is predicted in all open-source prediction models. Take the disease with the most occurrences as the final prediction result, and at the same time divide the number of occurrences by the total number of models as the reliability score.

[0150] (3) Weighted voting method based on model performance: Combine the weighted average method and the voting method, assign a weight to each model, and multiply its prediction result by the weight. Then, count the weighted voting results, select the disease with the highest weighted vote as the final prediction result, and divide its weighted vote count by the sum of the weights as the reliability score.

[0151] After obtaining the reliability scores 504 corresponding to multiple sample data respectively, perform screening on them to obtain the training dataset 505. The specific process is as follows:

[0152] Set a reliability threshold: According to the actual needs and requirements for data quality, set a reliability threshold. This threshold can be a fixed value or a value dynamically adjusted according to the characteristics of the dataset. For example, the threshold can be set as u, indicating that only samples with a reliability score greater than or equal to u are retained.

[0153] (1) Filter samples: Traverse all samples in the dataset and check the reliability score of each sample. If the reliability score of a certain sample is lower than the set threshold, remove it from the dataset. In this way, a filtered dataset will be obtained, which only contains samples with higher confidence.

[0154] (2) Update the dataset: Use the filtered dataset as the training data for pre-training the medical large model. This will help improve the performance of the model and remove low-quality samples that may introduce noise and mislead the model.

[0155] (3) Iterative optimization: In practical applications, the quality of the dataset can be continuously improved through multiple iterative optimization processes. For example, after training a new large medical model using the filtered dataset, the original dataset can be input into the model again to obtain new reliability scores, and filtering can be performed based on the new scores. Through this iterative process, the quality of the dataset can be gradually optimized, thereby improving the performance of the model.

[0156] Finally, the sample prediction model 506 is trained using the training dataset 505 to obtain the result prediction model 507.

[0157] In summary, for the method provided in this application, after obtaining the first sample data labeled with domain labels, multiple category predictions are made on the first sample data according to the domain labels, so as to obtain multiple category prediction results corresponding to the first sample data. Then, data integration is performed on the multiple category prediction results according to the predicted categories and confidence levels corresponding to the category prediction results, so as to obtain the accuracy of the first sample belonging to the predicted category as its reliability score. Finally, the first sample data whose reliability score meets the reliability conditions is used to train the sample prediction model, and a result prediction model capable of predicting input data of multiple domain categories is obtained. That is, after obtaining the first sample data, by making category predictions on it and screening the first sample data according to the data accuracy of the first sample data, the sample data corresponding to the predicted category is used for model training, which can not only reduce the number of training data and improve the accuracy of model training, but also remove noise data and improve the prediction accuracy of the result prediction model.

[0158] The data filtering technical solution based on uncertainty proposed in this application aims to provide higher-quality training data for relevant enterprises and research institutions in the medical field, so as to train a pre-trained model with better performance. The application of this technical solution can improve the performance of the pre-trained model, making it perform more excellently in various medical tasks, thus attracting more potential customers. In addition, by removing noise and low-quality samples from the training data, we can ensure that the trained model has higher accuracy, so as to provide more accurate diagnosis and treatment suggestions for users. At the same time, this technical solution can also reduce the scale of the training data set, thereby reducing the computing resources and time required in the model training process, which helps to reduce the R & D costs of enterprises. By reducing the noise interference and overfitting phenomenon in the training data, we can improve the generalization ability of the model, making the product more adaptable when dealing with the diversification and complexity in actual applications. With the high-quality data processed by the method of this patent, the pre-trained model can better understand and interpret medical texts. More importantly, the method of this patent can be applied to a variety of medical tasks, such as disease diagnosis, treatment plan recommendation, drug R & D, etc. The data filtering technical solution based on uncertainty proposed in this patent provides an effective means of data quality control for relevant enterprises and research institutions in the medical field. From improving model performance, enhancing product accuracy, reducing training costs, strengthening model generalization ability, optimizing product experience to expanding product application fields, this technical solution creates value for enterprises in multiple aspects and is expected to become an important technical means for data processing in the medical field.

[0159] Figure 6 is the structural block diagram of the training device of the result prediction model provided by an exemplary embodiment of this application, as Figure 6 shown, this device includes the following parts.

[0160] An acquisition module 610, configured to acquire first sample data, where the first sample data corresponds to a domain label, and the domain label is used to indicate the first domain category corresponding to the first sample data;

[0161] A prediction module 620, configured to perform multiple category predictions on the first sample data based on the domain label, to obtain multiple category prediction results corresponding to the first sample data, where the category prediction results include a predicted category and a confidence score, and the confidence score is used to indicate the probability value that the first sample data belongs to the predicted category;

[0162] An integration module 630, configured to perform data integration on the multiple category prediction results based on the predicted categories and the confidence scores respectively corresponding to the multiple category prediction results, to obtain a reliability score corresponding to the first sample data, and the reliability score is used to indicate the accuracy situation of the first sample data belonging to the predicted category;

[0163] A training module 640, configured to train a sample prediction model with the first sample data to obtain the result prediction model in response to the reliability score corresponding to the first sample data meeting the reliability condition, where the result prediction model is used to predict input data corresponding to multiple domain categories respectively to obtain a prediction result corresponding to the input data, and the multiple domain categories include the first domain category.

[0164] In some embodiments, the prediction module 620 is further configured to obtain a plurality of class prediction models corresponding to the first domain category based on the domain label; input the first sample data into the plurality of class prediction models, and respectively output class prediction results corresponding to the first sample data.

[0165] In some embodiments, the plurality of class prediction models include an i-th class prediction model, where i is a positive integer;

[0166] The prediction module 620 is further configured to input the first sample data into the i-th class prediction model, and output confidence scores corresponding to a plurality of candidate classes respectively, where the confidence scores are used to indicate probability values that the first sample data belongs to the candidate classes; determine a prediction class corresponding to the first sample data based on the candidate classes corresponding to the plurality of confidence scores; and determine the prediction class and the confidence scores as the class prediction result corresponding to the i-th class prediction model.

[0167] In some embodiments, the prediction module 620 is further configured to determine a candidate class corresponding to a target confidence score with the highest confidence score among the plurality of confidence scores as the prediction class.

[0168] In some embodiments, the integration module 630 is further configured to obtain first weights corresponding to the plurality of class prediction models based on model prediction capabilities corresponding to the plurality of class prediction models, where the model prediction capabilities are used to indicate prediction accuracy metrics corresponding to the class prediction models; obtain first prediction scores corresponding to the plurality of class prediction results based on product results between the confidence scores in the plurality of class prediction results and the first weights; and fuse the plurality of first prediction scores to obtain a reliability score corresponding to the first sample data.

[0169] In some embodiments, the integration module 630 is further configured to obtain the quantity distribution of the predicted categories corresponding to the multiple category prediction results, where the quantity distribution includes the occurrence times corresponding to the multiple predicted categories respectively; determine the target quantity corresponding to the predicted category with the highest occurrence times among the multiple predicted categories based on the quantity distribution; and obtain the reliability score corresponding to the first sample data based on the proportional relationship between the target quantity and the number of models of the category prediction model.

[0170] In some embodiments, the integration module 630 is further configured to obtain the first weight corresponding to each of the multiple category prediction models based on the model prediction capabilities corresponding to the multiple category prediction models, where the model prediction capabilities are used to indicate the prediction accuracy metrics corresponding to the category prediction models; obtain the second prediction score corresponding to each of the multiple category prediction results based on the product result between the confidence scores in the multiple category prediction results and the first weights; use the predicted category corresponding to the highest second prediction score among the multiple second prediction scores as the target predicted category; and perform data operations on the second prediction score corresponding to the target predicted category and the sum of the multiple first weights to obtain the reliability score.

[0171] In some embodiments, the training module 640 is further configured to obtain a preset reliability threshold; and in response to the reliability score corresponding to the first sample data reaching the reliability threshold, train the sample prediction model with the first sample data to obtain the result prediction model.

[0172] In some embodiments, the first sample data includes a first sample image and first text content corresponding to the first sample image;

[0173] The acquisition module 610 is further configured to perform domain recognition on the first sample image based on the first text content to obtain the domain label corresponding to the first sample image;

[0174] The training module 640 is further configured to, in response to the reliability score corresponding to the first sample data meeting the reliability condition, train the sample prediction model with the first sample image in the first sample data to obtain the result prediction model.

[0175] In summary, for the training device of the result prediction model provided in this application, after obtaining the first sample data labeled with domain tags, multiple category predictions are performed on the first sample data according to the domain tags, so as to obtain multiple category prediction results corresponding to the first sample data. Then, data integration is performed on the multiple category prediction results according to the predicted categories and confidence levels corresponding to the category prediction results, so as to obtain the accuracy of the first sample belonging to the predicted category as its reliability score. Finally, the first sample data whose reliability score meets the reliability condition is used to train the sample prediction model, and a result prediction model capable of predicting input data of multiple domain categories is obtained. That is, after obtaining the first sample data, by performing category prediction on it, the first sample data is screened according to the data accuracy of the first sample data, so that the sample data corresponding to the predicted category is used for model training. This can not only reduce the number of training data and improve the accuracy of model training, but also remove noise data and improve the prediction accuracy of the result prediction model.

[0176] It should be noted that: for the training device of the result prediction model provided in the above embodiment, only the above division of each functional module is used for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the training device of the result prediction model provided in the above embodiment and the embodiment of the training method of the result prediction model belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be elaborated here.

[0177] Figure 7 The block diagram of a computer device 700 provided by an exemplary embodiment of this application is shown. The computer device 700 can be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV) player, a notebook computer or a desktop computer. The computer device 700 may also be referred to by other names such as user equipment, portable terminal, laptop terminal, desktop terminal, etc.

[0178] Generally, the computer device 700 includes: a processor 701 and a memory 702.

[0179] The processor 701 may include one or more processing cores, such as a 4-core processor, a 7-core processor, etc. The processor 701 may be implemented in at least one of the following hardware forms: DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 701 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the wake state, also known as the CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 701 may be integrated with a GPU (Graphics Processing Unit), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 701 may further include an AI (Artificial Intelligence) processor, and the AI processor is used to process computational operations related to machine learning.

[0180] The memory 702 may include one or more computer-readable storage media, and the computer-readable storage media may be non-transitory. The memory 702 may further include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash storage devices. In some embodiments, the non-transitory computer-readable storage media in the memory 702 is used to store at least one instruction, and the at least one instruction is used to be executed by the processor 701 to implement the training method of the result prediction model provided in the method embodiments of the present application.

[0181] In some embodiments, the computer device 700 further includes other components. Those skilled in the art can understand that Figure 7 the structure shown does not limit the computer device 700, and it may include more or fewer components than shown, or combine certain components, or adopt a different component layout.

[0182] Optionally, the computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), solid state drives (SSD), or optical discs, etc. Among them, the random access memory may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM). The serial numbers of the embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0183] An embodiment of the present application further provides a computer device, which includes a processor and a memory. At least one instruction, at least one program, a code set, or an instruction set is stored in the memory. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the training method of the result prediction model as described in any one of the embodiments of the present application above.

[0184] An embodiment of the present application further provides a computer-readable storage medium. At least one instruction, at least one program, a code set, or an instruction set is stored in the storage medium. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the training method of the result prediction model as described in any one of the embodiments of the present application above.

[0185] An embodiment of the present application further provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the training method of the result prediction model as described in any one of the above embodiments.

[0186] Those of ordinary skill in the art can understand that all or part of the steps to implement the above embodiments can be completed by hardware, or can be completed by a program instructing related hardware. The program can be stored in a computer-readable storage medium. The storage medium mentioned above can be read-only memory, a magnetic disk, or an optical disc, etc.

[0187] The above are only optional embodiments of the present application and are not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A training method for a result prediction model, characterized in that The method includes: Obtain first sample data, where the first sample data corresponds to a domain label, and the domain label is used to indicate the first domain category corresponding to the first sample data; Perform multiple category predictions on the first sample data based on the domain label to obtain multiple category prediction results corresponding to the first sample data. The category prediction results include a predicted category and a confidence score, and the confidence score is used to indicate the probability value that the first sample data belongs to the predicted category; Integrate the multiple category prediction results based on the predicted categories and the confidence scores respectively corresponding to the multiple category prediction results to obtain a reliability score corresponding to the first sample data, and the reliability score is used to indicate the accuracy of the first sample data belonging to the predicted category; In response to the reliability score corresponding to the first sample data meeting the reliability condition, train a sample prediction model with the first sample data to obtain the result prediction model. The result prediction model is used to predict input data corresponding to multiple domain categories respectively to obtain a prediction result corresponding to the input data, and the multiple domain categories include the first domain category.

2. The method according to claim 1, wherein The performing multiple category predictions on the first sample data based on the domain label to obtain multiple category prediction results corresponding to the first sample data includes: Obtain multiple category prediction models corresponding to the first domain category based on the domain label; Input the first sample data into the multiple category prediction models, and respectively output the category prediction results corresponding to the first sample data.

3. The method according to claim 2, characterized in that The multiple category prediction models include the i-th category prediction model, where i is a positive integer; The inputting the first sample data into the multiple category prediction models and respectively outputting the category prediction results corresponding to the first sample data includes: Input the first sample data into the i-th category prediction model, and output confidence scores corresponding to multiple candidate categories respectively. The confidence score is used to indicate the probability value that the first sample data belongs to the candidate category; Determine the predicted category corresponding to the first sample data based on the candidate categories respectively corresponding to the multiple confidence scores; Determine the predicted category and the confidence score as the category prediction result corresponding to the i-th category prediction model.

4. The method according to claim 3, wherein The determining the predicted category corresponding to the first sample data based on the candidate categories respectively corresponding to the multiple confidence scores includes: Determine the candidate category corresponding to the target confidence score with the highest confidence score among the multiple confidence scores as the predicted category.

5. The method according to any one of claims 2 to 4, characterized in that, The integrating the multiple category prediction results based on the predicted categories and the confidence scores respectively corresponding to the multiple category prediction results to obtain a reliability score corresponding to the first sample data includes: Obtain first weights respectively corresponding to the multiple category prediction models based on the model prediction capabilities respectively corresponding to the multiple category prediction models. The model prediction capability is used to indicate the prediction accuracy index corresponding to the category prediction model; Based on the product result between the confidence scores in the multiple category prediction results and the first weights, obtain the first prediction scores corresponding to the multiple category prediction results respectively; Fuse multiple first prediction scores to obtain the reliability score corresponding to the first sample data.

6. The method according to any one of claims 2 to 4, characterized in that, The data integration of the multiple category prediction results based on the predicted categories and the confidence scores corresponding to the multiple category prediction results respectively to obtain the reliability score corresponding to the first sample data includes: Obtain the quantity distribution of the predicted categories corresponding to the multiple category prediction results, where the quantity distribution includes the occurrence times corresponding to multiple predicted categories respectively; Based on the quantity distribution, determine the target quantity corresponding to the predicted category with the highest occurrence times among the multiple predicted categories; Obtain the reliability score corresponding to the first sample data based on the proportional relationship between the target quantity and the number of models of the category prediction model.

7. The method according to any one of claims 2 to 4, characterized in that, The data integration of the multiple category prediction results based on the predicted categories and the confidence scores corresponding to the multiple category prediction results respectively to obtain the reliability score corresponding to the first sample data includes: Obtain the first weights corresponding to the multiple category prediction models respectively based on the model prediction capabilities corresponding to the multiple category prediction models, where the model prediction capabilities are used to indicate the prediction accuracy indicators corresponding to the category prediction models; Based on the product result between the confidence scores in the multiple category prediction results and the first weights, obtain the second prediction scores corresponding to the multiple category prediction results respectively; Take the predicted category corresponding to the second prediction score with the highest score among the multiple second prediction scores as the target predicted category; Perform data operations on the second prediction score corresponding to the target predicted category and the sum of the multiple first weights to obtain the reliability score.

8. The method according to any one of claims 1 to 4, characterized in that, In response to the reliability score corresponding to the first sample data meeting the reliability condition, train the sample prediction model with the first sample data to obtain the result prediction model, including: Obtain a preset reliability threshold; In response to the reliability score corresponding to the first sample data reaching the reliability threshold, train the sample prediction model with the first sample data to obtain the result prediction model.

9. The method according to any one of claims 1 to 4, characterized in that The first sample data includes a first sample image and the first text content corresponding to the first sample image; Before obtaining the multiple category prediction results corresponding to the first sample data by performing multiple category predictions on the first sample data based on the domain label, it further includes: Perform domain recognition on the first sample image based on the first text content to obtain the domain label corresponding to the first sample image; In response to the reliability score corresponding to the first sample data meeting the reliability condition, train the sample prediction model with the first sample data to obtain the result prediction model, including: In response to the reliability score corresponding to the first sample data meeting the reliability condition, training the sample prediction model with the first sample image in the first sample data to obtain the result prediction model.

10. A training device for a result prediction model, characterized in that, The device includes: An acquisition module, configured to acquire first sample data, where the first sample data corresponds to a domain label, and the domain label is used to indicate a first domain category corresponding to the first sample data; A prediction module, configured to perform multiple category predictions on the first sample data based on the domain label to obtain multiple category prediction results corresponding to the first sample data, where the category prediction results include a predicted category and a confidence score, and the confidence score is used to indicate a probability value that the first sample data belongs to the predicted category; An integration module, configured to perform data integration on the multiple category prediction results based on the predicted categories and the confidence scores respectively corresponding to the multiple category prediction results to obtain a reliability score corresponding to the first sample data, where the reliability score is used to indicate the accuracy of the first sample data belonging to the predicted category; A training module, configured to, in response to the reliability score corresponding to the first sample data meeting the reliability condition, train the sample prediction model with the first sample data to obtain the result prediction model, where the result prediction model is used to perform predictions on input data corresponding to multiple domain categories respectively to obtain prediction results corresponding to the input data, and the multiple domain categories include the first domain category.

11. A computer device, characterized in that, The computer device includes a processor and a memory, and at least one program is stored in the memory, and the at least one program is loaded and executed by the processor to implement the training method of the result prediction model according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, At least one program is stored in the storage medium, and the at least one program is loaded and executed by the processor to implement the training method of the result prediction model according to any one of claims 1 to 9.

13. A computer program product, characterized in that, Including a computer program, where when the computer program is executed by a processor, it implements the training method of the result prediction model according to any one of claims 1 to 9.

Images