Cross-validation method, device and electronic equipment for disease prediction model

Abstract

The present application provides a cross-validation method, device and electronic device for a disease prediction model. Based on the original medical data obtained from the two regions, the original feature sample sets and the common feature sample sets of the two regions are determined; based on the training sample sets in the original feature sample sets and the common feature sample sets, the disease prediction original models and the disease prediction common models corresponding to the two regions are constructed; the test sample sets in the original feature sample sets and the common feature sample sets are used to perform corresponding tests and cross-tests on the models of the two regions to obtain multiple test results; cross-validation is performed based on the multiple test results to determine the substitutability between the models of the two regions. The present application can perform cross-validation based on the medical data collected from the two regions and the constructed models to determine the substitutability between the models, thereby providing data support and reference for model construction in other regions, so as to assist other regions in quickly building disease prediction models.

Description

Technical Field

[0001] The present application relates to the field of underwriting technology, and in particular to a cross-validation method, device and electronic equipment for a disease prediction model. Background Art

[0002] Underwriting is a very important part of the insurance company's risk control process, and medical underwriting is the most rigorous and valuable part of it. This process is usually conducted by underwriters who comprehensively judge the insured's physical health based on their medical health data (including age, gender, physical signs, medical history, etc.). In recent years, with the rapid development of the Internet and information technology, the informatization and electronicization of the medical field have gradually improved, various types of medical data have increased dramatically, and the data dimensions have become richer. Therefore, with the help of machine learning and big data models, the efficiency and quality of medical and health data analysis can be effectively improved, the health risks of the insured can be judged more scientifically, and strong data support can be provided for insurance underwriting.

[0003] However, building a complete machine learning model is time-consuming and labor-intensive. First, it requires enough samples for training. Second, the modeling ideas need to be scientific and reasonable (including data selection, feature selection, data preprocessing, etc.). Third, model parameter adjustment is cumbersome and time-consuming. Fourth, training the model requires certain hardware support. The entire process usually takes several months. When expanding business in a new region, how to learn from the existing data results and experience of data laboratories in other regions, and how to easily and quickly build data models in the new region and provide data support is an important issue currently faced. Summary of the invention

[0004] The purpose of this application is to provide a cross-validation method, device and electronic equipment for a disease prediction model. The original model and the common model are built based on the medical data collected in the two places, and then the four models in the two places are tested and cross-tested using various test sample sets. The cross-validation is performed based on the test results to determine the substitutability of the four models, thereby providing data support and reference for model construction in other regions, so as to assist other regions in quickly building disease prediction models.

[0005] In a first aspect, an embodiment of the present application provides a cross-validation method for a disease prediction model, the method comprising: obtaining first original medical data of a first region and second original medical data of a second region; determining a first original feature sample set and a first common feature sample set corresponding to the first region, and a second original feature sample set and a second common feature sample set corresponding to the second region based on the first original medical data and the second original medical data; each sample set includes a training sample set and a test sample set; based on the training sample sets in the first original feature sample set and the first common feature sample set, constructing a first disease prediction original model and a first disease prediction common model corresponding to the first region respectively; based on the second original feature sample set The training sample sets in this set and the second common feature sample set are used to construct the second disease prediction original model and the second disease prediction public model corresponding to the second region respectively; the test sample sets in the first original feature sample set, the first common feature sample set, the second original feature sample set and the second common feature sample set are used to perform corresponding tests and cross tests on the first disease prediction original model, the first disease prediction public model, the second disease prediction original model and the second disease prediction public model to obtain multiple test results; cross-validation is performed based on the multiple test results to determine the substitutability among the first disease prediction original model, the first disease prediction public model, the second disease prediction original model and the second disease prediction public model.

[0006] Furthermore, the above-mentioned step of determining the first original feature sample set and the first common feature sample set corresponding to the first region, and the second original feature sample set and the second common feature sample set corresponding to the second region based on the first original medical data and the second original medical data, includes: determining the corresponding common features in the first original medical data and the second original medical data; taking the first original medical data and the second original medical data as current medical data, respectively, and performing the following steps: preprocessing the current medical data, and dividing the current medical data into a training sample set and a test sample set according to a specified ratio, to obtain the original feature sample set corresponding to the current medical data; extracting corresponding sample data from the current medical data based on the common features, and dividing the sample data into a training sample set and a test sample set according to a specified ratio, to obtain the common feature sample set corresponding to the current medical data.

[0007] Furthermore, the above-mentioned steps of constructing the first disease prediction original model and the first disease prediction public model corresponding to the first region based on the training sample set in the first original feature sample set and the first common feature sample set; and constructing the second disease prediction original model and the second disease prediction public model corresponding to the second region based on the training sample set in the second original feature sample set and the second common feature sample set, include: taking the training sample sets in the first original feature sample set, the first common feature sample set, the second original feature sample set and the second common feature sample set as the current training sample sets, and performing the following operations: performing model training based on the current training sample set and a preset algorithm; the preset algorithm includes one of the following: CatBoost algorithm, LightGBM algorithm, XGBoost algorithm; performing model parameter adjustment processing on the trained model to obtain the disease prediction model corresponding to the current training sample set; the model parameter adjustment processing includes at least one of the following: adjusting the number of iterations, adjusting the learning rate, adjusting the depth of the tree, and adjusting the L2 regularization coefficient.

[0008] Furthermore, the above-mentioned step of applying the first original feature sample set, the first common feature sample set, the second original feature sample set and the test sample set in the second common feature sample set to perform corresponding tests and cross tests on the first disease prediction original model, the first disease prediction public model, the second disease prediction original model and the second disease prediction public model to obtain multiple test results includes: applying the test sample set in the first original feature sample set to test the first disease prediction original model to obtain a first test result; applying the test sample set in the first common feature sample set to test the first disease prediction public model to obtain a second test result; applying the test sample set in the second common feature sample set to test the first disease prediction public model to obtain a third test result; applying the test sample set in the first common feature sample set to test the second disease prediction public model to obtain a fourth test result; applying the test sample set in the second common feature sample set to test the second disease prediction public model to obtain a fifth test result; applying the test sample set in the second original feature sample set to test the second disease prediction original model to obtain a sixth test result; wherein each test result includes AUC and lift.

[0009] Furthermore, the above-mentioned step of performing cross-validation based on multiple test results to determine the interchangeability between the first disease prediction original model, the first disease prediction public model, the second disease prediction original model and the second disease prediction public model includes: comparing the second test result with the first test result, and if the corresponding AUC and lift decrease rates are both less than the first preset threshold, determining that the first disease prediction public model can replace the first disease prediction original model; comparing the fifth test result with the sixth test result, and if the corresponding AUC and lift decrease rates are both less than the second preset threshold, determining that the second disease prediction public model can replace the second disease prediction original model; comparing the fourth test result with the first test result, and if the corresponding AUC and lift decrease rates are both less than the third preset threshold, determining that the second disease prediction public model can replace the first disease prediction original model; comparing the third test result with the sixth test result, and if the corresponding AUC and lift decrease rates are both less than the fourth preset threshold, determining that the first disease prediction public model can replace the second disease prediction original model.

[0010] Furthermore, the above method also includes: if there are sufficient data samples in the third region, based on the common features, obtaining a third common feature sample set of the third region; applying the third common feature sample set to perform model training to obtain a target disease prediction model corresponding to the third region.

[0011] Furthermore, if the data samples in the third region are insufficient, the model with higher model accuracy between the first disease prediction public model and the second disease prediction public model is used as the target disease prediction model corresponding to the third region.

[0012] In a second aspect, an embodiment of the present application further provides a cross-validation device for a disease prediction model, the device comprising: a data acquisition module for acquiring first original medical data of a first region and second original medical data of a second region; a sample set determination module for determining, based on the first original medical data and the second original medical data, a first original feature sample set and a first common feature sample set corresponding to the first region, and a second original feature sample set and a second common feature sample set corresponding to the second region; each sample set comprises a training sample set and a test sample set; a model construction module for constructing, based on the training sample sets in the first original feature sample set and the first common feature sample set, a first disease prediction original model and a first disease prediction common model corresponding to the first region, respectively; The training sample sets in the second original feature sample set and the second common feature sample set are used to construct the second disease prediction original model and the second disease prediction common model corresponding to the second region respectively; the testing module is used to perform corresponding tests and cross tests on the first disease prediction original model, the first disease prediction common model, the second disease prediction original model and the second disease prediction common model based on the first original feature sample set, the first common feature sample set, the second original feature sample set and the testing sample sets in the second common feature sample set to obtain multiple test results; the verification module is used to perform cross-validation based on the multiple test results to determine the substitutability among the first disease prediction original model, the first disease prediction common model, the second disease prediction original model and the second disease prediction common model.

[0013] In a third aspect, an embodiment of the present application further provides an electronic device, including a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the method described in the first aspect above.

[0014] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method described in the first aspect above.

[0015] In the cross-validation method, device and electronic device of the disease prediction model provided in the embodiments of the present application, firstly, based on the first original medical data of the first region and the second original medical data of the second region, the first original feature sample set and the first common feature sample set corresponding to the first region, and the second original feature sample set and the second common feature sample set corresponding to the second region are determined; wherein each sample set includes a training sample set and a test sample set; then, based on the training sample sets in the first original feature sample set and the first common feature sample set, the first disease prediction original model and the first disease prediction common model corresponding to the first region are constructed; based on the second original feature sample set and the second common feature sample set, the first disease prediction original model and the first disease prediction common model corresponding to the first region are constructed. The training sample set in the sample set is used to construct the second disease prediction original model and the second disease prediction public model corresponding to the second region respectively; then the test sample set in the first original feature sample set, the first public feature sample set, the second original feature sample set and the second public feature sample set is used to perform corresponding tests and cross tests on the first disease prediction original model, the first disease prediction public model, the second disease prediction original model and the second disease prediction public model to obtain multiple test results; finally, cross-validation is performed based on the multiple test results to determine the substitutability between the first disease prediction original model, the first disease prediction public model, the second disease prediction original model and the second disease prediction public model. In the embodiment of the present application, the original model and the public model are firstly modeled based on the medical data collected in the two places, and then the four models of the two places are correspondingly tested and cross-tested using each test sample set, and cross-validation is performed based on the test results to determine the substitutability between the four models, thereby providing guidance or reference for the model construction of the third region. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 A flow chart of a cross-validation method for a disease prediction model provided in an embodiment of the present application;

[0018] Figure 2 A schematic diagram of a cross-validation method for a disease prediction model provided in an embodiment of the present application;

[0019] Figure 3 A flowchart of a cross-validation process provided in an embodiment of the present application;

[0020] Figure 4A structural block diagram of a cross-validation device for a disease prediction model provided in an embodiment of the present application;

[0021] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The technical solution of the present application will be described clearly and completely below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0023] At present, it is time-consuming and labor-intensive to establish a complete machine learning disease prediction model in a new region. It often needs to meet the following four conditions: first, there needs to be enough samples for training, second, the modeling ideas need to be scientific and reasonable, third, a certain amount of time is needed to adjust the model parameters, and fourth, the training model requires certain hardware support. The whole process usually takes several months. When expanding business in a new region, how to learn from the existing data results and experience of data laboratories in other regions, and how to easily and quickly build data models in the new region and provide data support is an important issue currently faced.

[0024] Based on this, the embodiments of the present application provide a cross-validation method, device and electronic device for a disease prediction model, which can model an original model and a common model based on the medical data collected in the two places, and then use each test sample set to perform corresponding tests and cross-tests on the four models in the two places, and cross-validate according to the test results to determine the substitutability of the four models, thereby providing data support and reference for model construction in other regions, so as to assist other regions in quickly building disease prediction models.

[0025] To facilitate understanding of this embodiment, a cross-validation method for a disease prediction model disclosed in an embodiment of the present application is first introduced in detail.

[0026] Figure 1 A flowchart of a cross-validation method for a disease prediction model provided in an embodiment of the present application specifically includes the following steps:

[0027] Step S102, obtaining first original medical data of a first region and second original medical data of a second region.

[0028] The first region and the second region are two regions with different geographical locations, that is, the original medical data of the populations corresponding to them are different. The original medical data includes the basic information of the patient, historical hospitalization behavior data, etc.

[0029] Step S104, based on the first original medical data and the second original medical data, determine the first original feature sample set and the first common feature sample set corresponding to the first region, and the second original feature sample set and the second common feature sample set corresponding to the second region; each sample set includes a training sample set and a test sample set.

[0030] The above original feature sample set is determined by extracting features from the original medical data. There are differences in the feature selection of original feature sample sets in different regions, that is, the features of the first original feature sample set and the second original feature sample set are not completely corresponding; while the public features in the first common feature sample set and the second common feature sample set are the same. Based on the same features, the corresponding sample data is selected from each original medical data to obtain the first common feature sample set and the second common feature sample set. Moreover, each sample set includes a training sample set and a test sample set divided according to a certain ratio.

[0031] Step S106, based on the first original feature sample set and the training sample set in the first common feature sample set, construct the first disease prediction original model and the first disease prediction common model corresponding to the first region respectively; based on the second original feature sample set and the training sample set in the second common feature sample set, construct the second disease prediction original model and the second disease prediction common model corresponding to the second region respectively.

[0032] There are many specific ways to train models based on training sample sets, which are not specifically limited here. Based on the training sample set in the first original feature sample set and the specified algorithm, a first disease prediction original model corresponding to the first region can be trained; based on the training sample set in the first common feature sample set and the specified algorithm, a first disease prediction common model corresponding to the first region can be trained; based on the training sample set in the first original feature sample set and the specified algorithm, a first disease prediction original model corresponding to the first region can be trained; based on the training sample set in the first common feature sample set and the specified algorithm, a first disease prediction common model corresponding to the first region can be trained.

[0033] Step S108, using the first original feature sample set, the first common feature sample set, the second original feature sample set and the test sample set in the second common feature sample set, to perform corresponding tests and cross tests on the first disease prediction original model, the first disease prediction common model, the second disease prediction original model and the second disease prediction common model to obtain multiple test results.

[0034] The above-mentioned corresponding test refers to testing the test sample set and the model one-to-one, such as using the test sample set in the first original feature sample set to test the first disease prediction original model (trained by the training sample set in the first original feature sample set), using the test sample set in the first common feature sample set to test the first disease prediction common model (trained by the training sample set in the first common feature sample set), using the test sample set in the second original feature sample set to test the second disease prediction original model (trained by the training sample set in the second original feature sample set), and using the test sample set in the second common feature sample set to test the second disease prediction common model (trained by the training sample set in the second original feature sample set).

[0035] Cross-testing means that the test sample set and the model do not correspond, such as using the test sample set in the second common feature sample set to test the first disease prediction public model; and using the test sample set in the first common feature sample set to test the second disease prediction public model.

[0036] Step S110, performing cross-validation based on multiple test results to determine the substitutability among the first disease prediction original model, the first disease prediction public model, the second disease prediction original model and the second disease prediction public model.

[0037] The above test results include two parameters commonly used in machine learning. One is AUC, which refers to the area under the receiver operating characteristic curve (ROC). It is an evaluation indicator for measuring the quality of a binary classification model, indicating the probability that the predicted positive case is ranked before the negative case. That is, the probability value of the prediction score of a person who will suffer from a serious disease in the future being higher than the prediction score of a person who will not suffer from a serious disease in the future by the major disease prediction model. The other parameter is the degree of improvement: the age group (5-year interval) depicts the improvement of the serious disease incidence rate of the top XX% of the population screened by the major disease prediction model compared with the serious disease incidence rate of the overall population.

[0038] By comparing the AUC and lift in multiple test results, the decrease or increase in AUC and lift can be determined. When they exceed the preset threshold, the substitutability, i.e., equivalence, of the two models can be determined.

[0039] In the cross-validation method of the disease prediction model provided in the embodiment of the present application, the original model and the common model are firstly built based on the medical data collected in the two places, and then the four models in the two places are correspondingly tested and cross-tested using various test sample sets. Cross-validation is performed based on the test results to determine the substitutability between the four models, thereby providing guidance or reference for the construction of the model in the third region.

[0040] An embodiment of the present application also provides a cross-validation method for a disease prediction model. The method is implemented on the basis of the previous embodiment. This embodiment focuses on describing the sample set determination process, model training process, testing process and verification process.

[0041] The sample set determination process is as follows:

[0042] (1) determining corresponding common features in the first original medical data and the second original medical data;

[0043] See also Figure 2 As shown in the figure, A and B are data laboratories in two regions, which have basic information of their local populations, historical hospitalization data, etc. Since the data sources in the two regions are different, the data features are not completely consistent. It is necessary to filter out common features to build the model. Finally, 236 common features were determined, including: age, gender, number of previous hospitalizations, previous hospitalization time, total previous hospitalization expenses, number of times suffering from various diseases, etc.

[0044] (2) The first original medical data and the second original medical data are respectively used as current medical data, and the following steps are performed: preprocessing the current medical data, and dividing the current medical data into a training sample set and a test sample set according to a specified ratio, so as to obtain an original feature sample set corresponding to the current medical data; based on common features, extracting corresponding sample data from the current medical data, and dividing the sample data into a training sample set and a test sample set according to a specified ratio, so as to obtain a common feature sample set corresponding to the current medical data.

[0045] The data laboratories in locations A and B each use the original features, select their own sample data, and preprocess the data, including missing value processing, outlier processing, etc. All data are divided according to the ratio of training set: validation set: test set = 3:1:1, and A1 samples (such as the first original feature sample set mentioned above) and B1 samples (such as the second original feature sample set mentioned above) are generated respectively. Similarly, the data laboratories in locations A and B select their own sample data based on the 236 identified common features, preprocess the data, divide each data set, and generate A2 samples (such as the first common feature sample set mentioned above) and B2 samples (such as the second common feature sample set mentioned above).

[0046] The model training process is as follows:

[0047] The training sample sets in the first original feature sample set, the first common feature sample set, the second original feature sample set and the second common feature sample set are respectively used as the current training sample sets, and the following operations are performed: model training is performed based on the current training sample set and a preset algorithm; the preset algorithm includes one of the following: CatBoost algorithm, LightGBM algorithm, XGBoost algorithm; model parameter adjustment is performed on the trained model to obtain a disease prediction model corresponding to the current training sample set; the model parameter adjustment includes at least one of the following: adjusting the number of iterations, adjusting the learning rate, adjusting the depth of the tree, and adjusting the L2 regularization coefficient.

[0048] Taking the above data laboratories A and B as examples, the data laboratories A and B use the CatBoost algorithm to train the model, and adjust the parameters such as iterations, learning_rate, max_depth, and l2_leaf_reg to adjust the model parameters. Figure 2 As shown, finally, the A1 model (such as the aforementioned first disease prediction original model), the B1 model (such as the aforementioned second disease prediction original model), the A2 model (such as the aforementioned first disease prediction public model), and the B2 model (such as the aforementioned second disease prediction public model) are generated.

[0049] The testing process is as follows:

[0050] (1) using a test sample set in the first original feature sample set to test the first disease prediction original model to obtain a first test result;

[0051] (2) using the test sample set in the first public feature sample set to test the first disease prediction public model to obtain a second test result;

[0052] (3) using the test sample set in the second public feature sample set to test the first disease prediction public model to obtain a third test result;

[0053] (4) using the test sample set in the first public feature sample set to test the second disease prediction public model to obtain a fourth test result;

[0054] (5) using the test sample set in the second public feature sample set to test the second disease prediction public model to obtain a fifth test result;

[0055] (6) Using the test sample set in the second original feature sample set to test the second disease prediction original model, obtaining a sixth test result; wherein each test result includes an AUC and a lift.

[0056] See also Figure 2As shown, the sample data and models of A and B are tested correspondingly and cross-tested. Experimental result 1 is the result of sample A1 on model A1, that is, the test result; experimental result 2 is the result of sample A2 on model A2, experimental result 3 is the result of sample B2 on model A2, experimental result 4 is the result of sample A2 on model B2, experimental result 5 is the result of sample B2 on model B2, and experimental result 6 is the result of sample B1 on model B1.

[0057] The verification process is as follows, see Figure 3 As shown:

[0058] Step S302, comparing the second test result with the first test result, if the corresponding AUC and lift decreases are both less than a first preset threshold, determining that the first disease prediction public model can replace the first disease prediction original model;

[0059] Step S304, comparing the fifth test result with the sixth test result, if the corresponding AUC and lift decreases are both less than the second preset threshold, it is determined that the second disease prediction public model can replace the second disease prediction original model.

[0060] The above comparison is also the comparison process between the original features and the common features: that is, the experimental result 2 is compared with the experimental result 1, and the experimental result 5 is compared with the experimental result 6. The AUC decreases by about 0.002 and the improvement decreases by about 0.02. The main reason for the decrease is that the common features have fewer region-specific features than the original features. However, the decrease in effect is very small compared to the original effect, so it can be used as a substitute for the original model.

[0061] Step S306, comparing the fourth test result with the first test result, if the corresponding AUC and lift decreases are both less than the third preset threshold, determining that the second disease prediction public model can replace the first disease prediction original model;

[0062] Step S308, comparing the third test result with the sixth test result, if the corresponding AUC and lift decreases are both less than the fourth preset threshold, it is determined that the first disease prediction public model can replace the second disease prediction original model.

[0063] The above comparison is also the model cross-comparison process: that is, experimental result 4 is compared with experimental result 1, and experimental result 3 is compared with experimental result 6. The AUC decreases by about 0.008 and the improvement decreases by about 0.05. The main reason for the decrease is the lack of some regional-specific features and the differences in the attributes and behaviors of the people in the two places. However, the decrease in effect is very small compared with the original effect, so it can be used as a substitute for the original model.

[0064] Guided by the above verification results, when expanding business in new areas, you can quickly build models for new regions. The specific process is as follows:

[0065] (1) If the data samples of the third region are sufficient, based on the common features, a third common feature sample set of the third region is obtained; the third common feature sample set is used to perform model training to obtain a target disease prediction model corresponding to the third region.

[0066] Specifically, if the data samples in area C are sufficient, the data quality is complete, and there is a modeling foundation, we can use cross-validation modeling ideas and general codes (including sample selection, feature screening, data processing, algorithm selection, effect evaluation and other modules) to quickly establish and deploy machine learning models to provide data support for the business.

[0067] (2) If the data samples of the third region are insufficient, the model with higher model accuracy between the first disease prediction public model and the second disease prediction public model is used as the target disease prediction model corresponding to the third region.

[0068] If area C has insufficient samples, poor data quality, and no modeling basis, you can use models from other areas and deploy them directly to quickly provide data support for your business. For example, choose the best model among A2 and B2 and use it.

[0069] The cross-validation method of the disease prediction model provided in the embodiment of the present application first builds an original model and a public model based on the medical data collected in the two places, such as the first disease prediction original model and the first disease prediction public model corresponding to the first region, and the second disease prediction original model and the second disease prediction public model corresponding to the second region; then, each test sample set is used to perform corresponding tests and cross-tests on the above four models of the two places to obtain six test results, and cross-validation is performed based on the six test results to determine the substitutability of the four models, thereby providing guidance or reference for model construction in the third region.

[0070] Based on the above method embodiment, the present application embodiment also provides a cross-validation device for a disease prediction model, see Figure 4 As shown, the device comprises:

[0071] The data acquisition module 402 is used to acquire the first original medical data of the first region and the second original medical data of the second region; the sample set determination module 404 is used to determine the first original feature sample set and the first common feature sample set corresponding to the first region, and the second original feature sample set and the second common feature sample set corresponding to the second region based on the first original medical data and the second original medical data; each sample set includes a training sample set and a test sample set; the model construction module 406 is used to construct the first disease prediction original model and the first disease prediction common model corresponding to the first region respectively based on the training sample sets in the first original feature sample set and the first common feature sample set; based on the second original feature sample set and the second common feature sample set The training sample set in the sample set is used to construct the second disease prediction original model and the second disease prediction public model corresponding to the second region respectively; the testing module 408 is used to perform corresponding tests and cross-tests on the first disease prediction original model, the first disease prediction public model, the second disease prediction original model and the second disease prediction public model based on the first original feature sample set, the first public feature sample set, the second original feature sample set and the test sample set in the second public feature sample set to obtain multiple test results; the verification module 410 is used to perform cross-validation based on multiple test results to determine the substitutability between the first disease prediction original model, the first disease prediction public model, the second disease prediction original model and the second disease prediction public model.

[0072] The above-mentioned sample set determination module 404 is also used to determine the corresponding common features in the first original medical data and the second original medical data; the first original medical data and the second original medical data are respectively used as the current medical data, and the following steps are performed: pre-processing the current medical data, and dividing the current medical data into a training sample set and a test sample set according to a specified ratio, so as to obtain the original feature sample set corresponding to the current medical data; based on the common features, extracting the corresponding sample data from the current medical data, and dividing the sample data into a training sample set and a test sample set according to a specified ratio, so as to obtain the common feature sample set corresponding to the current medical data.

[0073] The above-mentioned model building module 406 is also used to use the training sample sets in the first original feature sample set, the first common feature sample set, the second original feature sample set and the second common feature sample set as the current training sample set, and perform the following operations: model training based on the current training sample set and a preset algorithm; the preset algorithm includes one of the following: CatBoost algorithm, LightGBM algorithm, XGBoost algorithm; model parameter adjustment processing is performed on the trained model to obtain a disease prediction model corresponding to the current training sample set; the model parameter adjustment processing includes at least one of the following: adjusting the number of iterations, adjusting the learning rate, adjusting the depth of the tree, and adjusting the L2 regularization coefficient.

[0074] The above-mentioned testing module 408 is also used to use the test sample set in the first original feature sample set to test the first disease prediction original model to obtain a first test result; use the test sample set in the first common feature sample set to test the first disease prediction common model to obtain a second test result; use the test sample set in the second common feature sample set to test the first disease prediction common model to obtain a third test result; use the test sample set in the first common feature sample set to test the second disease prediction common model to obtain a fourth test result; use the test sample set in the second common feature sample set to test the second disease prediction common model to obtain a fifth test result; use the test sample set in the second original feature sample set to test the second disease prediction original model to obtain a sixth test result; wherein each test result includes AUC and lift.

[0075] Furthermore, the above-mentioned verification module 410 is also used to compare the second test result with the first test result. If the corresponding AUC and lift decrease rates are both less than the first preset threshold, it is determined that the first disease prediction public model can replace the first disease prediction original model; compare the fifth test result with the sixth test result. If the corresponding AUC and lift decrease rates are both less than the second preset threshold, it is determined that the second disease prediction public model can replace the second disease prediction original model; compare the fourth test result with the first test result. If the corresponding AUC and lift decrease rates are both less than the third preset threshold, it is determined that the second disease prediction public model can replace the first disease prediction original model; compare the third test result with the sixth test result. If the corresponding AUC and lift decrease rates are both less than the fourth preset threshold, it is determined that the first disease prediction public model can replace the second disease prediction original model.

[0076] Furthermore, the above-mentioned device also includes a new model determination module, which is used to obtain a third common feature sample set for the third region based on common features if the data samples of the third region are sufficient; apply the third common feature sample set to model training to obtain a target disease prediction model corresponding to the third region; if the data samples of the third region are insufficient, use the model with higher model accuracy between the first disease prediction public model and the second disease prediction public model as the target disease prediction model corresponding to the third region.

[0077] The device provided in the embodiment of the present application has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the embodiment of the device, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0078] The present application also provides an electronic device, such as Figure 5As shown, it is a schematic diagram of the structure of the electronic device, wherein the electronic device includes a processor 51 and a memory 50, the memory 50 stores computer executable instructions that can be executed by the processor 51, and the processor 51 executes the computer executable instructions to implement the above method.

[0079] exist Figure 5 In the illustrated embodiment, the electronic device further includes a bus 52 and a communication interface 53 , wherein the processor 51 , the communication interface 53 and the memory 50 are connected via the bus 52 .

[0080] Among them, the memory 50 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 53 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used. The bus 52 can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 52 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0081] The processor 51 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 51. The above processor 51 can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined and executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor 51 reads the information in the memory and completes the steps of the method of the above embodiment in combination with its hardware.

[0082] An embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the above method. The specific implementation can be found in the aforementioned method embodiment, which will not be repeated here.

[0083] The computer program products of the methods, devices, and electronic devices provided in the embodiments of the present application include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be found in the method embodiments, which will not be repeated here.

[0084] Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0085] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0086] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0087] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A cross-validation method for a disease prediction model, characterized in that: The method comprises: Acquire first original medical data of a first region and second original medical data of a second region; Based on the first original medical data and the second original medical data, determining a first original feature sample set and a first common feature sample set corresponding to the first region, and a second original feature sample set and a second common feature sample set corresponding to the second region, including: determining corresponding common features in the first original medical data and the second original medical data; taking the first original medical data and the second original medical data as current medical data, respectively, and performing the following steps: preprocessing the current medical data, and dividing the current medical data into a training sample set and a test sample set according to a specified ratio, to obtain an original feature sample set corresponding to the current medical data; extracting corresponding sample data from the current medical data based on the common features, and dividing the sample data into a training sample set and a test sample set according to a specified ratio, to obtain a common feature sample set corresponding to the current medical data; Based on the first original feature sample set and the training sample set in the first common feature sample set, a first disease prediction original model and a first disease prediction common model corresponding to the first region are constructed respectively; based on the second original feature sample set and the training sample set in the second common feature sample set, a second disease prediction original model and a second disease prediction common model corresponding to the second region are constructed respectively, including: taking the first original feature sample set, the first common feature sample set, the second original feature sample set and the training sample set in the second common feature sample set as the current training sample set, and performing the following operations: performing model training based on the current training sample set and a preset algorithm; the preset algorithm includes one of the following: CatBoost algorithm, LightGBM algorithm, XGBoost algorithm; performing model parameter adjustment processing on the trained model to obtain a disease prediction model corresponding to the current training sample set; the model parameter adjustment processing includes at least one of the following: adjusting the number of iterations, adjusting the learning rate, adjusting the depth of the tree, and adjusting the L2 regularization coefficient; Apply the test sample set in the first original feature sample set to test the first disease prediction original model to obtain a first test result; apply the test sample set in the first common feature sample set to test the first disease prediction common model to obtain a second test result; apply the test sample set in the second common feature sample set to test the first disease prediction common model to obtain a third test result; apply the test sample set in the first common feature sample set to test the second disease prediction common model to obtain a fourth test result; apply the test sample set in the second common feature sample set to test the second disease prediction common model to obtain a fifth test result; apply the test sample set in the second original feature sample set to test the second disease prediction original model to obtain a sixth test result; wherein each test result includes AUC and lift; The second test result is compared with the first test result. If the corresponding AUC and lift decrease rates are both less than the first preset threshold, it is determined that the first disease prediction public model can replace the first disease prediction original model; the fifth test result is compared with the sixth test result. If the corresponding AUC and lift decrease rates are both less than the second preset threshold, it is determined that the second disease prediction public model can replace the second disease prediction original model; the fourth test result is compared with the first test result. If the corresponding AUC and lift decrease rates are both less than the third preset threshold, it is determined that the second disease prediction public model can replace the first disease prediction original model; the third test result is compared with the sixth test result. If the corresponding AUC and lift decrease rates are both less than the fourth preset threshold, it is determined that the first disease prediction public model can replace the second disease prediction original model.

2. The method according to claim 1, characterized in that The method further comprises: If the data samples of the third region are sufficient, a third common feature sample set of the third region is obtained based on the common features; the third common feature sample set is used to perform model training to obtain a target disease prediction model corresponding to the third region.

3. The method according to claim 1, characterized in that If the data samples of the third region are insufficient, the model with higher model accuracy between the first disease prediction public model and the second disease prediction public model is used as the target disease prediction model corresponding to the third region.

4. A cross-validation device for a disease prediction model, characterized in that: The device comprises: A data acquisition module, used for acquiring first original medical data of a first region and second original medical data of a second region; The sample set determination module is used to determine the first original feature sample set and the first common feature sample set corresponding to the first region, and the second original feature sample set and the second common feature sample set corresponding to the second region based on the first original medical data and the second original medical data, including: determining the common features corresponding to the first original medical data and the second original medical data; taking the first original medical data and the second original medical data as current medical data, respectively, and performing the following steps: preprocessing the current medical data, and dividing the current medical data into a training sample set and a test sample set according to a specified ratio to obtain the original feature sample set corresponding to the current medical data; extracting corresponding sample data from the current medical data based on the common features, and dividing the sample data into a training sample set and a test sample set according to a specified ratio to obtain the common feature sample set corresponding to the current medical data; A model building module, used to build a first disease prediction original model and a first disease prediction public model corresponding to the first region based on the first original feature sample set and the training sample set in the first common feature sample set, and to build a second disease prediction original model and a second disease prediction public model corresponding to the second region based on the training sample set in the second original feature sample set and the second common feature sample set, including: taking the first original feature sample set, the first common feature sample set, the second original feature sample set and the training sample set in the second common feature sample set as the current training sample set, and performing the following operations: performing model training based on the current training sample set and a preset algorithm; the preset algorithm includes one of the following: CatBoost algorithm, LightGBM algorithm, XGBoost algorithm; performing model parameter adjustment processing on the trained model to obtain a disease prediction model corresponding to the current training sample set; the model parameter adjustment processing includes at least one of the following: adjusting the number of iterations, adjusting the learning rate, adjusting the depth of the tree, and adjusting the L2 regularization coefficient; A testing module, used to apply the test sample set in the first original feature sample set to test the first disease prediction original model to obtain a first test result; apply the test sample set in the first common feature sample set to test the first disease prediction common model to obtain a second test result; apply the test sample set in the second common feature sample set to test the first disease prediction common model to obtain a third test result; apply the test sample set in the first common feature sample set to test the second disease prediction common model to obtain a fourth test result; apply the test sample set in the second common feature sample set to test the second disease prediction common model to obtain a fifth test result; apply the test sample set in the second original feature sample set to test the second disease prediction original model to obtain a sixth test result; wherein each test result includes AUC and lift; A verification module is used to compare the second test result with the first test result. If the corresponding AUC and lift decrease rates are both less than a first preset threshold, it is determined that the first disease prediction public model can replace the first disease prediction original model; compare the fifth test result with the sixth test result. If the corresponding AUC and lift decrease rates are both less than the second preset threshold, it is determined that the second disease prediction public model can replace the second disease prediction original model; compare the fourth test result with the first test result. If the corresponding AUC and lift decrease rates are both less than a third preset threshold, it is determined that the second disease prediction public model can replace the first disease prediction original model; compare the third test result with the sixth test result. If the corresponding AUC and lift decrease rates are both less than the fourth preset threshold, it is determined that the first disease prediction public model can replace the second disease prediction original model.

5. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method according to any one of claims 1 to 3.

Images