A method for constructing a disease recognition model for medical data based on multi-task self-supervised learning

Abstract

The present invention discloses a method for constructing a disease recognition model of medical data based on multi-task self-supervised learning, which belongs to the field of laboratory medicine and disease recognition technology, and includes: S1, extracting medical data of a target population to form an original sample data set; S2, preprocessing the original sample data set; S3, constructing a missing value filling model and a sample data homology test model using the preprocessed missing value filling data set and the sample data homology analysis data set; S4, using a multi-task self-supervised learning model, taking missing value filling and sample data homology prediction as auxiliary tasks, taking known disease screening as a downstream task, and realizing disease risk assessment of the original sample data through transfer learning. The present invention adopts the above-mentioned method for constructing a disease recognition model of medical data based on multi-task self-supervised learning, which can extract more comprehensive and accurate features to achieve the purpose of improving the generalization performance of downstream tasks.

Description

Technical Field

[0001] The present invention relates to the field of laboratory medicine and disease identification technology, and in particular to a method for constructing a medical data disease identification model based on multi-task self-supervised learning. Background Art

[0002] Disease recognition models built based on medical data usually only use labeled data for supervised learning, or rely on a single self-supervised auxiliary task. However, these methods fail to fully tap the rich information in unlabeled data, nor effectively coordinate the differences and synergies between multiple tasks. For example, labeled data may be insufficient or of poor quality, resulting in poor generalization performance of the trained model, and annotated data increases costs. A single auxiliary task may not cover all features and relationships in the results of routine blood tests, and may be inconsistent or irrelevant to downstream target tasks, thereby reducing the effectiveness of the model. In addition, existing deep learning models may also lack the ability to handle factors such as uncertainty, outliers, and noise in medical data, resulting in unstable and unreliable performance of the model when faced with complex and changing actual situations.

[0003] Therefore, the problem faced by existing technologies is how to cleverly use unlabeled or low-label data to improve the generalization performance and robustness of deep learning models in medical data disease identification. This requires not only the fusion of multi-source information to more comprehensively characterize blood routine features, but also the construction of an intelligent multi-task self-supervised learning framework to collaboratively learn different tasks to achieve the goal of improving model performance. Summary of the invention

[0004] The purpose of the present invention is to provide a method for constructing a medical data disease recognition model based on multi-task self-supervised learning. Through self-supervised learning, the missing value filling and sample data homology prediction auxiliary tasks are used to pre-train unlabeled or few-labeled medical data, thereby extracting more comprehensive and accurate features to achieve the purpose of improving the generalization performance of downstream tasks.

[0005] To achieve the above object, the present invention provides a method for constructing a disease recognition model for medical data based on multi-task self-supervised learning, comprising the following steps:

[0006] S1, extract the medical data of the target population to form the original sample data set;

[0007] The sample data sets include missing value filling data sets, sample data homology analysis data sets, and disease screening data sets;

[0008] S2. Clean, standardize, and normalize the original sample data set to remove outliers and noise, so that the data meets the requirements of the disease recognition model input, and obtain the preprocessed missing value filling data set, sample data homology analysis data set, and disease screening data set;

[0009] S3, using the preprocessed missing value filling data set and sample data homology analysis data set to respectively construct a missing value filling model and a sample data homology test model;

[0010] S4. Based on the preprocessed disease screening dataset, a multi-task self-supervised learning model is used to fill in missing values ​​and test sample data homology as auxiliary tasks, and known disease screening as a downstream task, and disease risk assessment of the original sample dataset is achieved through transfer learning.

[0011] Preferably, in step S1, the missing value filling data set is to collect medical data of the target population from hospitals or institutions, and randomly mask the medical data of the target population to form a data set with missing values;

[0012] In the sample data homology analysis data set, the positive data is the combination of the first data and the nth data, indicating that the two data belong to the sample data of the same person; the negative sample is the mirror image of the first data and the nth data, indicating that the two data do not belong to the sample data of the same person; n is the data volume threshold.

[0013] Preferably, in step S3, the missing value filling model is constructed by using a regression model based on a fully connected neural network, taking data with missing values ​​as input and a complete data set as output; the specific operation is:

[0014] The input data X with missing values ​​is {x1,x2,...,x n}, which contains n samples, each sample x i Contains m features; let the missing feature be x i,j , that is, the jth feature of the i-th sample is missing, then the regression model is expressed as:

[0015]

[0016] Where, f(·; θ) is the regression model; θ is the model parameter;

[0017] The training goal of the regression model is to minimize the prediction error, that is:

[0018]

[0019] Where L(·,·) is the loss function;

[0020] By training the regression model, we can obtain a complete data set after missing values ​​are filled. Right now in, represents the samples after missing values ​​are filled.

[0021] Preferably, in step S3, the sample data homology test model is constructed using twin neural network technology, and the specific operations are:

[0022] Assume that the two input data sets are W and Z, that is, {w1,w2,...,w n} and {z1,z2,...,z n}, where w c and z c They correspond to the cth sample in the two data sets respectively;

[0023] w c and z c Combine the sample pairs required to form the twin neural network and mark the homologous labels. If w c and z c Sample data belonging to the same person is marked as 1, otherwise it is marked as 0;

[0024] The Siamese network model consists of two sub-networks that share parameters and a distance metric layer;

[0025] The two sub-networks g(·,φ) are implemented using a fully connected neural network structure, where φ refers to the model parameters;

[0026] The distance metric layer is implemented in the form of Euclidean distance, namely:

[0027] d c =d(g(w c ,φ),g(z c ,φ)) (3)

[0028] Among them, d c for similarity;

[0029] Finally, the binary cross entropy loss function is used to train the twin network model to obtain the homology prediction results for each pair of samples.

[0030] Therefore, the present invention adopts the above-mentioned method for constructing a medical data disease recognition model based on multi-task self-supervised learning, and its technical effects are as follows:

[0031] (1) By introducing "missing value filling combined with sample data homology prediction" as an auxiliary task, the model's mining of intra-individual and inter-individual differences in sample information is improved from both horizontal and vertical dimensions, making better use of the complex associations of medical data and thus extracting more effective feature information;

[0032] (2) It solves the technical problems in existing technologies such as insufficient feature extraction of disease screening based on medical data and poor model generalization ability, and has high superiority and innovation.

[0033] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 The present invention is a flowchart of a method for constructing a medical data disease recognition model based on multi-task self-supervised learning. DETAILED DESCRIPTION

[0035] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0036] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0037] Embodiment 1

[0038] like Figure 1 As shown, it is a flow chart of a method for constructing a medical data disease recognition model based on multi-task self-supervised learning in the present invention. This embodiment adopts a multi-task self-supervised learning method to improve the generalization performance of the brucellosis recognition model based on routine blood data. The method includes the following steps:

[0039] Step 1: Extract blood routine test data related to brucellosis and construct a data set.

[0040] The data set includes a positive experimental group and a negative control group, which are composed of blood routine test data of brucellosis patients and non-brucellosis patients, respectively. In order to ensure the validity of the data, all samples are selected from the first visit data, that is, blood routine samples without brucellosis treatment.

[0041] Step 2: Preprocess the data to make it more suitable for subsequent model training and prediction, and improve the accuracy and efficiency of the model.

[0042] Preprocessing includes the following:

[0043] 2a) Remove outliers: Delete data points that are significantly different from the normal range or inconsistent with other samples to prevent them from affecting the learning effect of the model.

[0044] 2b) Standardization: The Z-score standardization method is used to standardize the data. The mean of the feature is subtracted from each data point and then divided by the standard deviation of the feature to convert it into a normal distribution with a mean of 0 and a standard deviation of 1, eliminating the dimensional differences between different features.

[0045] Step 3: Train two auxiliary task models, namely the missing value filling model and the sample data homology test model (blood routine homology analysis model). The auxiliary task model is used to learn the feature extraction rules of blood routine data from other related data and transfer them to the brucellosis screening model to improve its generalization performance. The specific training steps are as follows:

[0046] 3a) Missing value filling model: This model is a regression model built using a fully connected neural network, which is used to learn the missing value filling rules from large-scale blood routine test data collected from multiple hospitals or institutions. The specific steps are as follows:

[0047] 3a.1) Collect blood routine test data, a total of 4,055,132 samples, each sample has 24-dimensional features such as age, gender, and blood routine test indicators.

[0048] 3a.2) Randomly mask the data. First, randomly determine the column index of the masked data according to the masking probability of 30%, 50%, 60%, 70%, and 90%, and then mask to generate data sets with different missing values.

[0049] 3a.3) Use a fully connected neural network to build a regression model, take a dataset with missing values ​​as input, and use a complete dataset as output to train a missing value filling model under different occlusion probabilities. The regression model consists of a 24-dimensional input layer, 6 fully connected layers with 256 neurons, each followed by a Dropout layer, a 24-dimensional output layer, and a residual layer. The Dropout layer is a random inactivation layer set to prevent the model from overfitting; the residual layer is a jump connection layer set to prevent the model from falling into a local optimum. The neural network is compiled using the RMSprop optimizer and the mean square error loss function, and the model training is performed with epochs set to 100 and batch_size set to 12000.

[0050] 3b) Blood routine homology analysis model: This model is a classification model built using a twin neural network, which is used to learn the judgment criteria of whether blood routine data belongs to the same person from the blood routine homology analysis data set. The specific steps are as follows:

[0051] 3b.1) Collect blood routine homologous analysis data sets and construct positive sample pairs and negative sample pairs. A positive sample pair is a combination of the first data and the nth data (n is the data volume threshold), indicating that the two data belong to the blood routine of the same person; a negative sample pair is a combination of the first data and the nth data in a mirror-flipped manner, indicating that the two data do not belong to the blood routine of the same person. In the end, there were 1,232,591 positive and negative samples.

[0052] 3b.2) Use the twin neural network to build a blood routine homology prediction model, which includes two feature extractors with shared parameters and a similarity output layer. The feature extractor consists of a 24-dimensional input layer, four fully connected layers with 128 neurons, and each layer is followed by a Dropout layer. The similarity output layer consists of a fully connected layer with Euclidean distance and Sigmoid activation function. The process of the blood routine homology prediction model is: first, split the sample pairs in the data set into two 24-dimensional single samples, send them to two input layers and feature extractors respectively, and obtain two 128-dimensional feature vectors; then perform Euclidean distance calculation to obtain the similarity between feature vectors, and output a value between 0 and 1 through the fully connected layer to indicate whether they belong to the same person. Use the binary_crossentropy loss function and RMSprop optimizer to compile the model, and set epochs to 100 and batch_size to 50000 to train and optimize the model.

[0053] Step 4: Use different auxiliary tasks to build a brucellosis screening model based on transfer learning.

[0054] Different auxiliary tasks include no auxiliary task (fully connected neural network), missing value filling under different occlusion probabilities, blood routine homology prediction, and a multi-task of missing value filling under different occlusion probabilities combined with blood routine homology prediction.

[0055] The brucellosis screening model is a classifier that is constructed by adding an output layer after the feature extractor of the auxiliary task model. It is used to fine-tune and verify the preprocessed brucellosis dataset. The specific steps are as follows:

[0056] 4a) For a single auxiliary task, the feature extractor of the auxiliary task model is directly extracted, and an output layer is added thereto to form a brucellosis screening classifier.

[0057] 4b) For multi-task, the feature extractor of missing value filling and blood routine homology prediction model is extracted, the feature vector is concatenated through the concatenate layer, and the output layer is added thereafter to form the brucellosis screening model.

[0058] 4c) Using the brucellosis dataset preprocessed in step 2, the model was fine-tuned using the binary_crossentropy loss function, RMSprop optimizer, epochs set to 100, batch_size set to 250, and the model parameters were optimized. The model performance was evaluated on the validation set. The evaluation index results are shown in Table 1.

[0059] Table 1 Evaluation indicators of brucellosis screening model

[0060]

[0061] The evaluation indicators are AUC, accuracy, specificity, and recall:

[0062] The calculation formulas for precision, specificity and recall are shown in equations (1), (2) and (3), where TP refers to true positive class predicted positive class, FP refers to true negative class predicted positive class, TN refers to true negative class predicted negative class, and FN refers to true positive class predicted negative class.

[0063]

[0064]

[0065]

[0066] The AUC indicator is the area under the ROC curve, which can intuitively reflect the classification ability expressed by the ROC curve. The larger the value, the better the classifier performance. Conversely, the smaller the AUC value, the worse the classifier performance.

[0067] Step 5: Use external datasets to verify the generalization performance of the brucellosis screening model.

[0068] An external dataset was obtained from a hospital. The external dataset contained routine blood data of non-brucellosis patients and brucellosis patients. After preprocessing, the brucellosis screening model based on different auxiliary tasks was used for prediction. The AUC, accuracy, specificity, sensitivity and other indicators of the prediction results were calculated to evaluate the generalization performance of the model. The results are shown in Tables 2, 3, 4 and 5.

[0069] Table 2 Evaluation indicators of external test set 1

[0070]

[0071] Table 3 Evaluation indicators of external test set 2

[0072]

[0073] Table 4 Evaluation indicators of external test set 3

[0074]

[0075]

[0076] Table 5 Evaluation indicators of external test set 4

[0077]

[0078] The experimental results show that the present invention adopts "blood routine homology prediction combined with missing value filling under 90% occlusion probability" as an auxiliary task, which can better extract the characteristic information of blood routine data, thereby improving the performance of the disease screening model.

[0079] For example, on external test set 2, the AUC value of the auxiliary task combination is 0.0506 higher than that of the fully connected model and 0.0197 higher than the highest AUC in a single auxiliary task. On external test set 4, the AUC value of the auxiliary task combination is 0.0559 higher than that of the fully connected model and 0.0169 higher than the highest AUC in a single auxiliary task. This illustrates the advantages of the combination on different test sets, indicating that in the brucellosis screening task, the auxiliary task of "blood routine homology prediction combined with missing value filling under occlusion probability of 90%" can help the model learn and understand blood routine data and improve generalization performance. It should be noted that on external test sets 1 and external test sets 3, although the AUC value of the auxiliary task combination is better than that of the fully connected neural network, it is not much different from the highest AUC value in a single auxiliary task. This may suggest that in some cases, the combination does not significantly surpass a single auxiliary task, which may be affected by the characteristics of the data set or other factors. But even so, on these two test sets, the combination still shows high prediction performance and can assist the model in disease screening.

[0080] Therefore, the present invention adopts the above-mentioned method for constructing a medical data disease identification model based on multi-task self-supervised learning. Through self-supervised learning, it uses missing value filling and sample data homology prediction auxiliary tasks to pre-train unlabeled or few-labeled medical data, thereby extracting more comprehensive and accurate features to achieve the purpose of improving the generalization performance of downstream tasks.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for constructing a disease recognition model for medical data based on multi-task self-supervised learning, characterized in that: The following steps are involved: S1, extract the medical data of the target population to form the original sample data set; The sample data sets include missing value filling data sets, sample data homology analysis data sets, and disease screening data sets; S2. Clean, standardize, and normalize the original sample data set to remove outliers and noise, and obtain the preprocessed missing value filling data set, sample data homology analysis data set, and disease screening data set; S3, using the preprocessed missing value filling data set and sample data homology analysis data set to respectively construct a missing value filling model and a sample data homology test model; S4. Based on the preprocessed disease screening dataset, a multi-task self-supervised learning model is used to fill missing values ​​and sample data homology test as auxiliary tasks, and known disease screening as a downstream task, and disease risk assessment of the original sample dataset is achieved through transfer learning; In step S3, the missing value filling model is constructed by using a regression model based on a fully connected neural network, taking data with missing values ​​as input and a complete data set as output; the specific operations are: The input data X with missing values ​​is {x1,x2,...,x n }, which contains n samples, each sample x i Contains m features; let The missing feature is x i,j , that is, the jth feature of the i-th sample is missing, then the regression model is expressed as: Where, f(·; θ) is the regression model; θ is the model parameter; The training goal of the regression model is to minimize the prediction error, that is: Where L(·,·) is the loss function; By training the regression model, we can obtain a complete data set after missing values ​​are filled. Right now in, represents the samples after missing values ​​are filled.

2. According to claim 1, a method for constructing a medical data disease recognition model based on multi-task self-supervised learning is characterized in that: In step S1, the missing value filling data set is to collect medical data of the target population from hospitals or institutions, and randomly mask the medical data of the target population to form a data set with missing values; In the sample data homology analysis data set, the positive data is the combination of the first data and the nth data, indicating that the two data belong to the sample data of the same person; the negative sample is the mirror image of the first data and the nth data, indicating that the two data do not belong to the sample data of the same person; n is the data volume threshold.

3. The method for constructing a medical data disease recognition model based on multi-task self-supervised learning according to claim 1, characterized in that: In step S3, the sample data homology test model is constructed using the twin neural network technology, and the specific operations are as follows: Assume that the two input data sets are W and Z, that is, {w1,w2,...,w n } and {z1,z2,...,z n }, where w c and z c They correspond to the cth sample in the two data sets respectively; w c and z c Combine the sample pairs required to form the twin neural network and mark the homologous labels. If w c and z c Sample data belonging to the same person is marked as 1, otherwise it is marked as 0; The Siamese network model consists of two sub-networks that share parameters and a distance metric layer; The two sub-networks g(·,φ) are implemented using a fully connected neural network structure, where φ refers to the model parameters; The distance metric layer is implemented in the form of Euclidean distance, namely: d c =d(g(w c ,φ),g(z c ,φ)) (3) Among them, d c for similarity; Finally, the binary cross entropy loss function is used to train the twin network model to obtain the homology prediction results for each pair of samples.

Images