A chronic disease complication early risk warning method based on small sample deep learning

Abstract

The application discloses a chronic disease complication early risk warning method based on small sample deep learning, and early risk warning tasks include two parts of collecting and processing chronic disease patient data and predicting model construction. For data collection and processing, Logistics regression analysis and SVM-RFE methods are used to screen feature factors of biochemical examination data of different complications, and missForest method is used to carry out data interpolation on features with many missing values; BioBERT model is used to carry out text vector embedding on outpatient and inpatient documents of electronic medical records. Then, the obtained vectors and complication labels are connected to form data forms. For the problem of sample data class imbalance, HTTE oversampling of fused condition entropy and TF-IDF is used to solve the small sample problem. For the prediction model construction part, a DBN model based on particle swarm optimization is used to construct a multi-label classification model in a classifier chain mode, and complications are predicted in the early stage of chronic disease.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of chronic disease complication prediction, and particularly relates to a chronic disease complication early risk warning method based on small sample deep learning. BACKGROUND

[0002] Chronic disease, full name chronic non-communicable disease, is a general term for a class of diseases with insidious onset, long course and persistent illness. Common ones are cardiovascular and cerebrovascular diseases, chronic obstructive pulmonary disease (chronic bronchitis, emphysema, etc.), diabetes, chronic respiratory disease, etc. Long-term accumulation will cause damage to important organs such as heart, brain, kidney and eyes. Clinical data shows that 30% to 40% of patients will have at least one complication in the later stage of chronic disease, and once the complication occurs, it is difficult to reverse by drug treatment, which is easy to cause disability, and the medical expenses are extremely expensive, increasing the economic burden of society and family. "Early diagnosis, early treatment, early standard, early benefit", the probability of complication can be greatly reduced. In order to better protect the safety of patients' life and property, avoid further damage to patients' body system, and reduce the possibility of death and disability caused by chronic disease complications, it is of practical significance to use deep learning technology to construct chronic disease complication early risk warning model.

[0003] At present, the research work of chronic disease complications in China is concentrated in the field of auxiliary diagnosis, that is, using the test data of chronic disease patients at the present stage, constructing machine learning diagnosis model, and assisting doctors to judge whether they have a certain complication.

[0004] Among the related 10,000 articles, 87 ML models are included. Neural network model is the most commonly used model for complication prediction, and random forest performs best in microvascular and macrovascular related complication prediction. The method used for auxiliary diagnosis is to find the characteristic factors related to complications, such as body mass index, age, gender, etc., to form a data set such as the input of machine learning model, train the classification model, find the appropriate parameters, that is, the classification decision boundary, and complete the judgment of complications.

[0005] However, the existing prediction scenarios, data processing and models have certain problems. First, the onset time of complications is 3-5 years after the onset of chronic diseases. Only the construction of a prediction model for auxiliary diagnosis can provide efficient inquiry assistance for doctors to a certain extent, but it cannot provide better early warning to patients, such as prevention program development, auxiliary medication and the like for possible complications. Secondly, the existing models mostly use biochemical examination data to form data vector form, or use "1" and "0" to represent whether a certain symptom or lesion feature is present. With the development of the field of natural language processing, more and more researchers have paid attention to extracting more valuable information from medical documents, so the processing of electronic medical records is no longer limited to single data quantization. While retaining the semantic information of the document, attention is paid to the correlation between words in the document, which can more fully extract the knowledge available for classification and realize multi-dimensional and overall expression of the semantic information. However, at the same time, the classification model of machine learning cannot well fit the decision boundary, and it is necessary to introduce a deep learning model to enhance the learning ability of the model. In addition, in the real scene, the positive and negative sample ratio of patients is unbalanced, and there is a class imbalance problem. If the negative samples are lost, there is a small sample problem; if the negative samples are oversampled, a simple sampling method may cause overfitting and other problems. At the same time, the problem of a large number of missing values of the features also needs to be solved. SUMMARY

[0006] TECHNICAL PROBLEM

[0007] The application provides a chronic disease complication early risk warning method based on small sample deep learning, which uses Logistics regression analysis and SVM-RFE method to screen the biochemical examination data of different complications, uses missForest method to perform data interpolation on the features with a large number of missing values, and uses BioBERT model to perform text vector embedding on the outpatient and inpatient documents of electronic medical records. For the problem of sample data class imbalance, HTTE oversampling is used to solve the small sample problem by fusing the conditional entropy and TF-IDF. Based on the DBN model optimized by the particle swarm algorithm, a multi-label classification model is constructed in the form of a classifier chain, so that scientific and accurate complication risk prediction can be realized in the early stage of chronic disease.

[0008] TECHNICAL SCHEME

[0009] The present application is just for the problems existing in the prior art, and provides a chronic disease complication early risk warning method based on small sample deep learning, which comprises the following steps: first, collecting outpatient and inpatient biochemical examination data and electronic medical record documents of chronic disease complication patients during the first visit to the hospital system; then, performing Logistics analysis and SVM-RFE feature screening on the biochemical examination data to determine the feature factor combination, constructing a feature table, and using missForest to perform data interpolation of missing values. At the same time, the electronic medical record documents are processed, the Biobert model is used for text vector embedding, and principal component analysis dimension reduction is performed on the result vector to splice the data format [biochemical examination vector, document embedding vector, and complication single label]. For the unbalanced data set, the HTTE oversampling method fusing conditional entropy and TF-IDF is used to create a minority class sample set, and the training set and test set required for model training are divided; a deep belief network based on particle swarm algorithm optimization is established, the training set data is input into the model for training, and a single label trained model is obtained; label correlation test is performed to determine the classifier chain structure, a multi-label data set is constructed, and the single label trained model is input into the model for tuning; finally, the test patient data is input into the model obtained by tuning, and whether the target patient will suffer from complications in the middle and later stages of chronic disease is predicted. The method of the present application considers the influence of biochemical examination data and electronic medical record documents on the prediction result in the early risk warning of chronic disease, comprehensively and deeply processes the rough data set in the real scene, optimizes the selection and construction method of the prediction model combined with the characteristics of the data set, can realize scientific and accurate prediction, and improves the accuracy of the early risk warning of chronic disease complications.

[0010] In order to achieve the above-mentioned purpose, the technical scheme adopted by the present application is: a chronic disease complication early risk warning method based on small sample deep learning, comprising the following steps:

[0011] 1) Data collection: collecting outpatient and inpatient biochemical examination data and electronic medical record documents of chronic disease complication patients during the first visit to the hospital system, and obtaining a disease dictionary strongly related to complications according to the document content, and excluding patients who suffer from related complications during the first visit;

[0012] 2) Biochemical examination data processing: processing the biochemical examination data obtained in step 1), using Logistics analysis to select features in a statistical sense, and then using SVM-RFE to further perform feature sorting and feature screening in a model sense. After determining the feature factor combination, a feature table is constructed, and missForest is used to perform data interpolation of missing values.

[0013] 3) Electronic medical record document processing: the electronic medical record document obtained in step 1) is processed, text vector embedding is carried out by using a Biobert model, and principal component analysis dimension reduction is carried out on the result vector.

[0014] 4) Class imbalance processing: the biochemical examination vector, the document embedding vector and the complication label obtained in steps 2) and 3) are spliced to form a data format [biochemical examination vector, document embedding vector and complication single label]. For unbalanced data, an HTTE oversampling method fusing conditional entropy and TF-IDF is used to create a minority class sample set, solve the small sample problem, obtain a class balanced data set, and divide to obtain a training set and a test set required for model training.

[0015] 5) Model establishment: a deep belief network based on particle swarm algorithm optimization is established, and the number of neurons in the hidden layer of the DBN and the learning rate are optimized. The model includes six layers of an input layer, four hidden layers and an output layer, the input layer includes three aspects of vector combination [biochemical examination vector, document embedding vector and complication single label]; the number of neurons contained in the hidden layer is obtained by particle swarm algorithm optimization. The label correlation test is carried out, the classifier chain structure is determined, and the multi-label data set is constructed.

[0016] 6) Model training: the training set data obtained in step 4) is input into the deep belief network model based on particle swarm algorithm optimization established in step 5) for training, and a single label trained model is obtained. Then the multi-label data set obtained in step 5) is input into the single label trained model for model tuning, and a trained model is obtained.

[0017] 7) Early risk warning of complications: the test data obtained in step 4) is input into the trained model in step 6) to predict whether the target patient will suffer from complications in the middle and late stages of chronic diseases.

[0018] In the early risk warning method of chronic disease complications based on small sample deep learning, in step 1), the first visit data of the chronic disease complication patient is derived from the city hospital patient information data system, covering all related chronic disease patients in nearly twenty years, and the obtained data is processed for privacy. The obtained data includes biochemical examination data and electronic medical record documents, and the data format is [biochemical examination vector, document embedding vector and complication single label].

[0019] In the small sample deep learning-based early risk warning method for chronic disease complications of the application, after the step 2), after the basic Logistics analysis, the relevant variables in the statistical sense are obtained, the application further performs SVM-RFE feature sorting and screening to select the variables meaningful to the warning model. SVM-RFE is a feature selection algorithm based on the maximum interval principle of SVM. It calculates the importance score of each feature by training the SVM model with samples, and then realizes feature sorting. The importance score here takes the influence of each feature on the classification hyperplane function y=W T X bio +b as the evaluation criterion, that is, the weight vector ω i of each feature. Specifically, the input is the feature set sample obtained by Logistics analysis, and the feature score is obtained after entering the SVM model training. The feature with the smallest weight ω i 2 is removed each time, and the SVM is retrained, until the sorting of all features is completed, and the output is a feature sorting list r. For the feature sorting list r, all feature subsets {r1, r2, r3, …, r m} are obtained, and then the SVM is trained using each feature subset to obtain the corresponding scoring function, and the best feature factor combination is the one with the highest score.

[0020] In the small sample deep learning-based early risk warning method for chronic disease complications of the application, in the step 2), missForest is used for data imputation of feature missing values. MissForest is a data imputation algorithm based on random forest algorithm, which makes up for the single nature of the "lazy" imputation method (such as median or average value), and the shortcomings of KNN method, such as the need for data preprocessing, parameter k adjustment, and sensitivity to outliers. It can be applied to mixed data types of continuous values and discrete values. The specific process is as follows: the median / mode imputation is used to fill in the missing values, and the value is marked as the predicted value, and the others are training rows. According to the missing rate of the variable from small to large, use missForest to fill in, iterate this process until a certain condition (such as the latest filling result and the last change is small) is met.

[0021] In the small sample deep learning-based early risk warning method for chronic disease complications of the application, in the step 3), the electronic medical record document is subjected to text vector embedding. Although the BERT model performs well in most similar tasks, its performance on professional texts such as medicine is relatively poor. BioBERT is initialized by BERT weights, and is further pre-trained based on biomedical domain corpus. It is essentially an extension of BERT, and its performance in the information mining task of electronic medical record documents is much better than that of BERT.

[0022] In the small sample deep learning based early risk warning method for chronic disease complications of the application, in the step 4), the HTTE oversampling method fusing conditional entropy and TF-IDF firstly encodes the discrete data by one-hot encoding, and processes the continuous data by segment labeling, and splices to obtain different feature combinations X and complication labels Y. The uncertainty of Y under the combination X, i.e. the conditional entropy of variable Y, is calculated. For each combination X, if the label Y is only a few classes, X is a new combination X_new, and the value of TF-IDF is calculated to obtain the data selection index value = conditional entropy / TF-IDF. The index is sorted in ascending order, and the safe samples are selected according to the input parameter α and the difference diff between the positive and negative samples. If the number of X_new is ≤diff / α, all X_new can be used as safe samples; if the number of X_new is >diff / α, and diff / α>1, the first diff / α of X_new are taken as safe samples, otherwise the first 5 rows are taken. Finally, the minority class samples are created. A safe sample is randomly taken, the continuous data is restored to the interval block, and any value in the interval block can be taken.

[0023] In the deep belief network DBN in the small sample deep learning based early risk warning method for chronic disease complications of the application, the particle swarm algorithm PSO is used for parameter optimization. In order to make the model have strong clustering learning ability, the application sets a four-layer DBN structure, and the number of neurons in each layer is an important factor affecting the classification performance, and the particle swarm algorithm can optimize the number of neurons in the hidden layer of the model. The basic principle of the algorithm is to convert the problem of seeking the optimal solution position in the solution space into the actual scene of the bird group seeking habitat, and because the groups communicate with each other, the optimal solution moves towards the example feasibility direction. Assuming that N particles seek the optimal position in an n-dimensional search space, the speed of each particle at any time is represented as V i =(v i1 ,v i2 ,…,v in ), the position is represented as X i =(x i1 ,x i2 ,…,x in ), and the optimal position is represented as pbest i =(pbest i1 ,pbest i2 ,…,pbest in ).

[0024] Let f(x) be the minimized objective function, and the expression formula of the optimal position is as follows:

[0025]

[0026] The position corresponding to one of the N particles is the best position, i.e. the global optimal position, which can be expressed as gbest(t) = min{f(pbest1(t), f(pbest2(t)),..., f(pbestN(t))}. The velocity and position update equations of the particle can be expressed as: N ij ij ij ij j ij ij ij ij ij 1i 2i 1f 2f

[0027] ij ij ij ij j ij ij ij ij ij 1i 2i 1f 2f

[0028] ij ij ij ij j ij ij ij ij ij 1i 2i 1f 2f

[0029] The first term of the velocity formula is the inertia part, which is composed of the inertia weight and the particle velocity, and represents the particle's trust in the previous motion state. Wherein ω is the inertia weight, which affects the balance between global and local performance. When ω is between (0.8, 1.2), the convergence speed and global search ability of the algorithm are the strongest, and in this interval, the value of ω can be changed using a linear adjustment strategy, i.e.:

[0030]

[0031] The second term is the cognitive part, i.e. the distance and direction between the current position of the particle and its own historical optimal position, which is the experience part of the particle. The third term is the social part, which is the information sharing between particles, i.e. the distance and direction between the current position of the particle and the historical optimal position of the group. Wherein c1, c2 are acceleration factors, and r1, r2 are random numbers in [0, 1]. The introduction of acceleration factors is conducive to the gradual convergence and stability of particles, avoiding falling into local optimum, and the calculation formula is as follows, c 1i ,c 2i for initial value, c 1f ,c 2f for final value:

[0032]

[0033] The particle will optimize the local solution in each iteration until the best classification accuracy is produced in a certain iteration or a certain condition is met (i.e. the fitness function is determined), and then the global optimal solution is produced in the local solution, and the optimal number of neurons of the DBN is obtained.

[0034] In the small sample deep learning based chronic disease complication early risk warning method of the application, in the step 5) multi-label data set construction, label correlation test is performed to determine the order of the classifier chain. Since the classification involves multiple chronic disease complications, the order of the classifier chain affects the classification model. A simple solution is to randomly generate several classifier chain orders to train the model, and use voting to determine the optimal order. The application uses a label relationship network node centrality measurement order selection method based on label correlation test. The nodes in the network are complications, and the edges represent the correlation between complications. The weight of the edge represents the size of the correlation. The size of the correlation between two label nodes can be represented as ω ij ω represents the correlation size of node i and node j. The centrality of the node in the network can measure the influence of the node on other nodes, that is, the importance of the node, and the calculation method is as follows:

[0035]

[0036] Wherein, n represents the number of nodes in the network.

[0037] According to the importance degree of the label, the higher the importance degree, the more information provided to other labels, and the earlier the order in the classifier chain.

[0038] Advantages:

[0039] Compared with the prior art, the application has the following advantages:

[0040] 1. The particle swarm algorithm is used to optimize the deep belief network. This algorithm can optimize the model parameter adjustment process and find the optimal number of neurons, avoiding the situation of local optimization and unable to converge in the parameter adjustment process, and wasting a lot of time and energy. In addition, in the early warning model part, the single label prediction is changed to multi-label prediction, and the label order is considered. Compared with single label prediction, the model is further trained and adjusted, which is helpful to improve the prediction accuracy and has more prominent performance in the scene of multiple disease prediction.

[0041] 2. In the process of chronic disease complication, in addition to the biochemical examination data which can reflect the patient's disease, the disease characteristics recorded in the electronic medical record are also important factors. However, in the past prediction, only some related diseases in the medical record part were extracted as discrete type features for consideration. In the application, the biomedical language processing model BioBERT is used for text vector embedding, which retains the relationship between the semantics of the documents and further mines the information.

[0042] 3. In the feature selection of biochemical examination data, on the basis of Logistics analysis, the SVM-RFE feature selection algorithm is used for feature sorting and further screening. The final determined feature factor combination is not a general statistical variable, but a feature factor closely related to the model.

[0043] 4. In the data interpolation part, the missForest data interpolation algorithm is used to make up for the single of the "lazy" interpolation method (such as median or average) and the shortcomings of KNN method, such as the need for data preprocessing, parameter k adjustment, and sensitivity to outliers. And it can be applied to mixed data types of continuous values and discrete values, and the difference between the predicted value and the true value obtained by interpolation is smaller.

[0044] 5. The HTTE oversampling method fusing conditional entropy and TF-IDF used in the present application can avoid the occurrence of overfitting phenomenon compared with general sampling methods, and the new samples obtained by sampling are closer to the real sample situation, solving the problem of small sample scene. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is a step flow chart of a chronic disease complication early risk warning method based on small sample deep learning of the present application;

[0046] Figure 2 is a structural schematic diagram of a deep belief network model in step 5) of the method of the present application;

[0047] Figure 3 is a structural schematic diagram of an RBM layer in the method of the present application;

[0048] Figure 4 is a complication relationship network schematic diagram in the embodiment of the present application. DETAILED DESCRIPTION

[0049] The present application will be further illustrated in combination with the drawings and specific embodiments, and it should be understood that the following specific embodiments are only used to illustrate the present application and not to limit the scope of the present application.

[0050] Embodiment: Taking the data of patients with diabetic complications as an example, the chronic disease complication early risk warning method based on small sample deep learning is as shown in Figure 1 , which includes the following steps:

[0051] 1) Data collection and preliminary screening.

[0052] The data of patients with diabetes complications need to be collected twice. First, the outpatient and inpatient electronic medical record documents of diabetic patients in the hospital system are obtained, and a disease dictionary related to the three chronic complications of diabetic nephropathy, diabetic retinopathy and diabetic neuropathy is obtained according to the electronic medical record documents. According to the disease dictionary, the first screening of patients is carried out in all the data of diabetic patients, and the index number of all patients related to the three complications is found. Further, in order to better realize risk prediction, secondary screening is required to exclude patients who are diagnosed with complications in the early stage of the disease. According to the index number obtained by secondary screening, the second data collection is carried out to obtain the basic information and biochemical examination data of related patients in the information database. The data set obtained by merging the electronic medical record documents is the whole required data set. The above data set is the fundamental source of subsequent feature selection and other operations, and is also the basis for model training.

[0053] 2) Feature selection and data interpolation processing are performed on the biochemical examination data part of the obtained data.

[0054] The most common influencing factors of different diabetic complications are mainly gender, age, diabetes duration, blood pressure, glucose, glycosylated hemoglobin, creatinine, hemoglobin, total cholesterol, total protein, albumin, globulin, triglyceride, high-density lipoprotein cholesterol (HDL), low-density lipoprotein cholesterol (LDL-C), urinary microalbumin, fasting blood glucose, etc. In specific implementation, feature factor selection needs to be performed from the following four aspects.

[0055] (1) First, search for keywords such as auxiliary diagnosis of diabetic complications, influencing factors, etc. on academic websites, and consult relevant Chinese and English literature. Consult professional doctors for direct guidance, or find the corresponding medical diagnosis standard in the "China Type II Diabetes Prevention and Treatment Guidelines". Second, combine the data set of the three complications of patients screened out in practice, and perform data mining and analysis to preliminarily sort out the key indicators affecting the corresponding complications, and determine an effective medical complication influencing factor table, as shown in the following table.

[0056]

[0057] Complication influencing factor table

[0058] (2) The above process only defines a characteristic range, and data analysis is also required. Thirdly, the Logistics analysis is used, and the statistical significance is P≤0.05, to further determine the variables with high correlation and statistical significance. The method of Logistic analysis mainly constructs a model through single factor analysis combined with multi-factor analysis, and finally obtains the related factors of diabetic complications. Single factor Logistics analysis can explain the influence of a single factor on the occurrence of complications in diabetic patients. After analyzing the above influence factor table, there are 12 variables with statistical significance. These variables mainly include glycosylated hemoglobin, disease duration, fasting blood glucose and part of biochemical examination indexes, and the specific P value results are shown in the following table.

[0059] Influencing factors Nephropathy Retinopathy Neuropathy Course of disease 0.001 0.001 0.001 Glycated hemoglobin 0.003 0.001 0.004 Triglycerides 0.026 0.873 0.003 Total cholesterol 0.005 0.002 0.003 HDL-C 0.036 0.072 0.830 LDL-C 0.311 0.116 0.001 Fasting blood glucose 0.001 0.001 0.001 eGFR (MDRD) 0.001 0.048 0.037 Urine microalbumin / creatinine 0.001 0.274 0.683 Fasting C-peptide 0.563 0.002 0.480 ... ... ... ... Fasting insulin 0.475 0.002 0.103 2-hour postprandial blood glucose 0.002 0.003 0.002

[0060] Different influence factors of complications P value results

[0061] Then, multi-factor Logistics analysis is performed on the 12 related variables. The results show that in the results of multi-factor Logistics analysis, 10 variables are strongly related to the occurrence of complications. The characteristic factor table obtained by statistical analysis is shown in the following table.

[0062]

[0063] Characteristic factor table obtained by statistical analysis

[0064] (3) Fourthly, SVM-RFE feature screening is performed. The present application uses the scikit-learn library to construct the algorithm, selects the data set with complete features as the input, and selects AUC as the evaluation function to select the optimal feature subset. Since statistical analysis has obtained 10 related variables for the three complications respectively, the SVM-RFE algorithm will produce 10 feature subsets. Through AUC analysis, when the number of selected features is less than 10, the score increases as the number of features increases, and the score fluctuates and rises to the maximum value; when the number of selected features is greater than 10, the score decreases as the number of features increases. Therefore, the optimal feature subset is the result obtained in the above table.

[0065] (4) The data set constructed according to the finally determined characteristic factor lacks many characteristic values, especially data related to lipids. For these missing data, the application uses missForest for data imputation, and the selected parameters are 100 trees and 100 iterations. In order to test the performance of the imputation strategy, only patient instances without missing data are considered to assemble a data complete set, and then the data complete set is changed by randomly deleting attribute records. The proportion of random deletion is based on the percentage of missing values of each attribute on the original data set, and only the same percentage is randomly deleted from the data complete set, so as to create an artificial missing data set to test the imputation capability. The comparison of the imputation performance is compared by calculating the root mean square error (RMSE) of the true value and the artificial missing value, and the comparison results are as follows:

[0066] Characteristic factors Glucose Glycated hemoglobin Total cholesterol Triglycerides missForest 0.60 3.65 23.1 49.04 Mean 3.23 11.51 36.37 73.35 Median 3.23 11.81 36.37 75.37

[0067] Data imputation RMSE results

[0068] It can be seen that the data imputation algorithm missForest used in the application is superior to the mean or median imputation method.

[0069] 3) Text vector embedding and PCA dimensionality reduction of electronic medical record documents.

[0070] The application uses the open source BioBERT_v1.1_pubmed version as the pre-training model of vector embedding, and fine-tunes it to embed the outpatient and inpatient medical record document content of the complication patients as a whole, to obtain the feature vector of each patient's diagnosis text. Each description text will be converted into a 768-dimensional vector. Due to the high dimension, the original biochemical examination features may be weakened, so the embedded vector needs to be reduced in dimension. In order to avoid the loss of vector information caused by excessive dimension reduction, the vector is reduced from 768 dimensions to 30 dimensions.

[0071] 4) Connect the vectors obtained in steps 2) and 3) and perform class imbalance processing to solve the influence of small sample scenarios.

[0072] The application adopts a hybrid oversampling method of combining conditional entropy and TF-IDF to supplement small sample data. In the machine learning standard library, an unbalanced data set is selected for method verification, and the commonly used undersampling, resampling and SMOTE oversampling methods are compared. Taking Accuracy as the evaluation index, the HTTE method performs better than the ordinary method on different data sets, and the experimental results are shown in the following table:

[0073]

[0074]

[0075] Comparison of sampling method experimental results

[0076] 5) Establish a deep belief network based on particle swarm optimization algorithm.

[0077] The network model of the early risk warning method of chronic disease complications based on small sample deep learning is composed of six layers: input layer, four hidden layers and output layer. The overall structure of the model is shown in Figure 2 The input layer includes the vector combination of [biochemical examination vector, document embedding vector, and complication single label].

[0078] The hidden layer is composed of a BP neural network layer and three layers of restricted Boltzmann machine (RBM). In order to find the global optimal value, optimal selection is needed between each layer, that is, each layer will be used as an input layer for training. The advantage is that the hidden layer has a large number of neurons, which is more conducive to the construction of feature vectors and has stronger learning ability; In training, the method of layer-by-layer initialization can be adopted to overcome the difficulty of deep neural network training. Specifically, an RBM is composed of a visible layer and a hidden layer connected and restricted, as shown in Figure 3 where a represents the bias coefficient of the visible layer v, b represents the bias coefficient of the hidden layer h, and the weight between them is represented by ω, θ = {ω ij ,a i ,b j}, then the function of the RBM can be represented as:

[0079]

[0080] The joint probability distribution of the visible layer and the hidden layer can be calculated as:

[0081]

[0082] When the number of neurons in the visible layer is determined, the activation state of the hidden layer unit is also determined, and σ is the Sigmoid activation function, so the probability of a single hidden layer unit being activated is:

[0083]

[0084] Similarly, the probability of a single visible layer being activated is:

[0085]

[0086] The training process of DBN is equivalent to the training of multiple RBMs using the contrast divergence algorithm (CD) to determine the activation state of the hidden layer unit. The layer-by-layer training process is as follows:

[0087] (a) The bottommost RBM is trained with the original input data;

[0088] (b) continue training with the features extracted from the bottom RBM as input to the top RBM;

[0089] (c) repeat this process until the top RBM layer is trained;

[0090] (d) fine-tune the model using BP neural network layers.

[0091] Although the training process is clear, the number of neurons in the hidden layer, i.e. the RBM, is uncertain, and the number of neurons in the next RBM layer is determined by the previous RBM layer. Using PSO to optimize this parameter can avoid the overfitting problem or the problem of too simple model caused by experience value. In the specific implementation, the number of initial population particles is set to 30, and the solution position is updated for 100 iterations. In the iteration process, 5-fold cross-validation is used to seek the optimal solution. Finally, when the number of neurons in each hidden layer is (70, 82, 150, 141), the DBN achieves the best classification effect on single label.

[0092] 6) Determine the order of the classifier chain based on the label correlation test and construct a multi-label data set.

[0093] According to the correlation of diabetic nephropathy, diabetic neuropathy and diabetic retinopathy, a simple complication label network can be constructed as shown in Figure 4 The tight centrality value of each diabetic complication node can be calculated according to the formula The results are: diabetic nephropathy 0.3114, diabetic neuropathy 0.4385, and diabetic retinopathy 0.3576, so the label order in the classifier chain is diabetic neuropathy-diabetic retinopathy-diabetic nephropathy. According to the order of the labels in the classifier chain, the multi-label data set is constructed as [biochemical examination vector, document embedding vector, diabetic neuropathy label, diabetic retinopathy label, diabetic nephropathy label].

[0094] 7) Use the multi-label data obtained in step 6) to retrain the DBN model in step 5).

[0095] The DBN model trained on single-label data set is retrained using multi-label data set, and the optimal solution of multi-label data set is achieved when the number of neurons in the hidden layer of the network structure is (71, 82, 150, 140).

[0096] 8) Use the model trained in step 7) to predict the test set obtained by dividing the data set.

[0097] The embodiment finally obtains the prediction results of three diabetic complications in the late stage of diabetes. Here, Accuracy and F1-score are used as evaluation indexes to evaluate the effect of the model.

[0098] The following table shows the comparison of the early risk warning of the three diabetic complications using the model of the present application and the model used by the prior art.

[0099]

[0100] Comparison of different models for predicting three diabetic complications

[0101] The results show that the model proposed in the present application achieves the best Accuracy and F1-score compared with the model proposed by the prior art, which indicates that the model proposed in the present application achieves the best effect compared with the previous prediction method. It can be seen that even with a small-scale data set, PSO-DBN is better than the relatively optimal model of random forest RF in machine learning, and the effect of PSO-DBN will be more prominent for more complex classification and prediction tasks.

[0102] In summary, the method of the present application incorporates electronic medical record documents as a consideration factor in the early risk warning process, combines the data characteristics in the biomedical field, and performs text vector embedding, feature selection and data interpolation to complete data processing. For multiple complication warning scenarios, a multi-label deep network model is constructed, and the particle swarm algorithm is used to optimize the model, realizing scientific and efficient risk warning and improving the accuracy of chronic disease complication prediction.

[0103] It should be noted that the above content only illustrates the technical idea of the present application and cannot limit the protection scope of the present application. For ordinary skilled persons in the technical field, they can make several improvements and refinements without departing from the principles of the present application, and these improvements and refinements fall within the protection scope of the claims of the present application.

Claims

1. A small sample deep learning-based early risk warning method for chronic disease complications, characterized in that, It comprises the following steps: 1) Data collection: collect the outpatient and inpatient biochemical examination data and electronic medical record documents of patients with chronic disease complications at their first visit to the hospital system, and obtain the disease dictionary strongly related to the complications according to the document content, exclude patients with related complications at their first visit, 2) Biochemical examination data processing: process the biochemical examination data obtained in step 1), use Logistics analysis for statistical significance feature selection, and then use SVM-RFE for further feature sorting and model significance feature selection, determine the feature factor combination, construct a feature table, and use missForest for missing value data interpolation, 3) Electronic medical record document processing: process the electronic medical record documents obtained in step 1), use the Biobert model for text vector embedding, and perform principal component analysis dimension reduction on the result vector, 4) Class imbalance processing: splice the biochemical examination vector, document embedding vector and complication label obtained in steps 2) and 3) to form a data format [biochemical examination vector, document embedding vector, complication single label], for unbalanced data, use the HTTE oversampling method that integrates conditional entropy and TF-IDF to create a minority class sample set, solve the small sample problem, obtain a class-balanced data set, and divide it to obtain the training set and test set required for model training, 5) Model establishment: establish a deep belief network optimized based on a particle swarm algorithm, optimize the number of neurons and learning rate of the hidden layer of the DBN, the model includes six layers of input layer, four hidden layers and output layer, the input layer includes three aspects of vector combination [biochemical examination vector, document embedding vector, complication single label]; the number of neurons contained in the hidden layer is obtained by particle swarm optimization, label correlation test is performed to determine the classifier chain structure, and a multi-label data set is constructed, 6) Model training: input the training set data obtained in step 4) into the deep belief network model established in step 5) based on particle swarm optimization for training, obtain a single-label trained model, and then input the multi-label data set obtained in step 5) into the single-label trained model for model tuning, and obtain a trained model, 7) Early risk warning of complications: input the test data obtained in step 4) into the trained model in step 6) to predict whether the target patient will suffer from complications in the later stage of chronic disease.

2. The small sample deep learning-based early risk warning method for chronic disease complications according to claim 1, characterized in that: In step 1), the data of patients with chronic disease complications at their first visit to the hospital is obtained from the city hospital patient information data system, covering all related chronic disease patients in nearly twenty years, the obtained data is processed for privacy, and the obtained data includes biochemical examination data and electronic medical record documents, and the data format is as follows: [biochemical examination vector, document embedding vector, complication single label].

3. The small sample deep learning-based early risk warning method for chronic disease complications according to claim 2, characterized in that: In step 2), after the Logistics analysis and obtaining the statistically significant variables, SVM-RFE feature ranking and screening are performed to select the variables meaningful to the early warning model. SVM-RFE is a feature selection algorithm based on the maximum interval principle of SVM. It calculates the importance score of each feature by training the SVM model with samples, and then realizes feature ranking. The importance score is the influence of each feature on the classification hyperplane function y=W T X bio +b as the evaluation criterion, that is, the weight vector ω i of each feature. Specifically, the input is the feature set sample obtained by Logistics analysis, and the feature score is obtained after entering the SVM model training. The feature with the smallest weight ω i 2 is removed each time, and the SVM is retrained until all features are ranked. The output is a feature ranking list r. For the feature ranking list r, all feature subsets {r1, r2, r3, …, r m} are obtained, and then the SVM is trained using each feature subset to obtain the corresponding scoring function. The best feature factor combination is the one with the highest score.

4. The small sample deep learning-based early risk warning method for chronic disease complications according to claim 2, characterized in that: In step 2), data imputation is performed on feature missing values using missForest, which is a data imputation algorithm based on random forest algorithm, making up for the single nature of "lazy" imputation method, and overcoming the shortcomings of KNN method such as data preprocessing, parameter k adjustment and sensitivity to outliers. It is applicable to mixed data types of continuous and discrete values. The specific process is as follows: missing values are filled using median / mode imputation, and this value is marked as the predicted value, and the others are training rows. The missing rate of variables is used to fill in the missing values from small to large using missForest, and this process is iterated until the condition is met.

5. The small sample deep learning-based early risk warning method for chronic disease complications according to claim 3, characterized in that: In step 3), text vector embedding is performed on electronic medical record documents. BioBERT is a continuation of BERT initialized with weights, based on biomedical domain corpus. It is essentially an extension of BERT, and its performance in electronic medical record document information mining tasks is much better than BERT.

6. The small sample deep learning-based early risk warning method for chronic disease complications according to claim 4, characterized in that: In step 4), the HTTE oversampling method fuses conditional entropy and TF-IDF. First, one-hot encoding is performed on discrete data, and segmentation labeling is performed on continuous data. After splicing, different feature combinations X and complication labels Y are obtained. The uncertainty of Y under the combination X is calculated, i.e. the conditional entropy of variable Y. For each combination X, if the label Y is only a few classes, then X is a new combination X_new, and the TF-IDF value is calculated. The data selection index value = conditional entropy / TF-IDF is obtained, and the index is sorted in ascending order. According to the input parameters α and the difference diff between positive and negative samples, the safe samples are selected. If the number of X_new is ≤diff / α, all X_new are safe samples. If the number of X_new is >diff / α, and diff / α>1, the first diff / α rows of X_new are taken as safe samples. Otherwise, the first 5 rows are taken. Finally, the minority class samples are created, a safe sample is randomly taken, the continuous data is restored to the interval block, and any value in the interval block can be taken.

7. The small sample deep learning-based early risk warning method for chronic disease complications according to claim 5, characterized in that: In the step 5) deep belief network DBN, a particle swarm optimization PSO is used for parameter optimization, in order to make the model have strong clustering learning ability, a four-layer DBN structure is set, and the number of neurons of each layer is an important factor affecting the classification performance, and the particle swarm optimization is used for optimization of the number of neurons of the hidden layer of the model, the basic principle of the algorithm is to convert the problem of seeking the optimal solution position in the solution space into the actual scene of the bird group seeking the habitat, and due to the mutual communication between the groups, the optimal solution moves towards the example feasibility direction, assuming that N particles seek the optimal position in an n-dimensional search space, the speed of each particle at any time is represented as V i =(v i1 ,v i2 ,…,v in ), the position is represented as X i =(x i1 ,x i2 ,…,x in ), and the optimal position, i.e. the individual optimal solution, is represented as pbest i =(pbest i1 ,pbest i2 ,…,pbest in ), Let f(x) be the minimized objective function, then the optimal position is represented by the following formula: where pbest i (t) denotes the optimal position of particle i at time t, f(pbest i (t)) denotes the fitness value of the optimal position of particle i at time t, and f(X i (t+1)) denotes the fitness value of the position of particle i at time t+1. Among N particles, there exists a position corresponding to one particle that is the optimal position, i.e., the global optimal position, which is represented as gbest(t) = min{f(pbest1(t), f(pbest2(t)), ..., f(pbest... N (t))}, then the velocity and position update equations of the particle are expressed as: v ij (t+1) = ωv ij (t) + c1r1 x (pbest ij (t) - x ij (t)) + c2r2 x (gbest j (t) - x ij (t)) x ij (t+1) = x ij (t) + v ij (t) + v ij (t+1) where v ij (t), x ij (t), pbest ij (t) are the velocity, position and the best position of particle i in dimension j at time t, respectively, gbest j (t) is the global best position of the swarm in dimension j at time t, and c1, c2 are acceleration factors and r1, r2 are random numbers in [0, 1], Specifically, the first term of the velocity formula is the inertia part, which is composed of the inertia weight and the particle velocity, representing the particle's trust in the previous motion state, where ω is the inertia weight, affecting the balance between global and local performance. When ω is between (0.8, 1.2), the convergence speed and global search ability of the algorithm are the strongest. In this interval, a linear adjustment strategy is used to change the value of ω, i.e. The second term is the cognitive part, that is, the distance and direction between the current position of the particle and the historical optimal position of itself, which is the experience part of the particle. The third term is the social part, which is the information sharing between particles, that is, the distance and direction between the current position of the particle and the historical optimal position of the group. The introduction of the acceleration factor is beneficial to the gradual convergence and stability of the particle, and avoids falling into local optimum. The calculation formula is as follows, c 1i ,c 2i is the initial value, c 1f ,c 2f is the final value: The particle optimizes the local solution in each iteration until the best classification accuracy or a certain condition is met, i.e. the fitness function, then the global optimal solution is generated in the local solution, and the optimal number of DBN neurons is obtained.

8. The small sample deep learning-based early risk warning method for chronic disease complications according to claim 7, characterized in that: In the step 5) of the multi-label dataset construction, the label correlation test is performed to determine the order of the classifier chain. Since the classification involves multiple chronic disease complications, the order of the classifier chain affects the classification model. A simple solution is to randomly generate several classifier chain orders to train the model, and use voting to determine the optimal order. The order selection method based on label correlation test and label relationship network node centrality measurement is used. The nodes in the network are complications, and the edges represent the correlation between complications. The weight of the edge represents the size of the correlation. The correlation size between two label nodes is represented as ω ij The correlation size between node i and node j is represented as ω ij. The centrality of a node in the network measures the influence of the node on other nodes, i.e., the importance of the node, which is calculated as follows: According to the importance degree of the label, the higher the importance degree, the more information it provides to other labels, and the earlier the order in the classifier chain.

Images