Medical data large-scale diabetes screening method based on attention-enhanced deep neural network
By introducing attention-enhanced deep neural networks and multi-head attention mechanisms, the problems of low accuracy and insufficient generalization ability in existing diabetes screening models are solved, achieving efficient and accurate diabetes screening and improving the model's prediction accuracy and stability.
Patent Information
- Application Number
- CN202411104939.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2026-01-20
AI Technical Summary
Existing technologies have low model prediction accuracy and insufficient generalization ability in diabetes screening, making it difficult to achieve efficient and accurate early screening and prediction.
An attention-enhanced deep neural network algorithm is adopted, combining multi-head attention mechanism and deep learning. By introducing multi-head attention layer, input layer, hidden layer and output layer, using ReLU and Sigmoid activation functions, adding dropout layer, optimizing weights and bias, and using cross-entropy loss and Adaptive Moment Estimation optimization algorithm, the number of hidden layers and neurons is dynamically adjusted, and data preprocessing and feature weighting are performed.
The model improved the accuracy and robustness of diabetes classification, achieving 98.4% accuracy, 98% precision, and 99% recall on the test set. It also reduced the gradient vanishing problem and enhanced the model's generalization ability and prediction accuracy.
Smart Images
Figure CN121366740A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of medical data analysis, in particular to a large-scale diabetes screening technology. BACKGROUND
[0002] Early screening and prediction of diabetes is of great significance for delaying complications, improving the quality of life of patients and reducing mortality. Diabetes is a serious metabolic disorder. Insulin levels directly affect blood glucose levels, and insufficient secretion of insulin by the pancreas can lead to elevated blood glucose levels, which can lead to the development of diabetes over time.
[0003] According to the World Health Organization (WHO), diabetes is one of the four major non-communicable diseases with high mortality, and one person dies from diabetes and its complications every five seconds. Kidney disease, retinopathy and diabetic foot caused by diabetes are the main causes of kidney failure, blindness and amputation. Diabetes is mainly divided into type 1, type 2 and gestational diabetes. Type 1 diabetes is more common in young people and children, characterized by the inability of the pancreas to produce insulin, requiring long-term insulin injections. Type 2 diabetes usually affects middle-aged, elderly and obese people, characterized by pancreatic dysfunction and insulin resistance. Gestational diabetes (GDM) is characterized by high blood sugar during pregnancy. Early screening and targeted treatment of pre-diabetic patients is crucial, which can significantly delay the occurrence of diabetes complications and improve the quality of life of patients, but the existing technology has problems such as low model prediction accuracy and insufficient generalization ability, and there is an urgent need for an efficient and accurate diabetes screening algorithm.
[0004] Artificial intelligence, particularly deep learning, is playing an increasingly important role in disease screening. Deep learning is an important branch of machine learning that mimics the structure and function of the human brain's neural system. It uses multi-layer neural networks to learn hierarchical representations of data. This technology has shown great potential in predicting and assessing the risk of chronic diseases such as diabetes and kidney disease. The core of deep neural networks is artificial neural networks, which consist of an input layer, multiple hidden layers and an output layer. Each layer consists of multiple nodes (neurons) that are connected to each other through weights. SUMMARY
[0005] The present application proposes a medical data large-scale diabetes screening algorithm based on attention-enhanced deep neural networks, which improves the feature extraction ability and prediction accuracy of the model by introducing a multi-head attention mechanism.
[0006] The multi-head attention mechanism algorithm in the Transformer is introduced, and the deep learning algorithm is combined to improve the classification performance of diabetes. The model includes four independent layers, namely the multi-head attention layer, the input layer, the hidden layer and the output layer. Among them, the multi-head attention layer uses 4 attention heads, which automatically adjusts the attention weight of each feature according to the weight matrix to converge the attention of the model, and inputs into the fully connected neural network model. The input layer receives 8 features with weights, and then the data propagates through 3 hidden layers. The neurons of each hidden layer are connected to the output results of the previous layer, and the output generated by the activation function of this layer is connected to the next layer. The output of each layer absorbs the abstract calculation results of the previous layer, and the output layer generates the classification results. Fixing the initial random weights and biases ensures the reproducibility and stability of the model, and the weights and biases are updated and optimized in the process of back propagation. The model combines the advantages of the multi-head attention mechanism and designs three hidden layers to realize a robust and efficient model for diabetes classification. The model is implemented on the famous Pima Indian Diabetes dataset.
[0007] After the multi-head attention layer, a new dataset is generated, which has 8 features and 10000 samples. Each column of feature data has an attention weight. The input layer receives the above data, and the data is transmitted to the output layer through four fully connected layers. The input data combines weights and biases as a complete parameter to pass to the activation function. The activation function uses ReLU() and sigmoid() functions. A Dropout layer is added after each hidden layer to prevent overfitting. The ReLU() function has high computational efficiency and can alleviate the problem of gradient disappearance. The gradient is more stable during back propagation. The Sigmoid() function maps the input to the (0, 1) interval, which has continuity and smoothness, and is crucial for the convergence of the optimization algorithm.
[0008] The deep neural network starts from the input layer and calculates the output of each layer through linear transformation and activation function. The loss function is calculated using cross-entropy loss, and the gradient of the loss function with respect to the model parameters is calculated using the back propagation algorithm. The learning rate of each parameter is adjusted adaptively using the optimization algorithm Adaptive Moment Estimation, and the momentum term is combined to make the model converge to the global optimal solution quickly during training. The model parameters are updated according to the gradient. Each epoch of training includes forward propagation, loss calculation, back propagation, parameter update and other processes. The above processes are repeated until the model output perfect prediction results. After training, the test set is used to evaluate the model performance and output the prediction results. The whole process optimizes the parameters through back propagation technology to minimize the loss function, so that the model can adapt to the training data and achieve good generalization ability.
[0009] Algorithm 1: Diabetes screening algorithm based on attention enhanced deep neural network
[0010] Step
[0011] 1. The dataset is preprocessed, i.e., fill in abnormal values, missing values, oversample to balance the dataset, and normalize.
[0012] 2. After normalizing the data, the multi-head attention layer is used to extract the attention weight of each feature, and the weight matrix is generated.
[0013] 3. The extracted data is multiplied by element by element, i.e., Hadamard product operation, and the weight matrix of each feature is added to each original data. Then the input data X is the data with weight, and each column feature contains the attention of the feature on the result in the entire dataset.
[0014] 4. Input the data into the deep neural network for learning.
[0015] First part of the algorithm: multi-head attention layer: weight each column of feature data
[0016] Input parameters: feature data X number of attention heads H
[0017] Calculate the query weight matrix set:
[0018] Calculate the key weight matrix set:
[0019] Calculate the value weight matrix set:
[0020] Calculate the scaling factor:
[0021] Split the input for each attention head:
[0022] For each h = 1 to H
[0023] Split X to get subvector X h
[0024] Calculate the attention weight:
[0025] Calculate the query vector:
[0026] Calculate the key vector:
[0027] Calculate the value vector:
[0028] Calculate the attention score:
[0029]
[0030] Weighted sum:
[0031] Attention h = Attention Score h ·V h
[0032] Merge multiple heads:
[0033] Multi-head Attention = [Attention1, Attention2,..., Attention H ]
[0034] Element-wise multiplication of Multi-head Attention and original data X, i.e. Hadamard product
[0035] Second part of the algorithm: Deep neural network layer: get the prediction result
[0036] ①. Calculate the linear combination of input features and weights plus bias:
[0037] Z = W T · X + b
[0038] ②. Apply activation function g to Z:
[0039]
[0040] (Note: g is ReLU in hidden layers and Sigmoid in output layer)
[0041] ③. Apply Dropout layer after each hidden layer:
[0042]
[0043] Where p is the probability of retaining nodes
[0044] ④. Calculate the loss function L:
[0045]
[0046] ⑤. Calculate the cost function J containing the L2 regularization term:
[0047]
[0048] ⑥. Learning rate adjusts the step size of gradient descent:
[0049]
[0050]
[0051] 7. Calculate the gradient:
[0052]
[0053]
[0054]
[0055] 8. Update the weights and biases using the gradient:
[0056] W := W - a · dW
[0057] b := b - a · db
[0058] 9. Make a diagnosis based on the prediction:
[0059] If pred > 0.5 Diagnose as "Diabetes"
[0060] Else, diagnose as "Non-Diabetes"
[0061] Third part of the algorithm: dynamically adjust the number of hidden layers and the number of hidden layer neurons
[0062] Number of hidden layers (L): L = log2(m) - 1
[0063] Number of neurons in each hidden layer (H):
[0064] Based on the cost function J state, dynamically adjust the number of hidden layers and the number of hidden layer neurons. If the cost function quickly decreases and tends to be stable during the training process, it means that the current network architecture complexity is sufficient. If the cost function hovers at a high level and the training error and validation error are both large, consider increasing the number of hidden layers. For example, increase one layer each time and observe the change in the loss function. If the cost function has decreased but is still not ideal, consider increasing the number of neurons in each hidden layer. For example, increase 10-20 neurons each time until the loss function decreases and tends to be stable.
[0065] Input
[0066] - X: input features with weights after attention layer
[0067] - Y: target label
[0068] - a: learning rate
[0069] - theta, b: weights and biases
[0070] - n: number of iterations
[0071] - m: number of samples
[0072] Output
[0073] - Predicted output
[0074] - Updated weights W and bias b BRIEF DESCRIPTION OF DRAWINGS
[0075] The above and other objects, features and advantages of the disclosed example embodiments will be more apparent from the following detailed description read in conjunction with the accompanying drawings, in which certain example embodiments of the disclosure are illustrated. In the drawings, several embodiments of the disclosure are illustrated by way of example and not limitation in which like references indicate similar, or corresponding, elements in the various figures, and in which:
[0076] Figure 1 Core diagram of the diabetes screening algorithm based on attention enhanced deep neural network
[0077] Figure 2 Training flowchart of the diabetes prediction algorithm
[0078] Figure 3 Boxplot showing outliers of each feature
[0079] Figure 4 Scatterplot for visualizing outliers and missing values
[0080] Figure 5 Histogram of raw data
[0081] Figure 6 Histogram of grouped medians
[0082] Figure 7 Comparison of prediction performance of three imputation methods
[0083] Figure 8 Multi-head attention diagram
[0084] Figure 9 Multi-head attention feature extraction layer
[0085] Figure 10 Five-fold cross-validation principle
[0086] Figure 11 Five-fold cross-validation results
[0087] Figure 12 Test accuracy, precision, recall, F1 score, mean squared error, R2 score, AUC curve of deep neural network
[0088] Figure 13 Area under ROC curve DETAILED DESCRIPTION
[0089] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It should be understood by those skilled in the art that the described embodiments of the present application are only part of the embodiments of the present application, not all the embodiments. Therefore, all other embodiments obtained by those skilled in the art without creative work on the basis of the embodiments in the present application are within the scope of protection of the present application.
[0090] It should be understood that the step numbers used herein are only for the convenience of description, and are not limited to the order of execution of the steps.
[0091] Referring to Figure 1 The present application discloses a medical data large-scale diabetes screening algorithm based on attention-enhanced deep neural network, referring to Figure 2 The diabetes prediction algorithm training flowchart is shown, which includes:
[0092] Step 1, data preprocessing: including identifying missing values and outliers based on box plot outlier detection algorithm, using grouping (diabetes group, non-diabetes group) median filling method. Using oversampling technology to balance the data of diabetes group and non-diabetes group. Using the min-max scaling method to normalize the data.
[0093] Step 2, feature extraction and weighting: introducing the multi-head attention mechanism in Transformer, constructing the attention-based feature extraction layer, i.e. Attention-Based Feature Weighting Layer, to extract the weight of each feature, generating the weight matrix of the feature, and multiplying the extracted data by element by element, i.e. Hadamard product operation, adding the weight matrix of each feature to each original data to obtain the data set with weight.
[0094] Step 3, input the data processed in step 2 into the deep neural network model for training: dynamically adjust the depth of hidden layer and the number of hidden layer neurons according to the size of data, the design of this example is 3 layers of hidden layer, each layer has 70 hidden layer neurons. And add L2 regularization term and Dropout layer to improve the generalization performance of the model.
[0095] Step 4, model verification and evaluation: accuracy, precision, recall, F1 score, mean square error (MSE), R2 score, ROC curve and AUC curve are used to measure the performance of the model.
[0096] Specifically, the present study adopts the Pima Indian Diabetes Dataset provided by the National Institute of Diabetes and Digestive and Kidney Diseases, which collects the clinical characteristics of a group of Pima Indian women and whether they have diabetes information, the dataset contains 768 records, each record has 8 characteristics, including age, pregnancy times, blood pressure, skinfold thickness, insulin level, body mass index (BMI), diabetes pedigree function, age and relationship with diabetes. The output label is binary, 0 means no diabetes, 1 means diabetes, and the women's age is between 21-81 years old. The data in this dataset has been carefully collected and processed, but there are still a large number of missing values, abnormal values of characteristics, and unbalanced data, among which the number of samples with diabetes is 268, and the number of samples without diabetes is 500, which is close to twice the number of samples with diabetes. This is a serious unbalanced data, so appropriate data cleaning and preprocessing is needed when training the model.
[0097] Specifically, Figure 3 The box plot is used to show the abnormal data of each feature, Figure 4 A scatter plot of each feature distribution is generated for each feature to visually observe the outliers and missing values. In the sample, a total of 50 outliers are detected. Among them, the pregnancy times abnormal value range is 14 to 17, the plasma glucose concentration abnormal value is 0, the blood pressure abnormal value range is 24 to 122, the triceps skinfold thickness abnormal value is 99, the serum insulin abnormal value range is 321 to 846, the BMI abnormal value range is 52.3 to 67.1, and the diabetes family history function abnormal value range is 1.213 to 2.42. These outliers may be caused by data collection errors.
[0098] Specifically, the data filling methods include mean filling, mode filling, and median filling. Since there is a large gap between the data of the diabetes group and the non-diabetes group, the various indicators of the human body have changed greatly after suffering from diabetes, and it is not appropriate to simply rely on mean, mode to fill the data. The method of filling according to the grouped median is scientific and conforms to the data rule. The so-called grouped median filling method is to divide the data of the 8 characteristics into diabetes group and non-diabetes group, and fill the data according to the median of the group category to which the missing value of a certain characteristic belongs. This method can well reflect the influence of disease state and ensure the integrity of data, and fully considers the influence of disease state on data, Figure 5 、 Figure 6 Draw the original data and grouped median filling histogram respectively, Figure 5 、 Figure 6The median filling is very consistent with the distribution characteristics of the original data.
[0099] Specifically, using mean filling, mode filling, and median filling, the three filling methods fill the missing values and outliers, and the data set produces different prediction performances under the same algorithm, Figure 7 The comparison of the prediction performances of the three filling methods well verifies this, Figure 7 The test accuracy, precision, and AUC value of the three methods are compared respectively, from Figure 7 It can be seen that the mode filling effect is the worst, the Test Accuracy is only 76.5%, the mean filling reaches 81%, and the median filling reaches 98%.
[0100] Specifically, in the Pima Indian Diabetes dataset, the diabetes patient data only has 268, while the healthy data has 500, and there is a significant class imbalance problem. The basic idea of oversampling is to balance the class distribution by increasing the number of samples of the minority class, so that the number of samples between the two classes is closer. This helps to improve the model's recognition ability of the minority class, thereby improving the overall classification performance. This study uses the SMOTE method in the imbalanced-learn library, which can easily oversample the data. In the code, we first import the SMOTE module, and then use the fit_resample() method to oversample the original data to get the new sample set after oversampling. After oversampling, the data set is expanded to 10000 samples, of which the number of diabetes patient samples is 5000 and the number of non-diabetes patient samples is 5000.
[0101] Specifically, since the Pima Indian Diabetes dataset has 8 features, the data scales of each feature are different, so scaling the feature data is crucial. Common feature data scaling techniques include normalization and standardization. Normalization is to scale the features to a fixed range, usually [0, 1] or [-1, 1]. The commonly used method is min-max scaling, which sets the minimum value of the feature to 0 and the maximum value to 1, and then scales other feature values in linear proportion. The latter is to make the mean of the feature 0 and the standard deviation 1. The most commonly used method is to use z-score standardization, which subtracts the mean and then divides by the standard deviation. This study uses the normalization method of min-max scaling to process the data. where X is the original feature value, X min and X max are the minimum and maximum values of the feature, respectively.
[0102] Specifically, Figure 8For the theoretical diagram of the multi-head attention feature extraction layer, the Pima Indian diabetes dataset is calculated through a multi-head attention layer before the deep neural network, and reference is made to Figure 9 The model can dynamically assign different attention weights to the features, so that the model pays more attention to the features that have an impact on the diabetes result. The multi-head attention mechanism is a technique that enhances the model's representation ability by applying attention mechanisms to different representation subspaces of the input. It obtains multiple different attention representations by using multiple attention heads to calculate in parallel, and combines these representations together to improve the model's expression ability and generalization ability. The attention mechanism allows the model to dynamically assign different attention weights to different parts of the input when processing sequential data, thereby improving the model's performance. Considering that the Pima Indian Diabetes Dataset is a medium-sized data and considering the complexity and computational cost of the model, this example selects 4 attention heads, each of which calculates in parallel and pays attention to different feature subspaces, increasing the model's expression ability and better capturing information in the input sequence, improving the model's robustness and generalization ability. The 4 attention heads are averaged by the average pooling layer to obtain an average attention weight for each feature, forming a complete attention weight matrix, and then the input data is multiplied element-wise with the attention weight matrix to strengthen the attention to key features such as blood glucose level and BMI. For less important features, by multiplying the attention weight close to zero, their influence in the model can be weakened, thereby reducing the impact of noise and improving the model's robustness. In this way, the importance of the features is quantified, enhancing the model's interpretability and providing a clearer understanding of how the model evaluates each feature and can infer the basis for the model's classification decision.
[0103] Specifically, the model validation of the present application adopts five-fold cross-validation as Figure 10 In this study, the dataset is first divided into a training+validation set (80%) and a test set (20%). Then, the training+validation set is further divided into a training set (80%) and a validation set (20%), resulting in a final training set of 64%, a validation set of 16%, and a test set of 20%. The present application uses StratifiedKFold for 5-fold cross-validation, trains and validates the model on each fold, and records the performance metrics of each fold, including accuracy, precision, and AUC.
[0104] Specifically, Figure 11 The scores of the five-fold cross-validation are shown: [0.9795 0.9845 0.981 0.985 0.982], and the average accuracy of cross-validation is 0.982.
[0105] Specifically, the application uses the grid search algorithm GridSearchCV to find the number of hidden layers, the number of hidden layer neurons, the activation function, the optimizer, the learning rate, the batch size, the number of iterations, etc. Hyperparameters, try different values to find the best parameter combination, for example: the number of hidden layers is set to (3, 4, 5...100), the number of hidden layer neurons is set to (10, 20, 30, 40,...100), and the final best number of hidden layers is 3, the number of hidden layer neurons is (70, 70, 70), and the optimizer is (’SGD’, ’RMSprop’, ’Adagrad’, ’Adadelta’, ’Adam’, ’Adamax’, ’Nadam’) 7 kinds of forms, activation function ’activation’:[’tanh’, ’softmax’, ’softplus’, ’softsign’, ’relu’, ’sigmoid’, ’hard_sigmoid’, ’linear’] 8 kinds, L2 regularization parameter ’alpha’;[0.001, 0.01, 0.02, 0.03, 0.04, 0.05], learning rate lr(0.0001, 0.001, 0.01, 0.1), in the training process, PyTorch library and TensorFlow library are used to build a deep learning model, and Adam optimizer is used in the model to reduce the output error in the forward propagation process. The number of hidden layers is 3, each hidden layer contains 70 neurons, the hidden layer uses ReLU activation function, the output layer uses Sigmoid activation function, the learning rate is 0.0001, the optimizer uses Adam, the binary cross entropy is used to calculate the loss function, the number of training rounds (Epochs) is 10000 times, and the sci-kit-learn library train_test_split function is used to perform dataset, and StandardScaler is used to standardize the feature data.
[0106] In a specific implementation case, it should be noted that the test platform and experimental environment of the application are: using Windows operating system, equipped with NVIDIA GeForce RTX 3060 graphics card for operation, and the size of the video memory is 12GB GDDR6. The CPU configuration is 12th Gen Intel(R)Core(TM)i7-12700, and the main frequency is 2.10GHz. The CUDA version is 12.4.131, and the Python language environment is 3.9.19.
[0107] In a specific implementation case, Figure 12The test accuracy, precision, recall, F1 score, mean square error, R2 score and AUC curve are shown, and the model reaches 98.4% accuracy, 98% precision, 99% recall and 98% F1 score on the test set, R2 score is 93.4%, and mean square error is only 1.65%, which shows that the predicted value of the model is very close to the actual value. Figure 13 The area under the ROC curve reaches 99.65%, which shows that the model correctly identifies most positive cases and rarely mislabels negative as positive, proving that the model has perfect classification ability.
[0108] Compared with the current algorithm, such as decision tree (DT), k-nearest neighbor (KNN), random forest (RF), naive Bayes (NB), AdaBoost (AB), logistic regression (LR) and support vector machine (SVM), the prediction accuracy reaches 88.6%, and the prediction accuracy of the present application is improved by 11.06%.
Claims
1. A medical data large-scale diabetes screening method based on attention-enhanced deep neural network, comprising the following steps: 1.1 Data preprocessing: - Using Pima Indian Diabetes Dataset as the training dataset. - Identifying missing values and outliers based on the boxplot outlier detection algorithm, using the grouping (diabetes group, non-diabetes group) median filling method. - Using oversampling techniques to balance the data of the diabetes group and the non-diabetes group. - Using the min-max scaling method to normalize the data. 1.2 Feature extraction and weighting: - Introducing the multi-head attention mechanism in the Transformer, constructing an attention-based feature extraction layer, i.e., Attention-Based Feature Weighting Layer, to extract the weight of each feature and generate a weight matrix of the feature. - The extracted data is multiplied element by element, and the weight matrix of each feature is added to each original data. 1.3 Model construction and training: - Constructing an Attention-Enhanced Deep Neural Network (AEDNN) model based on the multi-head attention mechanism and deep neural network. - Training in the deep neural network layer, where the depth of the deep neural network can be flexibly increased or decreased according to the size and complexity of the dataset, including the number of hidden layers and the number of neurons in each hidden layer, adding L2 regularization term and Dropout layer during training to improve the generalization performance of the model. - Using ReLU for hidden layer and Sigmoid activation function for output layer, using the optimization algorithm Adaptive Moment Estimation to adjust the hyperparameters adaptively. 1.4 Model evaluation: - Using accuracy, precision, recall, F1 score, mean square error (MSE), R2 score, ROC curve and AUC curve to measure the performance of the model. 1.5 Model prediction: - Use the trained model to screen and predict diabetes. - The model can predict the association between other medical data and a certain disease.
2. The method of claim 1.1, wherein, Based on the box plot outlier detection algorithm, missing values and outliers are identified and marked. In the box plot, the upper boundary of the box represents the upper quartile (Q3), the lower boundary represents the lower quartile (Q1), and the line in the middle of the box represents the median (Q2). The length of the box is the interquartile range (IQR, Q3-Q1) of the data distribution. The upper and lower limits are calculated as follows: Upper limit: Q3+1.5*IQR Lower limit: Q1-1.5*IQR Outliers are points that fall outside the upper and lower limits of the box plot.
3. The method of claim 1.1, wherein, The grouping median filling method calculates the median for different feature groups, i.e. the median of each feature in the diabetes group and the median of each feature in the non-diabetes group, and fills in the missing values and outliers with the median.
4. The method of claim 1.1, wherein, The oversampling technique refers to balancing the positive and negative sample ratio in the dataset by increasing the number of samples with the disease. When there is a serious bias in the dataset, such as the non-diabetes group data being much larger than the diabetes group data, SMOTE algorithm can be used to oversample the diabetes group data to balance the sample size of the diabetes group and the non-diabetes group.
5. The method of claim 4, wherein, This study uses the SMOTE method in the imbalanced-learn library, which can easily oversample the data. In the code, first import the SMOTE module, then use the fit_resample() method to oversample the original data, and get the new sample set after oversampling. After oversampling, the dataset is expanded to 10000 samples, with 5000 diabetes patients and 5000 non-diabetes patients.
6. The method of claim 1.1, wherein, This method scales different feature values of different data. Since the Pima Indian Diabetes dataset has 8 features, the data scales of each feature are different, so scaling the feature data is crucial. Common feature data scaling techniques include normalization and standardization. Normalization scales features to a fixed range, usually [0, 1] or [-1, 1]. The most commonly used method is min-max scaling, which sets the minimum value of the feature to 0 and the maximum value to 1, then scales other feature values in linear proportion. The latter is to make the mean of the feature 0 and the standard deviation 1. The most commonly used method is to use z-score standardization, which subtracts the mean of the feature and then divides by the standard deviation. This study uses the normalization method of min-max scaling to process the data. where X is the original feature value, X min and X max are the minimum and maximum values of the feature, respectively.
7. The method of claim 1.2, wherein, The Attention-Based Feature Weighting Layer is designed to generate a weight matrix based on the influence of each feature on the result and apply the weight matrix to the dataset.
8. The method of claim 1.2, wherein, The extracted data is multiplied element by element, i.e. Hadamard product operation, to add the weight matrix of each feature to each original data, and then input the result into the deep neural network for training and learning.
9. The method of claim 1.2, wherein, The deep neural network can flexibly increase or decrease the depth according to the size of the dataset, adjust the number of hidden layers and the number of neurons in each hidden layer according to the number of features of the dataset.
10. The method of claim 1.1, wherein, The training data was provided by the National Institute of Diabetes and Digestive and Kidney Diseases, which was collected by medical experts from women in the Pima Indian community in Arizona. It is widely used for the development and evaluation of diabetes prediction models. Researchers and data scientists can use this data to train machine learning models to predict whether a person has diabetes, thereby helping medical professionals to intervene and treat early.
11. The method of claim 1.3, wherein, The GridSearchCV algorithm was used in the training process of the model to adjust the hyperparameters such as the number of hidden layers, the number of neurons in the hidden layer, the activation function, the optimizer, the learning rate, the batch size, and the number of iterations.
12. The method of claim 1.4, wherein, The accuracy of the model on the test set reached 98.4%, the precision reached 98%, the recall reached 99%, the F1 score was 98%, the R2 Score was 93.4%, the mean squared error was 1.65%, and the area under the ROC curve was 99.65%.
13. The method of claim 1.5, wherein, The model not only can handle the prediction of diabetes, but also can be applied to other high-dimensional, multi-feature, large-scale biomedical data and the prediction of the association between a disease.