Coronary heart disease risk assessment method fusing tongue diagnosis image and structured data

By constructing a deep learning model that integrates tongue diagnosis images and structured data, the problem of insufficient accuracy in early detection of coronary heart disease in traditional diagnostic methods has been solved, achieving more efficient risk assessment of coronary heart disease and improving early identification capabilities and model stability.

CN120878218APending Publication Date: 2025-10-31YANBIAN UNIV
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202510983657.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-07-17
Publication Date
2025-10-31

AI Technical Summary

Technical Problem

Existing traditional methods for diagnosing coronary heart disease have limitations in early detection, especially in terms of accuracy, making timely intervention difficult. Furthermore, the combination of tongue diagnosis images and structured data has limitations in terms of accuracy and reliability in non-invasive diagnosis.

Method used

A deep learning model integrating tongue diagnosis images and structured data was constructed. The features of the tongue image and structured data were extracted by CNN_GPT2 and CNN_Tabular models respectively, and the probability was predicted by the XGBoost ensemble model to build an early prediction model for coronary heart disease.

Benefits of technology

It improves the accuracy and reliability of coronary heart disease risk assessment, enhances the stability and generalization ability of the model, makes up for the limitations of single data types, and improves the early identification ability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120878218A_ABST
    Figure CN120878218A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of medical health risk assessment, and particularly discloses a coronary heart disease risk assessment method fusing a tongue diagnosis image and structured data, and the method comprises the steps: collecting the tongue image of a patient with coronary heart disease and the structured data such as age and gender; and extracting tongue image features by using a convolutional neural network, encoding the structured data through a multi-layer perceptron, and fusing the encoded structured data with the structured data. And a CNNGPT2 deep learning model and a CNNTabular deep learning model are constructed. After training, a test set prediction result is extracted to construct a metadata set, XGBoost is used as a meta classifier for training and prediction, a coronary heart disease early prediction model is constructed, and coronary heart disease risk assessment is performed based on the early prediction model. According to the method, the tongue diagnosis image and the structured data are fused, the limitation of a single data type is made up, and the comprehensiveness of evaluation is improved. According to the method, two deep learning models of CNNGPT2 and CNNTabular are constructed, association information between data is fully mined, and the processing capability of complex data is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of medical and health risk assessment technology, specifically to a method for assessing the risk of coronary heart disease by integrating tongue diagnosis images and structured data. Background Technology

[0002] Coronary heart disease (CHD) is one of the leading causes of death worldwide and poses a serious threat to public health. Despite significant advances in diagnosis and treatment in recent years, the global burden of CHD continues to increase, particularly in low- and middle-income countries.

[0003] Traditional methods for diagnosing coronary artery disease (CAD) primarily rely on clinical symptoms, electrocardiograms (ECG), and coronary angiography (CAG). However, these methods have limitations in early detection. For example, resting ECG has limited diagnostic value and low accuracy in detecting obstructive coronary artery disease in patients with stable angina. Furthermore, while coronary angiography is considered the gold standard for diagnosing CAD, its accuracy is limited, and the assessment of the degree of coronary artery stenosis depends on experienced physicians. These limitations render traditional methods insufficient for early detection of CAD, hindering timely intervention.

[0004] Therefore, exploring more efficient and scalable diagnostic strategies to improve the early identification of coronary heart disease has become an important research direction. In recent years, the application of artificial intelligence, especially deep learning technology, in medical image analysis has made significant progress.

[0005] Tongue images, as an important component of traditional Chinese medicine diagnosis, contain rich physiological and pathological information and have the potential to become an auxiliary diagnostic tool. Combining tongue images with structured data (such as blood glucose and blood lipids) to construct a multimodal deep learning model is expected to improve the accuracy of early prediction of coronary heart disease and make up for the shortcomings of traditional methods.

[0006] In the diagnosis and prevention of coronary heart disease, traditional Chinese medicine has accumulated a rich theoretical foundation and clinical experience since ancient times. The *Huangdi Neijing* (Yellow Emperor's Inner Classic) states that "the heart governs the blood vessels" and "the tongue is the sprout of the heart," indicating that changes in the tongue's appearance can reflect the state of cardiac function. This provides a theoretical basis for the application of tongue diagnosis in cardiovascular diseases. In recent years, with the development of evidence-based medicine, more and more studies have begun to attempt to verify the effectiveness of tongue diagnosis in the identification of coronary heart disease using modern scientific methods.

[0007] Although existing research has made some progress in tongue image analysis and deep learning applications, there are still some limitations, especially in terms of the accuracy of non-invasive diagnostic methods and their dependence on clinical medical imaging and clinical data.

[0008] Therefore, a method for assessing the risk of coronary heart disease that integrates tongue diagnosis images and structured data has become an urgent problem to be solved. Summary of the Invention

[0009] The technical problem this invention aims to solve is to provide a method for assessing the risk of coronary heart disease by fusing tongue images and structured data. Based on deep learning technology, it integrates tongue images and structured text data to construct a coronary heart disease risk prediction model. Two independent fusion models are designed to extract features from the tongue images and structured data respectively and then make predictions. Subsequently, a stacking ensemble strategy is used to integrate the predicted probabilities of the two fusion models and use them as input data for further prediction, thereby improving the overall predictive performance of the model.

[0010] To address the aforementioned technical problems, the present invention provides a method for assessing the risk of coronary heart disease by integrating tongue diagnosis images and structured data, comprising the following steps:

[0011] S1. Collect structured data including tongue images and basic personal information of patients with coronary heart disease;

[0012] S2. Extract tongue image features using CNN and encode structured data using MLP, then fuse the tongue image features with the structured data.

[0013] S3. Construct the CNN_GPT2 deep learning model and the CNN_Tabular deep learning model;

[0014] The CNN_GPT2 deep learning model extracts local structural features from tongue images using CNN, embeds these local structural features into the input sequence of GPT2, and models them together with structured text features.

[0015] The CNN_Tabular deep learning model uses a pre-trained ResNet-18 to extract image features, concatenates the image features extracted by CNN with the structured features after MLP encoding, and directly uses them for the final classification prediction.

[0016] S4. After training the CNN_GPT2 deep learning model and the CNN_Tabular deep learning model, extract the prediction results of the test set to construct a meta-dataset. Use XGBoost as a meta-classifier for training and prediction to construct an early prediction model for coronary heart disease. Based on this early prediction model, assess the risk of coronary heart disease.

[0017] Furthermore, the basic personal information includes age, gender, BMI, waist circumference, systolic blood pressure, diastolic blood pressure, whether one smokes, whether one has diabetes, whether one has hypertension, whether one has coronary heart disease, and a family history of cardiovascular disease.

[0018] Furthermore, after completing step S1, the structured data is organized, outliers are removed, and standardized coding and consistency verification are performed.

[0019] Statistical analysis methods were used to perform descriptive analysis on the basic characteristics of the processed dataset, and visualization tools were used to create relevant charts to show the relationships between variables and the overall distribution characteristics.

[0020] Furthermore, the data in the table is preprocessed by selecting the required features based on the column names and dividing them into numerical data and categorical data. The numerical data is standardized using the StandardScaler method, which adjusts the mean of each feature to 0 and the standard deviation to 1, thereby ensuring that all features are on the same scale.

[0021] Furthermore, the tongue image data is preprocessed, and the trained U-Net is used for image segmentation to extract the required parts from the original image. The image size is standardized to 128×128, then converted into a tensor, and normalized using the Normalize method. The mean and standard deviation of the normalization are set according to the statistical values ​​of the dataset.

[0022] Furthermore, the specific method for constructing the CNN_GPT2 deep learning model is as follows:

[0023] A pre-trained ResNet-18 model is used to extract features from the image to obtain its high-level semantic features: F img =ResNet18(I); where I is the input image, F img These are the extracted features;

[0024] The input image is a three-channel color image with a size of 128×128×3. After being processed by the convolution and pooling layers of ResNet-18, a 512-dimensional feature vector is extracted before the fully connected layer, which serves as the deep representation of the image.

[0025] To adapt the extracted image features to the input dimension of GPT-2, a fully connected layer is used to project the original 512-dimensional feature vector to 256 dimensions: F proj =Linear(F img )∈R 256 ;

[0026] The projected image features are input into the lightweight GPT-2 model for semantic modeling;

[0027] Meanwhile, the structured features are encoded using an MLP; the shape of the input structured features is: after passing through two fully connected layers, they are mapped into a 128-dimensional feature vector, which is used for subsequent fusion with image semantic features;

[0028] The image features output by GPT-2 are concatenated with the structured features output by MLP to obtain the fused feature representation: F combined =Concat(F gpt ,F tab ); where F gpt F represents the semantic features of the image generated by the mini GPT-2 model. tab This represents a high-level feature representation from structured tabular data; the concatenated feature dimension is 384.

[0029] The final classification is performed through a fully connected layer, which maps the fused features to the target category to complete the classification task.

[0030] Furthermore, the specific method for constructing the CNN_Tabular deep learning model is as follows:

[0031] The image features are extracted using a pre-trained ResNet-18 network. The input image size is 128×128×3. After passing through multiple convolutional layers, residual blocks, and pooling operations, a 512-dimensional image feature vector is finally output through a global average pooling layer, representing the deep semantic information of the image.

[0032] Meanwhile, the structured tabular data is encoded through an MLP, which maps the structured features into a 128-dimensional representation vector;

[0033] Image features and structured features are concatenated together to form a 640-dimensional fused feature vector, representing the multimodal information of the image and structured data;

[0034] These fused features are processed through a linear classifier for binary classification tasks.

[0035] Furthermore, the specific methods for constructing an early prediction model for coronary heart disease are as follows:

[0036] Obtain the predicted probability outputs of the CNN_GPT2 deep learning model and the CNN_Tabular deep learning model, and input them as meta-features into the XGBoost classifier to construct a fusion model;

[0037] XGBoost iteratively optimizes the objective function using a gradient boosting framework. In each iteration, XGBoost constructs a new decision tree to minimize the loss of the current model. In binary classification tasks, XGBoost uses cross-entropy loss to optimize the model's predicted probabilities, making the predicted probabilities as close as possible to the true labels, thereby improving classification accuracy. By reducing misclassification, cross-entropy loss is also used to prevent overfitting and enhance the model's robustness and generalization ability.

[0038] Furthermore, the optimal classification threshold for the binary classification task is found using the ROC curve. This optimal classification threshold θ is determined by maximizing the difference between the TPR and FPR.

[0039] Where TPR(θ) is the true positive rate at the optimal classification threshold θ, and FPR(θ) is the false positive rate at the optimal classification threshold θ. The optimal classification threshold is used to convert the predicted probabilities into the final classification result.

[0040] Where p i y is the predicted probability of the i-th sample, θ is the optimal classification threshold, and y is the predicted probability of the i-th sample. pred This is the final classification result.

[0041] The advantages of this invention compared to the prior art are:

[0042] This invention integrates tongue diagnosis images with structured data, making full use of the information contained in different types of data, making up for the limitations of single data types in coronary heart disease risk assessment, and improving the comprehensiveness of the assessment.

[0043] This invention constructs two deep learning models, CNN_GPT2 and CNN_Tabular, which process and fuse multimodal data from different perspectives, fully exploring the correlation information between data and improving the model's ability to process complex data.

[0044] This invention uses XGBoost as a meta-classifier to fuse the prediction results of two models, further integrating the advantages of different models and improving the accuracy and reliability of coronary heart disease risk assessment.

[0045] This invention enhances the stability and generalization ability of the model by performing rigorous preprocessing and feature engineering on the data, and by adopting the optimal classification threshold determination method, thus enabling the model to perform better in practical applications. Attached Figure Description

[0046] Figure 1 This is a flowchart of a method for assessing the risk of coronary heart disease that integrates tongue diagnosis images and structured data, according to the present invention.

[0047] Figure 2 It is a data feature distribution map.

[0048] Figure 3 It is a characteristic heatmap.

[0049] Figure 4 This is a U-Net image segmentation result diagram.

[0050] Figure 5 This is a diagram illustrating the process of image feature extraction using the CNN_GPT2 deep learning model.

[0051] Figure 6 It is a high-level semantic feature mining graph.

[0052] Figure 7 It is the MLP encoding and feature map of structured features in the CNN_GPT2 deep learning model.

[0053] Figure 8 This is a fusion diagram of image features and structured features in the CNN_GPT2 deep learning model.

[0054] Figure 9 This is a fusion diagram of image features and structured features in the CNN_Tabular deep learning model.

[0055] Figure 10 This is a flowchart of the construction process of a stacked fusion model based on tongue image and structured data.

[0056] Figure 11 It is a scatter plot of the performance of 100 independent experiments.

[0057] Figure 12 It is a trend chart of key indicators.

[0058] Figure 13 This is a key performance indicator (KPI) assessment chart.

[0059] Figure 14 It is a heatmap of tongue features in the test dataset. Detailed Implementation

[0060] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0061] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0062] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0063] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0064] The following detailed description, in conjunction with the accompanying drawings, illustrates the present invention's method for assessing coronary heart disease risk by integrating tongue diagnosis images and structured data. Figure 1-14 The specific implementation process of the coronary heart disease risk assessment method integrating tongue diagnosis images and structured data of the present invention is as follows:

[0065] The data used in this invention comes from 409 healthy volunteers in the Park Community of XX City.

[0066] I. Dataset

[0067] Basic information of the participants was systematically collected, including height, weight, hypertension, diabetes, family history of cardiovascular disease, and daily habits. Tongue images were also taken for each participant. After processing and preprocessing, the experimental dataset required for this invention was finally constructed. The specific features of the dataset are shown in Table 1.

[0068] Table 1: Dataset Characteristics

[0069]

[0070] In the data preprocessing stage, this invention systematically cleaned and organized the collected raw data. Outliers were specifically checked and processed, and data items significantly exceeding normal physiological ranges were removed. Standardized coding and consistency checks were performed on categorical variables such as hypertension and diabetes to ensure data accuracy and standardization. After data cleaning, statistical analysis methods were further employed to conduct descriptive analysis of the dataset's basic characteristics to comprehensively understand the distribution of variables. Simultaneously, relevant charts were created using visualization tools (as attached). Figure 2 As shown in the figure, the correlation between the variables and the overall distribution characteristics are clearly displayed.

[0071] Combined with appendix Figure 2The scatter plots illustrate the relationship between BMI and age (AGE), waist circumference (WC), systolic blood pressure (SP), and diastolic blood pressure (DP), and are stratified and visualized according to coronary artery disease (CAD) status. The plots show that individuals with CAD are primarily concentrated in the 65+ age group. The distribution of CAD-positive samples in the relationship between BMI and systolic and diastolic blood pressure is more dispersed, indicating potentially more complex individual variability in the relationship between blood pressure and BMI within the affected population.

[0072] Furthermore, histograms and kernel density estimation (KDE) curves of the numerical variables revealed the overall distribution characteristics of each indicator. Both age and BMI showed a right-skewed distribution, indicating that the sample was mainly concentrated in the middle-aged and elderly stage and the moderately overweight range. Regarding blood pressure, diastolic blood pressure was mainly concentrated in the 70–80 mmHg range, while systolic blood pressure was mainly concentrated at approximately 140 mmHg, showing a certain proportion of hypertension. To further explore the correlation structure between variables, this invention plotted a heatmap (see attached diagram). Figure 3 This visually presents the correlation coefficient matrix between various feature variables, which helps to identify potential linear relationships and provides guidance for feature selection and fusion strategies in subsequent modeling.

[0073] II. Data Preprocessing

[0074] The preprocessing method for tabular data is to select the required features based on the column names and divide them into numerical data and categorical data. The numerical data is standardized using the StandardScaler method, which adjusts the mean of each feature to 0 and the standard deviation to 1, thereby ensuring that all features are on the same scale.

[0075] The image data preprocessing method involves first segmenting the image using a trained U-Net to extract the desired portions from the original image. (See attached image.) Figure 4 As shown, the original image, feature labels, and segmentation results are presented. It can be seen that the segmented image clearly highlights the tongue region. Segmenting the original image using U-Net allows for better extraction of the tongue region. The segmented image not only removes background noise but also highlights the features of the target region, enabling subsequent models to focus more on key information such as the tongue's shape and texture.

[0076] Next, the image size is standardized to 128×128, then converted into a tensor, and normalized using the Normalize method. The mean and standard deviation of the normalization are set according to the statistical values ​​of the dataset, which helps the subsequent model to learn image features better.

[0077] III. Model Design

[0078] To achieve non-invasive intelligent prediction of coronary heart disease risk, this invention constructs a multimodal deep learning framework that integrates tongue image and structured data to improve the accuracy and applicability of the model in practical applications. This framework comprises two main sub-models: the CNN_Tabular deep learning model and the CNN_GPT2 deep learning model.

[0079] The CNN_Tabular deep learning model aims to jointly model tongue images and structured numerical features. It uses a shared convolutional neural network to extract image information, while introducing a multilayer perceptron to process structured features. Features are then concatenated in a fusion layer for joint prediction. The CNN_GPT2 deep learning model encodes image information as an "image-text sequence," introducing a mini-GPT-2 model to achieve deep semantic modeling and contextual mining, enhancing the model's understanding of high-level tongue image features. Finally, this invention uses a stacking ensemble strategy to fuse the prediction results of the two sub-models, fully leveraging the complementary advantages of different modalities to improve the overall prediction performance and generalization ability of the entire model.

[0080] First, the CNN_GPT2 deep learning model is a multimodal model that integrates convolutional neural networks (CNNs) and a mini-GPT-2 model, aiming to achieve deep fusion of tongue image features and textual semantic information. The CNN_GPT2 deep learning model first extracts local structural features from the tongue image using CNNs, then embeds these image features into the input sequence of GPT2, jointly modeling them with structured text features to achieve cross-modal information fusion and interactive learning. The entire operation flow of the CNN_GPT2 deep learning model is shown in the attached figure. Figure 5 As shown, this fusion strategy not only enhances the CNN_GPT2 deep learning model's ability to represent complex patterns, but also improves its generalization performance in few-shot tasks.

[0081] First, as attached Figure 5 As shown, a pre-trained ResNet-18 model is used to extract features from the image to obtain its high-level semantic features, such as: F img =ResNet18(I), where I is the input image, F img The extracted features are typically a 128×128×3 three-channel color image. After processing by the convolution and pooling layers of ResNet-18, a 512-dimensional feature vector is finally extracted before the fully connected layer, serving as the deep representation of the image.

[0082] To adapt the extracted image features to the input dimension of GPT-2, a fully connected layer is used to project the original 512-dimensional feature vector to 256 dimensions, such as: F proj =Linear(F img )∈R 256 This step projects the image features from the original 512 dimensions to 256 dimensions through a linear transformation, ensuring that its dimensionality matches the input requirements of the GPT-2 model. Subsequently, the projected image features are input into the lightweight GPT-2 model for semantic modeling. Leveraging its powerful contextual modeling capabilities, GPT-2 can further extract high-level semantic information from the feature sequence, enhancing the expressiveness of the image representation, as shown in the attached figure. Figure 6 As shown.

[0083] Meanwhile, as attached Figure 7 As shown, structured features (such as age, gender, BMI, etc.) are encoded using a multilayer perceptron (MLP). The input structured features are of shape [shape missing], and after passing through two fully connected layers, they are mapped into a 128-dimensional feature vector, which is used for subsequent fusion with image semantic features.

[0084] Multilayer perceptrons (MLPs) encode structured data through a series of nonlinear transformations, extracting its high-level feature representations and ultimately outputting a 128-dimensional vector. Meanwhile, the image semantic features from the lightweight GPT-2 model are 256-dimensional.

[0085] The following is attached. Figure 8 As shown, the image features output by GPT-2 are concatenated with the structured features output by MLP to obtain the fused feature representation, such as: F combined =Concat(F gpt ,F tab ), where F gpt F represents the semantic features of the image generated by the mini GPT-2 model. tab This indicates a high-level feature representation from structured tabular data; the concatenated feature dimension is 384.

[0086] The feature concatenation (Concat) operation fuses information from two modalities into a unified multimodal representation F. combined This fused feature not only contains the deep semantic content of the image but also preserves key pattern information from the structured data. Finally, a fully connected layer is used for final classification. The fully connected layer maps the fused features to the target category, completing the classification task.

[0087] The advantage of this method lies in the powerful context modeling capabilities of the Transformer architecture, which can deeply mine the semantic information in images, thereby improving the richness and discriminativeness of feature representation. Secondly, by adopting the Transformer architecture, the stability problem of traditional CNN features with a limited number of samples can be effectively alleviated, thus significantly improving the model's generalization ability in low-resource environments.

[0088] Model 2 uses a fusion structure of CNN and TabularMLP (denoted as CNN_Tabular deep learning model). This model also uses pre-trained ResNet-18 to extract image features, but no longer introduces the Transformer structure. Instead, it concatenates the image features extracted by CNN with the structured features encoded by a multilayer perceptron (MLP) and uses them directly for the final classification prediction.

[0089] In the experiment, the CNN_Tabular deep learning model served as the control group, forming the two main components of an ensemble model together with the CNN_GPT2 deep learning model. The CNN_Tabular model independently utilizes image and structured data features for classification tasks. It does not rely on the GPT-2 generative model but instead employs a more traditional deep learning structure for fusion and prediction. Specifically, the CNN_Tabular model uses a pre-trained ResNet-18 network to extract features from the image. The input image size is 128×128×3. After passing through multiple convolutional layers, residual blocks, and pooling operations, a 512-dimensional image feature vector is output through a global average pooling layer, representing the deep semantic information of the image.

[0090] Meanwhile, the structured tabular data is encoded using a multilayer perceptron (MLP), mapping the structured features into a 128-dimensional representation vector. Subsequently, the image features and structured features are concatenated to form a 640-dimensional fused feature vector, representing the multimodal information of the image and structured data. Finally, these fused features are processed by a linear classifier for a binary classification task, as shown in the attached diagram. Figure 9 As shown.

[0091] As a control group, the CNN_Tabular deep learning model excels in its simple and efficient structure, leveraging the complementarity of image and structured data for classification. It avoids the generative mechanisms of the GPT-2 model, focusing instead on the fusion of image and tabular data in deep learning. This approach can serve as a benchmark, comparing it to more complex models such as the CNN_GPT2 deep learning model to evaluate the performance differences of different architectures in handling multimodal data. Therefore, the CNN_Tabular deep learning model provides a comparative reference for performance optimization of ensemble models.

[0092] To improve the overall performance and discrimination ability of coronary heart disease prediction models, this invention constructs a stacking ensemble model based on tongue images and structured data. The specific method involves processing the tongue images using a convolutional neural network (CNN) and processing the structured features using a deep learning model, obtaining the prediction probability outputs of the two sub-models, and then inputting these as meta-features into an XGBoost classifier to construct a fusion model for better classification results. The overall process is shown in the attached figure. Figure 10 As shown.

[0093] In this invention, the input features of the XGBoost model are derived from the predicted probabilities of the base models CNN_GPT2 and CNN_Tabular deep learning models. These probability values, as high-level meta-features, represent the confidence level of each base model in that the sample belongs to the positive class (coronary heart disease). The target label is the true class label of the sample, used to supervise the training process, thereby guiding the XGBoost model to learn how to integrate the outputs of the base models to achieve a more accurate final classification.

[0094] XGBoost iteratively optimizes the objective function using the gradient boosting framework. In each iteration, XGBoost constructs a new decision tree to minimize the loss of the current model. In binary classification tasks, XGBoost typically employs cross-entropy loss, which effectively measures the difference between the model's predicted probabilities and the true labels. Its purpose is to improve classification accuracy by optimizing the model's predicted probabilities to be as close as possible to the true labels. By reducing misclassification, cross-entropy loss also helps the model prevent overfitting, enhancing its robustness and generalization ability.

[0095] To further optimize classification performance, the optimal classification threshold is then found using the ROC curve. The optimal classification threshold θ is typically determined by maximizing the difference between the True Positive Rate (TPR) and the False Positive Rate (FPR), as follows: Here, TPR(θ) is the true positive rate at the optimal classification threshold θ, and FPR(θ) is the false positive rate at the optimal classification threshold θ. This process helps to find the optimal balance point in classification decision, thereby improving the overall performance of the model.

[0096] The predicted probabilities are converted into the final classification result using the optimal classification threshold, such as: Where p i y is the predicted probability of the i-th sample, θ is the optimal classification threshold, and y is the predicted probability of the i-th sample. pred This is the final classification result.

[0097] IV. Experimental Results

[0098] To comprehensively evaluate the performance of the proposed model, this experiment designed and implemented 100 stability experiments with different random seeds to ensure the reliability and consistency of the results. In each experiment, the model was trained and predicted on the test set. The classification threshold was optimized by calculating the prediction probability and combining it with the ROC curve to maximize the model's classification ability. Based on this, key performance indicators such as accuracy, precision, recall, F1 score, and ROCAUC were systematically recorded. A systematic comparison with several comparative models is shown in Table 2.

[0099] Table 2: Comparative Experiments

[0100]

[0101]

[0102] Table 2 shows the comparison results between the present invention and various traditional machine learning methods on five key performance indicators (Accuracy, Precision, Recall, F1Score, and ROCAUC). Overall, the "graph-parameter integrated" model proposed in this invention significantly outperforms the comparison models on all evaluation indicators, demonstrating stronger classification ability and higher stability.

[0103] Among them, the "Graphical Parametric Integrated Model" achieved an accuracy of 0.963, significantly higher than other models; in terms of F1 score, the Graphical Parametric Integrated Model reached 0.914, far exceeding Logistic Regression's 0.531, indicating its superior balance between precision and recall. Furthermore, its ROCAUC reached 0.996, demonstrating its extremely strong discriminative ability in handling class imbalance problems.

[0104] To further evaluate the stability of the model under different random seeds, this invention plotted a scatter plot of the performance of the "graphical and parameter integrated" model in 100 independent experiments, covering key indicators such as accuracy, F1 score, and ROC, as shown in the appendix. Figure 11 As shown.

[0105] The results show that when the threshold is set between 0.425 and 0.5, the model's F1 score and AUC distribution are more concentrated at higher levels, and the performance fluctuation is relatively small, indicating that the model achieves a better balance in distinguishing between positive and negative samples within this range. Meanwhile, the ROCAUC index remains consistently high within this range, further validating the model's stable performance in overall recognition ability. Based on this observation, this invention employs a dynamic threshold strategy in subsequent evaluations, selecting the optimal threshold based on the validation set results in each experiment to improve the model's actual classification performance.

[0106] To analyze the performance changes of the model across multiple experiments, this invention plotted a line graph of 100 experiments, showing the trends in key metrics such as accuracy, F1 score, and ROCAUC under different random seeds, as shown in the attached figure. Figure 12 As shown.

[0107] The line graph clearly shows that the model's performance remained highly stable in most experiments, especially in the threshold range of 0.425 to 0.5, where the fluctuation range of the line was small, exhibiting relatively consistent high performance. This indicates that within this range, the model of this invention can stably achieve superior classification results, avoiding large performance fluctuations in extreme cases.

[0108] Furthermore, the line graph also demonstrates the model's sensitivity under different threshold settings. In particular, when the threshold deviates from the optimal range, performance begins to decline significantly, further validating the crucial role of selecting the optimal threshold range in the model's stability and efficiency. Through these experimental results, this invention confirms 0.425 as the lower threshold limit.

[0109] This invention comprehensively presents four key evaluation charts, which fully demonstrate the model's performance and stability under different performance metrics. Each sub-chart corresponds to a different dimension of model performance, providing a deeper understanding of the model's capabilities in classification tasks, as shown in the appendix. Figure 13 As shown.

[0110] First, the ROC curve shows the model's performance on the classification task (as shown in the attached figure). Figure 13 (As shown in Figure (a)). The shape of the ROC curve indicates that the model can effectively distinguish between positive and negative classes, and its AUC value is close to 1, demonstrating its excellent performance in positive and negative class identification. By maximizing TPR and minimizing FPR, the model effectively reduces the occurrence of false positives and false negatives, thereby improving its classification accuracy and reliability.

[0111] Secondly, the Precision-Recall curve and AP section show the model's Precision-Recall curve, indicating that the AP (Average Precision) value is 0.98 (see attached). Figure 13 (As shown in Figure (b)). The results show that the model of this invention can maintain high precision while improving recall, demonstrating its excellent performance in identifying positive samples. Especially in the case of class imbalance, the model can effectively identify positive samples and keep the false positive rate at a low level. The Precision-Recall curve reflects the efficiency and accuracy of the model in classifying positive samples.

[0112] Next, attached Figure 13 Figure (c) shows the confusion matrix of the model on the test set, where 63 represents true positives in the negative class (class 0), 2 represents false positives in the negative class, 1 represents false negatives in the positive class (class 1), and 16 represents true positives in the positive class. The accuracy for the positive class is approximately 0.8889, meaning that about 88.89% of all samples predicted as positive are actually positive. The accuracy for the negative class is approximately 0.9844, meaning that about 98.44% of all samples predicted as negative are actually negative. The confusion matrix results show that the model of this invention exhibits high accuracy in both positive and negative class classification, especially in the prediction of negative samples, where the model accurately identifies the majority of negative samples.

[0113] Appendix Figure 13 Figure (d) shows the predicted probability distribution of the model on different samples. This figure clearly presents the predicted probability distribution of the model, revealing its performance in the classification task. As can be seen from the figure, when the threshold is set to 0.46, the model of this invention can effectively distinguish most of the data, demonstrating good classification performance.

[0114] To further verify the effectiveness and generalization ability of the proposed model in practical applications, this invention used an independent test dataset for validation. This dataset contains 10 data points, 5 from diagnosed coronary heart disease patients and 5 from healthy control groups. The test results show that the model of this invention still has good classification ability on the new dataset, with an accuracy of 90%. The prediction results of the test dataset are shown in Table 3 below. Table 3 lists the ID, predicted probability, predicted label, and true label for each data point. For example, the data with ID 0 has a predicted probability of 0.608267, a predicted label of 1 (indicating disease), and a true label of 1, indicating that the model's prediction is correct.

[0115] Table 3: Prediction results for the test dataset

[0116]

[0117] Furthermore, a detailed analysis of the tongue images in the test dataset was conducted, extracting a series of representative feature images for diagnostic assistance. This invention uses heatmaps to illustrate the image features of interest to the model, as shown in the attached figure. Figure 14 As shown in the image, by generating a heatmap, the image regions that the model focuses on when making predictions can be seen intuitively.

[0118] Combined with appendix Figure 14 The test dataset contains tongue feature images, which demonstrate the tongue characteristics corresponding to different syndrome types. Heatmaps show the model's focus on these key areas. Test results indicate that this invention can effectively identify different syndrome types.

[0119] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. A method for assessing the risk of coronary heart disease by integrating tongue diagnosis images and structured data, characterized in that, Includes the following steps: S1. Collect structured data including tongue images and basic personal information of patients with coronary heart disease; S2. Extract tongue image features using CNN and encode structured data using MLP, then fuse the tongue image features with the structured data. S3. Construct the CNN_GPT2 deep learning model and the CNN_Tabular deep learning model; The CNN_GPT2 deep learning model extracts local structural features from tongue images using CNN, embeds these local structural features into the input sequence of GPT2, and models them together with structured text features. The CNN_Tabular deep learning model uses a pre-trained ResNet-18 to extract image features, concatenates the image features extracted by CNN with the structured features after MLP encoding, and directly uses them for the final classification prediction. S4. After training the CNN_GPT2 deep learning model and the CNN_Tabular deep learning model, extract the prediction results of the test set to construct a meta-dataset. Use XGBoost as a meta-classifier for training and prediction to construct an early prediction model for coronary heart disease. Based on this early prediction model, assess the risk of coronary heart disease.

2. The method for assessing coronary heart disease risk by integrating tongue diagnosis images and structured data according to claim 1, characterized in that: The basic personal information includes age, gender, BMI, waist circumference, systolic blood pressure, diastolic blood pressure, whether the person smokes, whether they have diabetes, whether they have hypertension, whether they have coronary heart disease, and a family history of cardiovascular disease.

3. The method for assessing coronary heart disease risk by integrating tongue diagnosis images and structured data according to claim 2, characterized in that: After completing step S1, the structured data is organized, outliers are removed, and standardized coding and consistency verification are performed. Statistical analysis methods were used to perform descriptive analysis on the basic characteristics of the processed dataset, and visualization tools were used to create relevant charts to show the relationships between variables and the overall distribution characteristics.

4. The method for assessing coronary heart disease risk by integrating tongue diagnosis images and structured data according to claim 3, characterized in that: The data in the table is preprocessed by selecting the required features based on the column names and dividing them into numerical data and categorical data. The numerical data is standardized using the StandardScaler method, which adjusts the mean of each feature to 0 and the standard deviation to 1, thereby ensuring that all features are on the same scale.

5. The method for assessing coronary heart disease risk by integrating tongue diagnosis images and structured data according to claim 4, characterized in that: The tongue image data is preprocessed by using a trained U-Net for image segmentation to extract the required parts from the original image. The image size is standardized to 128×128, then converted into a tensor, and normalized using the Normalize method. The mean and standard deviation of the normalization are set according to the statistical values ​​of the dataset.

6. The method for assessing coronary heart disease risk by integrating tongue diagnosis images and structured data according to claim 5, characterized in that: The specific method for constructing the CNN_GPT2 deep learning model is as follows: A pre-trained ResNet-18 model is used to extract features from the image to obtain its high-level semantic features: F im g = ResNet18(I); where I is the input image, F img These are the extracted features; The input image is a three-channel color image with a size of 128×128×3. After being processed by the convolution and pooling layers of ResNet-18, a 512-dimensional feature vector is extracted before the fully connected layer, which serves as the deep representation of the image. To adapt the extracted image features to the input dimension of GPT-2, a fully connected layer is used to project the original 512-dimensional feature vector to 256 dimensions: F projx Linear(F img )∈R 256 ; The projected image features are input into the lightweight GPT-2 model for semantic modeling; Meanwhile, the structured features are encoded using an MLP; the shape of the input structured features is: after passing through two fully connected layers, they are mapped into a 128-dimensional feature vector, which is used for subsequent fusion with image semantic features; The image features output by GPT-2 are concatenated with the structured features output by MLP to obtain the fused feature representation: F combined =Concat(F gpt ,F tab ); where F gpt F represents the semantic features of the image generated by the mini GPT-2 model. tab This represents a high-level feature representation from structured tabular data; the concatenated feature dimension is 384. The final classification is performed through a fully connected layer, which maps the fused features to the target category to complete the classification task.

7. The method for assessing coronary heart disease risk by integrating tongue diagnosis images and structured data according to claim 6, characterized in that: The specific method for constructing the CNN_Tabular deep learning model is as follows: The image features are extracted using a pre-trained ResNet-18 network. The input image size is 128×128×3. After passing through multiple convolutional layers, residual blocks, and pooling operations, a 512-dimensional image feature vector is finally output through a global average pooling layer, representing the deep semantic information of the image. Meanwhile, the structured tabular data is encoded through an MLP, which maps the structured features into a 128-dimensional representation vector; Image features and structured features are concatenated together to form a 640-dimensional fused feature vector, representing the multimodal information of the image and structured data; These fused features are processed through a linear classifier for binary classification tasks.

8. The method for assessing coronary heart disease risk by integrating tongue diagnosis images and structured data according to claim 7, characterized in that: The specific methods for constructing an early prediction model for coronary heart disease are as follows: Obtain the predicted probability outputs of the CNN_GPT2 deep learning model and the CNN_Tabular deep learning model, and input them as meta-features into the XGBoost classifier to construct a fusion model; XGBoost iteratively optimizes the objective function using a gradient boosting framework. In each iteration, XGBoost constructs a new decision tree to minimize the loss of the current model. In binary classification tasks, XGBoost uses cross-entropy loss to optimize the model's predicted probabilities, making the predicted probabilities as close as possible to the true labels, thereby improving classification accuracy. By reducing misclassification, cross-entropy loss is also used to prevent overfitting and enhance the model's robustness and generalization ability.

9. The method for assessing coronary heart disease risk by integrating tongue diagnosis images and structured data according to claim 8, characterized in that: The optimal classification threshold θ for a binary classification task is found using the ROC curve. This optimal classification threshold is determined by maximizing the difference between the TPR and FPR. Where TPR(θ) is the true positive rate at the optimal classification threshold θ, and FPR(θ) is the false positive rate at the optimal classification threshold θ. The optimal classification threshold is used to convert the predicted probabilities into the final classification result. Where p i y is the predicted probability of the i-th sample, θ is the optimal classification threshold, and y is the predicted probability of the i-th sample. pred This is the final classification result.

Citation Information

Cited By

  • Brain function connection intelligent screening method and system for autism spectrum disorder

    CN121943222A