Prediction method and device for disease-free lifetime risk of thyroid cancer patient
Through the multimodal feature fusion neural network model based on the gating mechanism, combined with multiple data sources, the information loss and diagnostic error problems caused by singlemodal feature extraction are solved, which significantly improves the prediction of disease-free survival risk in patients with thyroid cancer accuracy.
Patent Information
- Application Number
- CN202411080624.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-11-29
AI Technical Summary
The existing cancer prognosis prediction model focuses on the extraction of single-modal features, resulting in information loss and diagnostic errors, making it difficult to effectively integrate multimodal data to improve diagnostic and prediction performance.
Using a multimodal feature fusion neural network model based on gating mechanism, combining gene data, clinical data and pathological image data, bilinear fusion and attention mechanism are used through multi-task learning models such as TOAD, GAT, SNN and AttnNetGated. Dynamically adjust feature weights to enhance feature expression ability.
It significantly improves the prediction accuracy of disease-free survival risk in patients with thyroid cancer, reduces information loss, enhances the performance of diagnostic and prognostic prediction, and provides more comprehensive feature expression and more accurate risk assessment.
Smart Images

Figure CN119049707A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of thyroid disease risk prediction, and in particular to a method and device for predicting the disease-free survival risk of thyroid cancer patients. Background Art
[0002] Thyroid cancer is a common endocrine malignancy worldwide, and the occurrence of lymph node metastases (LNM) significantly affects the treatment prognosis of thyroid cancer. Therefore, accurately predicting the prognostic risk of thyroid cancer, that is, the risk of disease-free survival (DFS), is of great significance for formulating personalized treatment plans.
[0003] Existing cancer prognosis prediction models focus on the extraction of single-modal features, such as analyzing only genetic data or pathological image data. However, in complex prognosis prediction tasks, the feature expression ability of a single data source is limited, which can easily lead to information loss and diagnostic errors.
[0004] Therefore, how to effectively integrate multimodal data to improve the diagnostic and predictive performance of the model has become an urgent problem to be solved. Summary of the invention
[0005] This paper proposes a multimodal feature fusion neural network model based on a gating mechanism. By combining multimodal data such as genetic data and clinical data, the advantages of each data source are fully utilized to achieve a more comprehensive feature expression. Using the gating mechanism, the model can dynamically adjust the weights of different modal features to ensure that important information is retained during the fusion process and enhance the expressiveness of features, thereby solving the problems existing in the prior art and improving the accuracy of diagnosis and prognosis prediction.
[0006] According to one aspect of the present invention, a method for predicting the disease-free survival risk of thyroid cancer patients is provided, comprising the following steps: collecting data of thyroid cancer patients, including clinical characteristic data, pathological image data, gene expression data and immune cell data; training a multimodal deep learning model based on the collected data; wherein the trained model is used for classifying lymph node metastasis and scoring the disease-free survival risk to obtain classification results and scoring results; wherein the classification results and scoring results are used to predict the disease-free survival risk.
[0007] Preferably, collecting gene expression data and immune cell data includes: using transcriptome data to determine the expression levels of BRAF, RAS, and RET genes in thyroid cancer to obtain the gene expression data; based on the transcriptome data, using single-cell RNA sequencing scRNA-seq technology to measure the gene expression of individual immune cells in thyroid cancer to obtain the immune cell data.
[0008] Preferably, training a multimodal deep learning model includes: setting a multi-task learning model TOAD, a graph attention network GAT, a sequence neural network block SNN, and a gated attention network AttnNetGated; inputting the clinical feature data, the pathological image data, the gene expression data, and the immune cell data; performing bilinear fusion on the input data to obtain fused features; and inputting the fused features into a linear regression decoder to convert them into prediction results.
[0009] Preferably, setting TOAD, GAT, SNN, AttnNetGated includes: setting initial weight parameters for each component of each model; using labeled training data to gradually adjust the weight parameters of the model; wherein the GAT is used to learn the dependencies between nodes in the graph structure features using the attention mechanism; wherein the TOAD is used to combine the graph attention network and the fully connected layer to process various types of input data; wherein the SNN is used to serialize feature extraction and regularization and prevent overfitting, and includes a linear layer, an activation function and a dropout layer; wherein the AttnNetGated is used to use a gating mechanism to control the information flow, enhance the expressive power of the model, and extract features by learning the weighted representation of the input data.
[0010] Preferably, inputting the clinical feature data, the pathological image data, the gene expression data and the immune cell data includes: inputting the pathological image data into the AttnNetGated, the AttnNetGated uses the attention mechanism to identify and highlight the important areas in the pathological image, and extracts the key features in the pathological image within the important areas, and inputs the key features into the bilinear fusion module; inputting the gene expression data into the GAT to extract the graph structure features of gene expression; transferring the graph structure features of gene expression to the SNN to extract the deep features of gene expression, and inputting the deep features into the bilinear fusion module; directly inputting the clinical feature data and the immune cell data into the bilinear fusion module.
[0011] Preferably, after collecting the data of thyroid cancer patients, the method further includes: analyzing the cell type distribution according to the transcriptome data to obtain a characteristic analysis result; and identifying the correlation between the characteristic analysis result and the disease-free survival period and lymph node metastasis.
[0012] Preferably, training the multimodal deep learning model includes: using an Adam optimizer to train the multimodal deep learning model, combining early stopping and dropout techniques, and setting a learning rate and weight decay.
[0013] Preferably, using the trained model for the classification of lymph node metastasis and the scoring of the disease-free survival risk includes: predicting the situation of lymph node metastasis according to the output results of the trained model, and classifying it into a high-risk group or a low-risk group to obtain the classification result; quantifying the probability and level of the disease-free survival risk according to the output results of the trained model to obtain the scoring result.
[0014] According to another aspect of the present invention, a device for predicting the disease-free survival risk of thyroid cancer patients is provided, comprising: a data collection module, used to collect data of thyroid cancer patients, including clinical characteristic data, pathological image data and transcriptome data; a model training module, used to train a multimodal deep learning model based on the collected data; wherein the trained model is used for classifying lymph node metastasis and scoring the disease-free survival risk to obtain classification results and scoring results; wherein the classification results and scoring results are used to predict the disease-free survival risk.
[0015] Preferably, the model training module includes: a setting unit for setting TOAD, GAT, SNN, AttnNetGated; an input unit for inputting the clinical feature data, the pathological image data, the gene expression data and the immune cell data; a fusion unit for performing bilinear fusion on the input data to obtain fused features; and a conversion unit for inputting the fused features into a linear regression decoder to convert them into prediction results.
[0016] The present invention discloses a method and device for predicting the risk of disease-free survival of thyroid cancer patients, the method comprising the following steps: collecting data of thyroid cancer patients, including clinical feature data, pathological image data, gene expression data and immune cell data; training a multimodal deep learning model based on the collected data; wherein the trained model is used for the classification of lymph node metastasis and the scoring of the risk of disease-free survival, and the classification results and the scoring results are obtained; wherein the classification results and the scoring results are used to predict the risk of disease-free survival. The present invention solves the problems of information loss and diagnostic error in the diagnosis of PTC (papillary thyroid carcinoma), realizes the efficient fusion and utilization of multimodal data, and significantly improves the performance of diagnosis and prognosis prediction. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The drawings described herein are used to provide further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.
[0018] Figure 1 This is a flow chart of a method for predicting the risk of disease-free survival of thyroid cancer patients according to an embodiment of the present invention;
[0019] Figure 2A flowchart of a technical solution according to an embodiment of the present invention;
[0020] Figure 3 A diagram for initializing a model architecture according to an embodiment of the present invention;
[0021] Figure 4 A flow chart of the model operation according to an embodiment of the present invention; and
[0022] Figure 5 2 is a diagram showing the structure of a device for predicting the risk of disease-free survival of thyroid cancer patients according to an embodiment of the present invention. DETAILED DESCRIPTION
[0023] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0024] Reference will now be made in detail to various embodiments of the present invention, examples of which are shown in the accompanying drawings and described below. For ease of interpretation and precise definition in the appended claims, the terms "upper", "lower", "inner" and "outer" are used to describe features of the exemplary embodiments shown in the drawings with reference to their positions.
[0025] An embodiment of the present invention provides a method for predicting the disease-free survival risk of a thyroid cancer patient, comprising the following steps: collecting data of thyroid cancer patients, including clinical characteristic data, pathological image data, gene expression data, and immune cell data; training a multimodal deep learning model based on the collected data; wherein the trained model is used for classifying lymph node metastasis and scoring the disease-free survival risk to obtain classification results and scoring results; wherein the classification results and scoring results are used to predict the disease-free survival risk.
[0026] In the field of healthcare, especially in cancer diagnosis and treatment decision-making, in-depth analysis of patients' pathological images, genomic data, and transcriptome data is crucial. Existing models focus on the extraction of single-modal features, such as analyzing only genetic data or pathological image data. However, in complex medical diagnosis and prognosis prediction tasks, the feature expression ability of a single data source is limited, which can easily lead to information loss and diagnostic errors.
[0027] In the embodiment of the present invention, a multimodal feature fusion neural network model is proposed, which combines multimodal data such as genetic data and clinical data, fully utilizes the advantages of each data source, and realizes more comprehensive feature expression. The present invention can improve the accuracy of diagnosis and survival prognosis prediction of papillary thyroid carcinoma PTC.
[0028] According to an embodiment of the present invention, collecting gene expression data and immune cell data includes: using transcriptome data to determine the expression levels of BRAF, RAS, and RET genes in thyroid cancer to obtain the gene expression data; based on the transcriptome data, using single-cell RNA sequencing scRNA-seq technology to measure the gene expression of individual immune cells in thyroid cancer to obtain the immune cell data.
[0029] In the embodiment of the present invention, transcriptome data is used to accurately determine the expression levels of BRAF, RAS, and RET genes in thyroid cancer, and single-cell RNA sequencing technology is used to measure the gene expression of individual immune cells, which not only improves the accuracy and comprehensiveness of gene expression data, but also integrates transcriptome data and single-cell sequencing data to achieve a comprehensive understanding of the tumor microenvironment.
[0030] According to an embodiment of the present invention, training a multimodal deep learning model includes: setting TOAD, GAT, SNN, AttnNetGated; inputting the clinical feature data, the pathological image data, the gene expression data, and the immune cell data; performing bilinear fusion on the input data to obtain fused features; and inputting the fused features into a linear regression decoder to convert them into prediction results.
[0031] In an embodiment of the present invention, multimodal data is effectively fused and processed by using a gated multimodal unit, a graph neural network feature extraction unit, a bilinear transformation, and a jump connection mechanism. The model feature extraction part uses a graph neural network to extract and construct the expression correlation between different genes, i.e., co-expression, for the expression input form of the gene, and then uses an independent embedded feature layer to extract features. In the process of multimodal feature fusion, the use of a bilinear fusion method can effectively combine multiple features and improve the expressiveness and predictive ability of the fused features.
[0032] According to an embodiment of the present invention, setting TOAD, GAT, SNN, and AttnNetGated includes: setting initial weight parameters for each component of each model; using labeled training data to gradually adjust the weight parameters of the model; wherein the GAT is used to learn the dependencies between nodes in the graph structure features using the attention mechanism; wherein the TOAD is used to process various types of input data in combination with the graph attention network and the fully connected layer; wherein the SNN is used to serialize feature extraction and regularization and prevent overfitting, and includes a linear layer, an activation function, and a dropout layer; wherein the AttnNetGated is used to use a gating mechanism to control the information flow, enhance the expressive power of the model, and extract features by learning the weighted representation of the input data.
[0033] In the embodiment of the present invention, the modules TOAD, GAT, SNN, and AttnNetGated work together in a multi-task learning framework to process and analyze information from different data sources, and ultimately achieve prediction of multiple tasks.
[0034] According to an embodiment of the present invention, inputting the clinical feature data, the pathological image data, the gene expression data and the immune cell data includes: inputting the pathological image data into the AttnNetGated, the AttnNetGated uses the attention mechanism to identify and highlight the important areas in the pathological image, and extracts the key features in the pathological image within the important areas, and inputs the key features into the bilinear fusion module; inputting the gene expression data into the GAT to extract the graph structure features of gene expression; transferring the graph structure features of gene expression to the SNN to extract the deep features of gene expression, and inputting the deep features into the bilinear fusion module; and directly inputting the clinical feature data and the immune cell data into the bilinear fusion module.
[0035] In an embodiment of the present invention, pathological image data is input into AttnNetGated, and the attention mechanism is used to highlight important areas and extract key features; gene expression data is input into GAT to extract graph structure features, and then transmitted to SNN to extract deep features; clinical feature data and immune cell data are directly input into the bilinear fusion module, which can realize the fusion of multiple data types and improve the accuracy and comprehensiveness of data analysis.
[0036] According to an embodiment of the present invention, after collecting data of thyroid cancer patients, it also includes: analyzing cell type distribution based on the transcriptome data to obtain feature analysis results; identifying the correlation between the feature analysis results and the disease-free survival period and lymph node metastasis.
[0037] In the embodiment of the present invention, the obtained feature analysis results, i.e., the specific gene features and mutation features input by the model, can be used in the subsequent TOAD module. In addition, through correlation analysis, the potential relationship between the above features and the patient's disease-free survival and lymph node metastasis is identified, thereby providing important biomarkers for the prognosis assessment of thyroid cancer.
[0038] According to an embodiment of the present invention, training the multimodal deep learning model includes: using an Adam optimizer to train the multimodal deep learning model, combining early stopping and dropout techniques, and setting a learning rate and weight decay.
[0039] In the embodiment of the present invention, the Adam optimizer is used to train the multimodal deep learning model, and the early stopping and dropout techniques are combined, and the learning rate and weight decay are set at the same time. Compared with the related art, the present invention accelerates the convergence of the model by using the Adam optimizer, improves the training efficiency; combines the early stopping technique to improve the generalization ability of the model; applies the dropout technique to increase the robustness of the model and prevent overfitting; sets the learning rate and weight decay to further optimize the training process of the model and ensure the stability and accuracy of the model.
[0040] According to an embodiment of the present invention, using the trained model for the classification of lymph node metastasis and the scoring of the disease-free survival risk includes: predicting the situation of lymph node metastasis according to the output result of the trained model, and classifying it into a high-risk group or a low-risk group to obtain the classification result; and quantifying the probability and level of the disease-free survival risk according to the output result of the trained model to obtain the scoring result.
[0041] In an embodiment of the present invention, the trained model is used for the classification of lymph node metastasis and the scoring of disease-free survival risk. According to the output results of the model, the lymph node metastasis is predicted and classified into a high-risk group or a low-risk group to obtain a classification result; at the same time, according to the output results of the model, the probability and level of disease-free survival risk are quantified to obtain a scoring result. Compared with the related art, the present invention can more accurately predict and classify lymph node metastasis through a deep learning model trained with comprehensive multimodal data, and effectively distinguish high-risk groups from low-risk groups. In addition, through the precise quantification of the model, a specific score for disease-free survival risk is provided, making the risk assessment more comprehensive and accurate, and improving the accuracy and reliability of disease prognosis.
[0042] Another embodiment of the present invention provides a device for predicting the disease-free survival risk of thyroid cancer patients, including: a data collection module, used to collect data of thyroid cancer patients, including clinical characteristic data, pathological image data and transcriptome data; a model training module, used to train a multimodal deep learning model based on the collected data; wherein the trained model is used for classifying lymph node metastasis and scoring the disease-free survival risk to obtain classification results and scoring results; wherein the classification results and scoring results are used to predict the disease-free survival risk.
[0043] In an embodiment of the present invention, a device combining a multimodal feature fusion neural network model is proposed, which combines multimodal data such as genetic data and clinical data, fully utilizes the advantages of each data source, and realizes a more comprehensive feature expression. The present invention can not only improve the accuracy of PTC diagnosis and survival prognosis prediction, but also identify high-risk areas in pathological images through visualization technology, provide clinicians with intuitive and reliable diagnostic basis, and assist in formulating more effective treatment plans.
[0044] According to an embodiment of the present invention, the model training module includes: a setting unit for setting TOAD, GAT, SNN, AttnNetGated; an input unit for inputting the clinical feature data, the pathological image data, the gene expression data and the immune cell data; a fusion unit for performing bilinear fusion on the input data to obtain fused features; and a conversion unit for inputting the fused features into a linear regression decoder to convert them into prediction results.
[0045] In the embodiment of the present invention, by combining the multi-task learning model with the attention mechanism, the model's ability to process multimodal data is improved, and the accuracy and effectiveness of feature extraction are enhanced; the bilinear fusion method realizes efficient fusion of data and improves the expression ability of comprehensive features; the linear regression decoder ensures the accuracy and reliability of the prediction results. Combining these technologies, the present invention shows higher performance and stability in disease prediction and risk assessment.
[0046] The present invention is further described in detail below in conjunction with specific implementation modes and accompanying drawings.
[0047] Figure 1 This is a flow chart of a method for predicting the risk of disease-free survival of thyroid cancer patients according to an embodiment of the present invention. Figure 1 As shown, the present invention provides a method for predicting the disease-free survival risk of thyroid cancer patients, comprising the following steps: collecting data of thyroid cancer patients, including clinical characteristic data, pathological image data, gene expression data and immune cell data; training a multimodal deep learning model based on the collected data; wherein the trained model is used for classifying lymph node metastasis and scoring the disease-free survival risk to obtain classification results and scoring results; wherein the classification results and scoring results are used to predict the disease-free survival risk.
[0048] Figure 2 is a flow chart of the technical solution according to an embodiment of the present invention, such as Figure 2 As shown, the technical solution process of the present invention includes:
[0049] (1) Data collection and patient characteristics analysis
[0050] Patient inclusion criteria: Clarify the conditions for patients to be included in the study, such as diagnosis of PTC, clinical stage, etc.
[0051] Data collection: Collect patients’ basic information, clinical data, treatment history, and follow-up data.
[0052] Patient characteristics statistics: Descriptive statistical analysis was performed on the patients' age, gender, tumor size, lymph node status, etc.
[0053] (2) Molecular characteristic analysis
[0054] Gene mutation analysis: NGS (next generation sequencing) technology is used to identify common gene mutations in PTC, such as BRAF, RAS, and RET.
[0055] Mutation frequency statistics: Calculate the frequency of occurrence of different gene mutations in patients.
[0056] Molecular subtype classification: Patients are divided into different molecular subtypes according to the type of gene mutation.
[0057] (3) Identification of prognosis-related genes and cell types
[0058] Gene expression analysis: Analyze the relationship between different gene expression levels and prognosis.
[0059] Cell type distribution: Identify cell types and their distribution in the tumor microenvironment by scRNA-seq.
[0060] Correlation analysis: Investigate the correlation of specific genes and cell types with DFS and LNM.
[0061] (4) Development of multimodal deep learning models
[0062] Model architecture design: Design a deep learning model that can process multi-source data.
[0063] Data preprocessing: WSIs (whole section images of intraoperative frozen sections), genomic and transcriptomic data were standardized and normalized.
[0064] Feature fusion strategy: Determine the fusion method of features from different data sources.
[0065] (5) Feature extraction and model training
[0066] Image feature extraction: Use the ResNet50 model to extract features from pathological slice images.
[0067] Gene feature extraction: Extract features using graph neural networks based on gene expression and co-expression networks.
[0068] Attention Mechanism: Identify key areas in the image through a self-attention module.
[0069] Training strategy: Use Adam optimizer, set learning rate and weight decay, and apply early stopping and dropout techniques.
[0070] (6) Construction and optimization of prognostic models
[0071] Risk scoring system: Build a DFS risk scoring system based on the model output.
[0072] Multi-task learning: The model is trained simultaneously for LNM classification and DFS time prediction.
[0073] Loss function adjustment: Adjust the weights of different loss functions to optimize model performance.
[0074] (7) Model verification and performance evaluation
[0075] Cross-validation: Use cross-validation method to evaluate the generalization ability of the model.
[0076] Performance indicators: The area under the ROC curve (AUC) and Kaplan-Meier survival curves were calculated.
[0077] Independent Cohort Validation: The predictive accuracy of the model was validated on an independent patient cohort.
[0078] (8) Model interpretation and visualization
[0079] GradCAM Heatmap: Generates a heatmap showing the areas that the model focuses on.
[0080] Feature importance evaluation: Evaluate the contribution of different features to model prediction.
[0081] Visualization tool development: Develop tools to help clinicians understand model predictions.
[0082] (9) Statistical analysis
[0083] Descriptive Statistics: Descriptive statistics were performed for both continuous and categorical variables.
[0084] Survival analysis: Survival data were analyzed using the Kaplan-Meier method and the log-rank test.
[0085] Gene network construction: WGCNA was used to analyze the gene co-expression network.
[0086] (10) Interpretation of results and clinical application
[0087] Risk grouping: Patients are divided into high-risk and low-risk groups based on their risk scores.
[0088] Prognostic difference analysis: compare the DFS differences among different risk groups.
[0089] Clinical decision support: Explore how models can assist clinical decision making and personalized treatment
[0090] Figure 3 This is a diagram for initializing the model architecture according to an embodiment of the present invention. Figure 3 As shown, the model first randomly initializes the weight parameters of each module, and then gradually optimizes the iterative parameters through supervised learning. The model architecture initialization of the present invention includes:
[0091] (1) GAT (Graph Attention Network)
[0092] The GAT module implements a graph attention network, which is a neural network for graph-structured data. GAT learns the weights between nodes through the attention mechanism, so that it can capture the complex dependencies between nodes. In the code you provided, the GAT class defines multiple attention layers, each layer consists of multiple sub-attention heads, which process input features in parallel and use different attention scores to aggregate neighbor information.
[0093] SpGraphAttentionLayer is an implementation of a graph attention layer that aggregates features of neighboring nodes using a learnable attention score.
[0094] The forward method defines the flow of data through the GAT network, including feature pooling, fully connected layer processing, and classifier output.
[0095] (2)TOAD (Tumor Origin Assessment via Deep-learning)
[0096] The TOAD module is a multi-task learning model that combines graph attention networks and fully connected layers to process multiple types of input data, such as pathology, gene, and mutation data. The TOAD module uses an attention mechanism to extract features and may use different fusion strategies to combine features from different sources.
[0097] (3) SNN (Siamese Neural Network)
[0098] SNN module It is a sequential neural network block for feature extraction. This block contains linear layers, activation functions, and dropout layers for regularization and prevention of overfitting.
[0099] (4) AttnNetGated (Attention Network with Gating)
[0100] The AttnNetGated module is an attention network using a gating mechanism, which extracts features by learning a weighted representation of the input data. This network uses the Sigmoid function as a gating mechanism to control the flow of information. The Attn_Net_Gated class defines a gated attention network with two attention branches (a and b), which process the input data through different activation functions.
[0101] These modules work together in a multi-task learning framework to process and analyze information from different data sources, ultimately achieving predictions for multiple tasks. For example, GAT can be used to learn representations of graph-structured data, while TOAD can integrate these representations and features from other data sources to perform the final classification or regression task.
[0102] Figure 4 This is a flow chart of the model operation according to an embodiment of the present invention. Figure 4 As shown, the model operation process of the present invention includes:
[0103] (1) Initialization
[0104] TOAD, GAT, SNN, AttnNetGated: These are the components and modules for model initialization. Each module will load relevant parameters and configurations during initialization to prepare for subsequent data processing and feature extraction.
[0105] (2) Feature extraction
[0106] Pathology image (AttnNetGated): The pathology image is input into the AttnNetGated module to extract key features in the image. AttnNetGated uses the attention mechanism to identify and highlight important areas in the pathology image for more accurate analysis.
[0107] Gene expression (GAT & SNN): Gene expression data is first input into the Graph Attention Network (GAT) to extract the graph structure features of gene expression. Next, these features are passed to the SNN (Siamese Neural Network) to further extract the deep features of gene expression.
[0108] Immune cells: Immune cell data are directly input into the bilinear fusion module. These data may include information such as the proportion and activity status of different types of immune cells.
[0109] Clinical features: Clinical feature data are also directly input into the bilinear fusion module. These features include the patient’s basic information, medical history, treatment methods, etc.
[0110] (3) Feature Fusion
[0111] Bilinear fusion: In this stage, data features from different sources (pathological images, gene expression, immune cells, clinical features) are fused. The bilinear fusion method can effectively combine multiple features and improve the predictive ability of the model.
[0112] (4) Prediction
[0113] Linear regression decoder: The fused features are input into the linear regression decoder. This decoder is responsible for converting high-dimensional feature vectors into specific prediction results, such as survival time prediction, disease progression probability, etc.
[0114] Figure 5 This is a diagram showing the structure of a device for predicting the risk of disease-free survival of thyroid cancer patients according to an embodiment of the present invention. Figure 5 As shown, the device for predicting the disease-free survival risk of thyroid cancer patients provided by the present invention includes: a data collection module, used to collect data of thyroid cancer patients, including clinical characteristic data, pathological image data and transcriptome data; a model training module, used to train a multimodal deep learning model based on the collected data; wherein the trained model is used for classifying lymph node metastasis and scoring the disease-free survival risk to obtain classification results and scoring results; wherein the classification results and scoring results are used to predict the risk of disease-free survival.
[0115] The above embodiments are only examples for clearly explaining the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications may be made based on the following description, and these various changes, modifications, substitutions and deformations derived from the principles and spirit of the present invention still fall within the scope of protection of the present invention.
Claims
1. A method for predicting the risk of disease-free survival in patients with thyroid cancer, characterized in that: The steps include: Collect data on thyroid cancer patients, including clinical characteristics, pathological images, gene expression, and immune cell data; Based on the collected data, train a multimodal deep learning model; The trained model is used for the classification of lymph node metastasis and the scoring of the disease-free survival risk to obtain the classification result and the scoring result; The classification results and scoring results are used to predict the risk of disease-free survival.
2. The method according to claim 1, characterized in that Collection of gene expression data and immune cell data including: Using transcriptome data, determining the expression levels of BRAF, RAS, and RET genes in thyroid cancer to obtain the gene expression data; According to the transcriptome data, single-cell RNA sequencing scRNA-seq technology is used to measure the gene expression of individual immune cells in thyroid cancer to obtain the immune cell data.
3. The method according to claim 2, characterized in that Training a multimodal deep learning model involves: Set up the multi-task learning model TOAD, graph attention network GAT, sequence neural network block SNN, and gated attention network AttnNetGated; inputting the clinical characteristic data, the pathological image data, the gene expression data and the immune cell data; Perform bilinear fusion on the input data to obtain fused features; The fused features are input into a linear regression decoder to be converted into prediction results.
4. The method according to claim 3, characterized in that Setting up TOAD, GAT, SNN, AttnNetGated includes: Set initial weight parameters for each component of each model; Use labeled training data to gradually adjust the model's weight parameters; The GAT is used to learn the dependencies between nodes in the graph structure features using the attention mechanism; Among them, the TOAD is used to process various types of input data in combination with the graph attention network and the fully connected layer; The SNN is used for serialized feature extraction and regularization and to prevent overfitting, and includes a linear layer, an activation function, and a dropout layer; The AttnNetGated is used to control the information flow using a gating mechanism, enhance the expressiveness of the model, and extract features by learning a weighted representation of the input data.
5. The method according to claim 3, characterized in that: Inputting the clinical feature data, the pathological image data, the gene expression data and the immune cell data comprises: Input the pathological image data into the AttnNetGated, the AttnNetGated uses an attention mechanism to identify and highlight important areas in the pathological image, extract key features in the pathological image in the important areas, and input the key features into a bilinear fusion module; Inputting the gene expression data into the GAT to extract the graph structure features of gene expression; transferring the graph structure features of gene expression to the SNN to extract the deep features of gene expression, and inputting the deep features into the bilinear fusion module; The clinical feature data and the immune cell data are directly input into the bilinear fusion module.
6. The method according to claim 2, characterized in that After collecting data on thyroid cancer patients, it also included: Analyzing cell type distribution according to the transcriptome data to obtain characteristic analysis results; Correlations between the signature analysis results and the disease-free survival and lymph node metastasis are identified.
7. The method according to claim 1, characterized in that Training the multimodal deep learning model includes: The multimodal deep learning model is trained using the Adam optimizer, combined with early stopping and dropout techniques, while setting the learning rate and weight decay.
8. The method according to claim 1, characterized in that The trained model was used for the classification of lymph node metastasis and the scoring of the disease-free survival risk including: According to the output result of the trained model, the situation of lymph node metastasis is predicted and classified into a high-risk group or a low-risk group to obtain the classification result; According to the output results of the trained model, the probability and level of the disease-free survival risk are quantified to obtain the scoring result.
9. A device for predicting the risk of disease-free survival of thyroid cancer patients, characterized in that: include: A data collection module, used to collect data of thyroid cancer patients, including clinical characteristic data, pathological image data, and transcriptome data; A model training module, used to train a multimodal deep learning model based on the collected data; The trained model is used for the classification of lymph node metastasis and the scoring of the disease-free survival risk to obtain the classification result and the scoring result; The classification results and scoring results are used to predict the risk of disease-free survival.
10. The device according to claim 9, characterized in that include: The model training module includes: The setup unit is used to set up the multi-task learning model TOAD, the graph attention network GAT, the sequence neural network block SNN, and the gated attention network AttnNetGated; An input unit, used for inputting the clinical characteristic data, the pathological image data, the gene expression data and the immune cell data; A fusion unit is used to perform bilinear fusion on the input data to obtain fused features; The conversion unit is used to input the fused features into a linear regression decoder to convert them into prediction results.
Citation Information
Cited By
Disease diagnosis method and system based on multi-mode space-frequency domain adaptive fusion
CN121601221A
Disease Diagnosis Method and System Based on Multimodal Spatial-Frequency Domain Adaptive Fusion
CN121601221B
