Gastric cancer postoperative prognosis prediction method and system based on multi-modal data fusion

Abstract

The invention discloses a gastric cancer postoperative prognosis prediction method and system based on multi-modal data fusion, and belongs to the technical field of medical and industrial combination. According to the technical scheme, the method comprises the steps of obtaining clinical structured data and postoperative pathological text data of a gastric cancer patient; the clinical structural data is preprocessed; performing feature extraction on the postoperative pathological text data by using a pre-trained ERNIE model, and performing dimension reduction processing through a UMAP algorithm; fusing the preprocessed structured data and the dimensionality-reduced text features by using a multi-level adaptive attention mechanism to obtain fused features; and inputting the fusion features into a machine learning model for training and prediction, and outputting a gastric cancer postoperative three-year survival prediction result. The method has the beneficial effects that through an innovative multi-modal data fusion strategy, the precision, stability and interpretability of gastric cancer postoperative survival prediction are effectively improved, and a powerful technical tool and decision basis are provided for improving postoperative management of gastric cancer patients and realizing personalized treatment.

Description

Technical Field

[0001] This invention belongs to the field of medical-engineering interdisciplinary and medical-engineering integration technology, and relates to a method and system for predicting the postoperative prognosis of gastric cancer based on multimodal data fusion. Background Technology

[0002] Gastric cancer is one of the deadliest malignant tumors worldwide, and the prognosis is particularly complex for patients with advanced gastric cancer. In East Asia, the incidence rate of gastric cancer is 321 per 100,000, and the mortality rate is 13.22 per 100,000. [1] Statistics show that advanced-stage gastric cancer accounts for approximately 80% of all cases. These patients often face a higher risk of postoperative complications and death due to factors such as decreased bodily function and malnutrition. According to clinical practice guidelines, even in advanced stages, surgical resection remains the cornerstone of treatment, as it not only removes the primary lesion but also improves the patient's quality of life and long-term survival. However, despite continuous improvements in surgical techniques in recent years, the overall prognosis for gastric cancer patients remains poor. [2-3] Many factors influence patient prognosis, including gender, age, Eastern Collaborative Oncology Group score, tumor location, size, differentiation degree, histological type, TNM stage, and chemotherapy regimen. [4-6] The TNM staging system is widely used in clinical practice, but it mainly predicts prognosis based on anatomical factors (such as tumor invasion depth and lymph node metastasis), and cannot fully reflect the actual survival risk of patients after surgery. [7] Therefore, accurate prediction of 3-year postoperative survival not only helps identify high-risk patients, but also guides clinicians in developing more targeted follow-up plans and adjuvant treatment options.

[0003] In recent years, with the rapid development of artificial intelligence technology, machine learning methods have been widely applied to the early screening and classification prediction of cancer, providing an important means to improve cancer prognosis assessment. ML methods excel at capturing complex, interactive, or nonlinear relationships and can mine potential patterns and key features from massive amounts of data. [8] Through techniques such as feature selection, feature extraction, and dimensionality reduction, machine learning methods significantly improve the quality of data analysis results, especially demonstrating advantages in denoising and removing redundant features on high-dimensional feature datasets. [9] In cancer prognosis prediction, supervised learning techniques (such as Random Forest (RF), Support Vector Machine (SVM), Decision Tree (DT), and Convolutional Neural Network (CNN)) are increasingly used. These techniques can effectively predict patients' survival risk and provide scientific evidence for clinicians. [10-13]Furthermore, the prognostic model integrating multiple machine learning algorithms further enhances the model's predictive performance and generalization ability, laying a technological foundation for the implementation of precision medicine.

[14] .

[0004] While machine learning (ML) methods have shown significant potential in cancer research, their application to gastric cancer remains limited. Most current gastric cancer prognostic models are primarily based on patients' structured clinical data, neglecting potentially crucial information in unstructured textual data such as pathology reports and surgical records, making it difficult to comprehensively and accurately assess patients' survival risk. Furthermore, these models often lack deep fusion of multimodal data, limiting their applicability in complex clinical scenarios.

[0005] With the rapid development of large language models and the widespread application of pre-trained language models such as BERT and ERNIE, natural language processing (NLP) methods have demonstrated enormous potential in the medical field. These models can extract fine-grained semantic information from unstructured text data, providing technical support for optimizing prognostic assessment models. In particular, the ERNIE model, through deep integration of medical knowledge, can capture more complex semantic features, and its application in medical text analysis is gradually gaining attention. Furthermore, the introduction of adaptive attention mechanisms provides a new solution for feature alignment and fusion of multimodal data. These techniques can dynamically allocate weights in multimodal data, effectively improving data complementarity and model performance.

[0006] [1]Bray, F., et al. "Global Cancer Statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. [Internet] 2019."

[0007] [2]Kinoshita, Takahiro, et al. "Long-term outcomes of laparoscopicversus open surgery for clinical stage II / III gastric cancer: a multicentercohort study in Japan (LOC-A Study)." Annals of surgery 269.5(2019):887-894.

[0008] [3]Liu,Fenglin,et al."Morbidity and mortality of laparoscopic vs opentotal gastrectomy for clinical stage I gastric cancer:the CLASS02 multicenterrandomized clinical trial."JAMA oncology 6.10(2020):1590-1597.

[0009] [4]Touchefeu,Y.,et al."Prognostic factors in patients treated withsecond-line chemotherapy for advanced gastric cancer:results from therandomized prospective phase III FFCD-0307 trial."Gastric Cancer 22(2019):577-586.

[0010] [5]Chen,Qi-Yue,et al."Prognosis of young survivors of gastric cancerin China and the US:Determining long-term outcomes based on conditionalsurvival."The oncologist24.6(2019):e260-e274.

[0011] [6]Dixon,Matthew,et al."Prognostic factors in metastatic gastriccancer:results of a population-based,retrospective cohort study in Ontario."Gastric Cancer 19(2016):150-159.

[0012] [7]Cheong,Jae-Ho,et al."Predictive test for chemotherapy response inresectable gastric cancer:a multi-cohort,retrospective analysis."The LancetOncology 19.5(2018):629-638.

[0013] [8]Chekroud,AdamM.,et al."The promise of machine learning inpredicting treatment outcomes in psychiatry."World Psychiatry 20.2(2021):154-170.

[0014] [9]Kourou,Konstantina,et al."Machine learning applications in cancerprognosis and prediction."Computational and structural biotechnology journal13(2015):8-17.

[0015]

[10] Hwangbo,Suhyun,et al."Development of machine learning models topredict platinum sensitivity of high-grade serous ovarian carcinoma."Cancers13.8(2021):1875.

[0016]

[11] Jiang,Yuming,et al."Immunomarker support vector machineclassifier for prediction of gastric cancer survival and adjuvantchemotherapeutic benefit."Clinical Cancer Research 24.22(2018):5574-5584.

[0017]

[12] Brims, Fraser JH, et al. "A novel clinical prediction model for prognosis in malignant pleural mesothelioma using decision tree analysis." Journal of Thoracic Oncology 11.4 (2016): 573-582.

[0018]

[13] Shim, Won Sang, et al. "DeepRePath: identifying the prognostic features of early-stage lung adenocarcinoma using multi-scale pathology images and deep convolutional neural networks." Cancers 13.13(2021):3308.

[0019]

[14] Hopkins, Ashley M., et al. "Development and validation of aprognostic model for patients with advanced lung cancer treated with the immune checkpoint inhibitor atezolizumab." Clinical Cancer Research 26.13(2020):3280-3286. Summary of the Invention

[0020] To address the technical problems existing in the prior art, this invention, building upon previous work, proposes a multimodal data fusion-based prognostic assessment model for gastric cancer, combining the ERNIE model with machine learning algorithms. By integrating patients' structured clinical data and postoperative pathological text information, and utilizing multi-level adaptive attention feature fusion to deeply fuse heterogeneous features, this invention aims to significantly improve the accuracy and reliability of prognostic prediction while overcoming data type heterogeneity and information redundancy. The successful application of this method will provide crucial technical support for precision medicine in the field of gastric cancer, particularly demonstrating broad clinical application potential in postoperative survival assessment and personalized treatment strategy development.

[0021] To achieve the above objectives, the present invention provides the following technical solution:

[0022] A method for predicting postoperative prognosis of gastric cancer based on multimodal data fusion, comprising the following steps:

[0023] Step 1: Obtain structured clinical data and postoperative pathology text data from gastric cancer patients;

[0024] Step 2: Preprocess the clinical structured data, including missing value imputation, data normalization, and category coding;

[0025] Step 3: Use a pre-trained ERNIE model to extract features from the postoperative pathology text data, and then perform dimensionality reduction using the UMAP algorithm;

[0026] Step 4: Use a multi-level adaptive attention mechanism to fuse the preprocessed structured data and the dimensionality-reduced text features to obtain fused features;

[0027] Step 5: Input the fused features into the machine learning model for training and prediction, and output the 3-year survival prediction results after gastric cancer surgery.

[0028] Furthermore, the missing value imputation uses the K-nearest neighbor interpolation algorithm, the numerical features are imputed using the weighted average, and the categorical features are imputed using the majority voting method.

[0029] Furthermore, in the dimensionality reduction process, the UMAP algorithm reduces the text features to 15 dimensions.

[0030] Furthermore, the multi-level adaptive attention mechanism includes feature-level attention, modality-level attention, and cross-modality attention, which are used to dynamically and weightedly fuse features from different modalities.

[0031] Furthermore, the machine learning model is any one of random forest, logistic regression, extreme gradient boosting, lightweight gradient boosting machine, or stochastic gradient descent.

[0032] Furthermore, the machine learning model uses random search for hyperparameter optimization and is evaluated using k-fold cross-validation.

[0033] Furthermore, the structured clinical data includes at least one of the following: age, red blood cell count, hemoglobin, albumin, CEA, CA199, intraoperative blood loss, maximum tumor diameter, alcohol intake, extent of resection, reconstruction method, lymphovascular invasion, perineural invasion, and TNM staging.

[0034] This invention also includes a postoperative prognostic prediction system for gastric cancer based on multimodal data fusion, comprising:

[0035] The data acquisition module is used to acquire clinical structured data and postoperative pathology text data of gastric cancer patients;

[0036] The preprocessing module is used to impute missing values, normalize, and encode the clinical structured data;

[0037] The text processing module is used to extract text features using the ERNIE model and...

[0038] UMAP is used for dimensionality reduction;

[0039] The feature fusion module is used to fuse structured and textual features through a multi-level adaptive attention mechanism;

[0040] The prediction module is used to input fused features into the machine learning model for training and prediction, and output the prognostic results.

[0041] Furthermore, the feature fusion module includes a learnable weight matrix and a cross-modal mapping matrix for implementing feature-level and modality-level attention weighting.

[0042] The present invention also includes an electronic device comprising a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the above-described method.

[0043] The present invention also includes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0044] The beneficial effects of this invention are:

[0045] Compared with existing technologies, the method and system for predicting postoperative prognosis of gastric cancer based on multimodal data fusion described in this invention have the following technical features and beneficial effects:

[0046] (1) Significantly improved the accuracy and reliability of prognostic prediction: By deeply fusing structured clinical data with unstructured postoperative pathological text data, the limitations of incomplete information in traditional single-modal models were overcome. Experimental results show that multimodal fusion features achieved the best performance on all five machine learning models (LR, XGBoost, RF, SGD, LightGBM). All indicators were significantly better than models using only clinical features or only text features.

[0047] (2) Key information in unstructured text is fully explored and utilized: By using a pre-trained ERNIE model to fine-tune and extract features from postoperative pathology reports, the complex semantic information hidden in the text can be effectively captured, thus providing richer and more detailed feature representation for prognostic assessment.

[0048] (3) Optimized feature representation and improved computational efficiency: The UMAP algorithm was introduced to reduce the dimensionality of high-dimensional text features. Under the optimal 15-dimensional configuration, the best model classification performance was achieved while maintaining high information retention.

[0049] (4) Intelligent fusion of heterogeneous data and focus on key information: The designed multi-level adaptive attention mechanism can dynamically assign weights to different features and modalities and automatically focus on the information most relevant to survival risk, thereby achieving effective alignment and deep fusion of structured data and text features.

[0050] (5) It has broad model applicability and good robustness: The proposed multimodal fusion framework is decoupled from specific machine learning algorithms. Experiments have verified that it can be effectively combined with a variety of classic machine learning algorithms, and all of them can bring performance improvement. The stability and reliability of the model performance are confirmed by 5-fold cross-validation.

[0051] (6) It provides interpretable decision-making basis for clinical precision medicine: Through univariate analysis and LASSO regression, statistically significant prognostic features were screened out. The identification of these features not only verified their predictive value, but also provided clinicians with clear and interpretable directions and goals for identifying high-risk patients, developing personalized follow-up plans and adjuvant treatment plans.

[0052] This invention effectively improves the accuracy, stability, and interpretability of postoperative survival prediction for gastric cancer through an innovative multimodal data fusion strategy, providing a powerful technical tool and decision-making basis for improving postoperative management of gastric cancer patients and achieving personalized treatment. Attached Figure Description

[0053] To more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and detailed embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] in:

[0055] Figure 1 This is a case screening diagram for the present invention;

[0056] Figure 2 This is a graph showing the selection of predictor variables for the LASSO regression model of this invention.

[0057] Figure 3 This invention provides postoperative pathological text feature extraction and UMAP reduction based on ERNIE.

[0058] Vito;

[0059] Figure 4 This is a dimensional analysis diagram of the text data in this invention;

[0060] Figure 5 For the test subjects of the five machine learning models under the three modes of this invention

[0061] Comparison of characteristic curves (left: clinical features only; middle: postoperative pathology text features only)

[0062] (The right side shows mixed features);

[0063] Figure 6 This is a graph showing the 5x cross-validation results of this invention. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. The following description, in conjunction with the accompanying drawings... Figure 1-6 This paper further explains the method and system for predicting postoperative prognosis of gastric cancer based on multimodal data fusion.

[0065] Example 1

[0066] This embodiment provides a postoperative prognostic model for gastric cancer based on RENIE and machine learning algorithms. It includes the following steps: S1. Data preprocessing; S2. Case screening; S3. Structured data processing methods; S4. Text data processing methods; S5. Machine learning model for predicting postoperative survival in advanced gastric cancer.

[0067] S1. For the preprocessing of structured data, this invention employs a systematic approach to process both numerical and categorical data to optimize model performance. The specific preprocessing process includes three key steps: missing value imputation, data normalization, and class encoding.

[0068] Based on the distribution characteristics of missing values, the K-nearest neighbor interpolation algorithm is used to handle missing values. By calculating the similarity between samples, the K nearest neighbors are selected for imputation. For numerical features, a weighted average is used for imputation to preserve the data distribution characteristics. For categorical features, a majority voting method is used, imputing the category with the highest frequency among the neighboring samples.

[0069] To eliminate the influence of dimensional differences between features, numerical data is normalized, mapping feature values ​​to the 0,1 interval. For categorical data, label encoding is used to convert discrete categorical values ​​into continuous integer labels, making them suitable for numerical computation in machine learning models.

[0070] S2. This invention includes 526 patients with primary gastric adenocarcinoma who underwent laparoscopic radical gastrectomy at the Affiliated Zhongshan Hospital of Dalian University between December 2011 and December 2018. To ensure the scientific rigor and reliability of the results, the following strict exclusion criteria were established during the patient sample screening process:

[0071] (1) Early gastric cancer cases;

[0072] (2) Cases where peritoneal dissemination or distant metastasis is confirmed before or during surgery;

[0073] (3) Cases of gastric stump cancer;

[0074] (4) Patients who lack abdominal CT images before surgery or whose images are more than 1 month apart from the surgery;

[0075] (5) Cases with incomplete clinicopathological data. Ultimately, a total of 447 patients met the inclusion criteria and were included in this invention for further analysis. The inclusion and exclusion criteria are as follows: Figure 1 As shown.

[0076] S3. Effective feature selection can significantly improve model performance when predicting postoperative prognosis in advanced gastric cancer. Univariate analysis identified 16 statistically significant prognostic features (p<0.1), including age, red blood cell count, hemoglobin, albumin, CEA, CA199, intraoperative blood loss, maximum intraoperative tumor diameter, alcohol intake, CEA, extent of resection, reconstruction method, complications, lymphovascular invasion, perineural invasion, and TNM stage. To eliminate multicollinearity among features and ensure model stability, LASSO regression analysis was performed on these features, such as... Figure 2 As shown in the figure. The analysis results indicate that, after removing redundant feature complications, 15 corresponding non-zero coefficient predictors were selected.

[0077] S4. Postoperative pathology reports contain a wealth of important information related to the prognosis of patients with advanced gastric cancer, such as tumor tissue type, depth of invasion, and lymph node metastasis. To efficiently extract this potential information and generate feature representations suitable for machine learning modeling, this paper combines a pre-trained and optimized ERNIE model with UMAP to process unstructured text data. ERNIE is fine-tuned to enhance its feature representation capabilities in postoperative pathology text analysis of advanced gastric cancer. Fine-tuning is performed based on a training set constructed from patients' postoperative pathology reports to better adapt the model to specific medical terminology and context. The Adam optimizer is used to optimize the model, and a weight decay strategy is introduced. The cross-entropy loss function is used as the optimization objective. Figure 3The framework of the ERNIE model used in this invention is shown. The model was trained using the AdamW optimizer with a learning rate of 4e-5. The batch size was set to 128, and the model was trained for 10 epochs. During training, the cross-entropy loss function was used to measure the difference between the predicted and the true labels. To improve operational efficiency and remove redundant text features, this invention employs the UMAP algorithm to explore the optimal dimensionality reduction scheme, and systematically evaluates the classification performance (AUC score) and information retention (confidence index), such as... Figure 4 As shown in Table 1, experimental results indicate that reducing the feature space to 15 dimensions yields the best overall performance. In terms of classification performance, the AUC score for the 15-dimensional representation is 0.894 (±0.017), significantly better than other dimensions. It is worth noting that while information retention (confidence) shows locally high values ​​at very low dimensions, these configurations do not perform well in classification. However, adding more dimensions does not provide a significant performance improvement; instead, it increases computational overhead. For example, when the dimension increases to 200, the runtime increases significantly from 1.60 seconds to 48.02 seconds, but the AUC score is only 0.885, even slightly lower than that of the 15-dimensional configuration. Although information retention is improved in 10 dimensions, the AUC performance is significantly lower than that of the 15-dimensional configuration. Regarding information preservation, the confidence score for the 15-dimensional configuration is 0.972, indicating that the dimensionality reduction process preserves the local structural features of the data well. Although a slightly higher score of 0.973 is obtained in the 200-dimensional configuration, this small gain (0.001) is insufficient to justify the additional computational overhead. Specific data are shown in Table 1.

[0078] Table 1 Data Dimension Analysis

[0079] Dimension AUC Credibility Index time 2 0.856 0.969 0.984 3 0.856 0.972 1.057 5 0.874 0.972 1.147 10 0.875 0.973 1.418 15 0.894 0.972 1.595 20 0.875 0.972 1.870 30 0.871 0.972 2.291 40 0.873 0.973 1.801 50 0.886 0.973 2.269 100 0.872 0.973 4.575 200 0.885 0.973 48.015

[0080] To comprehensively characterize the postoperative features of patients with advanced gastric cancer, this invention proposes a feature fusion framework based on multi-level adaptive attention. This framework deeply integrates clinical structured data with postoperative pathological text features through feature-level and modality-level attention mechanisms, achieving effective integration of heterogeneous data. The implementation process is as follows:

[0081]

[0082]

[0083] Where W1 and W2 are learnable weight matrices, A s and A t These represent the attention weights for structured features and textual features, respectively.

[0084] To capture the relationships between different modalities, a cross-modal attention mechanism was designed:

[0085] M = tanh(W) m [X s ;X t (3)

[0086] Among them, W m It is a cross-modal mapping matrix, [;] represents feature concatenation, and M is the cross-modal feature representation.

[0087] Feature fusion based on attention weights:

[0088] F = σ(W) f [A s eX s A t eX t ;M]) (4)

[0089] Where e is the Hadamard product (element-by-element product), W f σ is the weight matrix of the fusion layer, σ is the non-linear activation function ReLU, and F is the final fusion feature.

[0090] S5. This invention employs five machine learning algorithms for postoperative survival prediction in advanced gastric cancer. These algorithms are all implemented in Python, including tree-based models, regression models, and ensemble learning methods.

[0091] (1) Random Forest (RF): It is an ensemble algorithm based on decision trees. It uses bootstrap sampling to randomly sample training data and builds multiple decision tree classifiers. Its prediction results are determined by voting among the decision trees.

[0092] (2) Logistic Regression (LR): It uses the sigmoid function to convert linear predictions into probability outputs. It is a classic binary classification algorithm with good interpretability.

[0093] (3) Extreme Gradient Boosting (XGBoost): As an efficient ensemble learning algorithm, it introduces a regularization term to prevent overfitting. Its objective function consists of two parts: a loss function and a regularization term. This algorithm has significant advantages in training speed and prediction accuracy.

[0094] (4) Lightweight Gradient Boosting Machine (LightGBM): Also based on the gradient boosting framework, but using a histogram-based decision tree construction method, it significantly reduces memory consumption. Through leaf growth strategy and efficient feature parallel algorithm, it greatly improves training efficiency while maintaining prediction accuracy.

[0095] (5) Stochastic Gradient Descent (SGD): SGD is a method that optimizes model parameters by calculating the gradient of random samples. Compared to traditional gradient descent methods, it has lower computational complexity and faster convergence speed, making it particularly suitable for online learning on large-scale datasets.

[0096] Various machine learning models were built using Python 3.12: logistic regression, XGBoost, random forest, LightGBM, and SGD. Model hyperparameters were optimized using random search, and model stability was evaluated using k-fold cross-validation. The dataset was randomly split into training and test sets in a 7:3 ratio. Model performance was evaluated using metrics such as the area under the receiver operating characteristic (ROC) curve (AUC), precision, recall, F1 score, and accuracy.

[0097] Example 2

[0098] This embodiment evaluated the predictive performance of unimodal clinical features, unimodal text features, and multimodal fusion features in five classic machine learning models through three sets of controlled experiments. The results are shown in Table 2.

[0099] Table 2 Comparison of prediction performance of five machine learning models in three modalities

[0100]

[0101] In the single-modal clinical feature experiments, model performance was limited, indicating that a single clinical feature cannot adequately capture prognostic risk characteristics. In the text feature experiments, the performance of each model significantly improved, validating the positive impact of semantic information from text features on predictive performance. In the multimodal feature fusion experiments, model performance was further optimized, with SGD showing the best overall performance. Figure 5 The ROC curve analysis shows that as the feature patterns become richer, the curve gradually shifts towards the upper left corner, and the AUC value continues to increase. Each model's ROC curve demonstrates excellent discriminative ability under multimodal fusion features. Table 3 shows the optimal hyperparameters for the five machine learning models under multimodal features.

[0102] Table 3 Hyperparameters of five optimal machine learning models under mixed features

[0103]

[0104]

[0105] To ensure the reliability of the optimal model, 5-fold cross-validation was used. Figure 6The results of five cross-validations are shown, demonstrating that in the case of multimodal data, SGD outperforms other models in accuracy, precision, recall, and F1 score, with a smaller standard deviation. Furthermore, to verify the effectiveness of its multi-level adaptive feature fusion, a comparative experiment was conducted using the best model SGD with fixed-weight fusion (0.5, 0.5) and direct concatenation under five-fold cross-validation to prove its effectiveness. The experimental results are shown in Table 4.

[0106] Table 4 Performance Comparison of Different Feature Fusion Methods

[0107] Fusion Method accuracy F1 value AUC Feature direct splicing 0.828 0.760 0.906 Fixed weight fusion 0.832 0.755 0.913 The method of the present invention 0.844 0.788 0.922

[0108] This invention proposes a machine learning model based on multimodal feature fusion, which integrates clinical structured data and postoperative pathological text data to accurately predict the 3-year survival rate of patients with advanced gastric cancer. The deep fusion of multimodal data significantly improves the model's predictive performance and emphasizes the importance of postoperative pathological text information in disease prognosis assessment, consistent with the current trend of using multimodal data for medical decision-making.

[0109] Compared to traditional unimodal prognostic models, this invention overcomes the problem of incomplete information from a single data source. The ERNIE language model effectively captures the complex semantic information implicit in the text, while UMAP dimensionality reduction avoids redundancy in text features, improving computational efficiency and model stability. The adaptive attention mechanism further enhances the interaction between structured data and text features, enabling the model to focus on information most relevant to survival risk.

[0110] In this invention, univariate analysis and LASSO regression analysis were used to identify risk factors affecting the survival of patients with advanced gastric cancer, including red blood cell count, hemoglobin, carcinoembryonic antigen (CEA), reconstructive methods, TNM stage, and lymphovascular invasion. The identification of these features validated their predictive value and provided clear direction for interventions in clinical practice.

[0111] This invention proposes a multimodal machine learning model that provides a novel approach to predicting postoperative survival in advanced gastric cancer. By integrating structured and unstructured data, the model achieves breakthroughs in predicting complex diseases and provides a reliable basis for developing personalized treatment plans. In the future, by expanding the sample size, introducing multi-center data, and optimizing text feature extraction methods, the model's performance and applicability can be further improved, providing stronger support for precision medicine and prognostic management of advanced gastric cancer. Simultaneously, developing a clinical decision support system to transform the model's predictions into actionable clinical recommendations is of great significance for advancing precision medicine practices.

[0112] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A gastric cancer postoperative prognosis prediction method based on multi-modal data fusion, characterized in that, The steps are as follows: Step one: obtaining clinical structured data and postoperative pathological text data of gastric cancer patients; Step two: preprocessing the clinical structured data, including missing value filling, data normalization and category encoding; Step three: using a pre-trained ERNIE model to extract features from the postoperative pathological text data, and performing dimensionality reduction processing by UMAP algorithm; Step four: using a multi-level adaptive attention mechanism to fuse the preprocessed structured data and the dimensionally reduced text features to obtain fusion features; Step five, input the fusion features into a machine learning model for training and prediction, and output the 3-year survival prediction result of gastric cancer after operation. 2.The gastric cancer postoperative prognosis prediction method based on multi-modal data fusion according to claim 1, characterized in that, The missing value filling adopts K nearest neighbor interpolation algorithm, the numerical features are filled with weighted average value, and the classification features are filled with majority voting method. 3.The gastric cancer postoperative prognosis prediction method based on multi-modal data fusion according to claim 1, characterized in that, In the dimensionality reduction processing, the UMAP algorithm reduces the text features to 15 dimensions. 4.The gastric cancer postoperative prognosis prediction method based on multi-modal data fusion according to claim 1, characterized in that, The multi-level adaptive attention mechanism includes feature-level attention, modal-level attention and cross-modal attention, which is used for dynamically weighted fusion of features of different modalities. 5.The gastric cancer postoperative prognosis prediction method based on multi-modal data fusion according to claim 1, characterized in that, The machine learning model is any one of random forest, logistic regression, extreme gradient boosting, light gradient boosting machine or stochastic gradient descent. 6.The gastric cancer postoperative prognosis prediction method based on multi-modal data fusion according to claim 5, characterized in that, The machine learning model uses random search for hyperparameter optimization, and is evaluated by k-fold cross validation. 7.The gastric cancer postoperative prognosis prediction method based on multi-modal data fusion according to claim 1, characterized in that, The clinical structured data includes at least one of age, red blood cell count, hemoglobin, albumin, CEA, CA199, intraoperative blood loss, maximum tumor diameter, alcohol intake, resection degree, reconstruction method, lymphatic vessel invasion, perineural invasion and TNM stage. 8.A gastric cancer postoperative prognosis prediction system based on multi-modal data fusion, characterized in that, It comprises: a data acquisition module for acquiring clinical structured data and postoperative pathological text data of gastric cancer patients; a preprocessing module for filling missing values, normalizing and encoding the clinical structured data; a text processing module for extracting text features using ERNIE model and performing dimensionality reduction by UMAP; a feature fusion module for fusing structured and text features by a multi-level adaptive attention mechanism; a prediction module for inputting fusion features into a machine learning model for training and prediction, and outputting prognosis results; The feature fusion module includes a learnable weight matrix and a cross-modal mapping matrix, which is used to realize attention weighting at feature level and modal level. 9.An electronic device comprising a memory and a processor, the memory storing a computer program, wherein, The processor executes the computer program to realize the method of any one of claims 1-6.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to realize the method of any one of claims 1 to 6.

Images