Colorectal cancer metastasis risk prediction method and system fusing radiomics and clinical features

Abstract

The invention discloses a colorectal cancer metastasis risk prediction method and system fusing radiomics and clinical characteristics, and the method comprises the steps: S1, collecting image data and clinical risk factor data of a patient, carrying out the preprocessing, and carrying out the feature extraction; s2, performing data expansion, and fusing expanded features and original features; s3, constructing and training a multi-modal large model, and outputting a first prediction result of the colorectal cancer metastasis risk of the patient; s4, layering the patients according to the first prediction result, and performing dynamic prediction; s5, performing interpretable analysis on a dynamic prediction result and generating a report; and S6, verifying the multi-modal large model. According to the method, the aspects of data fusion, prediction performance, dynamic assessment, clinical interpretation and the like are remarkably improved, key problems in the prior art are solved, an efficient, accurate and easy-to-use transfer risk assessment tool is provided for clinicians, and the method has important academic value and wide clinical application prospects.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method and system for predicting colorectal cancer metastasis risk by integrating radiomics and clinical characteristics. Background Art

[0002] Colorectal cancer is the third most common cancer in the world, and its metachronous liver and lung metastasis is one of the important causes of death in patients. According to the 2021 Global Colorectal Cancer Report, there are approximately 1.93 million new cases of colorectal cancer each year worldwide, of which more than 50% of patients will experience metachronous metastasis to the liver or lung after diagnosis, and the 5-year survival rate of these patients is only 10%. Early prediction of metachronous metastasis is of great significance for improving patient survival and optimizing treatment plans. In clinical practice, traditional risk assessment methods mainly rely on two types of data: one is the patient's clinical risk factors (such as age, gender, pathological stage, CEA, CA19-9 and other biochemical indicators), and the other is imaging omics features (such as tumor texture, morphology, blood flow density, etc. in CT / MRI images). Many studies have shown that imaging omics features can reflect the microscopic heterogeneity of tumors and are closely related to the patient's risk of metastasis; while clinical risk factors can provide overall background information on the development of the disease. The combination of these two types of data is considered to have higher predictive value, but there is still a lack of efficient integration methods.

[0003] Current risk prediction methods are mainly divided into single-modality clinical feature models and imaging omics models, both of which have significant limitations. First, models based on clinical features mainly use basic patient information (such as age, gender), pathological stage, and biochemical indicators, and their modeling methods are mostly logistic regression or simple machine learning algorithms (such as support vector machines and random forests). Although this method is easy to implement, it fails to capture the microscopic characteristics of the tumor due to the single data dimension, and has low predictive performance. The AUC value of the model is usually between 0.65-0.75. In addition, such models often use fixed thresholds to classify patients for risk, ignoring individual differences, and it is difficult to provide targeted risk assessments.

[0004] On the other hand, radiomics-based models extract high-dimensional imaging features of tumors through deep learning or feature extraction tools (such as PyRadiomics) to predict patients' risk of metastasis. Although this method can capture the microscopic information of tumors, its prediction results are less versatile and stable due to the lack of integration with clinical features. The performance of radiomics models fluctuates significantly among patients of different stages. For example, the AUC values ​​in stage II and stage III patients are 0.65 and 0.80, respectively. In addition, radiomics models have poor interpretability, and imaging features are difficult to intuitively associate with doctors' clinical experience, further limiting their credibility in clinical applications. In general, existing methods have the following major problems: 1) Insufficient utilization of single-modality information: Failure to combine radiomics features and clinical risk factors leads to insufficient comprehensiveness and accuracy of risk prediction; 2) Insufficient personalization: Lack of dynamic risk assessment based on individual patient differences, which can easily lead to missed diagnosis of high-risk patients and overtreatment of low-risk patients; 3) Unstable prediction performance: Single-modality models perform quite differently in different patient groups, and have poor versatility and stability; 4) Lack of interpretability: Model prediction results are difficult to combine with clinical experience, which limits doctors' trust and acceptance of the model. Summary of the invention

[0005] The purpose of the present invention is to overcome the deficiencies of the prior art and to provide a method and system for predicting the risk of colorectal cancer metastasis by integrating radiomics and clinical characteristics.

[0006] The objective of the present invention is achieved through the following technical solutions: In a first aspect, the present invention provides a method for predicting the risk of colorectal cancer metastasis by integrating radiomics and clinical characteristics, comprising the following steps: S1. Multimodal data preparation and preprocessing: collecting patient image data and clinical risk factor data, preprocessing the collected image data and clinical risk factor data respectively, performing feature extraction on the preprocessed image data and clinical risk factor data, and obtaining a first image feature and a first clinical feature; S2. Perform data expansion to solve the problem of insufficient diversity and imbalanced categories in patient data. The expanded features are fused with the original features to obtain the second imaging features and the second clinical features as input data. S3. Build and train a large multimodal model to output the first prediction result of the patient's colorectal cancer metastasis risk; S4. Based on the first prediction result of the patient's colorectal cancer metastasis risk output by the model, the patient is stratified and dynamically predicted; S5. Perform interpretable analysis on the results of dynamic prediction and generate an explanatory report containing key features; S6. Use indicators to verify the multimodal large model.

[0007] Based on the first aspect, the step S1 specifically includes: collecting the patient's imaging data and clinical risk factor data, wherein the clinical risk factor data includes age, gender, pathological stage, biochemical indicators CEA and CA19-9; the preprocessing of the imaging data includes normalization processing, specifically including grayscale normalization and size standardization, extracting the features of the imaging data through an imaging genomics feature extraction tool, and generating a first imaging feature ,in The omics features in the first image data include texture, shape and density; preprocessing the clinical risk factor data includes encoding the clinical risk factor data, encoding the discrete clinical risk factor data into continuous numerical features, and obtaining the first clinical feature ,in Indicates i The clinical risk factor data are standardized to convert the data of different dimensions into a unified range. ,in After standardization i The features corresponding to the clinical risk factor data are as follows: min(x) is the minimum value of the clinical risk factor data, and max(x) is the maximum value of the clinical risk factor data.

[0008] Based on the first aspect, step S2 specifically includes: generating diversified patient characteristics through a diffusion model , which is used to supplement the number of samples in the first image feature and the first clinical feature, and to fuse the expanded feature with the original feature to obtain the second image feature and second clinical features , as input data, let the size of the input data be N.

[0009] Based on the first aspect, step S3 specifically includes: constructing a multimodal large model, the multimodal large model includes an image modality encoder, a clinical modality encoder, a modality fusion module and a classification layer, the image modality encoder is based on a visual Transformer, and is used to process the second image feature And generate an embedding representation The clinical modality encoder uses a multi-layer perceptron to encode the structured second clinical feature Encoding The modality fusion module integrates the second image feature and the second clinical feature to obtain multimodal features through the self-attention mechanism. ; Multimodal features after fusion Input to the classification layer, output the patient's colorectal cancer metastasis risk probability, and pass the cross entropy loss function Optimize multimodal large models, where is a binary value, indicating whether the i-th patient has colorectal cancer metastasis. The first prediction result of the colorectal cancer metastasis risk probability of the patient represented by the model output.

[0010] Based on the first aspect, step S4 specifically includes: outputting a first prediction result of the patient's colorectal cancer metastasis risk probability through the model , dividing patients into three groups, including low risk, medium risk and high risk: Combined with the follow-up data of the patient, including imaging changes and fluctuations in biochemical indicators, the model is used to update the first prediction result of the patient's colorectal cancer metastasis risk probability output by the model to obtain the second prediction result , and generate time series risk curves ,in For time point, To realize dynamic prediction for risk prediction function based on multimodal time series.

[0011] Based on the first aspect, step S5 specifically includes: Perform interpretability analysis to generate an interpretability report containing key features; the interpretability analysis includes using the gradient attribution formula Calculate the feature contribution weight, where is the gradient attribution calculation factor, , generate corresponding treatment recommendations based on risk stratification results and explanatory reports of key characteristics.

[0012] Based on the first aspect, step S6 specifically includes: in order to verify the performance of the model, an evaluation is performed on an independent test set, and comprehensive verification is performed using indicators. , the indicators include AUC, precision and recall, where TPR is the true positive rate and FPR is the false positive rate.

[0013] In a second aspect, the present invention discloses a colorectal cancer metastasis risk prediction system integrating radiomics and clinical characteristics, which is used in the above-mentioned colorectal cancer metastasis risk prediction method integrating radiomics and clinical characteristics, comprising: A data acquisition module, used to collect patient imaging data and clinical risk factor data; A data processing module, used for preprocessing the collected patient's imaging data and clinical risk factor data; The data expansion module is used to expand the collected data to solve the problems of insufficient diversity and imbalanced categories in patient data; A multimodal large model building module, used to build a multimodal large model; Risk stratification and dynamic prediction module, used for risk stratification and dynamic prediction; The explainability analysis module is used to perform explainability analysis and generate an explanation report containing key features; The model verification module is used to verify the performance of the constructed multimodal large model.

[0014] Based on the second aspect, the multimodal large model includes: Image modality encoder, used to process image features; clinical modality coders, which process clinical features; Modality fusion module, which is used to fuse image features and clinical features through self-attention mechanism to obtain multimodal features; Classification layer: used to process the input multimodal features and output the patient's colorectal cancer metastasis risk probability.

[0015] The beneficial effects of the present invention are: 1) High prediction performance: higher accuracy. The present invention achieves a significant improvement in risk prediction performance through deep fusion of multimodal data. The AUC value on the test set reaches 0.92, and the recall rate is 89%, which is significantly better than the existing single-modal model (the AUC value is only 0.65-0.78). The missed diagnosis rate is significantly reduced. By accurately identifying high-risk patients, the missed diagnosis rate is reduced and the reliability of risk prediction is improved.

[0016] 2) Comprehensively capture metastasis risk characteristics: The present invention makes full use of imaging genomics features (such as tumor texture, shape, density, etc.) and clinical characteristics (such as pathological stage, CEA, CA19-9, etc.), and captures the complementarity of the two types of data through a multimodal model to ensure the comprehensiveness of risk assessment; the complex interactive relationship between modalities is modeled through a self-attention mechanism, and the relationship between images and clinical characteristics is deeply mined, making up for the defect of insufficient information utilization of single-modal models.

[0017] 3) Dynamic risk prediction and precise stratification. The present invention supports the updating of dynamic follow-up data and generates a dynamic change curve of the patient's metastasis risk in combination with time series analysis, helping doctors to understand the changes in the patient's condition in real time. The risk prediction results can accurately stratify patients into low, medium, and high risk groups, providing doctors with a clear stratification basis and guiding personalized treatment and intervention.

[0018] 4) Strong clinical interpretability. The system generates detailed explanatory reports through gradient attribution and attention weight analysis, intuitively displaying the key features of risk prediction and their contribution weights (such as tumor texture features contributing 35%, and CEA levels contributing 28%). These explanatory information not only increases doctors’ trust in the model, but also helps doctors develop personalized diagnosis and treatment plans more efficiently.

[0019] 5) High data utilization efficiency. The diffusion model generates diversified image features, effectively expands the data of minority patients, alleviates the problem of class imbalance, and further improves the robustness and performance of the model. The model is compatible with images and clinical data from different sources and formats, has strong adaptability, and is suitable for different devices and scenarios.

[0020] 6) Broad clinical applicability: The present invention is applicable to multiple clinical scenarios, including risk prediction of metachronous liver and lung metastasis of colorectal cancer, postoperative follow-up management, and personalized treatment plan design; efficient and intelligent risk assessment capabilities help reduce the workload of doctors and improve the utilization efficiency of medical resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of the steps of a method for predicting the risk of colorectal cancer metastasis by integrating radiomics and clinical characteristics according to an embodiment of the present invention; Figure 2 Schematic diagram of the structure of a colorectal cancer metastasis risk prediction system integrating radiomics and clinical characteristics according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0023] The present invention discloses a method for predicting the risk of colorectal cancer metastasis by integrating radiomics and clinical characteristics. In view of the problems of insufficient information utilization, limited prediction performance and poor clinical interpretability of single-modal models in the prior art, the accuracy, stability and clinical applicability of colorectal cancer metastasis risk prediction are significantly improved. A multimodal large model (MLLM) is used to achieve deep fusion of imaging and clinical data, providing patients with accurate personalized risk assessment. The complementarity of multimodal data is used to make up for the problem of insufficient information utilization of single-modal models. At the same time, the powerful modeling ability of multimodal large models is used to capture the complex relationship between imaging and clinical characteristics, thereby improving the prediction performance and stability of results. In addition, in order to solve the problem of clinical acceptability of the model, the present invention will construct an explanatory report system to intuitively display the key predictive factors and decision logic of the model, thereby helping doctors better understand and apply the prediction results of the model. Through the above innovations, the present invention will significantly improve the accuracy and stability of colorectal cancer metastasis risk assessment, provide a reliable basis for personalized treatment plans for patients, and play an important role in improving the trust of clinicians and the breadth of model application. The process flow chart is as follows: Figure 1 As shown, the specific steps include: S1. Multimodal data preparation and preprocessing: collecting patient image data and clinical risk factor data, preprocessing the collected image data and clinical risk factor data respectively, performing feature extraction on the preprocessed image data and clinical risk factor data, and obtaining a first image feature and a first clinical feature; S2. Perform data expansion to solve the problem of insufficient diversity and imbalanced categories in patient data. The expanded features are fused with the original features to obtain the second imaging features and the second clinical features as input data. S3. Build and train a large multimodal model to output the first prediction result of the patient's colorectal cancer metastasis risk; S4. Based on the first prediction result of the patient's colorectal cancer metastasis risk output by the model, the patient is stratified and dynamically predicted; S5. Perform interpretable analysis on the results of dynamic prediction and generate an explanatory report containing key features; S6. Use indicators to verify the multimodal large model.

[0024] Exemplarily, step S1 includes: collecting imaging data and clinical risk factor data of the patient, wherein the clinical risk factor data includes age, gender, pathological stage, biochemical indicators CEA and CA19-9, etc.; preprocessing the imaging data includes normalization processing, such as grayscale normalization and size standardization, extracting features of the imaging data through an imaging genomics feature extraction tool (such as PyRadiomics), and generating a first imaging feature ,in is the omics feature in the first image data, including texture, shape, density, etc.; preprocessing the clinical risk factor data includes encoding the clinical risk factor data, encoding the discrete clinical risk factor data into continuous numerical features, and obtaining the first clinical feature ,in Indicates i The clinical risk factor data are standardized to convert the data of different dimensions into a unified range. ,in After standardization i The features corresponding to the clinical risk factor data are as follows: min(x) is the minimum value of the clinical risk factor data, and max(x) is the maximum value of the clinical risk factor data.

[0025] Exemplarily, step S2 includes: since actual patient data may have problems such as insufficient diversity and imbalanced categories (such as fewer high-risk patients), the present invention generates diversified patient features through a diffusion model , which is used to supplement the problem of insufficient sample quantity in the first image feature and the first clinical feature, and to fuse the expanded feature with the original feature to obtain the second image feature and second clinical features , as input data, let the size of the input data be N.

[0026] Exemplarily, step S3 specifically includes: constructing a multimodal large model, the multimodal large model including an image modality encoder, a clinical modality encoder, a modality fusion module and a classification layer, the image modality encoder is based on a visual Transformer (ViT) for processing the second image feature And generate an embedding representation The clinical modality encoder uses a multi-layer perceptron (MLP) to encode the structured second clinical features. Encoding The modality fusion module integrates the second image feature and the second clinical feature to obtain multimodal features through the self-attention mechanism. ; Multimodal features after fusion Input to the classification layer, output the patient's colorectal cancer metastasis risk probability, and pass the cross entropy loss function Optimize multimodal large models, where Is a binary value, indicating i Whether the patient has colorectal cancer metastasis, The first prediction result of the colorectal cancer metastasis risk probability of the patient represented by the model output.

[0027] Exemplarily, step S4 specifically includes: outputting a first prediction result of the patient's colorectal cancer metastasis risk probability through the model , dividing patients into three groups, including low risk, medium risk and high risk: Combined with the follow-up data of the patient, including imaging changes and fluctuations in biochemical indicators, the model is used to update the first prediction result of the patient's colorectal cancer metastasis risk probability output by the model to obtain the second prediction result , and generate time series risk curves ,in For time point, To realize dynamic prediction for risk prediction function based on multimodal time series.

[0028] Exemplarily, step S5 specifically includes: by analyzing the second prediction result Perform interpretability analysis to generate an interpretability report containing key features; the interpretability analysis includes using the gradient attribution formula Calculate the feature contribution weight, where is the gradient attribution calculation factor, , based on the risk stratification results and the explanatory report of key features, the report content includes: key imaging features that the model focuses on (such as texture heterogeneity or tumor boundary shape); the contribution of important clinical features (such as the impact of CEA or CA19-9); specific explanations of risk stratification results to help doctors understand the basis of model predictions. Based on the risk stratification results and the results of interpretable feature analysis, the system generates corresponding personalized treatment recommendations. For example: for the low-risk group, routine follow-up is recommended, and imaging examinations are performed every 6 months; for the medium-risk group, intensive follow-up is recommended, combined with dynamic imaging monitoring, and adjuvant therapy is considered; for the high-risk group, further molecular diagnosis or targeted therapy is recommended.

[0029] Exemplarily, step S6 specifically includes: in order to verify the performance of the model, an evaluation is performed on an independent test set, and a comprehensive verification is performed using indicators The indicators include AUC, precision and recall, where TPR is the true positive rate and FPR is the false positive rate. The AUC of the model on the test set reached 0.92 and the recall rate was 0.89, which are significantly higher than the existing single-modal model.

[0030] This invention achieves a deep integration of radiomics and clinical characteristics through a multimodal large model, and builds a complete technical solution from data preparation to model training, risk stratification and clinical application. It significantly improves the accuracy of metastasis risk prediction (AUC value reaches 0.92), and provides clinicians with personalized diagnosis and treatment recommendations through dynamic prediction and interpretable analysis, which has broad application value and promotion potential.

[0031] The present invention also discloses a colorectal cancer metastasis risk prediction system integrating radiomics and clinical characteristics, which is used in the above-mentioned colorectal cancer metastasis risk prediction method integrating radiomics and clinical characteristics. Figure 2 As shown, including: A data acquisition module, used to collect patient imaging data and clinical risk factor data; A data processing module, used for preprocessing the collected patient's imaging data and clinical risk factor data; The data expansion module is used to expand the collected data to solve the problems of insufficient diversity and imbalanced categories in patient data; A multimodal large model building module, used to build a multimodal large model; Risk stratification and dynamic prediction module, used for risk stratification and dynamic prediction; The explainability analysis module is used to perform explainability analysis and generate an explanation report containing key features; The model verification module is used to verify the performance of the constructed multimodal large model.

[0032] The multimodal large model includes: Image modality encoder, used to process image features; clinical modality coders, which process clinical features; Modality fusion module, which is used to fuse image features and clinical features through self-attention mechanism to obtain multimodal features; Classification layer: used to process the input multimodal features and output the patient's colorectal cancer metastasis risk probability.

[0033] For example, in clinical applications, a colorectal cancer metastasis risk prediction system that integrates radiomics and clinical characteristics is deployed on the diagnostic platform of medical institutions, providing doctors with risk assessment results, explanatory reports and treatment recommendations through a user-friendly interface, simplifying the decision-making process of doctors. In addition, the system supports the continuous update of follow-up data to ensure the real-time and accuracy of risk assessment.

[0034] Specifically, the core technical difference of the present invention lies in the introduction and application of the Multimodal Large Model (MLLM), which achieves global optimization and personalized stratification of metastasis risk prediction by deeply integrating radiomics features and clinical risk factors. This technical difference not only overcomes the shortcomings of the existing single-modality prediction model, such as insufficient information utilization and limited performance, but also significantly improves the accuracy, stability and clinical interpretability of the prediction.

[0035] Existing technologies mainly rely on a single modality (such as clinical features or imaging genomics features) for risk prediction, which makes it difficult to fully capture the metastasis risk characteristics of colorectal cancer patients. The present invention uses a multimodal large model to uniformly process imaging data and structured clinical data, and constructs an intelligent model architecture with high fusion capabilities and diversified input support. Deep fusion of imaging and clinical features: Through a multimodal large model, the deep integration of imaging genomics features (such as tumor texture, shape, density, etc.) and clinical risk factors (such as age, pathological stage, biochemical indicators, etc.) is realized for the first time, overcoming the problem of information fragmentation of each modality in existing methods. Modeling of interactive relationships between modalities: The self-attention mechanism is used to mine the complex interactive relationship between imaging and clinical features, enhancing the model's ability to process high-dimensional heterogeneous data. Dynamic risk prediction and stratification: Through time series modeling, dynamic updates of follow-up data are supported, realizing dynamic assessment of patient risks, rather than static prediction based on a single time point. The unified modeling of imaging and clinical data is achieved through a multimodal Transformer architecture, including an imaging modality encoder, a clinical modality encoder, and a modality fusion module. The self-attention mechanism is used to model the complex interaction between modalities. A feature fusion method for radiomics features and clinical features is provided, which can significantly improve the prediction performance of metastasis risk. Dynamic risk assessment based on time series is supported to generate a metastasis risk curve that changes over time. Personalized stratification and intervention recommendations are provided based on risk prediction results. Based on gradient attribution and attention weights, a detailed prediction explanation report is generated to clearly show the contribution of key features to the prediction results. Diversified radiomics features are generated using diffusion models to supplement minority class samples in the data set and solve the problem of class imbalance. Personalized follow-up plans and treatment recommendations are generated for patients with different risk stratifications, significantly improving the accuracy and efficiency of clinical treatment.

[0036] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not deviate from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A method for predicting the risk of colorectal cancer metastasis by integrating radiomics and clinical characteristics, characterized in that: The following steps are involved: S1. Multimodal data preparation and preprocessing: collecting patient image data and clinical risk factor data, preprocessing the collected image data and clinical risk factor data respectively, performing feature extraction on the preprocessed image data and clinical risk factor data, and obtaining a first image feature and a first clinical feature; S2. Perform data expansion to solve the problem of insufficient diversity and imbalanced categories in patient data. The expanded features are fused with the original features to obtain the second imaging features and the second clinical features as input data. S3. Build and train a large multimodal model to output the first prediction result of the patient's colorectal cancer metastasis risk; S4. Based on the first prediction result of the patient's colorectal cancer metastasis risk output by the model, the patient is stratified and dynamically predicted; S5. Perform interpretable analysis on the results of dynamic prediction and generate an explanatory report containing key features; S6. Use indicators to verify the multimodal large model.

2. The method for predicting the risk of colorectal cancer metastasis by integrating radiomics and clinical characteristics according to claim 1, characterized in that: The step S1 specifically includes: collecting the patient's imaging data and clinical risk factor data, wherein the clinical risk factor data includes age, gender, pathological stage, and biochemical indicators CEA and CA19-9; the preprocessing of the imaging data includes normalization processing, specifically including grayscale normalization and size standardization, extracting features of the imaging data through an imaging genomics feature extraction tool, and generating a first imaging feature ,in The omics features in the first image data include texture, shape and density; preprocessing the clinical risk factor data includes encoding the clinical risk factor data, encoding the discrete clinical risk factor data into continuous numerical features, and obtaining the first clinical feature ,in Indicates i The clinical risk factor data are standardized to convert the data of different dimensions into a unified range. ,in After standardization i The features corresponding to the clinical risk factor data are as follows: min(x) is the minimum value of the clinical risk factor data, and max(x) is the maximum value of the clinical risk factor data.

3. The method for predicting the risk of colorectal cancer metastasis by integrating radiomics and clinical characteristics according to claim 2, characterized in that: The step S2 specifically includes: generating diversified patient characteristics through a diffusion model , which is used to supplement the number of samples in the first image feature and the first clinical feature, and to fuse the expanded feature with the original feature to obtain the second image feature and second clinical features , as input data, let the size of the input data be N.

4. The method for predicting the risk of colorectal cancer metastasis by integrating radiomics and clinical characteristics according to claim 3, characterized in that: Step S3 specifically includes: constructing a multimodal large model, the multimodal large model includes an image modality encoder, a clinical modality encoder, a modality fusion module and a classification layer, the image modality encoder is based on a visual Transformer, and is used to process the second image feature And generate an embedding representation The clinical modality encoder uses a multi-layer perceptron to encode the structured second clinical feature Encoding The modality fusion module integrates the second image feature and the second clinical feature to obtain multimodal features through the self-attention mechanism. ; Multimodal features after fusion Input to the classification layer, output the patient's colorectal cancer metastasis risk probability, and pass the cross entropy loss function Optimize multimodal large models, where is a binary value, indicating whether the i-th patient has colorectal cancer metastasis. The first prediction result of the colorectal cancer metastasis risk probability of the patient represented by the model output.

5. The method for predicting the risk of colorectal cancer metastasis by integrating radiomics and clinical characteristics according to claim 4, characterized in that: Step S4 specifically includes: outputting the first prediction result of the patient's colorectal cancer metastasis risk probability through the model , dividing patients into three groups, including low risk, medium risk and high risk: ; Combined with the patient's follow-up data, including imaging changes and biochemical index fluctuations, to update the first prediction result of the patient's colorectal cancer metastasis risk probability output by the model to obtain the second prediction result , and generate time series risk curves ,in For time point, To realize dynamic prediction for risk prediction function based on multimodal time series.

6. The method for predicting the risk of colorectal cancer metastasis by integrating radiomics and clinical characteristics according to claim 5, characterized in that: Step S5 specifically includes: Perform interpretability analysis to generate an interpretability report containing key features; the interpretability analysis includes using the gradient attribution formula Calculate the feature contribution weight, where is the gradient attribution calculation factor, , generate corresponding treatment recommendations based on risk stratification results and explanatory reports of key characteristics.

7. The method for predicting the risk of colorectal cancer metastasis by integrating radiomics and clinical characteristics according to claim 6, characterized in that: Step S6 specifically includes: In order to verify the performance of the model, an evaluation is performed on an independent test set, and a comprehensive verification is performed using indicators , the indicators include AUC, precision and recall, where TPR is the true positive rate and FPR is the false positive rate.

8. A colorectal cancer metastasis risk prediction system integrating radiomics and clinical characteristics, used in a colorectal cancer metastasis risk prediction method integrating radiomics and clinical characteristics as claimed in any one of claims 1 to 7, characterized in that: include: A data acquisition module, used to collect patient imaging data and clinical risk factor data; A data processing module, used for preprocessing the collected patient's imaging data and clinical risk factor data; The data expansion module is used to expand the collected data to solve the problems of insufficient diversity and imbalanced categories in patient data; A multimodal large model building module, used to build a multimodal large model; Risk stratification and dynamic prediction module, used for risk stratification and dynamic prediction; The explainability analysis module is used to perform explainability analysis and generate an explanation report containing key features; The model verification module is used to verify the performance of the constructed multimodal large model.

9. The colorectal cancer metastasis risk prediction system integrating radiomics and clinical characteristics according to claim 8, characterized in that: The multimodal large model includes: Image modality encoder, used to process image features; clinical modality coders, which process clinical features; Modality fusion module, which is used to fuse image features and clinical features through self-attention mechanism to obtain multimodal features; Classification layer: used to process the input multimodal features and output the patient's colorectal cancer metastasis risk probability.

Images