A multi-modal fusion based system and method for predicting the risk of postoperative recurrence of renal tumors

Abstract

The application belongs to the technical field of medical information processing, and relates to a kidney tumor postoperative recurrence risk prediction system and method based on multi-modal fusion, which comprises the following steps: acquiring multi-modal data of a patient at multiple time points after operation, generating fusion correlation features reflecting the cooperative relationship between different modal data, obtaining kidney-specific physiological parameters quantifying the physiological and pathological state of the kidney, and a recurrence risk index; constructing the fusion correlation features and the recurrence risk index into a fusion feature time sequence and a recurrence risk index time sequence changing with time, and performing cooperative analysis on the fusion feature time sequence and the recurrence risk index time sequence to generate a recurrence probability value of the kidney tumor after operation; the application solves the problems that the fine characterization of the physiological and pathological dynamic evolution process of the kidney, a specific organ, after operation is lacked, the pertinence of the model is not strong, and the timeliness and accuracy of the prediction are limited.

Description

Technical Field

[0001] This invention belongs to the technical field of medical information processing and relates to a system and method for predicting the risk of recurrence after renal tumor surgery based on multimodal fusion. Background Technology

[0002] Kidney tumors are among the most common malignant tumors of the urinary system. Even after surgical resection, patients still face a certain risk of recurrence. Predicting postoperative recurrence risk is crucial for developing individualized follow-up monitoring plans and adjuvant therapy strategies. In recent years, with the widespread application of information technology in the medical field, utilizing computer-aided systems to process massive amounts of medical data to build intelligent disease risk prediction models has become a research hotspot. These systems provide quantitative support for clinical decision-making by integrating and analyzing patients' clinicopathological information, medical imaging data, and other data.

[0003] In the field of predicting the risk of recurrence after kidney tumor surgery, existing computer-aided systems typically rely on postoperative clinicopathological parameters, such as tumor size, stage, and grade, and combine them with static imaging features for modeling. Some systems attempt to introduce data from different sources, such as genomic data or blood biomarkers, and fuse these multimodal data through simple weighted summation or feature splicing to improve the accuracy of prediction. Most of these systems use machine learning algorithms, such as support vector machines or conventional neural networks, to classify the fused features and output a high-risk or low-risk judgment.

[0004] However, most existing models use general oncology indicators as predictive parameters, lacking a detailed characterization of the dynamic physiological and pathological evolution of the kidney as a specific organ after surgery, resulting in weak model specificity. Furthermore, the vast majority of models perform static assessments based on data from a single time point, failing to effectively utilize the dynamic changes in various indicators during the patient's postoperative recovery period, thus limiting the timeliness and accuracy of predictions. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a system and method for predicting the risk of recurrence after kidney tumor surgery based on multimodal fusion.

[0006] A multimodal fusion-based system for predicting the risk of recurrent kidney tumors after surgery includes:

[0007] The data acquisition module is used to acquire multimodal data of patients at multiple time points after surgery. The multimodal data includes imaging data, genomic data, liquid biopsy data, and dynamic monitoring data of renal function.

[0008] The fusion association feature extraction module constructs a heterogeneous data graph from multimodal data and uses a graph attention network to process the heterogeneous data graph to generate fusion association features that reflect the collaborative relationships between different modalities of data.

[0009] The kidney-specific parameter calculation module, based on imaging data, genomic data, liquid biopsy data, and dynamic monitoring data of kidney function, obtains kidney-specific physiological parameters that quantify the physiological and pathological state of the kidney.

[0010] The recurrence risk index calculation module is used to perform a weighted geometric mean calculation on kidney-specific physiological parameters to obtain the recurrence risk index.

[0011] The fusion prediction module constructs fusion-related features and recurrence risk index into time series of fusion features and recurrence risk index that change over time. It then performs a collaborative analysis of the fusion feature time series and recurrence risk index time series to generate the recurrence probability value after kidney tumor surgery.

[0012] A further aspect of this invention involves constructing a heterogeneous data graph from multimodal data, comprising the following steps:

[0013] Acquire imaging features, genomic data, liquid biopsy data, and dynamic monitoring data of renal function from multimodal data;

[0014] Imaging features, genomic data, liquid biopsy data, and dynamic monitoring data of renal function are defined as different types of nodes;

[0015] Based on the co-occurrence relationships of nodes in clinical cases or the biological association rules pre-set according to existing biomedical knowledge, connection edges are established between different types of nodes to form a heterogeneous data graph.

[0016] A further aspect of this invention utilizes graph attention networks to process heterogeneous data graphs, including the following steps:

[0017] For each node in the heterogeneous data graph, the association weight between it and its neighboring nodes is dynamically calculated through an attention mechanism;

[0018] Aggregate information from adjacent nodes based on association weights to generate enhanced node representations;

[0019] When two or more specific data features appear simultaneously, the enhanced node representation is output as a fused correlation feature.

[0020] A further aspect of the present invention provides kidney-specific physiological parameters for quantifying the physiological and pathological state of the kidneys, comprising the following steps:

[0021] Based on imaging data, the ratio of tumor volume to ipsilateral renal cortex volume is calculated to obtain the tumor-renal cortex volume ratio.

[0022] The postoperative renal function recovery slope was obtained based on dynamic monitoring data of renal function.

[0023] The imaging data were analyzed to obtain the immune response index predicted by radiomics.

[0024] Based on the tumor-to-renal cortex volume ratio, the postoperative renal function recovery slope, and the immune response index predicted by radiomics, kidney-specific physiological parameters were obtained.

[0025] A further aspect of the present invention, obtaining the postoperative renal function recovery slope, includes the following steps:

[0026] Estimated glomerular filtration rate values ​​at multiple time points in the dynamic monitoring data of renal function are obtained to form glomerular filtration rate time series data;

[0027] Curve fitting was performed on the time series data of glomerular filtration rate, and the slope characteristics of the fitted curve were extracted.

[0028] Using the slope characteristic as the slope of postoperative renal function recovery, the dynamic trend of renal function recovery or deterioration over time was obtained.

[0029] A further aspect of the present invention involves performing a weighted geometric mean calculation on kidney-specific physiological parameters, comprising the following steps:

[0030] A score was calculated for each kidney-specific physiological parameter to obtain the parameter score.

[0031] The relapse risk index is obtained by taking a weighted geometric mean of the parameter scores.

[0032] A further embodiment of the present invention includes a fusion prediction module comprising a long short-term memory network unit, which generates a recurrence probability value after kidney tumor surgery, comprising the following steps:

[0033] At each time point, the fusion association features and the relapse risk index are used simultaneously as inputs to the long short-term memory network units;

[0034] By utilizing the memory gating mechanism of long short-term memory network units, the evolution trajectory of the fusion feature time series and the evolution trajectory of the relapse risk index time series were learned and memorized respectively.

[0035] By integrating the analysis results of the two evolutionary trajectories, a relapse probability value is generated.

[0036] A further aspect of this invention, which integrates the analysis results of two evolutionary trajectories to generate a recurrence probability value, also includes the following steps:

[0037] The hidden state of the long short-term memory network unit at the end time step of the fusion feature time series and the recurrence risk index time series is input into the fully connected layer and processed by the activation function to output the recurrence probability value after kidney tumor surgery.

[0038] In a further embodiment of the present invention, after the data acquisition module acquires multimodal data, a preprocessing module performs preprocessing operations on the multimodal data to obtain standardized multimodal data, which is then used as input to the fusion correlation feature extraction module and the kidney-specific parameter calculation module.

[0039] The preprocessing operations include cleaning, aligning, and normalizing data from different sources to generate standardized multimodal data.

[0040] A further aspect of the present invention is a method for predicting the risk of recurrence after renal tumor surgery based on multimodal fusion, comprising the following steps:

[0041] S1. Acquire multimodal data of the patient at multiple time points after surgery. The multimodal data includes at least imaging data, liquid biopsy data, and dynamic monitoring data of renal function. Input the multimodal data in parallel into the first processing channel and the second processing channel.

[0042] S2. In the first processing channel, the multimodal data is constructed into a heterogeneous data graph, and the graph attention network is used to process the heterogeneous data graph to generate fusion association features that reflect the collaborative relationship between different modal data.

[0043] S3. In the second processing channel, based on imaging data, liquid biopsy data and dynamic monitoring data of renal function, kidney-specific physiological parameters that quantify the physiological and pathological state of the kidney are obtained.

[0044] S4. The recurrence risk index is obtained by performing a weighted geometric mean calculation on the kidney-specific physiological parameters.

[0045] S5. Construct the fusion correlation features and recurrence risk index into time series of fusion features and time series of recurrence risk index that change over time;

[0046] S6. Perform synergistic analysis on the fusion feature time series and the recurrence risk index time series to generate the recurrence probability value after kidney tumor surgery.

[0047] In summary, the present invention has the following beneficial technical effects:

[0048] 1. Through a dual-channel parallel processing framework, the system not only explores the nonlinear synergistic relationships between different modalities of data such as images, genes, and biomarkers, but also introduces specific parameters that reflect the physiological state of the kidney organ, thereby improving the accuracy of prediction results.

[0049] 2. By using long short-term memory networks to model data sequences at multiple time points, the system can capture and present the evolution trajectory of patients' postoperative recovery period, rather than simply providing isolated risk scores. At the same time, it introduces kidney-specific parameters with clear clinical significance, making the biological driving factors behind the prediction results clear and providing doctors with more forward-looking and targeted decision-making basis.

[0050] 3. It can learn and utilize the dynamic regulatory effect of macroscopic physiological state on microscopic data anomalies. In the early stage of relapse, when various indicators show only subtle but coordinated abnormal changes, it issues an early warning and constructs a dynamic fusion analysis system that can reflect the synergistic effect. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. The drawings are used to provide a further understanding of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This discloses a first frame schematic diagram of an embodiment of this application.

[0053] Figure 2 This discloses a second frame schematic diagram in an embodiment of this application.

[0054] Figure 3 This discloses a flowchart of an embodiment of this application. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] The following is in conjunction with the appendix Figure 1 - Figure 3 A preferred description of the present invention is provided below.

[0057] See attached document Figure 1 - Figure 3 This invention proposes a multimodal fusion-based system for predicting the risk of recurrence after kidney tumor surgery, comprising the following modules:

[0058] The data acquisition module is used to acquire multimodal data of patients at multiple time points after surgery. The multimodal data includes imaging data, genomic data, liquid biopsy data, and dynamic monitoring data of renal function.

[0059] The fusion association feature extraction module constructs a heterogeneous data graph from multimodal data and uses a graph attention network to process the heterogeneous data graph to generate fusion association features that reflect the collaborative relationships between different modalities of data.

[0060] The kidney-specific parameter calculation module, based on imaging data, liquid biopsy data, and dynamic monitoring data of kidney function, obtains kidney-specific physiological parameters that quantify the physiological and pathological state of the kidney.

[0061] The recurrence risk index calculation module is used to perform a weighted geometric mean calculation on kidney-specific physiological parameters to obtain the recurrence risk index.

[0062] The fusion prediction module constructs fusion-related features and recurrence risk index into time series of fusion features and recurrence risk index that change over time. It then performs a collaborative analysis of the fusion feature time series and recurrence risk index time series to generate the recurrence probability value after kidney tumor surgery.

[0063] In one embodiment of the present invention, constructing multimodal data into a heterogeneous data graph includes the following steps:

[0064] Acquire imaging features, genomic data, liquid biopsy data, and dynamic monitoring data of renal function from multimodal data;

[0065] Imaging features, genomic data, liquid biopsy data, and dynamic monitoring data of renal function are defined as different types of nodes;

[0066] Based on the co-occurrence relationships of nodes in clinical cases or the biological association rules pre-set according to existing biomedical knowledge, connection edges are established between different types of nodes to form a heterogeneous data graph.

[0067] Specifically, this step aims to transform fragmented multimodal patient data from different medical fields into a structured, heterogeneous data graph containing potentially interconnected information. First, the multimodal data needs to be characterized to obtain entities that can serve as graph nodes.

[0068] For image data, radiomics analysis is performed on the segmented tumor regions in CT or MRI images to extract quantitative image features, such as gray-level co-occurrence matrix texture features describing the heterogeneity of the tumor interior, and shape features such as sphericity or compactness describing the tumor morphology. Each specific image feature is regarded as a node of an image feature type.

[0069] For genomic data, gene sequencing results from tumor tissues are analyzed to identify key gene mutation sites or specific gene expression levels, such as VHL gene mutations or PBRM1 gene deletions. Each specific gene event is defined as a node of a genomic data type. For example, the VHL gene mutation status (yes / no) is defined as a node of a genomic data type. These nodes, along with other data nodes, are connected by edges based on the aforementioned co-occurrence relationships or biological association rules to form a heterogeneous data graph.

[0070] For liquid biopsy data, the concentration of circulating tumor DNA (ctDNA) in blood samples is quantified by analyzing blood samples, such as through next-generation sequencing technology (NGS) or digital PCR, or the level of specific protein biomarkers is quantified by immunoassay methods such as ELISA. Each monitored biomarker is defined as a node in a liquid biopsy data type.

[0071] For dynamic monitoring data of renal function, blood samples are collected and indicators reflecting renal function, such as estimated glomerular filtration rate (eGFR), are calculated. The eGFR values ​​monitored at different time points after surgery constitute dynamic monitoring data of renal function. After obtaining imaging feature nodes, genomic data nodes, liquid biopsy data, and dynamic monitoring data nodes of renal function, it is necessary to establish connection edges between these different types of nodes to form a heterogeneous data graph.

[0072] The establishment of connecting edges is based on two complementary mechanisms:

[0073] The first approach is data-driven co-occurrence analysis, which calculates the co-occurrence probability between different node characteristics through statistical analysis of a large-scale historical clinical case database. For example, Point Mutual Information (PMI) can be used to quantify the association strength between any two nodes x and y, and its calculation is as follows:

[0074]

[0075] in, Node features and The probability of co-occurrence within the same patient population and These are the marginal probabilities of their respective occurrences, estimated through frequency statistics on a historical database. For example, frequency statistics are estimated based on a historical database containing anonymized data from at least 1000 post-renal tumor patients, encompassing imaging, genomics, liquid biopsy data, and dynamic renal function monitoring data. The joint probability is estimated similarly. When the value exceeds the preset association strength threshold, then at the node and A connection edge is established between them. For example, the association strength threshold can be set to 0.1. Through historical database analysis, it was found that elevated circulating tumor DNA concentration and unclear tumor boundaries in imaging features often occur simultaneously, that is, liquid biopsy nodes and imaging feature nodes often appear at the same time, and the PMI value exceeds the threshold of 0.1. Therefore, a connection edge is established. It should be noted that the association strength threshold is optimized and determined on an independent validation set through grid search to maximize the AUC value of the prediction model. For example, a historical database containing at least 1000 patients is used, which is randomly divided into a training set and a validation set in a 7:3 ratio. On the training set, the PMI value of all node pairs is calculated, and a heterogeneous data graph is constructed based on this to train the complete prediction model. On the validation set, the prediction performance of the model is evaluated under different association strength thresholds, for example, in the range of 0.05 to 0.2, with a grid search step size of 0.01. The performance index is optimized based on the AUC value. The threshold that results in the highest AUC value on the validation set is selected as the final association strength threshold.

[0076] The second approach is based on expert knowledge-based biological association rules, which encodes known biomedical knowledge as graph connections. For example, if it is medically known that a specific gene mutation promotes tumor angiogenesis (a genomics data node), and this angiogenesis manifests as a specific perfusion pattern on enhanced CT images (an image feature node), then a connection edge is directly established between the genomics data node and the image feature node. By integrating these two mechanisms to establish all connections, a complete heterogeneous data graph is obtained. This heterogeneous data graph not only integrates multi-source data, but more importantly, its topological structure itself encodes deep association information between data, providing structured input for subsequent feature fusion analysis. For example, according to biomedical knowledge, VHL gene mutations are associated with cystic changes in tumors on CT images (genomics node and image feature node), because VHL mutations may lead to abnormal tumor angiogenesis, forming cystic structures. Therefore, in the heterogeneous data graph, a connection edge is directly established between the VHL mutation node and the cystic change feature node. Another example is the association between PBRM1 gene deletion and uniformity in image texture features, because PBRM1 deletion may affect tumor intratumoral heterogeneity.

[0077] In one embodiment of the present invention, a graph attention network is used to process heterogeneous data graphs, including the following steps:

[0078] For each node in the heterogeneous data graph, the association weight between it and its neighboring nodes is dynamically calculated through an attention mechanism;

[0079] Aggregate information from adjacent nodes based on association weights to generate enhanced node representations;

[0080] When two or more specific data features appear simultaneously, the enhanced node representation is output as a fused correlation feature.

[0081] Specifically, this step receives the heterogeneous data graph generated in the previous step as input and processes it using a graph attention network (GAT). The core of this process lies in assigning dynamic association weights to connections in the graph through a self-learning attention mechanism. For any target node i in the heterogeneous data graph, the method first calculates the attention coefficient between i and each of its neighboring nodes j. This coefficient is used to evaluate the importance of node j's features to node i. The calculation process for this coefficient can be described as follows:

[0082]

[0083] in, and These are the initial feature vectors of the target node i and its neighboring node j, respectively. These vectors are obtained directly from the original multimodal data through quantization. It is a learnable weight matrix that linearly transforms initial feature vectors of different modalities, which may have different dimensions and scales, into the same high-dimensional feature space, thus making them comparable. Its initial values ​​are set using the Xavier initialization method to ensure the stability of the gradients in the early stages of training; the symbol " " " represents a vector concatenation operation, which connects two transformed feature vectors into a longer vector; These are the weight vectors of a learnable single-layer feedforward neural network, which can also be initialized using Xavier. At the start of model training, the weight matrix... and attention vector The initial values ​​are set using the Xavier initialization method. This is because the linear transformation layer and attention calculation layer in the GAT network are sensitive to the initial weights. Xavier initialization can adaptively adjust the scale of the initial weights according to the input and output dimensions of the layer, thereby maintaining the stability of the variance of the data flow between layers in the early stage of training, effectively alleviating the gradient vanishing or exploding problem, and accelerating the convergence of the model.

[0084] LeakyReLU is a non-linear activation function that acts on the concatenated vector, outputting a scalar value, namely the attention coefficient. After calculating the attention coefficients between the target node i and all its neighboring nodes, the softmax function is used to normalize these coefficients to obtain the final association weights. After calculating the attention coefficients between the target node i and all its neighboring nodes, the softmax function is used to normalize these coefficients, thus obtaining the final association weights. The associated weight It was then used for weighted aggregation of all adjacent nodes. The features after matrix transformation generate the updated and enhanced node representation of target node i. Ultimately, the set of enhanced node representations obtained after this update of all nodes in the heterogeneous data graph constitutes the required fusion correlation features. Here, specific data features refer to feature combinations that have synergistic indicative significance in clinical diagnosis, such as the simultaneous occurrence of imaging feature A—suggesting vascular invasion and liquid biopsy feature B—increased circulating tumor DNA concentration. The model learns which feature combinations are specific through training and outputs their enhanced node representations as key fusion correlation features.

[0085] It is worth noting that the weight matrix and attention vector The initial values ​​are automatically learned throughout the training process of the prediction model, with the goal of minimizing the prediction error. Specifically, the initial values ​​are usually randomly set using well-established initialization methods in deep learning, such as Xavier initialization or He initialization, to ensure stability and convergence speed in the early stages of training. Subsequently, during model training, the backpropagation algorithm and gradient descent optimizers, such as the Adam optimizer, are used to iteratively update the values ​​based on a historical patient dataset with labeled recurrence patterns. and The numerical value ultimately enables the model to accurately capture the strength of the association between nodes.

[0086] In one embodiment of the present invention, obtaining kidney-specific physiological parameters that quantify the physiological and pathological state of the kidney includes the following steps:

[0087] Based on imaging data, the ratio of tumor volume to ipsilateral renal cortex volume is calculated to obtain the tumor-renal cortex volume ratio.

[0088] The postoperative renal function recovery slope was obtained based on dynamic monitoring data of renal function.

[0089] The imaging data were analyzed to obtain the immune response index predicted by radiomics.

[0090] Based on the tumor-to-renal cortex volume ratio, the postoperative renal function recovery slope, and the immune response index predicted by radiomics, kidney-specific physiological parameters were obtained.

[0091] Specifically, this step aims to extract a set of kidney-specific physiological parameters from multimodal data that have clear biological significance and can directly reflect the unique physiological and pathological state of the kidney organ. First, the tumor-to-cortical volume ratio is calculated. Based on known physiological and anatomical knowledge, namely that the functional units of the kidney are mainly concentrated in the renal cortex, the ratio of tumor volume to cortical volume can serve as a proxy indicator for assessing the relative extent of tumor damage to functional kidney tissue. This ratio effectively reflects the relative scale of tumor damage to functional kidney tissue, and its calculation formula is as follows:

[0092]

[0093] in, The volume of the tumor is precisely delineated and calculated from preoperative CT or MRI images using image segmentation algorithms, such as three-dimensional region growth and U-Net neural networks. The volume of the cortex of the affected kidney is represented by this ratio. Image segmentation algorithms are used to distinguish the cortex and medulla of the kidney in the image, and the volume of the cortex is calculated separately. This ratio is a dimensionless value; a higher ratio indicates a more severe relative invasion of the kidney's functional units by the tumor, and it is positively correlated with the risk of postoperative recurrence.

[0094] It should be noted that the image segmentation algorithm described above can be any applicable algorithm known in the art, and no specific limitations are imposed here. This ratio provides an imagingly measurable parameter reflecting the structural volume proportion of the tumor relative to the functional renal cortex region.

[0095] Secondly, the postoperative renal function recovery slope is calculated. The calculation of this parameter depends on the dynamic monitoring data of renal function at multiple time points after the operation. The details of this process will be described later.

[0096] Finally, the radiomics-predicted immune response index is calculated. This requires combining imaging data with a pre-set radiomics-based immune analysis model. The pre-set model is a pre-trained machine learning model that has learned the correspondence between specific image texture features and the density of immune cell infiltration in immunohistochemical sections. The patient's postoperative imaging data is input into this model, and the model outputs a quantitative value representing the level of immune cell infiltration in the tumor bed region or renal parenchyma—the radiomics-predicted immune response index.

[0097] The pre-defined radiomics immunoassay model can be represented as a linear regression model:

[0098]

[0099] in, The immune response index value predicted by radiomics; To extract the first from the tumor area or peritumoral area of ​​the patient's postoperative imaging Image-based features, such as texture features like entropy and contrast based on the gray-level co-occurrence matrix; For the first The weight coefficients corresponding to each feature are obtained during the model training phase using optimization algorithms such as least squares. This is the bias term of the model, i.e., the intercept; This represents the total number of radiomics features used; after model training is complete, the radiomics feature vectors of new patients will be... Inputting this model will yield a predicted value for its immune response index. .

[0100] For example, the pre-trained radiomics-based immune analysis model is a regression model obtained through the following methods: Historical data of a group of patients who underwent renal tumor surgery are collected, including postoperative images such as enhanced CT or MRI and matched postoperative pathological sections; the pathologist quantitatively analyzes the infiltration density of immune cells on the pathological sections, such as the number of CD8+ T cells per unit area, as a true label value for the intensity of the immune response; radiomics features, such as texture features based on the entropy, contrast, and uniformity of the gray-level co-occurrence matrix, are extracted from the tumor region and / or the peritumoral renal parenchyma region of the postoperative images; machine learning regression algorithms such as random forest or support vector regression are used to establish a mapping model from the extracted radiomics features to the true label value of the immune response intensity; the model is then evaluated on an independent validation set and saved for use, for example, by calculating the Pearson correlation coefficient or root mean square error between the predicted and actual values. For new patients, only the radiomics features of their postoperative images need to be extracted and input into the trained model to obtain their radiomics-predicted immune response index.

[0101] In one embodiment of the present invention, obtaining the postoperative renal function recovery slope includes the following steps:

[0102] Estimated glomerular filtration rate values ​​at multiple time points in the dynamic monitoring data of renal function are obtained to form glomerular filtration rate time series data;

[0103] Curve fitting was performed on the time series data of glomerular filtration rate, and the slope characteristics of the fitted curve were extracted.

[0104] Using the slope characteristic as the slope of postoperative renal function recovery, the dynamic trend of renal function recovery or deterioration over time was obtained.

[0105] Specifically, this step details how to extract the postoperative renal function recovery slope, which reflects the trend of functional changes, from discrete renal function monitoring data. First, it is necessary to acquire dynamic renal function monitoring data collected at different preset time points post-surgery, specifically a series of time points paired with corresponding estimated glomerular filtration rate (eGFR) values. For example, eGFR values ​​are measured at 1 month, 3 months, and 6 months post-surgery, thus constructing an eGFR time series data containing at least three data points. Subsequently, to quantify the overall trend of change from this time series data, a curve fitting method is used; a direct and effective approach is linear regression analysis. Time is used as the independent variable. eGFR value as dependent variable Establish a linear model The key here is to solve for the slope. This represents the average rate of change of eGFR over time. The slope can be obtained using the least squares method. The calculation formula is as follows:

[0106]

[0107] In this formula, This is the slope characteristic to be determined; This is the total number of eGFR measurement time points; It is the time from the i-th measurement point to the surgery, for example, in months; At a certain point in time Measured value; Indicates all The summation operation is performed on each data point. This is done by summing all collected time points and their corresponding... Substituting the values ​​into this formula will allow you to calculate the slope characteristic. This calculated slope characteristic It was then defined as the slope of postoperative renal function recovery.

[0108] For example, suppose patient A's eGFR values ​​measured at 1, 3, and 6 months post-surgery are 65 mL / min / 1.73 m², 70 mL / min / 1.73 m², and 75 mL / min / 1.73 m², respectively. The time points are in months, i.e., t=[1,3,6], and eGFR=[65,70,75]. Using the formula above, the result is calculated as follows: ; ; ; ; ; calculate based on this The slope is approximately 1.97 mL / min / 1.73 m² / month, indicating an average monthly improvement of 1.97 units in renal function, suggesting good recovery. This parameter dynamically reflects the trend of renal function changes and, compared to traditional static indicators, can predict the risk of relapse earlier, as renal function deterioration often precedes clinical relapse. By capturing dynamic trends, the timeliness and accuracy of prediction are improved.

[0109] In one embodiment of the present invention, the weighted geometric mean calculation of kidney-specific physiological parameters includes the following steps:

[0110] A score was calculated for each kidney-specific physiological parameter to obtain the parameter score.

[0111] The relapse risk index is obtained by taking a weighted geometric mean of the parameter scores.

[0112] The weighted geometric mean calculation method ensures that when the score of any parameter decreases significantly, the relapse risk index will decrease disproportionately, thereby strengthening the synergistic constraint relationship between the parameters and simulating the biological mechanism that the inhibition of any key link in the relapse process can lead to relapse failure.

[0113] Specifically, this step aims to integrate the heterogeneous set of kidney-specific physiological parameters obtained in the previous step into a single, comprehensive recurrence risk index (RRI) using an algorithm that reflects synergistic constraints. First, each kidney-specific physiological parameter, such as the tumor-to-renal cortex volume ratio and the postoperative renal function recovery slope, needs to be standardized and converted into a uniform, dimensionless parameter score. This conversion is typically achieved through a pre-defined scoring function or normalization function, mapping the raw value of each parameter to a range of 0 to 1, where higher values ​​represent a higher risk of recurrence, thus obtaining a set of parameter scores.

[0114] For example, the raw values ​​of each kidney-specific physiological parameter are processed using a min-max normalization method. Convert to parameter score value :

[0115]

[0116] in, The normalized parameter score value, ranging from between; This refers to the original calculated value of a specific kidney physiological parameter, such as the modified value. Postoperative renal function recovery rate ; and These are the lower and upper limits of the reference population or the preset risk threshold. The preset risk threshold can be determined based on clinical studies or expert consensus, such as the slope of postoperative renal function recovery. It can be set to a typical slope value representing functional deterioration. It can be set to a typical slope value that represents a good recovery.

[0117] It should be noted that, for example, this involves including a representative cohort of patients who have undergone renal tumor surgery in a clinical study, and statistically analyzing the distribution range of each kidney-specific physiological parameter, such as using the 5th percentile and the 95th percentile as... and ;

[0118] The method determined by expert consensus involves multiple senior urologists and oncologists setting clear cutoff ranges for each parameter that are significantly correlated with clinical outcomes, based on their clinical experience. For example, regarding the postoperative renal function recovery slope, the expert panel might unanimously agree that a slope below -2.0 mL / min / 1.73 m² / month represents a clear deterioration in function, and set this as... A slope greater than +1.0 mL / min / 1.73 m² / month represents good recovery, and is set as follows: .

[0119] Subsequently, a weighted geometric mean algorithm was used to fuse the scores of this set of parameters. The calculation formula is as follows:

[0120]

[0121] in, This is the final calculated recurrence risk index. The parameter score represents the i-th kidney-specific physiological parameter. This is the weight assigned to the i-th parameter, reflecting its relative importance in the overall risk assessment. Its value can be learned during model training or pre-set by clinical experts based on experience, and is the sum of all weights. The value is usually 1. This is the total number of kidney-specific physiological parameters. The final recurrence risk index can be calculated by substituting the scores of all parameters and their corresponding weights into this formula, performing a product, and then taking the appropriate power. .

[0122] For example, suppose there are three parameters: tumor-to-renal cortical volume ratio score P1=0.7, postoperative renal function recovery slope score P2=0.3, and radiomics-predicted immune response index score P3=0.5. The weights are set by expert consensus as w1=0.5, w2=0.3, w3=0.2, and the sum is 1; then the recurrence risk index... The weighted geometric mean emphasizes the synergistic constraint of parameters, meaning that a low score for any parameter will lead to a significant decrease in the RRI, simulating the weakest link effect in biology, thus more accurately reflecting the overall risk, enhancing the model's characterization of kidney-specific physiological pathways, and improving the specificity of predictions.

[0123] It should be noted that, The determination is the same as above. and Similarly, for example, optimizing a machine learning model involves: using kidney-specific physiological parameters as features and patient relapse status as a label; training a logistic regression model or a Cox proportional hazards model on a historical patient dataset; and normalizing the regression coefficients of each feature in the trained model, for example, by using the Softmax function to make their sum equal to 1, thus obtaining a set of optimal weights. ;

[0124] The process, pre-defined by clinical experts based on experience, includes: a structured decision-making process such as the Delphi method, gathering opinions from multiple field experts and assigning weights to each parameter representing its relative importance. For example, an expert panel of at least five senior clinical experts, covering specialties such as urology, oncology, and radiology, is formed. The Delphi method is used for multiple rounds of back-to-back consultation and feedback: first, each expert independently assigns weights to each parameter, with a sum of 1; then, the weight distribution and main reasons from the first round are summarized and anonymously reported; experts adjust their judgments based on the feedback; this process is repeated 2-3 rounds until expert opinions converge; finally, the median or average of the weights assigned by each expert in the final round is calculated and normalized to serve as the final pre-defined weights. For example, the expert panel may consider tumor invasiveness to be the most critical factor, thus assigning a higher weight, such as 0.5, to the tumor-to-renal cortex volume ratio, and relatively lower weights, such as 0.3 and 0.2, to the postoperative renal function recovery slope and immune response index. This reflects that the expert consensus considers tumor invasiveness to be the most important risk driver.

[0125] In one embodiment of the present invention, the fusion prediction module includes a long short-term memory network unit to generate a recurrence probability value after kidney tumor surgery, including the following steps:

[0126] At each time point, the fusion association features and the relapse risk index are used simultaneously as inputs to the long short-term memory network units;

[0127] By utilizing the memory gating mechanism of long short-term memory network units, the evolution trajectory of the fusion feature time series and the evolution trajectory of the relapse risk index time series were learned and memorized respectively.

[0128] By integrating the analysis results of the two evolutionary trajectories, a relapse probability value is generated.

[0129] Specifically, this step describes how to use a dynamic fusion prediction model incorporating Long Short-Term Memory (LSTM) network units to perform collaborative analysis on the two time-series information generated in the previous step, in order to output the final relapse probability value. First, at each time point t, the fusion-related feature vector corresponding to that time point is concatenated with the scalar value of the relapse risk index RRI to form a combined input vector. Then, this combined input vector is... The data is fed into an LSTM network unit. It should be noted that an LSTM unit is a variant of a recurrent neural network (RNN), which internally contains three key gating mechanisms: an input gate, a forget gate, and an output gate, as well as cell state control. When combining input vectors When entering the LSTM unit, the forget gate will... and the hidden state of the previous moment To determine cell state Which information needs to be forgotten or retained? Next, the input gate determines the combined input vector at the current time step. What new information will be updated in the cell state? Through the regulation of these two gates, the cell state at the previous moment... Updated to the current cell state This process effectively learns and memorizes the evolution trajectory of the fusion feature time series and the relapse risk index time series. Finally, the output gate is adjusted based on the updated cell state. To determine the output at the current moment, i.e., the hidden state. This hidden state It is a comprehensive representation of historical information encoded by the LSTM unit at the current time point. The hidden state at the last time point in the sequence, i.e., the hidden state at the last time step, is input into one or more fully connected layers and passed through the Sigmoid activation function. The final output is a value between 0 and 1, which is the predicted recurrence probability value.

[0130] Specifically, the calculation process of an LSTM network unit at each time point t is defined by the following formula:

[0131] Forgotten Gate:

[0132] Input Gate:

[0133] Candidate cell status:

[0134] Cell status update:

[0135] Output gate:

[0136] Hidden state output

[0137] in, For time step The input vector is formed by concatenating the fusion correlation features at that time point with the recurrence risk index; This is the hidden state from the previous time step; This represents the cell state at the previous time step; , , and These are the learnable weight matrices corresponding to the forget gate, input gate, candidate cell state, and output gate, respectively. , , and These are the learnable bias vectors corresponding to the above gates, respectively; The Sigmoid activation function compresses the output to the (0,1) interval, simulating the degree of door opening. is the hyperbolic tangent activation function, which compresses the output to the interval (-1, 1); ⊙ represents the Hadamard product;

[0138] The weight matrix and bias vector constitute the learnable parameter set of the LSTM unit. At the start of model training, each parameter in the learnable parameter set is randomly initialized using the Xavier or He initialization method described above. The training objective of the model is to minimize the cross-entropy loss function between the predicted relapse probability and the actual relapse state. The learnable parameter set is iteratively optimized on a historical patient dataset containing multi-time-point sequence data using the backpropagation time-series algorithm (BPTT) and the Adam optimizer. Through this optimization process, the LSTM unit acquires long-term dependencies in the time series, enabling a dynamic adjustment mechanism—that is, adjusting the sensitivity to fused correlation features based on the evolution trend of the relapse rate (RRI).

[0139] Through the synergistic effect of these gating mechanisms, the LSTM unit can learn the dynamic regulatory effect of the recurrence risk index time series on the fused feature time series; for example, when the RRI input indicates a worsening risk, the model will automatically adjust the gating parameters during training to make the input gate... It is more sensitive to small fluctuations in the fusion features, thereby amplifying early recurrence signals; that is, the moderating effect of the LSTM unit learning the evolution trajectory of the recurrence risk index time series on the analysis of the fusion feature time series is not achieved through preset hard rules, but rather is an inherent law that the model learns autonomously in order to minimize prediction error during the end-to-end training process.

[0140] Specifically, during model training, it continuously adjusts all learnable parameters in the LSTM unit, including the weight matrix, through the backpropagation algorithm and gradient descent optimizer. , , and When the RRI input shows a deteriorating trend, that is, when the RRI value itself is high or continues to rise, those small abnormal fluctuations in the fusion correlation features are actually highly correlated with subsequent relapse events. In order to accurately predict these relapse cases, the model needs to pay attention to these small fluctuations in this specific context.

[0141] Therefore, the gradient descent algorithm drives the model to optimize its parameters, making the input gate of the LSTM more efficient under high RRI input conditions. It is more sensitive to changes in fusion association characteristics, allowing more new information to update the cell state, including minute fluctuations; at the same time, the forgetting gate... It may become more conservative, reducing the forgetting of historical risk signals; this entire adjustment process is implicitly encoded in the LSTM network parameters through multiple iterative optimizations. Once the model is trained, it possesses the ability to dynamically adjust its sensitivity to micro-features, i.e., the fusion of related features, based on macro-physiological states, thereby amplifying collaborative abnormal signals and achieving early warning.

[0142] This system uses historical patient datasets for end-to-end supervised training. The training set consists of data sequences from multiple patients who have undergone renal tumor surgery. Complete multimodal data from multiple time points acquired during the follow-up period of each patient, such as imaging data, liquid biopsy data, genomic data, and dynamic monitoring data of renal function, are input into the system for processing. Finally, a fusion-related feature vector and recurrence risk index (RRI) corresponding to each time point are generated.

[0143] The target label for training is the patient's recurrence status, typically defined as the period from the initial postoperative moment to the end of a specified observation period, such as whether tumor recurrence or metastasis occurred 5 years postoperatively. If recurrence occurs within the observation period, the entire time series is labeled as a positive sample, i.e., labeled 1; if no recurrence occurs within the complete observation period, it is labeled as a negative sample, i.e., labeled 0.

[0144] The overall training objective of the model is to minimize its final output, i.e., the difference between the predicted recurrence probability and the true recurrence label. Here, a binary cross-entropy loss function can be used as the optimization objective for model training. The model optimization employs an adaptive moment estimation optimizer, updating all learnable parameters in the network through a time-based backpropagation algorithm. This includes the node feature transformation weight matrix in the graph attention network, the attention mechanism weight vector, the individual gate weight matrices and bias vectors in the long short-term memory network units, and the weights in the weighted geometric mean. The training process is typically divided into multiple batches for iterative optimization, and independent validation sets are used to monitor model performance to prevent overfitting. Optional strategies include adding an L2 weight regularization term to the loss function or applying Dropout to improve the model's generalization ability.

[0145] In one embodiment of the present invention, the collaborative analysis using long short-term memory network units further includes the following steps:

[0146] The Long Short-Term Memory Network unit learns the evolution trajectory of the relapse risk index time series and dynamically adjusts the fusion feature time series. When the relapse risk index time series shows a deteriorating trend, the risk weight of small abnormal fluctuations in the fusion feature time series is increased to establish a nonlinear association, amplify and obtain early relapse signals.

[0147] Specifically, this step elucidates how the Long Short-Term Memory (LSTM) network unit learns the dynamic adjustment effect between two time series. During LSTM model training, through backpropagation and gradient descent, the model continuously adjusts all learnable parameters, including the weight matrices of each gating unit, with the goal of minimizing the difference between the predicted relapse probability and the actual relapse result. It is through this optimization process that the model learns the dynamic adjustment effect. At each time point, when the combined input vector containing the fusion correlation feature and the relapse risk index enters the LSTM unit, the activation state of the input gate and forget gate depends not only on the fusion correlation feature itself but also on the value of the relapse risk index (RRI). For example, when the RRI value is low and shows a downward trend, i.e., the relapse risk index time series shows a deteriorating trend, even if the fusion correlation feature changes only slightly, the model will find during training that ignoring this slight change will increase the final prediction loss. To reduce loss, the gradient descent algorithm drives the model to adjust the weights of the gating units. This makes the input gate more open to minor anomalous fluctuations in the fused correlation features, allowing them to influence cell states more significantly, especially given the deteriorating RRI. Simultaneously, the forget gate may become more conservative, tending to retain previously accumulated risk signals. This weight adjustment essentially means the model automatically learns non-linear gating logic; the RRI state becomes a dynamic valve regulating the flow rate of information from the fused correlation features. Therefore, once the model is trained, its internal parameters solidify this dynamic adjustment mechanism, enabling it to automatically and disproportionately amplify or reduce the risk weights of anomalous signals in the fused correlation features based on the real-time RRI state during prediction, thereby capturing early relapse signals.

[0148] In one embodiment of the present invention, the recurrence risk prediction system for renal tumors based on multimodal fusion, after the data acquisition module acquires multimodal data, a preprocessing module performs preprocessing operations on the multimodal data to obtain standardized multimodal data, which is then used as input to the fusion correlation feature extraction module and the kidney-specific parameter calculation module.

[0149] The preprocessing operations include cleaning, aligning, and normalizing data from different sources to generate standardized multimodal data;

[0150] Specifically, this step, serving as the data foundation of the entire prediction process, involves a series of preprocessing operations to generate standardized multimodal data before sending the multimodal data into subsequent parallel processing channels. The first step is data cleaning, which aims to identify and correct noise and errors in the raw data. For image data, this may include using filters to remove random noise or correcting motion artifacts caused by patient movement; for numerical data such as biomarkers or renal function indicators, this includes handling missing values, for example, estimating missing eGFR values ​​using time-series-based interpolation methods, and identifying and removing outliers that exceed reasonable physiological ranges. Next is data alignment, which ensures spatiotemporal consistency of data from different sources and at different time points. Temporal alignment requires that all data collected within a predetermined time window, such as the first month post-surgery, regardless of the specific day, be labeled as the same time point, ensuring that the LSTM model receives synchronized time series. Spatial alignment primarily targets image data of different modalities, for example, aligning CT and MRI images of the same patient in three-dimensional space using image registration techniques, so that the same voxel in the two images corresponds to the same physical location. Finally, normalization is performed. This process scales all numerical features, such as image texture values ​​and biomarker concentrations, to a uniform scale to eliminate model training bias caused by large differences in units and numerical ranges between different features. A commonly used method is Z-score normalization, which is calculated as follows:

[0151]

[0152] in, These are the normalized data values; It is the original data value; while and These are the mean and standard deviation of the feature in the reference group, respectively. Adaptability is reflected in... and Dynamic selection based on specific patient subgroups, such as age and gender, rather than using globally fixed values, better preserves the relative distribution characteristics of the data within a specific group. After cleaning, alignment, and normalization, the original multimodal data is transformed into standardized multimodal data with consistency and comparability, providing reliable input for all subsequent analytical steps.

[0153] In one embodiment of the present invention, a method for predicting the risk of recurrence after renal tumor surgery based on multimodal fusion is provided, specifically including the following steps:

[0154] S1. Acquire multimodal data of the patient at multiple time points after surgery. The multimodal data includes at least imaging data, liquid biopsy data, and dynamic monitoring data of renal function. Input the multimodal data in parallel into the first processing channel and the second processing channel.

[0155] S2. In the first processing channel, the multimodal data is constructed into a heterogeneous data graph, and the graph attention network is used to process the heterogeneous data graph to generate fusion association features that reflect the collaborative relationship between different modal data.

[0156] S3. In the second processing channel, based on imaging data, liquid biopsy data and dynamic monitoring data of renal function, kidney-specific physiological parameters that quantify the physiological and pathological state of the kidney are calculated.

[0157] S4. The recurrence risk index is obtained by performing a weighted geometric mean calculation on the kidney-specific physiological parameters.

[0158] S5. Construct the fusion correlation features and recurrence risk index into time series of fusion features and time series of recurrence risk index that change over time;

[0159] S6. Perform synergistic analysis on the fusion feature time series and the recurrence risk index time series to generate the recurrence probability value after kidney tumor surgery.

[0160] Prior to the step of inputting multimodal data into two processing channels in parallel, the following steps are included:

[0161] After acquiring multimodal data, preprocessing is performed to obtain standardized multimodal data.

[0162] Preprocessing operations include cleaning, aligning, and normalizing data from different sources to generate standardized multimodal data, thereby improving data consistency in subsequent processing.

[0163] In subsequent steps, standardized multimodal data is used to replace the multimodal data for processing.

[0164] By acquiring multimodal data from multiple postoperative time points, including imaging and renal function information, a foundation for dynamic analysis is laid. Next, the method feeds the data in parallel into two fundamentally different and complementary processing channels. The first processing channel utilizes a graph attention network to construct a graph structure from heterogeneous data from different sources. By learning the attention weights between nodes, it mines and quantifies the hidden, nonlinear collaborative relationships between data, generating fusion correlation features. This solves the problem that traditional methods can only perform shallow data concatenation. The second processing channel focuses on biological significance, extracting and calculating a set of kidney-specific physiological parameters that directly reflect the unique physiological and pathological state of the kidney organ. These parameters are then fused into a comprehensive recurrence risk index using a weighted geometric mean method, which effectively simulates the bottleneck effect in biology. Finally, the method inputs the time-series information from these two channels, representing microscopic correlation patterns and macroscopic physiological states respectively, into a dynamic fusion prediction unit, such as an LSTM network. This unit can learn and memorize the evolutionary trends of the two trajectories and their dynamic regulatory effects on each other, ultimately outputting a recurrence probability value that integrates multi-level and multi-dimensional information through collaborative analysis.

[0165] It should be noted that the formulas described above, through the principle of dimensional consistency and mathematical standardization methods (such as normalization, dimensionless parameter conversion, or unit system unification), can translate physical quantities with different properties into unitless standard values ​​or superimposed parameters of the same dimension. This eliminates the interference of different dimensions on the computational logic, allowing the formulas to retain the original data distribution characteristics while possessing mathematical rationality and adaptability to objective laws. The descriptions are merely exemplary embodiments of the present invention and should not be construed as limiting the scope of the invention.

[0166] Each of the modules can be implemented in whole or in part through software, hardware, or a combination thereof. It supports hardware embedded in or independent of the processor in the computer device, and also supports software stored in the memory of the computer device, so that the processor can call and execute the operations corresponding to each of the above modules.

[0167] It should be noted that the human information (including but not limited to human device information and personal information) and data (including but not limited to data used for analysis, data stored and data displayed) involved in this invention are all information and data authorized by the human body or fully authorized by all parties. The collection, use and processing of related data require relevant legal standards.

[0168] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A multimodal fusion-based system for predicting the risk of recurrent renal tumors after surgery, characterized in that, include: The data acquisition module is used to acquire multimodal data of patients at multiple time points after surgery. The multimodal data includes imaging data, genomic data, liquid biopsy data, and dynamic monitoring data of renal function. The fusion association feature extraction module constructs a heterogeneous data graph from multimodal data and uses a graph attention network to process the heterogeneous data graph to generate fusion association features that reflect the collaborative relationships between different modalities of data. The kidney-specific parameter calculation module calculates the ratio of tumor volume to ipsilateral renal cortex volume based on imaging data to obtain the tumor-renal cortex volume ratio; it obtains the postoperative renal function recovery slope based on dynamic renal function monitoring data; it analyzes the imaging data to obtain the radiomics-predicted immune response index; and it obtains kidney-specific physiological parameters that quantify the physiological and pathological state of the kidney based on the tumor-renal cortex volume ratio, the postoperative renal function recovery slope, and the radiomics-predicted immune response index. The recurrence risk index calculation module is used to perform a weighted geometric mean calculation on kidney-specific physiological parameters to obtain the recurrence risk index. The fusion prediction module constructs fusion-related features and recurrence risk index into time series of fusion features and recurrence risk index that change over time. It then performs a collaborative analysis of the fusion feature time series and recurrence risk index time series to generate the recurrence probability value after kidney tumor surgery.

2. The renal tumor postoperative recurrence risk prediction system based on multimodal fusion according to claim 1, characterized in that, Constructing multimodal data into a heterogeneous data graph includes the following steps: Acquire imaging features, genomic data, liquid biopsy data, and dynamic monitoring data of renal function from multimodal data; Imaging features, genomic data, liquid biopsy data, and dynamic monitoring data of renal function are defined as different types of nodes; Based on the co-occurrence relationships of nodes in clinical cases or the biological association rules pre-set according to existing biomedical knowledge, connection edges are established between different types of nodes to form a heterogeneous data graph.

3. The renal tumor postoperative recurrence risk prediction system based on multimodal fusion according to claim 1, characterized in that, Processing heterogeneous graph data using graph attention networks includes the following steps: For each node in the heterogeneous data graph, the association weight between it and its neighboring nodes is dynamically calculated through an attention mechanism; Aggregate information from adjacent nodes based on association weights to generate enhanced node representations; When two or more specific data features appear simultaneously, the enhanced node representation is output as a fusion-related feature. Here, the specific data features refer to the combination of features that have synergistic indicative significance in clinical diagnosis. The model learns and determines the feature combination through training and outputs its enhanced node representation as a fusion-related feature.

4. The renal tumor postoperative recurrence risk prediction system based on multimodal fusion according to claim 1, characterized in that, To obtain the postoperative renal function recovery slope, the following steps are included: Estimated glomerular filtration rate values ​​at multiple time points in the dynamic monitoring data of renal function are obtained to form glomerular filtration rate time series data; Curve fitting was performed on the time series data of glomerular filtration rate, and the slope characteristics of the fitted curve were extracted. Using the slope characteristic as the slope of postoperative renal function recovery, the dynamic trend of renal function recovery or deterioration over time was obtained.

5. The renal tumor postoperative recurrence risk prediction system based on multimodal fusion according to claim 1, characterized in that, The weighted geometric mean calculation of kidney-specific physiological parameters includes the following steps: A score was calculated for each kidney-specific physiological parameter to obtain the parameter score. The relapse risk index is obtained by taking a weighted geometric mean of the parameter scores.

6. The renal tumor postoperative recurrence risk prediction system based on multimodal fusion according to claim 1, characterized in that, The fusion prediction module includes long short-term memory network units to generate recurrence probability values ​​after kidney tumor surgery, including the following steps: At each time point, the fusion association features and the relapse risk index are used simultaneously as inputs to the long short-term memory network units; By utilizing the memory gating mechanism of long short-term memory network units, the evolution trajectory of the fusion feature time series and the evolution trajectory of the relapse risk index time series were learned and memorized respectively. By integrating the analysis results of the two evolutionary trajectories, a relapse probability value is generated.

7. The renal tumor postoperative recurrence risk prediction system based on multimodal fusion according to claim 6, characterized in that, The analysis results of the two evolution trajectories are integrated to generate a recurrence probability value. This includes the following steps: inputting the hidden state of the long short-term memory network unit at the end time step of the fusion feature time series and the recurrence risk index time series into a fully connected layer, and processing it through an activation function to output the recurrence probability value after kidney tumor surgery.

8. The renal tumor postoperative recurrence risk prediction system based on multimodal fusion according to claim 1, characterized in that, After the data acquisition module acquires multimodal data, a preprocessing module performs preprocessing operations on the multimodal data to obtain standardized multimodal data. The standardized multimodal data is then used as input to the fusion correlation feature extraction module and the kidney-specific parameter calculation module. The preprocessing operations include cleaning, aligning, and normalizing data from different sources to generate standardized multimodal data.

9. A method for predicting the risk of recurrence after kidney tumor surgery, implemented using the multimodal fusion-based kidney tumor recurrence risk prediction system as described in any one of claims 1-8, characterized in that, Includes the following steps: S1. Acquire multimodal data of the patient at multiple time points after surgery. The multimodal data includes at least imaging data, liquid biopsy data, and dynamic monitoring data of renal function. Input the multimodal data in parallel into the first processing channel and the second processing channel. S2. In the first processing channel, the multimodal data is constructed into a heterogeneous data graph, and the graph attention network is used to process the heterogeneous data graph to generate fusion association features that reflect the collaborative relationship between different modal data. S3. In the second processing channel, based on the imaging data, the ratio of tumor volume to ipsilateral renal cortex volume is calculated to obtain the tumor-renal cortex volume ratio; based on the dynamic monitoring data of renal function, the postoperative renal function recovery slope is obtained; the imaging data is analyzed to obtain the radiomics-predicted immune response index; based on the tumor-renal cortex volume ratio, the postoperative renal function recovery slope, and the radiomics-predicted immune response index, kidney-specific physiological parameters that quantify the physiological and pathological state of the kidney are obtained. S4. The recurrence risk index is obtained by performing a weighted geometric mean calculation on the kidney-specific physiological parameters. S5. Construct the fusion correlation features and recurrence risk index into time series of fusion features and time series of recurrence risk index that change over time; S6. Perform synergistic analysis on the fusion feature time series and the recurrence risk index time series to generate the recurrence probability value after kidney tumor surgery.

Images