Tumor recurrence and metastasis risk prediction method and system based on anatomical atlas

Abstract

The application discloses a tumor recurrence and metastasis risk prediction method and system based on an anatomical atlas, and the method comprises the following steps: acquiring a patient image report, a chief complaint text and non-equidistant tumor marker data; based on a tumor metastasis knowledge graph, the image report and the chief complaint text are mapped to corresponding anatomical nodes, the semantic alignment weight of objective lesions and subjective symptoms on the nodes is calculated by using a cross-attention mechanism, and multi-modal fusion node features are generated; the biochemical trend features of the marker data are extracted as global state variables by using a continuous time model to inject the graph; finally, the multi-modal fusion node features and the global state variables are input into a graph neural network for feature aggregation and reasoning, and a risk prediction result and a high-risk anatomical region are output. The application realizes early and accurate early warning and an interpretable path reasoning for tumor recurrence and metastasis risk.

Description

Technical Field

[0001] This invention belongs to the field of medical data analysis technology, and specifically relates to a method and system for predicting the risk of tumor recurrence and metastasis based on anatomical atlases. Background Technology

[0002] Tumor recurrence and metastasis are a major cause of death in cancer patients. During clinical follow-up, physicians typically need to comprehensively assess the trends of changes in imaging examinations (such as computed tomography (CT) and magnetic resonance imaging (MRI), tumor markers (such as carcinoembryonic antigen CEA and CA19-9), and the patient's subjective symptoms to determine the risk of recurrence or metastasis. With the development of deep learning technology, using artificial intelligence to assist in tumor prognostic analysis has become a research hotspot.

[0003] Chinese invention application CN117238489A discloses a method for predicting distant metastases of head and neck tumors in multimodal nuclear medicine imaging. This method involves registering, normalizing, and preprocessing PET and CT images, constructing an encoder using a residual network to extract image features, and building a decoder by combining time points to predict the probability of distant metastases at specific time points. While this method achieves some progress in image feature extraction and multi-time-point prediction, it is essentially still a single-dimensional visual modality analysis. However, tumor recurrence is often insidious; abnormal fluctuations in biochemical indicators or minor patient symptoms often precede lesions visible on imaging. Relying solely on imaging data can easily lead to missed diagnoses of early biochemical recurrence or micrometastases.

[0004] Chinese invention patent application CN119007197A discloses a method and device for predicting tumor cell metastasis tendency. This method, based on an improved Yolov8 model, introduces a channel attention mechanism to classify and predict tumor cell images, thereby determining the cell's metastatic tendency. This approach focuses on microscopic pathological cell morphology analysis and is mainly used for postoperative pathological typing or research analysis; it cannot be applied to long-term clinical follow-up monitoring of patients after discharge. Furthermore, this approach cannot effectively model dynamic physiological parameters and subjective feelings experienced by patients during follow-up.

[0005] In summary, existing tumor recurrence and metastasis prediction technologies mainly suffer from the following technical bottlenecks and limitations:

[0006] In clinical diagnosis, the consistency between the chief complaint and imaging is an important basis for judgment. Existing models cannot handle the hidden risk scenario of significant symptoms but negative imaging. Existing deep learning models only explore the statistical correlation between data, but cannot perform interpretable path reasoning based on anatomical logic. Conventional recurrent neural networks (RNNs) or long short-term memory networks (LSTMs) are difficult to accurately capture the dynamic evolution rate and acceleration of tumor markers on a continuous time axis, resulting in insufficient early warning ability for disease mutations. Summary of the Invention

[0007] This invention provides a method and system for predicting the risk of tumor recurrence and metastasis based on anatomical atlases, aiming to solve the problems of insufficient processing and early warning capabilities in the existing technology when assessing the risk of tumor metastasis.

[0008] To address the aforementioned technical problems, this invention proposes a method for predicting the risk of tumor recurrence and metastasis based on anatomical atlases, comprising the following steps:

[0009] Acquire patient imaging reports, chief complaint texts, and non-equidistant tumor marker data;

[0010] Based on the tumor metastasis knowledge graph, the imaging reports and chief complaint texts are mapped to the corresponding anatomical nodes. The semantic alignment weights of objective lesions and subjective symptoms on the nodes are calculated using the cross-attention mechanism to generate multimodal fusion node features.

[0011] Biochemical trend features of biomarker data extracted using a continuous-time model are injected into the map as global state variables.

[0012] Finally, the multimodal fusion node features and global state variables are input into the graph neural network for feature aggregation and inference, and the risk prediction results and high-risk anatomical regions are output.

[0013] Preferably, the method for constructing the tumor metastasis knowledge graph is as follows:

[0014] The key points of human organs, tissues, lymph node groups, and blood vessels are used as map nodes;

[0015] Directed edges of a graph based on lymphatic drainage pathways, blood circulation pathways, and anatomical proximity relationships;

[0016] Each directed edge has a pre-defined prior transition probability weight based on medical statistics, which is used to initialize the edge weights of the graph neural network.

[0017] Preferably, the mapping of the image report and chief complaint text to the corresponding anatomical node specifically involves:

[0018] Using a pre-trained medical named entity recognition model, anatomical site entities and lesion description entities are extracted from the image report and chief complaint text;

[0019] Calculate the semantic similarity between the extracted anatomical site entities and the nodes in the knowledge graph;

[0020] The feature vector of the lesion description entity is used as a node attribute and attached to the corresponding graph node with the highest semantic similarity.

[0021] Preferably, the step of calculating the semantic alignment weights between objective lesions and subjective symptoms at a node using the cross-attention mechanism specifically involves:

[0022] The subjective symptom feature vector of the anatomical node is used as the query vector, and the feature vector of the objective lesion is used as the key vector and value vector.

[0023] Calculate the attention score of the dot product between the query vector and the key vector, and use it as the subjective-objective consistency coefficient;

[0024] When the subjective-objective consistency coefficient is lower than a preset threshold and the magnitude of the subjective symptom feature vector is greater than a preset symptom intensity threshold, a high-weight differentiation mask is generated to amplify the feature weight of the corresponding node during fusion.

[0025] Preferably, the generation of the multimodal fusion node features also incorporates a time decay mechanism:

[0026] For historical image reports or chief complaint texts, the corresponding feature vectors are weighted using an exponential decay function based on the time difference between their generation time and the current prediction time, so that recent clinical evidence has a higher weight in feature fusion.

[0027] Preferably, the continuous-time model adopts a neural ordinary differential equation model;

[0028] By solving ordinary differential equations, the evolution trajectory of tumor markers sampled at non-equidistant intervals is fitted on a continuous time axis;

[0029] The biochemical trend features include the current predicted values ​​of the biomarkers, as well as the first and second derivatives of the biomarker's change trajectory.

[0030] Preferably, the biochemical trend features are injected into the graph as global state variables, specifically as follows:

[0031] The biochemical trend features are used as a global context vector;

[0032] The global context vector is modulated into the feature representation of each anatomical node in the graph neural network through an adaptive instance normalization layer or gating mechanism.

[0033] Preferably, the graph neural network adopts a graph attention network architecture;

[0034] During feature aggregation, the model dynamically adjusts the attention coefficients of the directed edges between each anatomical node based on the global state variables to simulate the probability changes of tumor metastasis pathways under different biochemical backgrounds.

[0035] Preferably, after outputting the risk prediction results and high-risk anatomical areas, the method further includes:

[0036] Generate a counterfactual explanation report;

[0037] Specifically, by fine-tuning the tumor marker values ​​at the input end or removing specific chief symptom descriptions, the magnitude of change in risk prediction results is observed, and the contribution ranking of each modality data to the current risk prediction results is calculated and output.

[0038] Secondly, the present invention also proposes a tumor recurrence and metastasis risk prediction system based on anatomical atlases, said system being used to implement the risk prediction method as described in the first aspect of the present invention, comprising:

[0039] The data acquisition module is used to acquire patient imaging reports, chief complaint texts, and non-equidistant tumor marker data;

[0040] The semantic mapping and feature fusion module is used to map the image report and chief complaint text to the corresponding anatomical nodes based on the tumor metastasis anatomy knowledge graph, and run a cross-attention mechanism to calculate the semantic alignment weights of objective lesions and subjective symptoms on the nodes, thereby generating multimodal fusion node features.

[0041] The biochemical trend evolution module is used to run a continuous-time model to extract the biochemical trend features of the biomarker data and inject them as global state variables into the knowledge graph.

[0042] The graph neural network inference module is configured to input the multimodal fusion node features and global state variables into the graph neural network for feature aggregation and inference, and output risk prediction results and high-risk anatomical regions.

[0043] Compared with the prior art, the present invention has the following technical effects:

[0044] 1. The tumor recurrence and metastasis risk prediction method proposed in this invention performs semantic alignment between the patient's subjective symptoms and objective imaging features at the anatomical node level. The model can capture the hidden risk pattern of significant symptoms but negative imaging, simulate the diagnostic thinking of doctors in clinical practice, and significantly reduce the risk of missed diagnosis caused by imaging lag.

[0045] 2. The tumor recurrence and metastasis risk prediction method proposed in this invention uses neuronormal differential equations for continuous-time modeling. It can not only fit the continuous evolution trajectory of biomarkers using sparse sampling points, but also accurately capture the rate of change and acceleration of tumor biomarkers by calculating the first and second derivatives, thereby triggering early warnings when the indicator values ​​have not yet exceeded the standard but the trend is abnormal.

[0046] 3. The tumor recurrence and metastasis risk prediction method proposed in this invention utilizes global state injection technology to map systemic biochemical indicators to background risks of local anatomical nodes. This enables deep learning models to deduce specific risk metastasis paths based on the medical logic of lymphatic drainage and blood circulation, and intuitively output the high-risk anatomical regions predicted by the model.

[0047] 4. The tumor recurrence and metastasis risk prediction method proposed in this invention introduces an explanation generation module based on counterfactual reasoning, which can quantify the marginal contribution of each modality of data to the risk and improve the efficiency of screening for false positive interference. Attached Figure Description

[0048] Figure 1 This is a flowchart illustrating the risk prediction method described in this invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present application and with reference to the accompanying drawings.

[0050] Example 1

[0051] This embodiment describes a method for predicting the risk of tumor recurrence and metastasis based on anatomical atlases. Figure 1 As shown, it includes the following steps one through four:

[0052] Step 1: Obtain the patient's imaging report, chief complaint text, and non-equidistant tumor marker data.

[0053] The imaging report does not refer to the raw DICOM image data, but rather to a text-based diagnostic report generated by a radiologist after interpreting the images. This data is either uploaded by the patient voluntarily through the platform's app or automatically retrieved via relevant interfaces after patient authorization. It includes, but is not limited to, reports from examinations such as computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET-CT), ultrasound, or bone scans. The retrieved data is a text string, from which the imaging findings and diagnostic opinions are extracted.

[0054] For example, the imaging findings field of a chest CT report may include: a ground-glass nodule with a diameter of about 5 mm and indistinct borders is visible in the upper lobe of the right lung; several lymph nodes are visible in the mediastinum, with a short diameter of about 4-6 mm.

[0055] The chief complaint reflects the patient's subjective physical experience and is an important source for early symptom detection. Data sources are divided into two parts: first, fields such as the chief complaint and present illness history recorded in the medical record; second, symptom descriptions self-reported by the patient through the platform's app, presented in natural language text format. The focus is on collecting descriptions related to pain (location, nature, severity), respiratory symptoms (cough, hemoptysis), digestive symptoms (abdominal distension, rectal bleeding), and systemic symptoms (fatigue, weight loss, fever).

[0056] Non-equidistant tumor marker data are key objective indicators reflecting tumor biological activity, characterized by the uncertainty of sampling time. This data is either actively uploaded by patients via the platform's app or automatically retrieved through relevant interfaces after patient authorization. Appropriate marker combinations are selected based on the primary cancer type. For example, for lung cancer patients, the combination includes CEA, CYFRA21-1 (soluble fragment of cytokeratin 19), NSE (neuron-specific enolase), and SCC (sigmoid cell count); for gastrointestinal tumor patients, the combination includes CA19-9, CA72-4 (carbohydrate antigen 72-4), CEA, and CA125 (carbohydrate antigen 125). Because patient follow-up times are not fixed, the data are organized into time-value pairs, defined as follows:

[0057]

[0058] in, Indicates the first The timestamp of each test is usually standardized as the number of days since the surgery or diagnosis; This indicates the corresponding detection value, with a unified unit of measurement, such as ng / ml.

[0059] Example data:

[0060] One day post-surgery (1 month), CEA = 2.5 ng / ml;

[0061] 4 days post-surgery, CEA = 3.1 ng / ml;

[0062] The test was performed on the day of [date] (an additional test was conducted due to illness), and CEA was 5.8 ng / ml.

[0063] Here and The time interval between them is significantly shorter than and This demonstrates the non-equidistant spacing characteristic, which serves as the data foundation for subsequent steps using neural differential equation modeling.

[0064] To ensure the quality of the input model data, the following processing is also included after obtaining the above data:

[0065] Desensitization process: Removing patient's name, ID number, and other private information.

[0066] Text cleaning: Remove special symbols and stop words from imaging reports and chief complaint texts, and standardize the abbreviations of medical terms.

[0067] Time alignment: All image report times, chief complaint recording times, and biomarker detection times are mapped onto the same relative time axis to facilitate subsequent model capture of the temporal correlation of multimodal events.

[0068] Step Two: Based on the tumor metastasis knowledge graph, the imaging reports and chief complaint text are mapped to corresponding anatomical nodes. A cross-attention mechanism is used to calculate the semantic alignment weights of objective lesions and subjective symptoms on each node, generating multimodal fusion node features. This step aims to transform unstructured text data into structured graph node features and simulates the logic of doctors' judgments regarding the consistency of symptoms and signs through algorithms. The specific implementation process includes the construction of the tumor metastasis knowledge graph, entity mapping and feature mounting of multimodal text, cross-attention mechanism and latent risk mining, and a time decay mechanism.

[0069] First, the system needs to pre-set a static anatomical atlas structure in the database. The method for constructing the tumor metastasis knowledge graph is as follows: key points of human organs, tissues, lymph node groups and blood vessels are used as atlas nodes; lymphatic drainage paths, blood circulation paths and anatomical proximity relationships are used as directed edges of the atlas; each directed edge is pre-set with a prior metastasis probability weight based on medical statistical data, which is used to initialize the edge weights of the graph neural network.

[0070] Nodes: Referencing the FMA (Foundational Model of Anatomy) or SNOMED-CT (Systematized Nomenclature of Medicine - Clinical Terms) medical ontology libraries, define the set of nodes. .For example, Represents the upper lobe of the left lung. Represents the left supraclavicular lymph nodes. It represents the 3rd thoracic vertebra.

[0071] Edges definition: Defines the adjacency matrix If there is anatomical lymphatic drainage (e.g., left lung -> hilar lymph nodes) or a high-risk pathway for hematogenous metastasis (e.g., lung -> brain), then a directed edge is established.

[0072] Prior weight initialization: initial weights of edges Instead of random initialization, the model is set based on large-scale epidemiological statistics. For example, based on literature data, the probability of breast cancer metastasizing to the ipsilateral axillary lymph nodes is 60%, so the weight of the corresponding side is set to 0.6; the probability of metastasis to the contralateral axillary node is extremely low, so it is set to 0.01. This allows the model to possess medical common sense from the initial training phase.

[0073] After constructing the knowledge graph skeleton, the personalized patient data obtained in step one needs to be populated into the knowledge graph. The mapping of image reports and chief complaint texts to corresponding anatomical nodes specifically involves: using a pre-trained medical named entity recognition model to extract anatomical site entities and lesion description entities from the image reports and chief complaint texts; calculating the semantic similarity between the extracted anatomical site entities and nodes in the knowledge graph; and attaching the feature vector of the lesion description entity as a node attribute to the corresponding knowledge graph node with the highest semantic similarity.

[0074] For model selection, a BERT-BiLSTM-CRF (Bidirectional Encoder Representations from Transformers-Bidirectional LongShort-Term Memory-Conditional Random Field) model, fine-tuned with a Chinese medical corpus, was used for Named Entity Recognition (NER). For example, if the input text is: "A soft tissue nodule is visible in the posterior segment of the right lower lobe, accompanied by pleural traction," then the extracted anatomical entity is: "posterior segment of the right lower lobe"; the extracted pathological entities are: "soft tissue nodule" and "pleural traction." After NER entity extraction, cosine similarity is used to calculate the vector distance between the extracted anatomical entity and the standard node name in the atlas. For example, the "posterior segment of the right lower lobe" has the highest similarity to the atlas node "right lower lobe," and is therefore assigned to that node.

[0075] Feature vectorization and embedding utilize the BERT model to transform lesion descriptions into fixed-dimensional embedding vectors, denoted as objective lesion feature vectors. Similarly, the symptom descriptions in the chief complaint text are transformed into subjective symptom feature vectors. If a node is not mentioned in the report, it is filled with a zero vector.

[0076] The method utilizes a cross-attention mechanism to calculate the semantic alignment weights of objective lesions and subjective symptoms at a node, simulating the thought process of a doctor examining a patient based on symptoms. Specifically, the subjective symptom feature vector of the anatomical node is used as the query vector, and the feature vector of the objective lesion is used as the key vector and value vector. The dot product attention score of the query vector and the key vector is calculated as the subjective-objective consistency coefficient. When the subjective-objective consistency coefficient is lower than a preset threshold, and the magnitude of the subjective symptom feature vector is greater than a preset symptom intensity threshold, a high-weight differential mask is generated to amplify the feature weights of the corresponding node during fusion.

[0077] In mathematical implementation, the query vector is first defined. key vector value vector Calculate attention score :

[0078]

[0079] in, The learnable weight matrix represents the linear projection matrices of the query, key, and value, respectively. These are parameters automatically updated by the neural network during training using the backpropagation algorithm, and are used to transform the original anatomical node subjective symptom feature vectors. and objective lesion feature vector They are mapped to a common latent semantic space to facilitate similarity calculation; Represents the key vector The transpose of the matrix; This represents the dimension of the key vector.

[0080] Scenario A of a confirmed diagnosis: The patient cries out in pain ( Strong), imaging shows a large tumor ( (Strong), the two are semantically similar, then High, with normal fusion of features.

[0081] Another example of a hidden risk scenario B: the patient cries out in pain ( Strong), imaging showed no abnormalities or minor lesions ( (weak or semantic mismatch), at this time Low.

[0082] The specific logic of the differentiation mask is as follows:

[0083] The system detected (Inconsistent) and (Severe symptoms), among which, It is a preset or learnable scalar threshold, with a value range of [0,1]. This threshold is used to measure the magnitude of the subjective symptom feature vector, filtering out descriptions of mild discomfort. A mask is then generated. ( (A scalar coefficient greater than 1) forcibly increases the attention of this node in the entire image, indicating that although the image at this location appears normal, there is an extremely high risk of missed diagnoses or micrometastases. Final fusion feature calculation:

[0084]

[0085] Since the patient data is a time series spanning several years, cough symptoms from two years ago have a relatively small impact on current risk. Therefore, the generation of the multimodal fusion node features also incorporates a time decay mechanism:

[0086] For historical image reports or chief complaint texts, the corresponding feature vectors are weighted using an exponential decay function based on the time difference between their generation time and the current prediction time, so that recent clinical evidence has a higher weight in feature fusion.

[0087] Regarding the setting of the decay function, the current prediction time is... The generation time of a certain report or chief complaint is Then calculate the time weight. :

[0088]

[0089] in, This is a preset, learnable decay constant.

[0090] The above calculations take This yields the multimodal fusion node features of the final input graph neural network.

[0091] Step 3: Utilize a continuous-time model to extract biochemical trend features from biomarker data and inject them into the atlas as global state variables. Traditional recurrent neural networks (RNN / LSTM) assume data is input at fixed time steps (e.g., daily, weekly), making it difficult to handle irregular sampling such as patients on days 1, 45, and 52. This embodiment introduces neural frequent differential equations to solve this problem; that is, the continuous-time model adopts a neural frequent differential equation model.

[0092] By solving ordinary differential equations, the evolution trajectory of tumor markers sampled at non-equidistant intervals is fitted along a continuous time axis; specifically, let... For time The potential state vector of the tumor marker at time step [time]. The rate of change of the marker state over time is defined as a derivative function:

[0093]

[0094] in, It is a function parameterized by a neural network, used to fit the dynamic changes of tumor biomarkers. These are network parameters.

[0095] Given an initial time status of marker observations For any future point in time The system uses a numerical integrator to solve the integral:

[0096]

[0097] This allows the model to extrapolate the state of markers with arbitrary time precision, filling the gaps between sampling points.

[0098] In response to the invasiveness and explosiveness of tumors, the model not only predicts numerical values ​​but also focuses on the trends of change. That is, the biochemical trend features include the current predicted values ​​of biomarkers, as well as the first and second derivatives of the biomarker's change trajectory.

[0099] Current forecast value That is, the current state obtained by integration. This reflects the current tumor burden level.

[0100] First derivative calculate The rate of increase of a marker is characterized. For example, the CEA rises from 5 to 10, and rises from 5 to 50. Although the starting values ​​are the same, the first derivatives are very different, and the latter indicates a very high risk.

[0101] Second derivative Taking the derivative of the first derivative again, that is... The second derivative represents the rate of disease progression. If the second derivative is positive and large, it suggests that the disease is experiencing an exponential surge rather than a linear increase.

[0102] By combining the above three elements, a global biochemical trend feature vector is formed. .

[0103] Blood parameters are systemic environmental variables; they do not belong to any specific organ, but they affect the probability of metastasis to all organs. Therefore, this embodiment introduces a global feature graph injection mechanism. Specifically, the biochemical trend features are injected into the graph as global state variables. This is achieved by using the biochemical trend features as a global context vector; and by using an adaptive instance normalization layer or gating mechanism, the global context vector is modulated into the feature representation of each anatomical node in the graph neural network.

[0104] In the graph injection mechanism, the global state variable is as described above. For each anatomical node in the atlas Its input features are This refers to the multimodal fusion features generated in step two. The system utilizes two independent linear projection layers, based on global vectors. Generate scaling factor and bias factor :

[0105]

[0106]

[0107] In the formula, This represents the nonlinear mapping operation of a multi-layer perceptron; specifically, it consists of at least two fully connected layers and an intermediate activation function. The first layer maps the global biochemical trend feature vector. Mapped to the hidden layer space; after nonlinear activation, the second layer projects it to a space with the same dimension as the anatomical node feature vectors. The scaling factor is then calculated. For example, this operation can be described as:

[0108]

[0109] in, This is the weight matrix. For bias terms, This is the activation function.

[0110] Injection calculation:

[0111]

[0112] in, This represents the modulated node feature vector, which will replace the original... These serve as input node features for message passing and feature aggregation in subsequent steps of the graph neural network; and These are the mean and standard deviation of the node features, respectively; This represents the alignment operation of the feature statistics, which includes two sub-steps: standardization and remodulation.

[0113] Standardization: Standardization of the local node feature vectors of the input Perform mean and unit variance removal processing, i.e. The purpose is to eliminate the original statistical distribution characteristics of the node;

[0114] Remodulation: Scaling factor generated using global biochemical trends and bias factor Perform an affine transformation on the standardized features, i.e., multiply And add .

[0115] Under the map injection mechanism, if This indicates an outbreak of the disease. and It amplifies the risk characteristics of all nodes, making even minor image anomalies significantly captured in graph network inference.

[0116] Step 4: Finally, the multimodal fusion node features and global state variables are input into a graph neural network (GNN) for feature aggregation and inference, outputting risk prediction results and high-risk anatomical regions. This step utilizes the relational reasoning capabilities of the graph neural network to simulate the spread of tumor cells in human anatomical structures. Compared to traditional GNNs, this embodiment introduces a dynamic attention mechanism controlled by the global biochemical environment.

[0117] The graph neural network adopts a graph attention network (GAT) architecture. In this embodiment, it is specifically a multi-layer (preferably 2-3 layers) stacked graph attention network; the input of each layer is a set of node features. and global state variables ,in This corresponds to the features modulated by AdaIN in step three. During feature aggregation, the model dynamically adjusts the attention coefficients of the directed edges between each anatomical node based on the global state variables to simulate the probability changes of tumor metastasis pathways under different biochemical backgrounds.

[0118] This embodiment will use global state variables. Explicitly incorporated into the attention scoring function, for any edge in the graph pointing from node i to node j, its unnormalized attention coefficient... The calculation formula is:

[0119]

[0120] in, This represents a vector concatenation operation; It is a learnable linear transformation matrix of node features; It is a dedicated projection matrix for global state variables; It is the weight vector of a single-layer feedforward neural network.

[0121] Then use the Softmax function to Conversion to normalized attention weights :

[0122]

[0123] in, Represents the set of neighboring nodes, here The transition probability was directly simulated. When When a CEA trend indicates a high risk, the model will automatically learn through training to enhance the edge weights that point from the primary lesion node to nodes prone to metastasis. This allows risk information to flow more smoothly to these high-risk areas.

[0124] Then, by weighted summation, the feature representation of each node is updated to simulate the propagation of risk in the dissected network.

[0125] The aggregation formula is:

[0126]

[0127] in, For the updated feature representation, It is a non-linear activation function.

[0128] To ensure stable training, this embodiment adopts... Each independent attention head is computed in parallel, and the results are then concatenated or averaged. After propagation through the layered graph network, the final node feature representations are obtained. Two downstream tasks are then executed: full-map risk prediction and high-risk region localization.

[0129] In the full-map risk prediction task, the system performs global pooling or readout operations on all node features to obtain the full-map feature vector. Through a fully connected layer and an activation function, the output is a scalar between 0 and 1. This represents the total probability of a patient experiencing recurrence or metastasis within a specified future timeframe.

[0130] In high-risk anatomical region localization tasks, the system identifies the final features of each anatomical node. Each dissected node is processed by a shared-weight MLP classifier, which outputs a local risk score. Select Nodes exceeding a set threshold will generate a list of high-risk anatomical regions. For example: L4 vertebral body -0.85, right supraclavicular lymph node -0.92.

[0131] In one embodiment of the present invention, after outputting the risk prediction results and high-risk anatomical regions, the method further includes step five: generating a counterfactual interpretation report; specifically, by fine-tuning the tumor marker values ​​at the input end or removing specific chief symptom descriptions, observing the magnitude of change in the risk prediction results, calculating and outputting the contribution ranking of each modality data to the current risk prediction results.

[0132] This step is a post-hoc interpretive analysis. The model does not need to be retrained; instead, the pre-trained model is used as the inference engine to explore the causal logic of decisions through systematic input perturbations. The specific implementation process is as follows:

[0133] First, the system records the original risk prediction probability calculated based on the patient's real data, denoted as... Then, the system generates counterfactual samples for data from different modalities.

[0134] Fine-tuning strategies for tumor markers:

[0135] For the latest tumor marker value obtained in step one, the system does not directly reset it to zero, but instead adjusts it down by percentage steps to simulate the hypothetical scenario of the patient's condition improving.

[0136] Specifically, for the first a marker Generate counterfactual input ,in As a perturbation factor, in this embodiment, The preferred setting is [0.1, 0.3]. Keeping all other inputs constant, only changing this indicator, a counterfactual sample set is constructed. .

[0137] Removal strategies targeting the chief complaint symptoms:

[0138] The subjective symptom feature vector extracted and attached to the anatomical node in step two What if the patient didn't have this symptom? This relates to the differential diagnostic thinking in clinical practice. (Regarding nodes...) The symptoms mentioned above will force the system to disable this node. Set it to an all-zero vector, or force its attention weights to 0. Keeping image features and global biochemical indices unchanged, construct a counterfactual sample set. .

[0139] Then, each counterfactual sample generated above is re-input into the already trained model (after the divine frequent differential equation in step three and GAT in step four), and forward propagation is performed to obtain new prediction probabilities.

[0140] For markers Counterfactual probability:

[0141]

[0142] For symptoms Counterfactual probability:

[0143]

[0144] Next, the importance of features is quantified using the finite difference method:

[0145]

[0146]

[0147] The higher the CIS value, the more significant the risk reduction is after the indicator or symptom is removed or improved, proving that this characteristic is the core reason for the current high risk.

[0148] Finally, the system calculates all the results. The values ​​are sorted in descending order to generate a visual explanatory report. Those skilled in the art will understand that, to reduce noise in the report, the system may preset a display threshold. (For example Only when Only then will this feature be included in the final contribution ranking table. This is to eliminate features with negligible impact on risk, thus reducing the reading burden.

[0149] For example:

[0150] An abnormally high CEA level (contribution: 0.35) – a major source of risk;

[0151] L4 vertebral body complaint of pain (contribution: 0.22) – secondary risk factor;

[0152] Lung imaging nodules (contribution: 0.05) – non-major factor.

[0153] If needed, preset natural language generation templates can also be used to output text reports, such as:

[0154] The model predicts a recurrence risk of 85%. Counterfactual analysis shows that if the patient's CEA level decreases by 20%, the risk will drop to 50%; if the patient's persistent low back pain is removed, the risk will drop to 63%. It is recommended to investigate the possibility of biochemical recurrence and bone metastasis.

[0155] Example 2

[0156] This embodiment is a tumor recurrence and metastasis risk prediction system based on anatomical atlases. The system is used to implement the risk prediction method as described in Embodiment 1, including:

[0157] The data acquisition module is used to acquire patient imaging reports, chief complaint texts, and non-equidistant tumor marker data;

[0158] The semantic mapping and feature fusion module is used to map the image report and chief complaint text to the corresponding anatomical nodes based on the tumor metastasis anatomy knowledge graph, and run a cross-attention mechanism to calculate the semantic alignment weights of objective lesions and subjective symptoms on the nodes, thereby generating multimodal fusion node features.

[0159] The biochemical trend evolution module is used to run a continuous-time model to extract the biochemical trend features of the biomarker data and inject them as global state variables into the knowledge graph.

[0160] The graph neural network inference module is configured to input the multimodal fusion node features and global state variables into the graph neural network for feature aggregation and inference, and output risk prediction results and high-risk anatomical regions.

[0161] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A method for predicting the risk of tumor recurrence and metastasis based on anatomical atlases, characterized in that, Includes the following steps: Acquire patient imaging reports, chief complaint texts, and non-equidistant tumor marker data; Based on the tumor metastasis knowledge graph, the imaging reports and chief complaint texts are mapped to the corresponding anatomical nodes. The semantic alignment weights of objective lesions and subjective symptoms on the nodes are calculated using the cross-attention mechanism to generate multimodal fusion node features. Biochemical trend features of biomarker data extracted using a continuous-time model are injected into the map as global state variables. The multimodal fusion node features and global state variables are input into a graph neural network for feature aggregation and inference, and the risk prediction results and high-risk anatomical regions are output.

2. The method according to claim 1, characterized in that, The method for constructing the tumor metastasis knowledge graph is as follows: The key points of human organs, tissues, lymph node groups, and blood vessels are used as map nodes; Directed edges of a graph based on lymphatic drainage pathways, blood circulation pathways, and anatomical proximity relationships; Each directed edge has a pre-defined prior transition probability weight based on medical statistics, which is used to initialize the edge weights of the graph neural network.

3. The method according to claim 1, characterized in that, The mapping of image reports and chief complaint text to corresponding anatomical nodes specifically involves: Using a pre-trained medical named entity recognition model, anatomical site entities and lesion description entities are extracted from the image report and chief complaint text; Calculate the semantic similarity between the extracted anatomical site entities and the nodes in the knowledge graph; The feature vector of the lesion description entity is used as a node attribute and attached to the corresponding graph node with the highest semantic similarity.

4. The method according to claim 1, characterized in that, The method of calculating the semantic alignment weights between objective lesions and subjective symptoms at a node using the cross-attention mechanism is as follows: The subjective symptom feature vector of the anatomical node is used as the query vector, and the feature vector of the objective lesion is used as the key vector and value vector; Calculate the attention score of the dot product between the query vector and the key vector, and use it as the subjective-objective consistency coefficient; When the subjective-objective consistency coefficient is lower than a preset threshold and the magnitude of the subjective symptom feature vector is greater than a preset symptom intensity threshold, a high-weight differentiation mask is generated to amplify the feature weight of the corresponding node during fusion.

5. The method according to claim 1, characterized in that, The generation of the multimodal fusion node features also incorporates a time decay mechanism: For historical image reports or chief complaint texts, the corresponding feature vectors are weighted using an exponential decay function based on the time difference between their generation time and the current prediction time, so that recent clinical evidence has a higher weight in feature fusion.

6. The method according to claim 1, characterized in that, The continuous-time model adopts a neural ordinary differential equation model; By solving ordinary differential equations, the evolution trajectory of tumor markers sampled at non-equidistant intervals is fitted on a continuous time axis; The biochemical trend features include the current predicted value of the biomarker, as well as the first and second derivatives of the biomarker's change trajectory.

7. The method according to claim 1, characterized in that, The biochemical trend features are injected into the graph as global state variables, specifically: The biochemical trend features are used as a global context vector; The global context vector is modulated into the feature representation of each anatomical node in the graph neural network through an adaptive instance normalization layer or gating mechanism.

8. The method according to claim 1, characterized in that, The graph neural network adopts a graph attention network architecture; During feature aggregation, the model dynamically adjusts the attention coefficients of the directed edges between each anatomical node based on the global state variables to simulate the probability changes of tumor metastasis pathways under different biochemical backgrounds.

9. The method according to claim 1, characterized in that, After outputting the risk prediction results and high-risk anatomical areas, the method also includes: Generate a counterfactual explanation report; Specifically, by fine-tuning the tumor marker values ​​at the input end or removing specific chief symptom descriptions, the magnitude of change in risk prediction results is observed, and the contribution ranking of each modality data to the current risk prediction results is calculated and output.

10. A tumor recurrence and metastasis risk prediction system based on anatomical atlas, characterized in that, The system is used to implement the risk prediction method as described in any one of claims 1-9, including: The data acquisition module is used to acquire patient imaging reports, chief complaint texts, and non-equidistant tumor marker data; The semantic mapping and feature fusion module is used to map the image report and chief complaint text to the corresponding anatomical nodes based on the tumor metastasis anatomy knowledge graph, and run a cross-attention mechanism to calculate the semantic alignment weights of objective lesions and subjective symptoms on the nodes, thereby generating multimodal fusion node features. The biochemical trend evolution module is used to run a continuous-time model to extract the biochemical trend features of the biomarker data and inject them as global state variables into the knowledge graph. The graph neural network inference module is configured to input the multimodal fusion node features and global state variables into the graph neural network for feature aggregation and inference, and output risk prediction results and high-risk anatomical regions.

Images