A method and system for assessing activity of liver hydatid lesions using multi-modal imaging
By constructing a graph structure from multimodal image data and generating a mechanistic risk field, and combining it with a causal Bayesian network to assess the activity of hepatic echinococcosis lesions, the problems of subjectivity, insufficient information utilization, and lack of interpretability in existing technologies are solved, achieving highly accurate and reliable assessment of lesion activity.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- XINJIANG MEDICAL UNIV
- Filing Date
- 2026-02-25
- Publication Date
- 2026-05-29
AI Technical Summary
Existing technologies for assessing the activity of hepatic echinococcosis lesions suffer from problems such as strong subjectivity, poor consistency, insufficient information utilization, lack of dynamic evolution analysis, uninterpretable AI models, and lack of mechanistic modeling.
A graph structure was constructed using multimodal image data, a mechanistic risk field was generated using a graph neural network, and a causal Bayesian network was combined to assess lesion activity. The interaction between lesions and vascular structures was learned through the graph structure, and a causal relationship network was constructed for interpretability assessment.
It improves the accuracy and interpretability of lesion activity assessment, enhances clinical trust, and significantly improves the consistency and dynamic analysis capabilities of the assessment.
Smart Images
Figure CN122116069A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to a method and system for evaluating the activity of hepatic echinococcosis lesions using multimodal imaging. Background Technology
[0002] Hepatic echinococcosis is a zoonotic parasitic disease caused by the larvae of *Echinococcus granulosus* parasitizing the liver, with high rates of disability and mortality. According to the World Health Organization (WHO) classification, the evolution of hepatic echinococcosis lesions can be divided into different stages, which are directly related to the biological activity of the lesions. Accurate assessment of lesion activity is crucial for developing clinical treatment plans (such as choosing surgery, drug therapy, or observation). For example, active lesions usually require aggressive intervention, while inactive calcified lesions can be treated conservatively.
[0003] Currently, the clinical assessment of the activity of hepatic echinococcosis lesions mainly relies on imaging examinations, such as ultrasound, computed tomography (CT), and magnetic resonance imaging (MRI). Physicians make subjective judgments based on WHO classification criteria, combined with characteristics such as lesion size, shape, wall thickness, cystic fluid density, presence of daughter cysts, or calcification. However, this approach has the following significant problems:
[0004] 1. High subjectivity and poor consistency: Doctors with different seniority may have different judgments on the activity of the same lesion, especially for transitional lesions between active and inactive, which are more difficult to interpret, resulting in low consistency among observers;
[0005] 2. Insufficient information utilization: Traditional imaging assessment relies on a few macroscopic features, making it difficult to fully utilize the rich texture and functional information contained in the imaging data (such as cell density information reflected by MRI diffusion-weighted imaging DWI).
[0006] 3. Lack of dynamic evolution analysis: A single image is like a snapshot, making it difficult to reveal the dynamic trend of lesion evolution over time. Although multiple follow-up images can be compared, manual comparison is inefficient and difficult to quantify.
[0007] In recent years, artificial intelligence (AI), especially deep learning technology, has achieved great success in the field of medical image analysis. Studies have attempted to use models such as convolutional neural networks (CNNs) for automatic typing and activity assessment of hepatic echinococcosis. These models, by learning from large amounts of labeled data, can achieve high classification accuracy. However, these black-box models based on deep learning also bring new challenges:
[0008] 1. Unexplainable Process: The model's decision-making process from input to output is like a black box, making it impossible to explain the basis of its judgments to doctors. Doctors do not know which imaging features the model uses to arrive at its active conclusions, which seriously affects its credibility and widespread application in critical clinical decisions.
[0009] 2. Lack of mechanistic modeling: Most existing AI models make predictions based on data correlations and do not model the pathophysiological processes such as lesion invasion and growth. For example, lesions tend to invade along intrahepatic blood vessels, bile ducts, and other ductal systems; this crucial anatomical prior knowledge is difficult to effectively utilize in traditional CNN models. The models are unaware that risk is spatially anisotropic, meaning that the risk along certain directions is much higher than in others. Summary of the Invention
[0010] The present invention aims to at least partially solve the aforementioned technical problems. Therefore, the objective of the present invention is to provide a method and system for assessing the activity of hepatic echinococcosis lesions using multimodal imaging.
[0011] To achieve the above objectives, the present invention provides a method for evaluating the activity of hepatic echinococcosis lesions using multimodal imaging, characterized in that the method comprises:
[0012] a) Acquire multimodal imaging data of the target lesion and its surrounding vascular structures at the first and second time points;
[0013] b) Construct a graph structure based on the image data, wherein the nodes of the graph structure include lesion voxel nodes and vascular structure nodes, and the edges of the graph structure represent the spatial proximity relationship between nodes;
[0014] c) Input the graph structure into a graph neural network (GNN) to learn the interaction relationship between the lesion voxel nodes and the vascular structure nodes, and generate a mechanistic risk field, wherein the risk field assigns risk vectors to the voxels within the lesion.
[0015] d) Input the multimodal image data and the mechanistic risk field into a feature extraction network to extract image features at the first time point and the second time point, wherein the risk vector is used to guide the feature extraction network to perform adaptive feature learning;
[0016] e) Construct a causal Bayesian network based on the image features to describe the causal relationship between different imaging changes, and output the predicted probability of the activity state of the lesion based on the causal Bayesian network.
[0017] The core technical solution of this invention constructs a precise and interpretable evaluation process through two deeply integrated and synergistic innovations.
[0018] First, a mechanistic risk field is generated based on graph neural networks. This invention no longer treats lesions as isolated sets of pixels, but rather constructs a graph with surrounding key anatomical structures such as blood vessels and bile ducts. A powerful graph neural network is used to learn the complex interactions between lesion nodes and vascular nodes, thereby generating a mechanistic anisotropic risk field. This risk field not only indicates the magnitude of the risk, but more importantly, it explicitly provides a risk vector indicating the most likely vascular direction along which the lesion will invade. This risk field serves as strong prior knowledge to guide the attention or convolutional kernels of subsequent feature extraction networks, ensuring that they focus from the outset on the most pathologically significant invasion interface.
[0019] Second, the activity status assessment is based on a causal Bayesian network. This invention changes the black-box model of traditional end-to-end classification. After extracting high-quality features guided by risk, this invention does not directly perform classification, but instead constructs a causal Bayesian network (CBN) through a causal discovery process. This network reveals, in the form of a visual graph, the causal transmission path and probability between different dimensions of radiological changes in lesions (such as morphological changes and functional changes) and the final activity status. The final activity assessment result is derived from probabilistic inference based on this causal network with inherent logic and interpretability, rather than from an uninterpretable neural network output.
[0020] The present invention also provides a system for assessing the activity of hepatic echinococcosis lesions using multimodal imaging, and a computer-readable storage medium for implementing the above-described method. The system includes: one or more processors; and a memory storing instructions executable by the one or more processors, wherein, when executed, the instructions cause the system to perform the above-described method.
[0021] The beneficial effects of this invention are as follows:
[0022] The GNN risk field of this invention provides strong priors about the invasion path, enabling the model to capture key features that are easily overlooked by traditional methods (such as early interaction changes between lesions and blood vessel walls), thereby significantly improving the accuracy of activity assessment.
[0023] Causal Bayesian networks deconstruct the process of determining activity into a clear causal graph that conforms to pathological logic. Doctors can intuitively understand which imaging changes the model is based on to draw conclusions (e.g., because the functional indicators deteriorate, leading to irregular morphological growth, it is judged to be active), which greatly enhances clinical trust.
[0024] The innovative combination of GNN risk field and causal network provides a brand-new technical paradigm for solving the problem of black box evaluation of medical images, with strong technical barriers and broad application prospects. Attached Figure Description
[0025] Figure 1 This is a system functional structure block diagram according to an embodiment of the present invention.
[0026] Figure 2 This is a flowchart of a method according to an embodiment of the present invention. Detailed Implementation
[0027] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0028] It should be understood that, and also noted, in the embodiments, the functions / actions may appear in a different order than those shown in the figures. For example, depending on the functions / actions involved, they may actually be performed substantially concurrently, or sometimes the two figures shown consecutively may be performed in reverse order.
[0029] like Figure 1 As shown, this invention discloses a system for assessing the activity of hepatic echinococcosis lesions using multimodal imaging. This system can be implemented on a server, workstation, or embedded device. The system includes a processor and a memory. The memory stores a computer program executable by the processor. When the processor executes the program, it implements the method of this invention. Logically, the system can be divided into multiple functional modules, including: a risk perception module for generating a mechanistic risk field, a feature extraction module for extracting deep features, and a causal assessment module for performing interpretability evaluation. The system acquires multimodal time-series image data from the data layer, processes it, and outputs the predicted probability of the activity state on a display terminal.
[0030] like Figure 2 As shown, the method flow of the present invention will be described in detail.
[0031] Step S101: Acquire multimodal time-series image data.
[0032] First, the system acquires liver imaging data from the patient at two different time points (T1 and T2). These two time points are typically the baseline examination and follow-up examination in clinical practice, with an interval of several months or years.
[0033] Multimodal images refer to images derived from different imaging principles or different scanning sequences. In a preferred embodiment, multimodal image data may include:
[0034] CT imaging includes plain CT scans and contrast-enhanced CT scans in the arterial, portal venous, and delayed phases. CT imaging can clearly show the contour, density, calcification, and relationship of lesions to major blood vessels.
[0035] MRI images include T1-weighted imaging (T1WI), T2-weighted imaging (T2WI), and functional imaging sequences, especially diffusion-weighted imaging (DWI) and its derived apparent diffusion coefficient (ADC) map. The ADC map reflects the degree of restriction of water molecule diffusion within tissues and is negatively correlated with cell density, making it an important functional indicator for assessing the biological activity of lesions.
[0036] Step S102: Construct the lesion-vascular map structure.
[0037] To enable the model to understand the relationship between the lesion and surrounding anatomical structures, this step converts the imaging data into graphical structural data. Specifically, this includes:
[0038] 1. Segmentation: The images at T1 and T2 time points are processed using a pre-trained segmentation network (such as U-Net) to automatically segment the three-dimensional target lesion area and the intrahepatic vascular system (including portal vein, hepatic vein and bile duct) within a certain range around the lesion.
[0039] 2. Node Definition: The segmented 3D lesion region is discretized into a series of voxels, and each or each group of lesion voxels constitutes a lesion voxel node. The segmented vascular system is extracted as a skeleton, and each key branch segment or key point constitutes a vascular structure node.
[0040] 3. Edge definition: Establish connection relationships (edges) between nodes. If the Euclidean distance between a lesion voxel node and a vascular structure node is less than a preset threshold (e.g., 5mm), then an edge is established between the two nodes, which represents the spatial proximity relationship between the nodes.
[0041] 4. Node Feature Configuration: Each node is assigned an initial feature vector. These features can be local image intensity, texture features, or functional map values extracted from multimodal image data. For example, for a lesion voxel node, its feature vector may include the voxel's CT value on a plain CT scan, its ADC value on an ADC map, and texture features extracted through a local Gabor filter. For a vascular structure node, its features may include geometric information such as its diameter and direction of travel.
[0042] Step S103: GNN generates a mechanistic risk field.
[0043] The graph structure with node features constructed in the previous step is then input into a graph neural network (GNN). GNNs (such as graph attention networks (GAT) or graph convolutional networks (GCN)) are able to learn the complex interactions between nodes in the graph and their neighboring nodes through their unique message passing mechanisms.
[0044] In this invention, a GNN is trained to predict the invasive potential of lesion nodes toward adjacent vascular nodes. The training supervision signal can come from a large dataset of cases with known invasions, where known invasion paths are used as ground truth.
[0045] The output of a GNN is a mechanistic risk field. Specifically, for each voxel within the lesion, the GNN outputs a risk vector. This risk vector consists of two parts:
[0046] Risk level (scalar): A value between 0 and 1, representing the probability of the voxel undergoing malignant evolution or invasion.
[0047] Risk direction (3D unit vector): A 3D vector representing the predicted highest risk direction of invasion from the lesion to the vascular structure.
[0048] This risk field is mechanistic because it is not based on guesswork, but on learning from the structured interaction between lesions and blood vessels.
[0049] Step S104: Perform risk-guided feature extraction.
[0050] The goal of this step is to extract depth features from the original image that can effectively characterize the changes in lesions over the T1 to T2 time period. Unlike traditional methods, the feature extraction process of this invention is guided by the risk field generated in the previous step.
[0051] The feature extraction network used can be a CNN-based or Transformer-based encoder. Methods for guiding the feature extraction network to perform adaptive feature learning using risk vectors include, but are not limited to:
[0052] Attention weight modulation: In the Transformer's self-attention module, the risk level of each voxel is used as a modulating factor, multiplied by its corresponding attention score. This way, the model gives higher attention to high-risk regions when calculating features.
[0053] Dynamic kernel adjustment: For CNNs, the shape or weights of convolutional kernels can be dynamically adjusted using the risk direction vector. For example, the kernel can be stretched along the risk direction, making it more sensitive to edge and texture changes in that direction.
[0054] In this way, the feature extraction network can adaptively learn, focusing on capturing imaging changes that occur in high-risk areas and along high-risk directions. Finally, the network outputs high-dimensional image feature vectors F at two time points, T1 and T2. T1 and F T2 .
[0055] Step S105: Construct a causal Bayesian network (CBN).
[0056] This is a core step in achieving interpretability assessment. Specifically, it includes:
[0057] 1. Feature difference decomposition: First, calculate the feature difference ΔF = F T2 -F T1 Then, this abstract, high-dimensional feature difference vector ΔF is decomposed into a set of human-understandable, low-dimensional semantic change vectors using one or more decoders. These predefined semantic change categories include at least two of morphological changes, texture or density changes, and functional changes. In a specific embodiment, these change categories can be further refined: morphological changes can be specifically changes in lesion volume or sphericity. Texture or density changes can be specifically changes in the homogeneity or CT value within the lesion. Functional changes can be specifically changes in the apparent diffusion coefficient value in diffusion-weighted imaging.
[0058] 2. Causal Discovery: Collect a large set of semantic change vectors from hundreds or thousands of patients. Then, use causal discovery algorithms (such as PC, FCI, or gradient-based Notears algorithms) to learn the conditional probabilistic dependencies between categories, thereby constructing a causal Bayesian network. This algorithm outputs a directed acyclic graph (DAG), i.e., a causal Bayesian network (CBN).
[0059] In this network, nodes represent semantic change categories and a target active state node, while directed edges represent causal relationships between nodes. For example, there might be a directed edge from a decrease in ADC to an increase in volume, indicating that functional deterioration (increased cell density) is the cause of morphological growth. Each edge is accompanied by a conditional probability table, quantifying the strength of this causal effect.
[0060] Step S106: Infer and output the predicted probability of the active state based on CBN.
[0061] For a new lesion to be evaluated, the system first completes the first step from S101 to S105, and calculates its corresponding semantic change vector (e.g., the volume increased by 15% and the ADC value decreased by 20%).
[0062] These observed changes are then used as evidence and fed into the pre-constructed CBN. The posterior probability distribution of the target active state nodes is calculated using standard Bayesian inference algorithms (such as variable elimination or belief propagation).
[0063] Finally, the system outputs the predicted probability of the lesion being in each activity state. The activity state of the lesion can include: active, transitional, and inactive. For example, the output result is: {active: 0.92, transitional: 0.07, inactive: 0.01}. Simultaneously, the system can also visualize the causal path leading to this result (e.g., a significant decrease in ADC value is the strongest cause of the active lesion judgment) on the display terminal, thus providing doctors with decision-making explanations.
[0064] This invention generates a mechanistic risk field through GNN and uses it to guide feature extraction. It then combines this with a causal Bayesian network for interpretability assessment, successfully solving many problems in the prior art and providing a complete and innovative solution for the accurate and reliable assessment of the activity of hepatic echinococcosis lesions.
[0065] This invention is not limited to the above-described optional embodiments. Anyone can derive other various forms of products under the guidance of this invention. However, regardless of any changes made in their shape or structure, any technical solution that falls within the scope of the claims of this invention shall be protected by this invention.
Claims
1. A method for assessing the activity of hepatic echinococcosis lesions using multimodal imaging, characterized in that, The method includes: a) Acquire multimodal imaging data of the target lesion and its surrounding vascular structures at the first and second time points; b) Construct a graph structure based on the image data, wherein the nodes of the graph structure include lesion voxel nodes and vascular structure nodes, and the edges of the graph structure represent the spatial proximity relationship between nodes; c) Input the graph structure into a graph neural network to learn the interaction relationship between the lesion voxel nodes and the vascular structure nodes, and generate a mechanistic risk field, wherein the risk field assigns risk vectors to the voxels in the lesion. d) Input the multimodal image data and the mechanistic risk field into a feature extraction network to extract image features at the first time point and the second time point, wherein the risk vector is used to guide the feature extraction network to perform adaptive feature learning; e) Construct a causal Bayesian network based on the image features to describe the causal relationship between different imaging changes, and output the predicted probability of the activity state of the lesion based on the causal Bayesian network.
2. The method according to claim 1, characterized in that, In step c), the risk vector contains risk magnitude and risk direction information, and the risk direction represents the predicted highest risk invasion direction from the lesion to the vascular structure.
3. The method according to claim 1, characterized in that, In step b), node features are configured for the lesion voxel nodes and vascular structure nodes, and the node features include local image intensity, texture features or functional map values extracted from the multimodal image data.
4. The method according to claim 1, characterized in that, In step d), the risk vector guides the feature extraction network to perform adaptive feature learning by modulating the attention weights in the feature extraction network or dynamically adjusting the convolutional kernels using the risk vector.
5. The method according to claim 1, characterized in that, In step e), constructing the causal Bayesian network includes: e1) Decompose the differences in image features between the first time point and the second time point into semantic change vectors of multiple preset categories; e2) Based on the set of semantic change vectors extracted from multiple patient data, a causal discovery algorithm is used to learn the conditional probabilistic dependencies between the categories in order to construct the causal Bayesian network.
6. The method according to claim 5, characterized in that, The preset semantic change categories include at least two of the following: morphological changes, texture or density changes, and functional changes.
7. The method according to claim 6, characterized in that, The morphological changes include changes in lesion volume or sphericity; the texture or density changes include changes in the homogeneity or CT value within the lesion; and the functional changes include changes in the apparent diffusion coefficient value in diffusion-weighted imaging.
8. The method according to claim 5, characterized in that, The nodes of the causal Bayesian network are the semantic change categories and a target active state node, and the directed edges of the network represent the causal relationships between the nodes.
9. A system for assessing the activity of hepatic echinococcosis lesions, characterized in that, The system includes: One or more processors; A memory storing instructions executable by the one or more processors, wherein when executed, the instructions cause the system to perform the method as described in any one of claims 1 to 8.
10. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1 to 8.