System and method for identifying and segmenting aneurysms in different anatomical regions of different modalities

Abstract

The present invention belongs to the field of biomedical technology, and relates to medical image application systems and methods, specifically to a system and method for identifying and segmenting aneurysms in different anatomical regions of different modalities. The present invention includes a model training part or a model prediction part, which identifies or segments aneurysms by separately modeling the target vascular segment and the three-dimensional surface of the normal blood vessel and comparing them. The present invention can extract and retain common morphological features of all aneurysms, such as local or diffuse dilation and bulging of the vessel wall structure, making it possible to apply deep learning; and can significantly reduce the workload and difficulty of aneurysm data annotation, increase the quality and quantity of annotated data, and enhance the generalization ability of the deep learning solution.

Description

Technical Field

[0001] The present invention belongs to the field of biomedicine, and relates to medical image application systems and methods, and particularly relates to a system and method for identifying and segmenting aneurysms at different anatomical sites in different modalities. Background Art

[0002] In recent years, deep learning has received extensive attention and applications in the field of medical images, and has been increasingly used for the identification of aneurysms. However, the current deep learning solutions still have the following limitations: 1) The training models are limited to a single modality and a single site. For example, the identification of intracranial aneurysms based on MRA images [1], the identification of intracranial aneurysms based on CTA [2], the identification of abdominal aortic aneurysms based on CT [3], etc. This is mainly because the imaging data based on gray-scale information have too large differences in different anatomical sites and different modalities, resulting in a decline in the generalization ability of deep learning models [4]. 2) The amount of training data is small. For deep learning models based on medical images, the amount of data is generally in the hundreds [1-3]. The main reason is that the annotation of imaging data requires a large amount of time and effort, resulting in less high-quality annotated data, thus indirectly affecting the generalization ability of the model.

[0003] Based on the current situation of the existing technology, the inventors of the present application intend to provide a system and method for identifying and segmenting aneurysms at different anatomical sites in different modalities.

[0004] The prior art related to the present invention includes:

[0005] [1]Dai ju Ueda,Akira Yamamoto,Masataka Nishimoriet,et al.(2018).DeepLearning for MR Angiography:Automated Detection of CerebralAneurysms.Radiology.290.180901.

[0006] [2]Allison Park,Chris Chute,Pranav Rajpurkar,et,al.(2019).DeepLearning-Assisted Diagnosis of Cerebral Aneurysms Using the HeadXNetModel.JAMA network open.2.e195600.10.1001 / jamanetworkopen.2019.5600.

[0007] [3]Jen-Tang Lu,Rupert Brooks,Stefan Hahn,et,al.(2019).DeepAAA:clinically applicable and generalizable detection of abdominal aortic aneurysm using deep learning.10.1007 / 978-3-030-32245-8_80.

[0008] [4]John Zech,Marcus Badgeley,Manway Liu,et al.(2018).Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs:A cross-sectional study.PLOS Medicine.15.e1002683.10.1371 / journal.pmed.1002683.。 Summary of the Invention

[0009] The objective of the present invention is to provide a system and method for identifying and segmenting aneurysms at different anatomical sites in different modalities, based on the fact that aneurysms can occur in various parts of the human arterial system in the prior art. The present invention can extract and retain the common morphological features of all aneurysms (localized or diffuse dilation and bulging of the vessel wall structure), making it possible to apply deep learning; at the same time, it can significantly reduce the workload and difficulty of aneurysm data annotation, thereby increasing the quality and quantity of annotated data and enhancing the generalization ability of the deep learning solution.

[0010] Specifically, the present invention provides a system for identifying and segmenting aneurysms at different anatomical sites in different modalities, including a model training part or a model part, which identifies or segments aneurysms by separately modeling the target vascular segment and the three-dimensional surface of the normal blood vessel and comparing them.

[0011] The model training part includes the following modules:

[0012] Vessel extraction module, which extracts blood vessels by an adaptive threshold and region growing method;

[0013] Vessel three-dimensional surface model generation module, which uses the marching cubes algorithm to convert the binary image segmented above into a three-dimensional surface model and smooths it to remove noise and non-manifold surfaces;

[0014] Aneurysm annotation module, which performs annotation by using a three-dimensional surface model;

[0015] Aneurysm blood vessel segment extraction module, which selects the blood vessel range with a geodesic distance less than a certain threshold from the annotated aneurysm as the final blood vessel segment;

[0016] Normal blood vessel segment extraction module, which randomly selects the blood vessel range with a geodesic distance less than a certain threshold centered on the blood vessel intersection as the normal blood vessel segment;

[0017] Model training module, the three-dimensional surface model uses different convolutional structures to visually display the three-dimensional topological structure and morphology of the aneurysm.

[0018] Preferably, for the large arteries of the human body, blood vessel extraction is achieved through the following steps:

[0019] Blood vessel enhancement: Calculate the multi-scale Hessian eigenvalues of the image. Any one of the Frangi, Sato, and Jerman feature response functions can be specifically adopted;

[0020] Blood vessel centerline and radius estimation: Based on the above blood vessel enhanced image, perform Dijkstra shortest path search to obtain the centerline, and the radius is the scale value corresponding to the maximum feature response function value;

[0021] Blood vessel segmentation: Classify the foreground and background of the original image according to the centerline and radius, and then use graphcut for segmentation: The user can appropriately correct the final blood vessel segmentation result.

[0022] Preferably, compared with the method of layer-by-layer annotation based on the original image, each annotation of the aneurysm annotation module will only modify the aneurysm and blood vessel regions, and there is no coverage of the background region outside the blood vessels and aneurysms; and there is no need to repeatedly compare adjacent layers to confirm the range of the aneurysm during annotation.

[0023] Preferably, the normal blood vessel extraction module should ensure that the number of normal blood vessel segments selected is the same as that of aneurysm blood vessel segments to balance the positive and negative samples in the training dataset.

[0024] Preferably, the model prediction part includes:

[0025] Blood vessel extraction module, which extracts blood vessels through an adaptive threshold and region growing method;

[0026] Blood vessel three-dimensional surface model generation module, which uses the marching cubes algorithm to convert the above segmented binary image into a three-dimensional surface model and smooth it to remove noise and non-manifold surfaces;

[0027] A suspected aneurysm localization module that locates aneurysms in an automatic or semi-automatic manner;

[0028] A target vascular segment extraction module that selects a vascular range with a geodesic distance less than a certain threshold from the seed point as the target vascular segment;

[0029] An aneurysm segmentation module that loads the trained model and performs deep learning prediction on the extracted target vascular segment to obtain the segmentation result of the aneurysm;

[0030] An aneurysm display module that superimposes and displays the segmented aneurysm on the three-dimensional surface model of the blood vessels in the original image.

[0031] Among them, the automatic method is as follows:

[0032] Calculate the multi-scale Hessian eigenvalues for the vascular image region in the vascular extraction module, and extract the regions where all eigenvalues are negative and the corresponding function values of the features are higher than a certain threshold;

[0033] Perform connected component analysis to obtain multiple connected regions;

[0034] Take the centroid position of each connected region as the seed point.

[0035] The semi-automatic method refers to generating a three-dimensional vascular surface model based on the three-dimensional vascular surface model generation module, and manually selecting the suspected aneurysm location by the user and marking it with a seed point.

[0036] Correspondingly, the present invention provides a method for identifying and segmenting aneurysms. The method includes model training or model prediction;

[0037] The model training consists of the following steps:

[0038] Vascular extraction. For small arterial blood vessels in the human body such as carotid arteries and intracranial arteries, they can be extracted by the adaptive threshold and region growing method;

[0039] Generation of a three-dimensional vascular surface model. Use the marching cubes algorithm to convert the above-mentioned segmented binary image into a three-dimensional surface model and smooth it to remove noise and non-manifold surfaces;

[0040] Aneurysm annotation. By using the three-dimensional surface model for annotation, the user only needs a few strokes to complete the annotation work of a single aneurysm;

[0041] Extraction of aneurysm vascular segments. Select the vascular range with a geodesic distance less than a certain threshold from the annotated aneurysm as the final vascular segment;

[0042] Normal blood vessel segment extraction: Taking the blood vessel intersection as the center, randomly select a blood vessel range with a geodesic distance less than a certain threshold as the normal blood vessel segment; to ensure the balance of positive and negative samples in the training dataset, the number of selected normal blood vessel segments should be the same as that of aneurysm blood vessel segments.

[0043] Model training: The three-dimensional surface model can visually display the three-dimensional topological structure and morphology of the aneurysm, but there are significant differences from the three-dimensional grayscale image, and different convolutional structures need to be used.

[0044] The model prediction described above includes the following steps:

[0045] Blood vessel extraction;

[0046] Generation of the three-dimensional surface model of the blood vessel;

[0047] Suspected aneurysm localization: Locate the aneurysm in an automatic or semi-automatic manner.

[0048] Target blood vessel segment extraction: Select a blood vessel range with a geodesic distance less than a certain threshold from the seed point in the suspected aneurysm localization step as the target blood vessel segment.

[0049] Aneurysm segmentation: Load the trained model and perform deep learning prediction on the blood vessel segment extracted in the suspected aneurysm localization step to obtain the segmentation result of the aneurysm.

[0050] Aneurysm display: Superimpose and display the segmented aneurysm with the original image or the three-dimensional surface model of the blood vessel in the generation step of the three-dimensional surface model of the blood vessel.

[0051] The system and method of the present invention can be used to extract and retain the common morphological features of all aneurysms, making it possible to apply deep learning and even enhancing the generalization ability of the deep learning solution.

[0052] The advantages of the present invention are as follows:

[0053] 1) It can be applied to aneurysm recognition and segmentation tasks of multiple modalities and multiple anatomical sites. Such as carotid artery aneurysms or intracranial aneurysms based on CTA, MRA, 3D-DSA modalities, abdominal aortic aneurysms or thoracic aortic aneurysms based on CTA, MRA, etc.

[0054] 2) Data annotation is simpler and faster.

[0055] 3) The trained model has a small volume and is easier to converge. It can be deployed on mobile devices. Brief Description of the Drawings

[0056] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings required for the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0057] Figure 1 . Automatic aneurysm recognition process.

[0058] Figure 2 . Model prediction module.

[0059] Figure 3 . Example of rapid annotation of thoracic aortic aneurysm. The aneurysm can be annotated with only a few strokes. The annotation brush is restricted to the three-dimensional surface model area and does not affect the background area. The screenshot is from the open-source software MeshLab. Detailed implementation manners

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0061] The present invention is based on a deep learning solution and consists of two independent functional modules: training and prediction. Their flowcharts are respectively as Figure 1 and Figure 2 shown.

[0062] Part of the model training module in Embodiment 1

[0063] This part consists of the following steps:

[0064] 1) Vessel extraction. For small arterial vessels in the human body such as carotid arteries and intracranial arteries, they can be extracted by the adaptive threshold and region growing method. For large arterial vessels in the human body such as the abdominal aorta, it can be achieved through the following steps: a) Vessel enhancement. Calculate the multi-scale Hessian eigenvalue of the image. Any one of the Frangi, Sato, and Jerman feature response functions can be specifically adopted. b) Estimation of vessel centerline and radius. Based on the above vessel-enhanced image, perform Dijkstra shortest path search to obtain the centerline. The radius is the scale value corresponding to the maximum feature response function value. c) Vessel segmentation. Classify the foreground and background of the original image according to the centerline and radius, and then use graph cut for segmentation. The user can appropriately correct the final vessel segmentation result.

[0065] 2) Generation of three-dimensional vascular surface model. The marching cubes algorithm is used to convert the above segmented binary image into a three-dimensional surface model, and it is smoothed to remove noise and non-manifold surfaces.

[0066] 3) Aneurysm annotation. By using the three-dimensional surface model for annotation, the user only needs a few strokes to complete the annotation of a single aneurysm. Compared with the method of layer-by-layer annotation based on the original image, this method has obvious advantages: a) Each annotation only modifies the aneurysm and vascular regions, and there is no coverage of the background region outside the blood vessels and aneurysms; b) The aneurysm is more intuitively displayed, and there is no need to repeatedly compare adjacent layers to confirm the scope of the aneurysm during annotation. Figure 3 This is an example of using open-source tools to quickly annotate thoracic aortic aneurysms.

[0067] 4) Extraction of aneurysm vascular segments. The vascular range with a geodesic distance less than a certain threshold from the annotated aneurysm is selected as the final vascular segment.

[0068] 5) Extraction of normal vascular segments. With the vascular intersection as the center, a randomly selected vascular range with a geodesic distance less than a certain threshold is used as the normal vascular segment. To ensure the balance of positive and negative samples in the training dataset, the number of selected normal vascular segments should be the same as that of aneurysm vascular segments.

[0069] 6) Model training. The three-dimensional surface model can intuitively display the three-dimensional topological structure and morphology of the aneurysm, but there are significant differences from three-dimensional grayscale images, and different convolutional structures such as MeshCNN need to be used. Since the training set is derived from the extracted three-dimensional surface model, it effectively overcomes the defects of excessive data volume and insufficient GPU memory required for three-dimensional model training, thus avoiding alternative solutions that sacrifice model accuracy, such as downsampling the image, randomly dividing the three-dimensional image into blocks, or training layer by layer in 2D. Similarly, due to fewer model training parameters, the model volume is greatly compressed and can be controlled within 10M, which is beneficial for mobile deployment.

[0070] Example 2: Model prediction module part

[0071] This part consists of the following steps:

[0072] 1) Vascular extraction. The same as above.

[0073] 2) Generation of three-dimensional vascular surface model. The same as above.

[0074] 3) Suspected aneurysm localization. The aneurysm can be localized in an automatic or semi-automatic manner. The steps of the automatic method are as follows: a) Calculate the multi-scale Hessian eigenvalues for the vascular image region in step 1), and extract the regions where all eigenvalues are negative and the corresponding function values of the features are higher than a certain threshold; b) Perform connected component analysis to obtain multiple connected regions; c) Use the centroid positions of each connected region as seed points. The semi-automatic method is as follows: b) Based on the three-dimensional vascular surface model generated in step 2), the user manually selects the suspected aneurysm location and marks it with seed points.

[0075] 4) Target vascular segment extraction. Select the vascular range with a geodesic distance less than a certain threshold from the seed points in step 3) as the target vascular segment.

[0076] 5) Aneurysm segmentation. Load the trained model and perform deep learning prediction on the vascular segment extracted in step 4) to obtain the segmentation result of the aneurysm.

[0077] 6) Aneurysm display. Superimpose and display the segmented aneurysm on the original image or the three-dimensional vascular surface model in step 2).

[0078] As described above, it is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed in this application should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A system for identifying and segmenting aneurysms in different modalities and different anatomical regions, characterized in that, The described system includes a model training part or a model prediction part. By separately modeling the target blood vessel segment and the three-dimensional surface of the normal blood vessel and comparing them, aneurysms are identified or segmented; among which the model training part includes the following modules: A blood vessel extraction module that extracts blood vessels through an adaptive threshold and region growing method; A blood vessel three-dimensional surface model generation module that converts the segmented binary image into a three-dimensional surface model and smooths it to remove noise and non-manifold surfaces; An aneurysm annotation module that performs annotation by using the three-dimensional surface model; An aneurysm blood vessel segment extraction module that selects the blood vessel range with a geodesic distance less than a certain threshold from the annotated aneurysm as the final blood vessel segment; A normal blood vessel segment extraction module that randomly selects the blood vessel range with a geodesic distance less than a certain threshold centered on the blood vessel intersection point as the normal blood vessel segment; A model training module that uses different convolutional structures for the three-dimensional surface model to visually display the three-dimensional topological structure and morphology of the aneurysm; the model prediction part includes: A blood vessel extraction module that extracts blood vessels through an adaptive threshold and region growing method; A blood vessel three-dimensional surface model generation module that uses the marching cubes algorithm to convert the above-mentioned segmented binary image into a three-dimensional surface model and smooths it to remove noise and non-manifold surfaces; A suspected aneurysm localization module that locates aneurysms in an automatic or semi-automatic manner; A target blood vessel segment extraction module that selects the blood vessel range with a geodesic distance less than a certain threshold from the seed point as the target blood vessel segment; An aneurysm segmentation module that loads the trained model and performs deep learning prediction on the extracted target blood vessel segment to obtain the segmentation result of the aneurysm; An aneurysm display module that superimposes and displays the segmented aneurysm on the three-dimensional surface model of the blood vessel in the original image. In the described aneurysm annotation module, each annotation only modifies the aneurysm and blood vessel regions, and does not cover the background regions outside the blood vessels and aneurysms; and when annotating, it is not necessary to repeatedly compare adjacent layers to confirm the range of the aneurysm.

2. The system for identifying and segmenting aneurysms in different modalities and different anatomical regions according to claim 1, characterized in that, The system realizes blood vessel extraction from the large arteries of the human body through the following steps: a) Blood vessel enhancement: calculating the multi-scale Hessian eigenvalues of the image; b) Blood vessel centerline and radius estimation: based on the above blood vessel enhanced image, performing Dijkstra shortest path search to obtain the centerline, and the radius is the scale value corresponding to the maximum feature response function value; c) Blood vessel segmentation: classifying the foreground and background of the original image according to the centerline and radius, and then using graphcut for segmentation: the user can appropriately correct the final blood vessel segmentation result.

3. The system for identifying and segmenting aneurysms in different modalities and different anatomical regions according to claim 1, characterized in that, The selected normal blood vessel segments by the described normal blood vessel segment extraction module are consistent with the aneurysm blood vessel segments in quantity to ensure the balance of positive and negative samples in the training dataset.

4. The system for identifying and segmenting aneurysms in different modalities and different anatomical regions according to claim 1, characterized in that, The described automatic method is:

5. The system for identifying and segmenting aneurysms in different modalities and different anatomical regions according to claim 1, characterized in that, a) Calculate the multi-scale Hessian eigenvalues for the blood vessel image region in the blood vessel extraction module, and extract the regions where all the eigenvalues are negative and the corresponding function values of the features are higher than a certain threshold; b) Perform connected component analysis to obtain multiple connected regions; c) Use the centroid position of each connected region as the seed point. The described semi-automatic method is that based on the three-dimensional blood vessel surface model generated by the blood vessel three-dimensional surface model generation module, the user manually selects the suspected aneurysm position and marks it with a seed point.

6. The system for identifying and segmenting aneurysms of different modalities and different anatomical sites according to claim 1, wherein, The described method includes model training or model prediction; 7. A method for identifying and segmenting aneurysms of a system for identifying and segmenting aneurysms of different modalities and different anatomical sites based on a claim, wherein, The model training consists of the following steps: A) Blood vessel extraction. For small arterial blood vessels in the human body such as the carotid artery and intracranial artery, extract them through an adaptive threshold and region growing method; ​ B) Generation of three-dimensional vascular surface model: The marching cubes algorithm is used to convert the segmented binary image into a three-dimensional surface model, and it is smoothed to remove noise and non-manifold surfaces; C) Aneurysm annotation: By using the three-dimensional surface model for annotation, the user only needs to draw a few strokes to complete the annotation of a single aneurysm; D) Extraction of aneurysm vascular segments: The vascular range with a geodesic distance less than a certain threshold from the annotated aneurysm is selected as the final vascular segment; E) Extraction of normal vascular segments: Centered on the vascular intersection point, a vascular range with a geodesic distance less than a certain threshold is randomly selected as the normal vascular segment; The number of selected normal vascular segments is the same as that of aneurysm vascular segments to ensure the balance of positive and negative samples in the training dataset; F) Model training: The three-dimensional surface model visually displays the three-dimensional topological structure and morphology of the aneurysm, but there are significant differences from the three-dimensional grayscale image, and different convolutional structures need to be used; The model prediction includes the following steps: 1) Vascular extraction; 2) Generation of three-dimensional vascular surface model; 3) Suspected aneurysm localization: Locate the aneurysm automatically or semi-automatically; 4) Extraction of target vascular segments: The vascular range with a geodesic distance less than a certain threshold from the seed point in step 3) is selected as the target vascular segment; 5) Aneurysm segmentation: Load the trained model and perform deep learning prediction on the vascular segments extracted in step 4) to obtain the segmentation result of the aneurysm; 6) Aneurysm display: Superimpose the segmented aneurysm on the original image or the three-dimensional vascular surface model in step 2).

Images