Automated 3D vascular modeling method for use in medical field

Abstract

The invention relates to an automatic three-dimensional vascular modeling method for use in the field of cardiology and cardiovascular surgical technique. The automated three-dimensional vascular modeling method of the invention is used in the diagnosis of cardiovascular diseases, pre-surgical planning for cardiovascular surgeries, and 3D reconstruction of vascular grafts for patient-specific treatments.

Description

[0001] AUTOMATED 3D VASCULAR MODELING METHOD FOR USE IN MEDICAL FIELD

[0002] Technical Field of the Invention

[0003] The invention relates to an automatic three-dimensional vascular modeling method for use in the field of cardiology and cardiovascular surgical technique. The automated three-dimensional vascular modeling method of the invention is used in the diagnosis of cardiovascular diseases, pre-surgical planning for cardiovascular surgeries, and 3D reconstruction of vascular grafts for patient-specific treatments. Said method is based on multilayer medical imaging.

[0004] State of the Art

[0005] The vascular system is a structure containing blood vessels (arteries, veins, and capillaries) that circulates blood throughout the body, and said system transports oxygen, nutrients, hormones, and other important substances to different parts of the body, enabling cells to function. The vascular system transports oxygen and nutrients from the lungs and organs such as the digestive system to all parts of the body, maintains the structure of the vessels and regulates blood pressure, transports white blood cells and antibodies to different parts of the body against infections, and ensures the continuity of blood circulation. Therefore, a healthy vascular system is vital for maintaining the overall health of the body. However, vascular diseases (e.g. arterial occlusion, aneurysm, venous insufficiency) can affect the vascular system and, by extension, lead to serious health problems. For this reason, cardiac imaging is crucial to identify biomarkers associated with the risk, progression, and response to treatment of heart diseases. There are many imaging methods used to diagnose cardiovascular diseases, including Magnetic Resonance Imaging (MRI), Digital Subtraction Angiography (DSA), Computed Tomography Angiography (CTA), and XRay Angiography (XRA). However, XRA provides 2-dimensional (2D) images to monitor blockages, aneurysms, narrowings, malformations, and other blood vessel problems in vessels, arteries, and organs on screens, which does not provide anything close to a real experience. The X-ray imaging procedure is not suitable to be repeated more than once as it requires surgical intervention and injection of contrast (radioactive) material directly into the arteries [2], In addition, said imaging techniques often produce outputs of low quality and resolution. Therefore, during patient assessment and treatment, when surgical planning is required, vascular modeling systems are widely preferred for training and simulation purposes, since in some complicated operations, it is important for the surgeon to use a model with a similar vascular structure that resembles the patient’s cardiovascular system to simulate the surgery beforehand. This helps the surgeon to be better prepared for problems that may arise during the operation, or vascular modeling systems help to create customized treatment plans based on the vascular anatomy of individual patients. Especially in complex vascular pathologies such as aneurysms, the use of these systems improves treatment outcomes. Furthermore, vascular modeling systems are used as training tools for medical students and surgical teams. Virtual surgical simulations help surgeons improve their skills and learn new surgical techniques. In addition, vascular modeling systems are used in the design and development of vascular devices such as stents, angioplasty balloons, and embolization materials. These systems allow devices to be tested in virtual environments to assess their effectiveness and safety. However, 3D modeling reconstruction of cardiovascular structures used in the present art is largely performed manually and dependent on the expertise of 3D modeling experts. This approach uses various medical image outputs, such as MRI or CT scans from the imaging techniques described above, and these outputs are interpreted by 3D artists or medical professionals to create 3D models. Furthermore, since the creation of models in this way is highly dependent on the human and the experience of the individual modeler, errors related to interpretation or level of expertise are often encountered. In addition, it should be noted that in the present art, hospital information systems often restrict access to raw data since these data require large storage space and are tightly managed due to security and privacy concerns. For this reason, medical images are usually stored in more compact formats (PDF, JPEG, PNG). If this is the case, it becomes difficult for the modeling expert to access image outputs such as MRI, CT or XRA, which require large storage space.

[0006] Focusing on cardiovascular imaging modalities, the applications of artificial intelligence (Al) in this field range from image acquisition to image analysis and ultimately assessment and prognosis [3]. Furthermore, deep learning, a class of artificial intelligence, plays a central and important role in the field of imaging, focusing on the ability to identify relevant patterns in images, such as edges, color gradients, and shapes by performing convolution between the 'convolutional layer', the image, and a set of specialized filters. The pattern learned by the network is then used to face two types of analysis: classification and segmentation. Classification aims to distinguish between two or more classes of patients, while segmentation aims to detect specific structures or objects by labeling at the pixel level [4], The patent application with the application no. CN116385667A in the state of the art relates to the field of medical image data processing, and in particular to a method, system, and computer storage medium for three-dimensional soft tissue reconstruction based on image segmentation. Herein, medical imaging data are multi-modal medical imaging data of soft tissue, and these consist of one or more of computed tomography, magnetic resonance imaging, and positron emission tomography. It is stated that the reconstructions obtained through this system are of high resolution and sensitivity. Furthermore, the system mentioned employs contrast stretching technology to enhance image details and median filtering technology to remove isolation in image noise spots, which are combined with Gaussian filtering to achieve image smoothing. However, there is no categorization of vessels here, the system described in the document mentioned only works with point cloud data.

[0007] In cardiovascular diseases, some irregularities in vessel wall morphology are of significant diagnostic value and need to be preserved during imaging analysis and consequently in 3D modeling methods. In particular, at the onset of coronary artery disease, atherosclerosis is characterized by such irregularities. The onset and progression of this condition is marked by noticeable changes in arterial wall structure that are critical for early diagnosis and management. Furthermore, pathological conditions such as dissections or aneurysms present areas where the vessel wall is noticeably thin. In these regions, inappropriate application of smoothing algorithms can inadvertently increase the apparent wall thickness and potentially obscure critical pathologic features. Therefore, it is imperative to apply computational modeling and image processing techniques in a sensible and ensure that clinically important diagnostic features are accurately preserved in the reconstructed images.

[0008] Another patent application with the application no. CN110570416A in the state of the art relates to a method for visualization, 3D printing, and mixed reality display of multimodal heart images. Herein, segmentation of cardiac tissue structure based on a multimodal heart image involves image segmentation of the multimodal heart image using a two-stage U-Net framework. The method described in said document enables fully automatic analysis, visualization, and 3D printing of heart images.

[0009] Due to the limitations and shortcomings of the solutions in the present art, the low resolution of cardiovascular imaging techniques, the high storage space requirement and difficult accessibility of images such as MR and CT, the human dependency of modeling systems and the resulting decrease in accuracy or errors, and the lack of categorization in modeling methods, an improvement in 3D modeling methods has become necessary.

[0010] Summary and Objects of the Invention

[0011] The invention describes an automatic three-dimensional vascular modeling method for use in the field of cardiology and cardiovascular surgical technique. The automated three-dimensional vascular modeling method of the invention is used in the diagnosis of cardiovascular diseases, pre-surgical planning for cardiovascular surgeries, and 3D reconstruction of vascular grafts for patient-specific treatments. Said method is based on multilayer medical imaging.

[0012] The object of the invention is to provide a 3D modeling method for use in the cardiovascular field. The 3D models obtained automatically by the method of the invention are used in computational fluid dynamics (CFD) for simulation, surgical planning and the design of patient-specific grafts, thus expanding the scope of application in cardiology and cardiovascular surgery and improving patient outcomes.

[0013] An object of the invention is to provide a 3D model that is close to the real patient vascular structure with high accuracy, in which structural details are preserved, and can be optimized for high-resolution, personalized simulations. Unlike traditional manual approaches, the method of the invention integrates a number of smoothing algorithms such as Butterworth Filter, Chebyshev Filter, Kalman Filter, Kernel Smoother, Laplacian Smoothing and Local Regression, which improve the accuracy and resolution of 3D models.

[0014] Another object of the invention is to enable the production of cardiovascular 3D models without the need for direct use of raw image data. In the present art, hospital information systems often restrict access to raw data since these data require large storage space and are tightly managed due to security and privacy concerns. For this reason, medical images are usually stored in more compact formats (PDF, JPEG, PNG). However, the method of the invention is able to process medical images in various formats (PDF, X-rays, JPEG, Output), overcoming the limitations of access to raw data. It also provides efficiency in data archiving and management by processing data in more compact formats instead of raw data that requires large storage space.

[0015] Another object of the invention is to reduce errors in medical imaging. In the present art, the reconstruction of 3D models of cardiovascular structures is largely performed manually and dependent on the expertise of 3D modeling experts. This approach uses various medical image outputs, such as MRI or CT scans, and these outputs are interpreted by 3D artists or medical professionals to create 3D models, however, errors are also reduced as the method of the invention eliminates the need for humans.

[0016] Another object of the invention is to provide a process of automation of the 3D model reconstruction process by developing a program that supports different types of inputs and can produce various types of outputs. This process is completely unsupervised and the user can only check the quality of the results and cannot intervene to change the process. By automatically processing medical imaging data, the program is able to reconstruct accurate and detailed 3D vascular models without the need for human intervention.

[0017] Brief Description of the Figures

[0018] Fig. 1. Flowchart of the 3D modeling method of the invention ( • : Start : Decision, E: Yes, H: No)

[0019] Description of the References in the Drawings

[0020] 1000. Checking the input

[0021] 1001. Determining whether the input is printed and obsolete

[0022] 1001.1 If the input is printed and obsolete, scanning radiographs with an X-ray film scanner

[0023] 1001.2. Scanning reports on white paper with a high resolution scanner (1200 dpi) 1001.3 Performing automatic enhancements (contrast, brightness, entropy, skewness, kurtosis) on the scanned images

[0024] 1001.4 If the input is not printed and obsolete, direct processing the digital input

[0025] 2000. Checking whether there is a 3D scan present, if there is a 3D scan present, respectively following the process steps 3001 -3004 and 4001 -4004 simultaneously

[0026] 2001. If there is no 3D scan present, calculating the approximate value of the third dimension for the 3D image using a dummy

[0027] 3000. Simultaneously determining whether multilayer images are used for processing multilayer images.

[0028] 3001. If multilayer images are used for processing multilayer images, selecting the function according to the specified vessel by the user

[0029] 3002. Creating layer contours by applying selected smoothing functions.

[0030] 3003. Selecting the correct smoothing function for lofting between contours of different vascular layers.

[0031] 3004. Lofting between contours of different vascular layers and creating a 3D model

[0032] 4000. Determining whether multi-angle images are used for processing simultaneous multi-angle images based on image characteristics

[0033] 4001. If multi-angle images are used, selecting the function according to the specified vessel by the user

[0034] 4002. Creating vessel contours on images by applying the selected smoothing function.

[0035] 4003. Extruding the contour of each angle and preserving their intersections

[0036] 4004. Applying the Elliptic function to smooth right angles

[0037] 5000. Checking for small objects, disconnection, and non-merging edges

[0038] 5001. Performing 3D model finite element analysis (FEA) and computational fluid dynamics (CFD) analysis

[0039] Detailed Description of the Invention

[0040] The invention relates to an automatic three-dimensional vascular modeling method for use in the field of cardiology and cardiovascular surgical technique. The automated three-dimensional vascular modeling method of the invention is used in the diagnosis of cardiovascular diseases, pre-surgical planning for cardiovascular surgeries, and 3D reconstruction of vascular grafts for patient-specific treatments. Said method is based on multilayer medical imaging, and due to the integration of multiple smoothing algorithms, vascular structures can be modeled more closely to reality. The 3D modeling method of the invention comprises the process steps of: i. checking the input (1000), ii. determining whether the input is printed and obsolete (1001 ) iii. if the input is printed and obsolete, scanning radiographs with an X-ray film scanner (1001.1), scanning reports on white paper with a high resolution scanner (1200 dpi) (1001.2), and performing automatic enhancements (contrast, brightness, entropy, skewness, kurtosis) on the scanned images (1001.3), or, if the input is not printed and obsolete, direct processing the digital input (1001.4), iv. checking whether there is a 3D scan present, if there is a 3D scan present, respectively following the process steps 3001 -3004 and 4001 -4004 simultaneously (2000), or, if there is no 3D scan present, calculating the approximate value of the third dimension for the 3D image using a dummy (2001), v. determining whether multilayer images are used for processing multilayer images based on image characteristics (3000), vi. if multilayer images are used for processing multilayer images, selecting the function according to the specified vessel by the user (3001), or, if multilayer images are not used, following process no. 5000 by the system, vii. creating layer contours by applying selected smoothing functions. (3002), viii. selecting the correct smoothing function for lofting between contours of different vascular layers (3003), ix. lofting between contours of different vascular layers and creating a 3D model (3004), x. Determining whether multi-angle images are used for processing simultaneous multi-angle images based on image characteristics (4000), xi. if multi-angle images are used, selecting the function according to the specified vessel by the user (4001 ), or, if multi-angle images are not used, following process no. 5000 by the system xii. if multi-angle images are used, creating vessel contours by applying the selected smoothing function for different angles. (4002), xiii. extruding the contour of each angle and preserving their intersections (4003), xiv. applying the Elliptic function to smooth right angles (4004), xv. checking for small objects, disconnection, and non-merging edges (5000). In the method of the invention, preferably, following the process step (xv), 3D model finite element analysis (FEA) and computational fluid dynamics (CFD) analyzes are performed (5001 ). Furthermore, in an embodiment of the invention, preferably in process step (vi), if multilayer images are not used, the system follows process steps 5000-5001 respectively. In addition, in another embodiment of the invention, preferably, if multi-angle images are not used in process step no. (xi), the system follows process steps 5000-5001 respectively.

[0041] The smoothing algorithms used in the invention are mainly effective on two main input types: multi-segment and multi-angle image data: In the case of smoothing for multilayer images, algorithms are used to smooth the contours generated by the coordinates of each layer. These contours are smoothed to allow for smooth transitions between layers as they are joined. The softening process contributes to the smooth lofting process, which then helps to assemble the multilayer structures to create a smooth 3D model. In smoothing for multi-angle images, the algorithms are applied on the contours determined for each angle. After extrusion of 3D objects created by combining contours obtained from different angles, algorithms are activated to smooth rough edges. This process is critical for joining multi-angle structures and giving the 3D model a smoother surface structure.

[0042] Table 1. Categorization and the smoothing algorithms used subject to the invention.

[0043] Table 1 briefly describes the shape and features of each vessel category. For example, the "tubular, multi branched" structure of the abdominal aorta suggests that smoothing functions such as the Gaussian Filter and Median Smoothing are suitable, while the "acute angles, complex structure" of the proximal coronary artery explains the reasons why the Butterworth Filter and Laplacian Smoothing are preferred.

[0044] Industrial Applicability of the Invention The invention relates to an automatic three-dimensional vascular modeling method for use in the field of cardiology and cardiovascular surgical technique, and is industrially applicable.

[0045] The invention is not limited to the above descriptions and the person skilled in the art can readily present other different embodiments of the invention. These should be considered within the protection scope of the invention claimed by the claims. REFERENCES

[0046] [1] Lopez-Perez, A., Sebastian, R., & Ferrero, J. M. (2015, April 17). Three- dimensional cardiac computational modelling: Methods, features and Applications -

[0047] [2] Biomedical Engineering Online. BioMed Central. Xiao, R., Yang, J., Fan, J., Ai, D., Wang, G., Wang, Y., Shape context and projection geometry constrained vasculature matching for 3d reconstruction of coronary artery, Neurocomputing, 195 (2016), 65-73,

[0048] [3] van Assen M., Muscogiuri G., Caruso D., Lee S.J., Laghi A., De Cecco C.N. Artificial intelligence in cardiac radiology. Radiol. Med. 2020 Sep 18;125(11):1186— 1199. PubMed PMID: 32946002. Epub 2020 / 09 / 19.

[0049] [4] Muscogiuri G;Volpato V;Cau R;Chiesa M;Saba L;Guglielmo M;Senatieri A;Chierchia G;Pontone G;Dell’Aversana S;Schoepf UJ;Andrews MG;Basile P;Guaricci AI;Marra P;Muraru D;Badano LP;Sironi S; (n.d.). Application of Al in Cardiovascular Multimodality Imaging. Heliyon. https: / / pubmed.ncbi.nlm.nih.gov / 36267381 /

Claims

CLAIMS1. A 3D modeling method, characterized in that it comprises the process steps of: i. checking the input (1000), ii. determining whether the input is printed and obsolete (1001) iii. if the input is printed and obsolete, scanning radiographs with an X-ray film scanner (1001.1), scanning reports on white paper with a high resolution scanner (1200 dpi) (1001.2), and performing automatic enhancements (contrast, brightness, entropy, skewness, kurtosis) on the scanned images (1001 .3), or, if the input is not printed and obsolete, direct processing the digital input (1001 .4), iv. checking whether there is a 3D scan present, if there is a 3D scan present, respectively following the process steps 3001 -3004 and 4001- 4004 simultaneously (2000), or, if there is no 3D scan present, calculating the approximate value of the third dimension for the 3D image using a dummy (2001 ), v. determining whether multilayer images are used for processing multilayer images based on image characteristics (3000), vi. if multilayer images are used for processing multilayer images, selecting the function according to the specified vessel by the user (3001), or, if multilayer images are not used, following process no. 5000 by the system, vii. creating layer contours by applying selected smoothing functions. (3002), viii. selecting the correct smoothing function for lofting between contours of different vascular layers (3003), ix. lofting between contours of different vascular layers and creating a 3D model (3004), x. Determining whether multi-angle images are used for processing simultaneous multi-angle images based on image characteristics (4000), xi. if multi-angle images are used, selecting the function according to the specified vessel by the user (4001 ), or, if multi-angle images are not used, following process no. 5000 by the system xii. if multi-angle images are used, creating vessel contours by applying the selected smoothing function for different angles. (4002),xiii. extruding the contour of each angle and preserving their intersections (4003), xiv. applying the Elliptic function to smooth right angles (4004), xv. checking for small objects, disconnection, and non-merging edges (5000).

2. A 3D model produced by a method according to claim 1 for use in cardiovascular surgery.

Images