Computer-implemented method, computer program and system for analyzing heart valve

Abstract

The invention relates to a method for analyzing a heart valve (52) of a subject, comprising the steps of: (a) providing a 3D or 4D volumetric data set (32) of the heart valve (52); (b) providing a valve model (2, 3) of the heart valve (52), where the valve model (2, 3) is adjusted to the morphology of the heart valve (52) and comprises a closure curve (9, 12) representing the annulus of the valve, where a region within the closure curve (9, 12) comprises at least one 2D portion (16), each 2D portion (16) corresponding to a morphological or physiological characteristic of the heart valve (52); and (c) projecting the valve model (2, 3) along a projection direction (36) into the 3D or 4D volume data set (32) to define a volume of interest (42), where the volume of interest (42) comprises at least one 3D region (26), each 3D region (26) being defined by a projection of a corresponding one of the at least one 2D portion (16); (d) tagging each voxel in the at least one 3D region (26) with a tag indicative of a morphological or physiological feature of the corresponding 2D portion (16). Preferably, a visualization of a cut plane through the 2D or 3D volume data set is provided in which voxels in at least one 3D region (26) of the volume of interest (42) are represented in colors dependent on their labels.

Description

Technical Field

[0001] The present invention relates to a computer-implemented method, computer program, and system for analyzing heart valves in a subject. Background Technology

[0002] Three-dimensional (3D) echocardiography is a common method for visualizing the human heart, particularly the heart valves. When 3D ultrasound images of the moving heart are acquired over a period of time, video clips (also known as 3D echo clips) are generated, which is called four-dimensional (4D) echocardiography. 3D or 4D echocardiography is commonly used for planning and performing minimally invasive cardiac surgeries, such as transcatheter valve surgery. For example, valve repair via leaflet clamping is becoming a common method for reducing regurgitation and significantly improving patients' quality of life. However, this procedure requires very careful and precise planning. Interventional cardiologists need to know which leaflets they must clamp during the procedure. Therefore, good planning tools and good methods for visualizing the heart valves during interventional procedures are required.

[0003] It is known to provide volumetric images of the heart and its parts, such as heart valves. However, many surgeons and echocardiologists still prefer to rely on two-dimensional (2D) planes derived from 3D ultrasound images. However, due to the complexity of the anatomy of heart valves, maintaining orientation relative to valve anatomy on such a 2D plane can be extremely difficult.

[0004] US2005187461A discloses a computerized method for facilitating cardiac interventions, comprising: inputting patient data; creating a computerized interactive model of the heart based on the patient data; simulating at least one proposed cardiac intervention by adding or removing features from the model; and determining the impact of the proposed cardiac simulation on the overall model. The simulation can be repeated to allow the user to determine the optimal cardiac intervention. Additionally, templates can be created from the model for use as guidance during cardiac interventions.

[0005] US2020082531A1 discloses an apparatus for dynamically evaluating a moving object based on a continuous sequence of volumetric image frames, the images being temporally separated by a certain time interval: identifying the object of interest in at least one image of the sequence; segmenting the object to identify its outline; propagating the identified object outline to other images in the sequence; and performing dynamic analysis of the object based on the propagated object outline.

[0006] Tomasso Mansi et al., in their April 2016 white paper "Quantifying Heart Valves: From Diagnostic to Personalized Valve Repair," described a method for valve modeling and editing. They proposed a combination of providing editing views (such as parallel or rotational cutting) and intelligent mesh editing.

[0007] EP 3683773 A1 relates to a method for visualizing dynamic anatomical structures, comprising the following steps: a) Provide a sequence of three-dimensional medical images spanning a time period, each of the three-dimensional medical images in the sequence showing dynamic anatomical structures at a point in time during the time period; b) Provide a dynamic model of at least a portion of the anatomical structure, wherein the dynamic model has been derived from and registered with the sequence of the three-dimensional medical images; c) Determine a volume of interest comprising an anatomical feature of interest within each of the three-dimensional images, wherein the volume of interest follows the position and / or shape of the anatomical feature of interest across the time period; and d) Provide a three-dimensional visualization environment for displaying the dynamic anatomical structures across the time period, wherein visualization corresponding to a specific time point within the time period includes: (i) Volume rendering of the volume of interest corresponding to the three-dimensional image at the specific time point; and (ii) The visualization of the dynamic model at the specific time point and in the same coordinate system as the volume of interest.

[0008] However, assessing the topology of heart valves remains cumbersome, and therefore preparing for surgery remains difficult.

[0009] The purpose of this invention Therefore, the object of the present invention is to provide a method and apparatus that can provide a better and more intuitive way to assess anatomical structures, especially structures like heart valves, which are complex and difficult to visualize. Summary of the Invention

[0010] The present invention addresses this problem by utilizing a computer-implemented method including the features of claim 1, a computer program including the features of claim 14, and a system including the features of claim 15.

[0011] According to one aspect, the present invention provides a computer-implemented method for analyzing the heart valves of a subject, the method comprising the following steps: (a) Provide a 3D or 4D volumetric dataset of the patient’s heart valves; (b) Providing a valve model of the heart valve, wherein the valve model is adapted to the morphology of the heart valve, and The valve model includes a closed curve representing the valve annulus, wherein the region within the closed curve includes at least one 2D portion, each 2D portion corresponding to a morphological or physiological feature of the heart valve; and (c) Project the valve model onto the 3D or 4D volume dataset along the projection direction to define the volume of interest, wherein the volume of interest includes at least one 3D region, each 3D region being defined by the projection of a corresponding 2D portion of the at least one 2D portion; (d) Each voxel in the at least one 3D region is labeled with a label indicating the morphological or physiological features of the corresponding 2D portion.

[0012] This invention is based on the concept of transforming anatomical knowledge of heart valves represented in a valve model into voxel labels within a 3D or 4D volumetric dataset, which can greatly facilitate navigation through the 3D or 4D volumetric dataset. Specifically, different regions (two parts), such as different leaflets, within the heart valve can be identified in a simplified representation (valve model), but the information contained therein can be projected onto the volumetric dataset. Users can then navigate through the structures in the 3D or 4D images (3D or 4D volumetric dataset) without having to switch between image and model information, because voxel labels can be used to differently represent 3D regions corresponding to different morphological or physiological features of the heart valve, for example, with different colors.

[0013] The method of this invention is performed on a computer, which may be connected to the ultrasound acquisition system, and may be, for example, its control computer. The computer may also be a standalone computer on which data analysis is performed. Therefore, the computer can be any PC, workstation, cloud computer, tablet, laptop, or mobile device.

[0014] The method of the present invention is based on a valve model, which is preferably a highly simplified model of the valve on a 2D or 3D surface enclosed within a closed curve representing the valve annulus. Preferably, it is two-dimensional (2D) because any extension of the valve annulus or leaflet in a direction perpendicular to the principal plane of the valve annulus is not represented in the valve model but is projected onto the surface within the closed curve. However, this surface can be three-dimensional (3D) to best fit the heart valve. For example, the valve model can have the shape of a potato chip. In this case, it is referred to herein as a 3D valve model. In another embodiment, the valve model can be a 2D model because the surface enclosed by the closed curve is in a plane, particularly in the plane corresponding to the principal plane of the valve annulus. An example of such a 2D model is the topology-based 2D model explained below. The “principal plane of the valve annulus” is a plane that can describe the orientation of the heart valve, for example, the plane that best fits the valve annulus of the corresponding 3D valve model, even if the valve annulus is not entirely located in this plane due to its three-dimensional shape.

[0015] The valve model includes a closed curve representing the valve annulus. The closed curve can preferably have any 3D shape, as long as it is closed. Such a 3D closed curve is also referred to herein as a 3D ring. A 3D ring can, for example, have the shape of the outer circumference of a potato chip. The closed curve of the valve model can be derived from the segmentation of the valve annulus. In other embodiments, or in a further simplified model that can be derived from the valve model, the closed curve can have a standardized shape, such as a circle, ellipse, polygon, or a polygon with rounded edges.

[0016] The valve model is preferably a simplified visualization of the major components of a heart valve, particularly the annulus and leaflets. The region within the closed curve includes at least one 2D portion, wherein the at least one 2D portion represents a morphological or physiological feature of the heart valve. Preferably, the morphological or physiological feature of the heart valve forms part of the heart valve. The morphological feature can be an anatomical structure within the heart valve, such as a leaflet or papillary muscle. In particular, at least one 2D portion can represent a leaflet of the heart valve. Thus, the valve model can include information about the location, area, and / or shape of the leaflets. For example, a leaflet can be characterized by a junction defined on the closed curve, wherein each junction defines one end of a junction line between two leaflets of the heart valve. The other end of the junction line can be the center point of the valve model. The junction line can be a division between two 2D portions. Physiological features can be blood flow phenomena, such as regurgitation.

[0017] At least one 2D portion preferably covers an area smaller than the area within the closed curve. Specifically, at least one 2D portion covers a portion of the region within the closed curve representing the valve annulus of the valve model, i.e., not the entire region within the closed curve. In useful embodiments, at least one 2D portion comprises at least two 2D portions. Preferably, at least one 2D portion comprises 2 to 8, more preferably 2 to 4 2D portions. Preferably, the region within the closed curve comprises multiple 2D portions. Thus, each 3D region preferably covers a volume smaller than the volume of interest. In useful embodiments, at least one 3D region comprises at least two 3D regions. Preferably, at least one 3D region comprises 2 to 8, more preferably 2 to 4 3D regions.

[0018] The valve model is preferably derived automatically or semi-automatically from a 3D or 4D volumetric dataset. In a semi-automatic embodiment, an initial valve model can be created automatically. This can be performed by first segmenting the heart valve from the 3D or 4D volumetric dataset and then fitting the 3D valve model to the segmented heart valve, particularly the valve annulus. When the 3D valve model is overlaid on the volumetric dataset or a visualization of the segmented heart valve, for example within a volumetric rendering, the user can adjust the 3D valve model. The 3D valve model can be further converted (particularly mapped) to a standardized topology-based 2D model, and the user can be given the opportunity to adjust the topology-based 2D model for the visualization of the heart valve, for example by adjusting the position of the junctions. This will be explained in more detail below. Preferably, the topology-based 2D model is a static symbolic representation of the heart valve because it does not change during the cardiac cycle. Therefore, even if the 3D valve model of the heart valve moves with the heartbeat, the topology-based 2D model will remain static.

[0019] 3D or 4D volumetric datasets can be obtained through medical imaging modalities or have already been obtained through medical imaging modalities, preferably by ultrasound, such as by 3D or 4D echocardiography, particularly by transesophageal 4D echocardiography. Other possible imaging modalities are magnetic resonance imaging (MRI), X-ray imaging, positron emission tomography (PET), or computed tomography (CT). A 3D volumetric dataset comprises a 3D matrix of voxels, each voxel including grayscale values ​​representing image information. In a 4D volumetric dataset, the fourth dimension is time. In other words, a 4D volumetric dataset can be a time-series video clip containing frames covering, for example, one or more heartbeat cycles (i.e., a 3D volumetric dataset). Therefore, a 4D volumetric dataset can be acquired as a sequence of 3D images obtained from a human or animal subject, the sequence of 3D images spanning a time period at, for example, a frame rate of 5-100 (preferably 20-60 images per second) to allow for a smooth representation of dynamically moving heart valves. This time period is typically at least one cycle of the periodic motion of the heart, such as at least one heartbeat. The subject can be a human or animal, particularly a patient. A heart valve can be any of the four heart valves, such as the mitral or tricuspid valve. 3D or 4D volumetric datasets can be provided in DICOM (Digital Imaging and Communications in Medicine) format or any other suitable image format. 3D or 4D volumetric datasets (also referred to herein as volumetric datasets, image datasets, or images) include image data containing morphological information about anatomical structures (i.e., the heart or parts of the heart). Morphological information about heart valves specifically includes anatomical features of the valve's components (such as leaflets and annulus), such as shape, structure, and size. Therefore, the morphological features of a heart valve can be, for example, the leaflet, parts of the leaflet, or any other part thereof. In contrast, the physiology of heart valves primarily deals with function. Therefore, the physiological features of heart valves can be, for example, areas of irregular blood flow within the heart, such as the location of regurgitation leakage, and more generally, the location and amplitude of flow phenomena.

[0020] A valve model can be derived, for example, from a 3D or 4D volumetric dataset using the following method: First, a 3D or 4D volumetric dataset of a moving heart is acquired as described above, wherein the dataset includes image data of the heart valves. In the next step, the heart valves are segmented. Thus, the anatomical structure of the heart valves is separated from blood or other tissues, such that preferably only voxels representing the portion of the heart valve or its surrounding environment are preserved. Segmentation methods are well known in the art. For example, methods such as those described by Dröge, Hannah, et al. Mitral Valve Segmentation Using Robust Nonnegative Matrix Factorization The execution segmentation is shown in (Journal of Imaging, 2021, Vol. 7, No. 10, p. 213), which is incorporated herein by reference.

[0021] A valve model can be derived from a segmented heart valve. In a preferred embodiment, the valve model is or is a 3D valve model based on the heart valve. The 3D valve model is preferably dynamic, i.e., it follows the movement of the anatomical structure (i.e., the heart valve) over a time period. The dynamic 3D valve model can be used to visualize the basic components of the heart valve, particularly the annulus and possibly the leaflets. In a preferred embodiment, the 3D valve model includes a 3D ring representing the annulus, which is located and oriented within a 3D or 4D volumetric dataset at the location and orientation of the annulus of the heart valve. This 3D ring can be obtained by fitting the 3D ring (in other words, a 3D closed curve that can have any shape in three dimensions) to the segmented heart valve. It can also be obtained by manually, automatically, or semi-automatically identifying certain landmarks of the heart valve on the 3D or 4D volumetric data or on the segmented heart valve, and then fitting the 3D ring to the landmarks. Alternatively, the 3D valve model can be a shape / surface model of the heart valve. A surface 3D valve model can be a simplified representation of a heart valve, such as a triangular surface covering the valve annulus. For each frame of a 3D or 4D volumetric dataset, the 3D valve model can include multiple points spanning a three-dimensional closed curve or an alternative ground surface. It can also be a mathematical model, such as a parametric model, like a closed loop, surface, or volume spanned by spline curves. The 3D valve model can be automatically fitted to a segmented heart valve. In some embodiments, the user may be given the opportunity to adjust the 3D valve model on a visualization of the heart valve.

[0022] Within the closed curve of a valve model, which may or may correspond to a 3D valve model, a 2D portion can be defined. For example, anatomical features of a heart valve can be represented in the valve model by junctions located on the closed curve. Furthermore, lines separating the leaflets from each other can be represented on the valve model. Such junction lines can originate from the junction points and extend toward the center point of the valve model, or extend to another point within the closed curve corresponding to the endpoint of a leaflet. The center point can be the centroid of the closed curve / 3D annulus of the 3D valve model representing the annulus. Thus, the portion separated by the junction lines can represent a 2D portion corresponding to a morphological feature (i.e., a leaflet). Other morphological features can also be represented as 2D portions, such as areas of leaflet damage. Additionally, physiological features such as certain regions or 2D portions can be represented on the valve model, wherein regurgitation occurs at least during a portion of the cardiac cycle. Preferably, the valve model is automatically created. In some embodiments, the user can adjust the valve model such that it best fits to a 3D or 4D volumetric dataset, as visualized in a visualization environment.

[0023] In this embodiment, the valve model is based on or associated with a topology-based 2D model of the heart valve. The topology-based 2D model can be derived from a 3D valve model. The topology-based 2D model is essentially a further simplification of the 3D valve model because the 2D model does not extend into a third dimension, specifically not into the dimension perpendicular to the principal plane of the valve annulus. Therefore, the topology-based 2D model can be derived by projecting or mapping the 3D valve model onto a plane. When mapping the 3D valve model onto a plane to obtain the topology-based 2D model, the proportions of the valve annulus in the 3D valve model can be preserved. In other embodiments, the projection of the 3D valve model onto a 2D surface can be distorted to obtain the topology-based 2D model. In yet another embodiment, the topology-based 2D model is created based on a developable surface of a segmented heart valve or a 3D valve model. A developable surface is a surface that can be flattened onto a plane without deformation (i.e., by “folding,” “bending,” etc.). In this context, the topology-based 2D model can be a developable surface surrounded by a 3D ring of a 3D valve model, which is flattened onto a 2D plane. The resulting "flattened" 2D closed curve can be further simplified by transforming it into a standardized shape (such as a circle or ellipse).

[0024] Preferably, the 4D volumetric data is focused on tracking a valve model, particularly a 3D valve model, over at least one heartbeat. In other words, the valve model can be visualized in any frame and is preferably tuned by the user to best fit a segmented heart valve, or a 3D or 4D visualization of a heart valve or a segmented heart valve. A mapping from the 3D valve model to a topology-based 2D model is also tracked over the heartbeat, allowing the model to be visualized at any point in time during the cardiac cycle. However, the topology-based 2D model is preferably static, as it does not change with the heartbeat.

[0025] The topology-based 2D model is preferably a standardized simplification of the actual segmentation of the heart valve. Therefore, the 3D representation of the heart valve (e.g., a segmented heart valve) is transferred to the 2D visualization of the heart valve. This standardized representation of the heart valve is very useful because it has been found that heart valves can vary significantly in their morphology, to the point that even the number of leaflets can differ between patients—in particular, some may have two leaflets while others may have three—and its standardized representation can significantly aid in diagnosis.

[0026] While topology-based 2D models are useful for visualizing heart valves, physicians may still need to reference the original 3D or 4D volumetric dataset for planning cardiac surgery or guiding surgical instruments during the procedure. This necessitates visualization using a 3D or 4D volumetric dataset, where the visualization can be, for example, 3D rendering, such as volumetric rendering, or alternatively or additionally, a 2D cutting plane derived from the 3D or 4D volumetric dataset. Such a cutting plane can be generated by multiplanar reconstruction (MPR), where pixels on a plane oriented at an oblique angle across the original 3D image matrix are calculated, for example, by interpolation from the nearest voxels of the original volumetric dataset. Therefore, the cutting plane can be oriented through the 3D or 4D image dataset in any user-selected orientation. Such cutting planes are commonly used by physicians when evaluating 3D or 4D volumetric datasets of heart valves, where the physician will move and tilt the cutting plane at various angles to view different portions of the 3D or 4D volumetric dataset. The 2D cutting plane can be usefully oriented along the axis (LaX plane) of the heart chamber (e.g., the ventricle). However, the 2D cutting plane can also be orthogonal to this axis orientation (SaX plane). It has been found that the visualization method of the present invention facilitates image analysis and orientation within the data in any type of cutting plane.

[0027] The present invention recognizes that navigation through volumetric datasets is typically very difficult, but can be facilitated by using a valve model of a heart valve, as this model usually defines 2D portions within the valve annulus, each corresponding to a morphological or physical feature of the heart valve, such as leaflets, leaflet portions, reflux regions, etc. According to the invention, this information is projected into a volume, where it can be used to facilitate user-defined navigation through the volumetric dataset. This is accomplished, in particular, by projecting the valve model into a 3D or 4D volumetric dataset along a projection direction to define a volume of interest. The volume of interest can be a cylinder with a region enclosed by a closed curve of the valve model as its base. In the case of a circular closed curve, the cylinder will be a perfect circle; obviously, depending on the shape of the closed curve representing the valve annulus, it can have other shapes. Preferably, the centroid of the cylinder's base can correspond to the center point of the valve model and can be aligned with the centroid of the heart valve. In the case of a circular cylinder, the cylindrical axis passing through the center point of the circle forming its base is aligned with the centroid of the heart valve. In a preferred embodiment, the height of the cylinder is the full height of the 3D or 4D volumetric dataset. In another embodiment, the valve model of the heart valve is projected only to a set height. The projection can be in one direction or in two directions starting from the principal plane of the valve annulus.

[0028] Therefore, the volume of interest describes and includes the 3D volume above and / or below the heart valve, particularly the region in which the valve leaflet can move during the cardiac cycle. The volume of interest includes at least one 3D region, defined as a projection of a 2D portion onto the valve model. The 2D portion may, for example, define a region corresponding to a leaflet of the heart valve or another morphological or physiological feature. The corresponding 3D region is the projection of this 2D portion along the projection direction. In other words, the volume of interest includes at least one sub-volume, called the 3D region, which also has a cylindrical shape, rather than a circular cylinder, wherein the base of the sub-cylinder is the portion corresponding to the morphological or physical feature. In a preferred embodiment, such a 3D region may have the shape of a slice of a pie, where the pie is the volume of interest.

[0029] When visualizing a volume of interest, a user may easily confuse one leaflet with another or with other structures such as the papillary muscle. Therefore, this invention proposes labeling voxels within a volume of interest according to the 3D region to which they belong, wherein each 3D region is defined by a projection of a 2D portion corresponding to a morphological or physiological feature of a heart valve. For example, a valve model may include one or more 2D portions, each corresponding to a leaflet. A 2D portion may, for example, be a segment representing the closed curve of the valve annulus. If the valve model is then placed at the position and orientation of the valve annulus within a frame of a 3D or 4D volume dataset and then projected into the volume, the 2D portions are projected into corresponding 3D regions, and each leaflet represented by a portion can move within that 3D region during heartbeat. Thus, each leaflet can move primarily within its own 3D region within the volume of interest. According to the invention, each voxel in at least one 3D region is labeled with a tag indicating the morphological or physiological feature of the corresponding 2D portion, such as a tag indicating a specific heart valve. Therefore, users can now navigate through volumes of interest in a 3D or 4D volume dataset (preferably across all frames of a 4D volume dataset) and find voxels labeled according to the leaflets to which they might belong.

[0030] In useful embodiments, the valve model, and particularly the projection orientation, is adapted to the position of the heart valve in each 3D frame of the 4D volume dataset. If the valve model is a 3D valve model or based on a 3D valve model, the 3D valve model is adapted over time and fitted to the valve annulus as it moves within the cardiac cycle. If the valve model is based on a topology-based 2D model, the topology-based 2D model can be placed in the principal plane of the valve annulus in each frame, for example, as identified on segmented heart valves. The projection orientation is determined as described below. In useful embodiments, the placement of the valve model and projection orientation can be identified using a 3D valve model of the valve annulus, which is located and oriented within the 3D or 4D volume dataset at the position and orientation of the heart valve annulus, and can be derived automatically or semi-automatically from the 3D or 4D volume dataset, particularly from the segmentation of the heart valve, using an automatic or semi-automatic fitting algorithm.

[0031] According to an embodiment, the method includes the further step (e): providing a visualization of a 3D or 4D volumetric dataset of a heart valve, the visualization including information from a projected valve model, particularly wherein voxels with different labels are visualized differently. The visualization can be, for example, a rendering of a 3D or 4D volumetric dataset; in the case of a 4D volumetric dataset, it is preferably a dynamic rendering within at least one cardiac cycle. In the visualization, different 3D regions within the volume of interest are preferably visualized differently, so that when viewing the visualization, the user always knows which anatomical region he is navigating through.

[0032] According to embodiments, voxels in at least one 3D region of the volume of interest are represented by color depending on their labels. In useful embodiments, at least two 3D regions of the volume of interest are represented by different colors. For example, 3D regions belonging to different leaflets can be colored differently. This color coding allows a user to always know which leaflet region he / she is navigating through. For example, 3D regions can be colored with one or more of red, green, yellow, magenta, orange, or blue, or any other color. Since it is sometimes difficult to determine the exact location where the leaflets meet at the center of the heart valve, an unlabeled central 3D region can be left. In a valve model, this can be represented by a central 2D portion that does not correspond to any particular morphological or physiological feature of the heart valve, but is simply left “empty”; in other words, the 2D portion of the valve model does not fill the entire surface surrounded by closed curves, but the central region remains undefined. Therefore, the 3D region defined by the projection of this central portion can be left unlabeled and can be represented by a neutral color instead of any particular color. According to another embodiment, when voxels with different labels are represented or visualized in different colors, there may be soft transitions between the different colors in the visualization, at least in some areas. This is particularly useful for the central region around the central axis of the volume of interest, especially around the central axis of the cylinder. In embodiments where there is a hard transition from color A to color B at this axis, this can be of great importance to the central axis, even though it is not so relevant on its own. Thus, a central region, such as a 3D region around the central axis, can be defined where the visualized shadows will transition softly from one color to the next, for example, with grayscale in between (color A - grayscale - color B). Alternatively, the 3D region around the central axis can be represented in a neutral color.

[0033] According to a preferred embodiment, visualization includes visualizing 2D cutting planes through a 3D or 4D volume dataset. The 2D cutting planes can be generated from the 3D or 4D volume dataset using MPR. Preferably, the 2D cutting planes can be oriented through the volume dataset in any user-selected orientation. Therefore, 3D regions of the volume of interest can also intersect at any angle. However, visualizing voxels from different 3D regions helps the user locate their orientation within the volume dataset. Preferably, all voxels in the 3D regions are colored the same color, corresponding to the corresponding 2D portion in the valve model. All 2D cutting planes at any location and orientation through the volume dataset, and preferably in any frame of the frame sequence, can display their corresponding color, indicating the location of specific morphological or physiological features of the heart valve. Thus, anatomical knowledge is transformed into color coding, allowing the user to always know that they are navigating through a lobular region. This enables navigation through structures in the image without having to switch between image information and model information.

[0034] Voxels can be represented in different colors in various ways. According to one implementation, an image of the 2D cut plane is first generated, and then colored shadows are projected onto the top of the image, as if the user were viewing the image through "stained glass." In another embodiment, the visualization of the 2D cut plane already includes color, which is associated with each voxel via a label. Therefore, when the 2D cut plane is generated, the voxels are already assigned shadows based on the color shadows associated with their respective labels, as the pixels of the image have been assigned shadows with those colors.

[0035] Anatomical structures outside the volume of interest can remain unshaded. Alternatively, structures outside the volume of interest may not be represented at all, or may be represented outside the volume of interest with a margin of safety. In other words, the volume of interest can also be used to "cut out" the heart valves to make them more visible.

[0036] This invention is not limited to visualizing 2D cut planes, but can also utilize color-coding schemes when visualizing 3D or 4D volumetric datasets in different ways, such as through volumetric rendering or surface rendering. Specifically, volumetric rendering refers to techniques for visualizing three-dimensional image datasets and employs parameters such as thresholds, opacity, and contrast, which can be advantageously adjusted by the user in "real-time," i.e., providing an immediate effect as if the user is viewing the rendering. Visualization can also be surface rendering, particularly surface rendering of segmented heart valves. In another embodiment, visualization is surface rendering of a surface 3D valve model of the heart valve (e.g., the 3D valve model described herein).

[0037] In the case of a 4D volumetric dataset, visualizations where different 3D regions are represented differently (particularly with different color shading) are preferably dynamic; that is, visualizations of complete 3D image segments are performed, at least for a 3D image sequence covering at least one cardiac cycle. Thus, users can analyze the dynamics of the heart valves while benefiting from the anatomical information propagated through the 3D volumetric dataset via projections of different 2D portions.

[0038] According to one embodiment, voxels outside the volume of interest are not visualized in the visualization. This is a useful embodiment because it can further facilitate in-dataset orientation by removing those regions of no interest in valve analysis. "Voxels outside the volume of interest" can be understood literally, or it can mean having a margin outside the volume of interest, such as a user-defined margin, e.g., 3-15 mm, to also cover adjacent regions of the valve. In an alternative embodiment, voxels outside the volume of interest are visualized differently from voxels within the volume of interest, for example, with a neutral color or a lighter shade.

[0039] According to an embodiment, the valve model includes a plurality of junction points positioned along the circumference of a closed curve, wherein each junction point defines the origin of a junction line between two leaflets of the heart valve, and wherein at least one portion is partially defined by at least one junction line and corresponds to a leaflet. Thus, the junction line can define a dividing line between two adjacent leaflets or the edges of leaflets of the heart valve. The junction point is located where the leaflets of the heart valve meet at their junction with the valve annulus. Therefore, the junction point can be used to define 2D portions corresponding to different leaflets. The endpoint of the junction line can be the center point of the valve model, such as its centroid or the center of a circle or ellipse representing the valve annulus. It can also be another junction point, particularly in the case of only two leaflets. It can also be another point on the valve model, which, for example, is identified as the endpoint of the junction line on the segmentation of the heart valve. The junction line itself can be a straight line between the origin and the endpoint. It can also be a curve, for example, crossed by several points on the valve model. Thus, by connecting the junction points to the center point or other points via junction lines, the leaflets of the heart valve can be represented in a simplified manner in the valve model. In one embodiment, a straight line extends from each junction toward the center of the closed curve representing the valve annulus, specifically the center of the corresponding circle representing the valve annulus. Thus, the valve model is divided into 2D portions, such as a pie chart, each portion representing a leaflet. However, in other embodiments, particularly those with only two leaflets, the lines extending from the junctions can have different shapes, more closely resembling the actual anatomy of the valve. Furthermore, since it is generally difficult to precisely define the heart valve at its center, i.e., along the cylindrical axis, in some embodiments, the central region of the valve model is not assigned to any particular portion but is not defined and is then represented in a neutral color. Therefore, the center of the valve model may include an inner circle as the central region, which may be concentric with the closed curve. Additionally, the junctions may describe fine structures that can be identified by two anatomical boundaries, such as the axes of corresponding papillary muscles and chordae tendineae with specific fan-shaped configurations.

[0040] The locations of the junction points on the valve model and in the 3D or 4D volumetric dataset, particularly on segmented heart valves, are preferably correlated with each other via a mapping function such that they have the same relative position around the valve annulus and / or the same relative position with respect to a reference point of the heart valve. The reference point can be, for example, the centroid of the heart valve. This applies when the valve model is a 3D valve model as described herein. It also applies when the valve model is or is based on a topology-based 2D model as described herein.

[0041] According to an embodiment, at least one 2D portion within a closed curve corresponds to a physiological feature, wherein, preferably, the physiological feature is a flow phenomenon. Therefore, a topology-based 2D model can also be used to define the physiological features of a heart valve in a standardized manner, particularly when projected onto a 2D plane of the topology-based 2D model. For example, certain regions where blood flow phenomena occur can be represented as 2D portions on a topology-based 2D model. Such a flow phenomenon can, for example, be a region of regurgitation flow. If this is represented differently in the visualization of a 3D or 4D volumetric dataset, it can provide physicians with valuable information about the exact location of valve prolapse, thereby assisting in surgical planning. In another embodiment, the flow phenomenon can also be blood flow from an adjacent valve that the user wants to exclude from the visualization of blood flow in the 3D or 4D volumetric dataset. In this case, the corresponding portion can be automatically defined on the topology-based 2D model or defined by the user. When the model is projected onto a 3D or 4D volumetric dataset, the 3D region associated with the portion where the unwanted flow measurement occurs can be marked as an area where no flow phenomenon will be visualized. In the next step, the user can choose to visualize a 3D or 4D volumetric dataset, which includes flow information, particularly flow information from Doppler ultrasound measurements. 3D regions associated with the 2D portion of unwanted flow measurements can be excluded from the flow visualization. In other words, although Doppler measurements are available for that 3D region, the measurements are not displayed to the user. Therefore, the user can focus on displaying Doppler measurements of regurgitation from the heart valves, which is important for the clinical problem at hand, without being distracted by unwanted flow phenomena.

[0042] According to an embodiment, the valve model is based on a 3D valve model that is located and oriented within a 3D or 4D volume dataset at the position and orientation of the valve annulus of the heart valve, wherein the projection direction is at least approximately perpendicular to the plane extension of the 3D valve model fitted to the valve annulus. In this context, "at least approximately" means up to ±10%, i.e., an angle between 81° and 99° with respect to the fitted plane. The above explains an example of how to generate a 3D valve model of the valve annulus from a 3D or 4D volume dataset of the heart valve. This method can be applied to each frame within a video clip (4D volume dataset). Alternatively, a 3D valve model of the valve annulus can be generated for one frame of the video clip, and the model can be tracked through a series of frames, for example, by following multiple anatomical landmarks from one frame to the next.

[0043] To project a 2D portion of a 3D valve model onto a 3D or 4D volumetric dataset, the 3D valve model can first be projected onto a plane, specifically a plane perpendicular to the projection direction. The 3D valve model is not necessarily limited to a 2D plane but can have a three-dimensional shape. Therefore, the orientation of the plane having the minimum quadratic distance to a non-planar surface enclosed by the closed curves of the valve model can be calculated. Here, for example, a set number of points (such as 5-30, preferably 10-20) are distributed on the non-planar surface of the 3D model. A minimization algorithm is then performed to find the plane that minimizes the sum of squares of the distances from each point to the plane. This plane is also called the "fitting plane" and can correspond to the principal plane of the valve annulus. In an alternative embodiment, the fitting plane can be determined solely based on points on the outer circumference of the 3D valve model (3D annulus), preferably also by minimizing the sum of squares of the distances from the outer circumference to the corresponding flat annulus. The flat annulus then defines the plane. The 3D valve model can be projected onto this plane along with any 2D portions included therein to produce a 2D valve model. Then, the 2D valve model is projected onto a 3D or 4D volumetric dataset.

[0044] In other embodiments, the orientation of the fitting plane is considered as the orientation of a topology-based 2D model, which is a valve model projected onto a 3D or 4D volumetric dataset. The topology-based 2D model is then positioned at the location of the 3D valve model, thus positioning it on the annulus of the heart valve. Furthermore, any junctions defined on the topology-based 2D model are positioned (e.g., calculated using a minimization algorithm) as close as possible to junctions defined in the 3D or 4D volumetric dataset, either on the 3D valve model or on the segmented heart valve. Therefore, it is possible to identify the projection direction extending from the region of the heart valve into the heart, and to perform the projection of the topology-based 2D model as accurately as possible.

[0045] According to one embodiment, the visualization is a dynamic 4D visualization of the heart valve, and the projection orientation of the valve model is adjusted according to the movement of the heart valve. In this embodiment, the projection orientation is calculated for each frame within a 4D volumetric dataset. This can be accomplished using a dynamic 3D valve model of the valve annulus, which is obtained, for example, by tracking a 3D valve model from one frame to the next. A plane passing through the 3D valve model can then be fitted for each frame within a 3D video segment of the heart valve. Therefore, the labels of voxels in the 3D region can be adjusted according to the movement of the heart during the cardiac cycle, and the visualization of the moving heart can be dynamically adapted to the position of the heart valve during the cardiac cycle. This greatly facilitates the analysis of the heart valve.

[0046] According to an embodiment, the volume of interest extends into the 3D or 4D volumetric dataset up to a predetermined height along the projection direction, preferably a height proportional to the size of the valve model. In other words, the height of the volume of interest is not the full height of the volumetric dataset. This has the advantage that the visualization of information from the projected valve model can be limited to the area around the valve so as not to distract the user. In particular, in visualizations that use different colors to represent different 3D regions, too many colors can be distracting to the user.

[0047] The height of the volume of interest (LOI) can be proportional to the size of the valve model. For example, it can correspond to the widest diameter of the closure line representing the valve annulus, which could be the diameter of the circle representing the valve annulus. It can also be proportional to that diameter, but multiplied by a predetermined factor, for example, between 0.5 and 1.5. In other embodiments, the height of the LIO can be set by the user. Therefore, the user can tailor the visualization to their specific requirements. Specifically, in embodiments where voxels outside the LIO are not visualized, the user can thus precisely determine the size of the data that should be visualized and analyzed. However, it is also useful to limit the color coding to a neighborhood of the valve when the visualization of the LIO involves color coding of voxels in different 3D regions. Therefore, the height of the LIO can be set manually or automatically. It can also be set automatically first, for example, to a height proportional to the size of the valve model. Then, the user can be given the opportunity to manually adjust the height of the LIO.

[0048] According to embodiments of the method of the present invention, a topology-based 2D model of the heart valve is used, preferably a highly simplified valve model represented in a plane, i.e., it is 2D in the sense that all elements of the topology-based 2D model can be represented in 2D coordinates. The topology-based 2D model preferably includes a closed curve representing the valve annulus. The closed curve preferably has a standardized shape, such as a circle, ellipse, polygon, or polygon with rounded edges. The size of the closed curve can be standardized regardless of the actual size of the heart valve annulus. In other embodiments, the size of the closed curve is proportional to the actual size of the heart valve annulus (e.g., the widest diameter). Therefore, the topology-based 2D model is preferably a standardized and simplified visualization of the major components of the heart valve (particularly the annulus and leaflets). The region within the closed curve may include at least one 2D portion, wherein at least the 2D portion represents a morphological or physiological feature of the heart valve. The advantage of the topology-based 2D model is that it provides a simplified visualization of the characteristics of the heart valve. Thus, the topology-based 2D model can be used to easily compare different heart valves with each other.

[0049] To enable adaptation of the topology-based 2D model, preferably, the user is also shown a 3D visualization of the 3D valve model from which the topology-based 2D model is derived, as well as at least a portion of a 3D or 4D volumetric dataset of the heart valve. In a preferred embodiment, the segmented heart valve is displayed as a volumetric or surface drawing, with the 3D valve model overlapping it. This visualization is preferably dynamic, i.e., it follows the movement of the heart valve over a period of time, such as a heartbeat. Since the 3D valve model is directly related to the topology-based 2D model via a mapping function, any changes made in the topology-based 2D model can be directly propagated to the 3D valve model and included in the visualization shown to the user. Conversely, in some embodiments, the user can also modify the 3D valve model, for example, by moving, deleting, or adding junctions, and this modification will be immediately propagated to the topology-based 2D model via the mapping function. Therefore, the user can adapt the topology-based 2D model to fit the actual heart valve.

[0050] According to another aspect, the present invention relates to a method for visualizing 2D cutting planes of a 3D or 4D volume dataset enriched with morphological information from a valve model. Specifically, voxels in the volume of interest are appropriately labeled according to the corresponding labels of the 2D portions. The rendering attributes of pixels in the 2D cutting plane are set according to the labels associated with the voxels of the volume of interest intersecting the 2D cutting plane. The visualization can be performed on a user interface that can be configured to display a visualization of the 3D or 4D volume dataset and information from the projected valve model. The visualization environment can be a three-dimensional visualization environment or a virtual reality environment. “Virtual reality” refers to any computer-generated visualization that provides a true three-dimensional experience of the depicted structure. Thus, the virtual reality (VR) environment of the present invention specifically provides visual feedback, but may also allow other types of sensory feedback, such as auditory feedback. The VR environment may also be an augmented reality environment, in which the user still sees the real environment, but VR objects (e.g., volume rendering) are overlaid or superimposed on real objects, or a mixed reality environment, in which real-world objects are superimposed on a virtual scene. The visualization environment also preferably allows users to adjust the valve model and, optionally, a topology-based 2D model, for example, via drag-and-drop functionality. The visualization environment can be used to view and analyze heart valves, particularly for planning interventions and / or determining the correct size, shape, and location of implants to be placed in future interventions.

[0051] As explained above, in addition to the 2D cutting planes, visualization of a 3D or 4D volumetric dataset of heart valves can include the rendering of segmented heart valves, such as volumetric or surface rendering, preferably overlaid with the valve model, particularly a 3D valve model. The corresponding topology-based 2D model can be visualized alongside the rendering. Therefore, the user can adjust the valve model to the anatomy of the heart valve on the visualized rendering and immediately see any changes in the color shading of the 2D cutting planes. The user can preferably further adjust the orientation of the 2D cutting planes through the 3D or 4D volumetric dataset. The visualization is preferably dynamic, i.e., it follows the movement of the heart valve over a period of time, for example, one heartbeat.

[0052] According to an embodiment, the method includes an additional step of receiving user input data via a user interface for creating or adjusting at least a 2D portion within a valve model. Preferably, the user interface includes a display configured to display visualizations and one or more user input devices operable by a user to provide user input. Such user input devices may be a computer mouse, joystick, touchpad, etc. Preferably, the display may be a touch-sensitive display. In this case, no additional user input device is required. The user interface may include a processor and a storage device for storing information. Preferably, the user interface is a computer or part of a computer.

[0053] In embodiments where the valve model is a 3D valve model as described herein or based on a 3D valve model as described herein, the 3D valve model is preferably visualized over a visualization of a 3D or 4D volumetric dataset (e.g., its volume rendering), and the user can modify the 3D valve model to better fit the anatomy of the heart valve. According to a preferred embodiment, a topology-based 2D model that can be derived from the 3D valve model as described herein is also displayed to the user, for example, next to, on the same screen as, or on another screen of the 3D or 4D volumetric dataset visualization, and any adjustments made by the user to the 2D portion of the topology-based 2D model are transferred to the 3D valve model. Therefore, the user can edit the topology-based 2D model (instead of the 3D valve model) to better fit the morphological or physiological characteristics of the heart valve. According to a preferred embodiment, any adjustments made by the user are immediately visible in the visualization of the 3D or 4D volumetric dataset. For example, if a user edits the 2D portion, for instance, by moving the junction points around a closed curve, adding or deleting junction points, and thereby adding or deleting leaflets, the mapping from the topology-based 2D model to the 3D valve model displayed superimposed on a 3D or 4D volumetric dataset is immediately adjusted accordingly. Consequently, the 3D region within the volume of interest also changes, and this is immediately visualized, for example, by changing the color shading. This has the advantage that users can immediately benefit from a better morphological fit from the topology-based 2D model to the actual heart valve, which will facilitate the analysis of 3D or 4D volumetric datasets.

[0054] According to another embodiment of the invention, user input includes instructions to modify the position of at least one junction point of the valve model. For example, the user can virtually grasp the junction point to be modified and move it to a desired position on the valve annulus (or the closed curve of the valve model). The user can preferably modify the junction point in a 3D visualization of the valve model (particularly a 3D valve model) overlaid on a visualization of the heart valve (e.g., a segmented heart valve). The corresponding junction point in the topology-based 2D model (whose position is not directly set by the user) automatically and simultaneously adjusts its position to match the position of the junction point directly set by the user. Preferably, the junction point and junction line are displayed on the topology-based 2D model and on the visualization of the 3D valve model overlaid on the (segmented) heart valve. This provides an opportunity to verify that the junction point is well placed in different images of a 3D or 4D volumetric dataset.

[0055] According to another embodiment of the invention, user input includes instructions to add new junction points and / or delete junction points. This is advantageous because different valves can have different numbers of leaflets, and even each person can have a different number of leaflets, for example, in their tricuspid valve. Therefore, the valve model can be adapted to the specific heart valve being examined. For example, the user can add additional junction points by clicking on the closed curve of the valve model. A new junction line extending from the new junction point to the center of the valve model can be automatically created. Thus, another leaflet can be depicted by the valve model. In a similar manner, junction points and the junction lines defined by the deleted junction points can be deleted. Thus, the valve model can be reduced by one leaflet and a 2D region can be reduced.

[0056] According to another embodiment, the method further includes labeling the 2D portions, particularly with labels corresponding to leaflets defined by at least one suture line. The labels may correspond to commonly used indications for the respective leaflets. For example, a tricuspid valve may include an anterior, posterior, and medial leaflets. An aortic valve may include a left coronary valve, a right coronary valve, and a non-coronary valve. In the case of a tricuspid valve, the leaflets may be labeled according to the nomenclature proposed by Rebeca T. Hahn, T. Weckbach, Thilo Noack, Nadira Hamid, Mitsunobu Kitamura, Richard Bae, Philipp Lurz, Susuel K. Kodali, Paul Soraja, Jörg Hausleiter, and Michael Nabauer in their “Proposal for a Standard Echocardiographic Tricuspid Valve Nomenclature” (JACC: Cardiovascular Imaging, Vol. 14, No. 7, 2021, pp. 1299-1305, ISSN 1936-878X), which is incorporated herein by reference. Labels for the corresponding leaflets can be used to annotate the 2D portions of the valve model. When the valve model is projected onto a volume, the various 2D portions representing different leaflets are transformed into 3D regions, which are then labeled accordingly. Therefore, the visualization of the cutting planes (where 3D regions with different labels are represented by different colors) can be color-coded according to the labels of the corresponding leaflets. The visualization may include a title, where the corresponding color is linked to the label of the corresponding leaflet. Thus, the leaflet labeling is transformed into color-coded, which allows a user observing a heart valve across various cutting planes of a 3D or 4D volumetric dataset to always know which leaflet region they are navigating through.

[0057] According to another embodiment of the present invention, the method further includes visualizing the flow phenomenon on a 3D visualization of the heart valve based on flow information, identifying the position on the valve model corresponding to the position of the flow phenomenon, and providing a 2D portion or marker at the identified position.

[0058] In the prior art, the projection of the leak location onto the valve is not intuitive for the user. The affected area of ​​the procedure cannot be easily marked and must be estimated using existing techniques. Currently, many of these procedures are performed without specific planning, increasing time on the operation schedule and the rate of termination due to incompatibility of methods or equipment. If image-based planning is performed, it is done entirely manually, consuming valuable physician time and exhibiting high variability and error-proneness. In this embodiment, a simple way is provided to indicate the leak flow in a valve model in the form of labels or 2D segments, which are marked accordingly, and this information is projected back into a 3D or 4D volume. Therefore, interventional planning can be facilitated and made more accurate. The flow phenomenon can be derived from Doppler information included in the 3D or 4D volume dataset. Thus, the flow (e.g., blood) can be visualized. The Doppler information can be in the same coordinate system as the segmented heart valve.

[0059] Therefore, the flow phenomenon can be visualized at its correct location relative to the (segmented) heart valve or a 3D valve model. This 3D location of the flow phenomenon in the visualization of the segmented heart valve can then be mapped to the valve model. This can be done using reference points present in both the visualized segmented heart valve and the valve model. Such reference points can be some or all of the junctions and / or the center of the heart valve. The location of the flow phenomenon is then known in the valve model and can be indicated by markers or 2D portions (e.g., regions of predefined size and shape around the markers). Alternatively, the region of the 2D portion can be calculated based on Doppler information included in a 3D or 4D volume dataset. The location and size of the region can then be projected onto a topology-based 2D model for easier localization in surgical planning. The locality of the leak can be indicated by contour lines or heatmaps. Thus, interventions can be planned precisely in advance.

[0060] In this embodiment, regurgitation can be dynamically indicated on the valve model during the periods when the valves are normally closed (mitral and tricuspid valves: systole; pulmonary and aortic valves: diastole). Furthermore, the location of the leakage area can be indicated in the valve model. The 2D portion indicating the flow phenomenon in the valve model can be dynamic. That is, the 2D portion indicating the flow phenomenon can change according to the current cardiac cycle depicted by the segmented visualization of the heart valve. Therefore, the user can examine the flow phenomenon at different locations on the heart valve.

[0061] According to another embodiment, the 2D portion includes markers comprising a computer graphical representation of the flow phenomenon, indicating the magnitude of the flow, such as contour lines, vector fields, streamlines, and / or color maps. In other words, the scale and size of the leakage area can be displayed on the valve model, and preferably on a topology-based 2D model. That is, the magnitude of the leakage can be displayed via heatmaps or contour maps (isolines) in a visualization of the segmented heart valve and / or in a topology-based 2D model. Thus, the meaning of the valve model can be further enhanced by indicating the location, extent, and direction of the flow phenomenon (e.g., regurgitation).

[0062] According to another embodiment of the invention, the method further includes receiving user input indicating a location or 2D region on the valve model where markings should not be displayed (i.e., no flow phenomena). In other words, the user can define exclusion regions in the valve model. For example, if the heart valve under examination is provided near a second heart valve, there may be lateral flow induced by the second heart valve, which does not describe the function of the heart valve under examination. Therefore, it may be helpful to exclude certain regions of the valve model from displaying flow phenomena.

[0063] According to another aspect, the present invention provides a computer program comprising program code instructions that, when executed by a processor, cause the processor to perform the method of the present invention. The computer program can be any code, particularly code suitable for computer graphics applications.

[0064] In another aspect, the present invention relates to a computer-readable medium comprising a computer program as defined above. The computer-readable medium can be any digital data storage device, such as a USB stick, hard disk, CRROM, SD card, or SSD card. Naturally, the computer program does not need to be stored on such a computer-readable medium for provision to a customer, but can be downloaded via the Internet.

[0065] According to another aspect, the present invention provides a system for analyzing heart valves, the system comprising a user interface configured to receive user input and a control unit configured to perform the methods described above. Any features and advantages of the system and computer program also apply to the method, and vice versa.

[0066] This system can be included in the acquisition modality, particularly in echocardiography systems. Therefore, valve models can be generated immediately during the acquisition of 3D or 4D volumetric datasets. Alternatively, the system can be centrally provided and connected to a database storing multiple 3D or 4D volumetric datasets. In this case, valve models can be created for each dataset and used for research and / or future intervention planning.

[0067] According to one aspect of the invention, a topology-based 2D model of a heart valve bidirectionally mapped to a 3D model of the heart valve is provided, which can be used as an editing control (“valve joystick”) and a parameter display (“valve dashboard”) for the valve. For the valve joystick, user input is mapped onto a topological model on a 4D valve segment. The user can add or delete junctions on the model and change their relative positions. These edits are based on the segmented heart valve displayed on the 3D valve model and tracked over time. Attached Figure Description

[0068] Useful embodiments of the invention will now be described with reference to the accompanying drawings. Similar elements or features are indicated by the same reference numerals in the drawings. In the drawings: Figure 1 This is a schematic diagram of a 3D valve model and a topology-based 2D model of a heart valve according to an embodiment of the present invention. Figure 2 This is a schematic diagram of a 3D valve model and a topology-based 2D model of a heart valve according to an embodiment of the present invention. Figure 3 This is a schematic diagram of several topology-based 2D models according to embodiments of the present invention. Figure 4 This is a schematic diagram of a topology-based 2D model adjusted according to an embodiment of the present invention. Figure 5A This is a schematic diagram of a 3D valve model and a topology-based 2D model of a heart valve according to an embodiment of the present invention. Figure 5B This is a schematic diagram of a visualization of a 3D valve model and a topology-based 2D model of a heart valve according to an embodiment of the present invention. Figure 6 This is a schematic diagram of a user interface according to an embodiment of the present invention. Figure 7 This is an illustration of the projection of a topology-based 2D model onto a 3D or 4D volumetric dataset according to an embodiment of the present invention. Figure 8 This is a diagram of the cutting plane. Detailed Implementation

[0069] In all the accompanying drawings, the same or corresponding features / elements of various embodiments are indicated by the same reference numerals.

[0070] Figure 1 This is a schematic visualization of a 3D valve model 2 and a topology-based 2D model 3 of a heart valve according to an embodiment of the present invention. Figure 1On the left, a visualization of the valve model (in this case, 3D valve model 2) is depicted. This can be usefully overlaid on visualizations, such as the volumetric drawing of segmented heart valves (not shown). Figure 1 On the right, a topology-based 2D model 3 is depicted. The 3D valve model can be derived from a heart valve segmented from a 3D or 4D volume dataset. The topology-based 2D model 3 is created from the 3D valve model, for example, by projecting it onto a 2D plane, as explained above. The heart valve has three leaflets 6. The topology-based 2D model 3 includes three junction points 4 on a closed curve 9 and three junction lines 5 connecting the junction points 4 to the center point 22 of the closed curve 9 (in this case, a circle). The three junction lines 5, together with the corresponding segments of the circle 9 representing the valve annulus, define three 2D portions 16 in the topology-based 2D model 3, each corresponding to a leaflet 6. In other words, the junction points 4 are connected to each other via the center 22 of the topology-based 2D model 3. The three junction points 4 and the three junction lines 5 also exist on a 3D ring 12 representing the valve annulus of the 3D valve model 2. The 3D ring 12 is a closed curve of the 3D valve model 2. Junction point 4 can be adjusted by the user via user interface 10 so that junction point 4 corresponds to the junction of the boundary line of leaflet 6 and valve annulus 12. By dragging and dropping, the topology-based 2D model 2 can be adjusted to fit the individual anatomy of the heart valve (e.g., leaflet 6), and the corresponding 3D valve model can be displayed on a segmented visualization of the heart valve. Junction point 4 can be modified as follows: a. Adding / removing joint points 4 on a topology-based 2D model based on morphology, and b. Drag and drop junction point 4 along closed curve 9. In this case, junction point 4 on the 3D valve model will be along 3D ring 12.

[0071] The 2D portions 16 between the junction lines 5 are labeled with the abbreviations of the corresponding valves according to standard anatomical nomenclature, in this case with the letters "A", "S", and "P". Each portion 16 represents a leaflet 6 of a heart valve. Furthermore, the mapping of 3D coordinates of the topology-based 2D model 3 to 4D volumetric dataset is tracked over a complete cardiac cycle, for example, through time tracking of the 3D ring 12 representing the valve annulus. Morphological nomenclature is mapped to the adapted topology-based 2D model. Figure 1 The arrows in the diagram illustrate the adaptation of junction point 4 in the topology-based 2D model 3, as well as in the visualization of the segmented heart valve or 3D valve model 2. Modifications are mapped to their corresponding counterparts.

[0072] Figure 2This is a schematic visualization of a 3D valve model 2 and a topology-based 2D model 3 according to another embodiment. This embodiment essentially corresponds to the previous embodiment, except that another heart valve is depicted as having only two leaflets. However, the process is the same as in the previous embodiment. Furthermore, in Figure 2 In this model, junction points 4 are directly connected to each other via junction line 5. In other words, the junction line is not routed at the center of the topology-based 2D model. In this case, the junction line can be adjusted by the user to fit the boundary lines of the two leaflets 6 of the heart valve.

[0073] Figure 3 This is a schematic visualization of several topology-based 2D models 3 according to embodiments of the present invention. Figure 3 At the center, the initial topology-based 2D model 3 is depicted. That is, the initial topology-based 2D model 3 is not adjusted to the actual heart valve, but has an initial setup, in this case, three junction points 4 and three junction lines 5. Figure 3 The topology-based 2D model 3 surrounding the initial topology-based 2D model 3 is an adapted topology-based 2D model 3. The topology-based 2D model 3 is created as in the previous embodiments. Therefore, the user can add, delete, and / or shift the junction points 4 to adjust the topology-based 2D model 3 to fit the actual heart valve depicted in the visualization of the segmented heart valve. Furthermore, the labels of the leaflets 6 can be manually modified by the user. Figure 3 In each of the topology-based 2D models 3, the center of the topology-based 2D model 3 is a circle coaxial with the circular closed curve 9 of the topology-based 2D model 3. A label indicating the type of heart valve is provided within the central circle. The 2D parts 16 of the topology-based 2D model are labeled with abbreviations of the corresponding valve names, such as P1, Ps, A, and S.

[0074] Figure 4 This is a schematic visualization of a topology-based 2D model adjusted according to an embodiment of the present invention. That is, in Figure 4 On the left, the initial topology-based 2D model 3 is depicted. Then, the user can use the pointing device 13 to grasp the junction point 4 and follow the model's petal loop or closed curve 9 (reference). Figure 4 The small arrow in the image is used to drag the joint point 4 in the desired direction. The adjustment of the joint point 4 is preferably automatically applied to the 3D valve model. Figure 4(Not depicted in the image), and the 3D valve model is displayed on the 4D echo segment (i.e., a visualization of the segmented heart valve 2). The entire topology-based 2D model 3 can be rotated to fit the alignment of the echo segment. Using the "+" and "-" buttons around the model's valve annulus, it is possible to add and remove junction points 4 in each sector of the topology-based 2D model 3 to fit the number and morphology of leaflets 6. Junction points 4 are adjusted by dragging and dropping along the closed curve 9 on the echo valve annulus 12. In the case of the tricuspid valve, leaflets 6 are labeled according to the nomenclature proposed by Hahn et al. Selected morphologies based on the number and location of junction points 4 are mapped to the proposed nomenclature, and types are assigned in the case of the tricuspid valve. The default type at initialization is the most common type I.

[0075] Figure 5A This is a schematic diagram of the visualization of a segmented heart valve along with a 3D valve model 2 and a topology-based 2D model 3 according to another embodiment of the present invention. In this embodiment, at least one flow phenomenon 7 is additionally indicated in the segmented heart valve 2 and the topology-based 2D model 3. A parameter display is provided that projects the flow phenomenon from 4D echo data (e.g., a 3D or 4D volume dataset including flow information, for example, from Doppler ultrasound) onto the topology-based 2D model 3. Specifically, as an example of flow phenomenon 7, in the segmented heart valve 2, regurgitation is dynamically displayed during the periods when the valve is normally closed (mitral and tricuspid valves: systole; pulmonary and aortic arteries: diastole). The location of the corresponding leakage region 8 on the topology-based 2D model 3 is then displayed. Furthermore, the size of the leakage region 8 is shown as scaled on the topology-based 2D model 3. In addition, the leakage amplitude is displayed on the projection via a heatmap or contour map (isolines). The visualization of flow phenomenon 7 in the topology-based 2D model 3 is achieved by marker 8, which can be changed relative to its visualization properties. The marker also represents the 2D part of the topology-based 2D model 3, whose marker has physiological characteristics, namely flow phenomena.

[0076] Figure 5B This is an illustrative visualization of the segmented heart valves and the topology-based 2D model visualization according to an embodiment of the present invention. Besides... Figure 5B In addition to having only two leaflets, the heart valves in the heart have... Figure 5B Basically corresponds to Figure 5A .

[0077] Figure 6A user interface 10 according to an embodiment of the present invention is shown. In this setup, a segmented heart valve 2 and a topology-based 2D model 3 are depicted on a conventional computer screen 14. The screen 14 may include a panel 15 with buttons and sliders that allow the user to tilt, zoom, move, or otherwise manipulate the segmented heart valve 2 and / or the topology-based 2D model 3. Also in such a user interface 10, a 2D cut plane through a 3D or 4D volumetric dataset may be displayed, wherein at least one 3D region is depicted with color shading characteristics of a label for the corresponding 3D region (not shown). The display may be controlled by a computer 16, such as a PC, including a processor 17 and a hard disk 18. The user interface may have input tools such as a keyboard 19 and / or a mouse (i.e., a pointing device 20).

[0078] Figure 7 The projection of a valve model according to an embodiment of the invention onto a 3D or 4D volumetric dataset 32 ​​is shown. At 30, a topology-based 2D model 3 is visualized. This particular topology-based 2D model 3 includes three junction points 4, namely A, B, and C, on a closed curve 9 (in this case, a circle), centered at a center point 22. Thus, it includes three 2D regions 16, each representing a leaflet, and each 2D portion 16 is surrounded by segments of the closed curve 9 and two junction lines 5. The topology-based 2D model 3 is derived from the corresponding 3D valve model 2. In visualization 31, the 3D valve model 2 is visualized as an overlay on a 3D echo segment 32, where the segmented heart valve is shown in a dynamic volumetric drawing view covering, for example, a time period of one or more cardiac cycles. For each frame of the 4D volumetric dataset (3D video segment), the calculation shown at 34 is performed. The calculation is performed according to the projection direction indicated by arrow 36. This is accomplished by first finding a plane 38 that best fits the 3D valve model 2, wherein the 3D valve model 2 specifically includes a three-dimensional ring 12 representing the valve annulus and a three-dimensional surface surrounded by the ring 12. Preferably, plane 38 is calculated to have a minimum squared distance from this 3D surface. Then, the projection direction 36 is a vector perpendicular to this “average” plane 38, which essentially represents the principal plane of the heart valve. This calculation is preferably repeated for each frame, i.e., for each shape and relative position of the 3D valve model 2 within a 3D or 4D volume dataset of the heart valve, which is visualized at 31 as a 3D video clip 32.

[0079] Therefore, the 3D valve model 2 can be projected onto plane 38 along with all the 2D parts included therein, preferably using parallel projection along the projection direction 36. This results in a projected 3D valve model. Alternatively, a topology-based 2D model 3 can be positioned in plane 38 and aligned with the 3D valve model 2, for example by aligning the position of the center point 22. From plane 38, the valve model, particularly the projected 3D valve model or the topology-based 2D model 3, is projected along the projection direction 36 into the corresponding frame of the 4D volumetric dataset 32, as shown at 40. Alternatively, the 3D valve model 2 can be projected directly along the projection direction 36 without first generating a projected 3D valve model.

[0080] As can be seen in Figure 40, the projection of the 3D valve model 2 onto the volume dataset 32 ​​results in a cylindrical volume of interest 42, which in this case has the full height of the volume dataset 32. Three 2D portions 16a, 16b, and 16c are transformed into 3D regions 26a, 26b, and 26c, each 3D region being substantially similar in shape to a slice of pie. Preferably, the voxels in the volume dataset 32 ​​within each of these 3D regions 26a, 26b, and 26c receive different labels depending on the 3D region they reside in. The different labels, which can be converted into different color shadings for each 3D region, are shown as “dashed lines” for 3D region 26a, “horizontal dashed lines” for 3D region 26b, and “vertical dashed lines” for 3D region 26c. In other embodiments, the 3D valve model may be projected only along a height less than the full height of the volume dataset 32, but rather, for example, up to a predefined multiple of the diameter of the 3D loop 12 or closed curve of the 3D valve model 3.

[0081] Figure 8 This illustrates how the method of the present invention can be used to generate visualizations, wherein voxels from different 3D regions are represented by different colors according to their labels. For example... Figure 7 As shown in section 40, plane 44 can be oriented in any orientation through volume dataset 32 ​​and the volume of interest 42, which is generated by the projection of the topology-based 2D model 3 along projection direction 36 into volume dataset 32. The user can adjust the orientation of plane 44 according to his / her needs to obtain a good view through the volume dataset. A corresponding cutting plane 44 can be generated from volume dataset 32 ​​by means of multi-plane construction. Figure 8As shown, the 2D cutting plane 44 can be visualized, for example, alongside the visualization of the volume dataset 32. In the example shown, the incision through the valve 52 is visible on the 2D cutting plane 44. Since the 2D cutting plane 44 intersects with two 3D regions 26a and 26b (i.e., two regions belonging to different leaflets), the regions belonging to different 3D regions are shown in different color shadings: 3D region 26a is colored one color and shown as a point, and 3D region 26b is colored another color and shown by a horizontally dashed surface. This color shading is maintained even when the user changes the orientation of the cutting plane 44. Therefore, the user can more easily navigate through the volume dataset 32 ​​because he always knows which leaflet region he is navigating through by moving the position and orientation of the observation plane 44.

[0082] The above discussion is intended to illustrate the system only and should not be construed as limiting the claims to any particular embodiment or group of embodiments. Therefore, while the system has been described in particular detail with reference to exemplary embodiments, it should be understood that many modifications and alternative embodiments can be devised by those skilled in the art without departing from the broader and contemplated spirit and scope of the system as set forth in the claims. Consequently, the specification and drawings should be considered illustrative and not intended to limit the scope of the claims.

Claims

1. A computer-implemented method for analyzing the heart valves (52) of a subject, the method comprising the following steps: (a) Provide a 3D or 4D volumetric dataset (32) of the patient’s heart valves (52); (b) Providing valve models (2, 3) of the heart valve (52), wherein the valve models (2, 3) are adapted to the morphology of the heart valve (52), and The valve model (2, 3) includes closed curves (9, 12) representing the valve annulus, wherein the region within the closed curves (9, 12) includes at least one 2D portion (16), each 2D portion (16) corresponding to a morphological or physiological feature of the heart valve (52); and (c) Project the valve model (2, 3) onto the 3D or 4D volume dataset (32) along the projection direction (36) to define the volume of interest (42), wherein the volume of interest (42) includes at least one 3D region (26), each 3D region (26) being defined by the projection of the corresponding 2D part in the at least one 2D part (16); (d) Each voxel in the at least one 3D region (26) is labeled with a label indicating the morphological or physiological characteristics of the corresponding 2D portion (16).

2. The method according to claim 1, further comprising the following step: (e) Provide a visualization of the 3D or 4D volume dataset (32) of the heart valve (52), the visualization including information from the projected valve model (2, 3), in particular, wherein voxels with different labels are visualized differently.

3. The method according to claim 2, wherein, Voxels in at least one 3D region (26) of the volume of interest (42) are represented by colors that depend on their labels.

4. The method according to claim 2 or 3, wherein, The visualization includes visualizing the 2D cutting plane (44) through the 3D or 4D volume dataset (32).

5. The method according to any one of claims 2 to 4, wherein, The voxels within the 3D region (26) of the at least one 3D region are excluded from the visualization and / or represented in neutral colors.

6. The method according to any one of claims 2 to 5, wherein, Voxels outside the volume of interest (42) are not visualized.

7. The method according to any one of the preceding claims, wherein, The valve model (2, 3) includes a plurality of junction points (4) located circumferentially along the closed curve (9), wherein each junction point (4) defines the origin of the junction line (9) between two leaflets (6) of the heart valve (52), and wherein the 2D portion of the at least one 2D portion is partially defined by at least one junction line (5) and corresponds to the leaflet.

8. The method according to any one of the preceding claims, wherein, The 2D portion of at least one 2D portion (16) within the closed curve (9) corresponds to a physiological feature, wherein, preferably, the physiological feature is a flow phenomenon (7).

9. The method according to any one of the preceding claims, wherein, The valve model (2, 3) is a 3D valve model (2) of the valve annulus or a 3D valve model (2) based on the valve annulus. The 3D valve model of the valve annulus is located and oriented within the 3D or 4D volume dataset (32) at the position and orientation of the valve annulus of the heart valve (52). Furthermore, the projection direction (36) is at least approximately perpendicular to the planar extension of the 3D valve model (2) fitted to the valve annulus.

10. The method according to any one of the preceding claims, wherein, The valve models (2, 3) are based on or associated with a topology-based 2D model (3) of the heart valve.

11. The method according to any one of the preceding claims, further comprising the step of receiving user input data via a user interface (10) for creating or adjusting at least a 2D portion (16) within the valve model (2, 3), wherein, Specifically, the topology-based 2D model (3) is displayed to the user, and any adjustments made by the user to the 2D portion (16) of the topology-based 2D model (3) are transmitted to the valve model (2).

12. The method according to any one of the preceding claims, wherein, The visualization is a dynamic 4D visualization of the heart valve (52), wherein the projection direction (36) of the topology-based 2D (3) model is adjusted according to the movement of the heart valve (52).

13. The method according to any one of the preceding claims, wherein, The volume of interest (42) extends into the 3D or 4D volume dataset (32) up to a set height along the projection direction (36), preferably a height proportional to the size of the topology-based 2D (3) model.

14. A computer program including program code, which, when executed on a control unit (16), is configured to perform the method according to any one of the preceding claims.

15. A system (52) for analyzing heart valves, comprising a user interface configured to visualize a 3D or 4D volumetric dataset (32), and a control unit configured to perform the method according to any one of claims 1 to 13.

Images