COMPUTER IMPLEMENTED METHOD, COMPUTER PROGRAM AND SYSTEM FOR ADJUSTING THE MORPHOLOGY OF A CARDIAC VALVE MODEL - Patent application

Abstract

A computer-implemented method for adjusting the morphology of a heart valve model segmented from a 3D or 4D volumetric data set, providing a 3D visualization of the segmented heart valve on a user interface, the heart valve being segmented from the 3D or 4D volumetric data set, generating a topology-based 2D model of the heart valve, displaying the 2D model on the user interface, and receiving user input data via the user interface for further adjusting the 2D model. Additionally, a computer program and system for analyzing a heart valve are provided.

Description

[Technical field]

[0001] The present invention relates to a computer-implemented method for adjusting the morphology of a heart valve model segmented from a 3D or 4D volume data set, a computer program and a system for analysing a heart valve. [Background technology]

[0002] Some anatomical structures are difficult to visualize due to the hidden apparatus and / or movement of the anatomical structures in the human body. An example of such an anatomical structure is the tricuspid valve, which is particularly difficult to visualize in echocardiography due to its location and structure. Specifically, the leaflets of the tricuspid valve can only be seen in two frames throughout the cardiac cycle, and often not all of them are in the same frame. Segmentation and tracking of the leaflets in echocardiographic clips has been unsuccessful so far. More and more implant devices and procedures are emerging to address common pathologies such as tricuspid regurgitation. These procedures need to be thoroughly planned to be as safe and successful as possible. Novel imaging approaches in echocardiography are required to plan and navigate these procedures. Knowledge of the morphology of the leaflets is mandatory for some procedures. Furthermore, the anatomical landmarks of the heart valves are not very intuitive to mark and are difficult to edit in a moving 4D clip.

[0003] US Patent Publication No. 2005187461 discloses a computerized method for facilitating cardiac intervention, the method including the steps of inputting patient data, generating a computerized interaction model of the heart based on the patient data, the model including features, simulating at least one proposed cardiac intervention treatment by adding or deleting features from the model, and determining the effect of the proposed cardiac simulation on the overall model. The simulation can be repeated to allow a user to determine an optimal cardiac intervention. Additionally, a template can be generated from the model for use as a guide during the cardiac intervention.

[0004] US2020082531 shows a device for dynamically assessing moving objects from a series of consecutive volumetric image frames of such an object separated in time by a fixed time interval by identifying an object of interest in at least one image of a sequence of such an object, segmenting the object to identify a contour of the object, propagating the identified object contour to other images of the sequence, and performing a dynamic analysis of the object based on the propagated object contour. The white paper "Quantifying Heart Valves: From Diagnostic to Personalized Valve Repair" by Tommaso Mansi et al. (04 2016) describes a method for valve modeling and editing, where it is proposed to provide a combination of editing views such as parallel or rotational cutting and smart mesh editing.

[0005] However, it remains cumbersome to assess the topology of the heart valve and therefore difficult to prepare for the surgical procedure. Summary of the Invention [Problem to be solved by the invention]

[0006] It is therefore one object of the present invention to provide a method and apparatus that can provide an easy and intuitive way to assess anatomical structures, even when the anatomical structures are difficult to visualize. [Means for solving the problem]

[0007] The present invention solves this problem by means of a computer program comprising the features of claim 14 and by means of a system comprising the features of claim 15 and by means of a computer implemented method comprising the features of claim 1.

[0008] According to one aspect, the present invention provides a computer-implemented method for adjusting the morphology of a heart valve model segmented from a 3D or 4D volumetric data set, comprising the steps of: providing, on a user interface, a 3D visualization of the segmented heart valve, said heart valve being segmented from a 3D or 4D volumetric data set; generating a topology-based 2D model of the heart valve, the topology-based 2D model being a simplified visualization of properties of the heart valve; displaying the 2D model on the user interface; receiving user input data via the user interface for further adjusting the 2D model; The present invention provides a method comprising:

[0009] According to one embodiment of the present invention, the method projects a topology-based model 2D based on the actual segmentation of the valve annulus. The model can be edited to depict any type of morphology of the heart valve. In other words, the 2D model can be basically generated based on the segmented heart valve and then further manually adjusted by the user to represent the morphology of the segmented heart valve. As a result, the 2D model can be accurately adjusted to the morphology of the actual heart valve. The 2D model can also be called a symbolic valve model, which can be used as a research and communication tool for valve studies, inter-patient comparison, longitudinal comparison of valves of one patient, disease progression analysis, consultation between doctors, etc. The method can provide a digital joystick that allows the user to transfer the topology on a 3D or 4D dataset, as well as a valve dashboard to visualize phenomena from the 3D or 4D dataset on the topology. The method is applicable to all heart valves.

[0010] The method can be used to adjust the morphology of the 2D model. Morphology can include aspects of the appearance (e.g., shape, structure, pattern, size, etc.) of the heart valve, i.e., external morphology (or aenomy), as well as the morphology and structure of the components of the heart valve, such as the leaflets, annulus, etc., i.e., internal morphology (or anatomy). This is in contrast to physiology, which deals primarily with function. Morphology can be considered a branch of life science that deals with the study of the macroscopic structure of anatomy and its components.

[0011] The 3D or 4D volume data set may include information of an anatomical structure in three dimensions (3D) or four dimensions (4D). From the anatomical structure, the heart valves may be segmented in a known manner. For example, the segmentation may be performed as shown in "Mitral Valve Segmentation Using Robust Nonnegative Matrix Factorization" by DROGE, Hannah et al. (Journal of Imaging, 2021, 7. Jg., Nr. 10, S. 213), which is incorporated herein by reference. Furthermore, the 3D or 4D volume data set may include the heart as an anatomical structure. The three-dimensional information may define the spatial position of a point in space. Furthermore, the fourth dimension may further include time, i.e., the position of the point over a period of time. In other words, the 4D volume data set may be, for example, a video clip including multiple frames (i.e., 3D information) covering a heartbeat cycle. The 3D or 4D volume data set may be provided in DICOM (Digital Imaging and Communications in Medicine) format. However, the 3D or 4D volume data set may be provided in any other suitable format, so long as it contains 3D or 4D information of the anatomical structure.

[0012] Heart valves are sometimes referred to as dynamic anatomical structures, since their dimensions change over time. A 3D or 4D volume data set may be captured in a series of three-dimensional medical images acquired from a human subject over a period of time, the images may be pre-operative images, but also intra-operative images. A 3D or 4D volume data set may include medical images. A sequence of 3D medical images may be referred to as 4D images. Three-dimensional (3D) images are usually digital images, for example of the DICOM standard, i.e. they include a three-dimensional array of voxels, each voxel including a grayscale value. Such 3D medical images have typically been acquired from a field of view including the dynamic anatomical structures using medical imaging modalities such as MR, computed tomography (CT), positron emission tomography (PET) or ultrasound (US). When the anatomical structure is the heart, ultrasound, in particular transesophageal echocardiography (TEE), may be advantageously used. One 3D image from a time sequence of 3D images is also referred to as a "frame" in the following. The 3D images are acquired at a frame rate of, for example, 5 to 100 images per second, preferably 20 to 60 images per second, allowing a smooth representation of dynamically moving anatomical structures, and are then displayed in cine mode. The time period is typically at least one cycle of the periodic motion, for example at least one heart beat.

[0013] 3D visualization (also called 3D or 4D echo) can be a 3D volume rendering of a 3D or 4D volume dataset. 3D volume rendering can comprise a set of techniques used to display a 2D projection of a 3D image dataset. However, while 3D volume rendering helps to form a mental 3D model of the anatomical structure, it strongly depends on the image quality as well as on the settings selected (thresholding, smoothing, etc.). Volume rendering has the advantage that it is suitable for more complex anatomical structures such as valve leaflets or tricuspid leaflets. Furthermore, parameters such as threshold, opacity, contrast can be adjusted "live", i.e. with immediate effect while looking at the volume rendering. On the other hand, volume rendering strongly depends on the image quality and therefore may be too complex for easy interpretation if used alone. Alternatively or additionally, the 3D visualization of the segmented heart valve may be a shape / surface 3D model. The surface 3D model may be a simplification of the heart valve, e.g. a triangular surface 3D model. The 3D model may comprise the number of points spanned by a line or surface per frame. It may also be a mathematical model, e.g. a parameterized model such as a surface or a volume spanned by a spline curve. The 3D model may be dynamic, i.e. it follows the movement of an anatomical structure (e.g. a heart valve) over a period of time. The purpose of a dynamic 3D model is to visualize the essential components of a heart valve (e.g. the leaflets) and to visualize the changes of the heart valve over time. A 3D model has the advantage that it does not require much computational power compared to a 3D rendering and still gives the user the essential features of the heart valve. However, a 3D model alone does not allow the user to track all the features of the heart valve over a complete heartbeat cycle. For example, the leaflets of a heart valve (e.g. the tricuspid valve) may only be seen in two frames over the entire cardiac cycle, and often not all of them are seen in the same frame. This limitation may be due to the nature of the imaging modality, which does not allow the acquisition angle to be freely changed, as in echocardiography.In this respect, a topology-based 2D model assists the user (more details follow below).

[0014] A three-dimensional visualization environment may be provided for visualizing a segmented heart valve visualization over a period of time to allow a user to view and analyze the heart valve (i.e., the region of interest). That is, a visualization of the segmented heart valve and a topology-based 2D model may be depicted within the same visualization environment. The visualization environment may be used in particular to view and analyze the heart valve for planning an intervention and / or to determine the exact size, shape and location of an implant to be implanted in a future intervention.

[0015] The visualization environment may be a three-dimensional visualization environment or a virtual reality environment. By "virtual reality" we mean any computer-generated visualization that provides a true three-dimensional experience of the depicted structure. Thus, the virtual reality (VR) environment of the present invention provides specifically visual feedback, but may also allow other types of sensory feedback, such as hearing. The VR environment may also be an augmented reality environment, where the user still sees the real environment, but where VR objects (e.g., volume renderings and dynamic models) are overlaid or superimposed on the real objects, or a mixed reality, where real-world objects are superimposed on the virtual scene.

[0016] The segmented heart valve may be part of an anatomical structure contained within a 3D or 4D volume data set. The segmented heart valve may be automatically acquired based on the 3D or 4D volume data set. That is, the 3D visualization of the segmented heart valve may depict only the heart valve. The surrounding anatomical structures may be omitted. Thus, the clarity of the visualization is improved and the user can focus on the essential parts of his evaluation.

[0017] The user interface may be a device configured to receive user input and visualize the 3D visualization of the segmented heart valve and the 2D model to the user. That is, the user interface may be an interface between a human and a machine. Preferably, the user interface may comprise a display configured to display the 3D visualization and the 2D model to the user. In addition, the user interface may comprise an input device that may be operated by the user and input user commands to the user interface. In particular, the display may be a touch-sensitive display. In this case, an additional input device may not be necessary since the user can input his / her commands via the display. The user interface may comprise a processor and a storage device for storing information. Preferably, the user interface is a computer.

[0018] A topology-based 2D model is a two-dimensional simplified visualization of a heart valve. A topology-based 2D model may be a standardized simplification of an actual segmentation of a heart valve. The simplification may mean that a three-dimensional representation of a heart valve (i.e., a segmented heart valve) is transferred to a two-dimensional visualization of the heart valve (i.e., a topology-based 2D model). The annulus of a segmented heart valve may be freely located in space, and points on the annulus may be described by three variable coordinates, each of which may be different. On the other hand, the annulus of a topology-based 2D model may be described by points with only two variable coordinates (i.e., the coordinates are the same for all points).

[0019] Preferably, the annulus may be represented by a circle. The length or circumference (e.g., circumferential length or circumferential shape) of the circle may correspond to the length or circumference of the valve annulus. The topology of the heart valve may be characterized by a number of leaflets of the heart valve. Furthermore, the topology of the heart valve may be further characterized by the position, area, and / or shape of the leaflets. Thus, the topology-based 2D model may provide a simplified visualization of the properties of the heart valve. Furthermore, the circumference of the circle that surrounds or defines the topology-based 2D model may be the same length regardless of the actual length of the heart valve annulus. Thus, the topology-based 2D model may be further simplified. Furthermore, the topology-based 2D models of different heart valves may be easily compared to each other.

[0020] The generation of the topology-based 2D model may be performed in an automated manner. That is, a rough topology-based 2D model may be automatically generated. The user may then adjust the topology-based 2D model to best fit the 3D visualization of the segmented heart valve (more details of the adjustment follow below). Furthermore, a standard topology-based 2D model may be provided first. The user may then adjust the topology-based 2D model to the visualization of the segmented heart valve. That is, the standard topology-based 2D model may always be the same, independent of the particular heart valve under examination. Furthermore, the topology-based 2D model may be a static symbolic representation of the heart valve. In other words, even if the 3D or 4D visualization of the segmented heart valve is depicted, for example, over a heartbeat, the topology-based 2D model may remain static (i.e., it does not move or modify its shape or configuration). Thus, the user may adapt the topology-based 2D model to the dynamic (4D) visualization of the heart valve using the static symbolic representation of the topology-based 2D model. Preferably, the user may immediately observe the results of the adaptation in the dynamic visualization of the heart valve. According to further embodiments, extensions to more dynamic surface models are possible (e.g., editing the shape of the joint lines). The model can be edited to depict any type of morphology of the heart valve. The topology-based 2D model can be depicted on the same user interface as the segmented heart valve. Receiving user input can mean that the user inputs a command to adjust the topology-based 2D model. This can be done directly by adjusting the topology-based 2D model. Alternatively or additionally, the user can also adjust the segmented heart valve and the topology-based 2D model can be adjusted indirectly by the user input.

[0021] According to a further embodiment, the topology-based 2D model is generated by mapping the segmented heart valve onto a plane to scale. Thus, the ratio along the circumference of the annulus of the segmented heart valve can be maintained. That is, the topology-based 2D model can be projected onto a plane or a surface. Furthermore, the topology-based 2D model may be distorted during projection onto the 2D surface. In this case, the topology-based 2D model is not a developable surface (it is not a ruled surface either). In an alternative embodiment, the topology-based 2D model is generated based on a developable surface of the segmented heart valve. For example, the topology-based 2D model may be a developable surface. In mathematics, a developable surface (or toss, classical) is a smooth surface with zero Gaussian curvature. That is, it is a surface that can be flattened onto a plane without distortion (i.e., it can be bent without stretching or compression). Conversely, it is a surface that can be made by deforming (i.e., "folding", "bending", "rolling", "cutting" and / or "gluing") a plane. In three dimensions, all developable surfaces are ruled surfaces (but not vice versa). In this case, the topology-based 2D model may be a developable surface based on a 3D visualization of the segmented heart valve. In this case, the topology-based 2D model may have dimensions related to the segmented 3D segmentation of the heart valve. That is, the circumferential length of the topology-based 2D model may correspond to the circumferential length of the segmented heart valve. Thus, further information can be derived from the topology-based 2D model.

[0022] According to a further embodiment of the invention, the topology-based 2D model is displayed on a user interface next to the 3D visualization of the segmented heart valve. That is, the topology-based 2D model and the 3D visualization of the segmented heart valve may be displayed on the same interface (e.g., display). Preferably, the 3D visualization of the segmented heart valve and the topology-based 2D model are depicted directly adjacent to each other. In this case, the user can easily adjust the topology-based 2D model while looking at the 3D visualization of the segmented heart valve. Thus, the process of adjusting the topology-based 2D model can be further facilitated. Preferably, the topology-based 2D model is displayed next to the segmented heart valve and is additionally projected onto the segmented heart valve. That is, the commissure points and / or commissure lines are projected onto the segmented heart valve. In this way, the topology-based 2D model can be easily adapted to the segmented heart valve.

[0023] According to a further embodiment, the 2D model includes commissure points that define at least one commissure line that indicates a line of separation between at least two leaflets of the heart valve. The commissure points may define a specific area where the leaflets meet at their junction with the valve annulus. For example, the points may be the locations where the anterior and posterior leaflets meet at their junction with the valve annulus. The commissure points may be used to define segments of the leaflets. More specifically, by connecting the commissure points with straight lines, the leaflets of the heart valve can be visualized in a simplified manner. Furthermore, the commissure points may describe fine structures that can be identified by two anatomical landmarks, for example, the axes of the corresponding papillary muscles and commissural chordae that have a specific fan-like configuration. A few millimeters of valve tissue separate the free edge of the commissure point from the valve annulus. This area must be observed in case of prolapse of the commissure point as well as the corresponding anterior and posterior leaflet segments (e.g., labeled P3 and A3), for example, by resection, otherwise residual regurgitation will occur in this area. The commissure points may be depicted on both the topology-based 2D model and the 3D visualization of the segmented heart valve. The commissure points depicted on the topology-based 2D model and the 3D visualization of the segmented heart valve may be related to each other such that both have the same relative position with respect to a reference point of the heart valve. Such a reference point may be, for example, the center of the heart valve. When a commissure point on the topology-based 2D model or the 3D visualization of the segmented heart valve is moved, the corresponding commissure point of the other topology-based 2D model or the 3D visualization of the segmented heart valve may be automatically moved accordingly. Thus, the user may compare the position of the commissure point with the segmented heart valve of the 3D visualization of the segmented heart valve. In other words, the user may evaluate whether the commissure point is located at the junction of the two leaflets and the annulus in the 3D visualization of the segmented heart valve. This may be done in any available frame of the visualization of the segmented heart valve. The commissure points may define a commissure line. More specifically, the commissure lines may extend from the annulus (eg, from the circle that defines the outer contour of the topology-based 2D model) to the center of the topology-based 2D model.More specifically, each commissure line may extend from one commissure point to the center of the topology-based 2D model. The commissure line may be a straight line connecting two commissure points, preferably through the center of the topology-based 2D model. The center of the topology-based 2D model may be represented by an inner circle that may be coaxial with the annulus. The commissure line may define a connecting line between two adjacent leaflets of the heart valve. In other words, the commissure line may define the edges of the two adjacent leaflets. The commissure line may be a rough representation of the boundary between two adjacent leaflets, since in reality the leaflets do not have a straight boundary line. That is, the commissure line may indicate the approximate extent of the boundary between the leaflets. A sector between the commissure lines may define the leaflets.

[0024] According to a further embodiment of the invention, the user input includes an instruction to modify the position of at least one commissure point of the topology-based 2D model. If the user wishes to adjust the position of one or more of the commissure points, the user can input an instruction to modify the position of at least one commissure point in the 3D visualization of the segmented heart valve and / or the topology-based 2D model. For example, the user can virtually grasp the commissure point to be modified and move it to a desired position on the valve annulus (or to the outer contour of the topology-based 2D model). The user can modify the commissure point in the 3D visualization of the segmented heart valve and / or the topology-based 2D model. The commissure point in the 3D visualization of the segmented heart valve or the topology-based 2D model modified by the user automatically adjusts its position according to the user's instructions. The corresponding commissure point in the other 3D visualization of the segmented heart valve or the topology-based 2D model, whose position is not directly set by the user, automatically and simultaneously adjusts its position to match the position of the commissure point directly set by the user. Preferably, the user modifies the topology-based 2D model (i.e., the commissure points in the topology-based 2D model) and observes the modification of the change in the position of the commissure points in the 3D visualization of the segmented heart valve. For example, by drag and drop, the commissure points in the topology-based 2D mode can be adjusted to fit the individual anatomical structure of the valve leaflets and displayed on the actual segmentation (i.e., the 3D visualization of the segmented heart valve). The 3D visualization of the segmented heart valve and the topology-based 2D model can be tracked over time. Pre-procedure planning can also be used during the procedure by projecting the model (e.g., commissure points, commissure lines, etc.) onto new image data. In this case, the topology-based 2D model can be considered as a virtual "joystick" that allows the user to adapt the position of the commissure points in the 3D visualization of the segmented heart valve.

[0025] According to an embodiment of the present invention, the commissure points and commissure lines are displayed in both the topology-based 2D model and the 3D visualization of the segmented heart valve. That is, the relative position of the commissure points with respect to a reference point may be the same in both the topology-based 2D model and the 3D visualization of the segmented heart valve. The reference point may for example be the canter of the heart valve. Alternatively, any other landmark may be used as a reference point. The representation of the commissure points and commissure lines in the 3D visualization of the segmented heart valve may provide an opportunity to verify that the commissure points are well located in different images of a 3D or 4D volume data set. The different images may be acquired at different times of the cardiac cycle and / or at different acquisition angles. Thus, the commissure points in each image may be adjusted to match the heart valve. Thus, the topology-based 2D model can be generated with high accuracy.

[0026] According to further embodiments of the present invention, the user input includes instructions to add new commissure points and / or delete commissure points. That is, the initial topology-based 2D model may include three different leaflets. That is, the initial topology-based 2D model may have three commissure lines. However, depending on the type of valve being examined, the heart valve may have more or less leaflets. For example, the mitral valve has only two leaflets, while the tricuspid valve may have more leaflets. Furthermore, the tricuspid valve may have a different number of leaflets depending on the individual. That is, each person may have a different number of leaflets in the tricuspid valve. Thus, the topology-based 2D model can be adapted to the particular heart valve being examined by adding commissure points or by deleting commissure points. For example, the user can add further commissure points by clicking on a circle that defines the outer contour of the topology-based 2D model. Furthermore, a new commissure line may be automatically generated extending from the new commissure point to the center of the topology-based 2D model. Thus, further leaflets may be depicted by the topology-based 2D model. In a similar manner, commissure points can be deleted. Thus, the commissure lines defined by the deleted commissure points may also be deleted. As a result, the topology-based 2D model may have one fewer leaflet. For example, a user may select a commissure point to be deleted and then provide instructions to delete that commissure point. Thus, the topology-based 2D model may be adapted to the segmented heart valve under inspection.

[0027] According to a further embodiment of the present invention, the method further comprises labeling a plurality of leaflets defined by at least one commissure line. That is, the sectors between the commissure lines may be labeled according to the morphological definition of each leaflet defined by the sector. The labels may correspond to commonly used indices of each leaflet. For example, the tricuspid valve may comprise an anterior leaflet, a posterior leaflet, and a medial leaflet. The aortic valve may comprise a left coronary leaflet, a right coronary leaflet, and a non-coronary leaflet. For example, the mitral valve has two leaflets. The anterior leaflet may have a semicircular shape and may be attached to two-fifths of the annular circumference. There is continuity between the anterior leaflet of the mitral valve and the left coronary and non-coronary leaflets of the adjacent aortic valve, called the aortic-mitral curtain. The movement of the anterior leaflet also defines the important boundary between the inflow tract (during diastole) and the outflow tract (during systole) of the left ventricle. The posterior leaflet of the mitral valve may have a square shape and may be attached to about three-fifths of the annular circumference of the heart valve. The posterior leaflet typically has two well-defined indentations that divide the leaflet into three individual scallop shapes identified as P1 (anterior or medial scallop shape), P2 (middle scallop shape), and P3 (posterior or lateral scallop shape). The three corresponding divisions of the anterior leaflet are A1 (anterior division), A2 (middle division), and A3 (posterior division). This nomenclature can be considered as a label for a topology-based 2D model. Moreover, this nomenclature is an important tool for describing a particular anatomical division anatomy between the echocardiographer and the surgeon. The indentations can be used to identify or analyze the morphology of each leaflet and thus to label each leaflet according to the above nomenclature. The labeling may be done automatically based on the identified morphology and / or location of each leaflet. For example, the angle at which the 3D or 4D volume data set is acquired may be known such that it is known which leaflets are expected and / or visible at a particular location on the heart valve depending on the acquisition angle. As a result, automatic labeling can be performed with high accuracy. For example, a learning algorithm may be provided to identify each leaflet of the heart valve.That is, leaflet morphology information may be input as input data to a learning algorithm, and the learning algorithm may output leaflet labels as output data. Thus, the learning algorithm may be pre-trained using training data that includes desired output data for particular input data. Using a learning algorithm to label the leaflets of a heart valve may speed up the process of generating a topology-based 2D model.

[0028] According to a further embodiment of the invention, the visualization of the segmented heart valve further includes structures surrounding the heart valve. The heart valve periphery may be the tissue surrounding the heart valve under examination. More specifically, the supporting structures of the heart valve may be shown in the visualization of the segmented heart valve. Thus, the commissure points may be set more accurately, since the supporting structures may help to define where the boundary between two adjacent leaflets actually is. For example, in a mitral valve, a part of the posteromedial papillary muscle and / or a part of the anterolateral papillary muscle may be visualized together with the segmented heart valve. Furthermore, the visualization of the segmented heart valve periphery may help to define the orientation of the segmented heart valve. Furthermore, the information of the segmented heart valve periphery may also be input to a learning algorithm for labeling the leaflets of the heart valve (see also overview above). Thus, the learning algorithm may be able to more accurately and / or select a particular valve by recognizing known landmarks in the surrounding tissue of the heart valve.

[0029] According to a further embodiment of the present invention, the visualization of the segmented heart valve is a dynamic 4D visualization of the heart valve. That is, the segmented heart valve may be visualized in four dimensions (4D). The fourth dimension may be time. That is, the 4D visualization of the segmented heart valve may dynamically represent the heart valve. For example, the segmented heart valve may be visualized in motion over the time span of a cardiac cycle. In other words, the segmented heart valve may be visualized by a series of frames such that the changes of the heart valve over time can be seen, preferably at a minimum of 20 frames per second. Thus, the segmented heart valve may be observed by a user over a cardiac cycle. The topology-based 2D model may be projected onto the segmented heart valve even when the segmented heart valve is moving. On the other hand, the topology-based 2D model is static and does not move. As a result, the topology-based 2D model may be adjusted to multiple different frames of the segmented heart valve. As a result, the commissure points (and therefore the commissure lines) may be accurately set by the user. In addition, the time period during which the segmented heart valve is visualized may also include changes in the acquisition angle of the 3D or 4D volume data set. Thus, the heart valve may be defined by commissure points and commissure lines that are only visible at different acquisition angles. As a result, a topology-based 2D model may be accurately generated based on the segmented heart valve even if the leaflets of the heart valve are not visible in one frame.

[0030] According to yet another embodiment of the present invention, the method projects a flow phenomenon on a 3D visualization of the heart valve based on the flow information, identifies a location on the topology-based 2D model that corresponds to the location of the flow phenomenon, and provides a marker at the identified location.

[0031] In the prior art, the projection of the leak location on the valve is not intuitive for the user. The diseased area for the procedure cannot be easily marked and must be estimated with the current state of the art. Currently, many of these procedures are performed without a specific plan, increasing the time on the operating table as well as the rate of abandonment due to method or device incompatibility. If image-based planning is performed, it is done in a fully manual manner that takes up valuable physician time and presents high variability and potential for error.

[0032] In this embodiment, an easy way is provided to show leak flow in a topology-based 2D model. Thus, planning of interventions can be made easier and more accurate. Flow phenomena can be derived from Doppler information contained in 3D or 4D volume data sets. Thus, flow (e.g., of blood) can be visualized. The Doppler information may be in the same coordinate system as the segmented heart valve.

[0033] Thus, the flow phenomenon can be projected in the correct position relative to the segmented heart valve. This position of the flow phenomenon in the visualization of the segmented heart valve can then be transferred to the topology-based 2D model. This can be done by reference points present in both the visualized segmented heart valve and the topology-based 2D model. Such reference points may be some or all of the commissures and / or centers of the heart valve. The position of the flow phenomenon is then known in the topology-based 2D model and can be indicated by markers. That is, according to the present embodiment, a parametric display in the topology-based 2D model is provided that can include additional information. This can be realized by projection from a 4D echo (as an example of a segmented heart valve) to a topology model (as an example of a topology-based 2D model). That is, a valve dashboard can be provided that includes information on the morphology and / or functionality of the valve under examination. For example, regurgitation can be dynamically shown on the topology-based 2D model during periods when the valve is typically closed (mitral, tricuspid, systole, pulmonary, aortic, diastole). Furthermore, the location of the leakage area may be indicated in the topology-based 2D model. The markers indicative of flow phenomena in the topology-based 2D model may be dynamic. That is, the indicators of the valve leaflets and annulus in the topology-based 2D model may be static (i.e., may not move or change), but the markers indicative of flow phenomena may change depending on the cardiac cycle currently depicted by the visualization of the segmented heart valve. Thus, the flow phenomena may be inspected by the user at different positions of the heart valve. In one embodiment, the user may select a specific point in the cardiac cycle to visualize the flow phenomena at that specific time in the topology-based 2D model. Thus, the meaning of the topology-based 2D model may be further enhanced. In addition, the standardized setting of the topology-based 2D model allows different patients or pathologies to be compared.

[0034] According to a further embodiment of the invention, the marker comprises a computer graphic representation of the flow phenomenon showing the magnitude of the flow phenomenon, in particular isolines, vector fields, streamlines, and / or color maps. In other words, the magnitude and area of ​​the leak may be displayed on the topology-based 2D model. That is, in the visualization of the segmented heart valve and / or the topology-based 2D model, the magnitude of the leak through a heat map or a contour graph (isolines) may be displayed. Thus, the meaning of the topology-based 2D model may be further increased by showing the location, extent, and direction of the flow phenomenon (e.g., regurgitation). The flow phenomenon may be visualized and calculated in several ways (e.g., as a vector field or isovelocity shell) in the visualization of the segmented heart valve. This information may then be projected onto the topology-based 2D model. The area of ​​the projection may be calculated based on the information contained in the 3D or 4D volume data set. The location and size of the area may then be projected onto the topology-based 2D model for easier localization in procedure planning. The local extent of the leak may be shown by a contour line or a heat map. Thus, the intervention may be precisely planned in advance. Thus, the occurrence of unexpected situations during an intervention can be significantly reduced.

[0035] According to yet another embodiment of the present invention, the method further includes receiving a user input indicating a location on the topology-based 2D model where the marker should not be displayed. That is, the user can define an area or sector in the topology-based 2D model where the marker (i.e., no flow phenomenon) should not be displayed. In other words, the user can define an exclusion region in the topology-based 2D model. For example, if a heart valve under examination is provided adjacent to a second heart valve, there may be a side flow induced by the second heart valve that does not describe the function of the heart valve under examination. Therefore, it may be useful to exclude some regions of the topology-based 2D model from displaying the flow phenomenon. This can improve the convenience of the examination. The user can anticipate flow phenomena near the commissure lines, and these lines indicate the borders between adjacent valve leaflets (at least the state where the heart valve is closed or should be closed). Therefore, the exclusion region may be defined in an area away from the commissure lines.

[0036] According to another aspect, the invention provides a computer program comprising program code instructions which, when executed by a processor, direct the processor to perform the method of the invention. The computer program may be any code, in particular code suitable for computer graphics applications.

[0037] In a further aspect, the present invention is directed to a computer readable medium comprising the computer program defined above. The computer readable medium can be any digital data storage device, such as a USB stick, a hard disk, a CR-ROM, an SD card, or an SSD card. Of course, the computer program does not have to be stored on such a computer readable medium to be provided to the customer, but can also be downloadable via the Internet.

[0038] According to another aspect, the present invention provides a system for analyzing a heart valve, comprising:

[0039] A user interface configured to receive user input and a control unit configured to perform the method.

[0040] The system may be included in an acquisition modality. Thus, a topology-based 2D model may be generated on the fly during acquisition of a 3D or 4D volumetric data set. Alternatively, the system may be provided centrally and connected to a database in which multiple 3D or 4D volumetric data sets are stored. In this case, a topology-based 2D model may be generated for each data set and used in studies and / or for future intervention planning.

[0041] According to one embodiment of the present invention, the topology-based 2D model is displayed on a 4D echo clip of a valve where the annulus has already been segmented. The topology-based 2D model can be rotated to fit the alignment of the echo clip. Via the "+" and "-" buttons on the user interface commissure points can be added and removed around the topology-based 2D model annulus in each sector of the topology-based 2D model to fit the number and morphology of the valve leaflets. Drag and drop of the commissure points along the topology-based 2D model annulus can adjust the commissure points on the segmented heart valve annulus. The leaflets are labeled according to the nomenclature proposed by Rebecca T. Hahn, Ludwig T. Weckwig T. Weckbach, Thilo Noack, Nadira Hamid, Mitsunobu Kitamura, Richard Bae, Philipp Lurz, Susheel K. Kodali, Paul Sorajja, Jerg Hausleiter, Michael Nabauer (JACC: Cardiovascular Imaging, Volume 14, Issue 7, 2021, Pages 1299-1305, ISSN 1936). All views of the topology-based 2D model for visualization of the segmented heart valve clips can be tracked through one complete cardiac cycle. The selected morphology based on the number and location of commissures can be mapped to the proposed nomenclature and the heart valve type can be assigned in the case of tricuspid valve. The default type at initialization is Type I, which is the most common.

[0042] In the topology-based 2D model, the valve leaflets can be continuously visualized even during periods of limited visibility in the visualization of the segmented heart valve (i.e., echo) clip. An alternative embodiment that can be used as an accelerated workflow option is a topology-based 2D model pre-selection based on a graphical representation of the morphology. Through the selection of the valve type, the topology-based 2D model can be displayed in the visualization of the segmented heart and, through landmarks (i.e., commissures), can be aligned to the visualization of the segmented heart itself. The landmarks can be tracked throughout one cardiac cycle.

[0043] The method or system of the present invention can be preferably implemented in the context of measuring 4D ultrasound data sets. The method or system of the present invention can be considered as a digital joystick that allows the user to transfer the topology-based 2D model on the 4D data set, as well as a valve dashboard for visualizing phenomena from the 4D data set on the topology-based 2D model. It can be applicable to all heart valves. Furthermore, the present invention can be used in imaging modalities such as cardiac CT (Computed Tomography) and MRI (Magnetic Resonance Imaging). The labeling of leaflets based on morphology can be standardized by the use of nomenclature in this technology. The resulting dynamic labels can also be useful in interventional navigation and live guidance such as Philips EchoNavigator. The topology-based 2D model can also be used as a research and communication tool for valve studies, inter-patient comparisons, longitudinal comparisons of valves of one patient, disease progression analysis, consultation between physicians, etc.

[0044] In summary, cardiac valves such as the tricuspid and mitral valves can be visualized with 4D echocardiography. This allows for diagnosis and subsequent planning of treatment. For planning of many procedures, it is essential to know and mark the morphology and position of the valve leaflets, as well as the location and magnitude of regurgitation. The anatomical landmarks of the valves are not very intuitive to mark and are difficult to edit in a moving 4D clip.

[0045] According to one aspect of the invention, a topological model is proposed that is bidirectionally mapped onto the valve segmentation that can act as both an editing control ("valve joystick") and a parametric display ("valve dashboard") of the valve. In the case of the valve joystick, user input is mapped onto the topological model on the 4D valve segmentation. The user can add or remove commissure points on the model and change their relative positions. These edits are displayed on the segmented model and tracked over time. According to a further aspect of the invention, the valve dashboard maps the localization and magnitude of regurgitant flow through the valve from the 4D echo to the topological model, thus enabling better localization identification.

[0046] Single features of the above-described embodiments can be combined with other features or other embodiments, thus forming new embodiments. Advantages and configurations of features are applicable to the new embodiments. Configurations and advantages defined in relation to the method are also applicable to the device and vice versa.

[0047] Useful embodiments of the present invention will now be described with reference to the accompanying drawings, in which similar elements or features are designated with the same reference numerals, and in which: [Brief description of the drawings]

[0048] [Figure 1] FIG. 1 is a schematic diagram of a segmented heart valve and a visualization of a topology-based 2D model according to an embodiment of the present invention. [Diagram 2]FIG. 1 is a schematic diagram of a segmented heart valve and a visualization of a topology-based 2D model according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a schematic diagram of multiple topology-based 2D models, according to an embodiment of the present invention. [Figure 4] FIG. 2 is a schematic diagram of a topology-based 2D model adjusted according to an embodiment of the present invention. [Figure 5A] FIG. 1 is a schematic diagram of a segmented heart valve and a visualization of a topology-based 2D model according to an embodiment of the present invention. [Figure 5B] FIG. 1 is a schematic diagram of a segmented heart valve and a visualization of a topology-based 2D model according to an embodiment of the present invention. [Figure 6] FIG. 2 is a schematic diagram of a user interface according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0049] Throughout the drawings, like or corresponding features / elements of various embodiments are designated with the same reference numerals.

[0050] FIG. 1 is a schematic diagram of a visualization of a segmented heart valve 2 and a topology-based 2D model 3 according to an embodiment of the present invention. On the left side of FIG. 1, a visualization of the segmented heart valve 2 is shown. On the right side of FIG. 1, a topology-based 2D model 3 is shown. The heart valve is segmented from a 3D or 4D volume data set. The segmentation is performed using known techniques. For example, the segmentation may be performed as shown in DROGE, Hannah et al., "Mitral Valve Segmentation Using Robust Nonnegative Matrix Factorization," Journal of Imaging, 2021, 7. Jg., Nr. 10, S. 213, which is incorporated herein by reference. Once the heart valve is segmented and displayed as a visualization of the segmented heart valve 2, a topology-based 2D model 3 is generated. The visualization of the segmented heart valve 2 includes at least two leaflets 6. In this case, the visualization of the segmented heart valve 2 includes three leaflets, each separated by a border. In other words, the heart valve represented by the visualization has three leaflets. The topology-based 2D model 3 includes three commissure points 4 on the model annulus 9 and three commissure lines 5 defined by the commissure points 4. The three commissure lines 5 define three leaflets 6 in the topology-based 2D model 3. The commissure lines 5 each connect a commissure point 4 to the center of the topology-based 2D model. That is, the commissure points 4 are connected to each other through the center of the topology-based 2D model. Furthermore, the model annulus 9 is scaled to the segmented annulus 12. The three commissure points 4 and the three commissure lines 5 are also shown in the visualization of the segmented heart valve 2. In other words, the topology-based 2D model 3 is projected onto the visualization of the segmented heart valve. Based on the visualization of the segmented heart valve 2, the commissure points 4 can be adjusted by the user via the user interface 10 (followed in more detail below) such that the commissure points 4 correspond to the junctions of the borders of the leaflets 6 with the annulus 12.By drag-and-drop, the topology-based 2D model 3 can be adjusted to fit the individual anatomical structures of the heart valve (e.g., the leaflets 6) and displayed on the actual segmentation (i.e., the visualization of the segmented heart valve 2). The topology-based 2D model 3 can be tracked over time. Pre-procedural planning can also be used during the procedure by projecting the topology-based 2D model 3 onto new image data.

[0051] In other words, the topology-based 2D model 3 of the heart valve includes the commissure points 4, the labeled cusps 6, and the commissure lines 5. The topology-based 2D model and modifications to the topology-based 2D model are projected onto the 4D echo segmentation of the heart valve (i.e., the visualization of the segmented heart valve 2). The commissure points 4 can then be edited on the topology-based 2D model. Modifications can be made to the commissure points 4, such as: a. Add / remove commissure points 4 on a topology-based 2D model based on morphology. b. By dragging and dropping the commissure points 4 along the model annulus 9, the commissure points 4 also remain along the annulus in the actual image (i.e., the segmented annulus 12). The sectors between the commissure lines 5 are then labeled according to the morphology definition. Each sector represents one cusp 6 of the heart valve. In addition, the projection of the topology-based 2D model 3 is tracked through the temporal tracking of the annulus 9, 12, through one complete cardiac cycle. The morphology nomenclature is mapped onto the adjusted topology-based 2D model.

[0052] The arrows in Fig. 1 indicate that an adaptation of the commissure points 4 can be performed in the topology-based 2D model 3 and in the visualization of the segmented heart valve 2. The corrections are projected onto each other one. The letters depicted in Fig. 1 are the labeling of the leaflets according to the nomenclature described above.

[0053] FIG. 2 is a schematic illustration of a segmented heart valve and a visualization of a topology-based 2D model according to another embodiment of the present invention. This embodiment essentially corresponds to the previous embodiment, except that another heart valve is shown in FIG. 2, i.e. the heart valve in FIG. 2 has only two cusps. However, the procedure is the same as in the previous embodiment. Moreover, in FIG. 2 the commissure points are directly connected to each other by a commissure line. In other words, the commissure line is not routed through the center of the topology-based 2D model. In this case, the commissure line can be adjusted by the user to fit the borders of the two cusps 6 of the heart valve.

[0054] FIG. 3 is a schematic visualization of multiple topology-based 2D models 3 according to an embodiment of the present invention. In the center of FIG. 3, an initial topology-based 2D model 3 is shown. That is, the initial topology-based 2D model 3 is not adjusted to the actual heart valve and has an initial setting. In this embodiment, the initial topology-based 2D model 3 has three commissure points 4 and three commissure lines 5. The topology-based 2D model 3 surrounding the initial topology-based 2D model 3 of FIG. 3 is an adaptive topology-based 2D model 3. The topology-based 2D model 3 is generated similarly to the previous embodiment. Thus, a user may add, remove, and / or shift commissure points 4 to adjust the topology-based 2D model 3 to the actual heart valve depicted in the segmented heart valve visualization. Furthermore, the labels of the valve cusps 6 may be manually modified by the user. In any topology-based 2D model 3 of FIG. 3, the center of the topology-based 2D model 3 is a circle coaxial with the outer contour of the topology-based 2D model 3. Within the central circle, a label is provided indicating the type of heart valve. The type of heart valve can be determined based on the morphology of the heart valve. Furthermore, each commissure line 5 is connected to the center of the topology-based 2D model 3. For simplicity, in FIG. 3, reference numbers are added only to the original topology-based 2D model.

[0055] FIG. 4 is a schematic visualization of a topology-based 2D model being adjusted according to an embodiment of the present invention. That is, on the left side of FIG. 4, an initial topology-based 2D model 3 is shown. Then, the user can grab the commissure points 4 with the pointing device 13 and drag the commissure points 4 in the desired direction along the model annulus 9 (see the small arrows in FIG. 4). In other words, the topology-based 2D model 3 can be a "valve joystick" since adjustments to the commissure points 4 can be automatically applied to the segmented heart valve (not shown in FIG. 4) as well. The topology-based 2D model 3 is displayed on a 4D echo clip (i.e., visualization of the segmented heart valve 2) of a valve whose annulus (i.e., the segmented annulus 12) has already been segmented. The complete topology-based 2D model 3 can be rotated to fit the alignment of the echo clip. Through the "+" and "-" buttons around the model annulus, commissure points 4 can be added and removed in any sector of the topology-based 2D model 3 to fit the number and morphology of the valve cusps 6. Drag and drop of the commissure points 4 along the model annulus 9 adjusts the commissure points 4 on the echo annulus 12. The leaflets 6 are labeled according to the nomenclature proposed by Hahn et al. in the case of tricuspid valves. All displays of the topology-based 2D model 3 on the echo clip are tracked through one complete cardiac cycle. The selected morphology based on the number and location of the commissure points 4 is mapped to the proposed nomenclature and, in the case of tricuspid valves, a type is assigned. The default type at initialization is the most common type I. An added value lies in the continuous visualization of the leaflets 6 even during periods when the visibility of the echo clip is limited. An alternative embodiment that can be used as an accelerated workflow option is a model pre-selection based on a graphical representation of the morphology. Once the valve type is selected, it is displayed on the echo clip and can be adjusted with the landmarks (commissure points) on the clip itself. The landmarks can be tracked through one cardiac cycle.

[0056] 5A is a schematic diagram of a visualization of a segmented heart valve 2 and a topology-based 2D model 3 according to an embodiment of the present invention. The segmented heart valve 2 and the topology-based 2D model 3 correspond to the segmented heart valve 2 and the topology-based 2D model 3 according to any one of the previous embodiments. In this embodiment of the segmented heart valve 2 and the topology-based 2D model 3, at least one flow phenomenon 7 is additionally shown. Thus, in this embodiment, a parametric display is provided that projects the flow phenomenon from the 4D echo (i.e., the segmented heart valve 2) to the topology-based 2D model 3. Thus, this embodiment can provide a "valve dashboard". That is, the topology-based 2D model 3 can be considered as a "valve dashboard" since it displays the flow phenomenon in the heart valve.

[0057] Specifically, as an example of a flow phenomenon 7 in the segmented heart valve 2, the regurgitant flow is dynamically projected or displayed in a 4D echo during the period when the valve is typically closed (mitral, tricuspid: systole, pulmonary artery, aorta: diastole). The location of the leak area on the topology-based 2D model 3 is then displayed. In addition, the area size of the leak is scaled and shown on the topology-based 2D model 3. Also displayed on the projection is the magnitude of the leak via a heat map or a contour graph (isoline). The visualization of the flow phenomenon 7 on the topology-based 2D model 3 is realized by markers 8 that can be modified with respect to their visualization properties, i.e. the markers 8 can be individually adapted to the user's preferences.

[0058] In other words, the "valve dashboard" of the present invention provides visualization of the location, extent, and direction of flow phenomena, e.g., regurgitation, in some manner on 4D echo (i.e., segmented heart valve 2) (e.g., as vector fields or isovelocity shells). This information is then projected onto a topology-based 2D model 3 on the 4D echo. The area of ​​the projection is calculated. The location and size of the area is then projected onto the topology-based 2D model 3 for easier localization in procedure planning. The local magnitude of the flow phenomenon (e.g., leakage) is shown through contour lines or heat maps.

[0059] 5B is a schematic visualization of a segmented heart valve and a topology-based 2D model visualization according to an embodiment of the present invention, which essentially corresponds to FIG 5A, except that the heart valve in FIG 5B has only two leaflets.

[0060] FIG. 6 shows a user interface 10 according to an embodiment of the present invention. In this setup, a segmented heart valve 2 and a topology-based 2D model 3 are depicted on a conventional computer screen 14. The screen 14 may include a panel of buttons and sliders 15 that allow the user to tilt, zoom, move, or otherwise manipulate the segmented heart valve 2 and / or the topology-based 2D model 3. Also, in such a user interface 10, it is a useful tool to have a volume-rendered VOI that is locked to the location of a feature of interest on a dynamic model of a dynamic anatomical structure such as a beating heart. The display may be controlled by a computer 16, such as a PC, including a processor 17 and a hard disk 18. The user interface may have input tools such as a keyboard 19 and / or a mouse (i.e., pointing device) 20.

[0061] The above discussion is intended to be merely illustrative of the system, and should not be construed as limiting the appended claims to any particular embodiment or group of embodiments. Thus, while the system has been invented in particular detail with reference to exemplary embodiments, it should also be understood that numerous modifications and alternative embodiments can be devised by those skilled in the art without departing from the broader and intended spirit and scope of the system as set forth in the following claims. Thus, the specification and drawings are to be regarded in an illustrative manner, and are not intended to limit the scope of the appended claims.

Claims

1. A computer implementation method for adjusting the mohrologic of a heart valve model segmented from a 3D or 4D volume dataset, A step of providing a 3D visualization of a segmented heart valve on a user interface, wherein the heart valve is segmented from a 3D or 4D volume dataset, and the annulus of the segmented heart valve is represented as a circle in the topology-based 2D model, A step of generating a topology-based 2D model of the heart valve, wherein the topology-based 2D model is a simplified visualization of the characteristics of the heart valve, The steps include displaying the 2D model on the user interface, The steps include receiving user input data via the user interface to further adjust the 2D model, and A method having

2. The method according to claim 1, wherein the topology-based 2D model is generated by mapping the segmented heart valves to scale on a plane.

3. The method according to claim 1, wherein the topology-based 2D model is displayed on the user interface next to the 3D visualization of the segmented heart valve.

4. The method according to claim 1, wherein the 2D model includes a commissure point that defines at least one commissure line indicating a separation line between at least two leaflets of the heart valve.

5. The method according to claim 4, wherein the user input includes an instruction to modify the position of at least one intersection point of the topology-based 2D model.

6. The method according to claim 4, wherein the commissure points and commissure lines are displayed in both the topology-based 2D model and the 3D visualization of the segmented heart valve.

7. The method according to claim 4, wherein the user input includes commands for adding new intersection points and / or commands for deleting intersection points.

8. The method according to claim 4, further comprising the step of labeling a plurality of valve leaflets defined by the at least one commissure.

9. The method according to claim 1, wherein the visualization of the segmented heart valve further includes a structure surrounding the heart valve.

10. The method according to claim 1, wherein the visualization of the segmented heart valve is a dynamic 4D visualization of the heart valve.

11. The aforementioned method, The steps include projecting the flow phenomenon onto a 3D visualization of the heart valve based on the flow information, The steps include identifying the location on a topology-based 2D model that corresponds to the location of the flow phenomenon, The steps include providing a marker at the identified location and The method according to claim 1, further comprising:

12. The method according to claim 11, wherein the marker includes a computer graphic representation of the flow phenomenon showing the magnitude of the flow phenomenon, in particular isolines, vector fields, streamlines, and / or color maps.

13. The method according to claim 11, further comprising the step of receiving user input indicating a location on the topology-based 2D model where a marker should not be displayed.

14. A computer program comprising program code configured to perform the method according to any one of claims 1 to 13 when executed on a control unit.

15. It is a system for analyzing heart valves. A user interface configured to receive user input, A control unit configured to perform the method described in any one of claims 1 to 13, and A system that includes this.

Images