Computer-implemented method, computer program and system for adjusting morphology of heart valve model

Abstract

There is provided a computer-implemented method for adjusting the morphology of a heart valve model segmented from a 3D or 4D volume dataset, the method comprising: providing a 3D visualization of a segmented heart valve (2) on a user interface wherein the heart valve is segmented from the 3D or 4D volume dataset, creating a topology-based 2D model (3) of the heart valve, and generating a topology-based 2D model (4) of the heart valve; the 2D model (3) is displayed on a user interface, via which user input data for further adjusting the 2D model (3) is received. Further, a computer program and system for analyzing a heart valve are provided.

Description

Technical Field

[0001] The present invention relates to a computer implemented method for adjusting the morphology of a heart valve model segmented from a 3D or 4D volume data set, a computer program and a system for analyzing a heart valve. Background Art

[0002] Some anatomical structures are difficult to visualize due to the hidden arrangement and / or movement of anatomical structures in the human body. An example of such an anatomical structure is the tricuspid valve, which is particularly difficult to visualize in echocardiography due to its position and structure. In particular, the leaflets of the tricuspid valve can only be seen in a few frames throughout the cardiac cycle, and usually not all of them are in the same frame. So far, the segmentation and tracking of leaflets in echocardiographic segments have not been successful. In order to address common pathologies such as tricuspid regurgitation, more and more implantable devices and surgeries are emerging. These surgeries require thorough planning to make them as safe and successful as possible. In order to plan and navigate these surgeries, novel imaging methods in echocardiography are needed. Knowledge of the morphology of the valve leaflets is necessary for several surgeries. In addition, the anatomical landmark markings of the heart valves are not very intuitive and are difficult to edit in moving 4D segments.

[0003] US2005187461 A discloses a computerized method for facilitating cardiac intervention, comprising: inputting patient data; creating a computerized interactive model of a heart based on the patient data, wherein the model includes features; simulating at least one proposed cardiac intervention by adding or deleting features to the model; and determining the impact of the proposed cardiac simulation on the entire model. The simulation can be repeated to allow the user to determine the best cardiac intervention. Additionally, a template can be created from the model for use as a guide during the cardiac intervention.

[0004] US2020082531 A1 shows a device for dynamically evaluating a moving object from a sequence of consecutive volumetric image frames of such an object, which images are separated in time by specific time intervals by: identifying an object of interest in at least one image of the sequence; segmenting the object to identify the object contour; propagating the identified object contour to other images of the sequence; and performing dynamic analysis of the object based on the propagated object contour.

[0005] A white paper “Quantifying Heart Valves: From Diagnostic to Personalized Valve Repair” written by Tommaso Mansi et al. in April 2016 describes a valve modeling and editing method, wherein a combination of providing editing views, such as parallel cutting or rotational cutting and intelligent mesh editing is proposed.

[0006] However, assessing the topology of heart valves remains cumbersome, and thus preparing for surgery remains difficult.

[0007] Object of the present invention

[0008] It is therefore an object of the present invention to provide a method and a device which provide a simple and intuitive way to assess anatomical structures even if these are difficult to visualize. Summary of the invention

[0009] The invention solves this problem with a computer implemented method comprising the features of claim 1 , with a computer program comprising the features of claim 14 and with a system comprising the features of claim 15 .

[0010] According to one aspect, the present invention provides a computer-implemented method for adjusting the morphology of a heart valve model segmented from a 3D or 4D volume dataset, the method comprising:

[0011] providing a 3D visualization of the segmented heart valve on a user interface, wherein the heart valve is segmented from the 3D or 4D volumetric dataset,

[0012] creating a topology-based 2D model of a heart valve, wherein the topology-based 2D model is a simplified visualization of properties of the heart valve,

[0013] Display 2D models on the user interface,

[0014] User input data for further adjusting the 2D model is received via the user interface.

[0015] According to an embodiment of the present invention, the method projects a topology-based model 2D onto the actual segmentation of the valve ring. The model can be edited to depict any type of morphology of the heart valve. In other words, the 2D model can be created essentially based on the segmented heart valve, and can then be further manually adjusted by the user to represent the morphology of the segmented heart valve. Therefore, the 2D model can be accurately adjusted to the morphology of the actual heart valve. The 2D model can also be referred to as a symbolic valve model, which can be used as a research and communication tool for valve research, comparison between patients, longitudinal comparison of a patient's valve, disease progression analysis, consultation between physicians, and the like. The method can provide a digital joystick with which the user can transfer the topology on a 3D or 4D data set, as well as a valve dashboard to visualize phenomena from a 3D or 4D data set on the topology. The method is applicable to all heart valves.

[0016] The method can be used to adjust the morphology of the 2D model. Morphology can include aspects of the appearance of the heart valve (e.g., shape, structure, pattern, size, etc.), i.e., external morphology (or circumcision), as well as the form and structure of the components of the heart valve, such as leaflets, annulus, etc., i.e., internal morphology (or anatomy). This is in contrast to physiology, which primarily deals with function. Morphology can be considered a branch of life science that deals with the study of the overall structure of anatomical structures and their components.

[0017] A 3D or 4D volume data set may include information about a three-dimensional (3D) or four-dimensional (4D) anatomical structure. The heart valve may be segmented in a known manner based on the anatomical structure. For example, Segmentation is performed as shown in "Mitral Valve Segmentation Using Robust Nonnegative Matrix Factorization" by Hannah et al. Journal of Imaging, Vol. 7, No. 10, No. 213, 2021, which is incorporated herein by reference. In addition, the 3D or 4D volume dataset may include the heart as an anatomical structure. The three-dimensional information can define the spatial position of a point in space. In addition, the fourth dimension can also include time, that is, the position of the point over a period of time. In other words, the 4D volume dataset can be a video clip containing multiple frames (i.e., 3D information) covering, for example, a heart cycle. The 3D or 4D volume dataset can be provided in a DICOM (Digital Imaging and Communications in Medicine) format. However, the 3D or 4D volume dataset can be provided in any other suitable format as long as it includes 3D or 4D information of the anatomical structure.

[0018] A heart valve may be referred to as a dynamic anatomical structure because its size changes over time. A 3D or 4D volume data set may be captured in a sequence of three-dimensional medical images obtained from a human subject over a period of time, wherein the images may be preoperative images, but may also be intraoperative images. A 3D or 4D volume data set may include medical images. A sequence of 3D medical images may be referred to as a 4D image. A three-dimensional (3D) image is typically a digital image such as in the DICOM standard, i.e., it contains a three-dimensional array of voxels, each voxel containing a grayscale value. Such a 3D medical image is typically obtained from a field of view containing a dynamic anatomical structure using a medical imaging modality such as MR, computed tomography (CT), positron emission tomography (PET), or ultrasound (US). When the anatomical structure is a heart, ultrasound, in particular transesophageal echocardiography (TEE), may be advantageously used. A 3D image from a time series of 3D images is also referred to as a "frame" hereinafter. 3D images can be acquired at a frame rate of, for example, 5-100 (preferably 20-60 images per second) to allow a smooth representation of dynamically moving anatomical structures, where it is displayed in a movie mode. The time period is typically at least one cycle of the cyclic movement, for example, at least one heartbeat.

[0019] 3D visualization (also known as 3D or 4D echo) can be a 3D volume rendering of a 3D or 4D volume dataset. 3D volume rendering can include a set of techniques for displaying 2D projections of a 3D image dataset. However, although 3D volume rendering helps to form a mental 3D model of the anatomical structure, it strongly depends on the image quality and the selected settings (threshold, smoothing, etc.). Volume rendering has the advantage of being suitable for more complex anatomical structures (such as valve leaflets or tricuspid valve leaflets). In addition, parameters such as threshold, opacity, contrast can be adjusted "in real time", that is, they take effect immediately when viewing the volume rendering. On the other hand, volume rendering is strongly dependent on image quality and may be too complex to interpret when used alone. Alternatively or additionally, the 3D visualization of the segmented heart valve can be a shape / surface 3D model. The surface 3D model can be a simplification of the heart valve, such as a triangular surface 3D model. The 3D model can include multiple points across a line or surface for each frame. It can also be a mathematical model, such as a parametric model, such as a surface or volume spanned by a spline curve. The 3D model can be dynamic, that is, it follows the movement of the anatomical structure (e.g., a heart valve) across time periods. The purpose of a dynamic 3D model is to visualize the basic components of the heart valve (e.g., leaflets) and the changes of the heart valve over time. The advantage of a 3D model is that it does not require as much computing power as a 3D rendering, but still provides the user with the basic features of the heart valve. However, using a 3D model alone, the user still cannot track all the features of the heart valve throughout the heart cycle. For example, the leaflets of a heart valve (e.g., the tricuspid valve) can be seen only in a few frames throughout the cardiac cycle, and typically not all of them in the same frame. This limitation may be due to the nature of the imaging modality, which does not allow the acquisition angle to be changed freely as in echocardiography. In this regard, a topology-based 2D model will help the user (described in more detail below).

[0020] In order to allow a user to view and analyze a heart valve (i.e., a region of interest), a three-dimensional visualization environment may be provided for visualizing a visualization of the segmented heart valve over a period of time. That is, a visualization of the segmented heart valve and a topology-based 2D model may be depicted within the same visualization environment. The visualization environment may be used to view and analyze the heart valve, in particular for planning interventions and / or determining the correct size, shape, and location of an implant to be implanted in a future intervention.

[0021] The visualization environment can be a three-dimensional visualization environment or a virtual reality environment. "Virtual reality" refers to any computer-generated visualization that provides a true three-dimensional experience of the depicted structure. Therefore, the virtual reality (VR) environment of the present invention provides certain visual feedback, but other types of sensory feedback, such as hearing, may also be allowed. The VR environment can also be an augmented reality environment, in which the user still sees the real environment, but VR objects (e.g., volume renderings and dynamic models) are overlaid or superimposed on the real objects, or a mixed reality, in which real-world objects are superimposed on the virtual scene.

[0022] The segmented heart valve may be part of an anatomical structure included in a 3D or 4D volume dataset. The segmented heart valve may be automatically obtained based on the 3D or 4D volume dataset. That is, a 3D visualization of the segmented heart valve may depict only the heart valve. The surrounding anatomical structures may be omitted. Thus, the clarity of the visualization is improved and the user may focus on the important parts of his or her assessment.

[0023] The user interface may be a device configured to receive user input and visualize the 3D visualization and the 2D model of the segmented heart valve to the user. That is, the user interface may be an interface between a person and a machine. Preferably, the user interface may include a display configured to display the 3D visualization and the 2D model to the user. In addition, the user interface may include an input device that can be operated by the user and can input user commands to the user interface. In particular, the display may be a touch-sensitive display. In this case, since the user can input his or her commands via the display, an additional input device may not be required. The user interface may include a processor and a storage device for storing information. Preferably, the user interface is a computer.

[0024] A topology-based 2D model is a two-dimensional simplified visualization of a heart valve. A topology-based 2D model may be a standardized simplification of an actual segmentation of a heart valve. Simplification may mean that a three-dimensional representation of a heart valve (i.e., a segmented heart valve) is transferred to a two-dimensional visualization of the heart valve (i.e., a topology-based 2D model). The ring of the segmented heart valve may be freely located in space, and a point on the ring may be described by three variable coordinates, each of which may be different. On the other hand, the ring of a topology-based 2D model may be described by points with only two variable coordinates (i.e., the coordinates are the same for each point).

[0025] Preferably, the ring can be represented by a circle. The length or circumference (e.g., circumference or circumference shape) of the circle can correspond to the length or circumference of the ring. The topology of the heart valve can be characterized by multiple leaflets of the heart valve. In addition, the topology of the heart valve can be further characterized by the position, area and / or shape of the leaflets. Therefore, the topology-based 2D model can simplify the visualization of the characteristics of the heart valve. In addition, the circumference of the circle surrounding or defining the topology-based 2D model can be the same length, regardless of the actual length of the heart valve ring. Therefore, the topology-based 2D model can be further simplified. In addition, the topology-based 2D models of different heart valves can be easily compared with each other.

[0026] The creation of a topology-based 2D model can be performed automatically. That is, a rough topology-based 2D model can be automatically created. Then, the user can adjust the topology-based 2D model so that it best fits the 3D visualization of the segmented heart valve (more details about the adjustment are as follows). In addition, a standard topology-based 2D model can be provided initially. Then, the user can adjust the topology-based 2D model to the visualization of the segmented heart valve. That is, the standard topology-based 2D model can always be the same independent of the specific heart valve being examined. In addition, the topology-based 2D model can be a static symbolic representation of the heart valve. In other words, for example, even if a 3D or 4D visualization of the segmented heart valve is depicted on a heartbeat, the topology-based 2D model can remain static (i.e., without moving or modifying its shape or configuration). Therefore, the user is able to adapt the topology-based 2D model to the dynamic (4D) visualization of the heart valve using the static symbolic representation of the topology-based 2D model. Preferably, the user can immediately observe the results of the adaptation in the dynamic visualization of the heart valve. According to another embodiment, extensions to a more dynamic surface model are possible (e.g., editing the shape of the joining lines). The model can be edited to depict any type of morphology of the heart valve. The topology-based 2D model can be depicted on the same user interface as the segmented heart valve. Receiving user input can mean that the user enters a command to adjust the topology-based 2D model. This can be done directly by adjusting the topology-based 2D model. Alternatively or additionally, the user can also adjust the segmented heart valve, and the topology-based 2D model can be adjusted indirectly by user input.

[0027] According to another embodiment, a topology-based 2D model is created by mapping the segmented heart valve onto a plane in proportion. Therefore, the proportion of the circumference of the ring of the segmented heart valve can be maintained. That is, the topology-based 2D model can be projected onto a plane or surface. In addition, the topology-based 2D model may be distorted during projection onto the 2D surface. In this case, the topology-based 2D model is not a developable surface (nor a ruled surface). In an alternative embodiment, the topology-based 2D model is created based on the developable surface of the segmented heart valve. For example, the topology-based 2D model can be a developable surface. In mathematics, a developable surface (called a twisted surface: an ancient body) is a smooth surface with a zero Gaussian distribution. That is, it is a surface that can be flattened on a plane without deformation (that is, it can be bent without stretching or compression). On the contrary, it is a surface that can be made by transforming a plane (that is, "folding", "bending", "rolling", "cutting" and / or "gluing"). In three dimensions, all developable surfaces are ruled surfaces (but not vice versa). In this case, the topology-based 2D model can be an expandable surface based on the 3D visualization of the segmented heart valve. In this case, the topology-based 2D model can have dimensions related to the segmented 3D segmentation of the heart valve. That is, the circumferential length of the topology-based 2D model can correspond to the circumferential length of the segmented heart valve. Therefore, further information can be derived from the topology-based 2D model.

[0028] According to another embodiment of the present invention, a topology-based 2D model is displayed on a user interface next to a 3D visualization of a segmented heart valve. That is, the topology-based 2D model and the 3D visualization of the segmented heart valve can be displayed on the same interface (e.g., a display). Preferably, the 3D visualization of the segmented heart valve and the topology-based 2D model are depicted as being directly adjacent to each other. In this case, the user can easily adjust the topology-based 2D model while viewing the 3D visualization of the segmented heart valve. Therefore, the process of adjusting the topology-based 2D model can be further facilitated. Preferably, the topology-based 2D model is displayed adjacent to the segmented heart valve and is additionally projected onto the segmented heart valve. That is, commissures and / or lines are projected onto the segmented heart valve. In this way, the topology-based 2D model can be easily adapted to the segmented heart valve.

[0029] According to another embodiment, the 2D model includes a commissure defining at least one commissure line indicating a separation line between at least two leaflets of a heart valve. The commissure point may define a specific region where the leaflets meet at their junction with the ring. For example, the point may be the location where the anterior leaflet and the posterior leaflet meet at their junction with the ring. The commissure point may be used to define the segmentation of the leaflet. In more detail, the leaflets of the heart valve may be visualized in a simplified manner by connecting the commissure point with a straight line. In addition, the commissure point may describe a fine structure that may be identified by two anatomical landmarks, such as the axis and commissure tendons of the corresponding papillary muscle having a specific fan-shaped structure. Several millimeters of valve tissue separate the free edge of the commissure point from the ring. If a commissure point and a corresponding anterior leaflet segmentation and a posterior leaflet segmentation (e.g., marked as P3 and A3) prolapse occurs, the region must be observed, for example, by resection, otherwise residual regurgitation will occur in the region. The commissure point may be depicted on both the topologically based 2D model and the 3D visualization of the segmented heart valve. The commissure points depicted on the topology-based 2D model and the segmented 3D visualization of the heart valve can be related to each other so that both have the same relative position relative to the reference point of the heart valve. For example, such a reference point can be the center of the heart valve. When the commissure points on the topology-based 2D model or the segmented 3D visualization of the heart valve move, the corresponding commissure points of another topology-based 2D model or the segmented 3D visualization of the heart valve can be automatically moved accordingly. Therefore, the user can compare the position of the commissure points with the segmented heart valve of the 3D visualization of the segmented heart valve. In other words, the user can evaluate whether the commissure points are located at the junction of the two leaflets and the ring in the 3D visualization of the segmented heart valve. This can be done in any available frame of the visualization of the segmented heart valve. The commissure points can define commissure lines. In more detail, the commissure lines can extend from the ring (for example, from the circle defining the outer contour of the topology-based 2D model) to the center of the topology-based 2D model. In more detail, each commissure line can extend from a commissure point to the center of the topology-based 2D model. The commissure line can be a straight line connecting two commissure points, preferably passing through the center of the topology-based 2D model. The center of the topology-based 2D model can be represented by an inner circle that can be coaxial with the ring. The commissure line can define a connection line between two adjacent leaflets of a heart valve. In other words, the commissure line can define the edge of two adjacent leaflets. The commissure line can be a rough representation of the boundary between two adjacent leaflets, because in reality the leaflets do not have a straight boundary line. That is, the commissure line can indicate the approximate extent of the boundary between the leaflets. The segment between the commissure lines can define the leaflets.

[0030] According to another embodiment of the present invention, the user input includes an instruction to modify the position of at least one commissure point of the topology-based 2D model. If the user wishes to adjust the position of one or more commissure points, he or she can input an instruction to modify the position of at least one commissure point in the 3D visualization of the segmented heart valve and / or the topology-based 2D model. For example, the user can virtually grasp the commissure point to be modified and move it to a desired position on the ring (or the outer contour of the topology-based 2D model). The user can modify the commissure point in the 3D visualization of the segmented heart valve and / or the topology-based 2D model. The commissure point in the 3D visualization of the segmented heart valve or the topology-based 2D model modified by the user automatically adjusts its position according to the user's instruction. The corresponding commissure point in another 3D visualization of the segmented heart valve or the topology-based 2D model (whose position is not directly set by the user) automatically and simultaneously adjusts its position to match the position of the commissure point directly set by the user. Preferably, the user edits the topology-based 2D model (i.e., the commissure points within the topology-based 2D model) and observes the correction of the change in the position of the commissure points within the 3D visualization of the segmented heart valve. For example, by dragging and dropping, the commissure points of the topology-based 2D model can be adjusted to fit the individual anatomy of the leaflets and displayed onto the real segmentation (i.e., the 3D visualization of the segmented heart valve). The 3D visualization of the segmented heart valve and the topology-based 2D model can be tracked over time. During surgery, pre-operative planning can also be used by projecting the model (e.g., commissure points, commissure lines, etc.) onto new imaging data. In this case, the topology-based 2D model can be considered a virtual "joystick" that the user can use to adapt the position of the commissure points within the 3D visualization of the segmented heart valve.

[0031] According to an embodiment of the present invention, commissure points and commissure lines are displayed on both a topology-based 2D model and a 3D visualization of a segmented heart valve. That is, the relative position of the commissure points relative to the reference point can be the same in both the topology-based 2D model and the 3D visualization of the segmented heart valve. For example, the reference point can be the center of the heart valve. Alternatively, any other landmark can be used as a reference point. The representation of commissure points and commissure lines in the 3D visualization of the segmented heart valve can provide an opportunity to verify that the commissure points are well placed in different images of a 3D or 4D volume data set. Different images can be acquired at different time points in the cardiac cycle and / or at different acquisition angles. Therefore, the commissure points in each image can be adjusted to match the heart valve. Therefore, a topology-based 2D model with high accuracy can be generated.

[0032] According to another embodiment of the present invention, the user input includes an instruction to add a new commissure point and / or delete a commissure point. That is, the initial topology-based 2D model may include three different leaflets. That is, the initial topology-based 2D model may have three commissure lines. However, depending on the type of valve being examined, the heart valve may have more or fewer leaflets. For example, the mitral valve has only two leaflets, while the tricuspid valve may have more leaflets. In addition, depending on the individual, the tricuspid valve may have a different number of leaflets. That is, each person may have a different number of leaflets in their tricuspid valve. Therefore, the topology-based 2D model can be adapted to the specific heart valve being examined by adding commissure points or by deleting commissure points. For example, a user can add additional commissure points by clicking on a circle defining the outer contour of the topology-based 2D model. In addition, a new commissure line extending from the new commissure point to the center of the topology-based 2D model can be automatically created. Therefore, additional leaflets can be depicted by the topology-based 2D model. Commissure points can be deleted in a similar manner. Thus, the commissures defined by the deleted commissures may also be deleted. Thus, the topology-based 2D model may have one less leaflet. For example, the user may select the commissures to be deleted and then provide instructions to delete the commissures. Thus, the topology-based 2D model may be adapted to the detected segmented heart valve.

[0033] According to another embodiment of the present invention, the method also includes marking a plurality of leaflets defined by at least one commissure. That is, the segments between the commissures can be marked according to the morphological definition of the corresponding leaflets defined by the segments. The labels can correspond to the commonly used indications of the corresponding leaflets. For example, the tricuspid valve may include an anterior cusp, a posterior cusp, and a middle cusp. The aortic valve may include a left coronary cusp, a right coronary cusp, and a non-coronary cusp. For example, the mitral valve has two leaflets. The anterior leaflet may have a semicircular shape and may be attached to two-fifths of the annular circumference. There is continuity between the anterior leaflet of the mitral valve and the left and non-coronary cusps of the adjacent aortic valve, referred to as the aortic-mitral curtain. The movement of the anterior leaflet also defines an important boundary between the inflow tract (during diastole) and the outflow tract (during systole) of the left ventricle. The posterior leaflet of the mitral valve may have a quadrilateral shape and may be attached to approximately three-fifths of the annular circumference of the heart valve. The posterior leaflet typically has two well-defined notches that divide the leaflet into three separate sectors, identified as P1 (anterior or inner sector), P2 (middle sector) and P3 (posterior or outer sector). The three corresponding segments of the anterior leaflet are A1 (anterior segment), A2 (middle segment) and A3 (posterior segment). This nomenclature can be considered as a labeling of the topology-based 2D model. In addition, this nomenclature is an important tool between echocardiographers and surgeons to describe the anatomical structure of a specific anatomical segment. The notches can be used to identify or analyze the morphology of the corresponding leaflet, and thus mark the corresponding leaflet according to the above nomenclature. The labeling can be automatically completed based on the morphology and / or position of the identified corresponding leaflet. For example, the angle at which the 3D or 4D volume data set is acquired can be known, so that depending on the acquisition angle, it can be known which leaflets are expected and / or visible at a specific location on the heart valve. Therefore, automatic labeling can be performed with high accuracy. For example, a learning algorithm can be provided to identify the corresponding leaflet of the heart valve. That is, the morphological information of the leaflets can be input as input data to the learning algorithm, and the learning algorithm can output the labels of the leaflets as output data. Therefore, the learning algorithm can be pre-trained using training data including the expected output data for the specific input data. Using the learning algorithm to label the leaflets of the heart valve can speed up the process of creating a topology-based 2D model.

[0034] According to another embodiment of the present invention, the visualization of the segmented heart valve also includes structures surrounding the heart valve. Surrounding the heart valve may be tissue surrounding the heart valve being examined. In more detail, the support structure of the heart valve may be depicted within the visualization of the segmented heart valve. Therefore, the commissure may be set more accurately because the support structure may help define where the boundary between two adjacent leaflets is actually located. For example, in the mitral valve, a portion of the posterior medial papillary muscle and / or a portion of the anterior lateral papillary muscle may be visualized together with the segmented heart valve. In addition, the visualization of the surroundings of the segmented heart valve may help define the orientation of the segmented heart valve. In addition, the surrounding information of the segmented heart valve may also be input into the tilt algorithm for marking the leaflets of the heart valve (also refer to the above overview). Therefore, the learning algorithm may more accurately and / or pick out a specific valve by identifying known landmarks in the surrounding tissue of the heart valve.

[0035] According to another embodiment of the present invention, the visualization of the segmented heart valve is a dynamic 4D visualization of the heart valve. That is, the segmented heart valve can be visualized in four dimensions (4D). The fourth dimension can be time. That is, the 4D visualization of the segmented heart valve can dynamically represent the heart valve. For example, the segmented heart valve can be visualized in motion within the time span of the cardiac cycle. In other words, the segmented heart valve can be visualized through a series of frames so that the changes of the heart valve over time can be seen, preferably at least 20 frames per second. Therefore, the user can observe the segmented heart valve within the heartbeat cycle. Even if the segmented heart valve is in motion, the topology-based 2D model can be projected onto the segmented heart valve. On the other hand, the topology-based 2D model is static and does not move. Therefore, the topology-based 2D model can be adjusted to multiple different frames of the segmented heart valve. Therefore, the fusion point (and therefore to the fusion line) can be accurately set by the user. In addition, the time period in which the segmented heart valve is visualized can also include a change in the acquisition angle of the 3D or 4D volume data set. Thus, the heart valve may be defined by commissures and commissure lines that are only visible at different acquisition angles. Thus, even in cases where the leaflets of the heart valve are not visible in one frame, a topology-based 2D model may be accurately created based on the segmented heart valve.

[0036] According to yet another embodiment of the present invention, the method further comprises:

[0037] Based on the flow information, the flow phenomena are projected onto the 3D visualization of the heart valve.

[0038] identifying locations on the topology-based 2D model that correspond to locations of flow phenomena, and

[0039] A marker is provided at the identified location.

[0040] In the prior art, the projection of the leak location on the valve is not intuitive for the user. In the prior art, the affected area of ​​the surgery cannot be easily marked and must be estimated. Currently, many of these surgeries are performed without specific planning, increasing the time on the operating table and the abortion rate due to incompatibility of methods or equipment. If image-based planning is performed, it is done entirely manually, taking up valuable physician time and exhibiting high variability and prone to errors.

[0041] In this embodiment, a simple way is provided to indicate leakage flow in a topology-based 2D model. Thus, planning of interventions can be facilitated and made more accurate. Flow phenomena can be derived from Doppler information included in a 3D or 4D volume data set. Thus, the flow (e.g., of blood) can be visualized. The Doppler information can be in the same coordinate system as the segmented heart valve.

[0042] Thus, the flow phenomenon can be projected at the correct position relative to the segmented heart valve. This position of the flow phenomenon in the visualization of the segmented heart valve can then be transferred to the topology-based 2D model. This can be done by reference points present in both the visualized segmented heart valve and the topology-based 2D model. Such reference points can be some or all of the commissures and / or centers of the heart valve. Then, the position of the flow phenomenon is known in the topology-based 2D model and can be indicated by a marker. That is, according to this embodiment, a parameter display that can include additional information is provided in the topology-based 2D model. This can be achieved by projection from 4D echo (as an example of a segmented heart valve) to a topology model (as an example of a topology-based 2D model). That is, a valve dashboard including information on the morphology and / or function of the valve being examined can be provided. For example, on a topology-based 2D model, regurgitation can be dynamically indicated in the time period (mitral valve, tricuspid valve: systolic period, pulmonary artery, aorta: diastolic period) in which the valve is typically closed. In addition, the position of the leakage area can be indicated in the topology-based 2D model. Markers indicating flow phenomena in a topology-based 2D model can be dynamic. That is, the indications of leaflets and rings in a topology-based 2D model can be static (i.e., may not move or change), while the marks indicating flow phenomena can change depending on the cardiac cycle currently depicted by the visualization of the segmented heart valve. Therefore, the user can examine flow phenomena in different positions of the heart valve. In one embodiment, the user can select a specific point within the cardiac cycle to visualize the flow phenomenon at that specific time in the topology-based 2D model. Therefore, the significance of the topology-based 2D model can be further increased. In addition, due to the standardized configuration of the topology-based 2D model, different patients or pathologies can be compared.

[0043] According to another embodiment of the present invention, the marking includes a computer graphic representation of the flow phenomenon indicating the magnitude of the flow phenomenon, in particular, isolines, vector fields, streamlines and / or color maps. In other words, the scale and area size of the leak can be displayed on a topologically based 2D model. That is, the leakage magnitude by a heat map or a contour map (isolines) can be displayed in a visualization of a segmented heart valve and / or a topologically based 2D model. Therefore, the significance of a topologically based 2D model can even be further increased by indicating the position, degree and direction of a flow phenomenon (e.g., regurgitation). The flow phenomenon can be visualized and calculated in several ways in the visualization of a segmented heart valve (e.g., as a vector field or a constant velocity shell). This information can then be projected onto a topologically based 2D model. The projected area can be calculated based on the information included in a 3D or 4D volume data set. The position and size of the region can then be projected onto a topologically based 2D model to make it easier to locate in surgical planning. The local range of the leak can be indicated by a contour line or a heat map. Therefore, interventions can be accurately planned in advance. Therefore, the occurrence of unexpected situations during the intervention can be significantly reduced.

[0044] According to another embodiment of the present invention, the method also includes receiving user input indicating a position on the topology-based 2D model where the mark should not be displayed. That is, the user can define an area or section in the topology-based 2D model where the mark should not be displayed (i.e., there is no flow phenomenon). In other words, the user can define an exclusion area in the topology-based 2D model. For example, if the heart valve being examined is disposed adjacent to a second heart valve, there may be a lateral flow caused by the second heart valve, which does not describe the function of the heart valve being examined. Therefore, it may be helpful to exclude some areas of the topology-based 2D model from displaying flow phenomena. Therefore, the simplicity of the inspection can be improved. The user can expect flow phenomena near the commissures because these lines indicate the boundary lines between adjacent leaflets (at least in a state where the heart valve is closed or should be closed). Therefore, the exclusion area can be defined in an area spaced apart from the commissures.

[0045] According to another aspect, the invention provides a computer program comprising program code instructions which, when executed by a processor, cause the processor to perform the method of the invention.The computer program may be any code, in particular a code suitable for computer graphics applications.

[0046] In another aspect, the invention relates to a computer readable medium comprising the computer program defined above. The computer readable medium may be any digital data storage device, such as a USB stick, a hard disk, a CR-ROM, an SD card or an SSD card. Naturally, the computer program does not need to be stored on such a computer readable medium to be provided to a customer, but may be downloaded via the Internet.

[0047] According to another aspect, the present invention provides a system for analyzing a heart valve, comprising:

[0048] A user interface is configured to receive user input,

[0049] A control unit is configured to execute the above method.

[0050] The system may be included in the acquisition modality. Thus, a topology-based 2D model may be generated immediately during the acquisition of a 3D or 4D volume dataset. Alternatively, the system may be provided centrally and may be connected to a database in which a plurality of 3D or 4D volume datasets are stored. In this case, a topology-based 2D model may be created for each dataset and may be used for studies and / or future intervention planning.

[0051] According to one embodiment of the present invention, a topology-based 2D model is displayed on a 4D echo segment of a valve, in which the valve annulus has been segmented. The topology-based 2D model can be rotated to fit the alignment of the echo segments. Through the "+" and "-" buttons on the user interface, commissure points can be added and removed around the topology-based 2D model annulus in each segment of the topology-based 2D model to fit the number and morphology of leaflets. Dragging and dropping the commissure points along the topology-based 2D model annulus can adjust the commissure points on the segmented heart valve annulus. In the case of the tricuspid valve, the commissure points can be adjusted according to the method described by Rebecca T. Hahn, Ludwig T. Weckbach, Thilo Noack, Nadira Hamid, Mitsunobu Kitamura, Richard Bae, Philipp Lurz, Susheel K. Kodali, Paul Sorajja, Hausleiter and Michael Nabauer proposed a nomenclture to label the leaflets, which is called "Proposal for a Standard Echocardiographic Tricuid Valve Nomenclture". Figure 3 Proposal for a Tricuspid Nomenclature", JACC: Cardiovascular Imaging, Vol. 14, No. 7, 2021, pp. 1299-1305, ISSN 1936-878X, which is incorporated herein by reference. All displays based on the visualization of the segmented heart valve segments based on the topology of the 2D model can be tracked through a complete cardiac cycle. The selected morphology based on the number and location of the commissures can be mapped to the proposed nomenclature and the type of the heart valve can be assigned in the case of the tricuspid valve. The default type at initialization can be the most common type I.

[0052] In the topology-based 2D model, leaflets can be continuously visualized even during periods of limited visibility in the visualization of the segmented heart valve (i.e., echo) segments. An alternative embodiment that can be used as an accelerated workflow option is a topology-based 2D model pre-selection based on a graphical representation of the morphology. By selecting the valve type, the topology-based 2D model can be displayed on the visualization of the segmented heart and can be adjusted to the visualization of the segmented heart itself by landmarks (i.e., commissures). The landmarks can be tracked through a cardiac cycle.

[0053] The method or system of the present invention can preferably be implemented in conjunction with the measurement of a 4D ultrasound data set. The method or system of the present invention can be viewed as a digital joystick with which a user can transfer a topology-based 2D model on a 4D data set, as well as a valve dashboard to visualize phenomena from a 4D data set on a topology-based 2D model. It can be applicable to all heart valves. In addition, the present invention can be used in imaging modes such as CT (computed tomography) and MRI (magnetic resonance imaging) of the heart. The labeling of morphological leaflets can be standardized by the nomenclature used in the technology. The resulting dynamic labeling can also be used for interventional navigation and real-time guidance, such as Philips EchoNavigator. The topology-based 2D model can also be used as a research and communication tool for valve research, comparisons between patients, longitudinal comparisons of valves in one patient, disease progression analysis, consultations between physicians, and the like.

[0054] In summary, the valves of the heart, such as the tricuspid and mitral valves, can be visualized in 4D echocardiography. This allows diagnosis and subsequent treatment planning. For the planning of many surgeries, it is crucial to understand and mark the morphology and position of the leaflets as well as the location and amount of regurgitation. The anatomical landmark markings of the valves are less intuitive and difficult to edit in a moving 4D clip.

[0055] According to one aspect of the invention, a topological model bidirectionally mapped to a valve segmentation is presented which can be used as an editing control ("valve joystick") and parameter display ("valve dashboard") of the valve. For the valve joystick, user input is mapped to the topological model on the 4D valve segmentation. The user can add or delete commissures on the model and change the relative positions of the commissures. These edits are displayed on the segmented model and tracked over time. According to another aspect of the invention, the valve dashboard maps the location and magnitude of regurgitation through the valve from the 4D echo to the topological model, thereby enabling better positioning.

[0056] The individual features of the above-described embodiments may be combined with other features or other embodiments, and thus may form new embodiments. The advantages and configurations of the features are applicable to the new embodiments. The configurations and advantages defined in conjunction with the method are also applicable to the device, and vice versa. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] A useful embodiment of the present invention will now be described with reference to the accompanying drawings. Similar elements or features are designated by the same reference numerals in the drawings. In the drawings:

[0058] Figure 1 is a schematic diagram of a visualization and topology-based 2D model of a segmented heart valve according to an embodiment of the present invention,

[0059] Figure 2 is a schematic diagram of a visualization and topology-based 2D model of a segmented heart valve according to an embodiment of the present invention,

[0060] Figure 3 is a schematic diagram of multiple topology-based 2D models according to an embodiment of the present invention,

[0061] Figure 4 is a schematic diagram of an adjusted topology-based 2D model according to an embodiment of the present invention,

[0062] Figure 5A is a schematic diagram of a visualization and topology-based 2D model of a segmented heart valve according to an embodiment of the present invention,

[0063] Figure 5B is a schematic diagram of a visualization and topology-based 2D model of a segmented heart valve according to an embodiment of the present invention, and

[0064] Figure 6 is a schematic diagram of a user interface according to an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In all the drawings, identical or corresponding features / elements of the various embodiments are indicated with identical reference numerals.

[0066] Figure 1 is a schematic diagram of a visualization of a segmented heart valve 2 and a topology-based 2D model 3 according to an embodiment of the present invention. Figure 1 On the left side of FIG, a visualization of a segmented heart valve 2 is depicted. Figure 1 , a topology-based 2D model 3 is depicted. The heart valve is segmented from the 3D or 4D volume dataset. The segmentation is performed using known techniques. For example, Segmentation is performed as shown in "Mitral Valve Segmentation Using Robust Nonnegative Matrix Factorization" by Hannah et al. Journal of Imaging, Vol. 7, No. 10, No. 213, 2021, which is incorporated herein by reference. Once the heart valve is segmented and displayed as a visualization of the segmented heart valve 2, a topology-based 2D model 3 is created. The visualization of the segmented heart valve 2 includes at least two leaflets 6. In the present case, the visualization of the segmented heart valve 2 includes three leaflets, each separated by a boundary line. In other words, the heart valve represented by the visualization has three leaflets. The topology-based 2D model 3 includes three commissures 4 on the model ring 9 and three commissures 5 defined by the commissures 4. The three commissures 5 define the three leaflets 6 in the topology-based 2D model 3. The commissures 5 each connect the commissure points 4 to the center of the topology-based 2D model. That is, the commissure points 4 are connected to each other via the center of the topology-based 2D model. In addition, the model ring 9 is proportionally expanded to the segmented ring 12. Three commissure points 4 and three commissure lines 5 are also depicted in the visualization of the segmented heart valve 2. In other words, the topology-based 2D model 3 is projected onto the visualization of the segmented heart valve. Based on the visualization of the segmented heart valve 2, the commissure points 4 can be adjusted by the user via the user interface 10 (described in more detail below) so that the commissure points 4 correspond to the junction of the boundary line of the leaflet 6 with the ring 12. By dragging and dropping, the topology-based 2D model 3 can be adjusted to fit the individual anatomical structure of the heart valve (e.g., to the leaflet 6) and displayed on the real segmentation (i.e., the visualization of the segmented heart valve 2). The topology-based 2D model 3 can be tracked over time. Preoperative planning can also be used during surgery by projecting the topology-based 2D model 3 onto new imaging data.

[0067] In other words, the topology-based 2D model 3 of the heart valve comprises commissures 4, leaflets 6 with markers and commissure lines 5. The topology-based 2D model and changes to the topology-based 2D model are projected on a 4D echo segmentation of the heart valve (i.e., a visualization of the segmented heart valve 2). The commissures 4 can then be edited on the topology-based 2D model. The following changes can be made to the commissures 4:

[0068] a. Adding / removing joints on a topology-based 2D model based on morphology 4, and

[0069] b. Commissure points 4 are dragged and dropped along the model annulus 9, which are also maintained along the annulus in real imaging (i.e., segmented annulus 12). Segments between commissure lines 5 are then labeled according to the morphological definition. Each segment represents a leaflet 6 of the heart valve. In addition, the projection of the topology-based 2D model 3 is tracked through a complete cardiac cycle by time tracking of the annulus 9, annulus 12. The morphological nomenclature is mapped to the adjusted topology-based 2D model.

[0070] Figure 1 The arrows in show that the adaptation of the commissure points 4 can be done in the topology-based 2D model 3 and in the visualization of the segmented heart valve 2. The modifications are projected onto the respective other modification. Figure 1 The letters depicted in are the labels for the leaflets according to the recognized nomenclature described above.

[0071] Figure 2 FIG. 1 is a schematic diagram of a visualization and topology-based 2D model of a segmented heart valve according to another embodiment of the present invention. Figure 2 This embodiment corresponds substantially to the previous embodiment, except that another heart valve is depicted in FIG. Figure 2 The heart valve in has only two leaflets. However, the process is the same as in the previous embodiment. Figure 2 In the embodiment of the present invention, the commissure points are directly connected to each other using commissure lines. In other words, the commissure lines are not routed on the center of the topology-based 2D model. In this case, the commissure lines can be adjusted by the user so as to fit the boundary lines of the two leaflets 6 of the heart valve.

[0072] Figure 3 is a schematic visualization of a plurality of topology-based 2D models 3 according to an embodiment of the present invention. Figure 3 In the center of the heart valve, an initial topology-based 2D model 3 is depicted. That is, the initial topology-based 2D model 3 is not adjusted to the real heart valve, but has an initial setting. In this embodiment, the initial topology-based 2D model 3 has three commissures 4 and three commissures 5. Figure 3 The topology-based 2D model 3 of the initial topology-based 2D model 3 in is an adapted topology-based 2D model 3. The topology-based 2D model 3 is created as in the previous embodiment. Therefore, the user can add, delete and / or shift the commissures 4 in order to adjust the topology-based 2D model 3 to the real heart valve depicted in the visualization of the segmented heart valve. In addition, the labels of the leaflets 6 can be manually modified by the user. Figure 3In any topology-based 2D model 3, the center of the topology-based 2D model 3 is a circle coaxial with the outer contour of the topology-based 2D model 3. A label indicating the type of heart valve is provided within the center circle. The type of heart valve can be determined based on the morphology of the heart valve. In addition, each commissure 5 is connected to the center of the topology-based 2D model 3. For simplicity, Figure 3 In , only reference labels are added to the original topology-based 2D model.

[0073] Figure 4 is a schematic visualization of a topology-based 2D model adjusted according to an embodiment of the present invention. Figure 4 The left side of the figure depicts the initial topology-based 2D model 3. The user can then grab the joint point 4 with the pointing device 13 and drag the joint point 4 in the desired direction along the model ring 9 (refer to Figure 4 In other words, the topology-based 2D model 3 can be a "valve joystick" because adjustments to the commissures 4 can also be automatically applied to the segmented heart valve ( Figure 4 ). The topology-based 2D model 3 is displayed on a 4D echo segment (i.e., a visualization of the segmented heart valve 2) of the valve in which the valve annulus (i.e., the segmented annulus 12) has been segmented. The complete topology-based 2D model 3 can be rotated to fit the alignment of the echo segments. By means of the "+" and "-" buttons around the model annulus, commissure points 4 can be added and removed in each segment of the topology-based 2D model 3 to fit the number and morphology of the leaflets 6. Drag-and-drop of the commissure points 4 along the model annulus 9 adjusts the commissure points 4 on the echo ring 12. In the case of the tricuspid valve, the leaflets 6 are labeled according to the nomenclature proposed by Hahn et al. All displays of the topology-based 2D model 3 on the echo segments are tracked through a complete cardiac cycle. The morphology selected based on the number and position of the commissure points 4 is mapped to the proposed nomenclature and a type is assigned in the case of the tricuspid valve. An added value is that the leaflets 6 can be continuously visualized even during limited visibility of the echo segments. An alternative embodiment that can be used as an option to accelerate the workflow is a model preselection based on a graphical representation of the morphology. By selecting the valve type, it is displayed on the echo segment and can be adjusted by landmarks on the segment itself (ie, commissures). The landmarks can be tracked through one cardiac cycle.

[0074] Figure 5Ais a schematic diagram of a visualization of a segmented heart valve 2 and a topology-based 2D model 3 according to an embodiment of the present invention. The segmented heart valve 2 and the topology-based 2D model 3 correspond to the segmented heart valve 2 and the topology-based 2D model 3 according to any one of the aforementioned embodiments. In the present embodiment, in the segmented heart valve 2 and the topology-based 2D model 3, at least one flow phenomenon 7 is additionally indicated. Therefore, in the present embodiment, a parameter display is provided, which projects the flow phenomenon from the 4D echo (i.e., the segmented heart valve 2) to the topology-based 2D model 3. Therefore, the present embodiment can provide a "valve dashboard". That is, the topology-based 2D model 3 can be considered as a "valve dashboard" because it displays the flow phenomena in the heart valve.

[0075] Specifically, as an example of a flow phenomenon 7 at a segmented heart valve 2, the regurgitant flow is dynamically projected or displayed in a 4D echo during a time period when the valve is typically closed (mitral valve, tricuspid valve: systole, pulmonary artery, aorta: diastole). Then, the location of the leakage area on the topology-based 2D model 3 is displayed. In addition, the area size of the leakage is also displayed proportionally on the topology-based two-dimensional model 3. In addition, the leakage magnitude through a heat map or contour map (isovalue line) is displayed on the projection. The visualization of the flow phenomenon 7 in the topology-based 2D model 3 is achieved by a marker 8, which can be changed relative to its visualization properties. That is, the marker 8 can be individually adapted to the user's preferences.

[0076] In other words, the "valve dashboard" of the present invention provides visualization of the location, extent and direction of flow phenomena (e.g., regurgitation) in a 4D echo (i.e., a segmented heart valve 2) in several ways (e.g., as a vector field or isokinetic shell). This information is then projected onto a topology-based 2D model 3 in the 4D echo. The projected area is calculated. The location and size of the area is then projected onto the topology-based 2D model 3 for easier positioning in surgical planning. The local magnitude of the flow phenomenon (e.g., leak) is indicated by contour lines or heat maps.

[0077] Figure 5B is a schematic visualization of a segmented heart valve and a topology-based 2D model according to an embodiment of the present invention. Figure 5B Basically Figure 5A Correspondingly, except Figure 5B The heart valve in the heart has only two leaflets.

[0078] Figure 6A user interface 10 according to an embodiment of the present invention is shown. In this setting, a segmented heart valve 2 and a topology-based 2D model 3 are depicted on a conventional computer screen 14. The screen 14 may include a panel 15 of buttons and sliders that allow the user to tilt, zoom, move or otherwise manipulate the segmented heart valve 2 and / or the topology-based 2D model 3. Also in such a user interface 10, having a volume rendered VOI that is locked to the location of a feature of interest on a dynamic model of a dynamic anatomical structure (such as a beating heart) is a useful tool. The display may be controlled by a computer 16 such as a PC, which includes a processor 17 and a hard disk 18. The user interface may have input tools such as a keyboard 19 and / or a mouse (i.e., a pointing device) 20.

[0079] The above discussion is intended only to illustrate the present system and should not be construed as limiting the appended claims to any particular embodiment or group of embodiments. Therefore, although the present system has been described in particular detail with reference to exemplary embodiments, it should also be understood that a person skilled in the art may devise numerous modifications and alternative embodiments without departing from the broader and intended spirit and scope of the present system as set forth in the appended claims. Accordingly, the description and drawings should be regarded as illustrative and are not intended to limit the scope of the appended claims.

Claims

1. A computer-implemented method for adjusting the morphology of a heart valve model segmented from a 3D or 4D volume dataset, the method comprising: providing a 3D visualization of a segmented heart valve (2) on a user interface, wherein the heart valve is segmented from the 3D or 4D volume data set, creating a topology-based 2D model of the heart valve (3), wherein the topology-based 2D model is a simplified visualization of the properties of the heart valve, displaying the 2D model on the user interface (3), User input data for further adjusting the 2D model (3) is received via the user interface.

2. The method according to claim 1, wherein: The topology-based 2D model (3) is created by mapping the segmented heart valve (2) to scale onto a plane.

3. The method according to claim 1 or 2, wherein: The topology-based 2D model (3) is displayed on the user interface next to the 3D visualization of the segmented heart valve (2).

4. A method according to any preceding claim, wherein: The 2D model comprises commissure points (4) defining at least one commissure line (5) indicating a separation line between at least two leaflets (6) of the heart valve.

5. The method according to claim 4, wherein: The user input comprises an instruction to modify the position of at least one commissure point (4) of the topology-based 2D model (3).

6. The method according to claim 4 or 5, wherein: The commissure points (4) and the commissure lines (5) are displayed both on the topology-based 2D model (3) and on the 3D visualization of the segmented heart valve (2).

7. The method according to any one of claims 4 to 6, wherein: The user input includes instructions to add new juncture points and / or delete juncture points.

8. The method according to any one of claims 4 to 7, wherein: The method further includes marking a plurality of leaflets (6) defined by the at least one commissure line (5).

9. A method according to any one of the preceding claims, wherein: The visualization of the segmented heart valve (2) also includes the structures surrounding the heart valve.

10. A method according to any one of the preceding claims, wherein: The visualization of the segmented heart valve (2) is a dynamic 4D visualization of the heart valve.

11. A method according to any one of the preceding claims, wherein: The method further comprises: projecting the flow phenomenon (7) onto the 3D visualization of the heart valve (2) based on the flow information, identifying locations on the topology-based 2D model (3) corresponding to locations of the flow phenomena, and A marker (8) is provided at the identified position.

12. The method according to claim 11, wherein: The marking (8) comprises a computer graphic representation of the flow phenomenon (7), in particular contour lines, vector fields, streamlines and / or a color map, indicating the magnitude of the flow phenomenon (7).

13. The method according to claim 11 or 12, wherein: The method further comprises receiving user input indicating locations on the topology-based 2D model (2) where markers should not be displayed.

14. A computer program comprising program code which, when executed on a control unit, is configured to perform the method according to any of the preceding claims.

15. A system (1) for analyzing a heart valve, comprising: A user interface (10) configured to receive user input, A control unit (11) configured to execute the method according to any one of claims 1 to 13.

Images