Systems and methods for medical image annotation and anatomical landmark height localization

Abstract

The invention relates to a system and method for medical image annotation and anatomical landmark height localization. Methods and systems for image annotation for anatomical landmark measurements are provided herein. In one example, a method includes obtaining an image data set acquired by an imaging system (902); displaying an image of the image data set within a graphical user interface (GUI) on a display device communicatively coupled to the imaging system (904); determining a first anatomical landmark point and a second anatomical landmark point of the image (906, 908); generating an annotation overlay (910) based at least in part on the first anatomical landmark point and the second anatomical landmark point, where the annotation overlay includes a first line (912) connecting the first anatomical landmark point and the second anatomical landmark point, a second line (914) perpendicular to the first line, and a third line (916) perpendicular to the second line, wherein the second line connects the first line to the third line and defines a height of the third line above the first line; and displaying and saving the annotation overlay.

Description

Technical Field

[0001] The present description relates generally to methods and systems for medical image annotation, and more particularly to determining the height and position of anatomical landmarks. Background Art

[0002] Medical ultrasound is an imaging modality that uses ultrasound waves propagated through the internal structures of a patient's body to produce corresponding images. For example, an ultrasound probe including multiple transducer elements transmits ultrasound waves that are reflected or returned, refracted, or absorbed by structures in the body. Medical ultrasound modalities (such as echocardiography) are also used before, during, and / or after a procedure to image internal structures related to the procedure. For example, for certain cardiac procedures (such as minimally invasive cardiac procedures) that employ catheters and / or other devices, ultrasound images (such as echocardiographic images) can be used to guide the positioning of the catheter. Summary of the invention

[0003] In one embodiment, a method includes obtaining an image data set acquired by an imaging system; displaying an image of the image data set within a graphical user interface (GUI) on a display device communicatively coupled to the imaging system; determining a first anatomical landmark point and a second anatomical landmark point of the image; generating an annotation overlay based at least in part on the first anatomical landmark point and the second anatomical landmark point; and displaying and saving the annotation overlay.

[0004] It should be understood that the above summary is provided to introduce a series of concepts further described in the detailed description in a simplified form. It is not meant to identify key features or essential features of the claimed subject matter, the scope of which is uniquely defined by the claims that follow the detailed description. Furthermore, the claimed subject matter is not limited to implementations that address any disadvantages noted above or in any part of this disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The present invention will be better understood by reading the following description of non-limiting embodiments with reference to the accompanying drawings, in which:

[0006] Figure 1 An imaging system according to an embodiment is schematically illustrated.

[0007] Figure 2 Shows Figure 1 A block diagram of a processor of an imaging system.

[0008] Figure 3 Shows the display by Figure 1 A first example graphical user interface (GUI) for images acquired by an imaging system.

[0009] Figure 4A second example GUI is shown in which a first mitral valve annulus point is indicated.

[0010] Figure 5 A second example GUI is shown displaying an annotation overlay having a first height displayed.

[0011] Figure 6 A second GUI is shown displaying the annotation overlay having a second height displayed.

[0012] Figure 7 A third example GUI displaying a saved annotation overlay is shown.

[0013] Figure 8 A three-dimensional ultrasound image displayed in a fourth GUI with an annotation overlay is shown.

[0014] Fig. 9 A flow chart illustrating a method for generating an annotation overlay via user input is shown.

[0015] Fig.10 A flow chart illustrating a method for generating annotation overlays via an automated algorithm is shown. DETAILED DESCRIPTION

[0016] The following description relates to various embodiments of systems and methods for annotating medical images. More specifically, systems and methods for medical image annotation for determining the location of a transseptal puncture (TSP) are provided herein. Preoperative, intraoperative, and postoperative imaging of the patient's relevant internal structures is often used to help surgeons and other medical personnel determine the proper positioning of instruments, the size of hardware, etc. In minimally invasive cardiac procedures, a device is manipulated through the venous system to reach the heart. In some examples, such as procedures such as mitral valve replacement, patent foramen ovale closure, left atrial appendage occlusion, the device is manipulated into the left atrium and then through the atrial septum via the TSP to the right atrium. The TSP requires precise positioning in order for the device to continue to be manipulated through the vascular system appropriately to perform the corresponding procedure.

[0017] Preoperative imaging is recommended as part of the procedure for patients undergoing TSP in order to determine the interatrial septum anatomy, including the location of a portion of the interatrial septum that may be used for TSP and that may be used to detect septal abnormalities. Preoperative imaging may include computed tomography (CT) images, ultrasound images (such as transesophageal echocardiogram (TEE)), fluorescence images, etc. Intraoperative imaging is also recommended to allow visualization of the interatrial septum, and particularly to determine the positioning of the device (such as a sheath) in real time (e.g., without intentional delay). Because fluorescence images do not display the interatrial septum anatomy well, and intraoperative CT is impractical for intraoperative purposes, echocardiography, TEE, or intracardiac echocardiography (ICE) have been used to visualize the relevant anatomical structures as well as the location of the device. For example, the real-time location of the device can be determined by visualizing a "bump" of the interatrial septum, which is a protrusion of the septum caused by pressure from the device on the septum.

[0018] The location of the TSP is typically planned to be above the height of the patient's mitral annulus (referred to as the puncture height). A common puncture height is 4 cm above the mitral annulus, although individual patient anatomy may result in a different puncture height being recommended. For both preoperative imaging and intraoperative imaging, measuring the ideal puncture height is performed by interactive annotation of a medical image (such as an echocardiogram image). Interactive annotations are typically time-consuming and may be inaccurate if the annotated measurement lines are improperly positioned or angled. Annotations may include drawing multiple lines on an image in a graphical user interface (GUI), and if any of these lines is repositioned, other lines may also need to be repositioned, which is time-consuming for the user. In addition, in order to appropriately define the puncture height, the level of the puncture (e.g., in some examples, measured at the ridge position) may be drawn parallel to the mitral annulus, and thus the annotated line indicating the puncture height therebetween may be perpendicular to the mitral annulus. Inaccurate positioning of the line and therefore inaccurate measurements may result in incorrect placement of the device within the atrium.

[0019] Systems and methods for annotating medical images to determine the location of a TSP or other application are presented herein, which solve the above-mentioned problems. An annotation tool within a GUI may allow a user to indicate a first anatomical landmark in a medical image using user input. An annotation overlay may be displayed based on user input defining a first anatomical landmark and a height (e.g., a puncture height), which may be predefined via user input and / or determined via detection of a landmark (such as a ridge location). The annotation overlay may indicate a location (such as the location of a puncture site), and may be able to edit the height via user input without changing the annotated location of the first anatomical landmark. The annotation tool may be used for intraoperative images as well as preoperative images and / or postoperative images. In a use case scenario, a live intraoperative image may be annotated to determine the current location of a device based on a visualized ridge. Portions of the methods described herein, including determining the location of an anatomical landmark (such as a mitral valve annulus) and determining the height based on another anatomical landmark (such as a ridge location), may be automatic. In this way, the time spent by the user may be reduced. Additionally, because the annotation overlay defines a vertical set of lines, accuracy may be improved.

[0020] Figure 1 An example imaging system including a display device and an imaging processing system is shown in FIG. Figure 2 As shown, the imaging processing system can be used to analyze and annotate Figure 1 Images acquired by the imaging system. Figures 3 to 8 shows that the Figure 1 An example GUI of various annotations and overlays displayed on a medical image acquired by an imaging system. Fig. 9 A method for generating a TSP annotation overlay via user input is illustrated, and Fig.10 A method for automatically determining TSP annotation coverage is illustrated.

[0021] The methods described herein may be implemented on various types of medical images, such as two-dimensional (2D) and / or three-dimensional (3D) ultrasound images like echocardiograms, CT images, etc. Such images may be acquired by an imaging system, such as an ultrasound system or a CT system. Figure 1An example system for implementing these methods is shown in . Therein, a block diagram of system 100 is depicted according to one embodiment. In the illustrated embodiment, system 100 is an imaging system, and more specifically an ultrasound imaging system 100. As shown, ultrasound imaging system 100 includes a plurality of components. The components may be coupled to each other to form a single structure. In one example, ultrasound imaging system 100 is an integrated system that can be moved from one room to another (e.g., portable). For example, ultrasound imaging system 100 may include one or more components that are configured to couple ultrasound imaging system 100 to a wheeled vehicle. However, in other examples, at least a portion of ultrasound imaging system 100 may be configured to remain stationary and / or fixed in place.

[0022] After the elements 104 of the probe 106 transmit the pulsed ultrasound signals into the body (of the patient), the pulsed ultrasound signals are backscattered from structures inside the body (such as blood cells or muscle tissue) to generate echoes that return to the elements 104. The echoes are converted into electrical signals or ultrasound data by the elements 104, and the electrical signals are received by the receiver 108. The electrical signals representing the received echoes pass through the receive beamformer 110, which outputs the ultrasound data. Additionally, the transducer elements 104 may generate one or more ultrasound pulses based on the received echoes to form one or more transmit beams.

[0023] According to some embodiments, the probe 106 may contain electronic circuits to perform all or part of the transmit beamforming and / or receive beamforming. For example, all or part of the transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110 may be located within the probe 106. In the present disclosure, the term "scan" may also be used to refer to the process of sending and receiving ultrasound signals to collect data. In the present disclosure, the term "data" may be used to refer to one or more data sets collected by the ultrasound imaging system. In one embodiment, the data collected via the ultrasound system 100 can be used to train a machine learning model. The user interface 115 can be used to control the operation of the ultrasound imaging system 100, including controlling the input of patient data (e.g., patient medical history), changing scanning or display parameters, starting a probe repolarization sequence, etc. The user interface 115 may include one or more of the following: a rotating element, a mouse, a keyboard, a trackball, a hard key linked to a specific action, a soft key that can be configured to control different functions, and a graphical user interface displayed on a display device 118.

[0024] The ultrasound imaging system 100 also includes a processor 116 that controls the transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110. The processor 116 is in electronic communication (e.g., communicatively connected) with the probe 106. For the purposes of this disclosure, the term "electronic communication" may be defined as including both wired communication and wireless communication. The processor 116 may control the probe 106 to acquire data according to instructions stored in the processor's memory and / or the memory 120. The processor 116 controls which of the elements 104 are active and the shape of the beam emitted from the probe 106. The processor 116 also communicates electronically with the display device 118, and the processor 116 may process data (e.g., ultrasound data) into an image for display on the display device 118. According to an embodiment, the processor 116 may include a central processing unit (CPU). According to other embodiments, the processor 116 may include other electronic components capable of performing processing functions, such as a digital signal processor, a field programmable gate array (FPGA), or a graphics board. According to other embodiments, the processor 116 may include a plurality of electronic components capable of performing processing functions. For example, the processor 116 may include two or more electronic components selected from a list of electronic components, including: a central processing unit, a digital signal processor, a field programmable gate array, and a graphics board. According to another embodiment, the processor 116 may also include a composite demodulator (not shown) that demodulates the RF data and generates raw data. In another embodiment, the demodulation may be performed earlier in the processing chain. The processor 116 is adapted to perform one or more processing operations according to a plurality of optional ultrasound modalities on the data. In one example, the data may be processed in real time during a scanning session because the echo signal is received by the receiver 108 and sent to the processor 116. For the purposes of this disclosure, the term "real time" is defined as including a program executed without any intentional delay. For example, an embodiment may acquire images at a real-time rate of 7 frames / second to 20 frames / second. The ultrasound imaging system 100 may acquire 2D data of one or more planes at a significantly faster rate. However, it should be understood that the real-time frame rate may depend on the length of time it takes to acquire each frame of data for display. Therefore, when a relatively large amount of data is acquired, the real-time frame rate may be slower. Thus, some embodiments may have real-time frame rates significantly faster than 20 frames / second, while other embodiments may have real-time frame rates less than 7 frames / second. Data may be temporarily stored in a buffer (not shown) during a scanning session and processed in less than real-time in real-time or off-line operations. Some embodiments of the present invention may include multiple processors (not shown) to handle processing tasks handled by processor 116 according to the exemplary embodiments described above.For example, a first processor may be utilized to demodulate and extract the RF signal prior to displaying an image, while a second processor may be utilized to further process the data (eg, by augmenting the data as further described herein). It should be appreciated that other embodiments may utilize different processor arrangements.

[0025] The ultrasound imaging system 100 can continuously acquire data at a frame rate of, for example, 10 Hz to 30 Hz (e.g., 10 to 30 frames per second). The image generated based on the data can be refreshed at a similar frame rate on the display device 118. Other embodiments can acquire and display data at different rates. For example, depending on the size of the frame and the intended application, some embodiments can acquire data at a frame rate less than 10 Hz or greater than 30 Hz. A memory 120 is included to store processed frames of the acquired data. In an exemplary embodiment, the memory 120 has sufficient capacity to store at least a few seconds of ultrasound data frames. The data frames are stored in a manner that is easy to retrieve according to their acquisition order or time. The memory 120 may include any known data storage medium.

[0026] In various embodiments of the present invention, the processor 116 may process data in different mode-related modules (e.g., B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastic imaging, TVI, strain, strain rate, etc.) to form 2D or 3D data. For example, one or more modules may generate B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastic imaging, TVI, strain, strain rate, and combinations thereof, etc. As an example, one or more modules may process color Doppler data, which may include traditional color blood flow Doppler, power Doppler, HD flow, etc. The image lines and / or frames are stored in the memory and may include timing information indicating the time when the image lines and / or frames are stored in the memory. These modules may include, for example, a scan conversion module that performs a scan conversion operation to convert the acquired image from beam space coordinates to display space coordinates. A video processor module may be provided that reads the acquired image from the memory and displays the image in real time while a procedure (e.g., ultrasound imaging) is performed on the patient. The video processor module may include a separate image memory, and the ultrasound image may be written to the image memory for reading and display by the display device 118 .

[0027] In various embodiments of the present disclosure, one or more components of the ultrasound imaging system 100 may be included in a portable handheld ultrasound imaging device. For example, the display device 118 and the user interface 115 may be integrated into the external surface of the handheld ultrasound imaging device, which may further contain a processor 116 and a memory 120. The probe 106 may include a handheld probe that electronically communicates with the handheld ultrasound imaging device to collect raw ultrasound data. The transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110 may be included in the same or different parts of the ultrasound imaging system 100. For example, the transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110 may be included in a handheld ultrasound imaging device, a probe, and a combination thereof.

[0028] After performing a two-dimensional ultrasound scan, a data block including scan lines and their samples is generated. After applying the back-end filter, a process called scan conversion is performed to transform the two-dimensional data block into a displayable bitmap image with additional scanning information (such as depth, angle of each scan line, etc.). During scan conversion, interpolation techniques are applied to fill in the missing holes (e.g., pixels) in the resulting image. These missing pixels occur because each element of the two-dimensional block should generally cover many pixels in the resulting image. For example, in current ultrasound imaging systems, bicubic interpolation is applied, which utilizes neighboring elements of the two-dimensional block.

[0029] The ultrasound images acquired by the ultrasound imaging system 100 may be further processed. In some examples, the ultrasound images generated by the ultrasound imaging system 100 may be sent to an image processing system, where in some examples, the ultrasound images may be analyzed to determine the locations of anatomical landmarks (e.g., the mitral valve annulus). In some examples, a machine learning algorithm may be employed to analyze the ultrasound images. As an example, different views showing the locations of the mitral valve annulus and protuberance from various ultrasound images are annotated and trained as a segmentation-based network. Based on the segmentation-based network, the acquired ultrasound images may be annotated in an automatic manner, thereby detecting the locations of the mitral valve annulus and protuberance based on the trained segmentation-based network.

[0030] Although described herein as separate systems, it should be understood that in some examples, the ultrasound imaging system 100 includes an image processing system, for example as part of the processor 116. In other examples, the ultrasound imaging system 100 and the image processing system may include separate devices. For example, the image processing system may be provided on a workstation.

[0031] It should be understood that although an ultrasound imaging system is described herein, this is a non-limiting example of an imaging system that can be used to acquire medical images on which the methods herein can be implemented. Other imaging systems (such as CT systems, magnetic resonance imaging (MRI) systems) can be used without departing from the scope of the present disclosure.

[0032] refer to Figure 2 , an image processing system 200 according to an exemplary embodiment is shown. In some examples, the image processing system 200 is incorporated into the ultrasound imaging system 100. For example, the image processing system 200 may be provided in the ultrasound imaging system 100 as a processor 116 and a memory 120. In some examples, at least a portion of the image processing system 200 is disposed at a device (e.g., an edge device, a server, etc.) that is communicatively coupled to the ultrasound imaging system 100 via a wired connection and / or a wireless connection. In some examples, at least a portion of the image processing system 200 is disposed at a separate device (e.g., a workstation) that can receive images / maps from the ultrasound imaging system or from a storage device storing images / data generated by the ultrasound imaging system. The image processing system 200 may be operably / communicatively coupled to a user input device 232 and a display device 234. At least in some examples, the user input device 232 may include a user interface 115 of the ultrasound imaging system 100, and the display device may include a display device 118 of the ultrasound imaging system 100.

[0033] The image processing system 200 includes a processor 204 configured to execute machine-readable instructions stored in a non-transitory memory 206. The processor 204 may be a single-core or multi-core processor, and the programs executed thereon may be configured for parallel processing or distributed processing. In some examples, the processor 204 may optionally include separate components spread across two or more devices, which may be remotely located and / or configured for collaborative processing. In some examples, one or more aspects of the processor 204 may be virtualized and executed by a remotely accessible networked computing device configured in a cloud computing configuration.

[0034] The non-transitory memory 206 may store an annotation module 208. The annotation module 208 may generate an annotation overlay within a graphical interface. The annotation module 208 may store a plurality of models. In some examples, the annotation overlay may be automatically generated based at least in part on an algorithm of a point detection model 210 and a height detection model 212. The point detection model 210 may detect one or more first anatomical landmark locations, and the height detection model 212 may detect a second type of anatomical landmark locations to notify the annotation module 208 of the anatomical landmarks so as to generate an annotation overlay based on those landmarks. In some examples, the annotation module 208 may also include a measurement model 214 that may determine a height measurement. In some examples, the height may be indicated via user input and / or may be automatically determined via the height detection model 212. An exact or near exact height of a second anatomical landmark location may be determined by the measurement model 214. The height of the second anatomical landmark may be measured relative to one or more first anatomical landmark locations. The use case examples as presented herein include the mitral annulus point as a first anatomical landmark and the TSP puncture location as a second anatomical landmark, the height being the puncture height above the mitral annulus.

[0035] The non-transitory memory 206 may also store image data 216, such as Figure 1 The images of the image data 216 may include images having different anatomical features (e.g., different views of the heart) and / or different image qualities that have been acquired by the ultrasound imaging system 100. In addition, the image data 216 may store images, ground truth outputs, iterations of machine learning model outputs, and other types of image data that may be used to train the annotation module 208.

[0036] In some embodiments, non-transitory memory 206 may include components disposed at two or more devices that may be remotely located and / or configured for collaborative processing. In some examples, one or more aspects of non-transitory memory 206 may include a remotely accessible networked storage device configured in a cloud computing configuration.

[0037] The user input device 232 may include one or more of a touch screen, a keyboard, a mouse, a trackpad, a motion sensing camera, or other devices configured to enable a user to interact with and manipulate data within the image processing system 200. In some examples, the user input device 232 may enable a user to make annotation inputs and otherwise interact with a GUI displayed on the display device 234. For example, the user input device 232 may enable a user to make annotation inputs indicating a first mitral valve annulus point and a second mitral valve annulus point, which may allow an annotation overlay to be generated. In addition, the user input device 232 may enable a user to edit the height of the annotation overlay, for example via a drag input and a release input.

[0038] The display device 234 may include one or more display devices utilizing almost any type of technology. In some examples, the display device 234 may include a computer monitor and may display images within a GUI, such as ultrasound images acquired using the ultrasound imaging system 100. The display device 234 may be combined with the processor 204, the non-transitory memory 206, and / or the user input device 232 in a common housing, or may be a peripheral display device and may include a monitor, a touch screen, a projector, or other display devices known in the art, which may enable a user to view images generated by an imaging system (such as the ultrasound imaging system 100) and / or interact with various data stored in the non-transitory memory 206. In some examples, the display device 234 may be configured to display images stored in a memory by a storage database (such as a picture archiving and communication system (PACS)). In other examples, the display device 234 may be configured to display images acquired in real time, for example, when the display device 234 is combined with the imaging system in a common housing, the display device 234 may display images acquired in real time. In some examples, display device 234 may be configured to display both images stored in the database and real-time images. In this way, both preoperative images stored in the PACS and intraoperative images acquired during the procedure may be annotated on the same or different display devices.

[0039] It should be understood that Figure 2 The image processing system 200 shown is for illustration and not for limitation. Another suitable image processing system may include more, fewer, or different components.

[0040] Figure 3 An example first GUI 300 is shown. The first GUI 300 may be displayed on a display device such as Figure 1 The display device 118 and / or Figure 2 The image displayed within the first GUI 300 may be displayed by an imaging system (such as a display device 234) that is communicatively coupled to the display device on which the first GUI 300 is displayed. Figure 1 In some examples, the first GUI 300 may display the first image 302 within a viewport 314 of the first GUI 300. The size and shape of the viewport 314 may depend on the type of image. For example, the first image 302 as displayed in the first GUI 300 may be an echocardiography image, and thus the viewport 314 may be shaped as a triangle, pie, or the like corresponding to the distribution of ultrasound waves from the transducer. Other types of images may be displayed in viewports of different shapes.

[0041] The first GUI 300 may include an image menu 304 through which a user may select one of a plurality of images to be displayed within a viewport 314. The plurality of images may include images from different scans of a patient, different views of the same scan (e.g., from different axes), etc. As an example, for a TEE scan, the available plurality of images may include mid-esophageal four-chamber, mid-esophageal two-chamber, mid-esophageal long axis, transgastric fundus short axis, and transgastric mid-papillary short axis. As another example, the available plurality of images may include a TEE scan (e.g., all images of a TEE scan included therein), an ICE scan (e.g., all images of an ICE scan included therein), and a cardiac CT scan. As noted, the shape of the viewport 314 may depend on which of the plurality of images is selected.

[0042] The first GUI 300 may also include a tool menu 306. The tool menu 306 may include a plurality of scan types that may be selected, for example, in the first GUI 300, the scan type is selected as cardiac. In some examples, the scan type may depend on the selected image that is displayed. The tool menu 306 may also include a plurality of tool types 308. The plurality of tool types 308 may include various annotation tools, measurement tools, parameter tools, etc. Each tool type in the tool types 308 may be selected to display a corresponding drop-down menu. For example, the plurality of tool types 308 may include an automatic ejection fraction calculator tool menu that displays one or more ejection fraction calculator tools when selected. One tool type in the plurality of tool types 308 may be an interventional tool type 310. The interventional tool type 310 may include various tool subtypes related to one or more procedures, interventions, surgical procedures, etc. in the drop-down menu. As a non-limiting example, the various tool subtypes may include left atrial appendage (LAA), TSP, and aortic valve. Each of the sub-types, when selected via user input, may initiate a corresponding drop-down menu displaying its corresponding tools.

[0043] A first GUI tool of interventional tool type 310 may include annotate height tool 312. As will be described further below, annotate height tool 312 may allow for the determination and measurement of the height of an anatomical location (eg, a TSP location) based on various anatomical landmarks determined via user input or an automated algorithm.

[0044] The first GUI 300 may also show a heart rhythm 330 in the case where the heart rhythm is classified for the selected and displayed image. The heart rhythm 330 as displayed may correspond to the displayed image. In some examples, a tracker 332 may indicate the position of the displayed image in the heart rhythm. For example, the heart rhythm may be monitored during an echocardiographic ultrasound image, and thus the heart rhythm 330 may be displayed when the echocardiographic image is selected for display within the first GUI 300. The tracker 332 may move along the heart rhythm 330 as the displayed echocardiographic image moves through the time frame in which it is captured. However, for cardiac CT, in the case of acquiring static images rather than an echocardiogram including moving images, the heart rhythm may not be monitored, and thus the heart rhythm 330 may not be displayed.

[0045] Although in Figure 3 Although not explicitly shown, the first GUI 300 may also include various demographic data of the patient for whom the first image 302 was acquired, including name, date of birth, age, etc., as well as data of the first image, including scan type, parameters, acquisition date, etc.

[0046] Steering Figure 4 , showing a portion of an example second GUI 400. In some examples, the second GUI 400 may be about Figure 3 The first GUI 300. The second GUI 400 may be displayed on a display device (such as Figure 1 The display device 118 and / or Figure 2 234) that is communicatively coupled to an imaging system (such as Figure 1 Ultrasound imaging system 100) for acquiring images displayed in the second GUI 400.

[0047] Similar to Figure 3 As described, the second GUI 400 may display a second image 402. In some examples, the second image 402 of the patient may be acquired before, during, or after a corresponding procedure. For example, the second image 402 may be acquired intraoperatively during a minimally invasive cardiac procedure and may be displayed by a display device in real time and / or at a later time. In such examples, as described herein, the second image 402 may include multiple anatomical features of the patient's heart. The multiple anatomical features include a mitral valve annulus, an interatrial septum, and the like. Each of the anatomical features may have one or more landmarks visible within the second image 402. One or more landmarks may be annotated and / or analyzed to generate an annotation overlay and / or determine a height.

[0048] The second image 402 may be annotated via various tools accessible through the second GUI 400 and / or via an automated algorithm. As an example, a TSP annotation tool (also referred to herein as a TSP height tool) may be accessible via the second GUI 400. When selected, the TSP height tool may allow a user to annotate the second image 402 to determine a recommended puncture height for the TSP.

[0049] The annotation of the second image 402 may include a first anatomical landmark 410. The first anatomical landmark 410 may be selected via a first type of user input via an annotation tool. In an example where the annotation tool is a TSP annotation tool, the first anatomical landmark 410 may indicate the location of a first side 490 of the mitral valve annulus. The first type of user input may be a left mouse click, a touch pad touch, a touch of a touch screen, or the like. The first anatomical landmark 410 may be displayed on the second image 402 as a cross-hatched mark or other suitable point indicator. In some examples, the first anatomical landmark 410 may be selectable to edit the location of the first anatomical landmark 410 or to indicate a connected second anatomical landmark, as will be further described below.

[0050] Reference now Figure 5 , Figure 4 The second GUI 400 is shown as including an annotation overlay 504. As noted above, the second GUI 400 may be displayed on a display device communicatively coupled to the imaging system, and an image may be displayed within the GUI, which in some examples was acquired by the imaging system. The second GUI 400 includes annotation tools accessible via a menu, such as with respect to Figure 3 When selected, one or more flags may be indicated via user input. A first type of user input may be indicated such as Figure 4 The annotation overlay 504 may include a plurality of lines, including a first line 510 , a second line 512 , and a third line 514 .

[0051] The second anatomical landmark 508 may be indicated via a second type of user input. In an example where the annotation tool is a TSP annotation tool, the second anatomical landmark 508 may indicate the location of the second side 594 of the mitral valve annulus. In some examples, the second type of user input may be the same as the first type of user input. In other examples, the second type of user input may be different from the first type of user input, for example, the second type of user input may be a cursor drag and a mouse click / touch pad touch / finger touch release.

[0052] The first line 510 may be generated between the first anatomical landmark 410 and the second anatomical landmark 508. The first line 510 may be a straight line connecting the two landmarks, thereby indicating an estimated angle of the mitral valve annulus within the second image 402. The first line 510 may have a first length 530. At the midpoint of the first length 530, a second line 512 may be generated perpendicular to the first line 510. The first end 590 of the second line 512 may contact the midpoint of the first line 510. The third line 514 may be generated in contact with the second end 592 of the second line 512. The third line 514 may be perpendicular to the second line 512 and parallel to the first line 510. In some examples, the third line 514 may extend toward the first side 490 by at least the same length as the first length 530. In other examples, the third line 514 may extend toward both the first side 490 and the second side 594. In examples where the third line 514 extends toward the first side 490 by at least the same length as the first length 530 , the second length 532 of the third line 514 may be equal to the first length 530 .

[0053] The third length 534 of the second line 512 extending between the first line 510 and the third line 514 can be a height. When the annotation tool is a TSP annotation tool, the height can be a puncture height. The height can be defined by a preset height (e.g., 4 cm) stored in a memory via a user input to the height element 520 and / or via a user input that changes the position of the third line 514. As an example, initially based on the preset height, the third length 534 can be 4 cm (or proportional to 4 cm according to display parameters). The user can input different heights into the height element 520 via a user input (such as a combination of a mouse click and a keyboard touch) to define a second height of 3.5 cm. The third length 534 can be changed to 3.5 cm (or proportional to 3.5 cm) in real time to match the defined second height. The change in height may not affect the position of the first line 510. As another example, the third length 534 can be changed via a user input (such as a mouse click and dragging the third line 514, etc.). The height displayed in the height element 520 can be updated in real time as the third length 534 changes. In this manner, the height may be set to a desired height based on the anatomy of the patient shown in the second image 402 and based on the height over the first anatomical landmark, as indicated by the third length 534. Furthermore, the annotation overlay may be interactive based on user input and may be updated in real-time in response to various user inputs.

[0054] Temporary turn Figure 6, shows an annotation overlay 504 with a modified third length 604 displayed within the second GUI 400. The modified third length 604 can extend between the first line 510 and the third line 514, just as the third length 534. As described, the third length 534 changes to the modified third length 604 in response to one or more of the various user inputs. For example, the position of the third line 514 can be changed via user input and / or the height within the height element 520 can be changed via user input. Figure 5 The height shown in the height element 520 is 4 cm, and the third length 534 corresponds to a height of 4 cm. Figure 6 The height displayed within the height element 520 in FIG. 5 is 2 cm, and the modified third length 604 may correspond to a height of 2 cm.

[0055] Return to Figure 5 In some examples, the image displayed within the second GUI 400 may be obtained intraoperatively. In a use case example presented by the TSP annotation tool, the tool may be used on an image acquired during a minimally invasive cardiac procedure so that a bulge in the interatrial septum caused by a device puncturing through the septum is visualized. The height of the third line 514 may be aligned with the location of the bulge to define a puncture height (e.g., a third length 534) based on the location of the bulge. The height of the third line 514 may be aligned with the location of the bulge in an automatic manner based on detection of the location of the bulge and / or based on user input to change the location of the third line 514.

[0056] When editable, the annotation overlay 504 may be displayed as a first color, indicating that the position of the mitral annulus point and the third line can be edited by the user. In some examples, the position of the mitral annulus point and the height of the third line may be determined by one or more algorithms. The annotation overlay 504 may be generated based on the determined points and displayed in a first color. The various points and positions determined by the algorithm may be changed via user input to the GUI. Once the user determines that the positions and points are desired, the annotation overlay 504 may be saved to memory.

[0057] Figure 7 A second GUI 400 is shown with a saved annotation overlay 702. The saved annotation overlay 702 may be displayed within the second GUI 400 in a second color that is different from the first color. The different colors may indicate to the user whether the annotation overlay is editable or whether it has been saved. For example, during surgery, the user may view an annotated preoperative image and compare it to the intraoperative image during annotation. The annotations in the annotated preoperative image may be saved and therefore displayed in the second color, thereby indicating to the user that the puncture height of the annotation overlay is not editable. In this way, the different colors may reduce the likelihood of the user confusing the two images, and thus more accurately measure the predicted puncture height and the actual puncture height.

[0058] The first image 302 and the second image 402 shown herein are 2D images, although other types of images, including 3D images, may also be displayed and annotated using the TSP height tool and other tools. Figure 8 An example of a 3D image 802 displayed within the third GUI 800 is shown. The 3D image 802 may be generated based on one or more 2D images, such as a first 2D image 804 and a second 2D image 806. An annotation overlay 810 may be displayed on top of the 3D image 802 based on a plurality of points and / or locations indicating the location and puncture height of the mitral annulus, similar to the description of the annotation overlay 810 in relation to the mitral annulus. Figures 4 to 7 The height of the puncture height above the mitral annulus may be indicated by a height element 808 , similar to as in the second GUI 400 .

[0059] Now turn to Fig. 9 , a flowchart illustrating a method 900 for generating an annotation overlay is shown. Figure 1 to Figure 2 The method 900 is described with reference to the system depicted in the present disclosure, but it should be understood that similar methods may be used with other systems without departing from the scope of the present disclosure. The method 900 may be executed via instructions stored in a non-transitory memory of one or more computing devices. For example, the method 900 may be executed by the processor 204 based on instructions stored in a non-transitory memory of the computing device. Figure 2 The instructions in the non-volatile memory 206 of the image processing system 200 are executed.

[0060] At 902, method 900 includes obtaining image data of a patient. The image data may be 2D image data or 3D image data acquired by an imaging system. For example, the image data may be acquired by Figure 1 The image data may be acquired by an ultrasound imaging system 100 and may include multiple 2D ultrasound images. In some examples, multiple 2D ultrasound images may be acquired over a period of time, such as acquiring images of the heart over a period of several heartbeats. In some examples, the image data may be obtained from a database (such as a PACS system) that stores the data in a memory. In other examples, the image data may be obtained in real time by the imaging system and may therefore include live images.

[0061] At 904, method 900 includes displaying an image of the image data within a GUI (such as first GUI 300 and / or second GUI 400 described above). The GUI may be displayed by a display device (such as a display device) that is in communication with one or more computing devices. Figure 1 The display device 118 and / or Figure 2 The image may be displayed on a display device 234 of the GUI. The image may be displayed within a viewport of the GUI, as described above.

[0062] At 906, method 900 includes receiving a first user input indicating a first landmark point. In some examples, the first landmark point may be a first mitral valve annulus point, wherein the first mitral valve annulus point indicates the location of a first side of a mitral valve annulus. The first side of the mitral valve annulus may be used as a first anatomical landmark for an annotation overlay. The first user input (such as a mouse click) may be entered by a user input device (such as a mouse). The first user input may indicate the location of the first landmark point within a corresponding image as displayed within the GUI.

[0063] At 908, method 900 includes receiving a second user input indicating a second landmark point. In some examples, the second landmark point can be a second mitral valve annulus point, wherein the second mitral valve annulus point indicates the location of a second side of the mitral valve annulus. The second side of the mitral valve annulus can be used as a second anatomical landmark for annotation overlay. The second user input can indicate the location of the second landmark point within the corresponding image as displayed within the GUI.

[0064] At 910, method 900 includes generating and displaying an annotation overlay based on the first user input and the second user input. Generating the annotation overlay may include generating a first line between the first landmark point and the second landmark point, as indicated at 912. The first line may be a straight line connecting the first landmark point and the second landmark point. In an example where the first landmark point and the second landmark point are mitral valve annulus points, the first line may approximate the location of the mitral valve annulus. Generating the annotation overlay may also include generating a second line perpendicular to the first line, as indicated at 914. As described with respect to Figure 5 As described above, the first end of the second line may be connected to the first line at the midpoint of the first line, and the second end of the second line may be away from the first line by a distance equal to the puncture height above the anatomical landmark (e.g., the mitral valve annulus). Generating the annotation overlay may also include generating a third line perpendicular to the second line, as indicated at 916. Also as described with respect to Figure 5 As described, the third line may extend in the direction of the first landmark (eg, toward the first end of the mitral valve annulus). In the example where the annotation overlay indicates the location of the TSP, the third line may extend at least to the interatrial septum to indicate the location of the TSP.

[0065] At 918, method 900 includes determining a height of the third line above the first line. Assuming that the second line is disposed between the first line and the third line and is perpendicular to the first line and the third line, the height of the third line above the first line can be the length of the second line. The height can be determined in various ways. For example, a preset height can be stored in a memory and can be obtained, as indicated at 920. The preset height can be an initial height to which the annotation overlay is set when it is initially displayed within the GUI. Alternatively or additionally, a third user input indicating a height can be received, as indicated at 922. As described with respect to Figure 5 and Figure 6As described, the third user input may be a user input to the third line that changes the position of the third line, which inherently changes the length of the second line. The third user input may also be an input height value to a height element within the GUI. The displayed annotation overlay may be updated in real time for both types of input, where the third line may be dragged to a different height in the first type of user input and / or may be updated to the input height value in response to the second type of user input. The third line at a specified height may indicate a third landmark point. In some examples, the third landmark point may be a TSP location along the interatrial septum.

[0066] The annotation overlay may be displayed on top of the displayed image corresponding to the various indicia within the GUI. During initial display and when the annotation overlay is editable via user input, it may be displayed as a first color.

[0067] At 924, method 900 includes saving the annotation data to a memory. The annotation data may include corresponding landmark locations and line lengths, including the height of the third line above the first line (e.g., the length of the second line). The annotation data may be saved for the corresponding display image and may be accessed by a user of the system or other users at a later time to view the annotation overlay on the image within the GUI. In some examples, the annotation data may be saved independently of the corresponding display image to which it was originally generated and may be displayed later on other images. For example, an annotation overlay may be generated and displayed on top of a preoperative echocardiographic image. The annotation overlay data may then be saved to a memory and may be accessed and displayed on an intraoperative echocardiographic image. In this manner, a preoperative TSP position / height may be determined based on preoperative imaging, and the TSP position / height may be compared to a real-time live intraoperative echocardiographic image showing a bulge in the septum from the puncture device to ensure that the position of the bulge is aligned with the TSP position / height. In this way, incorrect positioning of the puncture device may be reduced.

[0068] In general, the method for annotating an image as described herein can reduce the time a physician spends measuring the height between anatomical landmarks. In particular, for the use case of TSP presented herein, the method includes generating vertical lines, which can reduce incorrect positioning and / or measurement of TSP height. Furthermore, because the annotation interacts with user input, a reduced amount of user input can be used to determine the TSP height, thereby reducing the time spent by the user.

[0069] Now turn to Fig.10 , a flowchart illustrating a method 1000 for automatically generating annotation coverage is shown. Figure 1 to Figure 2The method 1000 is described with reference to the system depicted in the present disclosure, but it should be understood that similar methods may be used with other systems without departing from the scope of the present disclosure. The method 1000 may be executed via instructions stored in a non-transitory memory of one or more computing devices. For example, the method 1000 may be executed by the processor 204 based on instructions stored in a non-transitory memory of the computing device. Figure 2 The instructions in the non-volatile memory 206 of the image processing system 200 are executed.

[0070] At 1002, method 1000 includes obtaining image data of a patient. The image data may be 2D image data or 3D image data acquired by an imaging system. For example, the image data may be acquired by Figure 1 The ultrasound imaging system 100 may acquire the image data and may include multiple 2D ultrasound images. In some examples, multiple 2D ultrasound images may be acquired over a period of time, such as acquiring images of the heart over a period of several heartbeats. In some examples, the image data may be obtained from a database (such as a PACS system) that stores the data in a memory. In other examples, the image data may be acquired in real time by the imaging system.

[0071] At 1004, method 1000 includes displaying an image of the image data within a GUI (such as first GUI 300 and / or second GUI 400 described above). The GUI may be displayed by a display device (such as a display device) that is in communication with one or more computing devices. Figure 1 The display device 118 and / or Figure 2 The image may be displayed on a display device 234 of the GUI. The image may be displayed within a viewport of the GUI, as described above.

[0072] At 1006, method 1000 includes determining a first anatomical landmark. Fig. 9 As described, the first anatomical landmark can be a first mitral valve annulus point, although other anatomical landmarks are also possible. As described above, a machine learning algorithm (e.g., a segmentation-based network) can be trained for the first anatomical landmark, and the first anatomical landmark can be detected based on the machine learning algorithm and the acquired image data, as indicated at 1008.

[0073] At 1010, method 1000 includes determining a second anatomical landmark. The second anatomical landmark can be a second mitral valve annulus point, although other anatomical landmarks are possible. Similar to as described above, the second anatomical landmark can be detected based on a machine learning algorithm trained for the second anatomical landmark, as indicated at 1012.

[0074] At 1014, method 1000 includes generating an annotation overlay. Figure 5 and Fig. 9As described in detail, the annotation overlay may include a first line generated between a first anatomical landmark and a second anatomical landmark, a third line parallel to the first line, and a second line perpendicular to both the first line and the third line and positioned between the first line and the third line. The length of the second line may define the height of the third line above the first line, which in some cases may be the puncture height of the TSP. In order to generate the annotation overlay in this way, in some examples, a preset height of the third line above the first line (e.g., the length of the second line) may be obtained, as indicated at 1012. The preset height may be stored in a memory and used as a default value for generating the annotation overlay. Additionally or alternatively, the height may be determined via user input, as indicated at 1014. As described above, the user input may be an input to change the position of the third line to change the length of the second line and / or an input to a height element indicating the height of the third line above the first line in cm.

[0075] In other examples, the location of a bulge in the interatrial septum may be detected to determine the height, as indicated at 1016. In such examples, images acquired intraoperatively may be acquired as the device is manipulated through a blood vessel into the heart. Once in the heart, the device may be positioned against the septum so as to cause the septum to bulge or protrude. The location of the bulge may be the assumed location of the TSP. In some examples, the bulge may be detected based on a machine learning algorithm. The location of the bulge so determined may then define the location of a third line and may therefore define the height of the third line above the first line. The third line may always be parallel to the first line, and the second line so defining the height of the third line may always be perpendicular to the first line (e.g., proximal to the mitral annulus) and the third line (e.g., defining the location of the TSP). The location of the bulge may serve as a third anatomical landmark for annotation overlay.

[0076] At 1018, method 1000 includes displaying an annotation overlay on the image in the GUI. In some examples, the annotation overlay can be displayed in response to the determination / detection of each of the anatomical landmarks, including a first anatomical landmark (e.g., a first mitral valve annulus point), a second anatomical landmark (e.g., a second mitral valve annulus point), and a third anatomical landmark (e.g., a protuberance and / or TSP location). In this manner, an annotation overlay can be displayed on the corresponding anatomical structure showing the location of each of the landmarks therein. Similar to the description of FIG. Figure 5 , Figure 6 and Fig. 9 As described, the annotation overlay may be editable via user input. In addition, the annotated data may be saved to memory for later access by a different user and / or for a different image, such as with respect to Fig. 9 described.

[0077] The technical effect of the systems and methods provided herein is that the time spent in measuring the height / distance between anatomical landmarks can be reduced. Since these methods include generating two parallel lines, wherein a third line between the two parallel lines is perpendicular to the two parallel lines, the third line can allow the height of the second anatomical landmark above the first anatomical landmark to be accurately measured. In this use case example described herein, these methods and systems can allow for more accurate positioning of the puncture device during the procedure. In this way, the time spent in determining the puncture height can be reduced and the need to reposition the annotation line can be reduced. In addition, the generated annotation overlay is interacted via user input. Therefore, processing power can be reduced by reducing the number of inputs required to generate an accurate annotation overlay. Reducing the number of inputs can also reduce the time it takes for the user to generate the annotation overlay when measuring the height.

[0078] The present disclosure also provides support for a method, the method comprising: obtaining an image data set of a patient acquired by an imaging system; displaying an image of the image data set in a graphical user interface (GUI) on a display device communicatively coupled to the imaging system; determining a first anatomical landmark of the image; determining a second anatomical landmark of the image; generating an annotation overlay based at least in part on the first anatomical landmark and the second anatomical landmark; displaying the annotation overlay on top of the image in the GUI; and saving data of the annotation overlay to a memory. In a first example of the method, the annotation overlay comprises a first line connecting the first anatomical landmark and the second anatomical landmark, a second line perpendicular to the first line, and a third line perpendicular to the second line, wherein the second line connects the first line to the third line and defines a height of the third line above the first line. In a second example of the method (optionally including the first example), determining the first anatomical landmark comprises receiving a first user input to the GUI, and determining the second anatomical landmark comprises receiving a second user input to the GUI. In a third example of the method (optionally including one or both of the first and second examples), the first anatomical landmark is a first mitral valve ring point indicating a first end of the mitral valve ring, and the second anatomical landmark is a second mitral valve ring point indicating a second end of the mitral valve ring. In a fourth example of the method (optionally including one or more or each of the first to third examples), the first line connecting the first anatomical landmark to the second anatomical landmark is close to the position of the mitral valve ring. In a fifth example of the method (optionally including one or more or each of the first to fourth examples), the third line defines a third anatomical landmark. In a sixth example of the method (optionally including one or more or each of the first to fifth examples), the third anatomical landmark is the location of atrial septal puncture. In a seventh example of the method (optionally including one or more or each of the first to sixth examples), the height of the third line above the first line is defined by the length of the second line. In an eighth example of the method (optionally including one or more or each of the first to seventh examples), the method further includes: determining the height of the third line above the first line based on one of one or more user inputs, a preset height, and a detected ridge position. In a ninth example of the method (optionally including one or more or each of the first to eighth examples), the first anatomical landmark and the second anatomical landmark are determined in an automatic manner based on a machine learning algorithm.

[0079] The present disclosure also provides support for a system, the system comprising: a computing device, the computing device being communicatively coupled to an imaging system and a display device, the imaging system being configured to generate image data of a patient, the computing device being configured with instructions in a non-transitory memory, the instructions, when executed, causing the computing device to: display an image acquired by the imaging system in a graphical user interface (GUI) on the display device; generate an annotation in response to the determination of a plurality of landmarks; display the annotation as an overlay on the image; and save the annotation to a memory, wherein the annotation comprises a first line between a first landmark point in the plurality of landmarks and a second landmark point in the plurality of landmarks, a second line perpendicular to the first line, and a third line perpendicular to the second line and defining a third landmark point, wherein the third landmark point is a height above the first line defined by the length of the second line. In a first example of the system, the length of the second line is determined based on one or more user inputs. In a second example of the system (optionally including the first example), the imaging system is an ultrasound imaging system configured to acquire an echocardiographic image. In a third example of the system (optionally including one or both of the first and second examples), the saved annotations are accessible to be overlaid on one or more images of the patient. In a fourth example of the system (optionally including one or more or each of the first to third examples), the first landmark is a first mitral annulus point, the second landmark is a second mitral annulus point, and the third landmark is a transseptal puncture location.

[0080] The present invention also provides support for a method, the method comprising: obtaining one or more images acquired by an imaging system; displaying a first image of the one or more images in a graphical user interface (GUI) on a display device; determining a first mitral valve annulus of the first image; determining a second mitral valve annulus of the first image; generating a transseptal puncture (TSP) annotation based on the first mitral valve annulus and the second mitral valve annulus and TSP height, wherein the TSP annotation comprises a first line perpendicular to a second line and parallel to a third line, the first line extending between the first mitral valve annulus and the second mitral valve annulus and the second line extending between the first line and the third line, wherein the third line extends above the interatrial septum; and displaying the TSP annotation as an overlay on the first image. In a first example of the method, the first mitral valve annulus and the second mitral valve annulus are determined in an automatic manner. In a second example of the method (optionally including the first example), the one or more images acquired by the imaging system are acquired and displayed in real time. In a third example of the method (optionally including one or both of the first and second examples), the TSP height is defined by a detected location of a bulge in the interatrial septum. In a fourth example of the method (optionally including one or more or each of the first and third examples), the third line has a first length that is at least as long as a second length of the first line.

[0081] When introducing the elements of various embodiments of the present disclosure, the articles "one", "a kind of" and "the" are intended to mean that there are one or more such elements. The terms "first", "second", etc. do not represent any order, amount or importance, but are used to distinguish one element from another element. The terms "include", "comprises", and "have" are intended to be inclusive, and mean that additional elements may also exist in addition to the listed elements. As used herein, the terms "connected to", "coupled to", etc., an object (e.g., material, element, structure, member, etc.) may be connected to or coupled to another object, regardless of whether the one object is directly connected or coupled to another object, or whether there are one or more intervening objects between the one object and another object. In addition, it should be understood that reference to "an embodiment" or "embodiment" of the present disclosure is not intended to be interpreted as excluding the existence of additional embodiments that are also combined with the cited features.

[0082] In addition to any previously indicated modifications, those skilled in the art may devise numerous other variations and alternative arrangements without departing from the spirit and scope of the present specification, and the appended claims are intended to cover such modifications and arrangements. Therefore, although the information has been described in detail and in detail as described above in conjunction with what are currently considered to be the most practical and preferred aspects, it will be apparent to those of ordinary skill in the art that many modifications, including but not limited to form, function, mode of operation, and use, may be made without departing from the principles and concepts set forth herein. Likewise, as used herein, the examples and embodiments are intended to be illustrative in all respects only and should not be construed as limiting in any way.

Claims

1. A method, comprising: Obtaining an image dataset of a patient acquired by an imaging system (902); displaying an image of the image data set within a graphical user interface (GUI) on a display device communicatively coupled to the imaging system (904); determining a first anatomical landmark point of the image (906); determining a second anatomical landmark of the image (908); generating an annotation overlay based at least in part on the first anatomical landmark and the second anatomical landmark (910); displaying the annotation overlay over the image in the GUI (1022); and The annotation-covered data is saved to memory (924).

2. The method of claim 1 , wherein the annotation overlay comprises a first line (912) connecting the first anatomical landmark and the second anatomical landmark, a second line perpendicular to the first line, and a third line (914) perpendicular to the second line, wherein the second line connects the first line to the third line and defines a height of the third line above the first line.

3. The method of claim 1, wherein determining the first anatomical landmark comprises receiving a first user input to the GUI (906), and determining the second anatomical landmark comprises receiving a second user input to the GUI (908).

4. The method according to claim 1, wherein the first anatomical landmark point (410) is a first mitral valve ring point indicating a first end of a mitral valve ring, and the second anatomical landmark point (508) is a second mitral valve ring point indicating a second end of the mitral valve ring.

5. The method of claim 4, wherein the first line (510) connecting the first anatomical landmark to the second anatomical landmark is proximal to the location of the mitral valve annulus.

6. The method of claim 1, wherein the third line (514) defines a third anatomical landmark point.

7. The method of claim 6, wherein the third anatomical landmark is the location of atrial septal puncture.

8. The method of claim 1, wherein the height of the third line (514) above the first line (510) is defined by a length of the second line (512).

9. The method of claim 1, further comprising determining the height of the third line above the first line based on one of one or more user inputs (922), a preset height (920), and a detected bump location (1020).

10. The method according to claim 1, wherein the first anatomical landmark and the second anatomical landmark are determined in an automatic manner based on a machine learning algorithm (1000).

11. A system, comprising: A computing device communicatively coupled to an imaging system (100) and a display device (118), the imaging system being configured to generate image data of a patient, the computing device being configured with instructions in a non-transitory memory that, when executed, cause the computing device to: displaying an image acquired by the imaging system in a graphical user interface (GUI) on the display device (904); generating annotations in response to the determination of the plurality of landmark points (910); displaying the annotation as an overlay on the image (1022); and The annotation is saved to a memory (924), wherein the annotation includes a first line (510) between a first marker point (410) of the plurality of marker points and a second marker point (508) of the plurality of marker points, a second line (512) perpendicular to the first line, and a third line (514) perpendicular to the second line and defining a third marker point, wherein the third marker point is a height above the first line defined by a length of the second line.

12. The system of claim 11, wherein the length of the second line (512) is determined based on one or more user inputs (922).

13. The system of claim 11, wherein the imaging system (100) is an ultrasound imaging system configured to acquire echocardiographic images.

14. The system of claim 11, wherein the saved annotations (924) are accessible to be overlaid on one or more images of the patient.

15. The system of claim 11, wherein the first landmark point (410) is a first mitral valve annulus point, the second landmark point (508) is a second mitral valve annulus point, and the third landmark point is a transseptal puncture location.

Images