Ultrasound imaging system and method for providing feedback on acquisition quality

Abstract

A method and ultrasound imaging system includes generating a cine including a plurality of heart views based on cardiac ultrasound data, segmenting a plurality of heart chambers from each of the plurality of heart images, and automatically determining a heart chamber area for each of the plurality of heart chambers. The method and ultrasound imaging system includes displaying the cine on a display device and simultaneously displaying a plurality of single trace curves on the display device with the cine to provide feedback regarding acquisition quality of the cine, where each of the single trace curves represents the heart chamber area for a different one of the plurality of heart chambers over a plurality of cardiac cycles.

Description

TECHNICAL FIELD

[0001] Embodiments of the subject matter disclosed herein relate to methods and systems of ultrasound imaging, and more specifically, to concurrently displaying multiple single traces with cine simultaneously over multiple cycles in order to provide feedback regarding the quality of acquisition of the cine. BACKGROUND

[0002] In ultrasound imaging, it is often desirable to obtain a 4-chamber view of a patient's heart. It is important to acquire and identify multiple cardiac images with the desired view before performing any type of assessment or measurement based on the cardiac images. Identifying multiple cardiac images with a 4-chamber view having a high quality of acquisition can be challenging for some clinicians.

[0003] For example, a 4-chamber view is typically acquired during both adult and fetal ultrasound scanning protocols. A high quality 4-chamber view depicts the left ventricle, right ventricle, left atrium, and right atrium. The 4-chamber view is often used to perform routine measurements on a patient's heart. These measurements can be used to perform a cardiac assessment on that patient and / or help screen for potential abnormalities and defects. For example, the 4-chamber view can be used to detect congenital heart malformations such as a septal defect, hypoplastic left heart syndrome, persistent truncus arteriosus, or the presence of an intra-cardiac echogenic focus.

[0004] A high quality 4-chamber view should clearly show all four chambers of the heart: the left ventricle, right ventricle, left atrium, and right atrium. The septum should be in approximately vertical position in a high quality 4-chamber view. The plane of the 4-chamber view should ideally pass through the apex of the patient's heart. If the plane does not pass through the apex, the resulting view will be foreshortened, which can make it difficult or impossible to make some or all of the desired measurements for a definitive cardiac assessment.

[0005] In many ultrasound procedures, an ultrasound physician views images during a real-time scanning process. The ultrasound physician can adjust the position and / or orientation of the ultrasound probe and see the resulting changes in the ultrasound probe's adjusted position / orientation based on how the real-time images update. It can be difficult for the ultrasound physician to correctly obtain images with a high quality of acquisition having a desired cardiac view in real-time or based on saved ultrasound data. Accordingly, there is a need for an improved method and ultrasound imaging system for providing feedback regarding the quality of acquisition of a cardiac ultrasound image relative to a 4-chamber view. SUMMARY

[0006] In one embodiment, a method of ultrasound imaging includes accessing, with a processor, cardiac ultrasound data, generating a cine based on the cardiac ultrasound data, where the cine includes a plurality of cardiac images acquired over a plurality of cardiac cycles. The method includes automatically segmenting a plurality of heart chambers from each of the plurality of cardiac images, automatically determining a heart chamber area of each of the plurality of heart chambers segmented from each of the plurality of cardiac images, and displaying the cine on a display device. The method includes displaying a plurality of single trace curves on the display device concurrently with the cine to provide feedback on acquisition quality of the cine, where each of the plurality of single trace curves represents the heart chamber area of a different one of the plurality of heart chambers over the plurality of cardiac cycles. The method includes receiving a selection of a portion of the cine based on information displayed in the plurality of single trace curves, and saving the portion of the cine as a 4-chamber view in a memory based on the selection.

[0007] In one embodiment, an ultrasound imaging system includes an ultrasound probe; a user interface; a display device; and a processor in electronic communication with the ultrasound probe, the user interface, and the display device. The processor is configured to control the ultrasound probe to acquire cardiac ultrasound data and generate a cine based on the cardiac ultrasound data, where the cine includes a plurality of cardiac images. The processor is configured to automatically segment a plurality of heart chambers from each of the plurality of cardiac images, and automatically determine a heart chamber area of each of the plurality of heart chambers segmented from each of the plurality of cardiac images. The processor is configured to display the cine on the display device, and display a plurality of single trace curves on the display device concurrently with the cine to provide feedback on acquisition quality of the cine, where each of the plurality of single trace curves represents the heart chamber area of a different one of the plurality of heart chambers over the plurality of cardiac cycles. The processor is configured to receive a selection of a portion of the cine based on information displayed in the plurality of single trace curves, and save the portion of the cine as a 4-chamber view in a memory.

[0008] It is to be understood that the above brief description is provided to introduce in simplified form a selection of concepts that are further described in the detailed description. This brief description does not identify key or essential features of the claimed subject matter nor does it constitute an BRIEF DESCRIPTION OF DRAWINGS

[0009] The present application will be better understood with reference to the following description in conjunction with the accompanying drawings, wherein:

[0010] Figure 1 A schematic diagram of an ultrasound imaging system is shown in accordance with an example embodiment.

[0011] Figure 2 A flowchart of a method according to an embodiment is shown.

[0012] Figure 3 A cardiac image and a plurality of single-trace curves according to an embodiment are shown.

[0013] Figure 4 A flowchart of a method according to an embodiment is shown.

[0014] Figure 5 A cardiac image, a plurality of single-trace curves, and a landmark image according to an embodiment are shown.

[0015] Figure 6 A schematic diagram of a neural network according to an embodiment is shown.

[0016] Figure 7 Input and output connections of a neuron according to an embodiment are shown. DETAILED DESCRIPTION

[0017] Figure 1 A block diagram of an ultrasound imaging system 100 according to an example embodiment is depicted. Further, it should be appreciated that other embodiments do not actively acquire ultrasound data. Rather, embodiments can retrieve images or ultrasound data previously acquired by an ultrasound imaging system and analyze the image data as described herein. As shown, the ultrasound imaging system 100 includes a number of components. These components can be coupled to one another to form a single structure, can be separate but located within a common room, or can be remote relative to one another. For example, one or more of the modules described herein can operate in a data server having a different and remote location relative to other components of the ultrasound imaging system 100, such as an ultrasound probe and a user interface. The ultrasound imaging system 100 can be configured as a workstation; the ultrasound imaging system 100 can be a portable or hand-carried system, such as a laptop computer system, a cart-based system, or a system having a form factor similar to a smart phone; or the ultrasound imaging system 100 can be a vehicle-mounted system on a cart with wheels configured to be easily moved via the cart.

[0018] In the illustrated embodiment, the ultrasound imaging system 100 includes a transmit beamformer 101 and a transmitter 102 that drives an array of elements 104 (e.g., piezoelectric crystals) within an ultrasound probe 106 to transmit ultrasound signals (e.g., continuous or pulsed signals) into a subject's body or volume (not shown). According to other embodiments, the ultrasound probe 106 can be a micro-machined ultrasonic transducer (MUT) or a capacitive micro-machined ultrasonic transducer (CMUT). The elements 104 and the ultrasound probe 106 can have a variety of geometries. The ultrasound signals are backscattered from in-vivo structures to produce echoes that return to the elements 104. The received echoes are received by a receiver 108. The received echoes are provided to a receive beamformer 110 that performs beamforming and outputs radio frequency (RF) signals. The RF signals are then provided to an RF processor 112 that processes the RF signals. Alternatively, the RF processor 112 can include a complex demodulator (not shown) that demodulates the RF signals to form pairs of I / Q data representative of the echo signals. The RF or I / Q signal data can then be provided directly to a memory 114 for storage (e.g., temporary storage). The ultrasound imaging system 100 also includes a processor 116, which can be part of a single processing unit or distributed across multiple processing units. The processor 116 is configured to control the operation of the ultrasound imaging system 100. The processor 116 can include a central processing unit (CPU), one or more microprocessors, a graphics processing unit (GPU), or other electronics capable of processing input data according to specific logical instructions stored on a memory of the processor 116 or coupled to the processor 116. Optionally, the processor 116 can include and / or represent one or more hardware circuits or circuitry, including, connected to, or both including and connected to one or more processors, controllers, and / or other hardware logic-based devices.

[0019] For example, the processor 116 can include an image processing module that receives cardiac ultrasound data (e.g., ultrasound signals in the form of RF signal data or pairs of I / Q data) and processes the image data. For example, the image processing module can process the cardiac ultrasound data to generate 2D cardiac images or ultrasound waveforms (e.g., continuous or pulsed Doppler spectra or waveforms) for display to an operator. Similarly, the image processing module can process the ultrasound signals to generate 3D renderings based on the cardiac ultrasound data. The image processing module can be configured to perform one or more processing operations according to a plurality of selectable ultrasound modalities on the acquired ultrasound information. By way of example only, the ultrasound modalities can include color flow, acoustic radiation force imaging (ARFI), B-mode, A-mode, M-mode, spectral Doppler, acoustic streaming, tissue Doppler module, C-scan, and elastography.

[0020] When echo signals are received, the acquired cardiac ultrasound data can be processed in real-time during an imaging session (or scan session). Additionally or alternatively, the ultrasound data can be temporarily stored in memory 114 during the imaging session and processed in a less than real-time manner in real-time or offline operations. Included is memory 120 for storing processed slices or waveforms of acquired ultrasound information that are not scheduled for immediate display. The image memory 120 can include any known data storage medium, such as a permanent storage medium, a removable storage medium, or the like. Additionally, the image memory 120 can be a non-transitory storage medium.

[0021] In operation, the ultrasound imaging system 100 can acquire cardiac ultrasound data by various techniques, such as 3D scanning, real-time 3D imaging, volume scanning, 2D scanning with a probe having a positioning sensor, freehand scanning using voxel correlation techniques, scanning using a 2D or matrix array probe, or the like. Ultrasound spectra (e.g., waveforms) and / or images can be generated from the acquired cardiac ultrasound data (at the processor 116) and displayed on the display device 118 to an operator or user.

[0022] The processor 116 is operatively connected to a user interface 122 that enables an operator to control at least some operations of the ultrasound imaging system 100. The user interface 122 can include hardware, firmware, software, or a combination thereof that enables a person (e.g., an operator) to directly or indirectly control the operation of the ultrasound imaging system 100 and its various components. As shown, the user interface 122 includes a display device 118 having a display area 117. In some embodiments, the user interface 122 can also include one or more user interface input devices 115, such as a physical keyboard, a mouse, a touchpad, one or more sliders, one or more rotary controls, a trackball, or other control input devices. In one embodiment, a touchpad can be configured to the system processor 116 and display area 117 such that when a user moves a finger / glove / stylus over the surface of the touchpad, a cursor on an ultrasound image or Doppler spectrum on the display device 118 moves in a corresponding manner.

[0023] In example implementations, the display device 118 can be a touch-sensitive display (e.g., a touchscreen) that can detect the presence of a touch by an operator on the display area 117 and can also identify the location of the touch in the display area 117. The touch can be applied by, for example, at least one of an individual’s hand, a glove, a stylus, etc. As such, the touch-sensitive display can also be characterized as an input device configured to receive input from an operator, such as a request to adjust or update the orientation of a displayed image. The display device 118 also conveys information from the processor 116 to the operator by displaying information to the operator. The display device 118 is configured to present information to the operator during or after an imaging or data acquisition session. The presented information can include ultrasound images (e.g., one or more 2D images and / or volumetric renderings), graphical elements, measurement graphics of displayed images, user-selectable elements, user settings, and other information (e.g., administrative information, personal information of a patient, etc.). In other implementations, the display device 118 can be a display that is not touch-sensitive.

[0024] In addition to the image processing module, the processor 116 can also include one or more of a graphics module, an initialization module, a tracking module, and an analysis module. The image processing module, the graphics module, the initialization module, the tracking module, and / or the analysis module can coordinate with one another to present information to the operator during and / or after an imaging session. For example, the image processing module can be configured to display acquired images on the display device 118, and the graphics module can be configured to display designated graphics with the displayed images, such as selectable icons (e.g., image rotation icons) and measurement parameters (e.g., data) related to the images. The processor 116 can contain algorithms and one or more neural networks (e.g., a system of neural networks) stored within a memory of the processor 116 for automatically identifying a plurality of structures from each of a plurality of cardiac images. In some examples, the processor 116 can include a deep learning module including one or more deep neural networks and instructions for performing deep learning and feature recognition discussed herein.

[0025] The screen of the display area 117 of the display device 118 is composed of a series of pixels that display data acquired with the ultrasound probe 106. The acquired data includes one or more imaging parameters computed for each pixel or a group of pixels of the display (e.g., a group of pixels assigned the same parameter value), where the one or more computed image parameters include one or more of intensity, velocity (e.g., blood flow velocity), color blood flow velocity, texture, graininess, contractility, deformation, and deformation rate values. The series of pixels then make up a display image and / or Doppler spectrum generated from the acquired ultrasound data.

[0026] The acquired cardiac ultrasound data can be used to generate one or more cardiac ultrasound images, which can then be displayed via the display device 118 of the user interface input device 115. For example, the one or more generated cardiac ultrasound images can include 2D images and / or volume renderings based on 3D ultrasound data. For example, the image processing module discussed above can be programmed to generate and simultaneously display 2D image slices and 3D renderings.

[0027] Figure 2 is a flowchart of a method according to an example embodiment. The technical effect of the method 200 is to simultaneously display on a display device a movie comprising a plurality of cardiac images and a plurality of single trace curves. Each of the plurality of single trace curves represents an area of a heart chamber over a plurality of cardiac cycles. The simultaneous display of the plurality of single trace curves with the movie provides feedback about the quality of acquisition of the movie.

[0028] Reference is now made to Figure 3 , which shows a cardiac image 302 and a plurality of single trace curves 304 according to an example embodiment.

[0029] Reference is now made to Figure 1 depicted ultrasound imaging system and components describe the method 200, but it should be understood that the method 200 can be implemented with other ultrasound systems and components without departing from the scope of the present disclosure. In some embodiments, the method 200 can be implemented as executable instructions in any appropriate combination of the ultrasound imaging system 100, an edge device (e.g., an external computing device) connected to the ultrasound imaging system 100, a cloud in communication with the ultrasound imaging system, and the like.

[0030] Reference is now made to Figure 2 At step 206, the processor 116 accesses the cardiac ultrasound data. According to an example embodiment, the processor 116 can access one frame of the cardiac ultrasound data at step 206, but according to other embodiments, the processor 116 can access two or more frames of the cardiac ultrasound data at step 206. According to embodiments, the processor 116 can access the cardiac ultrasound data from a memory, such as the memory 120. The cardiac ultrasound data can be stored, for example, from a previous ultrasound examination. According to embodiments, the processor 116 can access the cardiac ultrasound data in real-time from the ultrasound probe 106 or the memory 120 as the cardiac ultrasound data is acquired as part of an ultrasound imaging examination. Alternatively, according to embodiments, the processor 116 can use deep learning, such as by applying a neural network or a deep neural network, to identify one or more possible four-chamber views from the cardiac ultrasound data. For example, a neural network or a deep neural network can be trained to identify frames of the cardiac ultrasound data that likely show a four-chamber view that includes all four chambers of a patient’s heart.

[0031] At step 208, the processor 116 generates an image based on the cardiac ultrasound data accessed during step 206. Generating the image can entail processing the cardiac ultrasound data into a format for display through processes such as scan conversion. Scan conversion involves converting the cardiac ultrasound data from the geometry used during acquisition to a geometry, i.e., a different coordinate system such as a Cartesian coordinate system, that is more well-suited for display on the display device 118.

[0032] Figure 3 is a representation of a cardiac image 302 and a plurality of single-tract curves 304 according to an example embodiment. The cardiac image 302 is a 4-chamber view according to an embodiment. At step 208, the processor can generate an image such as Figure 3 The cardiac image 302 shown. The 4-chamber view, such as the cardiac image 302, ideally shows all four heart chambers. The cardiac image 302 includes a left ventricle 306, a right ventricle 308, a left atrium 310, and a right atrium 312. The cardiac image 302 also includes a cusp 314.

[0033] At step 210 of the method 200, the processor 116 automatically segments a plurality of heart chambers from the cardiac image generated at step 208. According to embodiments in which the desired heart view is a 4-chamber view, the plurality of structures segmented by the processor 116 at step 210 can include the left ventricle 306, the right ventricle 308, the left atrium 310, and the right atrium 312 from the cardiac image 302.

[0034] The processor 116 can also optionally segment one or more additional structures, such as the cusp, the aorta, the interventricular septum (IVS), or another structure that can be used as a landmark. According to one example embodiment, the processor 116 can be configured to segment the plurality of heart chambers (i.e., the left ventricle 306, the right ventricle 308, the left atrium 310, and the right atrium 312) from the cardiac image 302. According to another example embodiment, the processor 116 can be configured to segment the plurality of heart chambers (i.e., the left ventricle 306, the right ventricle 308, the left atrium 310, and the right atrium 312) and one or more other structures (such as the cusp 314) from the image at step 210.

[0035] The processor 116 can be configured to use image processing techniques, such as shape-based recognition, to identify the plurality of heart chambers for segmentation at step 210. For example, the processor 116 can use a model (rigid or deformable model) or template in order to identify the plurality of heart chambers. The processor 116 can be configured to operate in one or both of the image domain or frequency domain by Fourier processing techniques in order to identify the plurality of heart chambers. According to other embodiments, the processor 116 can be configured to apply artificial intelligence techniques in order to segment the plurality of heart chambers from the image at step 210 representing landmarks. For example, the processor 116 can be configured to apply a neural network, such as a deep neural network or system of deep neural networks, in order to identify and segment the plurality of heart chambers from the heart image 302. For example, in an exemplary embodiment, the processor 116 can apply a system of neural networks in order to segment the left ventricle 306, the right ventricle 308, the left atrium 310, and the right atrium 312 from the heart image 302. For example, the neural network or system of neural networks can be trained with a curated set of training images for a 4-chamber view. For example, the curated set of training images can include various images obtained from different patients in which the plurality of heart chambers are identified. The neural network or system of neural networks can then “learn” to recognize the traits or characteristics of the heart chambers.

[0036] At step 212, the processor 116 is configured to determine a chamber area for each of the plurality of heart chambers in the heart image 302. The chamber area is computed / determined by the processor 116 based on the plurality of heart chambers segmented from the heart image 302. According to embodiments, the chamber area can be computed in pixels, as determined based on the heart chambers segmented during step 210. According to other embodiments, the processor 116 can compute the chamber area in area units, such as mm 2 or cm 2 .

[0037] At step 214, the processor 116 displays the heart image generated at step 208 on the display device 118. According to embodiments, the heart image 302 can be displayed as part of a movie.

[0038] At step 216, the processor 116 displays the plurality of single trace curves 304 on the display device 118 concurrently with the heart image 302. Figure 3 The plurality of single trace curves includes a first single trace curve 342, a second single trace curve 344, a third single trace curve 346, and a fourth single trace curve 348. Additional details regarding the plurality of single trace curves will be discussed below.

[0039] At step 220, a selection of a portion of the movie is optionally received. At step 220, the processor 116 can optionally receive a selection indicating a portion of the movie to store. The processor 116 can receive the selection of the portion of the movie based on input entered through the user interface, or the processor 116 can automatically receive the selection of the portion of the movie. For example, the processor 116 can use or apply a neural network or system of neural networks in order to identify the portion of the movie. According to embodiments, the processor 116 can automatically identify the portion of the movie based on other automated techniques. Step 220 will be described in greater detail below.

[0040] If the processor 116 receives a selection of a portion of the movie at step 220, the method proceeds to step 222 of storing the portion of the movie in the memory 120. According to embodiments, the processor 116 can associate a tag that identifies the portion of the movie as representative of a 4-chamber view. After storing the portion of the movie at step 222, the method proceeds to step 230. If a selection of the portion of the movie is not received at step 220, the method proceeds to step 230.

[0041] At step 230, the processor 116 determines whether additional cardiac images are desired to be generated. If additional cardiac images are desired to be generated, the method 400 returns to step 206. Steps 206, 208, 210, 212, 214, 216, 220, and 230 are iteratively repeated in order to display the movie image. It will be appreciated by those skilled in the art that if a selection of the portion of the movie is received at step 220, the method 200 will also perform step 222 during a particular iteration.

[0042] During each iteration of performing steps 206, 208, 210, 212, 214, 216, 220, and 230, the movie is refreshed each time a new cardiac image is generated and displayed based on a different frame of cardiac data. The plurality of single trace curves 304 are updated to include the chamber areas for each chamber based on the current cardiac image of the movie. The processor 116 displays the movie on the display device 118 by displaying a plurality of cardiac images, where each cardiac image is based on cardiac ultrasound data acquired at a different time. Movie images are well known to those skilled in the art and, as such, will not be described in additional detail. For example, Figure 3 The illustrated cardiac image 302 represents the movie at a single point in time. During the course of displaying the movie, the cardiac image 302 will be updated with updated cardiac images each time the method iteratively performs steps 206, 208, 210, 212, 214, 216, 220, and 230. However, the plurality of single trace curves additionally include a historical plot of chamber areas that are calculated or determined based on the plurality of structures segmented from cardiac images acquired over one or more prior cardiac cycles.

[0043] Each single trace curve of the plurality of single trace curves 304 represents a plot of the heart chamber area in one of the heart chambers plotted over time (i.e., the data for each single trace curve accumulated by repeating steps 206, 208, 210, 212, 214, 216, 220, and 230 multiple times to capture the heart chamber area based on different frames of the cardiac ultrasound data). According to Figure 3 According to the illustrated embodiment, each single trace curve of the plurality of single trace curves represents a heart chamber area associated with a different heart chamber of the plurality of heart chambers. The first single trace curve 342 represents the heart chamber area of the left ventricle 306 over a plurality of cardiac cycles, the second single trace curve 344 represents the heart chamber area of the right ventricle 308 over a plurality of cardiac cycles, the third single trace curve 346 represents the heart chamber area of the left atrium 310 over a plurality of cardiac cycles, and the fourth single trace curve 348 represents the heart chamber area of the right atrium 312 over a plurality of cardiac cycles. The plurality of single trace curves 304 are updated to include the heart chamber area as calculated by the processor 116 based on the current cardiac image 302 in the cine, but they also include the heart chamber area calculated based on previous data frames based on previous iterations of performing steps 206, 208, 210, 212, 214, 216, 220, and 230.

[0044] In Figure 3 According to the illustrated embodiment, the plurality of single trace curves 304 represent the heart chamber area of each of the plurality of chambers over a plurality of cardiac cycles. For example, according to Figure 3 According to the illustrated example embodiment, the plurality of single trace curves 304 represent the heart chamber area of each of the plurality of chambers over more than three complete cardiac cycles. As the method 200 iteratively performs steps 206, 208, 210, 212, 214, 216, 220, and 230, the processor 116 determines the heart chamber area of additional cardiac images. According to embodiments, the number of cardiac cycles represented in the plurality of single trace curves can increase as the method 200 iteratively performs steps 206, 208, 210, 212, 214, 216, 220, and 230. According to other embodiments, the plurality of single trace curves can display the heart chamber area over a predetermined number of cardiac cycles. The plurality of single trace curves 304 represent at least two cardiac cycles, but according to embodiments, the plurality of single trace curves 304 can show a different number of cardiac cycles. According to various embodiments, the plurality of single trace curves can be configured to show an integer or non-integer greater than two cardiac cycles. The number of cardiac cycles represented in the plurality of single trace curves can be preset, or can be adjustable by a user via a user interface according to various embodiments.

[0045] Figure 3The plurality of single trace curves 304 shown includes a first portion 350, a second portion 352, and a third portion 354. As previously described, the plurality of single trace curves 304 represents the heart chamber areas of each of the four heart chambers over time. In the first portion 350 and the third portion 354, the plurality of single trace curves does not exhibit a high degree of synchrony. In contrast, in the second portion 352, the plurality of single trace curves exhibits a high degree of synchrony. In the second portion 352, the plurality of single trace curves (each single trace curve representing the heart chamber area of one of the four heart chambers) has a high degree of synchrony. In the second portion 352, the single trace curves have well-defined periods, and the periods of each single trace curve are similar. Additionally, each single trace curve reaches local maxima or minima at approximately similar times. In contrast, the single trace curves in the first portion 350 and the third portion 354 are not strongly synchronized. The periodicity of the plurality of single trace curves in the first portion 350 and the third portion 354 is far less pronounced than the periodicity in the second portion 352.

[0046] When the 4-chamber view is acquired with the ultrasound probe 106 in the correct position, the plurality of single trace curves 304 representing the heart chamber areas over time exhibits a high degree of synchrony. For example, the second portion 352 indicates a high degree of synchrony. The high degree of synchrony in the second portion 352 provides the clinician with qualitative feedback about the quality of the acquisition of the cine. The clinician can quickly and easily identify, by viewing the degree of synchrony exhibited by the plurality of single trace curves 304, one or more portions of the cine acquired with the ultrasound probe 106 in the correct position and orientation to acquire a 4-chamber view with a high quality of acquisition. According to embodiments in which the clinician acquires the cardiac ultrasound data in real-time as the clinician views the cine and the plurality of single trace curves 304, the clinician can use the plurality of single trace curves, for example, to adjust the position and / or orientation of the ultrasound probe 106 to provide feedback about the quality of the acquisition. The clinician can adjust the ultrasound probe 106 to a position and orientation in which the plurality of single trace curves 304 has a high degree of synchrony, such as in the second portion 352. The plurality of single trace curves 304 can be used to provide real-time feedback to help the clinician position and orient the ultrasound probe 106 to acquire a 4-chamber view with a high quality of acquisition.

[0047] The processor 116 can display graphical indicators 360 on the plurality of single trace curves 304 to indicate locations of the movie where the acquisition quality exceeds the acquisition quality threshold. The graphical indicators 360 encompass the second portion 352 of the single trace curves that is the portion where the acquisition quality exceeds the threshold as determined by the synchronization metric in the area of the heart cavity. Other embodiments can use different types of graphical indicators to indicate locations where the acquisition quality exceeds the threshold. According to other embodiments, the graphical indicators can include one or more straight lines, one or more brackets, a color, and / or a highlight to graphically show locations of the movie where the acquisition quality exceeds the acquisition quality threshold. The processor 116 can optionally use a neural network to determine where to position the graphical indicators 360 on the plurality of single trace curves 304. The processor 116 can adjust the positioning of the graphical indicators 360 in real-time as additional cardiac ultrasound data is represented by the plurality of single trace curves 304.

[0048] According to another embodiment, the clinician can review cardiac ultrasound data acquired during a previous examination or scan session. The clinician can use the plurality of single trace curves 304 to quickly and easily identify a portion of the movie that has high acquisition quality. For example, the clinician can use the plurality of single trace curves 304 as a guide to identify a portion of the movie that has high acquisition quality.

[0049] Referring to step 220 of the method 200, the clinician can use input from the user interface to select a portion of the movie to store as a 4-chamber view. For example, the clinician can use the user interface to select, at step 220, the portion of the movie saved at step 222. The clinician can select the portion of the movie by interacting with a graphical user interface element associated with the plurality of single trace curves 304. Figure 3 A graphical user interface element is shown according to an example embodiment: Figure 3This includes a first edge marker 362 and a second edge marker 364. The first edge marker 362 and the second edge marker 364 are graphical user interface elements that can be controlled by a clinician to select a portion of a movie via a user interface. The clinician can store the selection of a movie portion based on input entered through the user interface. According to an exemplary embodiment, the clinician can be able to control the position of the first edge marker 362 and the second edge marker 364 to select a portion of the movie corresponding to a portion of a plurality of single-track curves between the first edge marker 362 and the second edge marker 364. For example, a user can position the first edge marker 362 at the left edge of the second portion 352 and the second edge marker 364 at the right edge of the second portion 352 to select a portion of the movie corresponding to the second portion 352 of a plurality of single-track curves 304. User input for selecting this portion of the movie may include input from a trackball, input from a mouse, or input from a touchpad or touch-sensitive display according to various embodiments.

[0050] According to the implementation scheme, multiple single-track curves can also be used as part of the user interface to select and view specific cardiac images from the film. For example, a user can locate an indicator (not shown) or select a position along the multiple single-track curves 304 to display a cardiac image from the film corresponding to the position selected on the multiple single-track curves 304. For example, a user can use a cursor to select a position along the multiple single-track curves 304, and the processor 116 can display the corresponding cardiac image acquired at the time indicated by the position selected along the multiple single-track curves 304.

[0051] Method 400 in Figure 4 It is shown in the text, and will be about Figure 1 and Figure 5 The following description is provided. Steps 206, 208, 210, 212, 214, 216, 220, 222, and 230 are identical to those previously described regarding method 200. Steps 206, 208, 210, 212, 214, 216, 220, 222, and 230 will not be described again regarding method 400. Compared to method 200, method 400 additionally includes a new step 218. At step 218, method 400 displays a landmark image. The following will be described regarding... Figure 5 Description method 400.

[0052] Figure 5 A cardiac image 302, multiple single-track curves 304, and a landmark image 370 according to an embodiment are shown. (Previous information regarding...) Figure 3 Method 200 describes cardiac images 302 and multiple single-trajectory curves 304, and will no longer describe them regarding... Figure 5 Describe it.

[0053] Landmark image 370 is a schematic view of the plurality of heart chambers segmented from heart image 302 displayed at step 214. Figure 5 Landmark image 370 shown in FIG. 3B is an example of a landmark image according to an embodiment. Landmark image 370 includes a schematic view of right ventricle 326, a schematic view of left ventricle 328, a schematic view of right atrium 330, and a schematic view of left atrium 332. Processor 116 can be configured to represent each of the plurality of structures in landmark image 370 in a different color. For example, processor 116 can represent the schematic view of right ventricle 326 in a first color, the schematic view of left ventricle 328 in a second color, the schematic view of right atrium 330 in a third color, and the schematic view of left atrium 332 in a fourth color. Processor 116 is configured to display landmark image 370 on display device 118 concurrently with heart image 302 and plurality of single trace curves 304. Landmark image 370 can include schematic views of other structures according to various embodiments. For example, Figure 5 Landmark image 370 shown in FIG. 3B includes a schematic view of cusps 333.

[0054] According to an embodiment, processor 116 can obtain a mask of the plurality of structures segmented at step 210. For example, processor 116 can generate a separate mask for each of the plurality of structures, or processor 116 can generate one mask for all of the plurality of structures. For example, the mask can define the locations of left ventricle 306, right ventricle 308, left atrium 310, and right atrium 312. According to various embodiments, the mask generated by processor 116 can be a binary mask. Displaying a landmark image at step 216 can include displaying a representation of the mask on display device 118 concurrently with heart image 302. According to other embodiments, processor 116 can be configured to generate different landmark images. For example, processor 116 can create a landmark image showing an outline or contour to schematically represent each of the plurality of structures segmented from the image. Landmark image 370 schematically represents the plurality of structures at the same relative locations and orientations in the image displayed at step 214. A relatively experienced clinician can be required to quickly identify each of the plurality of structures in heart image 302. Showing a schematic view of the plurality of structures, such as in landmark image 370, allows the clinician to quickly and easily determine whether the current heart image 302 includes the plurality of structures in the correct orientation of the desired heart view, especially when viewed in conjunction with plurality of single trace curves 304.

[0055] If at step 230 it is desired to generate additional images, the method 200 returns to step 206, where the processor 116 repeats steps 206, 208, 210, 212, 214, 216, 218, 220, and 230 with a different frame of cardiac ultrasound data. The different frame of cardiac data can be newly acquired as part of real-time acquisition, or can be accessed from memory 120. Each iteration through steps 206, 208, 210, 212, 214, 216, 218, 220, and 230 results in display of an updated cardiac image at step 214 (based on the updated frame of cardiac ultrasound data), display of an updated landmark image based on segmentation of the plurality of structures from the updated cardiac image, and display of an updated plurality of single-tract curves. The processor 116 can be configured to repeat steps 206, 208, 210, 212, 214, 216, 218, 220, and 230 of the method 400 in real-time as cardiac ultrasound data is acquired with the ultrasound probe 106. For example, if an updated frame of cardiac ultrasound data has been acquired with the ultrasound probe 106, the processor can return to step 206 at step 230. According to other embodiments, the processor 116 can iteratively loop through steps 206, 208, 210, 212, 214, 216, 218, 220, and 230 of the method 400 using cardiac ultrasound data acquired and stored during a prior scan session. For example, the processor 116 can access previously acquired cardiac ultrasound data from memory 120 of the ultrasound imaging system or from a remote memory device. According to embodiments, the processor 116 can access previously acquired cardiac ultrasound data via a server and a remote memory or data storage device. The landmark image 370 is synchronized with the cardiac image 302. This means that the illustration of the plurality of structures in the landmark image 370 is based on the cardiac image 302 that is currently displayed on the display device. The landmark image 370 is updated at the same refresh rate as the cardiac image 302 that is displayed on the display device 118 as part of a movie. In real-time acquisition scenarios in which the cardiac image 302 is generated in real-time as additional frames of cardiac ultrasound data are acquired, the landmark image 370 provides an illustration of the plurality of structures contained in the most recently acquired cardiac image 302 (i.e., the cardiac image 302 that is currently displayed on the display device 118). According to embodiments in which a clinician acquires real-time ultrasound data, the clinician can make adjustments to the position and / or orientation of the ultrasound probe 106 during the process in which steps 206, 208, 210, 212, 214, 216, 218, 220, and 230 of the method 200 are iteratively repeated. Each time the method repeats steps 206, 208, 210, 212, 214, 216, 218, 220, and 230, the cardiac image 302 displayed on the display device 118 and the landmark image 370 displayed on the display device 118 will represent data acquired from the most recent position and orientation of the ultrasound probe 106.An advantage of landmark image 370 is that it helps the clinician determine whether a desired heart view has been acquired. For example, a 4-chamber view should include all four heart chambers, the left ventricle should show the apex of the heart, and the 4-chamber view should include the tricuspid and mitral valves. Displaying landmark images, such as landmark image 370, in synchronization with the cine provides the clinician with easily interpretable real-time feedback about the quality of acquisition of the currently displayed heart image relative to a desired heart view. If the clinician is acquiring data from a plane that is never in the correct position for the desired view, the clinician can easily see that the multiple heart chambers (indicative of the landmarks) shown in the landmark image are not in the correct relative positions for the desired 4-chamber view. According to some embodiments, processor 116 can also display a target landmark image that would show the relative orientation of the multiple structures according to the example of a landmark image with high quality of acquisition. The user can then compare the landmark image to the target landmark image. This can help a less experienced clinician identify the desired 4-chamber view if the less experienced clinician is not as familiar with the expected relative orientation of the multiple structures represented in the 4-chamber view.

[0056] The following description refers to the ultrasonic imaging system 100 and components depicted in Figure 1 The method 400 is described in the context of the ultrasonic imaging system 100 and components depicted in Figure 4 but it should be understood that the method 400 can be implemented with other ultrasonic systems and components without departing from the scope of the present disclosure. In some embodiments, the method 400 can be implemented as executable instructions in any appropriate combination of the ultrasonic imaging system 100, an edge device (e.g., an external computing device) connected to the ultrasonic imaging system 100, a cloud in communication with the imaging system, etc. As one example, the method 400 can be implemented in the non-transitory memory of a computing device, such as Figure 1 the processor 116 of the ultrasonic imaging system 100 in FIG. 1. The method 400 includes many steps that are the same as or substantially the same as those described with respect to the method 200. Steps that are substantially the same as previously described with respect to the method 200 will not be described in detail with respect to the method 400.

[0057] The methods 200 and 400 advantageously provide users with qualitative feedback about the quality of acquisition of multiple heart images in a cine. The user can use the qualitative information in the landmark images and / or the plurality of single trace curves to quickly and easily identify a portion of previously acquired heart data that represents a 4-chamber view. The user can also use the method 200 or 400 when the cine represents real-time heart ultrasound data being acquired in real-time. For example, the user can adjust the position and / or orientation of the ultrasound probe based on feedback from the landmark images and / or the plurality of single trace curves until the current heart image in the cine represents a desired heart view.

[0058] According to various embodiments, the user can use the landmark image 370 in order to identify when the plurality of structures in the landmark image are arranged in relative positions associated with a desired view. For example, the landmark image 370 can include only the plurality of structures segmented by the processor 116. Thus, the landmark image helps the user to more quickly determine, based on the plurality of structures representing landmarks of a desired heart view, the portion of the cardiac ultrasound data acquired with the ultrasound probe in the proper position and orientation.

[0059] According to various embodiments, the user can use the plurality of single trace curves 304 in addition to or instead of the landmark image 370 in order to quickly and easily determine qualitative information about the quality of acquisition for a desired 4-chamber view. For example, when the desired heart view is a 4-chamber view and the areas of the heart chambers represented in the plurality of single trace curves are expected to vary with a generally synchronized periodicity. In contrast, when the cardiac images are not acquired from the correct plane, the plurality of single trace curves will not show the same synchronized periodicity. For example, the user can adjust the position and orientation of the ultrasound probe 106 in real-time until the plurality of single trace curves based on the cardiac images displayed in the movie exhibit a synchronized periodicity. The user is looking for an ultrasound probe position and orientation in which the motion of the four heart chambers is rhythmic and stable, as determined by one or both of the landmark image 370 and the plurality of single trace curves 304. According to some embodiments, the processor 116 can be configured to automatically save the best one or more cycles of cardiac ultrasound data in the movie, as determined based on the plurality of single trace curves and / or a metric for the plurality of structures segmented for the landmark image.

[0060] According to embodiments, the plurality of single trace curves can also be used as part of a user interface to select and view particular cardiac images from the movie and corresponding landmark views. For example, the user can position a pointer (not shown) or select a position along the plurality of single trace curves 304 to display a cardiac image from the movie and display a corresponding landmark image corresponding to the position selected on the plurality of single trace curves 304. According to embodiments, the landmark image can always correspond to the currently displayed cardiac image. For example, the user can select a position along the plurality of single trace curves 304 with a cursor and the processor 116 can display the corresponding cardiac image and the corresponding landmark image acquired from the time represented at the position selected along the plurality of single trace curves 304.

[0061] Reference is now made to Figure 6 and Figure 7 showing an example neural network for identifying and segmenting a plurality of structures from a cardiac image, according to an example embodiment. In some examples, the neural network can be trained with a training set of cardiac images.

[0062] Figure 6A schematic diagram of a neural network 500 having one or more nodes / neurons 502, which in some embodiments can be disposed in one or more layers 504, 506, 508, 510, 512, 514, and 516, is depicted. The neural network 500 can be a deep neural network. As used herein with respect to neurons, the term“layer” refers to a collection of analog neurons having inputs and / or outputs that are connected in a similar manner to other collections of analog neurons. Thus, as shown, neurons 502 can be connected to one another via one or more connections 518 such that data can propagate from an input layer 504 through one or more intermediate layers 506, 508, 510, 512, and 514 to an output layer 516. Figure 6

[0063] Figure 7 Input and output connections of a neuron are shown in accordance with example embodiments. As shown, Figure 7 connections (e.g., 518) of a single neuron 502 can include one or more input connections 602 and one or more output connections 604. Each input connection 602 of a neuron 502 can be an output connection of a preceding neuron, and each output connection 604 of a neuron 502 can be an input connection of one or more subsequent neurons. Although Figure 7 neurons 502 are depicted as having a single output connection 604, it should be understood that a neuron can have multiple output connections that send / transmit / communicate the same value. In some embodiments, a neuron 502 can be a data construct (e.g., a structure, an instantiated class object, a matrix, etc.), and input connections can be received by the neuron 502 as weighted numerical values (e.g., floating point or integer values). For example, as further shown, Figure 7 input connections X1, X2, and X3 can be weighted, summed, and sent / transmitted / communicated as an output connection Y via weights W1, W2, and W3, respectively. As will be appreciated, the processing of a single neuron 502 can generally be represented by the following equation:

[0064]

[0065] where n is the total number of input connections 602 to the neuron 502. In one embodiment, the value of Y can be based at least in part on whether the sum of the weighted inputs X i X i exceeds a threshold value. For example, if the sum of the weighted inputs does not exceed a desired threshold value, Y can have a value of zero (0).

[0066] From Figure 6 and Figure 7 ​It will be further appreciated that input connections 602 of neurons 502 in input layer 504 can be mapped to inputs 501, while output connections 604 of neurons 502 in output layer 516 can be mapped to outputs 530. As used herein, to "map" a given input connection 602 to an input 501 refers to the manner in which input 501 influences / indicates the value of said input connection 602. Similarly, as used herein, to "map" a given output connection 604 to an output 530 refers to the manner in which the value of said output connection 604 influences / indicates output 530.

[0067] Accordingly, in some embodiments, an acquired / obtained input 501 is passed / fed to input layer 504 of neural network 500 and propagates through layers 504, 506, 508, 510, 512, 514, and 516, such that mapped output connections 604 of output layer 516 generate / correspond to outputs 530. As shown, input 501 can comprise a cardiac image. The cardiac image can depict one or more structures that can be identified by neural network 500. Further, outputs 530 can comprise locations and contours of the one or more structures identified by neural network 500.

[0068] Neural network 500 can be trained using a plurality of training data sets. Each training data set can comprise, for example, annotated cardiac images. Based on the training data sets, neural network 500 can learn to identify a plurality of structures from cardiac images. The machine learning or deep learning therein (e.g., due to identifiable trends in the arrangement, size, etc. of anatomical features) can cause weights (e.g., Wi, W2, and / or W3) to vary, input / output connections to vary, or other adjustments to neural network 500. Further, as additional training data sets are employed, machine learning can continue to adjust various parameters of neural network 500 in response. As such, the sensitivity of neural network 500 can be periodically increased, resulting in higher accuracy of anatomical feature identification.

[0069] As used herein, an element or step recited in the singular and preceded with the word "a" or "an" should be understood as not excluding plural of said elements or steps, unless explicitly stated that such exclusion applies. Also, as used herein, "one" or "another" version of an item should be understood as one, some, or each of one or more items, unless explicitly stated that such reference to one of only one of the items has a different connotation. Furthermore, an embodiment of the present application can include a combination of one or more aspects of the application. Also, some embodiments can be implemented by one or more computer programs or software, which run on one or more computers or software, which can each be in the form of a program storage device. Additionally, the terms "first," "second," and "third," etc. are used herein not to denote a particular location or order, but to denote one of a number of such steps or elements, and do not imply a particular order or sequence except where explicitly so indicated.

[0070] This written description uses examples to disclose the application, including the best mode, and also to enable any person skilled in the art to practice the application, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the application is defined by the claims, and can include other examples that occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims.

Claims

1. An ultrasound imaging method, the method comprising: accessing, with a processor, cardiac ultrasound data; generating a cine based on the cardiac ultrasound data, wherein the cine comprises a plurality of cardiac images acquired over a plurality of cardiac cycles; automatically segmenting a plurality of heart chambers from each of the plurality of cardiac images; automatically determining a heart chamber area for each of the plurality of heart chambers segmented from each of the plurality of cardiac images; displaying the cine on a display device; displaying a plurality of single trace curves on the display device contemporaneously with the cine to provide feedback regarding an acquisition quality of the cine, wherein each of the plurality of single trace curves represents a heart chamber area of a different one of the plurality of heart chambers over the plurality of cardiac cycles; receiving a selection of a portion of the cine based on information displayed in the plurality of single trace curves; and saving the portion of the cine as a 4-chamber view in a memory based on the selection.

2. The method of claim 1, wherein the portion of the cine represents at least one complete cardiac cycle and comprises a second plurality of cardiac images, wherein the second plurality of cardiac images is a subset of the plurality of cardiac images.

3. The method of claim 2, wherein each of the second plurality of cardiac images exceeds an acquisition quality threshold determined based on information displayed in the plurality of single trace curves.

4. The method of claim 1, wherein the feedback regarding the acquisition quality of the cine comprises an amount of synchronization between the plurality of single trace curves.

5. The method of claim 1, wherein receiving the selection of the portion of the cine comprises receiving an input entered through a user interface.

6. The method of claim 5, further comprising displaying a graphical indicator on the plurality of single trace curves to indicate a location of the portion of the cine where the acquisition quality exceeds an acquisition quality threshold.

7. The method of claim 1, wherein receiving the selection comprises automatically receiving the selection from a processor based on an analysis of information displayed in the plurality of single trace curves to identify the portion of the cine where the acquisition quality exceeds an acquisition quality threshold.

8. The method of claim 7, wherein the automatically receiving the selection from the processor comprises using a neural network to identify the portion of the cine.

9. The method of claim 7, wherein the analysis of information displayed in the plurality of single trace curves comprises determining an amount of synchronization in the information displayed as the plurality of single trace curves.

10. The method of claim 1, further comprising displaying a landmark image on the display device contemporaneously with the cine and the plurality of single trace curves, wherein the landmark image comprises a schematic of the plurality of heart chambers currently displayed in the cine.

11. The method of claim 10, wherein each of a plurality of structures is represented in a different color in the landmark image.

12. An ultrasound imaging system, comprising: an ultrasound probe; a user interface; a display device; and a processor configured to: access cardiac ultrasound data; generate a cine based on the cardiac ultrasound data, wherein the cine comprises a plurality of cardiac images acquired over a plurality of cardiac cycles; automatically segment a plurality of heart chambers from each of the plurality of cardiac images; automatically determine a heart chamber area for each of the plurality of heart chambers segmented from each of the plurality of cardiac images; display the cine on a display device; display a plurality of single trace curves on the display device contemporaneously with the cine to provide feedback regarding an acquisition quality of the cine, wherein each of the plurality of single trace curves represents a heart chamber area of a different one of the plurality of heart chambers over the plurality of cardiac cycles; receive a selection of a portion of the cine based on information displayed in the plurality of single trace curves; and save the portion of the cine as a 4-chamber view in a memory based on the selection. a processor in electronic communication with the ultrasound probe, the user interface, and the display device, wherein the processor is configured to: control the ultrasound probe to acquire cardiac ultrasound data; generate a cine based on the cardiac ultrasound data, wherein the cine comprises a plurality of cardiac images; automatically segment a plurality of heart chambers from each cardiac image of the plurality of cardiac images; automatically determine a heart chamber area for each heart chamber of the plurality of heart chambers segmented from each cardiac image of the plurality of cardiac images; display the cine on the display device; and display a plurality of single trace curves on the display device concurrently with the cine, to provide feedback regarding acquisition quality of the cine, wherein each single trace curve of the plurality of single trace curves represents a heart chamber area of a different heart chamber of the plurality of heart chambers over a plurality of cardiac cycles; receive a selection of a portion of the cine based on information displayed in the plurality of single trace curves; and save the portion of the cine as a 4-chamber view in a memory.

13. The system of claim 12, wherein the portion of the cine represents at least one complete cardiac cycle and comprises a second plurality of cardiac images, wherein the second plurality of cardiac images is a subset of the plurality of cardiac images.

14. The system of claim 13, wherein each of the second plurality of cardiac images exceeds an acquisition quality threshold determined based on information displayed in the plurality of single trace curves.

15. The system of claim 12, wherein the feedback regarding acquisition quality comprises a measure of synchronization.

16. The system of claim 12, wherein the processor is configured to receive the selection of the portion of the cine based on input entered through the user interface.

17. The system of claim 12, wherein the processor is further configured to display a graphical indicator on the plurality of single trace curves to indicate a location of the cine where the acquisition quality exceeds an acquisition quality threshold.

18. The system of claim 12, wherein the processor is configured to identify the portion of the cine based on a measure of synchronization in the information displayed as the plurality of single trace curves.

19. The system of claim 18, wherein the processor is configured to receive the selection of the portion of the cine to identify the portion of the cine based on output from a neural network.

20. The system of claim 12, wherein the processor is further configured to display a landmark image on the display device concurrently with the cardiac images and the plurality of single trace curves.

Images