Ultrasonic Imaging System and Method for Low-Resolution Background Volume Acquisition

Abstract

The present invention is entitled "Ultrasound Imaging System and Method for Low-Resolution Background Volume Acquisition". An ultrasound imaging system (100) and method include acquiring low-resolution volume data using an ultrasound probe (106) during a process of acquiring high-resolution slice data of a plane using the same probe. The ultrasound imaging system (100) and method include displaying an ultrasound image on a display device (118) based on the high-resolution slice data; generating a low-resolution volume based on the low-resolution volume data; and implementing a neural network (500) using a processor (116) to calculate guidance information based on the low-resolution volume. The ultrasound imaging system (100) and method include automatically performing actions using the processor (116) based on the guidance information.

Description

Technical Field

[0001] This disclosure generally relates to a method and an ultrasound imaging system for acquiring low-resolution volumetric data using an ultrasound probe during a process of acquiring high-resolution slice data. The method and system further involve calculating guidance information based on the low-resolution volumetric data and automatically performing actions based on that guidance information. Background Technology

[0002] Diagnostic ultrasound imaging is an imaging modality that displays ultrasound images based on signals detected in response to transmitted ultrasound signals. Each time additional frames of slide data are acquired, the ultrasound image generated based on the slide data is typically refreshed. This provides clinicians with real-time images of the patient's anatomy. For various applications, such as echocardiography, one or more standard views are required, such as a parasternal long-axis view, a parasternal short-axis view, an apical four-chamber view, an apical two-chamber view, an apical three-chamber view, an apical five-chamber view, a subcostal four-chamber view, etc. To make an accurate clinical diagnosis and / or obtain accurate measurements of the patient's anatomy from the standard views, the desired standard views must be correctly obtained. If a clinician accidentally acquires slide data from an incorrect plane, any measurements or quantitative data obtained from the image may be inaccurate. Similarly, if a clinician accidentally acquires slide data from an incorrect plane, an accurate clinical diagnosis may be difficult or impossible because the image does not represent the standard view.

[0003] For example, in echocardiography, the slice plane of each apical view should pass through the apex of the heart. However, clinicians often struggle to accurately position the slice plane to include the apex. Attempts to acquire an apical view that does not include the apex are called shortening. Clinicians, especially relatively inexperienced ones, may have difficulty recognizing that a cardiac image has been shortened from a single image.

[0004] Therefore, for these and other reasons, there is a desire for improved ultrasound imaging systems and methods. Summary of the Invention

[0005] This article addresses the aforementioned defects, shortcomings, and problems, which will be understood by reading and comprehending the following instructions.

[0006] In one embodiment, an ultrasound imaging method includes acquiring low-resolution volumetric data using the ultrasound probe during a process of acquiring high-resolution slice data of a plane using the ultrasound probe. The method includes displaying an ultrasound image on a display device based on the high-resolution slice data; generating a low-resolution volume based on the low-resolution volumetric data; and implementing a neural network using a processor to calculate guidance information based on the low-resolution volume. The method includes automatically performing at least one of the following actions using the processor based on the guidance information: displaying guidance suggestions for adjusting the ultrasound probe to obtain a desired view; adjusting the orientation parameters of the ultrasound probe to acquire a desired view using the ultrasound probe in its current position and orientation; providing a warning that the ultrasound probe has been incorrectly positioned; or automatically placing a region of interest relative to the ultrasound image.

[0007] In another embodiment, an ultrasound imaging system includes an ultrasound probe, a user interface, a display device, and a processor that electronically communicates with the ultrasound probe, the user interface, and the display device. The processor is configured to control the ultrasound probe to acquire low-resolution volumetric data during a process of acquiring high-resolution slice data of an acquisition plane, display an ultrasound image on the display device based on the high-resolution slice data, and generate a low-resolution volume based on the low-resolution volumetric data. The processor is configured to implement a neural network to compute guidance information based on the low-resolution volume. The processor is configured to automatically perform at least one of the following actions based on the guidance information: displaying guidance suggestions for adjusting the ultrasound probe to obtain a desired view; adjusting the orientation parameters of the ultrasound probe to acquire a desired view using the ultrasound probe in its current position and orientation; providing a warning that the ultrasound probe has been incorrectly positioned; or placing a region of interest relative to the ultrasound image.

[0008] Various other features, objects, and advantages of the present invention will be apparent to those skilled in the art from the accompanying drawings and their detailed embodiments. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of an ultrasound imaging system according to one implementation scheme;

[0010] Figure 2 It is a flowchart of a method based on an implementation plan;

[0011] Figure 3 It is a representation of an ultrasonic probe and a planar region according to an implementation scheme;

[0012] Figure 4 This is a representation of an ultrasonic probe and a thin plate according to an implementation scheme;

[0013] Figure 5 It is a representation of the ultrasonic probe and its volume according to an implementation scheme;

[0014] Figure 6 This is a schematic diagram of a neural network based on one implementation scheme;

[0015] Figure 7 This is a schematic diagram illustrating the input and output connections of neurons in a neural network according to an exemplary embodiment;

[0016] Figure 8 This is a screenshot based on an exemplary implementation;

[0017] Figure 9 This is a screenshot based on an exemplary implementation;

[0018] Figure 10 This is a screenshot based on an exemplary implementation;

[0019] Figure 11 It is a screenshot based on an implementation plan; and

[0020] Figure 12 It is a screenshot based on an implementation plan. Detailed Implementation

[0021] In the following detailed description, reference is made to the accompanying drawings, which form a part thereof, and specific embodiments that can be practiced are illustrated therein. These embodiments have been described in sufficient detail to enable those skilled in the art to practice them, and it should be understood that other embodiments may be utilized, and logical, mechanical, electrical, and other changes may be made without departing from the scope of the embodiments. Therefore, the following detailed description should not be considered as limiting the scope of the invention.

[0022] Figure 1 This is a schematic diagram of an ultrasound imaging system 100 according to one embodiment. The ultrasound imaging system 100 includes a transmit beamformer 101 and a transmitter 102 that drive elements 104 within an ultrasound probe 106 to transmit pulsed ultrasound signals to a patient (not shown). The probe 106 is capable of volumetric acquisition. According to one embodiment, the ultrasound probe 106 may be a 2D matrix array probe capable of full beamforming in both the azimuth and elevation directions. According to other embodiments, the ultrasound probe 106 may be a mechanical 3D / 4D probe comprising an array of elements configured to move relative to the remainder of the ultrasound probe 106. See still Figure 1Pulsed ultrasound signals are backscattered from internal structures such as blood cells or muscle tissue to generate echoes that return to element 104. The echoes are converted into electrical signals or ultrasound data by element 104, and the electrical signals are received by receiver 108. The electrical signals representing the received echoes pass through receiver beamformer 110, which outputs ultrasound data. According to some embodiments, ultrasound probe 106 may include electronic circuitry to perform all or part of transmit beamforming and / or receive beamforming. For example, all or part of transmit beamformer 101, transmitter 102, receiver 108, and receive beamformer 110 may be located within probe 106. In this disclosure, the terms “scanning” or “under scanning” may also be used to refer to the acquisition of data through the process of transmitting and receiving ultrasound signals. In this disclosure, the terms “data” and “ultrasound data” may be used to refer to one or more datasets acquired using an ultrasound imaging system. User interface 115 can be used to control the operation of ultrasound imaging system 100. User interface 115 can be used to control the input of patient data or to select various modes, operations, and parameters, etc. User interface 115 may include one or more user input devices, such as a keyboard, hard keys, touchpad, touch screen, trackball, rotary control, slider, soft keys, or any other user input device. The user interface communicates electronically with processor 116.

[0023] The ultrasound imaging system 100 also includes a processor 116 for controlling the transmit beamformer 101, transmitter 102, receiver 108, and receive beamformer 110. As will be described in more detail below, some or all of the processors in processor 116 may be implemented as a neural network to perform one or more tasks. The receive beamformer 110 may be a conventional hardware beamformer or a software beamformer according to various embodiments. If the receive beamformer 110 is a software beamformer, it may include one or more of the following components: a graphics processing unit (GPU), a microprocessor, a central processing unit (CPU), a digital signal processor (DSP), or any other type of processor capable of performing logical operations. The beamformer 110 may be configured to perform conventional beamforming techniques as well as software beamforming techniques such as, for example, backhaul transmission beamforming (RTB).

[0024] Processor 116 may be one or more central processing units (CPUs), microprocessors, microcontrollers, graphics processing units (GPUs), digital signal processors (DSPs), etc. According to some embodiments, the processor may include one or more GPUs, some or all of which include tensor processing units (TPUs). According to embodiments, processor 116 may include a field-programmable gate array (FPGA) or any other type of hardware capable of performing processing functions. Processor 116 may be an integrated component or may be distributed across various locations. For example, according to one embodiment, processing functions may be partitioned among two or more processors based on the type of operation. For example, embodiments may include a first processor configured to perform a first set of operations and a second separate processor for performing a second set of operations. According to embodiments, one of the first and second processors may be configured to implement a neural network. Processor 116 may be configured to execute instructions accessing memory. According to one embodiment, processor 116 electronically communicates with ultrasound probe 106, receiver 108, receive beamformer 110, transmit beamformer 101, and transmitter 102. Processor 116 may control ultrasound probe 106 to acquire ultrasound data. Processor 116 controls which elements in element 104 are active and the shape of the beam emitted from ultrasound probe 106. Processor 116 also communicates electronically with display device 118 and can process ultrasound data into images for display on display device 118. For the purposes of this disclosure, the term "electronic communication" may be defined to include both wired and wireless connections. According to other embodiments, processor 116 may include multiple electronic components capable of performing various processing functions. According to embodiments, processor 116 may also include a composite demodulator (not shown) that demodulates RF data and generates raw data. In another embodiment, demodulation may be performed earlier in the processing chain. Processor 116 may be adapted to perform one or more processing operations based on multiple selectable ultrasound modes on the data. Data can be processed in real time during a scanning session as echo signals are received. For the purposes of this disclosure, the term "real time" is defined to include a procedure executed without any intentional delay. The real-time frame rate may vary based on the size of the area or volume from which data is acquired and the specific parameters used during acquisition. Data may be temporarily stored in a buffer (not shown) during a scanning session and processed in a less real-time manner during real-time or offline operation. According to various embodiments, for embodiments in which the receive beamformer 110 is a software beamformer, the processing functions of the processor 116 and the software beamformer described above may be performed by a single processor such as the receive beamformer 110, the processor 116, a portion of the processor 116, or a separate processor.Alternatively, the processing capabilities attributable to processor 116 and beamformer 110 can be distributed in different ways among any number of individual processing components.

[0025] Some embodiments provide a non-transitory computer-readable medium on which a computer program is stored, the computer program having at least one code segment. The at least one code segment is executable by a processor 116 for causing the ultrasound imaging system 100 to perform the steps of any of the methods and / or functions described in this application.

[0026] According to one embodiment, the ultrasound imaging system 100 can continuously acquire ultrasound data at frame rates, for example, from 10 Hz to 30 Hz. Images generated from the data can be refreshed at similar frame rates. Other embodiments can acquire ultrasound data and display ultrasound images at different frame rates. For example, some embodiments may acquire ultrasound data at frame rates less than 10 Hz or greater than 30 Hz, depending on the size and intended application. For example, many applications involve acquiring ultrasound data at frame rates of 50 Hz or higher. A memory 120 is included for storing frames of processed acquired data. In one exemplary embodiment, the memory 120 has sufficient capacity to store frames of ultrasound data acquired over a time period of at least several seconds. The data frames are stored in a manner that facilitates retrieval based on their acquisition order or time. The memory 120 can be random access memory (RAM) or other dynamic storage devices. The memory can be a hard disk drive, a solid-state drive, flash memory, or any other known data storage medium.

[0027] Optionally, contrast agents can be used to implement embodiments of the present invention. When an ultrasound contrast agent, including microbubbles, is used, contrast imaging generates enhanced images of in vivo anatomical structures and blood flow. After data acquisition using a contrast agent, image analysis includes separating harmonic and linear components, enhancing harmonic components, and generating ultrasound images by utilizing the enhanced harmonic components. A suitable filter is used to perform the separation of harmonic components from the received signal. Ultrasound imaging using contrast agents is well known to those skilled in the art and will therefore not be described in detail.

[0028] In various embodiments of the invention, processor 116 may process data through other or different mode-related modules (e.g., dual-plane mode, tri-plane mode, B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastography, TVI, strain, strain rate, etc.) to form 2D or 3D data. For example, one or more modules may generate dual-plane mode, tri-plane mode, B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastography, TVI, strain, strain rate, and combinations thereof, etc. Image beams and / or frames are stored, and timing information indicating the time of data acquisition in the memory may be recorded. These modules may include, for example, a scan conversion module for performing scan conversion operations to convert image frames from beam space coordinates to display space coordinates. A video processor module may be provided that reads image frames from memory such as memory 120 and displays image frames in real time during surgery on a patient. The video processor module may store image frames in an image memory, read from and display images from that image memory.

[0029] Figure 2 This is a flowchart of method 200 according to an exemplary embodiment. The various blocks of the flowchart represent steps that can be performed according to method 200. Additional embodiments may perform the steps shown in a different order, and / or additional embodiments may include... Figure 2 Additional steps not shown. Due to the nature of method 200, Figure 2 Some of the steps shown may be performed simultaneously or during overlapping time periods. Method 200 will be described according to an exemplary embodiment in which method 200 is performed by… Figure 1 The ultrasound imaging system 100 shown is used for execution. The technical effect of method 200 is that it automatically executes actions based on guidance information calculated from low-resolution volume.

[0030] Regarding Figure 1 and Figure 2 Method 200 is described. At step 202, processor 116 controls ultrasound probe 106 to acquire high-resolution slice data from the patient. The high-resolution slice data may be 2D ultrasound data acquired from a planar or substantially planar region, or it may be ultrasound data acquired from a thin plate. The 2D ultrasound data includes the thickness of a single sample in a direction perpendicular to the acquisition plane. Conversely, the thin plate has a thickness greater than that of a single sample in a direction perpendicular to the acquisition plane. Figure 3 This is a schematic diagram of an ultrasonic probe 301 and a planar region 302 according to one embodiment. Figure 3This includes coordinate axis 306. Coordinate axis 306 represents the azimuth direction 308, the depth direction 310, and the elevation direction 312. High-resolution slice data can be 2D ultrasound data acquired from planar region 302. Figure 4 This is a schematic diagram of the ultrasound probe 301 and the thin plate 304. According to one embodiment, high-resolution slice data can be acquired from the thin plate 304. Planar region 302 (e.g., Figure 3 (As shown) is only the width of a single sample in the elevation direction of 312. On the other hand, in Figure 4 In the example shown, the plate 304 has a thickness greater than that of a single sample in the elevation direction. The plate 304 can be defined as having an opening angle 303 of 15 degrees or less. According to an exemplary embodiment, the high-resolution slice data may include composite ultrasound data from two or more planes acquired from slightly different locations in the elevation direction 312.

[0031] At step 204, processor 116 controls display device 118 to display ultrasound images based on high-resolution slice data. According to one exemplary embodiment, the ultrasound image based on high-resolution slice data can be a real-time ultrasound image. For example, the ultrasound image can be repeatedly refreshed to reflect frames of the most recently acquired high-resolution slice data during the acquisition process.

[0032] According to one exemplary embodiment, steps 206 and 208 can be performed during the process of executing steps 202 and 204. At step 206, the processor 116 controls the ultrasound probe to acquire low-resolution volumetric data during the process of acquiring high-resolution slice data in step 202. Figure 5 This is a schematic diagram of an ultrasound probe 301 and a representation of a volume 320 from which low-resolution volumetric data can be acquired. The processor 116 can be configured to alternate between acquiring one or more frames of high-resolution slice data and acquiring one or more frames of low-resolution volumetric data while performing steps 202 and 206. For example, according to one exemplary embodiment, the processor 116 can be configured to control the ultrasound probe 106 to acquire frames of low-resolution volumetric data between two adjacent frames of high-resolution slice data. According to other embodiments, the processor 116 can be configured to control the ultrasound probe 106 to acquire multiple frames of high-resolution slice data between each adjacent frame of low-resolution volumetric data.

[0033] According to other embodiments, processor 116 may be configured to control ultrasound probe 106 to interleave the acquisition of low-resolution volumetric data with the acquisition of high-resolution slice data while performing steps 202 and 206. For example, processor 116 may control ultrasound probe 106 to acquire low-resolution volumetric data based on one or more transmission lines transmitted during the process of acquiring a single frame of high-resolution slice data. Additional details regarding various acquisition schemes for acquiring high-resolution slice data and low-resolution volumetric data will be described below.

[0034] A key aspect of this invention is that it allows for the acquisition of low-resolution volumetric data with minimal impact on the image quality of ultrasound images generated from high-resolution slice data. In contrast to high-resolution volumetric data, acquiring low-resolution volumetric data reduces the total acquisition time dedicated to volumetric acquisition. This frees up time during the acquisition sequence for acquiring high-resolution slice data. The objective of this invention is to use low-resolution volumetric data to compute guidance information. Since it is not intended for display, the resolution of low-resolution volumetric data can be significantly lower than that of conventional high-resolution volumetric data. This, in turn, minimizes the impact on the resolution of the high-resolution slice data, which is intended for display and diagnostic purposes.

[0035] In various implementations, low-resolution volumetric data can be acquired using a relatively small number of transmission events compared to conventional volumetric acquisition. For example, according to one implementation, low-resolution volumetric data can be acquired using 25 or fewer transmission events. For example, 25 transmission events could correspond to a 5×5 transmission pattern in the azimuth direction 308 and the elevation direction 312. According to other implementations, low-resolution volumetric data can be acquired using fewer than 16 transmission events. For example, 16 transmission events could correspond to a 4×4 transmission pattern in the azimuth direction 308 and the elevation direction 312. In other implementations, transmission events can be distributed in an irregular or staggered pattern. For example, transmission events in adjacent rows in the elevation direction 312 can be offset in the azimuth direction 308. It should be understood that other implementations can use different numbers of transmission events and / or the transmission events can be arranged in different transmission patterns. For example, implementations can use different numbers of transmission events in the elevation and azimuth directions.

[0036] Processor 116 can be configured to acquire low-resolution volumetric data at step 206 using defocused transport events. For example, processor 116 can be configured to transport a defocused wave for each transport event in the transport events. Each defocused transport event covers a larger field of view than a regular focused transport event. Using defocused transport events helps to acquire more sample in the volume while using fewer transport events. As previously discussed, using fewer transport events results in a reduction in the total time spent acquiring low-resolution volumetric data, which allows for the expenditure of more time and resources when acquiring high-resolution slice data and generating ultrasound images based on the high-resolution slice data.

[0037] According to an embodiment where the receiving beamformer 110 is a software beamformer, the processor 116 can be configured to use nonlinear beamforming techniques to beamform low-resolution volumetric data and / or high-resolution slice data. Linear beamforming techniques typically include delay-summing beamforming. Nonlinear beamforming techniques may include delay-multiplication-summing, or any other nonlinear operation applied during the process of beamforming low-resolution volumetric or high-resolution slice data. Nonlinear beamforming techniques can be used to reduce the level of sidelobes caused by otherwise defocused transmission beams. Therefore, nonlinear beamforming techniques can help recover some spatial resolution due to the relatively low number of transmission events used to acquire low-resolution volumetric data.

[0038] According to one embodiment, the receive beamformer 110 can be configured to apply massive multiline beamforming technology to low-resolution volumetric data acquired along each transmission line. The use of massive multiline beamforming technology may be particularly advantageous for embodiments using a relatively small number of transmission events. Massive multiline beamforming acquires data along more than 16 receive lines for each transmission line. According to one exemplary embodiment, the receive beamformer 110 can beamform 32 or more receive lines for each transmission line. According to other embodiments, the receive beamformer 110 can be configured to beamform 64 or more receive lines for each transmission line.

[0039] In addition to or instead of lower spatial resolution, low-resolution volumetric data can have lower temporal resolution. For example, conventional volumetric acquisition can have frame rates varying between approximately 20 frames per second (FPS) and 70 FPS. For example, low-resolution volumetric data can be acquired at much lower frame rates. For example, according to some embodiments, acquiring low-resolution volumetric data at frame rates between 0.25 FPS and 10 FPS may be sufficient. In other embodiments, the low-resolution frame rate can be higher than 10 FPS. According to one embodiment, the frame rate (i.e., temporal resolution) and spatial resolution of low-resolution volumetric data can be inversely proportional. That is, an embodiment using low-resolution volumetric data with relatively low spatial resolution can have a relatively high frame rate; and an embodiment using low-resolution volumetric data with relatively high spatial resolution can have a relatively low frame rate.

[0040] Based on the above description, it should be understood that, for the purposes of this application, the term "low-resolution volumetric data" can be defined as including volumetric data with a lower resolution than high-resolution slice data; volumetric data acquired using a lower number of transmission events than conventional volumetric data acquisition; or volumetric data acquired using a lower frame rate than conventional volumetric data acquisition. It should be understood that low-resolution volumetric data can include different combinations of spatial and temporal resolution based on the acquisition type and the amount of acceptable image quality reduction for high-resolution slice data.

[0041] As previously discussed, according to various embodiments, processor 116 can be configured to acquire low-resolution volume data in an interleaved manner with high-resolution slice data. According to one embodiment, one or more transmission events for low-resolution volume acquisition can be transmitted during the acquisition of a single frame of high-resolution slice data. One or more additional transmission events for the low-resolution volume can be transmitted during the acquisition of subsequent frames of high-resolution slice data. Using this technique, frames that do not fully acquire low-resolution volume data are acquired up to all different transmission events of the transmitted volume. Interleaving only a subset of transmission lines during the acquisition of a single frame of high-resolution slice data minimizes the time penalty on each frame of high-resolution slice data. For example, according to one embodiment, where only four transmission events are interleaved with the acquisition of transmission events for each frame of high-resolution slice data, the elapsed acquisition time for each frame of high-resolution slice data is extended only by the time spent transmitting those four transmission events and receiving data based on those four transmission events. Similarly, for an embodiment where only one transmission event is interleaved with the acquisition of each frame of high-resolution slice data, the elapsed acquisition time for each frame of high-resolution slice data is extended only by the time spent transmitting that one transmission event and receiving data based on that one transmission event.

[0042] According to one implementation, transmission events for acquiring low-resolution volumetric data can be uniformly spread across the transmission sequence of each frame of high-resolution slice data. Due to the addition of transmission events for low-resolution volumetric data, this interleaving technique may result in a slight loss of consistency due to the additional time between adjacent transmission events for high-resolution slice data.

[0043] According to one embodiment, transmission events for low-resolution volumes can be randomly distributed among transmission events in each frame of high-resolution slice data. The random distribution of transmission events for low-resolution volumes can be modified frame-to-frame such that transmission events for low-resolution volume data are positioned at different times / locations in adjacent or consecutive frames of the transmission sequence used for high-resolution slice data, minimizing the visual impact due to consistency loss caused by interleaving. According to one embodiment, the low-resolution volume data is not displayed as an image. To minimize the impact on the high-resolution slice data, it is anticipated that the resolution of the low-resolution volume data will be too low to be displayed. The low-resolution volume data is "background" data because it is not intended for display.

[0044] At step 208, processor 116 calculates guidance information based on the low-resolution volume generated from the low-resolution volume data. According to one embodiment, processor 116 may implement a neural network to calculate the guidance information. Additional information regarding the calculation of the guidance information will be discussed below.

[0045] See now Figure 6 and Figure 7 The illustration shows an exemplary neural network according to one exemplary embodiment. In some examples, the neural network can be trained using a training set of volumetric data. According to other embodiments, the neural network can be trained using both a set of low-resolution volumetric data and a set of high-resolution volumetric data.

[0046] Figure 6 A schematic diagram of a neural network 500 having one or more nodes / neurons 502 is depicted. In some embodiments, the one or more nodes / neurons may be arranged in one or more layers 504, 506, 508, 510, 512, 514, and 516. The neural network 500 may be a deep neural network. As used herein with respect to neurons, the term "layer" refers to a collection of analog neurons having inputs and / or outputs that are connected in a similar manner to other sets of analog neurons. Thus, as... Figure 6As shown, neurons 502 can be connected to each other via one or more connections 518, allowing data to propagate from the input layer 504 through one or more intermediate layers 506, 508, 510, 512, and 514 to the output layer 516. The one or more intermediate layers 506, 508, 510, 512, and 514 are sometimes referred to as "hidden layers".

[0047] Figure 7 The input and output connections of a neuron according to an exemplary embodiment are shown. Figure 7 As shown, the connections (e.g., 518) of a single neuron 502 may include one or more input connections 602 and one or more output connections 604. Each input connection 602 of neuron 502 may be an output connection of a preceding neuron, and each output connection 604 of neuron 502 may be an input connection of one or more subsequent neurons. Although Figure 7 Neuron 502 is depicted as having a single output connection 604; however, it should be understood that a neuron may have multiple output connections that send / transmit / pass the same value. In some embodiments, neuron 502 may be a data construct (e.g., a structure, an instantiated class object, a matrix, etc.), and the input connections may be received by neuron 502 as weighted numerical values ​​(e.g., floating-point or integer values). For example, as... Figure 7 As further shown, input connections X1, X2, and X3 can be weighted and summed by weights W1, W2, and W3, respectively, and sent / transmitted / passed as output connection Y. It will be understood that the processing of a single neuron 502 can typically be represented by the following formula:

[0048]

[0049] Where n is the total number of input connections 602 to neuron 502. In one implementation, the value of Y may be based at least in part on whether the sum of WiXi exceeds a threshold. For example, if the sum of the weighted inputs does not exceed the desired threshold, Y may have a zero (0) value.

[0050] from Figure 6 and Figure 7 Further understanding is needed; an input connection 602 of neuron 502 in input layer 504 can be mapped to input 501, and an output connection 604 of neuron 502 in output layer 516 can be mapped to output 530. As used herein, “mapping” a given input connection 602 to input 501 means how input 501 influences / indicates the value of said input connection 602. Similarly, as used herein, “mapping” a given output connection 604 to output 530 means how the value of said output connection 604 influences / indicates output 530.

[0051] Therefore, in some implementations, the acquired / obtained input 501 is passed / feeded to the input layer 504 of the neural network 500 and propagates through layers 504, 506, 508, 510, 512, 514, and 516, such that the mapping output connection 604 of the output layer 516 generates / corresponds to the output 530. As shown, the input 501 may include a low-resolution volume. The cardiac image may depict one or more structures recognizable by the neural network 500. Furthermore, the output 530 may include structures, landmarks, contours, or planes associated with a standard view.

[0052] Multiple training datasets can be used to train the neural network 500. Depending on the implementation, the neural network 500 can be trained with volumetric data. Low-resolution volumetric data, high-resolution volumetric data, or a combination of both can be used to train the neural network 500. Each training dataset may include, for example, annotated volumetric data. Based on the training datasets, the neural network 500 can learn to recognize multiple structures derived from the volumetric data. Machine learning or deep learning within this process (e.g., due to identifiable trends in the arrangement, size, etc. of anatomical features) may cause changes in weights (e.g., W1, W2, and / or W3), input / output connections, or other adjustments to the neural network 500. Furthermore, as additional training datasets are adopted, machine learning can continue to adjust various parameters of the neural network 500 in response. Thus, the sensitivity of the neural network 500 can be periodically increased, resulting in higher accuracy in anatomical feature recognition.

[0053] According to one embodiment, a neural network 500 can be trained to identify landmark structures in volumetric data. For example, according to one embodiment where the volumetric data is cardiac data, the neural network 500 can be trained to identify structures such as the right ventricle, left ventricle, right atrium, left atrium, one or more valves (such as the tricuspid valve), mitral valve, aortic valve, apex of the left ventricle, septum, etc. The neural network 500 can be trained, for example, using low-resolution volumetric data that has been annotated to identify one or more structures that can be used as landmarks. According to another embodiment, the neural network 500 can be trained using both low-resolution and high-resolution volumetric data. The high-resolution and low-resolution volumetric data used for training can, for example, have been acquired in pairs from the same patient, ultrasound probe, and probe position and orientation. For example, one such pair may include a low-resolution volumetric dataset acquired from a first position and orientation and a high-resolution volumetric dataset acquired from the same position and orientation. By using the same ultrasound probe, the same patient, and the same position and orientation, the pair may differ primarily in data resolution. The low-resolution volumetric data is closer to the data to be acquired at step 206 in method 200. However, using high-resolution volumetric data to train the neural network 500 may result in greater accuracy in structure recognition due to the higher resolution of the data. Furthermore, because low-resolution and high-resolution volumetric data are paired, the neural network 500 is able to learn how a low-resolution volumetric dataset corresponds to a high-resolution volumetric dataset of the same volume.

[0054] According to one embodiment, neural network 500 can be trained to identify the location of one or more standard views based on training data. A non-limiting list of exemplary standard views includes: a parasternal long-axis view, a parasternal short-axis view, an apical four-chamber view, an apical two-chamber view, an apical three-chamber view, an apical five-chamber view, and a coronary sinus view. Processor 116 can identify the use of standard views according to various techniques. According to one exemplary embodiment, neural network 500 can be trained to identify various standard views from a low-resolution volume. According to another embodiment, processor 116 can be configured to utilize neural network 500 to identify standard views using landmarks identified in a low-resolution volume.

[0055] Returning to step 208 of method 200, processor 116 calculates guidance information based on a low-resolution volume generated from the low-resolution volume data using the output of neural network 500. As discussed above, according to some embodiments, neural network 500 may determine the location of one or more landmark structures based on the low-resolution volume, or neural network 500 may determine the location of one or more standard views. Since low-resolution volume data is acquired during the acquisition of high-resolution slice data, it is acquired from the same or substantially the same location as the high-resolution slice data. According to many embodiments, the high-resolution slice data is displayed as a real-time ultrasound image. In other words, low-resolution volume data is acquired from the real-time location of ultrasound probe 106. Based on the correlation between the displayed ultrasound image, the low-resolution volume data acquired by ultrasound probe 106, and the current location and orientation of ultrasound probe 106, guidance information can be used to provide feedback to assist clinicians in acquiring the desired ultrasound data for a given clinical procedure.

[0056] At step 210 of method 200, processor 116 automatically performs an action based on the guidance information calculated at step 208. For example, according to one embodiment, processor 116 may display guidance on display device 118 for adjusting ultrasound probe 106 to obtain a desired view. Processor 116 may display guidance by displaying text describing how a clinician should move ultrasound probe 106, by displaying one or more graphic icons instructing how ultrasound probe 106 should be moved, or by displaying a combination of text and one or more graphic icons instructing how ultrasound probe 106 should be moved. Guidance may, for example, include directions associated with one or more of translating ultrasound probe 106, rotating ultrasound probe 106, and tilting ultrasound probe 106. For example, graphic icons may include arrows or other directional indicators to convey how ultrasound probe 106 should be adjusted to obtain high-resolution slice data from the desired view. According to various embodiments, guidance may be displayed on display device 118 along with probe icons to help clinicians better understand how ultrasound probe 106 should be adjusted to acquire the desired view. For example, Figure 8 This is a screenshot 800 including icons for ultrasound image 801, ultrasound probe 802, and a first graphic icon 804 and a second graphic icon 806. The first graphic icon 804 is an arrow indicating that the ultrasound probe 106 should be tilted in the direction indicated by the arrow. The second graphic icon 806 is a curved arrow indicating that the ultrasound probe 106 should be rotated in a clockwise direction as indicated by the curved arrow. According to various embodiments, other embodiments may use different graphic icons and / or different icons for the ultrasound probe to graphically convey how the ultrasound probe 106 should move (i.e., tilt, rotate, or translate, one or more of these).

[0057] According to one embodiment, processor 116 can automatically perform actions based on guidance information by automatically adjusting the orientation parameters of ultrasound probe 106 to acquire a desired view using ultrasound probe 106 in its current position and orientation. For example, previous embodiments involved displaying guidance to clinicians on how to adjust the position of ultrasound probe 106 to acquire high-resolution slice data from the desired view and / or acquire high-resolution slice data including desired anatomical regions. Instead of providing instructions for adjusting ultrasound probe 106, processor 116 can alternatively adjust the orientation parameters of ultrasound probe 106 to acquire additional high-resolution slice data from a plane or plate that has a different geometry from the ultrasound probe compared to previously acquired high-resolution slice data. For example, processor 116 can control ultrasound probe 106 to acquire additional high-resolution slice data relative to ultrasound probe 106 from one or more planes or plates with different tilts or rotations. For example, processor 116 can calculate adjustments to the current position and orientation of the ultrasound probe so that the high-resolution slice data includes a desired standard view based on a low-resolution volume. For example, processor 116 can use the output of neural network 500 to identify the position of the standard view relative to the low-resolution volume. Since the position of the current acquisition plane or plate relative to the low-resolution volume is known, the processor 116 can then calculate the necessary adjustments to the ultrasound probe 106 for acquiring high-resolution slice data of the standard view.

[0058] According to one embodiment, processor 116 can provide a warning that ultrasound probe 106 has been incorrectly positioned. For example, some standard views, such as an apical four-chamber view, should include the apex of the patient's heart. However, clinicians, especially inexperienced clinicians, may have difficulty correctly positioning the ultrasound probe to acquire high-resolution slice data across a plane passing through the apex of the patient's heart. When an ultrasound image that should include the apex does not, it is referred to as "shortening" because the left ventricle appears shorter in the image than its actual length. According to one embodiment, neural network 500 can be configured to detect one or more structures from low-resolution volumetric data, and then, based on the output from neural network 500, processor 116 can determine whether the current high-resolution slice view includes the apex of the heart or whether it has been shortened. If processor 116 determines that the current image has been shortened, processor 116 can, for example, display a warning that ultrasound probe 106 has been incorrectly positioned. For example, processor 116 can display notifications such as the words "Warning," "Incorrect View," "Shortening Detected," or other warning notifications. Figure 9This is a screenshot 900 according to one embodiment. Screenshot 900 includes an ultrasound image 901 and a notification 902, which is a text-based notification stating "Shortening detected." Other embodiments may display different text-based notifications and / or may use other technologies to provide a warning. Figure 9 The example shown involves detecting shortening; it should be understood that different text-based notifications can be used to provide warnings associated with different views that may not involve shortening. Other implementations may use audible alarms or colors to provide warnings that the ultrasound probe 106 has been mispositioned. Figure 2 As shown, at step 212, if additional high-resolution slice data and additional low-resolution volumetric data are desired, the method can return to steps 202 and 206. As long as the clinician continues to acquire additional data (both high-resolution slice data and low-resolution volumetric data) using the ultrasound probe 106, method 200 can automatically return from step 212 to steps 202 and 206. Once at steps 202 and 206, method 200 can repeat steps 202, 204, 206, 208, 210, and 212. With each iteration, method 200 can display ultrasound images based on the most recently acquired high-resolution slice data, and processor 116 can calculate guidance information based on the most recently acquired low-resolution volumetric data. Therefore, the processor can perform a guidance-based action at step 210 based on the most recently acquired low-resolution volumetric data, which reflects the real-time or near-real-time position and orientation of the ultrasound probe 106. As method 200 is repeated iteratively, processor 116 can update guidance information and actions to be performed based on the current position and orientation of ultrasound probe 106. For example, while performing method 200, a clinician may adjust one or both of the position and orientation of ultrasound probe 106. Processor 116 can be configured to provide a positive message when it is determined that high-resolution slice data of the desired plane is being acquired. For example, a positive message can identify that the correct view has been acquired by displaying text on display device 118 stating statements such as “Correct view acquired,” “Four-chamber view acquired,” or other positive messages. Alternatively, processor 116 can simply remove warnings displayed on display device 118 when it is determined that high-resolution slice data of the desired plane is being acquired.

[0059] According to one embodiment, the processor 116 can automatically position a region of interest (ROI) relative to the ultrasound image. According to various embodiments, the ROI can be a 2D, 3D, or 4D ROI. The ROI can be automatically positioned to indicate the area from which different modes of ultrasound data should be acquired. For example, the ROI can be a color box used to indicate the area from which color or Doppler ultrasound data should be acquired, or the ROI can be a Doppler gate to indicate the area from which pulse-wave (PW) Doppler data should be acquired. Figure 10 Screenshot 950 according to one embodiment is shown. Screenshot 950 includes an ultrasound image 951 and a ROI box 952 located on the image 951. The ultrasound image 951 is a four-chamber view, and the ROI box 952 has been automatically placed around the mitral valve by the processor 116. Although Figure 10 The ROI box 902 is used to represent the location of the region of interest. However, it should be understood that in other embodiments, the region of interest may be automatically placed by the processor 116 without physically positioning the ROI box or other ROI graphic indicators on the image.

[0060] According to other embodiments, low-resolution volumetric data can be used to identify scaling regions. For example, low-resolution volumetric data can be used to identify 2D or 3D scaling regions. Processor 116 can be configured to use image processing techniques to identify one or more regions using low-resolution volumetric data, and then automatically place 2D or 3D scaling around the one or more regions identified using the low-resolution volumetric data. For example, processor 116 can use low-resolution volumetric data to automatically magnify structures of interest. According to other embodiments, processor 116 can implement one or more neural networks to automatically identify one or more structures of interest in low-resolution volumetric data and / or locate 2D or 3D scaling regions around the identified structures of interest. According to one embodiment, processor 116 can be configured to display candidate scaling boxes for 2D or 3D scaling regions on an ultrasound image generated from high-resolution slice data. For example, the operator would have the option to accept candidate scaling boxes or adjust candidate scaling boxes before accepting them. According to one embodiment, processor 116 can display boxes around candidate 2D scaling regions in an ultrasound image, and processor 116 can display representations of 3D boxes around candidate 3D scaling regions. Those skilled in the art will understand that, depending on the various implementations, other geometries may be used to indicate candidate scaling regions.

[0061] Figure 11This is a screenshot 960 according to one embodiment. Screenshot 960 includes an ultrasound image 961 and a representation of a 3D bounding box 962 surrounding a candidate 3D zoomed region. Processor 116 is configured to zoom in on the 3D zoomed region indicated by the location of the representation of the 3D bounding box 962.

[0062] Processor 116 can be configured to simultaneously display a model of the patient's anatomy with ultrasound images generated based on high-resolution slice data. Processor 116 can be configured to display a planar representation relative to the model. According to one embodiment, processor 116 can register the position of ultrasound probe 106 to the model using low-resolution volumetric data. Processor 116 can then display a planar representation relative to the model. For most embodiments, the resolution of the rendering generated from low-resolution volumetric data is expected to be too low to be displayed. Registering low-resolution volumetric data to the model can display a more visually pleasing representation of the patient's anatomy than directly displaying images based on low-resolution volumetric data. Furthermore, by displaying a planar representation, clinicians can quickly and easily see the position of the patient's anatomy relative to the plane from which the ultrasound images were acquired.

[0063] Figure 12 Screenshot 970 is based on one embodiment. Screenshot 970 includes an ultrasound image 971 and a model 972. According to an exemplary embodiment, low-resolution volumetric data can be fitted to or registered to model 972. Screenshot 970 also includes a representation of plane 974. The representation of plane 974 is shown in its position relative to model 972 to indicate the plane from which image 971 is acquired. Clinicians can use the representations of model 972 and plane 974 to help correctly position the scanning plane to acquire the patient's desired anatomy or desired view. Figure 12 In the example shown, model 972 is a model of a heart. Model 972 can be a rendered solid, a rendered wireframe, a rendered mesh, or can be rendered using any other technique. Model 972 provides a higher resolution view of the heart than is possible by rendering low-resolution volumetric data.

[0064] This written description uses examples to disclose the invention, including the best mode, and also enables those skilled in the art to practice the invention, including making and using any device or system and performing any included methods. The scope of the invention is defined by the claims and may include other examples that would occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that are not indistinguishable from the literal language of the claims, or if they include equivalent structural elements that have minor differences from the literal language of the claims.

Claims

1. An ultrasound imaging system, the ultrasound imaging system comprising: Ultrasonic probe; user interface; Display devices; and A processor that electronically communicates with the ultrasound probe, the user interface, and the display device, wherein the processor is configured to: The ultrasound probe is controlled to acquire low-resolution volumetric data during the process of acquiring high-resolution slice data of the acquisition plane, wherein the low-resolution volumetric data has a lower resolution than the high-resolution slice data; The ultrasound image is displayed on the display device based on the high-resolution slice data; A low-resolution volume is generated based on the low-resolution volume data, wherein the low-resolution volume is not displayed. Implement a neural network to compute guidance information based on the low-resolution volume; and Based on the guidance information, at least one of the following actions will be automatically executed: The system displays instructions for adjusting the ultrasound probe to obtain the desired view. Adjust the orientation parameters of the ultrasound probe to acquire the desired view in the current position and orientation using the ultrasound probe; Provide a warning that the ultrasound probe has been incorrectly positioned; or place the region of interest relative to the ultrasound image.

2. The ultrasound imaging system according to claim 1, wherein the ultrasound image is a real-time ultrasound image.

3. The ultrasound imaging system of claim 1, wherein the low-resolution volumetric data is acquired at fewer than 25 transmission events per volume.

4. The ultrasound imaging system of claim 1, wherein the processor is configured to acquire the low-resolution volume data by transmitting a defocused wave.

5. The ultrasound imaging system of claim 1, wherein implementing the neural network to calculate the guidance information includes identifying the desired view based on the low-resolution volume, and wherein the processor is configured to automatically perform the actions that display the guidance suggestions for adjusting the ultrasound probe in order to obtain the desired view.

6. The ultrasound imaging system of claim 1, wherein implementing the neural network to calculate the guidance information includes identifying the desired view based on the low-resolution volume, and wherein the processor is configured to automatically perform the action of adjusting the steering parameters of the ultrasound probe to acquire the desired view in the current position and orientation using the ultrasound probe.

7. The ultrasound imaging system of claim 1, wherein implementing the neural network to calculate the guidance information includes identifying the desired view based on the low-resolution volume, wherein the processor is configured to determine that the ultrasound probe is mispositioned based on the desired view, and wherein automatically performing at least one of the following actions includes providing a warning that the ultrasound probe is mispositioned.

8. The ultrasound imaging system of claim 1, wherein implementing the neural network to compute the guidance information includes identifying a desired region of interest from the low-resolution volume, and wherein automatically performing at least one of the following actions includes automatically positioning the region of interest relative to the ultrasound image to include the desired region of interest.

9. An ultrasound imaging method, the ultrasound imaging method comprising: During the process of acquiring high-resolution slice data of a plane using an ultrasonic probe, low-resolution volume data is acquired using the same ultrasonic probe, wherein the low-resolution volume data has a lower resolution than the high-resolution slice data. Display ultrasound images on a display device based on the high-resolution slice data; A low-resolution volume is generated based on the low-resolution volume data, and the low-resolution volume is not displayed. A neural network is implemented using a processor to compute guidance information based on the low-resolution volume; as well as The processor automatically performs at least one of the following actions based on the guidance information: displays guidance suggestions for adjusting the ultrasound probe to obtain the desired view; adjusts the orientation parameters of the ultrasound probe to acquire the desired view in the current position and orientation; provides a warning that the ultrasound probe has been incorrectly positioned; or automatically places the region of interest relative to the ultrasound image.

10. The method of claim 9, wherein the low-resolution volume data is acquired at fewer than 25 transmission events per volume.

11. The method of claim 10, wherein acquiring the low-resolution volume data includes transmitting a defocused wave.

12. The method of claim 9, wherein implementing the neural network to calculate the guidance information includes implementing the neural network to identify the desired view based on the low-resolution volume, and wherein automatically performing at least one of the following actions includes displaying the guidance suggestions for adjusting the ultrasound probe to obtain the desired view.

13. The method of claim 9, wherein implementing the neural network to calculate the guidance information includes implementing the neural network to identify the desired view based on the low-resolution volume, and wherein automatically performing at least one of the following actions includes adjusting the orientation parameters of the ultrasound probe to acquire the desired view at the current position and orientation using the ultrasound probe.

14. The method of claim 9, wherein implementing the neural network to compute the guidance information includes implementing the neural network to identify the desired view based on the low-resolution volume, and wherein automatically performing at least one of the following actions includes providing a warning that the ultrasound probe has been mispositioned.

15. The method of claim 9, wherein the ultrasound image is a cardiac ultrasound image, and the desired view is an apical view, and wherein the warning includes a notification that the ultrasound image being displayed has been shortened.

Images