Systems and methods for image optimization - Patents.com

Abstract

The ultrasound imaging system includes a view recognition processor. The view recognition processor determines when a particular view has been acquired by the ultrasound imaging system. A signal indicating that the particular view has been acquired is provided to an optimization state controller included in the ultrasound imaging system. Based on the signal, the optimization state controller determines optimized imaging settings for the particular view. The optimized imaging settings are provided to other components of the ultrasound imaging system to acquire or process an ultrasound image of the particular view with the optimized imaging settings.

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE

[0001] The present disclosure relates to imaging systems and methods for automated view recognition and image acquisition parameter adjustment. Certain embodiments include a system configured to automatically recognize anatomical views and adjust acquisition parameters based on the anatomical view. [Background technology]

[0002] During an ultrasound examination, a sonographer scans a plane and / or volume of a subject to obtain one or more images. Typically, the sonographer obtains one or more standard views of the subject. A standard view is an image of an anatomical structure from a particular location and angle that is known to provide diagnostic value to a review (e.g., a radiologist). The number and type of standard views depend on the type of ultrasound examination. For example, an echocardiogram (e.g., an ultrasound examination involving the heart) includes several standard views. Some exemplary ultrasound images of the standard views of an echocardiogram are shown in FIG. 6. Standard echocardiogram views are used to evaluate the health of the heart. For example, the parasternal long axis (PLAX) view shown in panel (e) shows the left atrium, left ventricle, right ventricle, and mitral valve. PLAX views are used to diagnose certain heart conditions, such as pericardial effusion (e.g., excess fluid surrounding the heart). Summary of the Invention [Problem to be solved by the invention]

[0003]

[0003] In addition to different locations and angles, different standard views require different system parameters, such as ultrasound image acquisition parameters and / or post-acquisition processing parameters, collectively referred to as imaging parameters. The imaging parameters include parameters such as lateral gain control, time gain compensation, transmit frequency, and power. Different imaging parameters are required due to the location (e.g., deep or shallow) and / or acoustic properties (e.g., inhomogeneity, rigidity) of the anatomy being scanned, and / or the characteristics of the acoustic window (e.g., intercostal) in which the standard views are acquired. For example, lateral gain control is used in the apical view (panes (a-d) in FIG. 6 ) to help visualize the heart wall, but adds undesirable noise to the parasternal view (panes (e-h) in FIG. 6 ). In addition, time gain compensation (TGC) adjusted to better visualize the apex in the apical window results in near-field overcompensation in the parasternal long axis view.

[0004]

[0004] There are also optimization trade-offs to be made based on the view for color flow, contrast, xPlane, and 3D imaging. For example, in color flow, the main direction of blood flow is toward and away from the transducer in apical views, but is primarily perpendicular to the transducer in parasternal views. In contrast imaging, the apical window often requires a lower mechanical index than the deeper parasternal views. The user must manually adjust the power level to compensate for this increased attenuation.

[0005]

[0005] Individually optimizing all of the imaging parameters for each standard view is time consuming and often impractical in the time allotted for an ultrasound examination. Furthermore, many ultrasound examiners may not have sufficient expertise to fully optimize all of the imaging parameters for each standard view.

[0006]

[0006] Currently, many commercially available ultrasound systems are equipped with "presets," which are preprogrammed sets of imaging parameters. The sonographer selects the type of exam (e.g., echocardiogram), and the ultrasound system applies the echocardiogram preset. The presets allow the sonographer to acquire standard views of the exam without having to adjust the imaging parameters. However, the imaging parameters of the presets are the result of trade-offs and compromises between parameters. That is, the imaging parameters of the presets allow reasonable standard views to be acquired, but no standard view can be acquired with imaging parameters optimized for that particular standard view. Therefore, it is desirable to improve the imaging parameters of the standard views. [Means for solving the problem]

[0007]

[0007] This disclosure describes systems and methods for optimizing imaging parameters for a particular view, which allows for improved image quality without changing the user's workflow.

[0008]

[0008] An ultrasound imaging system according to one example of the present disclosure includes an ultrasound transducer array configured to acquire an ultrasound image, a controller configured to control acquisition by the ultrasound transducer array based at least in part on one or more imaging parameters, a recognition processor configured to determine whether the ultrasound image corresponds to a particular view, and an optimization state controller configured to receive an output of the view recognition processor and determine an update of one or more imaging parameters based at least in part on the output if the view recognition processor determines that the ultrasound image corresponds to a particular view, wherein the optimization state controller provides the updated one or more imaging parameters to the controller.

[0009]

[0009] A method according to one example of the present disclosure includes steps of acquiring an ultrasound image, determining whether the ultrasound image includes a particular view, and if the particular view is determined, providing an output based on the particular view, determining one or more imaging parameters based at least in part on the output, providing the one or more imaging parameters to a controller, and re-acquiring the ultrasound image with the one or more imaging parameters.

[0010]

[0010] According to one example of the present disclosure, a non-transitory computer readable medium includes instructions that, when executed, cause an ultrasound imaging system to acquire an ultrasound image, determine whether the ultrasound image includes a particular view, if the particular view is determined, provide an output based on the particular view, determine one or more imaging parameters based at least in part on the output, and if the particular view is not determined, determine one or more default imaging parameters, provide the one or more imaging parameters or the one or more default imaging parameters to a controller, and re-acquire an ultrasound image with the one or more imaging parameters or the one or more default imaging parameters by the controller. [Brief description of the drawings]

[0011] [Figure 1]

[0011] FIG. 1 is a block diagram of an ultrasound system according to the principles of the present disclosure. [Diagram 2]

[0012] 1 is a block diagram illustrating an example processor in accordance with the principles of the present disclosure. [Diagram 3]

[0013] FIG. 1 is a block diagram of a process for training and deploying a neural network in accordance with the principles of the present disclosure. [Figure 4]

[0014] 1 is a flow diagram of a method according to the principles of the present disclosure. [Diagram 5]

[0015] 1 is a flow diagram of a method according to the principles of the present disclosure. [Figure 6]

[0016] FIG. 1 illustrates exemplary standard views in an echocardiography examination. [Figure 7A-7B]

[0017] FIG. 2 illustrates an example ultrasound image of the lateral heart wall. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012]

[0018] The following description of specific embodiments is merely exemplary in nature and is in no way intended to limit the invention or its application or uses. In the following detailed description of embodiments of the present system and method, reference is made to the accompanying drawings, which form a part of this specification, and in which specific embodiments in which the described system and method are practiced are shown by way of illustration. These embodiments have been described in sufficient detail to enable those skilled in the art to practice the disclosed system and method, and it should be understood that other embodiments may be utilized, and that structural and logical changes may be made without departing from the spirit and scope of the present system. Moreover, for the sake of clarity, detailed descriptions of specific features will not be discussed where they would be apparent to those skilled in the art, so as not to obscure the description of the present system. Therefore, the following detailed description should not be taken in a limiting sense, and the scope of the present system is defined only by the appended claims.

[0013]

[0019] In cardiac ultrasound, there are several areas that are traditionally very difficult to image. In the apical four-chamber view, the lateral wall is not well visualized. In this view, the lateral wall is located at the edge of the sector, indicated by circle 702 in FIG. 7A and circle 704 in FIG. 7B. To better visualize this wall of the heart, the gain is increased. The system has a lateral gain control that allows the user to selectively control the gain at the edge of the image. However, when moving to the next view, the user must reset these gains, as high gains introduce excessive noise. In many cases, the excessive noise causes the user to never optimize the lateral gains.

[0014]

[0020] One emerging area of ​​interest in echocardiography that may improve workflow is automated view classification. For example, machine learning techniques are based on histogram analysis and statistical features. In another example, deep learning is used to automate view recognition. Automated view recognition improves workflow, particularly measurement and / or analysis, and aids in patient diagnosis. In accordance with the principles of the present disclosure, view information provided by automated view recognition techniques is used to adjust imaging parameters, such as RF filters, time gain compensation (TGC), lateral gain compensation (LGC), and transmit frequency, to improve image quality and workflow. Imaging parameters include both acquisition parameters (e.g., settings for transmitting and receiving ultrasound signals) and post-acquisition parameters (e.g., settings for processing received ultrasound signals). The principles apply to various imaging modes, such as 2D echo imaging, color flow, contrast, xPlane, and 3D volumetric imaging.

[0015]

[0021] For example, detecting the apical four-chamber view and automatically adjusting the lateral gain improves visualization of the apical four-chamber view without increasing noise in other views. A further method to improve visualization is to provide different lateral gains for different parts of the cardiac cycle by automatically detecting segments of the cardiac cycle in a given standard view. As the heart contracts, the location of the lateral walls changes, and the user of the ultrasound imaging system cannot compensate for this. However, compensation by the ultrasound imaging system is achieved by detecting the location of the lateral walls by view recognition and modifying the imaging parameters (e.g., modifying the gain) throughout the cardiac cycle. Another compensation strategy used is to modify the imaging parameters at a specific location (e.g., lowering the receive RF filter where the lateral wall is located as detected by view recognition). Compensation by locally modifying the imaging parameters is done in combination with modifying the transmit frequency of the acoustic line. A lower frequency reduces attenuation and improves the signal-to-noise of the lateral walls. However, in other areas of the image where the signal to noise is already sufficient, low frequencies can introduce undesirable reverberation artifacts and reduce resolution, so the original frequencies are preserved in these areas.

[0016]

[0022] In accordance with the principles of the present disclosure, view-specific optimization is performed by a view recognition processor, acquisition parameters optimized for imaging views identified by the view recognition processor, and an optimization state controller that monitors the output of the view recognition processor and applies imaging parameters (e.g., view-specific system settings) in a manner that improves system responsiveness while reducing unstable transitions between imaging parameters that can be confusing to a user.

[0017]

[0023] An ultrasound system according to the principles of the present invention includes or is operatively coupled to an ultrasound transducer array configured to transmit ultrasound signals toward a medium, such as a human body or a particular portion thereof, and to receive echoes in response to the ultrasound signals. The ultrasound system includes a transmit controller and a beamformer configured to perform transmit and receive beamforming, respectively, and in some embodiments a display configured to display an ultrasound image generated by the ultrasound imaging system.

[0018]

[0024] In some embodiments, the ultrasound imaging system includes one or more processors, such as a view recognition processor that includes at least one model of a neural network. The neural network is trained to determine whether a particular view (e.g., a standard view for a given exam type) has been acquired, and if so, which particular view. The view recognition processor provides an output that includes an indication of the acquired standard view. In some applications, the indication of the standard view includes an indication of a physiological state, e.g., a phase of the cardiac cycle in cardiac imaging. The indication of which standard view has been acquired is provided to an optimization state controller. Based on the output of the view recognition processor (e.g., the indication of the standard view), the optimization state controller searches for an appropriate set of imaging parameters that are optimized for the standard view determined by the view recognition processor. In some applications, the imaging parameters are optimized not only for the particular view acquired, but also for the location in the image, e.g., where the lateral wall of the heart is located, a different gain setting is used. The imaging parameters are provided to one or more elements of the ultrasound imaging system (e.g., a beamformer), such that the ultrasound imaging system acquires the particular view using the optimized imaging parameters. In some embodiments, the optimized state controller provides imaging parameters only if certain conditions are met, such as when the ultrasound images acquired by the ultrasound imaging system have been stable for a period of time, thereby preventing a user from being distracted by rapid changes in the imaging parameters.

[0019]

[0025] The principles of the present disclosure improve the quality of ultrasound images acquired for each particular view (eg, reduced noise, improved visualization of anatomical structures, reduced artifacts).

[0020]

[0026] FIG. 1 shows a block diagram of an ultrasound imaging system 100 constructed according to the principles of the present disclosure. The ultrasound imaging system 100 according to the present disclosure includes a transducer array 114 included in an ultrasound probe 112, e.g., an external probe, or an internal probe such as an intravascular ultrasound (IVUS) catheter probe. In other embodiments, the transducer array 114 is in the form of a flexible array configured to be conformally applied to a surface of a subject (e.g., a patient) to be imaged. The transducer array 114 is configured to transmit ultrasound signals (e.g., beams, waves) and receive echoes in response to the ultrasound signals. Various transducer arrays may be used, e.g., linear arrays, curvilinear arrays, or phased arrays. The transducer array 114 may include, for example, a two-dimensional array of transducer elements (as shown) that may be scanned in both elevation and azimuth dimensions for 2D and / or 3D imaging. As is commonly known, the axial direction is the direction perpendicular to the face of the array (for a curvilinear array, the axial direction is a sector), the azimuth direction is generally defined by the longitudinal dimension of the array, and the elevation direction is transverse to the azimuth direction.

[0021]

[0027] In some embodiments, the transducer array 114 is coupled to a microbeamformer 116, which is disposed within the ultrasound probe 112, and which controls the transmission and reception of signals by the transducer elements of the array 114. In some embodiments, the microbeamformer 116 controls the transmission and reception of signals by the active elements of the array 114 (e.g., an active subset of the elements of the array that define an active aperture at a given time).

[0022]

[0028] In some embodiments, the microbeamformer 116 is coupled, for example, by a probe cable or wirelessly, to a transmit / receive (T / R) switch 118 that switches between transmit and receive and protects the main beamformer 122 from high energy transmit signals. In some embodiments, for example, in a portable ultrasound system, the T / R switch 118 and other elements of the system are included in the ultrasound probe 112 rather than in the ultrasound system base that houses the image processing electronics. The ultrasound system base generally includes circuitry for signal processing and image data generation, as well as software and hardware components that contain executable instructions for providing a user interface.

[0023]

[0029] The transmission of ultrasound signals from the transducer array 114 under the control of the microbeamformer 116 is directed by a transmit controller 120 coupled to a T / R switch 118 and a main beamformer 122. The transmit controller 120 controls the direction in which the beam is steered. The beam is steered straight ahead (vertically) from the transducer array 114 or at a different angle for a wider field of view. The transmit controller 120 is also coupled to a user interface 124 to receive input from user manipulation of the user controls. The user interface 124 includes one or more input devices, such as a control panel 152, including one or more mechanical controls (e.g., buttons, encoders, etc.), touch-sensitive controls (e.g., trackpad, touch screen, etc.), and / or other known input devices.

[0024]

[0030] In some embodiments, the partially beamformed signals produced by the microbeamformer 116 are coupled to a main beamformer 122, where the partially beamformed signals from the individual patches of transducer elements are combined into a fully beamformed signal. In some embodiments, the microbeamformer 116 is omitted and the transducer array 114 is under the control of the beamformer 122, which performs all beamforming of the signals. In embodiments with and without the microbeamformer 116, the beamformed signals of the beamformer 122 are coupled to processing circuitry 150. The processing circuitry 150 includes one or more processors (e.g., signal processor 126, B-mode processor 128, Doppler processor 160, and one or more image generation and processing components 168) configured to produce ultrasound images from the beamformed signals (i.e., beamformed RF data).

[0025]

[0031] The signal processor 126 is configured to process the received beamformed RF data in various manners, such as bandpass filtering, decimation, I and Q component separation, and harmonic signal separation. The signal processor 126 also performs additional signal enhancements, such as speckle reduction, signal combining, and noise removal. The processed signals (also referred to as I and Q components or IQ signals) are coupled to additional downstream signal processing circuitry for image generation. The I and Q signals are coupled to multiple signal paths within the system, each of which is associated with a particular arrangement of signal processing components suitable for generating different types of image data (e.g., B-mode image data, Doppler image data). For example, the system includes a B-mode signal path 158, which couples signals from the signal processor 126 to a B-mode processor 128 to produce B-mode image data.

[0026]

[0032] The B-mode processor utilizes amplitude detection for imaging of body structures. The signals produced by the B-mode processor 128 are coupled to a scan converter 130 and / or a multiplanar reformatter 132. The scan converter 130 is configured to arrange the echo signals from the received spatial relationships into a desired image format. For example, the scan converter 130 arranges the echo signals into a two-dimensional (2D) sector format or into a pyramidal or otherwise shaped three-dimensional (3D) format. The multiplanar reformatter 132 converts echoes received from points within a common plane of a volumetric region of the body into an ultrasound image of that plane (e.g., a B-mode image), as described, for example, in U.S. Pat. No. 6,443,896 (Detmer). The scan converter 130 and the multiplanar reformatter 132 are implemented as one or more processors in some embodiments.

[0027]

[0033] The volume renderer 134 generates an image (also called a projection, render, or rendering) of the 3D dataset as seen from a given reference point, for example as described in U.S. Patent No. 6,530,885 (Entrekin et al.). The volume renderer 134, in some embodiments, is implemented as one or more processors. The volume renderer 134 generates renderings, such as positive or negative renderings, by known or future known techniques, such as surface rendering or maximum intensity rendering.

[0028]

[0034] In some embodiments, the system includes a Doppler signal path 162 that couples the output from the signal processor 126 to a Doppler processor 160. The Doppler processor 160 is configured to estimate the Doppler shift and generate Doppler image data. The Doppler image data includes color data, which is then overlaid on the B-mode (i.e., grayscale) image data for display. The Doppler processor 160 is configured to filter out unwanted signals (i.e., noise or clutter associated with non-moving tissue), for example, using a wall filter. The Doppler processor 160 is further configured to estimate velocity and power according to known techniques. For example, the Doppler processor includes a Doppler estimator, such as an autocorrelator, where the velocity (Doppler frequency) estimate is based on the argument of the lag 1 autocorrelation function and the Doppler power estimate is based on the amplitude of the lag zero autocorrelation function. The motion is further estimated by known phase domain (e.g., parametric frequency estimators such as MUSIC, ESPRIT, etc.) or time domain (e.g., cross-correlation) signal processing techniques. Other estimators related to the time or spatial distribution of velocity, such as estimators of acceleration or time and / or spatial velocity derivatives, may be used instead of or in addition to the velocity estimator. In some embodiments, the velocity and power estimates are further subjected to threshold detection to further reduce noise, as well as post-processing such as segmentation, filling and smoothing. The velocity and power estimates are then mapped to a desired range of display colors by a color map. The color data, also referred to as Doppler image data, is then coupled to a scan converter 130, which converts the Doppler image data to a desired image format and overlays it onto a B-mode image of the tissue structure.

[0029]

[0035] In accordance with the principles of the present disclosure, outputs from the scan converter 130, such as B-mode and Doppler images, collectively referred to as ultrasound images, are provided to a view recognition processor 170. The view recognition processor 170 analyzes the ultrasound images to determine whether a particular view has been acquired. For example, if the imaging system is performing cardiac imaging, the view recognition processor 170 is configured to determine whether a particular standard view of the heart has been acquired (e.g., long axis or short axis parasternal, apical four-chamber view, or another standard view acquired through a subcostal / subxiphoid or apical window). In some embodiments, the view recognition processor 170 further determines a physiological state of the anatomical structure in the particular view. For example, continuing with the cardiac imaging example, the physiological state is the phase of the cardiac cycle of the standard view of the heart.

[0030]

[0036] Based on the determination that a particular view has been acquired, view recognition processor 170 generates an output (e.g., a signal). The output includes one or more signals or data identifying a particular view from the multiple views analyzed by processor 170 and / or a physiological condition of the anatomical structure in the ultrasound image. In other examples, the output includes image data corresponding to the identified particular view and / or data representative of the physiological condition of the anatomical structure. In some embodiments, the output further includes a signal or data representative of a confidence score. The confidence score is a measure of the accuracy of the view identification by view recognition processor 170. That is, the confidence score represents the likelihood or probability that the view identified by processor 170 as a particular view actually corresponds to the desired particular view and / or physiological condition.

[0031]

[0037] In some embodiments, the view recognition processor 170 utilizes a neural network, such as a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), an autoencoder neural network, etc., to recognize a particular view. A neural network is implemented in hardware (e.g., neurons represented by physical components) and / or software (e.g., neurons and pathways implemented in a software application) components. A neural network implemented according to the present disclosure uses various topologies and learning algorithms to train the neural network to produce a desired output. For example, a software-based neural network is implemented using a processor (e.g., a single-core or multi-core CPU, a single GPU or a GPU cluster, or multiple processors configured for parallel processing) configured to execute instructions. The instructions are stored in a computer-readable medium and, when executed, cause the processor to execute a trained algorithm to determine whether a particular view has been acquired.

[0032]

[0038] In various embodiments, the neural network is trained using any of a variety of now known or later developed learning techniques to obtain a neural network (e.g., a trained algorithm, or a hardware-based node system) configured to analyze input data in the form of ultrasound images, measurements, and / or statistics and determine whether and which particular views have been acquired. In some embodiments, the neural network is trained statically; that is, the neural network is trained on a data set and deployed to the view recognition processor 170. In some embodiments, the neural network is trained dynamically. In these embodiments, the neural network is trained on an initial data set and deployed to the view recognition processor 170. However, the neural network continues to train and change based on ultrasound images acquired by the system 100 after the neural network is deployed to the view recognition processor 170.

[0033]

[0039] In other embodiments, the view recognition processor 170 does not include a neural network. In other embodiments, the view recognition processor 170 is implemented using other suitable image processing techniques, such as image segmentation, histogram analysis, edge detection, or other shape or object recognition techniques. In some embodiments, the view recognition processor 170 implements a neural network in combination with other image processing methods to recognize a particular view.

[0034]

[0040] Although reference is made to indications of standard views (e.g., views required for a particular exam type to perform a diagnosis or other evaluation), in some embodiments the neural network is trained to recognize and provide indications of views desired by a user, for example in clinical studies where the usefulness of non-standard views is being evaluated, or for novel indications where standard views have not yet been established (e.g., monitoring a new disease, imaging a new implanted medical device).

[0035]

[0041] In some embodiments, the view recognition processor 170 provides an output to an optimization state controller 172. The optimization state controller 172 is implemented in suitable hardware and / or software. In some embodiments, the optimization state controller 172 is implemented by one or more processors that, in response to the output of the view recognition processor 170, determine appropriate imaging parameters for a particular view. The imaging parameters determined by the optimization state controller 172 include, but are not limited to, RF filters, TGCs, LGCs, and transmit frequencies. In some embodiments, determining the appropriate imaging parameters includes referencing a lookup table stored in a memory (e.g., the local memory 142) and retrieving appropriate acquisition parameters for the particular view from the memory (e.g., the local memory 142). In some such examples, the lookup table is implemented using a suitable relational data structure that associates a particular view (e.g., a standard apical four-chamber view) with a particular set of imaging parameters (e.g., a particular TGC, LGC, and transmit frequency setting).

[0036]

[0042] In an embodiment, one or more of the imaging parameters are uniform across the scan area of ​​the ultrasound image. In other embodiments, one or more of the imaging parameters vary across the scan area of ​​the ultrasound image. For example, when an anatomical feature is located in a particular view, one or more imaging parameters are different. Continuing with the echocardiography example, when the side wall of the heart is located in a particular view, the imaging parameters are adjusted in the scan area where the side wall of the heart is located. For example, the transmit frequency is lowered in the side wall area to improve visualization of the side wall, while the transmit frequency is raised in other parts of the scan area to reduce the introduction of excessive noise. In other examples where the acoustic properties of tissues are more homogeneous, such as imaging of the liver, the gain or other imaging parameters are uniform across the scan area.

[0037]

[0043] Some or all of the imaging parameters determined by optimization state controller 172 are provided to transmit controller 120 and / or beamformer 122. Ultrasound images are acquired with the determined imaging parameters (e.g., view-specific imaging parameters) by transmit controller 120 and / or beamformer 122. Some or all of the determined imaging parameters may also or alternatively be provided to image processor 136. Image processor 136 processes the acquired ultrasound images based on the imaging parameters and provides the processed ultrasound images to display 138.

[0038]

[0044] The optimization state controller 172 is responsible for controlling the imaging parameters of the ultrasound imaging system 100 over time. The optimization state controller 172 maintains the current imaging parameters, monitors the output of the view recognition processor 170, and combines this information to determine if and when the imaging parameters should be changed. When the optimization state controller 172 triggers an imaging parameter change, the optimization state controller 172 replaces its record of the current imaging parameters with the newly selected imaging parameters, provides the new imaging parameters to other components of the system 100 as described above, and then resumes monitoring the output of the view recognition processor 170 for potential future imaging parameter changes.

[0039]

[0045] The optimization state controller 172 provides the user with an optimal balance between system responsiveness and stability. If the optimization state controller 172 responds too quickly to the output of a particular view recognition processor 170, the wrong imaging parameters will be selected and / or the system 100 will change the imaging parameters so quickly that the display 138 will become unstable and the image will become unusable. In either case, the user will lose confidence in the system 100's ability to provide reliable image diagnosis. Thus, in some embodiments, the optimization state controller 172 waits for one or more states before determining imaging parameters or providing the determined imaging parameters. For example, the optimization state controller 172 waits for the metrics provided by the view recognition processor 170 to stabilize for a particular period of time (e.g., 0.5 s, 1 s, 2 s) or for a particular number of image frames (e.g., 5, 10, 30). In some embodiments, the optimization state controller 172 analyzes the confidence scores provided by the view recognition processor 170, perhaps over many frames, to determine if and when the view recognition processor 170 is sufficiently confident, and then determines and provides imaging parameters, for example, when the confidence score exceeds a threshold (e.g., 70%, 90%) for one or more frames. In some embodiments, the confidence score threshold is preset. In other embodiments, the threshold is set by user input.

[0040]

[0046] Optionally, in some embodiments, the ultrasound probe 112 includes or is coupled to a motion detector 174. The motion detector 174 provides signals to the optimization state controller 172 to indicate when the ultrasound probe 112 is moving and when it is stationary. In some embodiments, the optimization state controller 172 waits for a signal to indicate that the ultrasound probe 112 is stationary before determining imaging parameters and providing the determined imaging parameters. In some embodiments, the optimization state controller 172 waits for a signal to indicate that the ultrasound probe 112 is stationary for a set period of time (e.g., 0.5 seconds, 1 second, 2 seconds) before determining imaging parameters and providing the determined imaging parameters.

[0041]

[0047] In some embodiments, when the metrics for a particular view are unstable and / or the confidence score is below a threshold, for example when the user is actively moving the transducer to find a suitable acoustic window, the optimization state controller 172 maintains the previous imaging parameters or provides default, non-view-specific imaging parameters until the confidence of the recognized view is established. In some embodiments, the default imaging parameters are based on the exam type or other preset.

[0042]

[0048] Outputs from the scan converter 130, the multiplanar reformatter 132, and / or the volume renderer 134 (e.g., B-mode images, Doppler images) are coupled to an image processor 136 for further enhancement, buffering, and temporary storage, and then displayed on an image display 138. Although the output from the scan converter 130 is shown provided to the image processor 136 via a view recognition processor 170, in some embodiments the output of the scan converter 130 is provided directly to the image processor 136. A graphics processor 140 generates graphic overlays for display along with the images. These graphic overlays include standard identifying information, such as, for example, the patient name, the date and time of the image, imaging parameters, and the like. For these purposes, the graphics processor is configured to receive inputs from the user interface 124, such as a typed patient name or other annotations. A user interface 144 is further coupled to the multiplanar reformatter 132 for selection and control of the display of multiple multiplanarily formatted (MPR) images.

[0043]

[0049] The system 100 includes a local memory 142. The local memory 142 may be implemented as a suitable non-transitory computer-readable medium (e.g., flash drive, disk drive). The local memory 142 stores data generated by the system 100, including ultrasound images, executable instructions, imaging parameters, training data sets, or other information required for the operation of the system 100.

[0044]

[0050] As previously described, the system 100 includes a user interface 124. The user interface 124 includes a display 138 and a control panel 152. The display 138 includes a display device implemented using various known display technologies, such as LCD, LED, OLED, or plasma display technologies. In some embodiments, the display 138 can include multiple displays. The control panel 152 is configured to receive user input (e.g., test type, threshold confidence score). The control panel 152 includes one or more hard controls (e.g., buttons, knobs, dials, encoders, mouse, trackball, etc.). In some embodiments, the control panel 152 additionally or alternatively includes soft controls (e.g., GUI control elements, or simply GUI controls) provided on a touch-sensitive display. In some embodiments, the display 138 is a touch-sensitive display that includes one or more soft controls of the control panel 152.

[0045]

[0051] In some embodiments, the various components shown in FIG. 1 are combined. For example, image processor 136 and graphics processor 140 are implemented as a single processor. In another example, scan converter 130 and multiplanar reformatter 132 are implemented as a single processor. In some embodiments, the various components shown in FIG. 1 are implemented as separate components. For example, signal processor 126 is implemented as a separate signal processor for each imaging mode (e.g., B-mode, Doppler). In some embodiments, one or more of the various processors shown in FIG. 1 are implemented by a general-purpose processor and / or microprocessor configured to perform a designated task. In some embodiments, one or more of the various processors are implemented as application-specific circuits. In some embodiments, one or more of the various processors (e.g., image processor 136) are implemented by one or more graphical processing units (GPUs).

[0046]

[0052] Figure 2 is a block diagram illustrating an example processor 200 in accordance with the principles of the present disclosure. Processor 200 may be used to execute one or more of the processors and / or controllers described herein, such as image processor 136 shown in Figure 1 and / or other processors or controllers shown in Figure 1. Processor 200 may be any suitable processor type, including, but not limited to, a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA) (wherein the FPGA is programmed to form a processor), a graphical processing unit (GPU), an application specific circuit (ASIC) (wherein the ASIC is designed to form a processor), or a combination thereof.

[0047]

[0053] Processor 200 includes one or more cores 202. Core 202 includes one or more arithmetic logic units (ALUs) 204. In some embodiments, core 202 includes a floating point logic unit (FPLU) 206 and / or a digital signal processing unit (DSPU) 208 in addition to or instead of the ALUs 204.

[0048]

[0054] The processor 200 includes one or more registers 212 communicatively coupled to the core 202. The registers 212 are implemented using dedicated logic gate circuits (e.g., flip-flops) and / or any memory technology. In some embodiments, the registers 212 are implemented using static memory. The registers provide data, instructions, and addresses to the core 202.

[0049]

[0055] In some embodiments, processor 200 includes one or more levels of cache memory 210 communicatively coupled to cores 202. Cache memory 210 provides computer-readable instructions to cores 202 for execution. Cache memory 210 provides data for processing by cores 202. In some embodiments, computer-readable instructions are provided to cache memory 210 by a local memory, such as a local memory attached to external bus 216. Cache memory 210 may be implemented in any suitable cache memory type, such as static random access memory (SRAM), metal-oxide semiconductor (MOS) memory such as dynamic random access memory (DRAM), and / or other suitable memory technologies.

[0050]

[0056] The processor 200 includes a controller 214 that controls inputs to the processor 200 from other processors and / or components in the system (e.g., the control panel 152 and the scan converter 130 shown in FIG. 1) and / or outputs from the processor 200 to other processors and / or components in the system (e.g., the display 138 and the volume renderer 134 shown in FIG. 1). The controller 214 controls the data paths of the ALU 204, the FPLU 206, and / or the DSPU 208. The controller 214 is implemented as one or more state machines, data paths, and / or dedicated control logic. The gates of the controller 214 are implemented as stand-alone gates, FPGAs, ASICs, or other suitable technologies.

[0051]

[0057] Registers 212 and cache 210 communicate with controller 214 and core 202 via internal connections 220A, 220B, 220C, and 220D. The internal connections may be implemented as buses, multiplexers, crossbar switches, and / or other suitable connection technologies.

[0052]

[0058] Inputs and outputs of processor 200 are provided via a bus 216 that includes one or more conductive lines. Bus 216 is communicatively coupled to one or more components of processor 200, such as controller 214, cache 210, and / or registers 212. Bus 216 is coupled to one or more components of the system, such as display 138 and control panel 152, previously described.

[0053]

[0059] The bus 216 is coupled to one or more external memories. The external memory includes a read only memory (ROM) 232. The ROM 232 may be a masked ROM, an electronically programmable read only memory (EPROM), or other suitable technology. The external memory includes a random access memory (RAM) 233. The RAM 233 may be a static RAM, a battery backed up static RAM, a dynamic RAM (DRAM), or other suitable technology. The external memory includes an electrically erasable programmable read only memory (EEPROM) 235. The external memory includes a flash memory 234. The external memory includes a magnetic storage device such as a disk 236. In some embodiments, the external memory is included in a system such as the ultrasound imaging system 100 shown in FIG. 1, for example, in the local memory 142.

[0054]

[0060] In some embodiments, the system 100 is configured to execute a neural network included in the view recognition processor 170, including a CNN, to determine whether a particular view was acquired, which particular view was acquired, the physiological state of the particular view, and / or a confidence score. The neural network is trained with imaging data, such as image frames, in which one or more items of interest have been labeled as being present. The neural network is trained to recognize target anatomical features associated with a particular ultrasound exam (e.g., various standard views of the heart in echocardiography), or a user trains the neural network to locate one or more custom target anatomical features (e.g., an implanted device, a liver tumor).

[0055]

[0061] In some embodiments, the neural network training algorithm associated with the neural network is presented with thousands or even millions of training data sets to train the neural network to determine the confidence level of each measurement taken from a particular ultrasound image. In various embodiments, the number of ultrasound images used to train the neural network ranges from about 50,000 to 200,000 or more. The number of images used to train the network increases when a larger number of different items of interest are to be identified or correspond to a larger variety of patient variables, such as weight, height, age, etc. The number of training images varies for different items of interest or their features and depends on the variability of the occurrence of a particular feature. For example, tumors generally have a larger range of variability than normal anatomical structures. Training the network to assess the presence of items of interest associated with features with a large variability across the population requires a larger amount of training images.

[0056]

[0062] FIG. 3 illustrates a block diagram of a neural network training and deployment process according to the principles of the present disclosure. The process illustrated in FIG. 3 is used to train a neural network included in the view recognition processor 170. Phase 1 on the left side of FIG. 3 illustrates training of the neural network. To train the neural network, a training set including a large number of instances of input arrays and output classifications is presented to a neural network training algorithm (e.g., the AlexNet training algorithm as described by Krizhevsky, A., Sutskever, I., and Hinton, GE, "ImageNet Classification with Deep Convolutional Neural Networks," NIPS 2012 or derivatives thereof). Training includes the selection of a starting network architecture 312 and the preparation of training data 314. The starting network architecture 312 can be a blank architecture (e.g., an architecture with defined layers and node arrangements but no previously trained weights), or a partially trained network such as an Inception network, which is then further tuned for classification of ultrasound images. The starting architecture 312 (e.g., blank weights) and training data 314 are provided to a training engine 310 to train the model. Upon a sufficient number of iterations (e.g., when the model is consistently performing within an acceptable error), the model 320 is said to be trained and ready for deployment, which is shown in the center of FIG. 3, phase 2. On the right side of FIG. 3, i.e., phase 3, the trained model 320 is applied (via an inference engine 330) for analysis of new data 332. The new data 332 is data that was not presented to the model during the initial training (in phase 1). For example, the new data 332 includes unknown images, such as live ultrasound images acquired during a patient scan (e.g., cardiac images during an echocardiogram).The trained model 320 executed via the engine 330 is used to classify unknown images according to the training of the model 320, thereby providing an output 334 (e.g., a particular view, a physiological state, a confidence score), which is then used by the system for a subsequent process 340 (e.g., as an input to the optimization state controller 172 to determine imaging parameters).

[0057]

[0063] In embodiments in which the neural network is dynamically trained, the engine 330 undergoes field training 338. The engine 330 continues to train and change based on data acquired after the deployment of the engine 330. The field training 338 is based at least in part on new data 332 in some embodiments.

[0058]

[0064] In an embodiment in which the trained model 320 is used to implement the neural network of the view recognition processor 170, the starting architecture can be that of a convolutional neural network or a deep convolutional neural network, trained to perform image frame indexing, image segmentation, image comparison, or a combination thereof. With the increasing amount of stored medical image information, the availability of high quality clinical images increases, which can be exploited to train the neural network to learn the probability (e.g., confidence score) of a given image frame containing a given particular view. The training data 314 includes a large number (hundreds, often thousands, or even more) of annotated / labeled images, also referred to as training images. It will be appreciated that the training images need not include the complete image (e.g., representing the entire field of view of the probe) produced by the imaging system, but can include patches or portions of images of labeled items of interest.

[0059]

[0065] In various embodiments, the trained neural network is at least partially implemented on a computer-readable medium that includes executable instructions executed by a processor, such as the view recognition processor 170.

[0060]

[0066] FIG. 4 is a flow diagram of a method 400 of ultrasound imaging performed in accordance with the principles of the present disclosure. The processes in each of the blocks of method 400 are performed in real-time or live, i.e., during real-time or live imaging of a subject. In block 402, a step of "acquiring an ultrasound image" is performed. For example, an ultrasound image is acquired by ultrasound probe 112 of system 100 in some embodiments. The ultrasound image is analyzed to determine whether it contains a particular view, as shown in block 404. This analysis and determination is performed by view recognition processor 170 according to any of the examples herein. View recognition processor 170, in some embodiments, includes a neural network. In other embodiments, view recognition processor 170 uses other image processing techniques to identify whether a particular view is represented in the acquired image. Processing of the ultrasound image in block 404 (e.g., by view recognition processor 170) further includes determining a physiological state of an anatomical structure in the particular view and / or generating a confidence score for the determination of the particular view. Upon determining that the ultrasound image corresponds to a particular view, an output (e.g., a confirmation or indication of the particular view, a confidence score, etc.) is provided to downstream components of system 100, for example, by view recognition processor 170, as shown in block 406. The output is a signal generated by view recognition processor 170. If the ultrasound image does not correspond to a particular view, no output is generated by view recognition processor 170, or alternatively, a low output (e.g., less than 40% or less than 30%) is output by view recognition processor 170. In some embodiments, method 400 includes repeating blocks 402 and 404, as shown by dashed arrow 414, until an output is generated in block 406 and / or until a confidence score of at least 50%, or in some cases a confidence score of at least 65%, is output in block 404.

[0061]

[0067] The method 400 then proceeds to block 408 where a step of "determine one or more view-specific imaging parameters" is performed. The determining step is performed by the optimization state controller 172 in some embodiments. The one or more imaging parameters are based at least in part on an output from block 406 (e.g., an index for a particular view). In some embodiments, in the absence of an output from block 406 (e.g., an index signal), the ultrasound system begins or continues to generate images using default imaging parameters. The one or more default imaging parameters are based in some embodiments on the exam type.

[0062]

[0068] At block 410, a step of "providing one or more view-specific imaging parameters" is performed. The one or more imaging parameters, which may be default imaging parameters, in some embodiments, are provided to a controller, such as the transmit controller 120 and / or the beamformer 122. In some embodiments, the optimization state controller 172 waits for an indication to be provided for a period of time and / or waits for an indication that the ultrasound probe is stationary before providing the one or more imaging parameters. At block 412, a step of "acquiring an ultrasound image with one or more view-specific imaging parameters" is performed. The acquiring step is performed by the ultrasound probe 112 under the control of the transmit controller 120 and / or the beamformer 122.

[0063]

[0069] FIG. 5 is a flow diagram of a method 500 according to the principles of the present disclosure. In some embodiments, the method 500 is performed by the optimization state controller 172. In block 502, a "receive an output signal" step is performed. In some embodiments, the output signal is provided by the view recognition processor 170. The output signal, in some embodiments, provides an indication of a particular view, a physiological state of an anatomical structure in the particular view, and / or a confidence score. In block 504, a "consult a lookup table for a particular view" step is performed. The particular view is provided as an output signal or as part of the output signal. In block 506, a "retrieve imaging parameters for a particular view" step is performed. The retrieved imaging parameters are based on the lookup table. In some embodiments, the imaging parameters are retrieved from the local memory 142. In some embodiments, based on the determined particular view, one or more algorithms are retrieved (e.g., from the local memory). The one or more algorithms are adaptive and are used to provide different imaging parameters based at least in part on the particular view. For example, one or more algorithms may provide different amounts of gain, different radio frequency (RF) filters, and / or image processing parameters to enhance the lateral walls of the heart when an apical four-chamber view is detected.

[0064]

[0070] In block 508, a "Compare output signal to a threshold" step is performed. In some embodiments, the threshold corresponds to a confidence score threshold. In some embodiments, the threshold is a number or period of ultrasound image frames during which the output signal remains stable, e.g., a particular view represented by the output signal remains constant. In some embodiments, the threshold corresponds to a period during which the ultrasound probe remains stationary. In some embodiments, the threshold is a combination of factors and / or multiple thresholds corresponding to different factors are analyzed (e.g., confidence scores above a threshold at a given number of frames). If the output signal meets or exceeds one or more thresholds, in block 510, a "Provide retrieved imaging parameters" step is performed. If the output signal is below the threshold, in block 512, a "Provide existing imaging parameters" step is performed. Alternatively, in block 512, a "Provide default imaging parameters" step is performed. In some embodiments, the default parameters are defined by exam type (e.g., liver, fetal, heart). In some embodiments, block 512 is executed in parallel with blocks 502, 504, 506, and / or 508 until the output signal meets or exceeds the threshold value.

[0065]

[0071] In some embodiments, block 508 is performed before blocks 504 and 506. In these embodiments, the output signal must meet or exceed the threshold before blocks 504 and 506 are performed, and block 510 is performed after block 506. Additionally, in these embodiments, block 512 is performed in parallel with 502 and / or 508.

[0066]

[0072] The systems and methods described herein allow for automatically adjusting imaging parameters based on a particular view acquired by an ultrasound imaging system, thereby allowing each particular view to be acquired with imaging parameters that are optimized for that particular view. Acquiring each view with optimized imaging parameters can improve the quality of the acquired images without increasing the workload of the user.

[0067]

[0073] Although the examples described herein refer to review of a single current or previous ultrasound study, the principles of this disclosure may be applied to review of multiple studies. An exam may be a single-subject exam, e.g., when reviewing a patient for disease progression. An exam may be a multi-subject exam, e.g., when identifying items of interest across a population for medical research.

[0068]

[0074] In various embodiments in which the components, systems, and / or methods are implemented using a computer-based system or a programmable device such as programmable logic, it should be appreciated that the above-described systems and methods may be implemented using any of a variety of known or later developed programming languages, e.g., "C", "C++", "C#", "Java", "Python", and the like. Accordingly, various storage media, such as magnetic computer disks, optical disks, electronic memory, and the like, are provided that contain information that instructs a device, such as a computer, to execute the above-described systems and / or methods. When an appropriate device accesses the information and programs contained on the storage media, the storage media provides the information and programs to the device, thereby enabling the device to execute the functions of the systems and / or methods described herein. For example, when a computer disk containing appropriate material, such as source files, object files, executable files, and the like, is provided to a computer, the computer receives the information and appropriately configures itself to execute the functions of the various systems and methods outlined in the figures and flow charts above to achieve the various functions. That is, the computer receives various portions of information from the disk that relate to the various elements of the above-described systems and / or methods, executes the individual systems and / or methods, and coordinates the functions of the individual systems and / or methods described above.

[0069]

[0075] In view of this disclosure, it should be noted that the various methods and devices described herein may be implemented in hardware, software, and firmware. Furthermore, the various methods and parameters are included merely as examples and are not meant to be limiting. In view of this disclosure, one skilled in the art may implement the teachings in determining their own techniques and the necessary equipment to effect these techniques while remaining within the scope of the present invention. One or more functions of the processor described herein may be incorporated into a fewer or a single processing unit (e.g., a CPU), or may be implemented using an application specific integrated circuit (ASIC), or a general-purpose processing circuit that is programmed in response to executable instructions to perform the functions described herein.

[0070]

[0076] Although the present system has been described with particular reference to an ultrasound imaging system, it is envisioned that the present system may be extended to other medical imaging systems in which one or more images are systematically obtained. Thus, the present system may be used to acquire and / or record image information related to, but not limited to, the kidneys, testes, breasts, ovaries, uterus, thyroid, liver, lungs, muscles and bones, spleen, heart, arteries, and vasculature, as well as other imaging applications related to ultrasound-guided interventions. Furthermore, the present system also includes one or more programs that may be used in conventional imaging systems, such that conventional imaging systems may incorporate the features and advantages of the present system. Certain additional advantages and features of the present disclosure will be apparent to those skilled in the art upon reviewing the present disclosure, or will be experienced by those utilizing the novel systems and methods of the present disclosure. Another advantage of the present systems and methods is that conventional medical imaging systems are easily upgraded to incorporate the features and advantages of the present systems, devices, and methods.

[0071]

[0077] Of course, it should be recognized that any of the examples, embodiments, or processes described herein may be combined with one or more other examples, embodiments, and / or processes, or may be separated and / or performed among separate devices or device portions according to the present systems, devices, and methods.

[0072]

[0078] Finally, the foregoing discussion is intended to be merely illustrative of the present system, and should not be construed as limiting the appended claims to any particular embodiment or group of embodiments. Thus, while the present system has been described in particular detail with reference to exemplary embodiments, it should also be recognized that numerous variations and alternative embodiments may be devised by those skilled in the art without departing from the broader and intended spirit and scope of the present system as set forth in the following claims. Accordingly, the specification and drawings are to be regarded as illustrative, and are not intended to limit the scope of the appended claims.

Claims

1. an ultrasonic transducer array for acquiring ultrasonic images; a controller for controlling acquisition by the ultrasound transducer array based at least in part on one or more imaging parameters; a view recognition processor for determining whether the ultrasound image corresponds to a particular view; an optimization state controller for receiving an output of the view recognition processor and determining an update to the one or more imaging parameters based at least in part on the output when the view recognition processor determines that the ultrasound image corresponds to the particular view, the optimization state controller providing the updated one or more imaging parameters to the controller; 1. An ultrasound imaging system comprising: and wherein the optimization state controller provides the one or more updated imaging parameters to the controller when the ultrasound image is stable for a period of time, and the controller controls the ultrasound transducer array to reacquire an ultrasound image of the same particular view with the updated one or more imaging parameters.

2. The ultrasound imaging system of claim 1 , wherein the view recognition processor comprises a neural network.

3. The ultrasound imaging system of claim 1 , further comprising an image processor for processing the ultrasound image based at least in part on the updated one or more imaging parameters determined by the optimization state controller.

4. The ultrasound imaging system of claim 1 , wherein the view recognition processor further determines a physiological condition of the particular view, the physiological condition being provided by the output.

5. The ultrasound imaging system of claim 4 , wherein the physiological condition is a phase of the cardiac cycle.

6. 2. The ultrasound imaging system of claim 1, wherein the view recognition processor further determines a confidence score for the particular view, and the optimization state controller provides updated the one or more imaging parameters to the controller if the confidence score is above a threshold.

7. The ultrasound imaging system of claim 6 , further comprising a user interface for receiving user input, said user input including said threshold value.

8. The ultrasound imaging system of claim 1 , wherein the updated one or more imaging parameters vary across a region of the ultrasound image.

9. The ultrasound imaging system of claim 1 , further comprising a memory, wherein the updated one or more imaging parameters are retrieved from the memory by the optimization state controller.

10. acquiring an ultrasound image; determining whether the ultrasound image includes a particular view; if the particular view has been determined, providing an output based on the particular view; determining one or more imaging parameters based at least in part on the output; providing the one or more imaging parameters to a controller when the ultrasound image is stable for a period of time; reacquiring an ultrasound image of the same particular view with the one or more imaging parameters; The method comprising:

11. The method of claim 10 , further comprising determining a physiological state of the particular view.

12. The method of claim 10 , further comprising waiting a period of time for the output to be provided before determining the one or more imaging parameters.

13. The method of claim 10 , further comprising determining one or more default imaging parameters if the output is not provided, and providing the one or more default parameters to the controller.

14. determining a confidence score for the particular view; The method of claim 10, further comprising determining one or more imaging parameters based at least in part on the output if the confidence score is above a threshold, and determining one or more default imaging parameters if the confidence score is below the threshold, and providing the one or more default imaging parameters to the controller.

15. A non-transitory computer readable medium containing instructions that, when executed, cause an ultrasound imaging system to: Acquire an ultrasound image; determining whether the ultrasound image includes a particular view; If the particular view is determined, providing an output based on the particular view; determining one or more imaging parameters based at least in part on the output; providing the one or more imaging parameters to a controller when the ultrasound image is stable for a period of time; A non-transitory computer readable medium for reacquiring an ultrasound image of the same particular view with the one or more imaging parameters.

Images