Fat layer identification using ultrasound imaging

Abstract

The present disclosure describes an imaging system that is configured to identify features within an image frame and improve the frame by implementing image quality adjustments. The ultrasound imaging system can include a transducer that is configured to acquire echo signals in response to an ultrasonic pulse transmitted toward a target. The system can also include a user interface configured to display an image and one or more processors configured to identify one or more features within the image. The processor can cause the interface to display elements associated with at least two image quality operations specific to the identified features. The first image quality operation can include a manual adjustment of a transducer setting, and the second image quality operation can include an automatic adjustment of the identified feature derived from a reference frame that includes the identified feature. The processor can receive a user selection of one or more elements and apply the operation to modify the image.

Description

Technical Field

[0001] The present disclosure relates to ultrasound systems and methods for identifying features such as fat layers via ultrasound imaging and modifying images based on the identified features. Detailed embodiments relate to systems configured to identify and remove fat layers and associated image artifacts from ultrasound images, thereby improving image quality. Background Art

[0002] Ultrasound imaging can be challenging when scanning patients with moderate to thick fat layers, particularly in the abdominal region. Fat causes increased sound attenuation, and because sound waves typically travel at different speeds through fat tissue relative to other soft tissues, ultrasound beams propagating through fat tissue often become defocused, an effect known as phase shift. More specifically, ultrasound beam focusing is achieved by applying a specific delay to each transducer element based on the time-of-flight of the acoustic pulse—for example, the length of time it takes for an ultrasound echo signal to travel from a specific anatomical point to each transducer receiving element, or the length of time it takes for a transmitted ultrasound signal to reach certain anatomical points from the transducer. Beamforming based on incorrect assumptions about the speed of sound can lead to incorrect distance-time relationship calculations, for example, generating a defocused ultrasound beam characterized by a wide main beam and large sidelobes. This defocusing can occur frequently when imaging abdominal fat layers, where sound waves typically travel at only approximately 1450 m / s, much slower than most surrounding tissue, where sound waves typically travel at approximately 1540 m / s. As a result, images of areas containing fat layers, such as the abdomen, often contain unwanted artifacts and are generally of poor quality. Existing techniques designed to correct or minimize image artifacts caused by fat are complex to implement and often ineffective. Therefore, new ultrasound systems are needed that can reduce or eliminate image degradation caused by fat layers to improve imaging and assessment of anatomical targets beneath the fat layer. Summary of the Invention

[0003] The present disclosure describes ultrasound systems and methods for identifying and locating at least one feature (e.g., a fat layer) within an ultrasound image. In some examples, the feature can be identified by implementing a neural network. Various measurements of the feature, such as the thickness of the identified fat layer, can also be automatically or manually determined, and an indication of the various measurements of the feature can be displayed on a graphical user interface. The system can generate an annotated ultrasound image in which the feature (e.g., the fat layer) is marked or highlighted for further evaluation, thereby alerting the user to the feature and any associated deviations. The system can also generate and display at least one recommended manual adjustment to a transducer setting based on the identified feature, such that implementation of the adjustment can remove deviations or image artifacts caused by the feature. In some embodiments, a second neural network can also be implemented that is trained to automatically remove or modify the identified feature from the ultrasound image. By removing the feature from the specific image, the second neural network can generate a modified image that lacks the feature and the associated image artifacts caused by the feature. The modified image, having enhanced quality relative to the original image, can then be displayed for analysis. The disclosed systems and methods are applicable to a wide range of imaging protocols, but may be particularly advantageous when scanning anatomical regions with high fat content, such as the abdominal region, where image degradation due to fat-induced artifacts may be most severe. Although example systems and methods are described with respect to fat layer identification and associated image modification, it should be understood that the present disclosure is not limited to fat layer application, and that a variety of anatomical features and / or image artifacts can be identified, modified, and / or removed according to the principles disclosed herein.

[0004] According to some examples of the present disclosure, an ultrasound imaging system may include: an ultrasound transducer configured to acquire echo signals in response to ultrasound pulses emitted toward a target area, and a graphical user interface configured to display an ultrasound image from at least one image frame generated based on the ultrasound echoes. The system may also include one or more processors in communication with the ultrasound transducer and the graphical user interface. The processor may be configured to identify one or more features within the image frame and cause the graphical user interface to display elements associated with at least two image quality operations specific to the identified features. The first image quality operation may include manual adjustment of a transducer setting, and the second image quality operation may include automatic adjustment of the identified features derived from a reference frame including the identified features. The processor may also be configured to receive a user selection of at least one of the elements displayed by the graphical user interface and apply the image quality operation corresponding to the user selection to modify the image frame.

[0005] In some examples, the second image quality operation can be dependent on the first image quality operation. In some embodiments, one or more features can be identified by inputting an image frame into a first neural network trained using imaging data including reference features. In some examples, the one or more features can include a fat layer. In some embodiments, a graphical user interface can be configured to display an annotated image frame with the one or more features labeled therein. In some examples, the first neural network can include a convolutional network defined by a U-net or V-net architecture, further configured to delineate visceral fat and subcutaneous fat layers within the image frame. In some embodiments, the processor can be configured to modify the image frame by inputting it into a second neural network trained to output a modified image frame omitting the identified features for display on the graphical user interface. In some examples, the second neural network can include a generative adversarial network. In some embodiments, the processor can also be configured to remove noise from the image frame before identifying the one or more features. In some examples, the one or more processors can also be configured to determine the size of the fat layer. In some embodiments, the size can include the thickness of the fat layer at a location within the fat layer specified by a user via the graphical user interface. In some examples, the target area can include the abdominal area.

[0006] According to some examples of the present disclosure, a method of ultrasound imaging can involve: acquiring echo signals in response to ultrasound pulses emitted toward a target area; displaying an ultrasound image from at least one image frame generated based on the ultrasound echoes; identifying one or more features within the image frame; and displaying elements associated with at least two image quality operations specific to the identified features. The first image quality operation can include manual adjustment of a transducer setting, and the second image quality operation can include automatic adjustment of the identified features derived from a reference frame that includes the identified features. The method can also involve: receiving a user selection of at least one of the displayed elements; and applying the image quality operation corresponding to the user selection to modify the image frame.

[0007] In some examples, the second image quality operation can be dependent on the first image quality operation. In some embodiments, the one or more features can be identified by inputting the image frame into a first neural network trained using imaging data including reference features. In some examples, the one or more features can include a fat layer. In some embodiments, the method can further involve displaying the annotated image frame in which the one or more features are labeled. In some examples, the image frame can be modified by inputting the image frame into a second neural network trained to output a modified image frame in which the identified features are omitted. In some embodiments, the method can further involve determining the size of the one or more features at an anatomical location specified by a user.

[0008] Any method described herein, or steps thereof, may be embodied in a non-transitory computer-readable medium comprising executable instructions that, when executed, cause a processor of a medical imaging system to perform the method or steps embodied herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 Schematic diagram of a cross-section of layered muscle, fat, and skin tissue and their corresponding ultrasound images.

[0010] Figure 2 is a block diagram of an ultrasound system according to the principles of the present disclosure.

[0011] Figure 3A is a block diagram of a U-net convolutional network configured for adipose tissue segmentation according to principles of the present disclosure.

[0012] Figure 3B is a block diagram of a V-net convolutional network configured for adipose tissue segmentation according to the principles of the present disclosure.

[0013] Figure 4 is a graphical user interface implemented according to the principles of the present disclosure.

[0014] Figure 5 is a block diagram of a neural network configured for fat layer image removal according to principles of the present disclosure.

[0015] Figure 6 is a block diagram of a coordinated neural network configured to identify and remove fat layers from ultrasound images in accordance with the principles of the present disclosure.

[0016] Figure 7 is a flow chart of a method of ultrasound imaging performed according to the principles of the present disclosure. DETAILED DESCRIPTION

[0017] The following description of certain embodiments is merely exemplary in nature and is in no way intended to limit the invention or its application or use. In the following detailed description of the embodiments of the present system and method, reference is made to the accompanying drawings that form a part thereof, and these drawings illustrate, by way of illustration, specific embodiments in which the described systems and methods may be practiced. These embodiments are described in sufficient detail to enable one of ordinary skill in the art to practice the presently disclosed systems and methods, and it will be understood that other embodiments may be utilized and that structural and logical changes may be made without departing from the spirit and scope of the present system. Moreover, for the purpose of clarity, when detailed descriptions of certain features are obvious to one of ordinary skill in the art, they will not be discussed so as not to obscure the description of the present system. Therefore, the following detailed description should not be taken in a limiting sense, and the scope of the present system is limited solely by the appended claims.

[0018] The system disclosed herein can be configured to implement a deep learning model to identify at least one fat layer present in a target region. The fat layer can be indicated on a user interface, and recommended solutions for eliminating or reducing image degradation caused by fat can be generated and optionally displayed. Embodiments also include a system configured to improve ultrasound images by employing a deep learning model trained to remove fat layers and associated image artifacts from images and generate new images lacking such features. The disclosed system can improve B-mode image quality, particularly when imaging high-fat areas, such as the abdominal region. The system is not limited to B-mode imaging or abdominal imaging and can be applied to imaging various anatomical features, such as the liver, lungs, and / or various limbs, as the system can be used to correct images containing fat at any anatomical location on a patient. In addition to or in lieu of B-mode imaging, the system can also be used with various quantitative imaging modalities to improve their accuracy and / or effectiveness. For example, the disclosed system can be implemented for shear wave elastography optimization, beam pattern adjustment for acoustic attenuation, and / or backscatter coefficient estimation.

[0019] Ultrasound systems according to the present disclosure can utilize various neural networks, such as deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), autoencoder neural networks, and the like, to identify fat layers and optionally remove them from newly generated images. In various examples, a first neural network can be trained using any of a variety of currently known or later developed learning techniques to obtain a neural network (e.g., a trained algorithm or a hardware-based node system) configured to analyze input data in the form of ultrasound image frames and determine the presence of at least one fat layer therein. A second neural network can be trained to modify and remove input data in the form of ultrasound image frames, or data containing or reflecting a fat layer. Image artifacts resulting from fat-induced phase shifts can also be selectively removed by the second neural network. Without the fat layer and associated artifacts, image quality is significantly enhanced, as evidenced by improved clarity and / or contrast.

[0020] An ultrasound system according to the principles of the present invention may include or be operatively coupled to an ultrasound transducer configured to transmit ultrasound pulses into a medium, such as a human body or a specific portion thereof, and to generate echo signals in response to the ultrasound pulses. The ultrasound system may include a beamformer configured to perform transmit and / or receive beamforming, and in some examples, a display configured to display ultrasound images generated by the ultrasound imaging system. The ultrasound imaging system may include one or more processors and at least one neural network, which may be implemented in hardware and / or software components. Embodiments may include two or more neural networks that may be communicatively coupled or integrated into a multi-layer network such that the output of a first network serves as an input to a second network.

[0021] Neural networks implemented according to the present disclosure can be hardware-based (e.g., neurons represented by physical components) or software-based (e.g., neurons and pathways implemented in a software application) and can utilize various topologies and learning algorithms for training neural networks to produce desired outputs. For example, a software-based neural network can be implemented using a processor configured to execute instructions that can be stored on a computer-readable medium (e.g., a single-core or multi-core CPU, a single GPU or GPU cluster, or multiple processors arranged for parallel processing), and when executed, the instructions cause the processor to execute a trained algorithm to identify fat layers present within an ultrasound image and / or generate new images lacking the identified fat layers. An ultrasound system can include a display or graphics processor operable to arrange the ultrasound image and / or additional graphical information in a display window for display on a user interface of the ultrasound system. The additional graphical information can include annotations, confidence levels, user instructions, tissue information, patient information, indicators, and other graphical components. In some embodiments, ultrasound images and associated measurements can be provided to a storage and / or memory device, such as a picture archiving and communication system (PACS), for reporting purposes or future training (e.g., to continue enhancing the performance of the neural network), particularly corrected images generated by a system configured to remove fat layers and associated artifacts from fat-labeled images.

[0022] Figure 1 A cross-sectional representation of normal tissue 102a is shown, comprising an outer layer of skin 104a, a fat layer 106a, and a muscle layer 108a. Ultrasound imaging of the tissue can produce corresponding images 102b of the skin layer 104b, the fat layer 106b, and the muscle layer 108b. As shown, each layer can be rendered differently on the ultrasound image 102b, with the muscle layer 108b appearing brighter than the fat layer 106b. Existing techniques require the user to manually identify and measure the fat layer 106b, and such techniques fail to remove the fat layer and associated artifacts from the image. The system herein can automatically identify one or more fat layers and, in some examples, process the corresponding image to improve image quality despite the presence of such fat layers. Specifically, the system herein may not be limited to identifying fat layers and can be configured to identify any form of fat, such as localized deposits, pockets, or accumulations of various shapes. The example system can also be configured to depict visceral fat and subcutaneous fat. Subcutaneous fat can include the area along the xiphoid-umbilical line approximately one centimeter above the umbilicus. The thickness of the subcutaneous fat layer can be measured during exhalation as the distance between the skin-fat interface and the outer edge of the linea alba. Visceral fat can be measured as the distance between the linea alba and the anterior aorta along the xiphoid-umbilical line approximately one centimeter above the umbilicus.

[0023] Figure 2 An example ultrasound system according to the principles of the present disclosure is shown. The ultrasound system 200 can include an ultrasound data acquisition unit 210. The ultrasound data acquisition unit 210 can include an ultrasound probe including an ultrasound transducer array 212 configured to transmit ultrasound pulses 214 to a target region 216 of a subject, which can include an abdominal region, a thoracic region, one or more limbs, and / or features thereof, and receive ultrasound echoes 218 in response to the transmitted pulses. The region 216 can include a fat layer 217 having a variable thickness. For example, the fat layer can range in thickness from approximately 0.1 to approximately 20 cm, approximately 1 to approximately 12 cm, approximately 2 to approximately 6 cm, or approximately 4 to approximately 5 cm. As further shown, the ultrasound data acquisition unit 210 can include a beamformer 220 and a signal processor 222, which can be configured to generate a stream of discrete ultrasound image frames 224 based on the ultrasound echoes 218 received at the array 212. Image frames 224 can be communicated to a data processor 226 , such as a computing module or circuit, which in some examples may include a pre-processing module 228 and may be configured to implement at least one neural network, such as neural network 230 , trained to identify fat layers within image frames 224 .

[0024] The ultrasound sensor array 212 may include at least one transducer array configured to transmit and receive ultrasonic energy. The configuration of the ultrasound sensor array 212 can be preset for performing a specific scan and can be adjustable during the scan. Various transducer arrays can be used, such as linear arrays, convex arrays, or phased arrays. The number and arrangement of transducer elements included in the sensor array 212 can vary in different examples. For example, the ultrasound sensor array 212 can include a 1D or 2D array of transducer elements, corresponding to a linear array probe and a matrix array probe, respectively. A 2D matrix array can be configured to electronically scan in elevation and azimuth dimensions (via phased array beamforming) for 2D or 3D imaging. In addition to B-mode imaging, imaging modalities implemented according to the disclosure herein can also include, for example, shear wave and / or Doppler. Various users can process and operate the ultrasound data acquisition unit 210 to perform the methods described herein.

[0025] The beamformer 220 coupled to the ultrasound sensor array 212 can include a microwave beamformer or a combination of a microwave beamformer and a main beamformer. The beamformer 220 can control the transmission of ultrasound energy, for example, by forming ultrasound pulses into a focused beam. The beamformer 220 can also be configured to control the reception of ultrasound signals so that discernible image data can be generated and processed with the aid of other system components. The role of the beamformer 220 can vary among different ultrasound probe types. In some embodiments, the beamformer 220 can include two separate beamformers: a transmit beamformer configured to receive and process a pulse sequence of ultrasound energy for transmission into an object; and a separate receive beamformer configured to amplify, delay, and / or sum received ultrasound echo signals. In some embodiments, the beamformer 220 may include a microbeamformer operating on a set of transducer elements for both transmit and receive beamforming, the microbeamformer being coupled to a main beamformer operating on a set of input and output elements for both transmit and receive beamforming, respectively.

[0026] The signal processor 222 may be communicatively, operatively, and / or physically coupled to the sensor array 212 and / or the beamformer 220. Figure 2 In the example shown in FIG, the signal processor 222 is included as an integral component of the data acquisition unit 210, but in other examples, the signal processor 222 can be a separate component. In some examples, the signal processor can be housed with the sensor array 212, or can be physically separate from the sensor array 212 but communicatively coupled thereto (e.g., via a wired or wireless connection). The signal processor 222 can be configured to receive unfiltered and unstructured ultrasound data representing ultrasound echoes 218 received at the sensor array 212. Based on this data, the signal processor 222 can generate ultrasound image frames 224 as the user scans the target area 216. In some embodiments, the ultrasound data received and processed by the data acquisition unit 210 can be utilized by one or more components of the system 200 before ultrasound image frames are generated therefrom. For example, as shown by the dashed lines and described further below, the ultrasound data can be communicated directly to the first neural network 230 or the second neural network 242, respectively, for processing prior to generating and / or displaying the ultrasound image frames.

[0027] The pre-processing module 228 can be configured to remove noise from the image frames 224 received at the data processor 226, thereby improving the signal-to-noise ratio of the image frames. In some examples, the noise reduction method employed by the pre-processing module 228 can vary and can include block matching with 3D filtering. By improving the signal-to-noise ratio of the ultrasound image frames, the pre-processing module 228 can improve the accuracy and effectiveness of the neural network 230 when processing the frames.

[0028] In certain embodiments, neural network 230 may comprise a deep learning segmentation network configured to detect and optionally measure one or more fat layers based on one or more unique features of fat detected in ultrasound image frames 224 or image data acquired by data acquisition unit 210. In some examples, network 230 can be configured to identify and segment fat layers present within an image frame and automatically determine the dimensions of the identified layers, such as thickness, length, and / or width, at various user-specified locations. Layers can be masked or labeled on the processed image. In some examples, different configurations of neural network 230 can segment fat layers present in 2D or 3D images. The specific network architecture can include a cascade of contracting and dilating convolutional and max pooling layers. Training neural network 230 can involve inputting a large number of images containing annotated fat layers and images lacking fat layers, such that, over time, the network learns to identify fat layers in non-annotated images in real time during ultrasound scanning.

[0029] The detected fat layer can be reported to the user via a display processor 232 coupled to a graphical user interface 234. The display processor 232 can be configured to generate an ultrasound image 235 based on the image frames 224, which can then be displayed in real time on the user interface 234 while the ultrasound scan is being performed. The user interface 234 can be configured to receive user input 236 at any time before, during, or after the ultrasound procedure. In addition to the displayed ultrasound image 235, the user interface can be configured to generate one or more additional outputs 238, which can include a variety of graphics displayed simultaneously with (e.g., overlaid on) the ultrasound image 235. Such graphics can mark certain anatomical features and measurements identified by the system, such as the presence and size of at least one fat layer (e.g., visceral and / or subcutaneous), along with various organs, bones, tissues, and / or tissue interfaces. In some examples, the fat layer can be highlighted by outlining the fat and / or color-coding the fat areas. Fat thickness can also be calculated by determining the maximum, minimum, and / or average vertical thickness of the masked fat region output from the segmentation network 230. In some embodiments, the output 238 can include selectable elements associated with image quality operations to improve the quality of a particular image 235. The image quality operations can include instructions for manually adjusting transducer settings in a manner that improves the image 235 by eliminating, reducing, or minimizing one or more image artifacts or deviations caused by the fat layer, such as adjusting an analog gain curve, applying a preload to compress a detected fat layer, and / or turning on harmonic imaging mode. The output 238 can include additional user-selectable elements and / or alerts to implement another image quality operation, which can be dependent on the first image operation and embody automatic adjustment of an identified feature (e.g., a fat layer) within the image 235 in a manner that eliminates, reduces, or minimizes the feature and / or any associated artifacts or deviations, as described further below. The graphical user interface 234 can then receive user input 236 to implement at least one of the quality operations, which can prompt the data processor 226 to modify the image frame 224 containing the feature. In some examples, user interface 234 can also receive image quality enhancement instructions that differ from instructions (e.g., instructions based on user knowledge and experience) embodied in output 238. Output 238 can also include annotations, confidence levels, user instructions, tissue information, patient information, indicators, user notifications, and other graphical components.

[0030] In some examples, the user interface 234 can be configured to receive user instructions 240 specific to automatic image quality operations. The user instructions 240 can be responsive to selectable alerts displayed on the user interface 234 or simply input by the user. According to such an example, the user interface 234 can prompt the data processor 226 to automatically generate an improved image based on the determined presence of a fat layer by implementing a second neural network 242 configured to remove the fat layer from the ultrasound image, thereby generating an improved image 244 lacking one or more fat layers and / or image artifacts associated therewith. Figure 2 As shown, the second neural network 242 can be communicatively coupled to the first neural network 230 such that the output of the first neural network (e.g., an annotated ultrasound image in which fat has been identified) can be directly input into the second neural network 242. In some examples, the second neural network 242 can include a Laplacian pyramid of adversarial networks configured to generate images in a coarse-to-fine manner using a cascade of convolutional networks. Large-scale adjustments made to an input image containing at least one fat layer can be minimized to preserve the most significant image features while maximizing fine changes specific to the identified fat layer and associated image artifacts. The input received by the second neural network 242 can include an ultrasound image containing a fat layer, or image data embodying a fat layer that has not yet been processed into a complete image. According to the latter example, the second neural network 242 can be configured to correct the image signal, for example, by removing the fat layer and associated artifacts from the image signal in the channel domain of the ultrasound data acquisition unit 210. The architecture and operating mode of the second neural network 242 can vary, as described below in conjunction with Figure 5 described.

[0031] Figure 2 The configuration of the components shown in the figure can be varied. For example, the system 200 can be portable or fixed. Various portable devices (e.g., laptops, tablets, smartphones, etc.) can be used to implement one or more functions of the system 200. In examples including such devices, the ultrasound sensor array can be connectable via, for example, a USB interface. In some examples, Figure 2 The various components shown in FIG. 2 can be combined. For example, neural network 230 can be merged with neural network 242. According to such an embodiment, the two networks can constitute subcomponents of a larger hierarchical network, for example.

[0032] The specific architecture of the network 230 can vary. In an example, the network 230 can include a convolutional neural network. In a specific example, the network 230 can include a convolutional autoencoder with skip connections from the encoder layer to the decoder layer at the same architectural network level. For 2D ultrasound images, a U-net architecture 302a can be implemented in a specific embodiment, such as Figure 3A As shown in the example of . The U-net architecture 302a includes a contracting path 304a and an expanding path 306a. In one embodiment, the contracting path 304a can include a cascade of repeated 3×3 convolutions followed by rectified linear units and a 2×2 max pooling operation with downsampling at each step, for example, as described in Ronneberger, O et al., “U-Net: Convolutional Networks for Biomedical Image Segmentation,” Medical Image Computing and Computer Assisted Intervention Society, conditionally accepted, published November 18, 2015 (“Ronneberger”). The expanding path 306a can include consecutive steps of upconvolution, each step halving the number of feature channels, as described by Ronneberger. The output 308a can include a segmentation map identifying one or more fat layers present within the initial image frame 224. In some embodiments, the fat layer or the surrounding non-fat area can be masked, and in some examples, the output 308a can depict the non-fat area, subcutaneous fat layer, and / or visceral fat layer using separate masks implemented for each tissue type. Training the network can involve inputting ultrasound images containing one or more fat layers and corresponding segmentation maps until the network learns to reliably identify the presence of fat layers in new images. As described by Ronneberger, data augmentation measures can also be implemented to train the network when a small number of training images are available.

[0033] For 3D ultrasound images, a convolutional V-net architecture 302b may be implemented in certain embodiments, such as Figure 3BAs shown in the example of . V-net architecture 302b can include a compression path 304b followed by a decompression path 306b. In one embodiment, each stage of compression path 304b can operate at a different resolution and can include one to three convolutional layers performing convolutions on voxels of different sizes, for example, as described in Milletari, F et al., “V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation,” 3D Vision (3DV), 2016 Fourth International Conference on 3D Vision, pp. 565-571, published October 25, 2016 (“Milletari”). In some examples, as further described by Milletari, each stage can be configured to learn a residual function that can converge in less time than existing network architectures. Output 308b can include a 3D segmentation map identifying one or more fat layers present in the initial image frame 224, the 3D segmentation map can include delineation of non-fat, visceral fat, and / or subcutaneous fat. Training the network can involve end-to-end training by inputting a three-dimensional image comprising one or more fat layers and corresponding annotated images in which the fat layers are identified. As described by Milletari, data augmentation measures can be implemented to train the network when a limited number of training images (especially annotated images) are available.

[0034] Figure 4An example of a graphical user interface 400 configured in accordance with the present disclosure is shown. As shown, interface 400 can be configured to show an ultrasound image 435 of a target region 416 containing at least one fat layer 417, the boundaries of which are represented by lines 417a and 417b. As further shown, the thickness of fat layer 417 is measured as 14 mm at a location that can be specified by the user, for example, by directly interacting with image 435 on a touch screen. Various example outputs are also shown, including a fat layer detection notification 438a, an "Auto Correct" button 438b, and recommended instructions 438c for improving the quality of image 435 by adjusting system parameters. Fat layer detection notification 438a includes an indication of the average thickness of fat layer 417, which in this particular example is 16 mm. By selecting "Auto Correct" button 438b, the user can initiate the automatic removal of fat layer 417 from the image via neural network generation of a corrected image that retains all features of image 435 except the fat layer and any associated artifacts. Signal attenuation may also be reduced in the corrected image. Recommended instructions 438c include instructions for starting harmonic imaging, applying more preload, and adjusting the analog gain curve. Instructions 438c can vary depending on the thickness and / or location of the fat layer detected in a given image and / or the extent to which the fat layer causes image artifacts to appear and / or generally reduces image quality. For example, instructions 438c can include recommended modifications to the position and / or orientation of the ultrasound probe used to acquire the image. In some embodiments, the user interface 400 can display the corrected image and a selectable option to revert to the original image, such as an "undo correction" button. According to such an example, the user can switch back and forth between an image containing the annotated fat layer and a new corrected image lacking the fat layer.

[0035] Figure 5An example neural network 500 is shown, configured to remove one or more fat layers and associated artifacts from an ultrasound image and generate a new, corrected image lacking these features. This specific example includes a generative adversarial network (GAN), but various network types can also be implemented. GAN 500 includes a generative network 502 and a competitive discriminative network 504, such as described in Reed, S. et al., "Generative Adversarial Text to Image Synthesis," Proceedings of the 33rd International Conference on Machine Learning, New York, NY (2016) JMLR: W&CP Vol. 48. In operation, generative network 502 can be configured to generate synthetic ultrasound image samples 506 lacking one or more fat layers and associated artifacts in a feedforward manner based on input 508 consisting of text-labeled images annotated with identified fat layers. Discriminative network 504 can be configured to determine the likelihood of samples 506 generated by generative network 502 being real or fake based in part on a plurality of training images containing fat layers and lacking fat layers. After training, generative network 502 can learn to generate images lacking one or more fat layers based on input images containing one or more fat layers, such that the modified fat-free images are substantially indistinguishable from actual ultrasound images containing fat and associated artifacts. In some examples, training network 500 can involve inputting pairs of controlled experimental images of phantom tissue with and without a fat layer near the surface. To generate a large number of sample images in a consistent manner, such that each image has the same field of view within the phantom tissue, various robotic components and / or motorized stages can be utilized, for example.

[0036] Figure 6 A coordinated system 600 of convolutional networks configured to identify and remove at least one fat layer from an original ultrasound image, in accordance with the principles of the present disclosure, is shown. An initial ultrasound image 602 can be input to a first convolutional network 604, which can be configured to segment and annotate a fat layer 606 present in the initial image, thereby generating an annotated image 608. The annotated image 608 can be input to a convolutional generator network 610, which is communicatively coupled to a convolutional discriminator network 612. As shown, the convolutional generator network 610 can be configured to generate a modified image 614 lacking the fat layer 606 identified and labeled by the first convolutional network 604. Due to the absence of the fat layer 606 and the resulting image degradation, multiple anatomical features 616 are more distinct in the modified image 614. The organization of networks 604, 610, and 612 can vary in various embodiments. In various examples, one or more of images 602 , 608 , and / or 614 can be displayed on a graphical user interface for user analysis.

[0037] Figure 7 is a flow chart of an ultrasound imaging method performed in accordance with the principles of the present disclosure. Example method 700 illustrates steps utilized by the systems and / or devices described herein, in any order, to identify and optionally remove one or more fat layers from an ultrasound image, for example, during an abdominal scan. Method 700 can be performed by an ultrasound imaging system such as system 100 or other systems including, for example, mobile systems such as LUMIFY by Koninklijke Philips NV ("Philips"). Additional example systems may include SPARQ and / or EPIQ, also produced by Philips.

[0038] In the illustrated embodiment, the method 700 begins at block 702 by acquiring echo signals in response to an ultrasound pulse transmitted toward a target region.

[0039] The method continues at block 704 by displaying an ultrasound image from at least one image frame generated from the ultrasound echoes.

[0040] The method continues at block 706 by identifying one or more features within the image frame.

[0041] The method continues at block 708 by “displaying elements associated with at least two image quality operations specific to the identified feature, wherein the first image quality operation comprises a manual adjustment of a transducer setting and the second image quality operation comprises an automatic adjustment of the identified feature derived from a reference frame containing the identified feature.”

[0042] The method continues at block 710 by receiving a user selection of at least one of the displayed elements.

[0043] The method continues at block 712 by applying an image quality operation corresponding to the user selection to modify the image frame.

[0044] In various embodiments, components, systems, and / or methods are implemented using programmable devices, such as computer-based systems or programmable logic. It should be understood that the systems and methods described above can be implemented using any of a variety of known or later developed programming languages, such as "C," "C++," "FORTRAN," "Pascal," "VHDL," and the like. Accordingly, various storage media, such as magnetic computer disks, optical disks, electronic memory, and the like, can be prepared that can contain information capable of directing a device, such as a computer, to implement the systems and / or methods described above. Once an appropriate device accesses the information and programs contained on the storage media, the storage media can provide the information and programs to the device, thereby enabling the device to perform the functions of the systems and / or methods described herein. For example, if a computer disk containing appropriate material, such as source files, object files, executable files, and the like, is provided to a computer, the computer can receive the information, configure itself appropriately, and perform the functions of the various systems and methods outlined in the figures and flow charts above to perform the various functions. That is, the computer can receive various portions of information related to the different elements of the systems and / or methods described above from the disk, implement the various systems and / or methods, and coordinate the functions of the various systems and / or methods described above.

[0045] In view of the present disclosure, it should be noted that the various methods and devices described herein can be implemented in hardware, software, and firmware. In addition, the various methods and parameters are included only by way of example and without any limiting meaning. In view of the present disclosure, those of ordinary skill in the art can implement the present teachings while determining their own techniques and the necessary equipment to implement these techniques, while still within the scope of the present invention. The functions of one or more processors described herein can be combined into a smaller number or a single processing unit (e.g., a CPU) and can be implemented using an application-specific integrated circuit (ASIC) or a general-purpose processing circuit that is programmed in response to executable instructions to perform the functions described herein.

[0046] Although the present system may have been described with specific reference to an ultrasound imaging system, it is also contemplated that the present system can be extended to other medical imaging systems that obtain one or more images in a systematic manner. Thus, the present system can be used to obtain and / or record image information about, but not limited to, the kidneys, testes, breasts, ovaries, uterus, thyroid, liver, lungs, musculoskeletal, spleen, heart, arterial and vascular systems, as well as other imaging applications related to ultrasound-guided interventions. In addition, the present system may also include one or more programs that can be used with conventional imaging systems so that they can provide the features and advantages of the present system. Certain additional advantages and features of the present disclosure may be apparent to those skilled in the art from studying the present disclosure, or may be experienced by those employing the novel systems and methods of the present disclosure. Another advantage of the present systems and methods may be that conventional medical imaging systems can be easily upgraded to incorporate the features and advantages of the present systems, devices and methods.

[0047] Of course, it should be understood that any of the examples, embodiments, or processes described herein may be combined with one or more other examples, embodiments, and / or processes, or may be separated and / or performed in separate devices or device portions in accordance with the present systems, devices, and methods.

[0048] Finally, the above discussion is merely illustrative of the present system and should not be construed as limiting the appended claims to any specific embodiment or group of embodiments. Therefore, although the present system has been described in detail with reference to exemplary embodiments, it should be understood that numerous modifications and alternative embodiments may be devised by those skilled in the art without departing from the broader and intended spirit and scope of the present system as set forth in the appended claims. Accordingly, the specification and drawings should be regarded in an illustrative manner and not as limiting the scope of the appended claims.

Claims

1. An ultrasound imaging system, comprising: an ultrasonic transducer configured to acquire echo signals in response to ultrasonic pulses transmitted toward a target area; a graphical user interface configured to display an ultrasound image from at least one image frame generated from the ultrasound echoes; as well as one or more processors in communication with the ultrasound transducer and the graphical user interface, the processors being configured to: identifying one or more features within the at least one image frame; causing the graphical user interface to display elements associated with at least two image quality operations specific to the identified one or more features, wherein a first image quality operation comprises recommended instructions for manually adjusting transducer settings (438c), wherein the recommended instructions for manually adjusting the transducer settings comprise instructions for manually adjusting the transducer settings in a manner that improves the ultrasound image by eliminating one or more image artifacts caused by the identified one or more features, and a second image quality operation comprises automatic adjustment of the identified one or more features (438b); receiving a user selection of at least one of the elements displayed by the graphical user interface; and The image quality operation corresponding to the user selection is applied to modify the at least one image frame.

2. The ultrasound imaging system according to claim 1, wherein: The second image quality operation is dependent on the first image quality operation.

3. The ultrasound imaging system according to claim 1, wherein: The one or more features are identified by inputting the at least one image frame into a first neural network trained using imaging data including reference features.

4. The ultrasound imaging system according to claim 1, wherein: The one or more features include a fat layer.

5. The ultrasound imaging system according to claim 1, wherein: The graphical user interface is configured to display the annotated image frame in which the one or more features are marked.

6. The ultrasound imaging system according to claim 3, wherein: The first neural network includes a convolutional network defined by a U-net or V-net architecture, and the U-net or V-net architecture is further configured to depict a visceral fat layer and a subcutaneous fat layer within the at least one image frame.

7. The ultrasound imaging system according to claim 1, wherein: The processor is configured to modify the at least one image frame by inputting the at least one image frame into a second neural network trained to output a modified image frame in which the identified one or more features are omitted for display on the graphical user interface.

8. The ultrasound imaging system according to claim 7, wherein: The second neural network includes a generative adversarial network.

9. The ultrasound imaging system according to claim 1, wherein: The one or more processors are further configured to remove noise from the at least one image frame prior to identifying the one or more features.

10. The ultrasound imaging system according to claim 4, wherein: The one or more processors are further configured to determine a size of the fat layer.

11. The ultrasound imaging system according to claim 10, wherein: The dimension includes a thickness of the fat layer at a location within the fat layer specified by a user via the graphical user interface.

12. The ultrasound imaging system according to claim 1, wherein: The target area includes the abdominal area.

13. A method of ultrasound imaging, comprising: collecting echo signals in response to ultrasonic pulses transmitted toward a target area; displaying an ultrasound image from at least one image frame generated based on the ultrasound echo; identifying one or more features within the at least one image frame; displaying elements associated with at least two image quality operations specific to the identified one or more features, wherein a first image quality operation comprises recommended instructions for manually adjusting a transducer setting (438c), wherein the recommended instructions for manually adjusting the transducer setting comprise instructions for manually adjusting the transducer setting in a manner that improves the ultrasound image by eliminating one or more image artifacts caused by the identified one or more features, and a second image quality operation comprises automatic adjustment of the identified one or more features (438b); receiving a user selection of at least one of the displayed elements; and The image quality operation corresponding to the user selection is applied to modify the at least one image frame.

14. The method according to claim 13, wherein The second image quality operation is dependent on the first image quality operation.

15. The method according to claim 13, wherein: The one or more features are identified by inputting the at least one image frame into a first neural network trained using imaging data including reference features.

16. The method according to claim 13, wherein: The one or more features include a fat layer.

17. The method of claim 13, further comprising displaying the annotated image frame in which the one or more features are marked.

18. The method according to claim 13, wherein The at least one image frame is modified by inputting the at least one image frame into a second neural network trained to output a modified image frame in which the identified one or more features are omitted.

19. The method of claim 13, further comprising determining a size of the one or more features at an anatomical location specified by a user.

20. A non-transitory computer-readable medium comprising executable instructions which, when executed, cause a processor of a medical imaging system to perform the method according to any one of claims 13-19.

Images