Identifying fat layers using ultrasound images

Abstract

To reduce or eliminate deterioration of an image caused by a fat layer.SOLUTION: The present disclosure describes an imaging system configured to identify features within image frames and improve the frames by executing adjustment of mage quality. An ultrasonic imaging system includes a transducer configured to acquire an echo signal responsive to an ultrasonic pulse transmitted toward a target. The system also includes a user interface configured to display an image and a processor configured to identify features within the image. The processor causes the interface to display elements associated with at least two image quality operations specific to the identified features. A first image quality operation can include a manual adjustment of transducer settings, and a second image quality operation can include an automatic adjustment of the identified features derived from reference frames having the identified features. The processor receives a user selection of an element and applies the operation to modify the image.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to ultrasound systems and methods for identifying features, such as fat layers, via ultrasound imaging and modifying the images based on the identified features. Certain embodiments include a system configured to identify and remove fat layers and associated image artifacts from ultrasound images, thereby improving image quality. [Background technology]

[0002] Ultrasound imaging can be challenging when scanning patients with moderate to thick fat layers, especially when scanning the abdominal region. Because fat increases acoustic attenuation and sound waves typically propagate through adipose tissue at a different speed relative to other soft tissues, ultrasound beams propagating through adipose tissue are often defocused, an effect known as phase aberration. More specifically, ultrasound beam focusing is achieved by applying a specific delay to each transducer element based on the time of flight of the acoustic pulse, e.g., the length of time it takes for an ultrasound echo signal to travel from a particular anatomical point to each transducer receiving element, or the length of time it takes for a transmitted ultrasound signal to reach a particular anatomical point from the transducer. Beamforming based on inaccurate sound speed assumptions can lead to inaccurate distance-time relationship calculations and can produce defocused ultrasound beams, characterized by, for example, wide primary beams and high side lobes. Such defocusing can occur frequently when imaging abdominal fat layers, through which sound waves typically propagate at only about 1450 m / s, significantly slower than most surrounding tissues, where waves often propagate at about 1540 m / s. As a result, images of areas with fat layers, such as the abdomen, typically contain undesirable artifacts and are generally of poor quality. Existing techniques aimed at correcting or minimizing image artifacts caused by fat are overly complex to implement and are often ineffective. Summary of the Invention [Problem to be solved by the invention]

[0003] Therefore, new ultrasound systems that can reduce or eliminate image degradation caused by the fat layer are needed to improve imaging and evaluation of anatomical targets beneath the fat layer. [Means for solving the problem]

[0004] The present disclosure describes ultrasound systems and methods for identifying and locating at least one feature, such as a fat layer, in an ultrasound image. In some examples, the feature can be identified by implementing a neural network. Various measurements of the identified feature, such as the thickness of the fat layer, can also be determined automatically or manually, and a representation thereof can be displayed on a graphical user interface. The system can generate an annotated ultrasound image in which the feature (e.g., the fat layer) is labeled or highlighted for further evaluation, thereby alerting the user to the feature and any associated aberrations. The system can generate and display at least one recommended manual adjustment of the transducer settings based on the identified feature, so that the adjustment can be implemented to remove the aberration or image artifact caused by the feature. In some embodiments, a second neural network can also be implemented that is trained to automatically remove or correct the identified feature from the ultrasound image. By removing the feature from a particular image, the second neural network can generate a corrected image that is devoid of the feature and the associated image artifact caused by the feature. The corrected image, which has improved quality compared to the original image, can then be displayed for analysis. The disclosed systems and methods are applicable to a wide range of imaging protocols, but may be particularly advantageous when scanning anatomical regions with high fat content, such as the abdominal region, where image degradation caused by fat-induced artifacts may be most severe. Although the exemplary systems and methods are described with respect to fat layer identification and associated image correction, it will be understood that the present disclosure is not limited to fat layer applications, and a variety of anatomical features and / or image artifacts may be identified, corrected, and / or removed in accordance with the principles disclosed herein.

[0005] According to some examples of the present disclosure, an ultrasound imaging system may include an ultrasound transducer configured to acquire echo signals responsive to ultrasound pulses transmitted toward a target region, and a graphical user interface configured to display an ultrasound image from at least one image frame generated from the ultrasound echoes. The system may have one or more processors in communication with the ultrasound transducer and the graphical user interface. The processor may be configured to identify one or more features in the image frame and cause the graphical user interface to display elements associated with at least two image quality operations specific to the identified features. The first image quality operation may include manual adjustment of transducer settings, and the second image quality operation may include automatic adjustment of the identified features derived from a reference frame having the identified features. The processor may be further configured to receive a user selection of at least one of the elements displayed by the graphical user interface and apply an image quality operation corresponding to the user selection to modify the image frame.

[0006] In some examples, the second image quality operation can depend on the first image quality operation. In some embodiments, the one or more features can be identified by inputting the image frames to a first neural network trained with the imaging data having the reference features. In some examples, the one or more features can include a fat layer. In some embodiments, the graphical user interface can be configured to display the annotated image frames in which the one or more features are labeled. In some examples, the first neural network can include a convolutional network defined by a U-net or V-net architecture further configured to delineate the visceral fat layer and the subcutaneous fat layer in the image frames. In some embodiments, the processor can be configured to modify the image frames by inputting the image frames to a second neural network, the second neural network trained to output modified image frames in which the identified features are omitted for display on the graphical user interface. In some examples, the second neural network can include a generative adversarial network. In some embodiments, the processor can be further configured to remove noise from the image frames prior to identifying the one or more features. In some examples, the one or more processors can be further configured to determine a dimension of the fat layer. In some embodiments, the dimension can comprise a thickness of the fat layer at a location within the fat layer specified by a user via a graphical user interface, hi some examples, the target area can comprise an abdominal area.

[0007] According to some examples of the present disclosure, a method of ultrasound imaging includes acquiring echo signals in response to ultrasound pulses transmitted toward a target region, displaying an ultrasound image from at least one image frame generated from the ultrasound echoes, identifying one or more features in the image frame, and displaying elements associated with at least two image quality operations specific to the identified features, where a first image quality operation includes manual adjustment of a transducer setting and a second image quality operation includes automatic adjustment of the identified features derived from a reference frame that includes the identified features. The method further includes receiving a user selection of at least one of the elements to be displayed and applying the image quality operation corresponding to the user selection to modify the image frame.

[0008] In some examples, the second image quality operation can depend on the first image quality operation. In some embodiments, the one or more features can be identified by inputting the image frame to a first neural network that is trained with imaging data having a reference feature. In some examples, the one or more features can include a fat layer. In some embodiments, the method can further include displaying the annotated image frame in which the one or more features are labeled. In some examples, the image frame can be modified by inputting the image frame to a second neural network, which is trained to output a modified image frame in which the identified feature is omitted. In some embodiments, the method can include determining a dimension of the one or more features at an anatomical location specified by the user.

[0009] Any of the methods described herein, or steps thereof, may be embodied in a non-transitory computer-readable medium comprising executable instructions that, when executed, may cause a processor of a medical imaging system to perform one or more of the methods or steps embodied herein. [Brief description of the drawings]

[0010] [Figure 1] 1 is a cross-sectional view of layered muscle, fat and skin tissue and its corresponding ultrasound image. [Diagram 2] FIG. 1 is a block diagram of an ultrasound system in accordance with the principles of the present disclosure. [Figure 3A] FIG. 1 is a block diagram of a U-Net convolutional network configured for adipose tissue segmentation in accordance with the principles of the present disclosure. [Figure 3B] FIG. 13 is a block diagram of a V-Net convolutional network configured for adipose tissue segmentation in accordance with the principles of the present disclosure. [Figure 4] 1 is a graphical user interface implemented in accordance with the principles of the present disclosure. [Diagram 5] FIG. 1 is a block diagram of a neural network configured for fat layer image removal in accordance with the principles of the present disclosure. [Figure 6] FIG. 1 is a block diagram of a collaborative neural network configured to identify and remove fat layers from ultrasound images in accordance with the principles of the present disclosure. [Figure 7] 1 is a flow diagram of a method of ultrasound imaging performed in accordance with the principles of the present disclosure; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The following description of specific embodiments is merely exemplary in nature and is not intended to limit the present invention or its application or uses in any way. In the following detailed description of the embodiments of the present system and method, reference is made to the accompanying drawings which form a part of this specification and which show, by way of example, specific embodiments in which the described system and method may be practiced. These embodiments have been described in sufficient detail to enable those skilled in the art to practice the presently disclosed system and method, and it should be understood that other embodiments may be utilized and structural and logical changes may be made without departing from the spirit and scope of the present system. Moreover, for the sake of clarity, detailed descriptions of specific features will not be discussed where they would be apparent to those skilled in the art so as not to obscure the description of the present system. Therefore, the following detailed description should not be taken in a limiting sense, and the scope of the present system is defined only by the appended claims.

[0012] The systems disclosed herein may be configured to implement a deep learning model to identify at least one layer of fat present in a target region. The fat layer may be indicated on a user interface, and a recommended solution to eliminate or reduce image degradation caused by fat may be generated and optionally displayed. An embodiment includes a system configured to improve ultrasound images by using a deep learning model that is trained to remove fat layers and associated image artifacts from an image and generate a new image lacking such features. The disclosed system may improve B-mode image quality, especially when imaging areas with high fat, such as abdominal regions. The system is not limited to B-mode imaging or abdominal imaging, but may be utilized to correct images with fat at any anatomical location of a patient, and may therefore be applied to image various anatomical features, such as the liver, lungs, and / or various extremities. The system may be utilized for various quantitative imaging modalities in addition to or instead of B-mode imaging to improve its accuracy and / or efficiency. For example, the disclosed system may be implemented for shear wave elastography optimization, beam pattern adjustment for acoustic attenuation, and / or backscatter coefficient estimation.

[0013] An ultrasound system according to the present disclosure may utilize various neural networks, such as deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), autoencoder neural networks, etc., to identify and optionally remove fat layers in newly generated images. In various examples, a first neural network may be trained using any of a variety of learning techniques now known or hereafter developed to obtain a neural network (e.g., a trained algorithm or a hardware-based node system) configured to analyze input data in the form of ultrasound image frames and determine the presence of at least one fat layer therein. A second neural network may be trained to modify the input data in the form of ultrasound image frames or data having or embodying a fat layer and remove the layer therefrom. Image artifacts generated by fat-induced phase aberrations may also be selectively removed by the second neural network. Without the fat layer and associated artifacts, image quality may be significantly improved and may be revealed with improved clarity and / or contrast.

[0014] An ultrasound system according to the principles of the present invention may include, or be operatively coupled to, an ultrasound transducer configured to transmit ultrasound pulses toward a medium, such as a human body or a particular portion thereof, and generate echo signals in response to the ultrasound pulses. The ultrasound system may include a beamformer configured to perform transmit and / or receive beamforming, and in some examples, a display configured to display ultrasound images generated by the ultrasound imaging system. The ultrasound imaging system may include one or more processors and at least one neural network, which may be implemented in hardware and / or software components. An embodiment may include two or more neural networks, which may be communicatively coupled or integrated into one multi-layer network, such that an output of a first network serves as an input to a second network.

[0015] Neural networks implemented according to the present disclosure may be hardware (e.g., neurons represented by physical components) or software-based (e.g., neurons and pathways implemented in a software application), and various topologies and learning algorithms may be used to train the neural network to generate desired outputs. For example, a software-based neural network may be implemented using a processor (e.g., a single-core or multi-core CPU, a single GPU or a GPU cluster, or multiple processors arranged for parallel processing) configured to execute instructions, which may be stored on a computer-readable medium, that when executed cause the processor to execute a trained algorithm for identifying a fat layer present in an ultrasound image and / or for generating a new image lacking the identified fat layer. The ultrasound system may include a display or graphics processor operable to configure the ultrasound image and / or additional graphical information, which may include annotations, confidence metrics, user instructions, tissue information, patient information, indicators, and other graphical components, into a display window for display on a user interface of the ultrasound system. In some embodiments, the ultrasound images and associated measurements may be provided to a storage and / or memory device, such as a picture archiving and communication system (PACS), for reporting purposes or for future training (e.g., to continue to improve the performance of the neural network), particularly for corrected images generated by a system configured to remove the fat layer and associated artifacts from the fat-labeled images.

[0016] FIG. 1 illustrates a cross-sectional view of normal tissue 102a having an outer layer of skin 104a, a layer of fat 106a, and a layer of muscle 108a. Ultrasound imaging of the tissue may generate a corresponding image 102b of the skin layer 104b, the fat layer 106b, and the muscle layer 108b. As shown, each layer may be clearly visible on the ultrasound image 102b, and the muscle layer 108b may appear brighter than the fat layer 106b. Existing techniques require a user to manually identify and measure the fat layer 106b, and such techniques are unable to remove the fat layer and associated artifacts from the image. The systems herein may automatically identify one or more fat layers, and in some instances, process the corresponding image to improve image quality despite the presence of such fat layers. The systems herein are not limited to identifying fat layers in particular, and may be configured to identify fat in any form, for example, localized deposits, pockets, or deposits of various shapes. Exemplary systems may also be configured to depict visceral fat and subcutaneous fat. Subcutaneous fat can include the area approximately 1 cm above the umbilicus along the axillary-umbilical line. The thickness of the subcutaneous fat layer can be measured as the distance between the skin-fat interface and the outer edge of the linea alba during expiration. Visceral fat can be measured as the distance between the linea alba and the anterior aorta approximately 1 cm above the umbilicus along the xiphoid-umbilical line.

[0017] 2 illustrates an exemplary ultrasound system according to the principles of the present disclosure. The ultrasound system 200 may include an ultrasound data acquisition unit 210. The ultrasound data acquisition unit 210 may include an ultrasound probe having an ultrasound sensor array 212 configured to transmit ultrasound pulses 214 to a target region 216 of a subject, which may include an abdominal region, a chest region, one or more limbs and / or features thereof, and to receive ultrasound echoes 218 in response to the transmitted pulses. The region 216 may have an adipose layer 217 of variable thickness. For example, the adipose layer may range in thickness from about 0.1 to about 20 cm, about 1 to about 12 cm, about 2 to about 6 cm, or about 4 to about 5 cm. As further shown, the ultrasound data acquisition unit 210 may include a beamformer 220 and a signal processor 222 that may be configured to generate a stream of discrete ultrasound image frames 224 from the ultrasound echoes 218 received at the array 212. The image frames 224 can be communicated to a data processor 226, e.g., a computational module or circuitry, which in some examples may include a pre-processing module 228, which can be configured to implement at least one neural network, such as neural network 230, that is trained to identify fat layers within the image frames 224.

[0018] The ultrasonic sensor array 212 may include at least one transducer array configured to transmit and receive ultrasonic energy. The settings of the ultrasonic sensor array 212 may be preset to perform a particular scan and may be adjusted during the scan. Various transducer arrays may be used, for example, linear arrays, convex arrays, or phased arrays. The number and arrangement of transducer elements included in the sensor array 212 may vary in different examples. For example, the ultrasonic sensor array 212 may include a 1D or 2D array of transducer elements, corresponding to linear array and matrix array probes, respectively. The 2D matrix array may be configured to electronically scan in both elevation and azimuth dimensions (via phased array beamforming) for 2D or 3D images. In addition to B-mode imaging, imaging modalities implemented in accordance with the disclosure herein may include, for example, shear wave and / or Doppler. Various users may handle and operate the ultrasound data acquisition unit 210 to perform the methods described herein.

[0019] The beamformer 220 coupled to the ultrasonic sensor array 212 may comprise a microbeamformer or a combination of a microbeamformer and a main beamformer. The beamformer 220 may control the transmission of ultrasonic energy, for example, by forming ultrasonic pulses into a focused beam. The beamformer 220 may also be configured to control the reception of ultrasonic signals so that identifiable image data may be generated and processed with the aid of other system components. The role of the beamformer 220 may vary for different ultrasonic probe types. In some embodiments, the beamformer 220 may include two separate beamformers, a transmit beamformer configured to receive and process pulse sequences of ultrasonic energy for transmission to the subject, and a separate receive beamformer configured to amplify, delay, and / or soften received ultrasonic echo signals. In some embodiments, the beamformer 220 may include a microbeamformer operating on groups of sensor elements for both transmit and receive beamforming, coupled to a main beamformer operating on group inputs and outputs, respectively, for both transmit and receive beamforming.

[0020] The signal processor 222 may be communicatively, operatively, and / or physically coupled to the sensor array 212 and / or the beamformer 220. In the example shown in FIG. 2, the signal processor 222 is included as an integral component of the data acquisition unit 210, while in other examples the signal processor 222 may be a separate component. In some examples, the signal processor may be housed together with the sensor array 212, or may be physically separated and communicatively coupled thereto (e.g., via a wired or wireless connection). The signal processor 222 may be configured to receive unfiltered, unorganized ultrasound data embodying the ultrasound echoes 218 received at the sensor array 212. From this data, the signal processor 222 may continuously generate ultrasound image frames 224 as the user scans the target region 216. In some embodiments, the ultrasound data received and processed by the data collection unit 210 may be utilized by one or more components of the system 200 prior to generating ultrasound image frames therefrom. For example, as indicated by the dashed lines and described further below, the ultrasound data may be communicated directly to the first neural network 230 or the second neural network 242, respectively, for processing before the ultrasound image frames are generated and / or displayed.

[0021] The pre-processing module 228 may be configured to remove noise from the image frames 224 received at the data processor 226, thereby improving the signal-to-noise ratio of the image frames. The noise removal methods employed by the pre-processing module 228 may vary and may include block matching with 3D filtering in some examples. By improving the signal-to-noise ratio of the ultrasound image frames, the pre-processing module 228 may improve the accuracy and efficiency of the neural network 230 in processing the frames.

[0022] In certain embodiments, the neural network 230 comprises a deep learning segmentation network configured to detect and optionally measure one or more fat layers based on one or more characteristic features of fat detected in the ultrasound image frames 224 or image data acquired by the data acquisition unit 210. In some examples, the network 230 may be configured to identify and segment fat layers present in the image frames and automatically determine dimensions, e.g., thickness, length, and / or width, of the identified layers at various locations that may be specified by a user. The layers may be masked or otherwise labeled on the images being processed. In some examples, different configurations of the neural network 230 may segment fat layers present in 2D or 3D images. Particular network architectures may include shrinking and expanding cascades of convolutional and max pooling layers. Training the neural network 230 may include inputting a large number of images, including annotated fat layers and images lacking fat layers, over time, such that the network learns to identify fat layers in unannotated images in real time during an ultrasound scan.

[0023] The detected fat layer can be reported to a user via a display processor 232 coupled to a graphical user interface 234. The display processor 232 can be configured to generate an ultrasound image 235 from the image frames 224, which can be displayed in real time on the user interface 234 as the ultrasound scan is being performed. The user interface 234 can be configured to receive user input 236 at any time before, during, or after the ultrasound procedure. In addition to the displayed ultrasound image 235, the user interface can be configured to generate one or more additional outputs 238, which can include a combination of graphics displayed, for example, overlaid, simultaneously with the ultrasound image 235. Such graphics can mark the presence and size of at least one fat layer (e.g., visceral and / or subcutaneous), along with certain anatomical features and measurements identified by the system, such as various organs, bones, tissues, and / or tissue interfaces. In some examples, the fat layer can be highlighted by outlining the fat contours and / or color-coding the fat regions. Fat thickness can be further calculated by determining a maximum, minimum, and / or average vertical thickness of the masked fat region output from the segmentation network 230. In some embodiments, the output 238 can have selectable elements associated with image quality operations to improve the quality of a particular image 235. Image quality operations can include instructions to manually adjust transducer settings, such as instructions to adjust an analog gain curve, instructions to apply a preload to compress a detected fat layer, and / or instructions to turn on a harmonic imaging mode, to improve the image 235 by removing, reducing, or minimizing one or more image artifacts or aberrations caused by the fat layer.The output 238 may include additional user-selectable elements and / or warnings to perform other image quality operations, depending on the first image quality operation that performs an automatic adjustment of the identified features, e.g., fat layer, in the image 235 in a manner that eliminates, reduces, or minimizes the features and / or any associated artifacts or aberrations, as described further below. The graphical user interface 234 may then receive user input 236 to implement at least one of the quality operations and prompt the data processor 226 to modify the image frame 224 that includes the features. In some examples, the user interface 234 may also receive image quality enhancement instructions that differ from the instructions embodied in the output 238, e.g., instructions based on the user's knowledge and experience. The output 238 may include annotations, confidence metrics, user instructions, tissue information, patient information, indicators, user notifications, and other graphical components.

[0024] In some examples, the user interface 234 may be configured to receive user instructions 240 specific to the automatic image quality operation. The user instructions 240 may be responsive to selectable alerts displayed on the user interface 234 or simply input by a user. According to such examples, the user interface 234 may prompt the data processor 226 to automatically generate an improved image based on the determined presence of a fat layer by implementing a second neural network 242 configured to remove the fat layer from the ultrasound image, thereby generating an improved image 244 that is devoid of one or more fat layers and / or image artifacts associated therewith. As shown in FIG. 2, the second neural network 242 may be communicatively coupled to the first neural network 230 such that the output of the first neural network, e.g., an annotated ultrasound image in which fat is identified, may be directly input to the second neural network 242. In some examples, the second neural network 242 may include a Laplacian pyramid of adversarial networks configured to utilize a cascade of convolutional networks to generate images in a sparse-to-fine manner. Large scale adjustments made to the input image having at least one fat layer can be minimized to preserve the most prominent image characteristics while maximizing the subtle changes characteristic of the identified fat layer and associated image artifacts. The input received by the second neural network 242 can include an ultrasound image including a fat layer, or image data embodying a fat layer that has not yet been processed into a complete image. In accordance with the latter example, the second neural network 242 can be configured to correct the image signal in the channel region of the ultrasound data acquisition unit 210, for example, by removing the fat layer and its associated artifacts. The architecture and mode of operation of the second neural network 242 can be modified as described below in connection with FIG. 5.

[0025] The configuration of the components shown in FIG. 2 may vary. For example, system 200 may be handheld or stationary. Various portable devices, such as laptops, tablets, smartphones, etc., may be used to implement one or more functions of system 200. In examples incorporating such devices, the ultrasonic sensor array may be connectable, for example, via a USB interface. In some examples, the various components shown in FIG. 2 may be combined. For example, neural network 230 may be merged with neural network 242. According to such an embodiment, the two networks may, for example, constitute subcomponents of a larger layered network.

[0026] The particular architecture of the network 230 may vary. In one example, the network 230 may comprise a convolutional neural network. In a particular example, the network 230 may comprise a convolutional autoencoder with skip connections from the encoder layer to the decoder layer on the same architecture network level. For 2D ultrasound images, a U-Net architecture 302a may be implemented in a particular embodiment, as shown in the example of FIG. 3A. The U-Net architecture 302a has a contraction path 304a and an expansion path 306a. For example, as described by Ronneberger, O. et al., "U-Net: A Convolutional Network for Biomedical Image Segmentation," provisionally accepted at the Society for Medical Image Computing and Computer-Aided Intervention, published November 18, 2015 ("Ronneberger"), in one embodiment, the contraction path 304a may have a cascade of repeated 3×3 convolutions, followed by a rectified linear unit and a 2×2 max pooling operation with downsampling at each step. As described by Ronneberger, the augmentation path 306a can have successive steps of up-convolution, each step halving the number of feature channels. The output 308a can include a segmentation map that identifies one or more fat layers present in the initial image frame 224. In some implementations, the fat layers or surrounding non-fatty areas can be masked, and in some examples, the output 308a can perform separate masking for each tissue type to delineate non-fatty areas, subcutaneous fat layers, and / or visceral fat layers. Training the network can include inputting ultrasound images that include one or more fat layers and corresponding segmentation maps until the network learns to reliably identify fat layers present in new images. When a small number of training images are available, a data augmentation procedure can also be performed to train the network, as described by Ronneberger.

[0027] For 3D ultrasound images, a convolutional V-Net architecture 302b can be implemented in certain embodiments, as shown in the example of FIG. 3B. The V-Net architecture 302b can include a compression path 304b followed by a decompression path 306b. In one embodiment, each stage of the compression path 304b can operate at different resolutions and can include, for example, one to three convolutional layers that perform convolutions on voxels of various sizes, as described by Milletari, F. et al. in “V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation,” 3D Vision (3DV), 2016 Fourth International Conference, 565-571, Published October 25, 2016 (“Milletari”). In some examples, each stage can be configured to learn a residual function that can achieve convergence in less time than existing network architectures, as further described by Milletari. The output 308b may include a three-dimensional segmentation map that identifies one or more fat layers present in the initial image frame 224, which may include delineation of non-fat, visceral fat, and / or subcutaneous fat. Training the network may include end-to-end training by inputting three-dimensional images that include one or more fat layers and corresponding annotated images in which the fat layers are identified. To train the network when a limited number of training images are available, especially annotated images, data augmentation measures may be implemented as described by Milletari.

[0028] FIG. 4 illustrates an example of a graphical user interface 400 configured in accordance with the present disclosure. As illustrated, the interface 400 may be configured to display an ultrasound image 435 of a target region 416 including at least one fat layer 417, the boundaries of which are indicated by lines 417a and 417b. As further shown, the thickness of the fat layer 417 is 14 mm at one location, which may be identified by a user, for example, by directly interacting with the image 435 on a touch screen. Various exemplary outputs are also shown, including a fat layer detection notification 438a, an "Auto-correct" button 438b, and recommended instructions 438c for improving the quality of the image 435 by adjusting system parameters. The fat layer detection notification 438a includes an indication of the average thickness of the fat layer 417, which in this particular example is 16 mm. By selecting the "Auto-correct" button 438b, the user may initiate automatic removal of the fat layer 417 from the image via neural network generation of a modified image that retains all features of the image 435 except for the fat layer and any associated artifacts. Also, the modified image may have reduced signal attenuation. The recommended instructions 438c include instructions to initiate harmonic imaging, apply more preload, and adjust the analog gain curve. The instructions 438c can vary depending on the thickness and / or location of the fat layer detected in a given image, and / or the extent to which the fat layer creates image artifacts and / or generally degrades image quality. For example, the instructions 438c can include recommended modifications to the position and / or orientation of the ultrasound probe used to acquire the image. In some implementations, the user interface 400 can display the image to be modified and a selectable option to revert to the original image, e.g., an "undo correction" button. According to such an example, the user can toggle back and forth between the image with the fat layer to be annotated and a new modified image lacking the fat layer.

[0029] FIG. 5 illustrates an example of a neural network 500 configured to remove one or more fat layers and associated artifacts from an ultrasound image and generate a new modified image lacking such features. This particular example includes a generative adversarial network (GAN), although various network types can be implemented. The GAN 500 includes a generative network 502 and a competitive discriminant network 504, as described, for example, by Reed, S. et al. in “Generative Adversarial Text to Image Synthesis” (Proceedings of 33rd International Conference on Machine Learning, New York, NY (2016) JMLR: W&CP vol. 48). In operation, the generative network 502 can be configured to generate synthetic ultrasound image samples 506 lacking one or more fat layers and associated artifacts in a feed-forward manner based on an input 508 consisting of text-labeled images annotated with the fat layers to be identified. The discrimination network 504 can be configured to determine the likelihood that a sampling 506 generated by the generative network 502 is real or fake based in part on a plurality of training images that include and lack a fat layer. After training, the generative network 502 can learn to generate images lacking one or more fat layers from input images that include one or more fat layers, such that the modified non-fat images are substantially indecipherable from the actual ultrasound images except for the presence of fat and associated artifacts. In some examples, the training network 500 can include inputting a pair of controlled experimental images of phantom tissue with and without a fat layer near the surface. For example, various robotic components and / or motorized stages can be utilized to generate multiple sample images in a consistent manner such that each image has the same field of view within the phantom tissue.

[0030] FIG. 6 illustrates a collaborative system 600 of convolutional networks configured to identify and remove at least one fat layer from an original ultrasound image in accordance with the principles of the present disclosure. An initial ultrasound image 602 can be input to a first convolutional network 604 that can be configured to segment and annotate a fat layer 606 present in the initial image, thereby generating an annotated image 608. The annotated image 608 can be input to a convolutional generator circuitry 610 that is communicatively coupled to a convolutional identification circuitry 612. As shown, the convolutional generator network 610 can be configured to generate a modified image 614 that lacks the fat layer 606 identified and labeled by the first convolutional network 604. A number of anatomical features 616 are more clearly visible in the modified image 614 due to the absence of the fat layer 606 and the image degradation it causes. The configuration of the networks 604, 610, and 612 may vary depending on the embodiment. One or more of the images 602, 608 and / or 614 may be displayed for user analysis on a graphical user interface in various examples.

[0031] 7 is a flow diagram of a method of ultrasound imaging performed in accordance with the principles of the present disclosure. Exemplary method 700 illustrates steps that may be utilized in any sequence by systems and / or devices described herein to identify and optionally remove one or more fat layers from an ultrasound image, for example during an abdominal scan. Method 700 may be performed by an ultrasound imaging system such as system 100 or other systems, including, for example, mobile systems such as LUMIFY by Koninklijke Philips NV ("Philips"). Additional exemplary systems may include SPARQ and / or EPIQ, also manufactured by Philips.

[0032] In the illustrated embodiment, the method 700 begins at block 702 by "acquiring echo signals in response to ultrasound pulses transmitted toward a target region."

[0033] The method continues at block 704 by "displaying an ultrasound image from at least one image frame generated from the ultrasound echoes."

[0034] The method continues at block 706 by "Identifying One or More Features in an Image Frame."

[0035] The method continues at block 708 by "displaying elements associated with at least two image quality operations specific to the identified feature, a first image quality operation comprising manual adjustment of transducer settings and a second image quality operation comprising automatic adjustment of the identified feature derived from a reference frame including the identified feature."

[0036] The method continues at block 710 by "receiving a user selection of at least one of the displayed elements."

[0037] The method continues at block 712 by "Applying Image Quality Operations Corresponding to User Selections to Modify the Image Frame."

[0038] In various embodiments in which the components, systems, and / or methods are implemented using a computer-based system or a programmable device such as programmable logic, it is understood that the above-described systems and methods can be implemented using any of a variety of known or later developed programming languages, such as "C", "C++", "FORTRAN", "Pascal", "VHDL", and the like. Accordingly, various storage media, such as magnetic computer disks, optical disks, electronic memory, and the like, can be provided that can have information capable of instructing a device, such as a computer, to perform the above-described systems and / or methods. When an appropriate device can access the information and programs contained therein, the storage medium can provide the information and programs to the device, and thus the device can perform the functions of the systems and / or methods described herein. For example, when a computer disk having appropriate material, such as source files, object files, executable files, and the like, is provided to a computer, the computer can receive the information and appropriately configure itself to perform the various functions, and perform the functions of the various systems and methods outlined in the figures and flow charts above. That is, the computer can receive various portions of information relating to different elements of the above-described systems and / or methods from the disk, and implement the individual systems and / or methods, and coordinate the functions of the individual systems and / or methods described above. In view of the present disclosure, it is noted that the various methods and devices described herein may be implemented in hardware, software, and firmware. Moreover, the various methods and parameters are included by way of example only and not in any limiting sense.

[0039] In view of this disclosure, one of ordinary skill in the art will be able to practice the present teachings in determining their own techniques and the equipment required to effect these techniques while remaining within the scope of the present invention. One or more functions of the processor described herein may be incorporated into fewer or a single processing unit (e.g., a CPU) or may be implemented using application specific integrated circuits (ASICs) or general purpose processing circuitry that is programmed in response to executable instructions to perform the functions described herein.

[0040] Although the present system has been described with particular reference to an ultrasound imaging system, it is also envisioned that the present system can be extended to other medical imaging systems in which one or more images are obtained in a systematic manner. Thus, the present system can be used to acquire and / or record image information for, but not limited to, the kidneys, testes, breasts, ovaries, uterus, thyroid, liver, lungs, musculoskeletal, spleen, heart, arterial blood and vasculature, and other imaging applications related to ultrasound-guided interventions. Furthermore, the present system can have one or more programs that can be used with conventional imaging systems to provide the features and advantages of the present system. Certain additional advantages and features of the present disclosure may be apparent to those skilled in the art upon reviewing the present disclosure or may be experienced by those skilled in the art employing the novel systems and methods of the present disclosure. Another advantage of the present systems and methods may be that conventional medical imaging systems can be easily upgraded to incorporate the features and advantages of the present systems, devices, and methods.

[0041] Of course, it will be understood that any one of the examples, embodiments, or processes described herein can be combined with one or more other examples, embodiments, and / or processes, or can be separated and / or performed among separate apparatus or device parts according to the present systems, devices, and methods.

[0042] Finally, the above discussion is intended to merely illustrate the present system, and should not be construed as limiting the appended claims to any particular embodiment or group of embodiments. Thus, while the present system has been invented in particular detail with reference to exemplary embodiments, it will also be understood that those skilled in the art can devise numerous modifications and alternative embodiments without departing from the broader intended spirit and scope of the present system as set forth in the following claims. Thus, the specification and drawings are to be regarded in an illustrative manner, and are not intended to limit the scope of the appended claims.

Claims

1. 1. An ultrasound imaging system comprising: an ultrasonic transducer configured to acquire echo signals in response to ultrasonic pulses transmitted toward a target area; a graphical user interface configured to display an ultrasound image from at least one image frame generated from the ultrasound echoes; one or more processors in communication with the ultrasound transducer and the graphical user interface, Identifying one or more features within the image frames; Modifying the image frame based on the identified one or more features. A processor configured to having the one or more features are identified by inputting the image frames to a first neural network that is trained with imaging data having reference features; the one or more features having a fat layer; the processor is configured to modify the image frame by inputting the image frame to a second neural network, the second neural network being trained to output a modified image frame for display on the graphical user interface in which the identified features are omitted. Ultrasound imaging system.

2. 10. The ultrasound system of claim 1, wherein the graphical user interface is configured to display annotated image frames in which the one or more features are labeled.

3. 2. The ultrasound imaging system of claim 1, wherein the first neural network comprises a convolutional network defined by a U-net or V-net architecture further configured to delineate visceral and subcutaneous fat layers in the image frames.

4. The ultrasound imaging system of claim 1 , wherein the second neural network comprises a generative adversarial network.

5. 10. The ultrasound imaging system of claim 1, wherein the one or more processors are further configured to remove noise from the image frames before identifying the one or more features.

6. The ultrasound imaging system of claim 1 , wherein the one or more processors are further configured to determine a size of the fat layer.

7. The ultrasound imaging system of claim 6 , wherein the dimension comprises a thickness of the fat layer at a location within the fat layer identified by a user via the graphical user interface.

8. The ultrasound imaging system of claim 1 , wherein the target region comprises an abdominal region.

9. 1. A method of ultrasound imaging, the method comprising: acquiring echo signals in response to ultrasound pulses transmitted toward a target area; displaying an ultrasound image from at least one image frame generated from the ultrasound echoes; identifying one or more features within the image frames; modifying the image frame based on the identified one or more features; having the one or more features are identified by inputting the image frames to a first neural network that is trained with imaging data having reference features; the one or more features having a fat layer; the image frame is modified by inputting the image frame to a second neural network, the second neural network being trained to output a modified image frame in which the identified features are omitted. method.

10. The method of claim 9 , further comprising displaying the annotated image frame in which the one or more features are labeled.

11. The method of claim 9 , further comprising determining a size of the one or more features at an anatomical location specified by a user.

12. A non-transitory computer readable medium having executable instructions which, when executed, cause a processor of a medical imaging system to perform the method of any one of claims 9 to 11.

Images