Systems and methods for improving ultrasound image resolution using neural networks

Abstract

The present disclosure provides a method for mapping an ultrasound image to a resolution-mapped ultrasound image using a generative neural network while preserving clinical quality of the resolution-mapped ultrasound image, thereby enabling a clinician to evaluate an ultrasound image at a preferred resolution without loss of clinically relevant content. In one embodiment, the present disclosure provides a method comprising: acquiring an ultrasound image of an anatomical region of a subject, wherein the ultrasound image is at a first resolution; selecting a target resolution, wherein the target resolution is different from the first resolution; selecting a clinical quality metric; selecting a trained resolution mapping network based on the target resolution and the clinical quality metric; mapping the ultrasound image to a resolution-mapped ultrasound image using the trained resolution mapping network, wherein the resolution-mapped ultrasound image is at the target resolution; and displaying the resolution-mapped ultrasound image via a display device.

Description

TECHNICAL FIELD

[0001] Embodiments of the subject matter disclosed herein relate to ultrasound imaging, and more particularly, to systems and methods for improving high resolution in ultrasound images using a generative neural network. BACKGROUND

[0002] Clinical ultrasound is an imaging modality that employs ultrasound waves to probe the internal structures of a patient's body and produce corresponding images. An ultrasound probe, which includes a plurality of transducer elements, emits ultrasound pulses that are reflected or backscattered by structures in the body, refracted, or absorbed. The ultrasound probe then receives the reflected echoes, which are processed into images. For example, medical imaging devices, such as ultrasound imaging devices, can be used to obtain images of a patient's heart, uterus, liver, lungs, and various other anatomical regions. An ultrasound beam produced by a single row of transducer elements (e.g., a one-dimensional (ID) ultrasound probe or a single row of a multi-row array) is focused by a lens such that the thickness of the beam is minimized at a first depth corresponding to the focal point of the lens. At the first depth, the high resolution of the image acquired by the probe is maximized, whereby anatomical features visible at the first depth are shown with higher resolution compared to anatomical features visible at a second depth lower or higher than the first depth. Thus, an ultrasound image acquired by an ID probe has a first resolution profile characterized by a narrow region of high resolution, and regions in the near field and far field shown with resolution that gradually decreases with distance from the high resolution region.

[0003] The inconsistency in high resolution has been addressed by using probes with additional rows of transducer elements (e.g., 1.5-dimensional / 1.5D or 2-dimensional / 2D probes). Multi-row probes have an advantage over single-row probes in that they can display ultrasound images with high resolution over an extended depth range by adjusting the lens of the probe so that each row of transducers is focused at a different depth, and combining the ultrasound images acquired by each row of the probe. Thus, higher resolution images with positive benefits can be acquired, which can include better patient experience, more accurate diagnoses, and / or improved clinical outcomes.

[0004] However, multi-row array probes, such as 1.5D and 2D probes, rely on more elements and thus can be more expensive than ID (single row array) probes. Additionally, the additional elements of a 1.5D or 2D probe can subject a subject to a higher level of energy than an ID probe, which can be undesirable in some cases (e.g., when acquiring fetal images, etc.). Ultrasound operators can also have less experience using 1.5D or 2D probes, which can result in lower image quality, longer examination times, and / or increased time and cost of training. SUMMARY

[0005] The present disclosure at least partially addresses one or more of the above-identified problems through a method comprising: acquiring a first ultrasound image having a first resolution distribution; inputting the first ultrasound image into a trained neural network algorithm; and generating a second ultrasound image having a second, higher resolution distribution as an output of the trained neural network algorithm.

[0006] The above advantages and other advantages and features of the present disclosure will be apparent from the following detailed description, either alone or in conjunction with the accompanying drawings. It should be understood that the above summary is provided to introduce a selection of concepts further described in the detailed description in a simplified form. It is not meant to identify key or essential features of the claimed subject matter, the scope of which is solely defined by the claims that follow the detailed description. Furthermore, the claimed subject matter is not limited to implementations that address any disadvantages noted above or in any part of this disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Various aspects of the present disclosure may be better understood by reading the following detailed description and referring to the accompanying drawings, in which:

[0008] Figure 1 A block diagram of an exemplary embodiment of an ultrasound system is shown;

[0009] Figure 2 A block diagram illustrating an exemplary embodiment of an image processing system configured to map an ultrasound image from an original resolution distribution to a target resolution distribution using a trained generative network is shown;

[0010] Figure 3 A block diagram illustrating an exemplary embodiment of a resolution mapping network training system is shown;

[0011] Figure 4A is a method that can be used according to an exemplary embodiment Figure 3 An exemplary generative neural network architecture diagram for use in the system;

[0012] Figure 4B An exemplary input image and an exemplary output image of a trained resolution mapping network are shown;

[0013] Figure 5 Shown is the preparation for Figure 3 A flowchart of an exemplary method for training / testing datasets of a resolution mapping network training system;

[0014] Figure 6 Shown for use Figure 3 A flowchart of an exemplary method for training a resolution mapping network using a resolution mapping network training system;

[0015] Figure 7 A flowchart showing an exemplary method for applying a trained resolution mapping network to map an ultrasound image having a first resolution profile to an ultrasound image having a second resolution profile.

[0016] The accompanying drawings illustrate particular aspects of the described systems and methods for mapping one or more ultrasound images at a first resolution to one or more corresponding ultrasound images at a target resolution using a generative neural network. The drawings, together with the description below, illustrate and explain the structures, methods and principles described herein. In the drawings, the size of components can be exaggerated or modified for clarity. Well-known structures, materials or operations are not shown or described in detail in order to avoid obscuring aspects of the described components, systems and methods. DETAILED DESCRIPTION

[0017] Clinical ultrasound imaging typically includes placing an ultrasound probe including one or more transducer elements onto an imaging subject, such as a patient, at the location of a target anatomical feature, e.g., an abdomen, a chest, etc. Images are acquired by the ultrasound probe and displayed on a display device in real-time or near real-time, e.g., the images are displayed as soon as they are generated and without intentional delay. An operator of the ultrasound probe can view the images and adjust various acquisition parameters and / or position, pressure and / or orientation of the ultrasound probe in order to obtain high quality images of the target anatomical feature, e.g., a heart, a liver, a kidney, or another anatomical feature. Adjustable acquisition parameters include transmit frequency, transmit depth, gain, e.g., total gain and / or time gain compensation, cross-beam, beam steering angle, beamforming strategy, frame averaging, and / or other parameters.

[0018] An ultrasound probe can include a single row array of transducer elements, herein a linear array probe or ID probe, that generates an ultrasound beam having a resolution defined in three dimensions. The axial resolution of a beam describes the resolving power between two points parallel to the path of the beam. The lateral resolution of a beam describes the resolving power between two points perpendicular to the path of the beam and parallel to the transducer array of the probe. The elevational resolution of a beam describes the resolving power between two points perpendicular to the transducer array of the probe at a fixed axial distance from the probe.

[0019] An ultrasound beam is typically focused by a lens, whereby the thickness of the beam varies with distance from the probe, such that the initial thickness of the beam decreases as it leaves the probe until a minimum thickness is reached at the focal point of the lens, after which the thickness of the beam increases as the beam gets further from the probe. The focal point of the lens is a fixed distance from the probe (e.g., a fixed depth into the subject). At the fixed distance, maximum height resolution of the beam is achieved. While a ID probe of a single row array exhibits good height resolution at the focal point of the lens, the height resolution is poor in the near field and the far field. For example, two points of an image that are close together in the height plane can be identifiable at the focal point of the beam, but not at depths above or below the focal point of the beam.

[0020] The poor near field and far field height resolution can be addressed by using a transducer element of a multi-row array (herein a 1.5D probe or a 2D probe), where each row of transducer elements is focused at a different focal point (e.g., a different distance or depth from the probe). The images produced by each row of transducers can be combined to create a single ultrasound image with high height resolution over a wider range of depths than the relatively narrow range of a ID probe. Thus, a 1.5D or 2D probe of a multi-row array provides uniform slice thickness and excellent contrast resolution over an extended imaging range.

[0021] Thus, an image can be acquired via a 2D probe with a transducer of a multi-row array, where anatomical features are shown in high resolution over a wider height range. This can result in a higher quality ultrasound image, where anatomical features can be more clearly identified and viewed, which can lead to a better patient experience, more accurate diagnoses, and / or improved clinical outcomes.

[0022] For example, in an ultrasound examination of a pregnant woman’s uterus performed using a ID probe with a single row of transducers, an ultrasound image can be acquired that shows anatomical features of the fetus (e.g., ears, nose, etc.) at a first depth focused by the beam produced by the single row of transducers in high resolution, while other anatomical features of the fetus (e.g., feet, genitalia, etc.) at a second depth (e.g., in the far field or the near field) can be shown in lower resolution. In contrast, in an ultrasound examination of a pregnant woman’s uterus performed using a 2D probe with a multi-row of transducers, an ultrasound image can be acquired that shows anatomical features of the fetus at a range of depths in high resolution. Generating a high resolution view of anatomical features of interest over a range of depths using a ID probe can involve adjusting the position of the probe over time to sweep the focal point of the ID probe over the range of depths at which the anatomical features of interest are visible, which can adversely affect diagnosis or patient experience.

[0023] However, although multi-row array (e.g., 2D) probes can produce higher resolution images than single-row array (e.g., 1D) probes, multi-row array probes may not be as widely adopted by clinicians as single-row array probes due to higher costs associated with a greater number of transducers and control elements, higher energy output, and / or a poor operator experience.

[0024] Thus, the present disclosure provides systems and methods for converting an original image having an original resolution distribution into an output ultrasound image having a target resolution distribution, wherein the target resolution distribution may include high resolution over a wider depth range than the original resolution distribution. For example, the original resolution distribution may be characterized as narrow, providing high resolution at a first depth, but providing poor resolution at a second depth in the near field or far field, while the target resolution distribution may be characterized as wide, including high resolution over a wide depth range in the near field and the far field. In one embodiment, a trained resolution mapping network may be used to convert the original image into a target image. The present disclosure also provides training systems and methods that enable the resolution mapping network to be trained to learn a mapping from a first resolution distribution to a target resolution distribution.

[0025] In one embodiment, the imaging system is ultrasound-assisted. Figure 1 The ultrasound imaging system 100) can be communicatively coupled to an image processing system, such as a 1D ultrasound probe, to acquire one or more ultrasound images. Figure 2 The image processing system 202 may include one or more neural network models stored in non-transitory memory, such as a generative neural network model and a generative adversarial network model. Figure 4A , which can be trained and deployed to use ultrasound images acquired via a 1D ultrasound probe as input to output images with a wider resolution distribution (e.g., similar to images produced by a 2D ultrasound probe), as shown schematically in FIG. Figure 4B As shown. Figure 3 The resolution map network training system 300 shown is performed by Figure 5 Method 500 and / or Figure 6 The method 500 includes preparing a training data set for the resolution mapping network training system 300, and the method 600 includes training the generative neural network algorithm to learn a mapping from a first resolution distribution to a target resolution distribution, wherein parameters of the generative neural network are adjusted according to a back-propagation algorithm using training image pairs including a 1D probe image and a corresponding ground truth 2D probe image. The training image pairs can be generated by scanning anatomical features of a patient using both a 1D and a 2D ultrasound probe device, as described below with reference to Figure 5The trained neural network model can be used by an image processing system (such as Figure 2 The image processing system 202) is deployed to execute Figure 7 One or more operations of the method 700 are used to map one or more ultrasound images from a first resolution distribution to corresponding resolution-mapped ultrasound images having a target resolution distribution.

[0026] Now see Figure 1 , which shows a schematic diagram of an ultrasound imaging system 100 according to an embodiment of the present disclosure. The ultrasound imaging system 100 includes a transmit beamformer 101 and a transmitter 102, which drives elements (e.g., transducer elements) 104 within a transducer array (referred to herein as a probe 106) to transmit pulsed ultrasound signals (referred to herein as transmit pulses) into a body (not shown). According to one embodiment, the probe 106 can be a one-dimensional transducer array probe. However, in some embodiments, the probe 106 can be a two-dimensional matrix transducer array probe. As further explained below, the transducer elements 104 can be made of piezoelectric material. When a voltage is applied to a piezoelectric crystal, the crystal physically expands and contracts, thereby emitting an ultrasonic spherical wave. In this way, the transducer elements 104 can convert the electronic transmit signal into an acoustic transmit beam.

[0027] After the elements 104 of the probe 106 transmit pulsed ultrasound signals into the body (of the patient), the pulsed ultrasound signals are backscattered from structures inside the body (such as blood cells or muscle tissue) to produce echoes that return to the elements 104. The echoes are converted into electrical signals or ultrasound data by the elements 104, and the electrical signals are received by the receiver 108. The electrical signals representing the received echoes pass through the receive beamformer 110, which outputs the ultrasound data. In addition, the transducer elements 104 may generate one or more ultrasound pulses based on the received echoes to form one or more transmit beams.

[0028] According to some embodiments, the probe 106 may include electronic circuitry to perform all or part of transmit beamforming and / or receive beamforming. For example, all or part of the transmit beamformer 101, transmitter 102, receiver 108, and receive beamformer 110 may be located within the probe 106. Throughout this disclosure, the terms "scan" or "scanning" may also be used to refer to the acquisition of data through the process of transmitting and receiving ultrasound signals. Throughout this disclosure, the term "data" may be used to refer to one or more data sets acquired by an ultrasound imaging system. In one embodiment, data acquired via the ultrasound system 100 may be used to train a machine learning model. A user interface 115 may be used to control the operation of the ultrasound imaging system 100, including input for controlling patient data (e.g., patient medical history), for changing scan or display parameters, for initiating probe repolarization sequences, and the like. The user interface 115 may include one or more of the following: a rotary element, a mouse, a keyboard, a trackball, hard keys linked to specific actions, soft keys configurable to control different functions, and / or a graphical user interface displayed on a display device 118.

[0029] The ultrasound imaging system 100 also includes a processor 116 to control the transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110. The processor 116 is in electronic communication (e.g., communicatively connected) with the probe 106. For the purposes of this disclosure, the term “electronic communication” can be defined to include both wired and wireless communication. The processor 116 can control the probe 106 to acquire data according to instructions stored on the processor’s memory, and / or the memory 120. The processor 116 controls which of the elements 104 are active and the shape of the beams transmitted from the probe 106. The processor 116 is also in electronic communication with the display device 118, and the processor 116 can process data (e.g., ultrasound data) into images for display on the display device 118. The processor 116 can include a central processor (CPU) according to one embodiment. According to other embodiments, the processor 116 can include other electronic components capable of performing processing functions, such as a digital signal processor, a field programmable gate array (FPGA), or a graphics board. According to other embodiments, the processor 116 can include multiple electronic components capable of performing processing functions. For example, the processor 116 can include two or more electronic components selected from the list of electronic components including: a central processor, a digital signal processor, a field programmable gate array, and a graphics board. According to another embodiment, the processor 116 can also include a complex demodulator (not shown) that demodulates RF data and generates raw data. In another embodiment, the demodulation can be performed earlier in the processing chain. The processor 116 is adapted to perform one or more processing operations according to a plurality of selectable ultrasound modalities on the data. In one example, the data can be processed in real-time during a scan session as the echo signals are received by the receiver 108 and transmitted to the processor 116. For the purposes of this disclosure, the term “real-time” is defined to include processes performed without any intentional delay. For example, embodiments can acquire images at a real-time rate of 7 to 20 frames / second. The ultrasound imaging system 100 can be capable of acquiring 2D data of one or more planes at a significantly faster rate. However, it should be understood that the real-time frame rate can depend on the length of time it takes to acquire each frame of data for display. Thus, the real-time frame rate can be slower when a relatively large amount of data is acquired. Thus, some embodiments can have a real-time frame rate that is significantly faster than 20 frames / second, while other embodiments can have a real-time frame rate that is lower than 7 frames / second. The data can be temporarily stored in a buffer (not shown) during a scan session and processed in a less real-time manner in real-time or offline operations. Some embodiments of the present invention can include multiple processors (not shown) to handle processing tasks handled by the processor 116 according to the example embodiments described above.For example, in the display of images, a first processor can be used to demodulate and decimate the RF signals, while a second processor can be used to further process the data (e.g., by augmenting the data as described further herein). It will be understood that other implementations can use different processor arrangements.

[0030] The ultrasound imaging system 100 can continuously acquire data at a frame rate of, for example, 10 to 30 Hz (e.g., 10 to 30 frames per second). Images generated from the data can be refreshed on the display device 118 at a similar frame rate. Other implementations can acquire and display data at different rates. For example, some implementations can acquire data at a frame rate of less than 10 Hz or greater than 30 Hz, depending on the size of the frames and the intended application. A memory 120 is included for storing frames of processed acquired data. In an exemplary implementation, the memory 120 has sufficient capacity to store at least several seconds of ultrasound data frames. The data frames are stored in a manner that facilitates retrieval according to their order of acquisition or time. The memory 120 can include any known data storage medium.

[0031] In various implementations of the present application, the processor 116 can process data through different mode dependent modules (e.g., B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastography, TVI, strain, strain rate, etc.) to form 2D or 3D data. For example, one or more modules can generate B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastography, TVI, strain, strain rate, and combinations thereof, etc. As one example, one or more modules can process color Doppler data, which can include traditional color blood flow Doppler, power Doppler, HD flow, etc. The image lines and / or frames are stored in memory and can include timing information indicative of the time at which the image lines and / or frames were stored in memory. These modules can include, for example, scan conversion modules for performing scan conversion operations to convert acquired images from beam space coordinates to display space coordinates. A video processor module can be provided that reads acquired images from memory and displays the images in real time as a procedure (e.g., ultrasound imaging) is performed on a patient. The video processor module can include a separate image memory, and ultrasound images can be written to the image memory for reading and display by the display device 118.

[0032] In various embodiments of the present disclosure, one or more components of the ultrasound imaging system 100 can be included in a portable handheld ultrasound imaging device. For example, the display device 118 and the user interface 115 can be integrated into an external surface of a handheld ultrasound imaging device, which can also include the processor 116 and the memory 120. The probe 106 can include a handheld probe in electronic communication with the handheld ultrasound imaging device to collect raw ultrasound data. The transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110 can be included in the same or different parts of the ultrasound imaging system 100. For example, the transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110 can be included in the handheld ultrasound imaging device, the probe, and combinations thereof.

[0033] The ultrasound scan can be performed using a ID ultrasound probe or a 2D ultrasound probe. As described above, a ID ultrasound probe includes a single row of transducers, which provides a focal region corresponding to the depth of the ID probe, where portions of the anatomical structure within a narrow height range of the focal depth are shown with high resolution (e.g., high height-wise resolution, or high resolution in the height direction, which is different from the resolution in the lateral and / or axial directions), while portions of the anatomical structure at heights progressively further away from the focal height are shown with progressively lower resolution (e.g., low height-wise resolution). In contrast, a 2D ultrasound probe includes an array of n rows of transducers, thereby providing n focal regions, each corresponding to a different depth of the 2D ultrasound probe, where portions of the anatomical structure within a narrow depth range from each focal region are shown with high resolution. Thus, images acquired via a 2D probe with a multi-row array of transducers show anatomical features with high resolution over a wider probe height range.

[0034] After performing the ultrasound scan, a two-dimensional data block including scan lines and their samples is generated for each row of transducers included in the ultrasound probe (e.g., one data block for a ID probe, or n data blocks for a 2D probe with n rows of transducers). After applying the back-end filters, a process called scan conversion is performed to transform the two-dimensional data block into a displayable bitmap image with additional scan information, such as depth, angle of each scan line, etc. During scan conversion, an interpolation technique is applied to fill in missing holes (i.e., pixels) in the resulting image. These missing pixels occur because each element of the two-dimensional block should typically cover many pixels in the resulting image. For example, in current ultrasound imaging systems, a bicubic interpolation is applied, which utilizes neighboring elements of the two-dimensional block. Thus, if the two-dimensional block is relatively small compared to the size of the bitmap image, the image after scan conversion will include areas of poor resolution or low resolution, particularly for areas that are large in depth.

[0035] If an ultrasound scan is performed using a 2D probe having n rows of transducers, n scan converted images will be generated, where each of the n images will show different regions of the image having high resolution, such that the n scan converted images can be combined into a single scan converted image in which anatomical features are shown at high resolution over a wider area of the image compared to a ID probe.

[0036] The ultrasound images acquired by the ultrasound imaging system 100 can be further processed. In some embodiments, the ultrasound images produced by the ultrasound imaging system 100 can be transmitted to an image processing system, where in some embodiments, the ultrasound images can be analyzed by using one or more machine learning models trained using the ultrasound images and corresponding ground truth images in order to assess image quality issues of the ultrasound images. As used herein, ground truth output refers to the expected or “correct” output based on a given input into a machine learning model. For example, if a machine learning model is being trained to improve resolution of a relevant portion of an ultrasound image, the ground truth output of the model when fed the input image is an ultrasound image in which the relevant portion is shown at high resolution.

[0037] Although described herein as separate systems, it should be understood that in some embodiments, the ultrasound imaging system 100 comprises the image processing system. In other embodiments, the ultrasound imaging system 100 and the image processing system can comprise separate devices. In some embodiments, the images produced by the ultrasound imaging system 100 can be used as a training dataset for training one or more machine learning models, where the machine learning models can be used to perform one or more steps of ultrasound image processing as described below.

[0038] Referring to Figure 2FIG. 2 shows an image processing system 202 in accordance with an embodiment. In some embodiments, the image processing system 202 is incorporated into the ultrasound imaging system 100. For example, the image processing system 202 can be provided in the ultrasound imaging system 100 as the processor 116 and the memory 120. In some embodiments, at least a portion of the image processing system 202 is provided at a device (e.g., an edge device, a server, etc.) that is communicatively coupled to the ultrasound imaging system via a wired connection and / or a wireless connection. In some embodiments, at least a portion of the image processing system 202 is provided at a separate device (e.g., a workstation) that can receive images from the ultrasound imaging system or from a storage device that stores images / data generated by the ultrasound imaging system. The image processing system 202 can be operatively / communicatively coupled to a user input device 232 and a display device 234. At least in some examples, the user input device 232 can comprise the user interface 115 of the ultrasound imaging system 100, and the display device 234 can comprise the display device 118 of the ultrasound imaging system 100. The image processing system 202 can also be operatively / communicatively coupled to an ultrasound probe 236. As discussed in further detail below, the ultrasound probe 236 can be a ID probe, or the ultrasound probe 236 can be a 2D probe.

[0039] The image processing system 202 includes a processor 204 configured to execute machine readable instructions stored in a non-transitory memory 206. The processor 204 can be single- or multi-core, and programs executing thereon can be configured for parallel or distributed processing. In some embodiments, the processor 204 can optionally include separate components distributed among two or more devices, which can be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of the processor 204 can be virtualized and executed by remotely accessible networked computing devices configured in a cloud computing configuration.

[0040] The non-transitory memory 206 can store a neural network module 208, a network training module 210, an inference module 212, and ultrasound image data 214. The neural network module 208 can include at least a deep learning model (e.g., a generative neural network or a generative adversarial network), as well as instructions for implementing the deep learning model to reconstruct ultrasound images acquired via a ID probe as images having a higher resolution over a wider height range typical of 2D probes, as described in greater detail below. The neural network module 208 can include trained and / or untrained neural networks, and can also include various data or metadata pertaining to the one or more neural networks stored therein.

[0041] The non-transitory memory 206 can also store a training module 210 including instructions for training one or more of the neural networks stored in the neural network module 208. The training module 210 can include instructions that, when executed by the processor 204, cause the image processing system 202 to perform one or more of the steps of the method 500 for generating a training dataset, the steps of the method 600 for training a neural network model using the training dataset, as discussed in further detail below with respect to Figure 5 and Figure 6 In some embodiments, the training module 210 includes instructions for implementing one or more gradient descent algorithms, applying one or more loss functions, and / or training routines for adjusting one or more of the neural networks of the neural network module 208.

[0042] The non-transitory memory 206 also stores an inference module 212 including instructions for testing new data with a trained deep learning model. The reconstruction and enhancement of ultrasound images with a trained deep learning model can be performed with the inference module 212 as discussed below. Figure 7 In particular, the inference module 212 can include instructions that, when executed by the processor 204, cause the image processing system 202 to perform one or more of the steps of the method 700, as discussed in further detail below.

[0043] The non-transitory memory 206 also stores ultrasound image data 214. The ultrasound image data 214 can include, for example, ultrasound images acquired via a ID ultrasound probe and images acquired via a 2D ultrasound probe. For example, the ultrasound image data 214 can store images acquired via a ID probe and images acquired via a 2D probe of the same anatomical feature of the same patient. In some embodiments, the ultrasound image data 214 can include multiple training sets generated as discussed at the method 500.

[0044] In some embodiments, the non-transitory memory 206 can include components disposed on two or more devices that can be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of the non-transitory memory 206 can include a networked storage device that is remotely accessible configured in a cloud computing configuration.

[0045] User input device 232 can include one or more of a touchscreen, a keyboard, a mouse, a trackpad, a motion-sensing camera, or other devices configured to enable a user to interact with and manipulate data within image processing system 202. In one example, user input device 232 can enable a user to select an ultrasound image for training a machine learning model, to indicate or label a location of an interventional device in ultrasound image data 214, or for further processing using a trained machine learning model.

[0046] Display device 234 can include one or more display devices utilizing almost any type of technology. In some embodiments, display device 234 can include a computer monitor and can display ultrasound images. Display device 234 can be combined in a shared housing with processor 204, non-transitory memory 206, and / or user input device 232, or can be a peripheral display device and can include a monitor, a touchscreen, a projector, or other display devices known in the art that can enable a user to view ultrasound images produced by an ultrasound imaging system and / or interact with various data stored in non-transitory memory 206.

[0047] It should be understood that Figure 2 The illustrated image processing system 202 is for illustration and not limitation. Another suitable image processing system can include more, less, or different components.

[0048] Referring to Figure 3 which shows an example of a resolution mapping network training system 300. Resolution mapping network training system 300 can be implemented by one or more computing systems, such as image processing system 202 of Figure 2 to train a resolution mapping network to learn a mapping from a first resolution distribution to a target resolution distribution. In one embodiment, resolution mapping network training system 300 includes a resolution mapping network 302 to be trained and a training module 304 that includes a training dataset that includes a plurality of image pairs split into training image pairs 306 and test image pairs 308. Training module 304 can be the same as or similar to training module 210 of image processing system 200. Figure 2

[0049] The plurality of training sets and the plurality of test sets can be selected to ensure that there is sufficient training data available to prevent overfitting, whereby resolution mapping network 302 learns a mapping that is not specific to a sample of the training set that is not present in the test set. As a non-limiting example, the number of training sets used is 10,000 and the number of test sets used is 1000.

[0050] ​Each of the training image pairs 306 and the test image pairs 308 includes an input image and a target image, wherein the input image is acquired via an ultrasound imaging system (e.g., Figure 1 The target image is acquired during a first examination of one or more anatomical structures of a patient by a 1D ultrasound probe 312 of an ultrasound imaging system 100, and the target image is acquired during a second examination of one or more anatomical structures of the patient by a 2D ultrasound probe 314 of the ultrasound imaging system. In one embodiment, the ultrasound operator can perform the second examination of the patient as soon as the first examination of the patient is completed, so as to minimize the duration between the first examination and the second examination, thereby ensuring that the anatomical structures being examined do not change between the first examination and the second examination. In addition, during the first examination, the position of the 1D ultrasound probe 312 can be adjusted to acquire input images via a mechanized, repeatable, automated process, and during the second examination, the position of the 2D ultrasound probe 314 can be adjusted to acquire corresponding target images via the same mechanized, repeatable, automated process. For example, during the first examination, the 1D ultrasound probe can be mechanically coupled to a device, such as a robotic arm, which adjusts the position of the 1D probe in one or more directions, so that when the 2D probe is mechanically coupled to the device during the second examination, the corresponding position of the 2D probe is adjusted in the one or more directions. Thus, a high correlation can be established between an input image acquired during a first examination (after a fixed time interval after the start of the first examination) via a 1D probe and a corresponding target image acquired during a second examination (after the same fixed time interval after the start of the second examination) via a 2D probe, relative to the position of the anatomical structure being examined. In one example, a set of ultrasound scanning parameters can be maintained between a first scan performed using the 1D probe and a second scan performed using the 2D probe. For example, the 1D probe can be adjusted to scan a first volume of an anatomical region using a first set of acquisition parameters (e.g., focus, depth, frequency, scan plane, aperture size, etc.), and the 2D probe can then be adjusted to scan the same first volume using the same first set of acquisition parameters, or vice versa. Furthermore, in order to generate a large training data set, multiple anatomical regions can be scanned using the 1D probe and the 2D probe. In this way, for training image pairs, a high correlation can be obtained between images acquired via the 1D probe and images acquired via the 2D probe.

[0051] Further, in some embodiments, the input images and the target images can be time-stamped, whereby a first input image acquired at the beginning of the first exam is assigned a time of 0, and a first target image acquired at the beginning of the second exam is assigned a time of 0, such that any subsequent input images acquired in the first exam will be associated with subsequent target images acquired in the second exam having the same time stamp. Thus, pairs of images comprising 1D probe input images and 2D probe target images of the same anatomical feature of the same patient can be effectively obtained via an automated process. Reference is made to FIG. 3 below for further details. Figure 5 An exemplary method for generating the training data is described in further detail.

[0052] In one embodiment, the input images and the target images can be pre-processed by the image processor 320 prior to generating the pairs of images to be included in the training image pairs 306 or the test image pairs 308. For example, the input images can be shifted in one direction in order to adjust the position of the examined anatomical structure relative to the frame of reference of the images acquired via the 1D probe to a position that matches the position of the examined anatomical structure relative to the frame of reference of the images acquired via the 2D probe in the target images.

[0053] Thus, pairs of images comprising an input image (e.g., one of the 1D probe images 316 generated by the 1D ultrasound probe 312) and a corresponding target image (e.g., one of the 2D probe images 318 generated by the 2D ultrasound probe 314) are generated, wherein the target image and the input image have the same time stamp. Once the pairs of images are generated, the pairs of images can be assigned to the training image pairs 306 dataset or the test image pairs 308 dataset. In one embodiment, the pairs of images can be randomly assigned to the training image pairs 306 dataset or the test image pairs 308 dataset in a pre-established ratio. For example, the pairs of images can be randomly assigned to the training image pairs 306 dataset or the test image pairs 308 dataset such that 90% of the generated pairs of images are assigned to the training image pairs 306 dataset and 10% of the generated pairs of images are assigned to the test image pairs 308 dataset. Alternatively, the pairs of images can be randomly assigned to the training image pairs 306 dataset or the test image pairs 308 dataset such that 85% of the generated pairs of images are assigned to the training image pairs 306 dataset and 15% of the generated pairs of images are assigned to the test image pairs 308 dataset. It should be appreciated that the examples provided herein are for illustrative purposes, and that the pairs of images can be assigned to the training image pairs 306 dataset or the test image pairs 308 dataset via different processes and / or in different ratios without departing from the scope of the present disclosure.

[0054] The resolution mapping network training system 300 can be configured to train the resolution mapping network 302 according to a training process 400, as illustrated in FIG. 4. The training process 400 can be performed by the image processor 320, the processor 302, or any other suitable processor. Figure 6The one or more operations of the method 600 can be implemented to train the resolution mapping network 302 to learn a mapping from a resolution distribution characterized by the ultrasound image 316 to a resolution distribution characterized by the target ultrasound image 318. The resolution mapping network 302 is configured to receive the training image pairs 306 from the training module 304 and iteratively adjust one or more parameters of the resolution mapping network 302 in order to minimize an error function based on an evaluation of a difference between the input image and the target image included in each image pair of the training image pairs 306. In one embodiment, the error function can be a per-pixel loss function in which the difference between the input image and the target image is compared on a pixel-by-pixel basis and summed. In another embodiment, the error function can be a perceptual loss function in which features extracted from the images are compared (e.g., in which the loss is defined by an average of squared errors between all pixels). In other embodiments, the loss function can be a least-squares loss function or a Wasserstein loss function. It should be appreciated that the examples provided herein are for illustrative purposes and that other types of loss functions can be used without departing from the scope of the present disclosure.

[0055] In some embodiments, the resolution mapping network 302 can include a generative neural network. In some embodiments, the resolution mapping network 302 can include a generative adversarial network. In some embodiments, the resolution mapping network 302 can include a generative neural network having a U-net architecture. In some embodiments, the resolution mapping network 302 can include a generative neural network having a variational autoencoder architecture including a first encoding portion that compresses information of an ultrasound image into a compressed representation / encoding and a decoder portion that decompresses the compressed representation / encoding into a variation of the ultrasound image. In some embodiments, the encoding portion includes one or more convolutional layers, which in turn include one or more convolutional filters (e.g., a convolutional neural network architecture). The convolutional filters can include a plurality of weights, where the values of the weights are learned during a training process, such as the training method of FIG. 6. Figure 6 The convolutional filters can correspond to one or more visual features / patterns, enabling the resolution mapping network 302 to recognize and extract features from the ultrasound image 316. The encoding portion can also include one or more down-sampling operations and / or one or more activation functions. The decoding portion can include one or more up-sampling and / or deconvolution operations that enable the compressed representation of the ultrasound image 316 to be reconstructed into an image having the same dimensions as the ultrasound image 316.

[0056] The resolution mapping network training system 300 can include a verifier 310 that verifies the performance of the resolution mapping network 302 from the test image pairs 308. The verifier 310 can take as input the trained or partially trained resolution mapping network 302 and the dataset of test image pairs 308, and can output an assessment of the performance of the trained or partially trained resolution mapping network 302 based on the dataset of test image pairs 308. In one embodiment, the assessment of the performance of the trained or partially trained resolution mapping network 302 can be determined based on an average of the minimum error rates achieved on each image pair of the test image pairs 308, where the minimum error rate is a function of one or more difference values between the image output by the trained or partially trained resolution mapping network 302 as an image of the input image in the image pair and the target ultrasound image in the image pair. In another embodiment, the assessment of the performance of the trained or partially trained resolution mapping network 302 can include a quality assessment of the ultrasound images output by the trained or partially trained resolution mapping network 302, where the quality assessment is determined by one or more pre-established target variables such as lateral, axial, and / or high resolution of the output images compared to the target images. In other embodiments, the assessment of the performance of the trained or partially trained resolution mapping network 302 can include a combination of the average minimum error rate and the quality assessment, or a different function of the minimum error rates achieved on each image pair of the test image pairs 308 and / or one or more quality assessments, or another factor for assessing the performance of the trained or partially trained resolution mapping network 302. It should be understood that the examples provided herein are for illustrative purposes and other error functions, error rates, quality assessments, or performance assessments can be included without departing from the scope of the present disclosure.

[0057] For example, the partially trained resolution mapping network 302 can be validated with a test dataset of 50 image pairs 308, where each of the 50 image pairs 308 includes an input image of a subject’s spleen acquired via a ID ultrasound probe, and a target image of the spleen acquired via a 2D ultrasound probe adjusted to the same position on the subject as the ID ultrasound probe. The input image acquired via the ID ultrasound probe can have a first height resolution profile characterized by a narrow high resolution region, and the target image acquired via the 2D ultrasound probe can have a second height resolution profile characterized by a wide high resolution region. The validator 310 can feed the input image into the partially trained resolution mapping network 302 and receive an output image including a reconstruction of the input image having an expanded resolution profile (e.g., a height resolution profile in which the output image has a greater high resolution region than the high resolution region of the input image). The validator 310 can then compare the output image of the spleen generated by the partially trained resolution mapping network 302 to the target image of the spleen from the relevant image pair, and output a value indicating a degree of similarity between the output image and the target image. The degree of similarity can be determined by a comparison of one or more measurements between anatomical features identified in the two images, or a contrast difference or another similar metric between the images or portions of the images. In one embodiment, the degree of similarity can be expressed as a percentage (e.g., a 90% similarity score), and the validator 310 can return a binary result of 1 indicating that the degree of similarity exceeds a threshold similarity percentage (e.g., 85%), and that the partially trained resolution mapping network 302 has successfully mapped the input image having the first resolution profile to an image (e.g., the output image) having the second resolution profile. Alternatively, the validator 310 can return a binary result of 0 indicating that the degree of similarity does not exceed the threshold similarity percentage (e.g., 95%), and that the partially trained resolution mapping network 302 has failed to successfully map the input image having the first resolution profile to an image (e.g., the output image) having the second resolution profile. The validator 310 can validate the partially trained resolution mapping network 302 based on each of the test image pairs 308, and average the results of the similarity assessments performed for each of the test image pairs 308 to determine an overall validation score. If the overall validation score exceeds a threshold (e.g., 0.8), then the partially trained resolution mapping network 302 passes validation, whereby the resolution mapping network 302 has been fully trained and can be used to map new ultrasound images acquired by the ID probe to a resolution mapped image having an expanded height resolution profile.Alternatively, if the overall validation score does not exceed the threshold (e.g., 0.8), the partially trained resolution mapping network 302 fails validation, indicating that the resolution mapping network 302 can not be suitable for mapping new ultrasound images acquired by the ID probe to resolution mapped images having an extended high resolution profile. In other embodiments, the validator 310 can output a similarity score or percentage instead of a binary value, and can average the similarity score or percentage for each image pair to determine the overall validation score. It should be appreciated that the examples provided herein are for illustrative purposes, and other processes and / or functions can be used to validate the performance of the partially trained resolution mapping network 302 without departing from the scope of the present disclosure.

[0058] The resolution mapping network training system 300 can include an inference module 322 that includes the validated resolution mapping network 324 that has been validated by the validator 310 as described above. The inference module 322 can also include instructions for deploying the validated resolution mapping network 324 to generate a set of resolution mapped images 328 from a set of new ID probe images 326. The resolution mapped ultrasound images 328 can include the same number of images as the new ID probe images 326, where for each image in the new ID probe images 326, a corresponding resolution mapped ultrasound image 328 is produced such that there is a one-to-one correspondence between the new ID probe images 326 and the resolution mapped ultrasound images 328. In this way, the resolution mapping network training system 300 enables the resolution mapping network 302 to learn a mapping from a first resolution profile to a target resolution profile.

[0059] Turning to Figure 4A FIG. 4 shows an architectural diagram of the CNN 400. The CNN 400 can take as input two-dimensional ultrasound images acquired via an ID ultrasound probe and generate as output two-dimensional ultrasound images having a wider resolution profile, similar to images acquired via a 2D ultrasound probe. The CNN 400 is representative of a U-net architecture, which can be divided into an encoding portion (downward portion, elements 402b-430) and a decoding portion (upward portion, elements 432-456a). The CNN architecture 400 includes a series of mappings from a pixel representation of an input image 402b, which can be received by an input layer, through a plurality of feature map images, to a pixel representation of an output image 456b, which can be produced by an output layer 456a.

[0060] Various elements comprising the CNN architecture 400 are labeled in the legend 458. As indicated by the legend 458, the CNN architecture 400 comprises a plurality of feature maps (and / or replicated feature maps), where each feature map can receive input from a previous feature map and can transform / map the received input into an output to produce a next feature map. Each feature map can comprise a plurality of neurons, where in some embodiments each neuron can receive input from a subset of neurons of a previous layer / feature map and can compute a single output based on the received input, where the output can be propagated to a subset of neurons in a next layer / feature map. The feature maps can be described using spatial dimensions such as length and width, which can correspond to features of each pixel of an input image, where the dimensions refer to the number of neurons comprising the feature map (e.g., the number of neurons along the length and the number of neurons along the width of a specified feature map).

[0061] In some embodiments, the neurons of a feature map can compute an output by performing a dot product of received inputs using a set of learned weights (each set of learned weights can be referred to herein as a filter), where each received input has a unique corresponding learned weight, where the learned weight is learned during training of the CNN.

[0062] The transformations / mappings performed by each feature map are indicated by arrows, where each type of arrow corresponds to a different transformation, as indicated by the legend 458. Rightward pointing solid black arrows indicate 3x3 convolutions with a stride of 1, where outputs from a 3x3 grid of feature channels of an immediately previous feature map are mapped to a single feature channel of the current feature map. Each 3x3 convolution can be followed by an activation function, where in one embodiment the activation function comprises a rectified linear unit (ReLU).

[0063] Downward pointing hollow arrows indicate 2x2 max pooling, where a maximum value from a 2x2 grid of feature channels is propagated from an immediately previous feature map to a single feature channel of the current feature map, resulting in an 8-fold reduction in spatial resolution of the immediately previous feature map. In some examples, this pooling occurs independently for each feature.

[0064] Upward pointing hollow arrows indicate 2x2 up-convolutions, which comprise mapping outputs from a single feature channel of an immediately previous feature map to a 2x2 grid of feature channels in the current feature map, resulting in an 8-fold increase in spatial resolution of the immediately previous feature map.

[0065] The right-pointing dashed arrow indicates the copying and cropping of a feature map for concatenation with another, later-appearing feature map. Cropping enables the dimensions of the copied feature map to match the dimensions of the feature channels to be concatenated with the copied feature map. It should be understood that no cropping may be performed when the size of the first feature map being copied and the size of the second feature map to be concatenated with the first feature map are equal.

[0066] A right-pointing arrow with a hollow elongated triangular head indicates a 1×1 convolution, in which each feature channel in the immediately previous feature map is mapped to a single feature channel of the current feature map, or in other words, in which a 1-to-1 mapping of feature channels occurs between the immediately previous feature map and the current feature map.

[0067] A right-pointing arrow with a V-shaped head indicates the incorporation of Gaussian noise into the received input feature map.

[0068] The right-pointing arrow with a bowed hollow head indicates a batch normalization operation, in which the distribution of activations of the input feature map is normalized.

[0069] Right-pointing arrows with short hollow triangular heads indicate a dropout operation, where random or pseudo-random dropping of input neurons (and their inputs and outputs) occurs during training.

[0070] In addition to the operations indicated by the arrows in legend 458, CNN architecture 400 also includes a solid filled rectangle corresponding to a feature map, where the feature map includes height (e.g., Figure 4A The length from top to bottom shown corresponds to the y spatial dimension in the xy plane), width ( Figure 4A , assuming the magnitude is equal to the height, corresponding to the x spatial dimension in the xy plane) and depth (as Figure 4A , the length from left to right as shown, corresponds to the number of features in each feature channel). Similarly, CNN architecture 400 includes a hollow (unfilled) rectangle corresponding to the copied and cropped feature map, where the copied feature map includes a height (e.g., Figure 4A The length from top to bottom shown corresponds to the y spatial dimension in the xy plane), width ( Figure 4A , assuming the magnitude is equal to the height, corresponding to the x spatial dimension in the xy plane) and depth (as Figure 4A The length from left to right shown corresponds to the number of features in each feature channel).

[0071] From an input image volume 402b (also referred to herein as an input layer), data corresponding to images from a ID ultrasound probe can be input (such as shown in input image 402a) and mapped to a first set of features. In some embodiments, the input data is acquired by scanning a target anatomical feature of a patient with a ID ultrasound probe in accordance with Figure 5 the method 500.

[0072] The output layer 456a can comprise an output layer of neurons, where each neuron can correspond to a pixel of an ultrasound image, and where the output of each neuron can correspond to a predicted anatomical feature (or lack of anatomical feature) in a given location within the input ultrasound image. For example, the output of a neuron can indicate whether a corresponding pixel of an output ultrasound image is part of a spleen or not part of a spleen. As shown, the output image 456b can show one or more features included in the input image 402a. Figure 4A

[0073] In this way, the CNN architecture 400 can enable an input ultrasound image to be mapped to a predicted ultrasound image having a wider range of high resolution regions. The CNN architecture 400 illustrates the feature map transformations that occur as the input image volume is propagated through the layers of neurons of the convolutional neural network to produce a predicted output image.

[0074] The weights (and biases) of the convolutional layers in the CNN 400 are learned during training, which will be discussed in detail below with reference to Figure 6 Briefly, a loss function is defined to reflect the difference between the output image predicted by the CNN 400 and a corresponding ground truth ultrasound image acquired via a 2D ultrasound probe. For example, the loss function can be a perceptual loss function, where a comprehensive introduction of perceptual loss can be used to control the appearance of artifacts. The loss can be backpropagated through the layers of the neural network to update the weights (and biases) of the convolutional layers. The neural network 400 can be trained using a plurality of training image pairs, which include ultrasound images acquired from an examination of a patient using a ID probe and corresponding ground truth images acquired from an examination of the patient using a 2D probe.

[0075] It will be appreciated that the present disclosure encompasses neural network architectures that include one or more regularization layers, including batch normalization layers, dropout layers, Gaussian noise layers, and other regularization layers known in the machine learning art, which can be used during training to mitigate overfitting and improve training efficiency, while reducing training time. The regularization layers are used during training of the CNN and are disabled or removed during post-training implementations of the CNN. These layers can be interspersed between the layers / feature maps shown, or can replace one or more of the layers / feature maps shown. Figure 4A illustrated layers / feature maps.​

[0076] It will be appreciated that, Figure 4A The architecture and configuration of the illustrated CNN 400 is for illustration and not limitation, and other suitable neural networks (such as ResNet, recurrent neural networks, generalized regression neural networks (GRNN), etc.) can be used herein to predict an ultrasound image having an extended range of high resolution regions from an ultrasound image having a more limited range of high resolution regions.

[0077] Referring to Figure 4B which illustrates an example of mapping an input image 482 having a first resolution profile to a resolution-mapped ultrasound image 484 having a second resolution profile by a trained resolution-mapping network 486. The trained resolution-mapping network 486 can be the same as or similar to the validated resolution-mapping network 324 of Figure 3 . The input image 482 can include an ultrasound image acquired by an ultrasound imaging system such as the ultrasound imaging system 100 of Figure 1 . The image acquired by the ultrasound imaging system can not satisfy the image appearance preferences of the clinician, and in response, the clinician can employ a method such as the method 700 to map the input ultrasound image 482 to the resolution-mapped ultrasound image 484, where the resolution-mapped ultrasound image 484 satisfies the clinician’s preferences regarding the resolution of the image in the near field and the far field. (For example, the resolution profile of the resolution-mapped ultrasound image 484 matches a target resolution profile that defines the image appearance preferred by the clinician.) Thus, the input ultrasound image 482 and the resolution-mapped ultrasound image 484 both include substantially the same anatomical content and are of the same anatomical region of the same patient, however, the input ultrasound image 482 and the resolution-mapped ultrasound image 484 include different appearance characteristics (i.e., different resolution profiles). The trained resolution-mapping network 486 includes a learned mapping from the first resolution domain to the target resolution domain.

[0078] Referring to Figure 5 which illustrates a flowchart of a method 500 for generating training data for training a resolution-mapping network (such as the resolution-mapping network 302 of the resolution-mapping network training system 300 of Figure 3 and / or the CNN 400 of FIG. 4), where the training data includes a set of image pairs. Each image pair of the set of image pairs can include an ultrasound image (e.g., an input image) of one or more anatomical features of a subject acquired via a 1D ultrasound probe at a probe location, and a corresponding ultrasound image (e.g., a ground truth image) of the one or more anatomical features of the subject acquired via a 2D ultrasound probe at the probe location. Using ultrasound images acquired by the ultrasound imaging system 100 of Figure 1 , the method 500 can be implemented as Figure 3resolution mapping network training system 300 and / or Figure 2 of the image processing system 202. In one embodiment, some operations of the method 500 can be stored in a non-transitory memory and executed by a processor 206 (such as, Figure 2 of the image processing system 202. In some embodiments, one or more training image pairs can be stored in a training dataset of the image processing system 202. Figure 2 of the image processing system 202. In some embodiments, one or more training image pairs can be stored in a training dataset of the image processing system 202.

[0079] The training data can include abnormal sample ultrasound images collected from subjects that can have one or more conditions, as well as sample ultrasound images of healthy tissue and / or sample ultrasound images collected from healthy subjects. For example, ultrasound images collected for training can include images of organs that are enlarged, swollen, and / or otherwise deformed, or images of anatomical features that are not present in healthy subjects (such as tumors, growths, scar tissue, etc.). In one embodiment, a process can be followed during preparation of the training data to collect images from a large number of subjects having different characteristics (e.g., age, gender, etc.) and / or different health levels and / or different normal / abnormal levels of anatomical structures. In other embodiments, a different process can be followed during preparation of the training data to collect images from a selected group of subjects, where the selected group of subjects have the same characteristic(s). For example, images can be collected from females (e.g., uteruses of pregnant women) but not males, or images can be collected from a group of subjects that are older or younger than a threshold age (e.g., children or infants). It should be understood that the examples provided herein are for illustrative purposes, and other criteria can be used to generate the training data.

[0080] The method 500 begins at 502, where the method 500 includes scanning a subject with a ID ultrasound probe (e.g., Figure 3 of the resolution mapping network training system 300). The images collected with the ID probe can be performed based on a first set of scan parameters for each anatomical feature or each set of anatomical features, including, for example, scan plane, scan frequency, probe position, focal point, aperture size, and scan depth.

[0081] In one example, method 500 may include scanning a uterus of a pregnant woman, whereby ultrasound images may be acquired by adjusting the position of a 1D ultrasound probe on the abdomen of the pregnant woman. The ultrasound images may show features of the uterus at different depths, wherein features of the uterus shown at a depth corresponding to the focus of a lens of the 1D ultrasound probe may be displayed at high resolution, while features of the uterus shown in the near field and / or far field may be displayed at a lower resolution. Scanning the uterus may involve acquiring a target number of ultrasound images (e.g., corresponding to the duration of the ultrasound examination), wherein the number of ultrasound images corresponds to the total number of image pairs in the training data. In some embodiments, at 502, the ultrasound images may be acquired by multiple ultrasound examinations, which may be performed, for example, on multiple objects and / or anatomical features of the objects so as to generate a sufficient number of image pairs in the training data.

[0082] At 504, method 500 includes selecting images at predetermined time intervals for a plurality of images acquired via a 1D ultrasound probe and storing them in a database, wherein the images are accessible based on their timestamps. For example, images (e.g., frames) may be selected from a stream of ultrasound images acquired via a 1D ultrasound probe at one-second intervals, such that 60 image frames are selected from a one-minute examination. Alternatively, images may be selected from a stream of ultrasound images acquired via a 1D ultrasound probe at 10-second intervals (e.g., if a greater difference between ultrasound images than between ultrasound images acquired per second is desired), or all image frames from the stream of ultrasound images acquired via a 1D ultrasound probe may be selected (e.g., if a data set size is desired to be maximized).

[0083] A timestamp may be associated with each selected image, the timestamp corresponding to the time at which each selected image was acquired relative to the start of the examination. As a non-limiting example, an image acquired 10.27 seconds after the start of the examination may receive a timestamp of 10.27. In one embodiment, a timestamp may be automatically associated with an image during acquisition by the ultrasound imaging system. In other embodiments, a timestamp may be associated with the image selected at 504 as part of an image processing phase (e.g., by Figure 2 The processor 204 of the image processing system 202 and / or by Figure 3 The selected image can be stored in a database where it can be accessed during later operations of the method 500. For example, the selected image can be stored in a relational database table with a timestamp for the selected image stored in a field of a row of the relational database table and the selected image stored in another field of the row of the relational database table so that the selected image can be retrieved by a processor (e.g., Figure 2the selected image based on the timestamp of the selected image from the database.

[0084] At 506, the method 500 includes scanning the subject scanned at 502 with a 2D ultrasound probe. The second set of scan parameters for acquiring images with the 2D probe can correspond to the first set of scan parameters for the ID probe. Specifically, for each anatomical feature or each set of anatomical features scanned with the ID probe, a corresponding image can be acquired with the 2D probe. The corresponding image acquired via the 2D probe can be acquired with the second set of scan parameters whose values are highly correlated with the values of the first set of scan parameters used to acquire the images acquired via the ID probe. For example, during scanning with the 2D probe, for each anatomical feature, the second set of scan parameters including probe position, scan frequency, scan depth, and scan plane can be adjusted to match the first set of scan parameters used for the acquisition with the ID probe.

[0085] For example, if the uterus of a pregnant woman is scanned at 502, the method 500 can include scanning the uterus of the pregnant woman at 506 with a 2D ultrasound probe, whereby the ultrasound images can be acquired by adjusting the position of the 2D ultrasound probe on the abdomen of the pregnant woman. Moreover, the position of the 2D ultrasound probe on the abdomen of the pregnant woman can be the same or substantially similar to the position of the ID ultrasound probe on the abdomen of the pregnant woman used to scan the subject at 502. In one embodiment, as described above with reference to Figure 3 The position of the ID ultrasound probe during the first examination can be adjusted so as to acquire ultrasound images via a mechanized, repeatable automatic process that can be reproduced during the second examination with the 2D ultrasound probe on the abdomen of the pregnant woman. For example, the position of the ID ultrasound probe during scanning of the subject at 502 and the position of the 2D ultrasound probe during scanning of the subject at 506 can be adjusted via a device that can include a robotic arm, whereby the position of the 2D ultrasound probe during scanning of the subject at 506 can be adjusted to the position of the ID ultrasound probe during scanning of the subject at 502.

[0086] The ultrasound images acquired via the 2D probe can show features of the uterus at different depths, where the features of the uterus shown at different depths can be shown with a similar high resolution as compared to the images acquired by the ID ultrasound probe for the reasons described above. Scanning the uterus at 506 can involve acquiring a target number of ultrasound images (e.g., corresponding to the duration of the ultrasound examination), where the target number of ultrasound images corresponds to the target number of ultrasound images acquired at 502 (e.g., the total number of pairs of images in the training data).

[0087] As described above in connection with 502, ultrasound images can be acquired at 506 by a plurality of ultrasound examinations performed on a plurality of subjects and / or anatomical features of subjects, where the number of ultrasound examinations performed, the number of subjects, and / or the number of anatomical features of subjects corresponds in a one-to-one manner to the number of ultrasound examinations performed, the number of subjects, and / or the number of anatomical features of subjects during the acquisition of the ID ultrasound images at 502. Thus, a high degree of correlation is established between the input images (e.g., input images to be used to train the resolution mapping network) acquired via the ID probe during the first examination (after a fixed time interval after the start of the first examination) and the corresponding target images (e.g., target images to be used to train the resolution mapping network) acquired via the 2D probe during the second examination (after the same fixed time interval after the start of the second examination) with respect to the position of the anatomical structure being examined. The input images and the target images can then be paired into training image pairs and added to the training dataset.

[0088] At 508, the method 500 includes, for the plurality of images acquired via the 2D ultrasound probe, selecting images at the established predetermined time interval used to select the ID probe images (e.g., input images) at 504 and storing them in the database used to store the selected ID probe images, where the images can be accessed based on their timestamps as described above.

[0089] At 510, the method 500 includes constructing a training dataset. In one embodiment, constructing the training dataset includes, at 512, accessing the ID probe images and the corresponding 2D probe images from the database by reference to the timestamps. For example, the processor 204 of the image processing system 202 of Figure 2 and / or the image processor 320 of the resolution mapping network training system 300 of Figure 3 can iteratively retrieve a plurality of ID probe images from the database. For each ID probe image (e.g., input image) retrieved, the processor can select the timestamp of the ID probe image and use the timestamp to retrieve a 2D probe image (e.g., ground truth image) from the database that corresponds to the same subject and anatomical feature as the ID probe image. At 514, the method 500 includes pairing the ID probe image and the corresponding ground truth 2D probe image as a training image pair. At 516, the method 500 includes storing the training image pair to a training dataset or a test dataset. As described above in connection with Figure 3 when generating the image pair, the image pair can be assigned to a training set (e.g., the training image pairs 306 of the resolution mapping network training system 300 of Figure 3 or a test set (e.g., the test image pairs 308 of the resolution mapping network training system 300 of Figure 3resolution mapping network training system 300). In one embodiment, the image pairs can be randomly assigned to the training data set or the test data set at a pre-established ratio.

[0090] As described above, generating the training data set and / or the test data set includes, for each subject, acquiring a plurality of scan images acquired via the ID probe, and subsequently acquiring a plurality of corresponding scan images acquired via the 2D probe, and associating each ID probe image with its corresponding 2D probe image based on a timestamp. In some embodiments, the ID scan and the corresponding 2D scan can be performed alternatively. For example, a first portion of a body can be scanned with the ID probe based on ID scan parameters to obtain a first ID probe image, and before proceeding to a next portion of the body, the first portion of the body can be scanned with the 2D probe based on 2D scan parameters (probe position, depth, plane, aperture, focus, and frequency) corresponding to the ID probe scan parameters to obtain a first 2D probe image corresponding to the first ID probe image. The first ID probe image and the first 2D probe image comprise a first image pair that can be used for training or validation. In this way, a plurality of image pairs (for training or validation) can be generated. For example, a robotic arm can be fitted with the ID probe and can initiate an ID scan to obtain an ID probe image, the ID scan starting at a first location on the body and ending at a second location to scan an anatomical feature or a set of features or a portion of anatomical features. While the image is being acquired with the ID probe, the robotic arm can be repositioned to the first location and fitted with the 2D probe. In some examples, the probe used for the ID scan can be changed to 2D mode. For example, the ultrasound probe (or, referred to herein as ultrasound transducer or transducer) can be a 2D matrix array probe, and the ID probe image can be obtained by using a linear array or a first number of rows of the 2D matrix array probe, and the 2D probe image can be obtained by using a second number of rows of the 2D matrix array, where the first number of rows is less than the second number of rows. The 2D scan can be performed based on the same scan parameters (probe position, depth, plane, aperture, focus, and frequency) as the scan parameters used for the ID scan to obtain images from the first location to the second location. The so-obtained ID probe image and 2D probe image can be used as a training pair or a validation pair.

[0091] Referring to Figure 6 which shows a system for training a resolution mapping network, such as Figure 34 ). In one embodiment, the resolution mapping network can be a deep neural network having multiple hidden layers. In one embodiment, the resolution mapping network can be a convolutional neural network, such as a convolutional autoencoder network (CAE). It should be understood that the examples provided herein are for illustrative purposes only, and that any type of neural network can be used with the method 600 without departing from the scope of this disclosure.

[0092] The training data used in method 600 may include the training data according to the above reference Figure 5 The process of method 500 selects and stores a set of image pairs, the set of image pairs including a 1D ultrasound image of one or more anatomical features of a subject acquired at a probe position (e.g., an input image), and a corresponding 2D ultrasound image of the one or more anatomical features of the subject acquired at the probe position (e.g., a ground truth image). Method 600 may be implemented as Figure 3 Resolution mapping network training system 300 and / or Figure 2 In one embodiment, one or more operations of method 600 may be stored in non-transitory memory and executed by a processor such as a processor. Figure 2 The image processing system 202's non-volatile memory 206 and processor 204) executes.

[0093] The method 600 begins at operation 602, where the method 600 includes receiving a training image pair from a training set (e.g., including an input image acquired via a 1D ultrasound probe and a ground truth image acquired via a 2D ultrasound probe). In one embodiment, the training set may be stored in a training module of an image processing system, such as Figure 2 The training module 210 of the image processing system 202. In other embodiments, the training image pairs may be acquired via a communication coupling between the image processing system and an external storage device, such as via an Internet connection to a remote server.

[0094] At 604, method 600 includes inputting an input image of the training image pair acquired via a 1D ultrasound probe into an input layer of a resolution mapping network. In some embodiments, the input image is input into an input layer of a CNN (such as CNN 400 of FIG. 4 ). In some embodiments, each pixel intensity value of the input image can be input into a different neuron of the input layer of the resolution mapping network.

[0095] At 606, the method 600 includes receiving an output image from the resolution mapping network. For example, the resolution mapping network can map an input image to an output image by propagating the input image from an input layer, through one or more hidden layers, until reaching an output layer of the resolution mapping network. In some embodiments, the output of the resolution mapping network includes values of a 2D matrix, where each value corresponds to a different intensity of a pixel of the input image, and where the different intensities of each pixel of the output image generate a reconstruction of the input image, where the resolution of one or more regions of the output image exceeds the resolution of the one or more regions of the input image.

[0096] At operation 608, the method 600 includes calculating a difference between the output image of the resolution mapping network and a target image of the training image pair. For example, the difference between the output image of the resolution mapping network and the target image (ground truth image) of the training image pair (e.g., the training image pair) can be calculated by determining a difference between the intensity of each pixel of the output image and the intensity of each corresponding pixel in the target image, and summing the differences for all pixels of the output image and the target image.

[0097] At operation 610, the weights and biases of the resolution mapping network are adjusted based on the difference between the output image and the ground truth image from the relevant data pair. The difference (or loss) as determined by the loss function can be backpropagated through the neural learning network to update the weights (and biases) of the convolutional layers. In some embodiments, the backpropagation of the loss can occur according to a gradient descent algorithm, where the gradient (first derivative or approximation of the first derivative) of the loss function is determined for each weight and bias of the deep neural network. Each weight (and bias) of the resolution mapping network is then updated by adding the negative of the product of the gradient determined (or approximation) for the weight (or bias) and a predetermined step size. The method 600 can then end. It should be noted that the method 600 can be repeated until the weights and biases of the resolution mapping network converge, or for each iteration of the method 600, the rate of change of the weights and / or biases of the deep neural network is below a threshold value.

[0098] Although not described in the method 600, it should be understood that to avoid overfitting, the training of the resolution mapping network can be periodically interrupted to validate the performance of the resolution mapping network based on a test set including test image pairs. The test image pairs can be similar to the training image pairs, but with different intensities of the pixels of the input images. For example, the test image pairs can be generated by the same process as the training image pairs, but with different intensities of the pixels of the input images. The test image pairs can be used to validate the performance of the resolution mapping network by calculating a difference between the output image of the resolution mapping network and a target image of the test image pair, and determining whether the difference is below a threshold value. If the difference is below the threshold value, the training of the resolution mapping network can continue. If the difference is not below the threshold value, the weights and biases of the resolution mapping network can be adjusted, and the training of the resolution mapping network can continue. Figure 5The method 600 may be generated as described in method 500 and may be randomly drawn from a larger training dataset. In one embodiment, training of the resolution mapping network may be terminated when the performance of the resolution mapping network relative to a test set of image pairs converges (e.g., when the error rate on the test set converges to a minimum). In this manner, the method 600 enables the resolution mapping network to be trained to generate a reconstruction of an input image that includes more regions with high resolution and / or uniformly high resolution throughout the reconstructed image.

[0099] Now refer to Figure 7 , which shows a method for using an ultrasound imaging system such as Figure 1 Flowchart of a method 700 for generating an ultrasound image using an ultrasound system. The ultrasound imaging system may include a processor such as Figure 2 During the operation mode of the ultrasound imaging system, a resolution mapping network (such as Figure 3 The resolution mapping network of the system 300 and / or the CNN 400 of FIG. 4 is trained to generate ultrasound images, wherein the ultrasound images have a higher resolution distribution, as described below. In one embodiment, the resolution mapping network can be a deep neural network such as a convolutional neural network having multiple hidden layers, which is based on Figure 6 The process described in method 600 is based on a training data set (relative to Figure 5 Method 700 can be implemented as Figure 3 Resolution mapping network training system 300 and / or Figure 2 In one embodiment, one or more operations of method 700 may be stored as executable instructions in a non-transitory memory (e.g., Figure 2 In addition, the method 700 can be deployed as part of an inference module, such as Figure 2 The inference module 212 of the image processing system 202 and / or Figure 3 The resolution mapping network training system 300 includes an inference module 322 .

[0100] The method 700 begins at operation 702, where the method 700 includes determining an operating mode, a processing mode, and a type of transducer to be used during a scanning operation using an ultrasound system. The operating mode may be B-mode, M-mode, Doppler mode, color M-mode, spectral Doppler, elastography, TVI, strain or strain rate, etc. The processing mode may be an artificial intelligence-based image processing mode that deploys a neural network algorithm, such as Figure 4BThe trained resolution mapping algorithm 486 can be a trained AI algorithm that is trained to map a resolution of a 1D transducer to a resolution of a 2D transducer. The type of transducer can be a ID transducer (which can be any of a linear array or a curved array transducer or a phased array transducer), a 1.25D transducer, a 1.5D transducer, or a 2D transducer. In one example, the processor can determine the operating mode, processing mode, and type of transducer based on user input via an ultrasound imaging interface on a display portion of a display coupled to the ultrasound system.

[0101] Next, the method 700 proceeds to 704. At 704, the method 700 includes determining whether the ultrasound scan is operating in an AI-based image processing mode with scan data acquired from a ID transducer. If the answer at 704 is yes, the method 700 proceeds to 706.

[0102] At 706, the method 700 includes acquiring first scan data with the ID transducer, where the first scan data is obtained by scanning a first volume of a given volume of interest. As a non-limiting example, an operator can scan a first number of parallel image planes with the ID probe in order to obtain the first scan data of the first volume. In one embodiment, the number of image planes can be one. In another embodiment, the number of image planes can be more than one but less than the number of image planes when the AI-based resolution mapping mode is not implemented. Specifically, the first volume is smaller than a second volume that can be scanned when the AI-based resolution mapping mode is not deployed. Thus, the amount of first scan data obtained when the AI-based resolution mapping mode is utilized is less than the amount of second scan data obtained when the AI-based resolution mapping mode is not utilized. For example, when the AI-based resolution mapping mode is not deployed, an operator can scan a larger volume and process a larger scan data (i.e., the second scan data) in order to generate a 2D image with a higher height resolution profile. In other words, the ID transducer array can acquire a plurality of 2D cross-sectional images by sweeping through the volume of interest in the height direction. Depending on the rendering mode (2D or 3D, which can be based on user selection), a 2D image or a 3D image with higher height resolution can be reconstructed from the plurality of 2D cross-sectional images. However, such volume scanning with the ID transducer to achieve the desired resolution in the height direction requires higher skill and richer experience, which can be challenging for a relatively inexperienced operator and can not be accurately reproduced even when operated by an experienced operator. Moreover, the volume scan data is processed by image processing algorithms (e.g., delay-and- multiply accumulation (DMAS) image reconstruction algorithms, image frame correlation or decorrelation algorithms, etc.) for generating the higher resolution 2D image, which has its own drawbacks such as image artifacts, distortion, insufficient dimensional accuracy, etc. Furthermore, the increased mechanical control to adjust the height movement of the probe increases the bulkiness and complexity of the processing, while the manual scanning lacks reproducibility (with or without experience and skill). In some examples, the reconstruction algorithm can be a neural network-based algorithm, which can still require a larger volume to be scanned and can require a larger number of parallel image planes to be scanned with the ID probe in order to achieve the desired height resolution. Thus, such neural network-based algorithm still suffers from the problems of insufficient dimensional accuracy, reproducibility, complexity of generating the scan image, and bulkiness and cost of the transducer (when the mechanical control is added).

[0103] To utilize scan data obtained from a ID transducer and improve the resolution of 2D images (such as higher resolution in the elevation direction) with reduced scan complexity and without bulky additional mechanical controls, a trained resolution mapping algorithm can be deployed, as described below. In brief, in an AI-based resolution mapping mode, a first volume of a given volume of interest can be scanned with a ID transducer, where the first volume is smaller than a second volume required when reconstructing multiple 2D cross-sectional images as described above. A first ultrasound image having a lower resolution profile is then generated using the scan data. The first ultrasound image is fed into a trained resolution mapping neural network algorithm to obtain a second ultrasound image having a higher resolution profile. In this way, for a given volume of interest, to obtain a desired resolution profile, the first volume (and thus the first amount of scan data) scanned with a ID transducer when AI-based resolution mapping is deployed is smaller compared to the second volume (and thus the second amount of second scan data) scanned with a ID probe when AI-based resolution mapping is not employed.

[0104] Returning to 706, after scanning the first volume with the ID probe to obtain the first scan data, the method 700 proceeds to 708. At 708, the method 700 includes generating a first ultrasound image using the first scan data. The first ultrasound image has a lower resolution profile. This lower resolution profile can have a lower resolution in one or more of the lateral, axial, and elevation directions.

[0105] Next, the method 700 proceeds to 710, where the method includes providing the first ultrasound image having the lower resolution profile as input to a trained resolution mapping algorithm. That is, the method includes feeding the first lower resolution image obtained by scanning with the ID ultrasound probe into the input layer of the trained resolution mapping network.

[0106] Next, at 712, the method 700 includes generating a second ultrasound image having features learned in one or more encoders of the trained resolution mapping network. The second ultrasound image has a higher resolution profile than the first ultrasound image used as input to the trained resolution mapping algorithm. This higher resolution profile can include a higher resolution in one or more of the lateral, axial, and elevation directions.

[0107] After obtaining the second higher resolution image, at 714, the method 700 includes displaying the second higher resolution image via a display portion of a display device (e.g., the display device 234 of the image processing system 202 of FIG. 2). Figure 2

[0108] ​In one embodiment, at 715, the second, higher resolution image (i.e., the generated image) can include one or more annotations on the generated image to indicate the regions modified by the neural network to improve resolution. As a non-limiting example, one or more graphical indications (such as outlines of one or more regions on the generated image) can be provided to enable a user to identify the regions that were modified. The user can then decide whether additional scans with a 2D probe are needed, or whether the images acquired via the ID probe and / or the generated image can have the desired resolution / clarity for diagnosis. The annotations (i.e., graphical indications) can be turned on or off, for example, based on a user input request.

[0109] In some embodiments, the generated image can include annotations that include confidence level indications of one or more features whose resolution is improved in the generated image. As a non-limiting example, if a portion of an anatomical structure is visible at a first, lower resolution in an image acquired via the ID probe, and after passing the image through the trained neural network algorithm, a generated image is obtained in which the portion of the anatomical structure has a second, higher resolution, and an additional portion of the anatomical structure is indicated in the generated image, a confidence level of the additional portion can be indicated on the generated image that indicates that the additional portion can be visible when scanned with an actual 2D probe.

[0110] Returning to 704, if the answer is no, the method 700 proceeds to 716. At 716, the method 700 includes acquiring second scan data by scanning a second volume of a volume of interest with the ID probe. In acquiring the second scan data, the ID probe can scan a second number of parallel image planes in the height direction to scan a second volume of interest. The second scan data is greater than the first scan data acquired when the AI-based resolution mapping mode is deployed. In operating without the AI-based resolution mapping mode deployed, the operator can scan a second volume that is greater than the first volume in order to obtain a second amount of scan data that is greater than the first amount of scan data. Further, the second number of image planes scanned to cover the greater second volume is greater than the first number of images scanned when the AI-based resolution mapping mode is implemented. Thus, to obtain the desired high resolution, a greater amount of scan data is needed and a greater volume can be scanned with the ID probe when operating without the AI-based resolution mapping network deployed.

[0111] After acquiring the second scan data, the method 700 proceeds to 718. At 718, the method 700 includes generating a third ultrasound image using the second scan data, where the third ultrasound image is reconstructed based on an image reconstruction algorithm. Exemplary image reconstruction algorithms include a delay-and-sum (DAS) beamforming algorithm, a delay multiply and sum (DMAS) image reconstruction algorithm, an image frame correlation or decorrelation algorithm, a pixel nearest neighbor (PNN), a voxel nearest neighbor (VNN), etc. It should be appreciated that other image reconstruction algorithms can be used; however, the scan data acquired for image reconstruction when the AI-based resolution mapping is not deployed can be greater than the scan data acquired during operation when the AI-based resolution mapping is deployed.

[0112] A technical effect of the following is that the scan data acquired to achieve a higher resolution profile is reduced: generating a dataset with image pairs (each image pair including a lower resolution image and a corresponding higher resolution image), training, validating, and testing a neural network-based algorithm for resolution mapping with the dataset, and deploying the trained resolution mapping algorithm during an ultrasound scan. Another technical effect of training and deploying the resolution mapping algorithm is that the resolution profile is improved, particularly in the elevation direction. Another technical effect of training and deploying the resolution mapping algorithm is that the image quality is improved without using highly complex control components for 2D probes. Further, by enabling 1D probe per elevation high resolution, users can obtain meaningful image quality improvements with 1D transducers without any additional hardware upgrades. Further, because the complexity of the scan and the bulk of the probe is reduced with the deployment of the resolution mapping algorithm, inexperienced users can generate high quality, high resolution images.

[0113] Embodiments of the method include: acquiring a first ultrasound image having a first resolution profile; inputting the first ultrasound image to a trained neural network algorithm; generating a second ultrasound image having a second, higher resolution profile as an output of the trained neural network algorithm; and displaying the generated second ultrasound image. In a first example of the method, the second, higher resolution profile has a higher resolution in one or more of an axial direction, a lateral direction, and an elevation direction compared to the first ultrasound image. In a second example of the method, which optionally includes the first example, the first ultrasound image is acquired with a linear array ultrasound probe or a single row array of a multi-row array probe. In a third example of the method, which optionally includes one or more or each of the first example and the second example, the trained neural network algorithm is trained with a training dataset comprising a plurality of image pairs, wherein each pair of the plurality of image pairs comprises a first training image and a second training image, the first training image having a lower resolution in one or more of an axial direction, a lateral direction, and an elevation direction compared to the second training image. In a fourth example of the method, which optionally includes one or more or each of the first example through the third example, the first training image is obtained via a linear array ultrasound probe and the second training image is obtained via a two-dimensional array ultrasound probe. In a fifth example of the method, which optionally includes one or more or each of the first example through the fourth example, the first training image is acquired via a first number of rows of a multi-array probe and the second training image is acquired via a second number of rows of a multi-array probe, the first number of rows being less than the second number of rows. In a sixth example of the method, which optionally includes one or more or each of the first example through the fifth example, the first training image and the second training image are obtained with a same set of scan parameters, the scan parameters comprising a scan depth, a probe position, a scan frequency, a scan aperture, a focal point, and a scan plane. In a seventh example of the method, which optionally includes one or more or each of the first example through the sixth example, the trained neural network algorithm is trained to extract one or more high resolution features in one or more of a lateral direction, an axial direction, and an elevation direction from the second training image to corresponding low resolution features in the first training image. In an eighth example of the method, which optionally includes one or more or each of the first example through the seventh example, the trained neural network algorithm has an autoencoder architecture. In a ninth example of the method, which optionally includes one or more or each of the first example through the eighth example, the trained neural network algorithm has a U-Net architecture.

[0114] Embodiments of the method include training a deep learning model to output a higher resolution medical image using a captured lower resolution medical image obtained from a medical imaging device as input, wherein the deep learning model is trained with a training data set comprising a plurality of medical image pairs for each of a plurality of anatomical regions; and wherein each image pair of the plurality of medical image pairs comprises a first lower resolution image of a selected anatomical region and a second higher resolution image of the selected anatomical region; and wherein the first lower resolution image and the second higher resolution image are obtained using a same set of scan parameters comprising a scan plane, a scan frequency, a position of a probe relative to a reference marker, a scan aperture, a focal point, and a scan depth. In a first example of the method, the first lower resolution image is captured via a one-dimensional ultrasound probe and the second higher resolution image is obtained via a two-dimensional ultrasound probe. In a second example of the method, which optionally includes the first example, the first lower resolution image is captured via a first number of rows of a multi-array probe and the second higher resolution image is obtained via a second number of rows of the multi-array probe, the first number of rows being less than the second number of rows. In a third example of the method, which optionally includes one or more of the first example and the second example, the first number of rows is equal to one row and the second number of rows is more than one row. In a fourth example of the method, which optionally includes one or more of the first example through the third example, the deep learning model has an autoencoder architecture or a U-Net architecture. In a fifth example of the method, which optionally includes one or more of the first example through the fourth example, the deep learning model is modeled as a generative adversarial network. In a sixth example of the method, which optionally includes one or more of the first example through the fifth example, training the deep learning model includes extracting one or more features corresponding to a higher resolution from the second higher resolution image and applying the one or more extracted features to the lower resolution image to generate a reconstructed higher resolution image.

[0115] Embodiments of the system include an image processing system comprising a display device, a user input device, a trained resolution mapping network, and a processor communicably coupled to the display device, the user input device, and a non-transitory memory storing the trained resolution mapping network and comprising instructions that, when executed, cause the processor to: receive an ultrasound image of an anatomical region of a subject, the ultrasound image having a first resolution; generate, using the trained resolution mapping network, a resolution mapped ultrasound image, the resolution mapped ultrasound image having a second resolution greater than the first resolution; and display, via the display device, the resolution mapped ultrasound image; and wherein the trained resolution mapping network has a convolutional neural network architecture. In a first example of the system, the ultrasound image is acquired with an ultrasound probe of a linear array of an ultrasound imaging device communicably coupled to the image processing system, and wherein the resolution mapped ultrasound image is generated based on the acquired ultrasound image. In a second example of the system, which optionally includes the first example, the trained resolution mapping network is trained with a first training set and / or a second training set, wherein the first training set comprises a plurality of pairs of ultrasound images for each of a plurality of anatomical portions of a human body, each pair of the plurality of pairs of ultrasound images comprising a first image of an anatomical region obtained from a linear array ultrasound probe, and a second image of the anatomical region obtained from a multi-array ultrasound probe; and wherein the second training set comprises a further plurality of pairs of ultrasound images for each of the plurality of anatomical portions, each pair of the further plurality of pairs of ultrasound images comprising a third image of the anatomical region obtained from a multi-array ultrasound probe of a first number of rows, and a fourth image of the anatomical region obtained from a multi-array ultrasound probe of a second number of rows, the first number of rows being less than the second number of rows.

[0116] When introducing elements of various embodiments of the present disclosure, the articles "a," "an," and "the" are intended to mean that there are one or more of the elements. The terms "first," "second," and the like do not denote any order, quantity, or importance, but are used to distinguish one element from another. The terms "include" and "comprise" and "have" are intended to be inclusive and permitting of additional elements. As used herein, the terms "connected to," "coupled to," and the like are intended to mean that two or more objects (e.g., materials, elements, structures, members, etc.) are either directly or indirectly connected or coupled to one another, whether by intervening objects or not. Additionally, it is to be understood that references to "one embodiment" or "an embodiment" of the present disclosure are not intended to be interpreted as excluding the presence of additional embodiments that also incorporate the referenced features.

[0117] In addition to any prior indicated modifications, many other variations and substitutions of the described arrangements can be designed by those skilled in the art without departing from the spirit and scope of the present description, and the appended claims are intended to cover such modifications and arrangements. Thus, although the information has been described above in connection with specific and preferred aspects, it will be apparent to one of ordinary skill in the art that many modifications, including but not limited to, form, function, manner of operation and use, can be made without departing from the principles and concepts set forth herein. Also, as used herein, in all aspects, the examples and embodiments are intended to be illustrative and not limiting in any way.

Claims

1. A method comprising: acquiring a first ultrasound image having a first resolution profile including high resolution at a first depth and differential resolution at a second depth in a near field or a far field; inputting the first ultrasound image to a trained neural network algorithm, wherein the trained neural network algorithm is trained with a training dataset including a plurality of image pairs, wherein each pair of the plurality of image pairs includes a first training image and a second training image, the first training image including a one-dimensional probe image, the second training image including a two-dimensional probe image; generating a second ultrasound image having a second higher resolution profile including high resolution over a wide depth range in the near field and the far field as an output of the trained neural network algorithm; and displaying the generated second ultrasound image.

2. The method of claim 1, wherein the first ultrasound image is acquired with a linear array ultrasound probe or a single row of an array probe in a multi-row array probe.

3. The method of claim 1, wherein the first training image has lower resolution in one or more of an axial direction, a lateral direction, and a height direction compared to the second training image.

4. The method of claim 3, wherein the first training image is obtained via a linear array ultrasound probe and the second training image is obtained via a two-dimensional array ultrasound probe.

5. The method of claim 3, wherein the first training image is acquired via a first number of rows of a multi-array probe and the second training image is acquired via a second number of rows of the multi-array probe, the first number of rows being less than the second number of rows.

6. The method of claim 3, wherein the first training image and the second training image are obtained with a same set of scan parameters including a scan depth, a probe position, a scan frequency, a scan aperture, a focal point, and a scan plane.

7. The method of claim 3, wherein the trained neural network algorithm is trained to extract one or more high resolution features in one or more of the lateral direction, the axial direction, and the height direction from the second training image to corresponding low resolution features in the first training image.

8. The method of claim 1, wherein the trained neural network algorithm has an autoencoder architecture.

9. The method of claim 1, wherein the trained neural network algorithm has a U-Net architecture.

10. A method comprising: training a deep learning model to output a higher resolution medical image using a lower resolution medical image acquired from a medical imaging device as input; wherein the deep learning model is trained with a training dataset including a plurality of medical image pairs for each of a plurality of anatomical regions; ​ wherein each image pair of the plurality of medical image pairs comprises a first lower resolution image of a selected anatomical region and a second higher resolution image of the selected anatomical region, the first lower resolution comprising high resolution at a first depth and differential resolution at a second depth in a near field or a far field, the second higher resolution comprising high resolution over a wide depth range in the near field and the far field; and wherein the first lower resolution image and the second higher resolution image are obtained using a same set of scan parameters, the same set of scan parameters comprising a scan plane, a scan frequency, a position of a probe relative to a reference marker, an aperture size, a focal point, and a scan depth.

11. The method of claim 10, wherein the first lower resolution image is acquired via a one-dimensional ultrasound probe and the second higher resolution image is obtained via a two-dimensional ultrasound probe.

12. The method of claim 10, wherein the first lower resolution image is acquired via a first number of rows of a multi-array probe and the second higher resolution image is acquired via a second number of rows of the multi-array probe, the first number of rows being less than the second number of rows.

13. The method of claim 12, wherein the first number of rows is equal to one row and the second number of rows is more than one row.

14. The method of claim 10, wherein the deep learning model has an autoencoder architecture or a U-Net architecture.

15. The method of claim 10, wherein the deep learning model is modeled as a generative adversarial network.

16. The method of claim 10, wherein training the deep learning model comprises extracting one or more features corresponding to a high resolution by height from the second higher resolution image and applying the one or more extracted features to the lower resolution image to generate a reconstructed higher resolution image.

17. An image processing system, comprising: a display device; a user input device; a trained resolution mapping network, wherein the trained resolution mapping network is trained with a training set comprising a plurality of ultrasound image pairs, wherein each pair of the plurality of ultrasound image pairs comprises a first image and a second image, the first image comprising a one-dimensional probe image, the second image comprising a two-dimensional probe image; a processor communicably coupled to the display device, the user input device, and a non-transitory memory storing the trained resolution mapping network and comprising instructions that, when executed, cause the processor to: receive an ultrasound image of an anatomical region of a subject, the ultrasound image having a first resolution comprising high resolution at a first depth and differential resolution at a second depth in a near field or a far field; generate, using the trained resolution mapping network, a resolution mapped ultrasound image having a second higher resolution greater than the first resolution, the second higher resolution comprising high resolution over a wide depth range in the near field and the far field; and displaying, via the display device, the resolution mapped ultrasound image; and wherein the trained resolution mapping network has a convolutional neural network architecture.

18. The image processing system of claim 17, wherein the ultrasound image is acquired with an ultrasound probe of a linear array of an ultrasound imaging device communicatively coupled to the image processing system, and wherein the resolution mapped ultrasound image is generated based on the acquired ultrasound image.

19. The image processing system of claim 17, wherein the trained resolution mapping network is trained with a first training set and / or a second training set; wherein the first training set includes a plurality of pairs of ultrasound images for each of a plurality of anatomical portions of a human body, each of the plurality of pairs of ultrasound images including a first image of an anatomical region obtained from a linear array ultrasound probe, and a second image of the anatomical region obtained from a multi-array ultrasound probe; and wherein the second training set includes a further plurality of pairs of ultrasound images for each of the plurality of anatomical portions, each of the further plurality of pairs of ultrasound images including a third image of the anatomical region obtained from a multi-array ultrasound probe of a first number of rows, and a fourth image of the anatomical region obtained from the multi-array ultrasound probe of a second number of rows, the first number of rows being less than the second number of rows.

Images