Systems and methods for automatic lesion characterization

Abstract

The present invention is titled Systems and Methods for Automated Lesion Characterization. The present invention provides methods and systems for automatically characterizing lesions in ultrasound images. In one example, the method includes: automatically determining an A / B ratio of a region of interest (ROI) via an A / B ratio model, the A / B ratio model being trained to output an A / B ratio using as inputs a B-mode image of the ROI and an elastography image of the ROI; and displaying the A / B ratio on a display device.

Description

Technical Field

[0001] Embodiments of the subject matter disclosed herein relate to ultrasound imaging and, more particularly, to characterizing lesions using ultrasound imaging. Background Art

[0002] Medical ultrasound is an imaging modality that uses ultrasonic waves to probe the internal structures of a patient's body and generate corresponding images. For example, an ultrasound probe including a plurality of transducer elements emits ultrasonic pulses that are reflected or backscattered, refracted, or absorbed by structures in the body. The ultrasound probe then receives the reflected echoes, which are processed into an image. The ultrasound image of the internal structure can be saved for later analysis by a clinician to aid in diagnosis and / or can be displayed on a display device in real time or near real time. Summary of the Invention

[0003] In one embodiment, a method includes: automatically determining an A / B ratio of a region of interest (ROI) via an A / B ratio model that is trained to output the A / B ratio using a B-mode image of the ROI and an elastography image of the ROI as inputs; and displaying the A / B ratio on a display device.

[0004] The foregoing advantages, as well as other advantages and features, of the present disclosure will become apparent from the following detailed description when taken in conjunction with the drawings. It should be understood that the above Summary of the Invention is provided to introduce in a simplified form a selected group of concepts that are further described in the Detailed Description. This is not meant to identify the key or essential features of the claimed subject matter, the scope of which is uniquely defined by the claims that follow the Detailed Description. Moreover, the claimed subject matter is not limited to embodiments that solve any disadvantages noted above or in any part of this disclosure. Brief Description of the Drawings

[0005] Aspects of the present disclosure are better understood by reading the following detailed description and referring to the drawings, in which:

[0006] Figure 1 A block diagram of an ultrasound system according to one embodiment is shown;

[0007] Figure 2 is a schematic diagram of a system for automatically characterizing lesions according to one embodiment;

[0008] Figure 3 is a flowchart of a method for automatically calculating an A / B ratio according to one embodiment; and

[0009] Figure 4shows an exemplary graphical user interface that shows an exemplary B-mode image, an exemplary elastography image, and an A / B ratio calculated according to Figure 3 the method. DETAILED DESCRIPTION

[0010] Ultrasound images acquired during a medical ultrasound examination can be used to diagnose a patient's condition, which can include one or more clinicians analyzing abnormalities in the ultrasound image, measuring certain anatomical features imaged in the ultrasound image, and so on. For example, when characterizing a lesion such as a breast lesion, a clinician can use standard B-mode ultrasound imaging as well as elastography to evaluate the lesion, where elastography is a mechanism for non-invasively measuring tissue stiffness. Certain characteristics of a lesion in an elastography image may be beneficial for semi-quantitative characterization of the lesion relative to the lesion in the B-mode image. For example, the width and / or area of a lesion in an elastography image relative to the width and / or area of the lesion in the B-mode image (referred to as the A / B ratio) can provide a semi-quantitative analysis of lesion malignancy because benign lesions typically have a smaller A / B ratio than malignant lesions.

[0011] Accordingly, when characterizing a lesion such as a breast lesion, a clinician can measure the A / B ratio by acquiring a B-mode image that includes the lesion and acquiring a corresponding elastography image that includes the lesion. The clinician can then identify the lesion in each image, measure the width of the lesion in each image, and then calculate the A / B ratio. However, this process is time-consuming and can result in inconsistent A / B ratio calculations between different clinicians and different patients, and even between different imaging sessions of the same patient. In particular, if the A / B ratio of a patient is monitored over time to track the development of a lesion, inconsistent A / B ratio calculations can lead to inaccurate determination of lesion growth and / or transformation, which can have a negative impact on patient care.

[0012] Accordingly, in accordance with the embodiments disclosed herein, an artificial intelligence-based model can be used to automatically calculate an A / B ratio of a target anatomical feature such as a lesion, where the model is trained to segment the target anatomical feature in both the B-mode image and the elastography image, measure the width and / or area of the segmented target anatomical feature in each image, and calculate the A / B ratio based on the measured width and / or area. The automatically calculated A / B ratio can be displayed on a display device and / or saved as part of a patient examination (e.g., saved in a patient's medical record). By doing so, A / B ratio measurements can be more consistent between different patients and between different imaging sessions, which can improve patient care and reduce the workflow requirements of clinicians.

[0013] An ultrasound imaging system such as Figure 1an ultrasound imaging system to obtain B-mode images and elastography images, and these images can be typed in as inputs to an A / B ratio model stored on an image processing system such as Figure 2 the image processing system. According to Figure 3 the method shown, the A / B ratio model can be trained to segment lesions in the B-mode image and the corresponding elastography image and calculate the A / B ratio. The calculated A / B ratio can be output and displayed on a part of a graphical user interface such as Figure 4 shown.

[0014] See Figure 1 , which shows a schematic diagram of an ultrasound imaging system 100 according to an embodiment of the present disclosure. The ultrasound imaging system 100 includes a transmit beamformer 101 and a transmitter 102 that drives elements (e.g., transducer elements) 104 within a transducer array (referred to herein as probe 106) to transmit pulsed ultrasound signals (referred to herein as transmit pulses) into a body (not shown). According to one embodiment, probe 106 can be a one-dimensional transducer array probe. However, in some embodiments, probe 106 can be a two-dimensional matrix transducer array probe. As further explained below, transducer element 104 can be composed of piezoelectric material. When a voltage is applied to the piezoelectric crystal, the crystal physically expands and contracts, thereby emitting an ultrasonic spherical wave. In this way, transducer element 104 can convert an electrical transmit signal into an acoustic transmit beam.

[0015] After the elements 104 of the probe 106 transmit pulsed ultrasound signals into the (patient's) body, the pulsed ultrasound signals are backscattered from structures inside the body (such as blood cells or muscle tissue) to generate echoes that return to the elements 104. The echoes are converted by the elements 104 into electrical signals or ultrasound data, and the electrical signals are received by the receiver 108. The electrical signals representing the received echoes pass through a receive beamformer 110 that outputs radio frequency (RF) data. Additionally, transducer element 104 can generate one or more ultrasound pulses based on the received echoes to form one or more transmit beams.

[0016] According to some embodiments, the probe 106 may include electronic circuitry to perform all or part of transmit beamforming and / or receive beamforming. For example, all or part of the transmit beamformer 101, transmitter 102, receiver 108, and receive beamformer 110 may be located within the probe 106. In the present disclosure, the term “scan” or “scanning” may also be used to refer to the process of acquiring data by transmitting and receiving ultrasonic signals. In the present disclosure, the term “data” may be used to refer to one or more data sets acquired using an ultrasonic imaging system. In one embodiment, data obtained via the ultrasonic system 100 may be used to train a machine learning model. The user interface 115 may be used to control the operation of the ultrasonic imaging system 100, including for controlling the input of patient data (e.g., patient history), for changing scan or display parameters, for initiating a probe repolarization sequence, etc. The user interface 115 may include one or more of the following items: a rotary element, a mouse, a keyboard, a trackball, hard keys linked to specific actions, soft keys configurable to control different functions, and a graphical user interface displayed on the display device 118.

[0017] The ultrasonic imaging system 100 further includes a processor 116 that controls the transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110. The processor 116 communicates electronically (e.g., communicatively connected) with the probe 106. For the purposes of this disclosure, the term "electronically communicate" can be defined to include both wired and wireless communication. The processor 116 can control the probe 106 to acquire data according to instructions stored in the memory of the processor and / or the memory 120. The processor 116 controls which of the elements 104 are active and the shape of the beam transmitted from the probe 106. The processor 116 also communicates electronically with the display device 118, and the processor 116 can process data (e.g., ultrasonic data) into an image for display on the display device 118. The processor 116 can include a central processing unit (CPU) according to one embodiment. According to other embodiments, the processor 116 can include other electronic components capable of performing processing functions, such as a digital signal processor, a field programmable gate array (FPGA), or a graphics board. According to other embodiments, the processor 116 can include multiple electronic components capable of performing processing functions. For example, the processor 116 can include two or more electronic components selected from a list of electronic components, including: a central processing unit, a digital signal processor, a field programmable gate array, and a graphics board. According to another embodiment, the processor 116 can further include a complex demodulator (not shown) that demodulates RF data and generates an IQ data pair representing the echo signal. In another embodiment, demodulation can be performed earlier in the processing chain. The processor 116 is adapted to perform one or more processing operations according to a plurality of optional ultrasonic modalities on the data. In one example, the data can be processed in real time during a scanning session as the echo signal is received by the receiver 108 and transmitted to the processor 116. For the purposes of this disclosure, the term "real time" is defined to include a process performed without any intentional delay. For example, an embodiment can acquire images at a real-time rate of 7 to 20 frames per second. The ultrasonic imaging system 100 is capable of acquiring 2D data of one or more planes at a significantly faster rate. However, it should be understood that the real-time frame rate can depend on the length of time it takes to acquire each frame of data for display. Therefore, when acquiring a relatively large amount of data, the real-time frame rate may be slow. Thus, some embodiments can have a real-time frame rate significantly faster than 20 frames per second, while other embodiments can have a real-time frame rate lower than 7 frames per second. The data can be temporarily stored in a buffer (not shown) during a scanning session and processed in a less real-time manner in real-time or offline operations. Some embodiments of the present invention can include multiple processors (not shown) to process the processing tasks processed by the processor 116 according to the exemplary embodiments described above.For example, before displaying an image, a first processor can be utilized to demodulate and extract the RF signal, while a second processor can be used to further process the data, such as by expanding the data. It should be understood that other embodiments may use different processor arrangements.

[0018] The ultrasound imaging system 100 can continuously acquire data, for example, at a frame rate of 10 Hz to 30 Hz (e.g., 10 frames per second to 30 frames per second). Images generated from the data can be refreshed on the display device 118 at a similar frame rate. Other embodiments can acquire and display data at different rates. For example, depending on the size of the frame and the intended application, some embodiments can acquire data at a frame rate less than 10 Hz or greater than 30 Hz. It includes a memory 120 for storing frames of the processed acquired data. In an exemplary embodiment, the memory 120 has sufficient capacity to store at least several seconds of ultrasound data frames. The data frames are stored in a manner that facilitates retrieval according to their acquisition order or time. The memory 120 can include any known data storage medium.

[0019] In various embodiments of the present invention, the processor 116 can process data through different mode-related modules (e.g., B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastography, TVI, strain, strain rate, etc.) to form 2D or 3D data. For example, one or more modules can generate B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastography, TVI, strain, strain rate, and combinations thereof, etc. As an example, one or more modules can process color Doppler data, which can include conventional color flow Doppler, power Doppler, HD flow, etc. Image lines and / or frames are stored in the memory and can include timing information indicating the time at which the image lines and / or frames are stored in the memory. These modules can include, for example, a scan conversion module for performing a scan conversion operation to convert the acquired image from beam space coordinates to display space coordinates. A video processor module can be provided that reads the acquired image from the memory and displays the image in real time when performing a procedure on a patient (e.g., ultrasound imaging). The video processor module can include a separate image memory, and the ultrasound image can be written to the image memory for reading and display by the display device 118.

[0020] The ultrasonic imaging system 100 includes an elastography circuit 103 configured to implement shear wave elastography and / or strain elastography. When in the shear wave mode, the elastography circuit 103 can control the probe 106 to generate shear waves at a site within a region of interest (ROI) of an imaging object (such as a patient). The elastography circuit 103 can control the probe 106 (or more specifically, the transducer element 104) to direct shear wave generation or push pulses towards a predetermined site to generate shear waves. Alternatively, the elastography circuit 103 can control another device capable of generating shear waves, and the probe 106 can measure or track the speed of the shear wave as it passes through the ROI. For example, the elastography circuit 103 can control a therapeutic transducer, a mechanical actuator, or an audio device to generate shear waves.

[0021] When in the strain mode, the elastography circuit 103 can control the probe 106 to generate a mechanical force (such as surface vibration, arbitrary or step quasi-static surface displacement, etc.) or a radiation force on the patient or ROI to measure the stiffness or strain of the patient's ROI. Alternatively, the elastography circuit 103 can control another device capable of generating a mechanical force on the patient or ROI. For example, a low-frequency mechanical vibrator can be applied to the skin surface, and the probe 106 can measure the compression motion induced on the underlying tissue, such as on the ROI.

[0022] In various embodiments of the present disclosure, one or more components of the ultrasonic imaging system 100 can be included in a portable handheld ultrasonic imaging device. For example, the display device 118 and the user interface 115 can be integrated into the outer surface of the handheld ultrasonic imaging device, which can also include a processor 116 and a memory 120. The probe 106 can include a handheld probe that communicates electronically with the handheld ultrasonic imaging device to collect raw ultrasonic data. The transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110 can be included in the same or different parts of the ultrasonic imaging system 100. For example, the transmit beamformer 101, the transmitter 102, the receiver 108, and the receive beamformer 110 can be included in the handheld ultrasonic imaging device, the probe, and combinations thereof.

[0023] After performing a two-dimensional ultrasound scan, a data block containing scan lines and their samples is generated. After applying a backend filter, a process called scan conversion is performed to transform the two-dimensional data block into a displayable bitmap image with additional scan information such as depth, angle of each scan line, etc. During scan conversion, interpolation techniques are applied to fill in the missing holes (i.e., pixels) in the resulting image. These missing pixels occur because each element of the two-dimensional block typically should cover many pixels in the resulting image. For example, in current ultrasound imaging systems, bicubic interpolation is applied, which utilizes adjacent elements of the two-dimensional block. Thus, if the two-dimensional block is relatively small compared to the size of the bitmap image, the scan-converted image will include regions of poor or low resolution, especially for regions with greater depth.

[0024] The ultrasound images acquired by the ultrasound imaging system 100 can be further processed. In some embodiments, the ultrasound images generated by the ultrasound imaging system 100 can be transmitted to an image processing system, where in some embodiments, the ultrasound images can be segmented by a machine learning model trained using the ultrasound images and corresponding ground truth outputs. As used herein, a ground truth output refers to the expected or "correct" output based on a given input into the machine learning model. For example, if a machine learning model is being trained to classify images of cats, the ground truth output of the model when fed an image of a cat is the label "cat". Additionally, the image processing system can further process the ultrasound images using one or more different machine learning models configured to calculate an A / B ratio based on the segmented ultrasound images.

[0025] Although described herein as separate systems, it should be understood that in some embodiments, the ultrasound imaging system 100 includes an image processing system. In other embodiments, the ultrasound imaging system 100 and the image processing system can include separate devices. In some embodiments, the images generated by the ultrasound imaging system 100 can be used as a training data set for training one or more machine learning models, where, as described below, the machine learning models can be used to perform one or more steps of ultrasound image processing.

[0026] See Figure 2, which shows an image processing system 202 according to an embodiment. In some embodiments, the image processing system 202 is incorporated into an ultrasound imaging system 100. For example, the image processing system 202 may be provided in the ultrasound imaging system 100 as a processor 116 and a memory 120. In some embodiments, at least a portion of the image processing system 202 is provided at a device (e.g., an edge device, a server, etc.) communicatively coupled to the ultrasound imaging system via a wired connection and / or a wireless connection. In some embodiments, at least a portion of the image processing system 202 is provided at a separate device (e.g., a workstation) that can receive images from the ultrasound imaging system or from a storage device storing images / data generated by the ultrasound imaging system. The image processing system 202 may be operably / communicatively coupled to a user input device 232 and a display device 234. At least in some examples, the user input device 232 may include a user interface 115 of the ultrasound imaging system 100, and the display device 234 may include a display device 118 of the ultrasound imaging system 100.

[0027] The image processing system 202 includes a processor 204 configured to execute machine-readable instructions stored in a non-transitory memory 206. The processor 204 may be single-core or multi-core, and the program executed thereon may be configured for parallel or distributed processing. In some embodiments, the processor 204 may optionally include separate components distributed across two or more devices, which may be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of the processor 204 may be virtualized and executed by a remotely accessible networked computing device configured in a cloud computing configuration.

[0028] The non-transitory memory 206 may store an A / B ratio model 208, a training module 210, and ultrasound image data 212. The A / B ratio model 208 may include one or more machine learning models (such as a deep learning network) that include a plurality of weights and biases, activation functions, loss functions, gradient descent algorithms, and instructions for implementing one or more deep neural networks to process input ultrasound images. For example, the A / B ratio model 208 may store instructions for implementing a segmentation model that is trained to identify and segment target anatomical features, such as lesions, in both B-mode images and elastography images. The A / B ratio model 208 may store additional instructions for calculating an A / B ratio based on the segmented B-mode images and elastography images. The A / B ratio model 208 may include one or more neural networks. The A / B ratio model 208 may include trained and / or untrained neural networks and may also include training routines or parameters (such as weights and biases) associated with one or more neural network models stored therein.

[0029] Accordingly, the A / B ratio model 208 described herein can be deployed to automatically calculate the A / B ratio of anatomic features such as lesions. In some examples, the A / B ratio model 208 can use a U-Net or other convolutional neural network architecture to segment lesions in corresponding B-mode images and elastography images (e.g., images taken from the same ROI and, in some examples, approximately at the same time), and can be trained using B-mode ultrasound images and elastography ultrasound images and / or cine loops in which the lesions have been annotated / identified by an expert. The A / B ratio model 208 can measure the width of the segmented lesions in both the B-mode image and the elastography image (e.g., the A / B ratio model 208 can identify the widest part of the lesion and measure the widest part of the lesion to determine the width of the lesion). The A / B ratio can be calculated as the ratio of the width of the lesion in the elastography image to the width of the lesion in the B-mode image. In other examples, the area of each lesion can be determined by the measured width or another suitable determination, and the A / B ratio can be calculated as the ratio of the area of the lesion in the elastography image to the area of the lesion in the B-mode image.

[0030] The non-transitory memory 206 can also include a training module 210 that includes instructions for training one or more machine learning models stored in the A / B ratio model 208. In some embodiments, the training module 210 is not provided at the image processing system 202. Accordingly, the A / B ratio model 208 includes a trained and validated network.

[0031] The non-transitory memory 206 can also store ultrasound image data 212, such as Figure 1The ultrasound images captured by the ultrasound imaging system 100. The ultrasound image data 212 can include both B-mode images and elastography images (whether obtained using shear wave elastography or strain elastography). Additionally, when the training module 210 is stored in the non-transitory memory 206, the ultrasound image data 212 can store ultrasound images, ground truth outputs, iterations of machine learning model outputs, and other types of ultrasound image data that can be used to train the A / B ratio model 208. In some embodiments, the ultrasound image data 212 can store the ultrasound images and ground truth outputs in an ordered format such that each ultrasound image is associated with one or more corresponding ground truth outputs. For example, the ultrasound image data 212 can store a collection of training data, where each collection includes a B-mode image and a ground truth containing a region of interest (ROI) annotated by an expert (e.g., a lesion annotated by a clinician), and / or an elastography image and a ground truth containing an ROI annotated by an expert (e.g., a lesion annotated by a clinician). In some examples, one or more collections of training data can include B-mode images and elastography images acquired on the same patient (e.g., such that the same lesion is annotated on both images). In some examples, one or more collections of training data can include B-mode images and / or elastography images that do not contain lesions and thus do not contain expert annotations. Additionally, in examples where the training module 210 is not provided at the image processing system 202, the image / ground truth outputs that can be used to train the A / B ratio model 208 can be stored elsewhere.

[0032] In some embodiments, the non-transitory memory 206 can include components disposed on two or more devices that can be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of the non-transitory memory 206 can include remotely accessible networked storage devices configured in a cloud computing configuration.

[0033] The user input device 232 can include a touch screen, keyboard, mouse, touchpad, motion sensing camera, or one or more of other devices configured to enable a user to interact with and manipulate data within the image processing system 202. In one example, the user input device 232 can enable a user to select an ultrasound image for training a machine learning model, to indicate or mark the location of an ROI in the ultrasound image data 212, or for further processing using the trained machine learning model.

[0034] The display device 234 may include one or more display devices that utilize almost any type of technology. In some embodiments, the display device 234 may include a computer monitor and may display ultrasound images. The display device 234 may be combined with the processor 204, the non-transitory memory 206, and / or the user input device 232 in a shared package or may be a peripheral display device and may include a monitor, a touch screen, a projector, or other display devices known in the art, which may enable a user to view ultrasound images generated by the ultrasound imaging system and / or interact with various data stored in the non-transitory memory 206.

[0035] It should be understood that Figure 2 the illustrated image processing system 202 is for illustration and not limitation. Another suitable image processing system may include more, fewer, or different components.

[0036] Figure 3 A flowchart according to one embodiment is shown, which shows an exemplary method 300 for automatically calculating an ROI such as an A / B ratio on a breast lesion. With reference to Figures 1 to 2 the system and components described, method 300 is described, but it should be understood that method 300 may be implemented with other systems and components without departing from the scope of the present disclosure. Method 300 may be executed according to instructions stored in the non-transitory memory of a computing device (such as Figure 2 the image processing system 202).

[0037] At 302, method 300 includes determining whether a request to calculate the A / B ratio has been received. The request to calculate the A / B ratio may be received via user input. For example, an operator of the ultrasound imaging system may type an input requesting the calculation of the A / B ratio via a user input device (such as the user interface 115 and / or the user input device 232). In some examples, the user input requesting the calculation of the A / B ratio may be received while the operator is actively imaging a patient, and thus the request may include a request to calculate the A / B ratio using a specific image or a series of images (such as the most recently acquired or stored elastography images). In some examples, the request to calculate the A / B ratio may include an indication of whether the A / B ratio is to be calculated based on the width of the region of interest (such as a lesion) in the elastography image and the B-mode image, or whether the A / B ratio is to be calculated based on the area of the region of interest in the elastography image and the B-mode image. In some examples, the request to calculate the A / B ratio may be received from the ultrasound imaging system as part of an automatic or semi-automatic workflow.

[0038] If a request to calculate the A / B ratio has not been received, method 300 returns. When a request to calculate the A / B ratio has not been received, the ultrasound system can continue to acquire ultrasound images (whether in B-mode, elastography mode, or other imaging modes) when requested (e.g., when the ultrasound probe is powered on and in contact with the imaging object), and can continue to evaluate whether a request to calculate the A / B ratio has been received.

[0039] If a request to calculate the A / B ratio is received, method 300 proceeds to 304 to obtain a B-mode image including a region of interest (ROI). The ROI can be a lesion or other anatomical feature of interest, such as the thyroid. In some examples, the request to calculate the A / B ratio can include an indication of which anatomical feature / ROI the A / B ratio is to be calculated for (e.g., a request to calculate the A / B ratio on a breast lesion). The B-mode image obtained at 304 can include the indicated anatomical feature / ROI. When the probe is operating in B-mode imaging, the B-mode image can be a standard 2D grayscale image obtained via the ultrasound probe (e.g., probe 106). The B-mode image obtained at 304 can be acquired by the ultrasound system in response to the request to calculate the A / B ratio. In other examples, the B-mode image can be obtained from memory. In some examples, the B-mode image obtained at 304 can be selected by the user. For example, an operator of the ultrasound imaging system can select a B-mode image from a plurality of B-mode images stored in the memory of the ultrasound imaging system, or the operator can indicate via user input that the currently displayed B-mode image can be used for A / B ratio calculation. In some examples, the B-mode image obtained at 304 and used for calculating the A / B ratio can be the B-mode image that serves as the basis for the elastography data in the elastography image obtained at 306, which will be explained in more detail below.

[0040] At 306, an elastography image is obtained. The elastography image can be a shear wave elastography image or a strain elastography image. To acquire a shear wave image, the ultrasound probe is controlled into the shear wave mode. Once the probe is in the shear wave mode, the probe is configured or controlled by the elastography circuitry (e.g., elastography circuitry 103) of the ultrasound imaging system to deliver a push pulse to generate a shear wave within the ROI. After generating the shear wave, the probe measures the echoes scattered from the ROI as the shear wave passes through the ROI. The processor (e.g., processor 116) of the ultrasound imaging system receives the electrical signals from the probe. The processor processes a set of vector data values corresponding to the shear wave data from the electrical signals, where each set defines a single shear wave image frame. The shear wave data vector values associated with the shear wave image frame can be converted to Cartesian coordinates to generate a shear wave image formatted for display. To generate a strain image, when the probe applies a mechanical force (e.g., surface vibration, arbitrary or step quasi-static surface displacement, etc.) or a radiation force on the patient or the ROI, the probe measures the echoes scattered from the ROI before and after the ROI is compressed by the mechanical force or the radiation force. The processor receives the electrical signals from the probe and processes a set of vector data values corresponding to the strain data from the electrical signals, where each set defines a single strain image frame. The strain data vector values can be converted to Cartesian coordinates to generate a strain image formatted for display.

[0041] Since the elastography image can be obtained approximately simultaneously (e.g., immediately following) with the B-mode image, the elastography image obtained at 306 can include the ROI. The elastography image obtained at 306 can be acquired by the ultrasound system in response to a request to calculate the A / B ratio. In other examples, the elastography image can be obtained from memory. In some examples, the elastography image obtained at 306 can be selected by the user. For example, an operator of the ultrasound imaging system can select an elastography image from a plurality of elastography images stored in the memory of the ultrasound imaging system, or the operator can indicate via user input that the currently displayed elastography image can be used for A / B ratio calculation. The elastography image can include color or grayscale elastography information indicating the measured tissue stiffness, and the elastography information can be displayed as an overlay on the B-mode image. For example, an operator of the ultrasound imaging system can image a patient in B-mode and can identify the ROI in the B-mode image. The operator can then enter a user input requesting to image the patient in elastography mode, and the final B-mode image can be displayed together with the elastography information overlaid on the B-mode image. Thus, the underlying B-mode image of the elastography image can be the B-mode image obtained at 304.

[0042] At 308, elastography images can be processed. Processing of the elastography images can include adjusting the gain and / or transparency of the elastography images. For example, the gain of the elastography image can be adjusted to the maximum allowable gain, and the transparency of the elastography image can be adjusted to the minimum transparency (e.g., transparency of zero). In another example, the transparency can be set to a level based on the brightness of the underlying B-mode image, such as a transparency that increases as the brightness decreases, such as a proportional linear relationship or based on a preset transfer function stored in the system and / or adjusted by the user. Such adjustments can advantageously allow a portion of the B-mode image information to be used to help improve segmentation through the A / B ratio model.

[0043] As described above, elastography data can be displayed on the B-mode image. When viewing the elastography image, the user can adjust the transparency and / or gain of the elastography information to make the underlying B-mode image features visible. However, if the transparency of the elastography information is set to the minimum transparency, the image segmentation of the elastography image used to calculate the A / B ratio may be more robust and consistent across the entire image. Similarly, if the gain of the elastography information is increased to the maximum gain, the robustness and consistency of the segmentation may be enhanced. The transparency and / or gain of the elastography image input to the A / B ratio model can be adjusted relative to and be different from the default transparency and / or gain. The default transparency and / or gain can be applied when the elastography image is first displayed, and the user can further adjust the transparency and / or gain based on user preferences. Thus, at least in some examples, processing of the elastography image can result in the processed elastography image having a different transparency and / or gain than the elastography image displayed to the user.

[0044] At 310, the B-mode image and the processed elastography image are typed as inputs to the A / B ratio model. The A / B ratio model (e.g., A / B ratio model 208) can include one or more deep learning / machine learning models that are trained to identify ROI / interesting anatomical features in the B-mode image and the elastography image. The A / B ratio model can perform image segmentation on the B-mode image and the elastography image to identify the boundaries of the ROI (e.g., the boundaries of the lesion in both the B-mode image and the elastography image), and then measure the width or area of the ROI in each image to calculate the A / B ratio. The segmentation of the ROI in the B-mode image can be performed independently of the segmentation of the ROI in the elastography image.

[0045] Thus, as shown at 312, the A / B ratio model can segment the elastography image to identify and define the boundaries of the ROI, and determine the width or area of the ROI (referred to as width or area A). As shown at 314, the A / B ratio model can segment the B-mode image to identify and define the boundaries of the ROI, and determine the width or area of the ROI (referred to as width or area B). The A / B ratio is then calculated by dividing the width or area (A) of the ROI in the elastography image by the width or area (B) of the ROI in the B-mode image.

[0046] At 316, the A / B ratio can be stored in the memory of the ultrasound imaging system and / or output for display on a display device (such as display device 118 or display device 234). Additionally, the A / B ratio can be sent to a remote device, such as a device that stores an electronic medical record database and / or a picture archiving and communication system (e.g., as part of a patient examination that includes the patient's ultrasound image). Then, method 300 returns.

[0047] Figure 4 An exemplary graphical user interface (GUI) 400 is shown that can be displayed on a display device 401 (such as display device 118 and / or display device 234). The GUI 400 can include a B-mode image 402 and an elastography image 404. The elastography image 404 can include a B-mode image 402 and a superimposed layer 405 of elastography information. The superimposed layer 405 is shown in grayscale, and its pixel brightness corresponds to an indication of tissue stiffness (e.g., Young's modulus) measured by the ultrasound probe in the elastography mode. Alternatively, the superimposed layer 405 can show the elastography information in color.

[0048] The results from the A / B model image segmentation are also shown in the GUI 400. For example, the boundaries of the lesion identified by the A / B model are shown as dashed lines on both images. Thus, the GUI 400 includes the boundary 406 of the lesion in the B-mode image 402 and the boundary 408 of the lesion in the elastography image 404. By showing the image typed as input to the A / B model and the identified ROI boundaries, the information used to calculate the A / B ratio can be communicated to the user. If the user does not agree with the identified boundaries (e.g., determines that the boundaries are too small, too large, or the lesion is misidentified) or determines that the image quality is not sufficient to reliably identify the ROI boundaries, the user can reject the calculated A / B ratio, request a new A / B ratio calculation, calculate the A / B ratio manually, etc. At 410, the A / B ratio calculated by the A / B ratio model is shown. As described above, the A / B ratio (1.2 in this case) is the ratio of the area / width of the ROI in the elastography image to the area / width of the ROI in the B-mode image. Thus, the A / B ratio shown at 410 is determined by dividing the width / area of boundary 408 by the width / area of boundary 406.

[0049] The technical effect of automatically determining the A / B ratio of a region of interest in an ultrasound image is to reduce the operator workflow and enhance the consistency of A / B ratio calculation between the patient and the imaging session.

[0050] Embodiments of the method include: automatically determining the A / B ratio of a region of interest (ROI) via an A / B ratio model that is trained to use the B-mode image of the ROI and the elastography image of the ROI as inputs to output the A / B ratio; and displaying the A / B ratio on a display device. In a first example of the method, the A / B ratio is the ratio of a first area of the ROI in the elastography image to a second area of the ROI in the B-mode image, or the A / B ratio is the ratio of a first width of the ROI in the elastography image to a second width of the ROI in the B-mode image. The area of the ROI in each image can be determined by identifying the maximum width / segment of the ROI and calculating the area based on the maximum width / segment. In other examples, the area of the ROI in each image can be determined by identifying the number of pixels in each ROI. When the A / B ratio is a ratio of widths, the width can be the maximum width / maximum segment of each ROI. In a second example of the method that optionally includes the first example, automatically determining the A / B ratio of the ROI via the A / B ratio model includes: identifying a first boundary of the ROI in the elastography image via the A / B ratio model and determining a first area of the ROI in the elastography image based on the identified first boundary; identifying a second boundary of the ROI in the B-mode image via the A / B ratio model and determining a second area of the ROI in the B-mode image based on the identified second boundary; and determining the A / B ratio based on the first area and the second area. In a third example of the method that optionally includes one or both of the first example and the second example, the elastography image is a shear wave elastography image or a strain elastography image. In a fourth example of the method that optionally includes one or more or each of the first example to the third example, the elastography image includes a B-mode image and a overlay on the B-mode image that includes elastography information indicating the measured stiffness of the tissue imaged in the B-mode image. In a fifth example of the method that optionally includes one or more or each of the first example to the fourth example, the method further includes adjusting the transparency and / or gain of the elastography image before entering the elastography image as an input to the A / B ratio model. In a sixth example of the method that optionally includes one or more or each of the first example to the fifth example, adjusting the transparency and / or gain of the elastography image includes adjusting the transparency to a minimum transparency and adjusting the gain to a maximum gain. In a seventh example of the method that optionally includes one or more or each of the first example to the sixth example, the method further includes storing the A / B ratio in a memory as part of a patient examination.

[0051] Embodiments of the system include: a display device; an ultrasound probe; a memory that stores instructions; and a processor communicatively coupled to the memory and configured, when executing the instructions, to: obtain a B-mode image of a region of interest (ROI) of a patient via the ultrasound probe; obtain an elastography image of the patient's ROI via the ultrasound probe; input the B-mode image and the elastography image as inputs to an A / B ratio model that is trained to output an A / B ratio of the ROI based on the B-mode image and the elastography image; and output the A / B ratio for display on the display device. In a first example of the system, the elastography image is acquired and / or processed to have maximum gain and minimum transparency. In a second example of the system optionally including the first example, the transparency of the elastography image is the transparency of a layer indicating the measured stiffness of the patient's tissue, the layer being superimposed on the B-mode image. In a third example of the system optionally including one or both of the first example and the second example, the A / B ratio model includes a first image segmentation model trained to identify a first boundary of the ROI in the elastography image and a second image segmentation model trained to identify a second boundary of the ROI in the B-mode image. In a fourth example of the system optionally including one or more or each of the first example to the third example, the A / B ratio model determines a first area of the ROI in the elastography image based on the identified first boundary and a second area of the ROI in the B-mode image based on the identified second boundary, and determines the A / B ratio as the ratio of the first area to the second area. In a fifth example of the system optionally including one or more or each of the first example to the fourth example, the elastography image is a shear wave elastography image or a strain elastography image.

[0052] Embodiments of a method for an ultrasound system include: receiving a request to determine an A / B ratio of a region of interest (ROI) of an elastography image, the elastography image including an underlying B-mode image and a stack on the B-mode image, the stack including elastography information of tissue imaged in the B-mode image and measured by an ultrasound probe of the ultrasound system; upon receiving the request, adjusting the transparency of the stack of the elastography image to generate a processed elastography image; entering the processed elastography image and the underlying B-mode image as inputs to a model, the model being trained to output an A / B ratio based on the processed elastography image and the underlying B-mode image; and outputting the A / B ratio for display on a display device. In a first example of the method, receiving the request includes receiving the request when the elastography image is displayed on the display device, the elastography image being displayed with the stack at a first transparency. In a second example of the method, optionally including the first example, adjusting the transparency includes adjusting the transparency from the first transparency to a second transparency, the first transparency being higher than the second transparency. In a third example of the method, optionally including one or both of the first example and the second example, the A / B ratio is a ratio of a first area of the ROI in the elastography image to a second area of the ROI in the B-mode image. In a fourth example of the method, optionally including one or more or each of the first example to the third example, the model is trained to identify and segment the ROI in the elastography image to determine the first area, and to identify and segment the ROI in the B-mode image to determine the second area. In a fifth example of the method, optionally including one or more or each of the first example to the fourth example, receiving the request to determine the A / B ratio of the ROI includes receiving the request to determine the A / B ratio of a lesion, the lesion being imaged in the elastography image and the B-mode image.

[0053] When introducing elements of various embodiments of the present disclosure, the words "a", "an", and "the" are intended to mean that there is one or more of these elements. The terms "first", "second", etc. do not denote any order, quantity, or importance, but are used to distinguish one element from another. The terms "comprising", "including", and "having" are intended to be inclusive and mean that additional elements may exist in addition to the listed elements. As used herein, terms such as "connected to", "coupled to", etc., an object (e.g., a material, element, structure, component, etc.) may be connected to or coupled to another object, regardless of whether the one object is directly connected or coupled to the other object, or whether there is one or more intervening objects between the one object and the other object. Further, it should be understood that references to "one embodiment" or "an embodiment" of the present disclosure are not to be construed as excluding the existence of additional embodiments that also incorporate the recited features.

[0054] Except for any previously indicated modifications, many other variations and alternative arrangements can be designed by those skilled in the art without departing from the substance and scope of this description, and the appended claims are intended to cover such modifications and arrangements. Thus, although the information has been specifically and detailedly described above in connection with the presently considered most practical and preferred aspects, it will be apparent to those of ordinary skill in the art that many modifications can be made without departing from the principles and concepts set forth herein, including but not limited to form, function, mode of operation, and use. Similarly, as used herein in all respects, the examples and embodiments are intended to be illustrative only and should not in any way be construed as restrictive.

Claims

1. A method, the method comprises: receiving a request to determine an A / B ratio of a region of interest (ROI) of an elastography image, the elastography image including a lower B-mode image and a stack on the B-mode image, the stack including elastography information of tissue imaged in the B-mode image and measured by an ultrasound probe of an ultrasound system; upon receiving the request, adjusting a gain of the stack of the elastography image to generate a processed elastography image; inputting the processed elastography image and the B-mode image as inputs to an A / B ratio model, the A / B ratio model being trained to output the A / B ratio based on the processed elastography image and the B-mode image; and displaying the A / B ratio on a display device, wherein the A / B ratio is a ratio of a width and / or area of the ROI in the elastography image to a width and / or area of the ROI in the B-mode image.

2. The method according to claim 1, wherein the A / B ratio is a ratio of a first area of the ROI in the elastography image to a second area of the ROI in the B-mode image.

3. The method according to claim 2, wherein the A / B ratio of the ROI is automatically determined via the A / B ratio model comprises: identifying a first boundary of the ROI in the elastography image via the A / B ratio model and determining the first area of the ROI in the elastography image based on the identified first boundary; identifying a second boundary of the ROI in the B-mode image via the A / B ratio model and determining the second area of the ROI in the B-mode image based on the identified second boundary; and determining the A / B ratio based on the first area and the second area.

4. The method according to claim 1, wherein the elastography image is a shear wave elastography image or a strain elastography image.

5. The method according to claim 1, wherein the stack includes elastography information indicating a measured stiffness of tissue imaged in the B-mode image.

6. The method according to claim 1, adjusting the gain of the stack of the elastography image to generate the processed elastography image further includes adjusting the transparency and gain of the stack.

7. The method according to claim 6, wherein adjusting the transparency and the gain of the elastography image includes adjusting the transparency to a minimum transparency and adjusting the gain to a maximum gain.

8. The method according to claim 1, the method further includes storing the A / B ratio in a memory as part of a patient examination.

9. A system, the system comprises: a display device; an ultrasound probe; a memory storing instructions; and a processor communicatively coupled to the memory and configured to, when executing the instructions: acquire a B-mode image of a region of interest (ROI) of a patient via the ultrasound probe; Obtain an elastography image of the ROI of the patient via the ultrasound probe, where the elastography image includes the B-mode image and a stack on the B-mode image, and the stack includes elastography information of tissue imaged in the B-mode image and measured by the ultrasound probe of the ultrasound system; Receive a request to determine the A / B ratio of the ROI; When the request is received, adjust the gain and / or transparency of the stack of the elastography image to generate a processed elastography image; Input the B-mode image and the processed elastography image as inputs to an A / B ratio model, and the A / B ratio model is trained to output the A / B ratio of the ROI based on the B-mode image and the processed elastography image; and Output the A / B ratio for display on the display device, where the A / B ratio is the ratio of the width and / or area of the ROI in the elastography image to the width and / or area of the ROI in the B-mode image.

10. The system according to claim 9, wherein the elastography image is acquired and / or processed to have maximum gain and minimum transparency.

11. The system according to claim 10, wherein the transparency of the elastography image is the transparency of a stack indicating the measured stiffness of the patient's tissue, and the stack is superimposed on the B-mode image.

12. The system according to claim 9, wherein the A / B ratio model includes a first image segmentation model trained to identify a first boundary of the ROI in the elastography image and a second image segmentation model trained to identify a second boundary of the ROI in the B-mode image.

13. The system according to claim 12, wherein the A / B ratio model determines a first area of the ROI in the elastography image based on the identified first boundary, and determines a second area of the ROI in the B-mode image based on the identified second boundary, and determines the A / B ratio as the ratio of the first area to the second area.

14. The system according to claim 9, wherein the elastography image is a shear wave elastography image or a strain elastography image.

15. A method for an ultrasound system, the method comprises: Receive a request to determine the A / B ratio of a region of interest (ROI) of an elastography image, the elastography image including a lower-layer B-mode image and a stack on the B-mode image, the stack including elastography information of tissue imaged in the B-mode image and measured by the ultrasound probe of the ultrasound system; When the request is received, adjust the transparency of the stack of the elastography image to generate a processed elastography image; Input the processed elastography image and the lower-layer B-mode image as inputs to a model, and the model is trained to output the A / B ratio based on the processed elastography image and the lower-layer B-mode image; and Output the A / B ratio for display on a display device, Wherein the A / B ratio is the ratio of the width and / or area of the ROI in the elastography image to the width and / or area of the ROI in the B-mode image.

16. The method according to claim 15, wherein receiving the request includes receiving the request while the elastography image is displayed on the display device, and the elastography image is displayed together with the overlay at a first transparency.

17. The method according to claim 16, wherein adjusting the transparency includes adjusting the transparency from the first transparency to a second transparency, and the first transparency is higher than the second transparency.

18. The method according to claim 15, wherein the A / B ratio is the ratio of a first area of the ROI in the elastography image to a second area of the ROI in the B-mode image.

19. The method according to claim 18, wherein the model is trained to identify and segment the ROI in the elastography image to determine the first area, and to identify and segment the ROI in the B-mode image to determine the second area.

20. The method according to claim 15, wherein receiving the request to determine the A / B ratio of the ROI includes receiving a request to determine the A / B ratio of a lesion that is imaged in the elastography image and the B-mode image.

Images