System and method for a measurement tool for medical imaging

Abstract

Methods and systems for assessing liver disease in a subject using ultrasound images are provided. In one example, a method includes, in response to a request to assess liver disease, determining, with a measurement model, that a selected medical image frame of a subject includes a target anatomic view and has an image quality above a threshold image quality, and in response, measuring, with the measurement model, a marker of liver disease in the selected medical image; and outputting the measurement of the marker for display on a display device.

Description

Technical Field

[0001] The embodiments of the subject matter disclosed herein relate to medical imaging, and more specifically to automated measurement tools for using medical imaging to assess liver diseases. Background Technology

[0002] Medical ultrasound is an imaging modality that uses ultrasound waves to probe the internal structures of a patient's body and generate corresponding images. For example, an ultrasound probe, comprising multiple transducer elements, emits ultrasound pulses that are reflected, refracted, or absorbed by structures within the body. The ultrasound probe then receives the reflected echoes, which are processed into images. Ultrasound images of internal structures can be saved for later analysis by clinicians to aid in diagnosis and / or can be displayed in real-time or near real-time on a display device. Summary of the Invention

[0003] In one embodiment, a method for assessing liver disease in a subject includes: in response to a request to assess liver disease, using a measurement model to determine the image quality of a selected medical image frame of the subject, which includes a target anatomical view and has an image quality above a threshold; and in response, using the measurement model to measure markers of liver disease in the selected medical image; and outputting the measurement of the markers for display on a display device.

[0004] The above-described advantages, as well as other advantages and features, of this specification will become apparent, either alone or in connection with the accompanying drawings, from the following detailed description. It should be understood that the above summary is provided to present a simplified version of the selected concepts further described in the detailed description. This is not intended to identify key or essential features of the claimed subject matter, the scope of which is uniquely defined by the claims following the detailed description. Furthermore, the claimed subject matter is not limited to embodiments that address any of the disadvantages mentioned above or in any part of this disclosure. Attached Figure Description

[0005] A better understanding of the various aspects of this disclosure can be achieved by reading the following detailed description and referring to the accompanying drawings, in which:

[0006] Figure 1 A block diagram illustrating an implementation scheme for an ultrasound system is shown;

[0007] Figure 2 This is a schematic diagram illustrating a system for measuring medical images using a measurement model, according to an embodiment.

[0008] Figures 3 to 5 Exemplary ultrasound images and corresponding annotations generated via a measurement model are shown;

[0009] Figure 6This is a flowchart illustrating an exemplary method for assessing liver disease using a measurement model; and

[0010] Figure 7 This is a flowchart illustrating an exemplary method for training a measurement model. Detailed Implementation

[0011] Medical ultrasound imaging typically involves placing an ultrasound probe, comprising one or more transducer elements, onto the object being imaged (such as a patient) at the location of a target anatomical feature (e.g., abdomen, chest, etc.). Images are acquired by the ultrasound probe and displayed on a display device in real-time or near real-time (e.g., images are displayed as soon as they are generated without intentional delay). The operator of the ultrasound probe can view the images and adjust various acquisition parameters and / or position of the ultrasound probe to obtain high-quality images of one or more target anatomical features (e.g., heart, liver, kidney, or another anatomical feature).

[0012] Because ultrasound imaging is relatively inexpensive, portable, non-invasive, and ubiquitous, it is often superior to biopsies or other types of medical imaging (e.g., computed tomography or magnetic resonance imaging) for assessing and monitoring patient conditions such as disease progression. However, the lack of standardization in acquired ultrasound images due to operator-to-operator variations can make the diagnosis and / or monitoring of certain conditions challenging. For example, non-alcoholic fatty liver disease (NAFLD), commonly found in obese patients, is defined as fat accumulation in more than 5% of hepatocytes and is the most prevalent liver disease. NAFLD is currently diagnosed via liver biopsy. Liver biopsies are invasive and not suitable for monitoring disease risk or progression in large numbers of prospective or diagnosed patients.

[0013] Therefore, NAFLD is a patient condition well-suited for diagnosis and monitoring via ultrasound, facilitating early detection and continuous monitoring of a disease currently undetectable by liver biopsy. However, given the operator-dependent nature of ultrasound imaging, diagnosing or monitoring NAFLD using ultrasound imaging of the liver itself has been challenging to date. For example, variations in operator experience, ultrasound equipment, and settings can result in acquiring images with different anatomical views and image quality. Using these different images for monitoring or diagnosing NAFLD can lead to inconsistent and unreliable assessments. Consequently, the widespread use of ultrasound for assessing NAFLD has not yet been adopted.

[0014] Therefore, according to the embodiments disclosed herein, NAFLD can be diagnosed and staged by measuring abdominal visceral fat on ultrasound images using a measurement tool. The measurement tool may include an artificial intelligence classification model (e.g., the measurement model described below) that is used to detect a standard plane suitable for visceral fat measurement, measure image quality for visceral fat measurement, process the visceral fat measurement (including calculating the average measurement of multiple visceral fat measurements), and predict the risk of NAFLD based on the visceral fat measurement. Measuring abdominal visceral fat to diagnose NAFLD by exposing the artificial intelligence classification model to multiple preliminary or early NAFLD cases facilitates the early diagnosis of NAFLD, and using an artificial intelligence classification model to assist visceral fat measurement may lead to more accurate testing, measurement, and prediction, including the diagnosis of NAFLD in asymptomatic patients. By ensuring that a standard plane and high-quality images are used for visceral fat measurement, the measurement model described herein improves the accuracy of visceral fat measurement, which reduces false positive and false negative results and reduces operator-to-operator variability in measurement.

[0015] Figure 1 An exemplary ultrasound system is illustrated, comprising an ultrasound probe, a display device, and an imaging processing system. Ultrasound images can be acquired via the ultrasound probe and displayed on the display device. Figure 2 The image processing system shown includes a measurement model that may include multiple sub-models trained to perform various tasks for assessing liver disease, such as anatomical view detection, image quality detection, anatomical segmentation and measurement, and disease risk / progression analysis. The measurement model can be based on... Figure 6 Deployment is carried out using the method and according to Figure 7 The method is used to train the measurement model to assess liver diseases such as NAFLD. Once the measurement model identifies images of sufficient quality in the target view, it measures the target anatomical features (in this case, visceral fat surrounding the liver) and determines disease risk or progression based on the measured target anatomical features. The measurements and disease risk or progression can be displayed along with the images, such as... Figures 3 to 5 As shown.

[0016] See Figure 1A schematic diagram of an ultrasound imaging system 100 according to an embodiment of the present disclosure is shown. The ultrasound imaging system 100 includes a transmit beamformer 101 and a transmitter 102 that drives elements (e.g., transducer elements) 104 within a transducer array (referred to herein as probe 106) to transmit pulsed ultrasound signals (referred herein as transmit pulses) into a body (not shown). According to one embodiment, probe 106 may be a one-dimensional transducer array probe. However, in some embodiments, probe 106 may be a two-dimensional matrix transducer array probe. Various geometries and configurations can be used for the transducer array (e.g., linear, convex, phased array), and the transducer array may be provided as part of, for example, different types of ultrasound probes, including but not limited to standard handheld linear or curved probes, endogenous probes, and cardiac probes. As further explained below, transducer element 104 may be made of a piezoelectric material. When a voltage is applied to a piezoelectric crystal, the crystal physically expands and contracts, thereby emitting ultrasound spherical waves. In this way, the transducer element 104 can convert the electronic emission signal into an acoustic emission beam.

[0017] After the element 104 of probe 106 transmits a pulsed ultrasound signal into (e.g., a patient's) body, the pulsed ultrasound signal is backscattered from internal structures such as blood cells or muscle tissue to generate an echo returning to element 104. The echo is converted into an electrical signal or ultrasound data by element 104, and the electrical signal is received by receiver 108. The electrical signal representing the received echo passes through receiver beamformer 110, which outputs ultrasound data. Additionally, transducer element 104 can generate one or more ultrasound pulses based on the received echo to form one or more transmit beams.

[0018] According to some embodiments, probe 106 may include electronic circuitry to perform all or part of transmit beamforming and / or receive beamforming. For example, all or part of transmit beamformer 101, transmitter 102, receiver 108, and receive beamformer 110 may be located within probe 106. In this disclosure, the terms “scanning” or “under scanning” may also be used to refer to the process of acquiring data by transmitting and receiving ultrasound signals. In this disclosure, the term “data” may be used to refer to one or more datasets acquired using an ultrasound imaging system. In one embodiment, data acquired via ultrasound imaging system 100 may be used to train a machine learning model. User interface 115 may be used to control the operation of ultrasound imaging system 100, including for controlling the input of patient data (e.g., patient history), for changing scan or display parameters, for initiating probe repolarization sequences, etc. User interface 115 may include one or more of the following: a rotary element, a mouse, a keyboard, a trackball, hard keys linked to specific actions, soft keys configurable to control different functions, and a graphical user interface displayed on display device 118.

[0019] The ultrasound imaging system 100 also includes a processor 116 for controlling the transmitting beamformer 101, the transmitter 102, the receiver 108, and the receiving beamformer 110. The processor 116 communicates electronically (e.g., is communicatively connected) with the probe 106. For the purposes of this disclosure, the term "electronic communication" may be defined to include both wired and wireless communication. The processor 116 can control the probe 106 to acquire data according to instructions stored in the processor's memory and / or memory 120. The processor 116 controls which of the elements 104 are active and the shape of the beam emitted from the probe 106. The processor 116 also communicates electronically with a display device 118 and can process data (e.g., ultrasound data) into images for display on the display device 118. The processor 116 may include a central processing unit (CPU) according to one embodiment. According to other embodiments, the processor 116 may include other electronic components capable of performing processing functions, such as a digital signal processor, a field-programmable gate array (FPGA), or a graphics board. According to other embodiments, processor 116 may include multiple electronic components capable of performing processing functions. For example, processor 116 may include two or more electronic components selected from a list of electronic components, including: a central processing unit, a digital signal processor, a field-programmable gate array, and a graphics board. According to another embodiment, processor 116 may also include a composite demodulator (not shown) that demodulates RF data and generates raw data. In another embodiment, demodulation may be performed earlier in the processing chain. Processor 116 is adapted to perform one or more processing operations based on multiple selectable ultrasound modalities on the data. In one example, data may be processed in real time during a scanning session because the echo signal is received by receiver 108 and transmitted to processor 116. For the purposes of this disclosure, the term "real time" is defined as including a procedure executed without any intentional delay. For example, embodiments may acquire images at a real-time rate of 7 frames / second to 20 frames / second. Ultrasonic imaging system 100 is capable of acquiring 2D data of one or more planes at a significantly faster rate. However, it should be understood that the real-time frame rate may depend on the length of time spent acquiring each frame of data for display. Therefore, the real-time frame rate may be slow when acquiring relatively large amounts of data. Therefore, some embodiments may have a real-time frame rate significantly faster than 20 frames per second, while other embodiments may have a real-time frame rate less than 7 frames per second. Data may be temporarily stored in a buffer (not shown) during a scanning session and processed in a less real-time manner in real-time or offline operation. Some embodiments of the invention may include multiple processors (not shown) to handle the processing tasks processed by processor 116 according to the exemplary embodiments described above.For example, before displaying an image, a first processor can be used to demodulate and extract the RF signal, while a second processor can be used to further process the data (e.g., by augmenting the data as further described herein). It should be understood that other embodiments may use different processor arrangements.

[0020] The ultrasound imaging system 100 can continuously acquire data at frame rates, for example, from 10 Hz to 30 Hz (e.g., 10 to 30 frames per second). Images generated from the data can be refreshed on a display device 118 at a similar frame rate. Other embodiments are capable of acquiring and displaying data at different rates. For example, depending on the frame size and the intended application, some embodiments may acquire data at frame rates less than 10 Hz or greater than 30 Hz. A memory 120 is included for storing frames of processed acquired data. In an exemplary embodiment, the memory 120 has sufficient capacity to store at least several seconds of ultrasound data frames. The data frames are stored in a manner that facilitates retrieval based on their acquisition order or time. The memory 120 may include any known data storage medium.

[0021] In various embodiments of the invention, processor 116 can process data through different mode-related modules (e.g., B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastography, TVI, strain, strain rate, etc.) to form 2D or 3D data. For example, one or more modules can generate B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastography, TVI, strain, strain rate, and combinations thereof. As an example, one or more modules can process color Doppler data, which may include conventional color flow Doppler, power Doppler, HD flow, etc. Image lines and / or frames are stored in memory and may include timing information indicating the time when image lines and / or frames are stored in memory. These modules may include, for example, a scan conversion module for performing scan conversion operations to convert the acquired images from beam space coordinates to display space coordinates. A video processor module may be provided that reads the acquired images from memory and displays the images in real time while performing procedures (e.g., ultrasound imaging) on ​​a patient. The video processor module may include a separate image memory, and ultrasound images may be written to the image memory for reading and display by the display device 118.

[0022] In various embodiments of this disclosure, one or more components of the ultrasound imaging system 100 may be included in a portable handheld ultrasound imaging device. For example, a display device 118 and a user interface 115 may be integrated into the external surface of the handheld ultrasound imaging device, which may also include a processor 116 and a memory 120. A probe 106 may include a handheld probe that electronically communicates with the handheld ultrasound imaging device to collect raw ultrasound data. Transmit beamformer 101, transmitter 102, receiver 108, and receive beamformer 110 may be included in the same or different parts of the ultrasound imaging system 100. For example, transmit beamformer 101, transmitter 102, receiver 108, and receive beamformer 110 may be included in the handheld ultrasound imaging device, the probe, and combinations thereof.

[0023] After performing a two-dimensional ultrasound scan, a data block containing scan lines and their samples is generated. Following the application of a back-end filter, a process called scan transformation is performed to transform the two-dimensional data block into a displayable bitmap image with additional scan information, such as depth, the angle of each scan line, etc. During scan transformation, interpolation techniques are applied to fill in any missing holes (i.e., pixels) in the resulting image. These missing pixels occur because each element of the two-dimensional block should typically cover many pixels in the resulting image. For example, in current ultrasound imaging systems, bicubic interpolation is used, which utilizes the adjacent elements of the two-dimensional block.

[0024] The ultrasound images acquired by the ultrasound imaging system 100 can be further processed. In some embodiments, the ultrasound images generated by the ultrasound imaging system 100 can be transmitted to an image processing system, wherein in some embodiments, the ultrasound images can be analyzed using one or more machine learning models trained with the ultrasound images and corresponding ground truth outputs to estimate current user actions (e.g., probe movement before and / or during the acquisition of the current ultrasound image), current anatomical views in the ultrasound images, and probe movement suggestions. As used herein, ground truth output refers to the expected or “correct” output based on a given input to the machine learning model. For example, if a machine learning model is being trained to classify images of cats, then when an image of a cat is fed, the model’s ground truth output is the label “cat”. As explained in more detail below, if a machine learning model is being trained to classify ultrasound images based on probe movement / user actions occurring before and / or during the acquisition of ultrasound images, then the model’s ground truth output can be a label indicating probe movement / user action, such as a label indicating translation, rotation, etc. Similarly, if a machine learning model is being trained to classify ultrasound images based on anatomical features in the ultrasound images, the model's ground truth output can be a label indicating one or more anatomical features in the ultrasound image.

[0025] Although described herein as a separate system, it should be understood that in some embodiments, the ultrasound imaging system 100 includes an image processing system. In other embodiments, the ultrasound imaging system 100 and the image processing system may comprise separate devices. In some embodiments, images generated by the ultrasound imaging system 100 can be used as a training dataset for training one or more machine learning models, wherein, as described below, the machine learning models can be used to perform one or more steps of ultrasound image processing.

[0026] See Figure 2 The image processing system 202 is illustrated according to an exemplary embodiment. In some embodiments, the image processing system 202 is integrated into the ultrasound imaging system 100. For example, the image processing system 202 may be provided as a processor 116 and a memory 120 within the ultrasound imaging system 100. In some embodiments, at least a portion of the image processing system 202 is located at a device (e.g., an edge device, server, etc.) communicatively coupled to the ultrasound imaging system via a wired and / or wireless connection. In some embodiments, at least a portion of the image processing system 202 is located at a separate device (e.g., a workstation) that can receive images / maps from the ultrasound imaging system or from a storage device storing images / data generated by the ultrasound imaging system. The image processing system 202 may be operatively / communically coupled to a user input device 232 and a display device 234. In at least some examples, the user input device 232 may include a user interface 115 of the ultrasound imaging system 100, while the display device 234 may include a display device 118 of the ultrasound imaging system 100.

[0027] Image processing system 202 includes processor 204 configured to execute machine-readable instructions stored in non-transitory memory 206. Processor 204 may be a single-core or multi-core processor, and programs executing thereon may be configured for parallel or distributed processing. In some embodiments, processor 204 may optionally include individual components distributed across two or more devices that can be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of processor 204 may be virtualized and executed by remotely accessible networked computing devices configured in a cloud computing configuration.

[0028] Non-transitory memory 206 may store measurement model 208, training module 210, and ultrasound image data 212. Measurement model 208 may include one or more machine learning models, such as deep learning networks, which include multiple weights and biases, activation functions, loss functions, gradient descent algorithms, and instructions for implementing one or more deep neural networks to process input ultrasound images. Measurement model 208 may include trained and / or untrained neural networks and may also include training routines or parameters (e.g., weights and biases) associated with one or more neural network models stored therein.

[0029] Measurement model 208 can generate multiple outputs for a given ultrasound image. For example, measurement model 208 can output an anatomical view 207 of the image, image quality 209, segmentation and measurement 211 of the image, and measurement-based disease risk or progression 213. In one example, measurement model 208 can be multiple models, where each output for a given ultrasound image can be generated by a separate model. In such examples, measurement models may include an anatomical view model, an image quality model, a segmentation and measurement model, and a risk or progression model. In some examples, the output of one model can be used as input to another model, or the output of one model can be evaluated to determine whether the ultrasound image should be input to other models. For example, the outputs from the anatomical view model and the image quality model can be evaluated to determine whether the ultrasound image should be rejected or should be input to the segmentation and measurement model.

[0030] The anatomical view 207 determined / output by measurement model 208 may include a current anatomical view (e.g., scan plane or anatomical region) of the current ultrasound image. For example, if the ultrasound image includes the liver and / or is acquired as part of an abdominal scan, measurement model 208 may be trained to determine whether the input ultrasound image includes a target anatomical view of the liver / abdomen, such as a long-axis aortic view. In some examples, anatomical view 207 may also include anatomical features identified in the current ultrasound image. For example, segmentation may be used to identify anatomical features in the current ultrasound image, and the identified anatomical features may then be used to determine the scan plane / anatomical region imaged in the current ultrasound image. In some examples, in addition to identifying the current anatomical view, measurement model may be trained to determine the proximity (e.g., in degrees and / or distance measurements) of the current anatomical view to a standard or target view for which visceral fat measurements will be performed. For example, if the target view is a long-axis aortic view, measurement model may be determined to identify whether the current anatomical view is a long-axis aortic view, and if so, to identify the degree of matching between the current view and a standard long-axis aortic view. The model can determine whether the current view is tilted at an angle relative to the standard long-axis aortic view (e.g., tilted at 5 degrees relative to the plane of the standard long-axis aortic view) and whether the centering of the current view is similar to or different from the standard long-axis aortic view (e.g., offset vertically and / or horizontally by a certain distance, such as a vertical offset of 3 cm).

[0031] The image quality 209 determined / output by measurement model 208 may include indicators (visual or non-visual) indicating whether the current ultrasound image is acceptable or unacceptable for visceral fat measurement. For example, if the ultrasound image includes the liver and / or is acquired as part of an abdominal scan, measurement model 208 may be trained to determine whether the input ultrasound image has threshold image resolution, image contrast, noise, and / or brightness. Furthermore, the image quality 209 determined by the measurement model may be a function of the measurement model's ability to segment and measure visceral fat (as described below), and may therefore be based on the sharpness of edges of specific anatomical features (e.g., liver, visceral fat, linea alba, etc.), the lack of artifacts in the image (particularly in the area to be measured), the lack of motion-induced blur, etc. In some examples, the image quality 209 output may include a quality metric, such as a value in the range of 1-10, which may be compared to a quality or adequacy threshold to determine whether the image quality of the current image frame is high enough to accurately perform visceral fat measurement, and this adequacy threshold may be learned by the measurement model during training, as explained in more detail below. In some examples, the adequacy threshold can be adjusted based on the current anatomical view (e.g., based on the proximity of the current anatomical view to the standard / target anatomical view).

[0032] The segmentation and measurement 211 determined / output by measurement model 208 may include the measurement of visceral fat thickness in the current ultrasound image. For example, if the ultrasound image includes the liver and / or is acquired as part of an abdominal scan, measurement model 208 may be trained to segment the ultrasound image and determine the visceral fat thickness in the ultrasound image. Segmentation performed by measurement model 208 may include determining and outputting visual indicators around the top and bottom visceral fat in the ultrasound image. In one example, for image clarity, measurement model 208 may output visual indicators around multiple segments of the ultrasound image, such as subcutaneous adipose tissue, the linea alba, and / or the top and bottom lobes of the liver. In one example, measurement model 208 may colorize segmented regions of the ultrasound image. Measurement model 208 may be trained to measure visceral fat thickness from a uniform location in the ultrasound image (e.g., along the central axis of the ultrasound image) to a uniform location. In one example, measurement model 208 may be trained to measure the visceral fat thickness at the thickest point of visceral fat in the ultrasound image. In another example, measurement model 208 can be trained to measure visceral fat thickness at multiple locations along the visceral fat layer and determine the average visceral fat thickness for a given image frame. In one example, measurement model 208 can be trained to measure visceral fat thickness across multiple frames and calculate an average value.

[0033] The disease risk or progression 213 determined / output by measurement model 208 may include a predicted value (e.g., a pictogram, scale, color map) indicating a prediction of the presence and / or progression of NAFLD based on the current ultrasound image. For example, if the ultrasound image includes the liver and / or is acquired as part of an abdominal scan, measurement model 208 may be trained to determine the risk of having NAFLD and output the prediction as a value, pictogram, progression bar, color map, etc. In one example, recommendations for future actions (e.g., outpatient visits, scans) and recommendations on when to schedule future actions based on the predictions made by measurement model 208 may be displayed. In one example, measurement model 208 may predict the progression of NAFLD and the possible outcomes for the patient in the event of no future action, in the event of taking the recommended future action, etc.

[0034] Therefore, the measurement model 208 can be one or more deep learning networks (e.g., neural networks) that take one or more ultrasound images as input and produce multiple outputs, including anatomical views, image quality, organ / anatomical feature segmentation, visceral fat measurement, and liver disease risk or progression. This network can be jointly trained with loss functions for all these tasks. To determine the anatomical views determined / output by the measurement model, expert-provided labels / annotations can be used to train the measurement model, and appropriate loss functions can be added to the joint model being trained. Annotations may include indications of scan planes / anatomical features in each image to train the model to determine the anatomical views. To segment and measure visceral fat, the measurement model 208 can be trained with images annotated by a clinician expert (e.g., the expert may measure visceral fat in multiple images, and these images may include labels indicating measurement points for determining visceral fat thickness and / or values ​​indicating visceral fat thickness).

[0035] To determine image quality, measurement model 208 can be trained with multiple images of varying qualities. A clinician expert or a standalone model (e.g., a neural network) can select or generate images whose quality is intentionally reduced and labeled as acceptable or unacceptable to train measurement model 208 to determine which factors constitute acceptable or unacceptable images. In another example, measurement model 208 can be trained using training data comprising multiple image sets to determine image quality, each set including high-quality images and subsets of images generated from the high-quality images, with each subset progressively decreasing in quality (e.g., generated by an expert or automatically by another model, such as a neural network). Lower-quality images can be generated by removing pixel information from high-quality images (e.g., progressively removing an increasing amount of pixel information), introducing artifacts into high-quality images (or not compensating for artifacts within the image data of high-quality images), blurring the edges of anatomical features within high-quality images, and so on. Once measurement model 208 has been trained to segment and measure visceral fat, images with intentionally reduced quality can be fed into a partially trained measurement model 208 capable of segmenting and measuring visceral fat in the images. Measurement model 208 can output a measurement and compare it to a ground-based measurement, which could be a measurement performed by an expert on the same image but of higher quality (e.g., before intentionally degrading the quality). Alternatively, measurement model 208 can perform measurements on both high-quality images and all corresponding degraded images, and compare the measurements performed on the degraded images with those performed on the high-quality images (e.g., the measurements performed on the high-quality images could be ground-based measurements). If measurement model 208 cannot accurately measure visceral fat on a given image, the image quality can be considered insufficient, and the model can learn which image features provide sufficient quality and insufficient quality. As an example, the model can be trained to output an image quality metric indicating the relative level of image quality of an input image, and the model can also learn an adequacy threshold against which the image quality metric can be compared to determine whether the input image has sufficient quality.

[0036] To determine the risk and / or progression of liver disease, historical data can be used to train a measurement model 208, where previous ultrasound images with known and quantifiable NAFLD risk and / or progression can be used to train risk / progression predictions based on given visceral fat thickness. Clinicians can select images for training to guide the measurement model 208 in identifying common historical use cases and testing it in identifying and responding to outliers and uncommon use cases. In some examples, the risk and / or progression of liver disease can also be determined based on the patient's medical history and current demographics and / or risk factors (such as the patient's weight, age, sex, geographic location, diagnostic laboratory tests (e.g., blood test results), which can be retrieved from the patient's electronic medical record). Therefore, in some examples, training data may include patient history, demographics, blood (or other biological sample) test results, and / or risk factors, in addition to ultrasound images labeled with visceral fat measurements and patient disease prognosis.

[0037] Furthermore, while measurement model 208 has been described herein as using 2D ultrasound images as input, in some examples, 3D ultrasound data may be used instead of 2D images, or in addition to 2D images. For example, 3D data may be used to determine anatomical views and anatomical feature segmentation (and visceral fat measurement). Additionally, while measurement model 208 has been described herein as being trained to analyze ultrasound images to determine the risk or progression of liver disease, similar approaches may be taken for other imaging modalities, particularly handheld or otherwise easily manipulated imaging modalities such as X-ray / fluorescence fluoroscopy scanners, near-infrared (NIR) spectroscopy scanners, and optical coherence tomography (OCT) scanners, to assess liver disease.

[0038] The non-transitory memory 206 may also include a training module 210, which includes instructions for training one or more machine learning models stored in the measurement model 208. In some embodiments, the training module 210 is not located at the image processing system 202. In such examples, the measurement model 208 therefore includes a trained and validated network.

[0039] The non-transitory memory 206 can also store ultrasound image data 212, such as Figure 1The ultrasound images captured by the ultrasound imaging system 100. The ultrasound images in ultrasound image data 212 may include ultrasound images acquired by the ultrasound imaging system 100 with different anatomical features (e.g., different amounts of visceral fat) and / or different image qualities. Furthermore, when the training module 210 is stored in the non-transitory memory 206, the ultrasound image data 212 may store ultrasound images, ground-based outputs, iterations of machine learning model outputs, and other types of ultrasound image data that can be used to train the measurement model 208. In some embodiments, the ultrasound image data 212 may store ultrasound images and ground-based outputs in an ordered format, such that each ultrasound image is associated with one or more corresponding ground-based outputs. However, in examples where the training module 210 is not located at the image processing system 202, the images / ground-based outputs that can be used to train the measurement model 208 may be stored elsewhere.

[0040] In some embodiments, the nontransitory memory 206 may include components disposed on two or more devices, which may be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of the nontransitory memory 206 may include a remotely accessible networked storage device configured in a cloud computing configuration.

[0041] User input device 232 may include one or more of the following: a touchscreen, keyboard, mouse, touchpad, motion-sensing camera, or other devices configured to enable a user to interact with and manipulate data within image processing system 202. In one example, user input device 232 may enable a user to select ultrasound images for training a machine learning model, to indicate or mark the location of an interventional device in ultrasound image data 212, or for further processing using the trained machine learning model.

[0042] Display device 234 may include one or more display devices utilizing virtually any type of technology. In some embodiments, display device 234 may include a computer monitor and may display ultrasound images. Display device 234 may be combined with processor 204, nontransitory memory 206 and / or user input device 232 in a shared housing, or it may be a peripheral display device and may include a monitor, touch screen, projector or other display devices known in the art that enable a user to view ultrasound images generated by the ultrasound imaging system and / or interact with various data stored in nontransitory memory 206.

[0043] It should be understood that Figure 2 The image processing system 202 shown is for illustration and not limitation. Another suitable image processing system may include more, fewer, or different components.

[0044] Figure 3 This is an exemplary view 300 of the liver during an ultrasound examination. View 300 may include a display on a display device (e.g., Figure 2 An ultrasound image 302 is displayed on a display device 234, the ultrasound image having a measurement model (such as...) Figure 2 The output of the measurement model 208 is a graphical representation of the result in response to an ultrasound image being input into the measurement model. View 300 shows a segmentation of visceral fat 304 defined by a first line 306 on the top side of visceral fat 304 (e.g., at the boundary between visceral fat and the linea alba) and a second line 308 on the bottom side of visceral fat 304 (e.g., at the boundary between the liver and visceral fat). View 300 shows a measurement of visceral fat thickness displayed in a graphical text box 310. In one example, view 300 is capable of displaying the visceral fat thickness measurement in multiple units (e.g., imperial, metric). View 300 may show a first measurement point 314 and a second measurement point 316 placed by a measurement model, and a measurement line 312 spanning from the first measurement point 314 to the second measurement point 316. The first measurement point 314 may be a point located on the top side of the visceral fat 304, the second measurement point 316 may be a point located on the bottom side of the visceral fat 304, and the measurement line 312 may be the measurement distance between the first measurement point 314 and the second measurement point 316. The measurement line 312 can be used to determine the thickness of the visceral fat 304, which is represented in the graphic text box 310. In one example, view 300 may show a visceral fat thickness measurement taken along the central axis of the image at locations such as the thickest part of the visceral fat.

[0045] For example, when calculating visceral fat measurements at the thickest part of a segmented visceral fat region, a trained measurement model can segment the visceral fat region and then calculate and automatically place the measuring calipers at the thickest part of the segmented region (e.g., the first measuring point 314 and the second measuring point 316), and then calculate the length of the thickest part of the visceral fat.

[0046] Additionally, the measurement model can be trained to segment subcutaneous adipose tissue regions, and based on the segmented regions of subcutaneous adipose tissue, it can calculate and automatically place two measuring calipers corresponding to the thickest areas of subcutaneous adipose tissue. The length between these calipers can be calculated and displayed.

[0047] Therefore, in addition to visceral fat thickness measurement, two additional measurements can be automatically calculated and displayed to the user: a) the thickness of subcutaneous adipose tissue, and b) the ratio of subcutaneous adipose tissue thickness to visceral fat thickness. In some examples, the user can choose whether to view only the visceral fat measurement or whether to view both visceral fat and subcutaneous adipose tissue measurements and / or the ratio of subcutaneous adipose tissue thickness to visceral fat thickness.

[0048] Figure 4 This includes the display on the display device (e.g., Figure 2 Another exemplary view 400 of the ultrasound image 302 on the display device 234. View 400 shows a thickness measurement of visceral fat 304 displayed in a graphical text box 310, similar to view 300 explained above. A predictive risk graphical element 404 may be displayed in view 400, showing a prediction of the presence of NAFLD output by a measurement model based on the thickness measurement of visceral fat 304. The predictive risk graphical element 404 may be displayed in view 400 as a partitioned bar with indicators, where an indicator on the left side of the partitioned bar indicates a lower predicted risk of the presence of NAFLD, and an indicator on the right side of the partitioned bar indicates a higher predicted risk of the presence of NAFLD. The predicted risk displayed within the graphical element 404 may be displayed in view 400 as text, including “low,” “high,” etc. The predicted risk may also be determined based on multiple patient data and visceral fat thickness (including but not limited to demographic patient data and patient medical data).

[0049] Figure 5 This includes the display on the display device (e.g., Figure 2 Another exemplary view 500 of the ultrasound image 302 on the display device 234. View 500 shows a thickness measurement of visceral fat 304 displayed in a graphical text box 310. A progression bar 504 may be displayed in view 500, showing a prediction of NAFLD progression in the liver as output by the measurement model. In the example shown, the progression bar 504 may include a color gradient (e.g., black / white / gray) that changes according to disease progression (e.g., from light to dark), and arrows may be placed along the progression bar 504 to indicate the currently estimated disease progression. In one example, view 500 may represent a view of a follow-up examination when a patient has been diagnosed with NAFLD, where the progression bar 504 may represent a comparative measurement of NAFLD based on visceral fat thickness measurements from the current examination and previous examinations. Although Figure 4 and Figure 5 This excludes lines dividing visceral fat or measurement lines between measurement points; however, it should be understood that lines dividing visceral fat and / or measurement lines between measurement points may be displayed, similar to... Figure 3 View 300.

[0050] Figure 6 This illustrates a measurement model (such as) according to an embodiment of the present disclosure. Figure 2 A flowchart of an exemplary method 600 of a measurement model 208 is provided, which takes ultrasound images from a scan as input and displays measurements of liver diseases such as NAFLD as output. (See also...) Figures 1 to 2The system and components described herein constitute method 600, but it should be understood that method 600 may be implemented with other systems and components without departing from the scope of this disclosure. Method 600 may be implemented based on data stored in a computing device (such as...) Figure 2 The image processing system 202 executes instructions from the non-transitory memory.

[0051] At point 602, an ultrasound image is acquired and displayed on a display device. For example, this can be achieved using... Figure 1 An ultrasound probe 106 is used to acquire ultrasound images of the subject and display them to the operator via a display device 118. The images can be acquired and displayed in real time or near real time, and can be acquired using default or user-defined scanning parameters (e.g., default depth, frequency, etc.).

[0052] At 604, method 600 includes determining whether a measurement tool configured to measure visceral fat as a marker of liver disease has been requested. The measurement tool may include a measurement model (such as...) Figure 2 The measurement model 208 and instructions for implementing the model on one or more ultrasound images and making decisions / processing the output of the measurement model. The operator can request a measurement tool during an ultrasound scan via user input (e.g., by pressing a button or selecting a user interface control button). If no measurement tool is requested, method 600 can proceed to 602 to acquire more ultrasound images.

[0053] If a measurement tool has been requested at 604, method 600 may proceed to 606, which includes inputting the current image frame into the measurement model. In one example, the measurement model is... Figure 2 The measurement model 208, and therefore includes anatomical view outputs (such as... Figure 2 Anatomical view 207), image quality output (such as...) Figure 2 Image quality 209), segmentation and measurement outputs (such as image ... Figure 2 Segmentation and measurement 211) and disease risk or progression output (such as Figure 2 The risk or progression of the disease (213).

[0054] At 608, method 600 includes using a measurement model to determine the anatomical view of the current frame. The measurement model can be referenced as described above. Figure 2 The anatomical view is determined as described. For example, the current image frame can be input into a measurement model, which can be trained to determine whether the current frame is in a target view in which specific anatomical features (e.g., liver, linea alba) are present in a specific cross-section and / or orientation. As an example, the target view may include a long-axis aortic view or another suitable view.

[0055] At 610, method 600 includes determining whether the current frame belongs to the target view. In some examples, the measurement model may output a binary indicator of the anatomical view. For example, the measurement model may output an indication (e.g., yes or no) of whether the anatomical view of the current frame is the target view. In other examples, the measurement model may output the name of the anatomical view of the current frame, and the measurement tool may determine whether the anatomical view matches the target view. In some examples, the measurement model may also output a confidence score that reflects the proximity / similarity between the anatomical view of the current frame and the target view.

[0056] If at 610 the current frame does not belong to the target view, method 600 may proceed to 612, which includes acquiring the next frame in the ultrasound image. Thus, if the current image frame is not the target / standard view, no measurement may be performed. In one example, guidance may be output (e.g., displayed on a display device) before acquiring the next frame, indicating factors or parameters regarding why the current frame does not belong to the target view and / or how to acquire frames of the target view. In some examples, the measurement tool may be configured to analyze one or more additional image frames stored in memory, rather than analyzing the current frame and then guiding the operator to acquire another frame. In one example, if no frame in the scan belongs to the target view, guidance on how to acquire the target view frame for subsequent scans may be displayed. Furthermore, while method 600 is described herein as being performed during an ongoing patient examination, the measurement tool may also be requested after the examination has been performed, and images may be retrieved from an image archive (e.g., PACS) and input into the measurement model. Method 600 may proceed to 606 to input the next frame into the measurement model (e.g., regardless of whether the next frame is acquired after the current frame or from another frame / image of a full examination).

[0057] If the current frame belongs to the target view at 610, then method 600 proceeds to 611 to determine the image quality of the current frame using the model. (See above for reference.) Figure 2As explained, the measurement model can output an image quality metric (e.g., a numerical value) indicating the relative image quality of the current frame, and this quality metric can be compared to an adequacy threshold determined during the training of the measurement model, as indicated at 613. For example, the image quality metric can be a numerical value in the range of 1-10, and the adequacy threshold can be a non-zero numerical value in the range of 1-10, such as 7. If the image quality metric of the current frame is higher than the adequacy threshold, it can be determined that the current frame has sufficient quality to perform visceral fat measurement, as described below. In some examples, the adequacy threshold can be adjusted based on the anatomical view of the current frame, as indicated at 615. For example, as explained above, the measurement model can be trained to output a confidence score when evaluating the anatomical view of the current frame, where the confidence score (e.g., in the range of 0-100) indicates the degree of certainty of the model that the current anatomical view is the target view and / or the closeness of the current anatomical view to the target view. If the confidence score of the current frame is within a threshold range (e.g., below the upper threshold but above the lower threshold, such as within 80-90), the adequacy threshold can be adjusted (e.g., increased). This allows for the application of a relatively low adequacy threshold when the current frame is at or very close to the target view (e.g., the confidence score is above the threshold range), while a higher adequacy threshold can be applied if the current frame is slightly close to the target view (e.g., within the threshold range). Doing so ensures accurate measurement of visceral fat while allowing some flexibility in varying the anatomical view or image quality (but not both) of the current frame. This may increase the number of images that might meet the criteria for visceral fat measurement and reduce the total number of images acquired to perform an accurate measurement.

[0058] At 614, method 600 includes determining whether the current frame has sufficient image quality, for example, whether the quality metric of the current frame output by the model is greater than a sufficiency threshold. If the current frame does not have sufficient quality (e.g., the image quality metric is below the sufficiency threshold), method 600 proceeds to 612 to acquire or obtain the next frame, and optionally provides guidance for acquiring the next frame. If the current frame has sufficient quality (e.g., the image quality metric is above the sufficiency threshold), method 600 proceeds to 616, which includes segmenting the current frame using a measurement model to identify measurement points. Segmenting the current frame may include identifying visceral fat, including indicating the top and bottom sides of visceral fat in the current frame. Measurement points may include a first measurement point and a second measurement point located at the top and bottom sides of the visceral fat, respectively. Measurement points may be placed along the central vertical axis of the current frame, wherein the first measurement point may be placed where the top side of the visceral fat intersects the central vertical axis, and the second measurement point may be placed where the bottom side of the visceral fat intersects the central vertical axis. In one example, when the measurement model is configured to measure visceral fat thickness at multiple locations to calculate the average visceral fat thickness, multiple measurement point pairs can be used.

[0059] At 618, method 600 includes measuring visceral fat using a measurement model. Measuring visceral fat thickness may include measuring the distance from a first measurement point to a second measurement point. In one example, multiple pairs of measurement points may be used, allowing for multiple measurements of visceral fat thickness.

[0060] At 620, method 600 includes displaying a measurement of visceral fat thickness and optionally displaying a risk or progression indicator. The measurement may be determined by a measurement model and displayed on one or more display devices (e.g., a computer monitor) in the form of text (displaying the value and units of the measurement) and graphical indicators of the measurement (showing the measurement points and the measurements between them, which are overlaid on visceral fat in an image)). Disease risk or progression may be determined by a measurement model and displayed on one or more display devices. In other examples, the measurement tool may determine risk or progression based on a set of rules or another suitable mechanism, using the output of the measurement model (e.g., visceral fat thickness) and patient data (e.g., demographics, risk factors, medical history) and / or past measurements of visceral fat thickness. Disease risk or progression may be displayed as text (such as an assessment of the likelihood of having NAFLD), as graphical bars (such as an assessment of the progression of NAFLD, where the first end of the bar indicates little or no progression and the second end indicates significant progression), or as a combination of text and graphical indicators. In one example, the patient may receive comparative results during a second or third examination, where the current disease risk or progression assessment may be displayed together with or in relation to previous disease risk or progression assessments. Method 600 then concludes.

[0061] Thus, method 600 can be performed to provide an indication of whether a patient undergoing ultrasound examination may be at risk of having a liver disease such as NAFLD. NAFLD is defined as fatty liver disease spread throughout the liver. Measuring the fat content of hepatocytes would provide a direct diagnosis, but it is a technically very challenging task. Instead, as disclosed herein, visceral fat is measured; visceral fat is a “marker” of fatty liver disease (i.e., excessive visceral fat is associated with excessive fat in the liver, but it is not a direct determining factor). Visceral fat is the fat deep within the abdomen that surrounds the kidneys and other abdominal organs (e.g., the stomach, intestines). Visceral fat measurement can be performed faster and easier than other methods of diagnosing NAFLD using ultrasound, and can be performed in a fully automated manner. Measuring visceral fat thickness does not require expensive or special hardware / equipment, such as equipment for shear wave / fiber scanning techniques. Similarly, measuring visceral fat does not require users to have special technical skills or be trained. Therefore, relying on visceral fat measurement is an inexpensive, simple, non-invasive, and scalable way to screen for NAFLD in the population, and may be particularly valuable for pediatric patients.

[0062] Figure 7 This illustrates an embodiment of the present disclosure for training a measurement model (such as...) Figure 2 A flowchart of an exemplary method 700 for measurement model 208. See also... Figures 1 to 2 The system and components described herein constitute method 700, but it should be understood that method 700 may be implemented with other systems and components without departing from the scope of this disclosure. Method 700 may be implemented based on data stored in a computing device (such as...). Figure 2 The image processing system 202 executes instructions from the non-transient memory of the training module 210.

[0063] At 702, method 700 includes using a first training dataset to train a measurement model to detect a standard or target anatomical view. The first training dataset may include multiple ultrasound images annotated by experts to indicate whether the images are correct standard anatomical views (e.g., target anatomical views as explained above). In another example, the first training dataset may include multiple images of various anatomical views, each labeled by an expert with the corresponding anatomical view.

[0064] At 704, method 700 includes using a second training dataset to train a measurement model to measure visceral fat. The second training dataset may include multiple ultrasound images annotated by experts using visceral fat measurements. The measurement model can be trained by measuring visceral fat from known measurement points and comparing that measurement to known measurements in the images. Alternatively, the measurement model can be trained by measuring visceral fat using measurement points placed by the measurement model instead of known measurement points, comparing the accuracy of that measurement to the accuracy of the measurement point placement.

[0065] At 706, method 700 includes using a third training dataset to train a measurement model to detect image quality adequacy. The third training dataset may include multiple ultrasound images annotated with indications of image quality adequacy or inadequacy. The third training dataset may include high-quality images, so the measurement model can identify the quality of high-quality images (e.g., high contrast-to-noise ratio, high resolution). The third training dataset may also include lower-quality images, so the measurement model can identify the quality of images with insufficient image quality.

[0066] In some examples, such as at 708, training the measurement model to detect image quality adequacy includes generating images of reduced quality from selected images in a second training dataset. This may include multiple images with progressively lower quality, ranging from images with less significant quality reduction to those with more significant reduction, and these images may include a third training dataset. In this example, these images may be labeled with predetermined indicators of image quality, such as a quality metric. For example, the original image may be labeled 10 (indicating the highest quality), and each intentionally reduced image generated from the original image may be labeled based on the level of quality reduction (e.g., the first image may be labeled 9, indicating some quality reduction, while the second image may be labeled 5, indicating a significant quality reduction).

[0067] In some examples, such as at 710, training the measurement model to detect image quality adequacy involves inputting the original image and degraded images (such as those generated at 708) into the measurement model and receiving measurement outputs from the measurement model for each image. As explained above, the measurement model can be trained to perform visceral fat measurements.

[0068] In some examples, such as at 712, training the measurement model to detect image quality adequacy involves comparing each measurement output from 710 with a corresponding ground-based measurement. The ground-based measurement can be a measurement performed by the measurement model on the original high-quality image (e.g., before the aforementioned quality degradation) or a measurement performed by an expert on the high-quality image. The measurement performed by the measurement model at 710 can be compared with the ground-based measurement for each image, wherein for each image, the absolute value of the error between the measurement from the measurement model and the corresponding ground-based measurement can be compared with an accuracy threshold. The accuracy threshold can be used to determine which measurements from the measurement model are not accurate, for example, whether the absolute value of the error exceeds the image's accuracy threshold.

[0069] In some examples, such as at 714, training the measurement model to detect image quality adequacy includes determining an image quality adequacy threshold based on the image's overall quality, where the measurement accuracy of the measurement output from 710 is below an accuracy threshold. By using the accuracy threshold, the level of image quality degradation can be identified, where the measurement model can still make accurate measurements that meet the accuracy threshold, and the model can learn the lowest image quality level, where the measurement model can still make accurate measurements by the accuracy threshold. (See above for reference.) Figure 6 As explained, the model can thus be trained to output an image quality metric of the input image and compare that metric with a sufficiency threshold to determine whether the image quality is sufficient for performing a measurement of visceral fat. Method 700 then concludes.

[0070] The technical advantage of using the measurement model described herein to measure visceral fat to predict or diagnose the presence of NAFLD is that it provides a rapid and inexpensive diagnosis of NAFLD, risk, or progression without invasive medical procedures. Another technical advantage of this measurement model is that the image plane and image quality used for visceral fat measurement can be standardized, which improves the accuracy of the measurement.

[0071] This disclosure also provides support for a method for assessing liver disease in a subject, the method comprising: in response to a request to assess liver disease, determining, using a measurement model, that a selected medical image frame of the subject includes a target anatomical view and has image quality above an image quality threshold, and in response, measuring a marker of liver disease in the selected medical image frame using the measurement model; and outputting the measurement of the marker for display on a display device. In a first example of the method, the method further comprises determining, using the measurement model, that a previous medical image frame of the subject acquired prior to the selected medical image frame does not include a target anatomical view and / or does not have image quality above an image quality threshold, and in response, rejecting the previous medical image frame such that the marker is not measured. In a second example of the method, the second example optionally includes the first example, the measurement model comprising one or more deep learning networks, and wherein determining that the selected medical image frame includes a target anatomical view and has image quality above an image quality threshold comprises: inputting the selected medical image frame into one or more deep learning networks, wherein the one or more deep learning networks are trained to output an anatomical view of the selected medical image frame and the image quality of the selected medical image frame. In a third example of the method, optionally including one or both of the first and second examples, measuring markers of liver disease includes: inputting selected medical image frames into one or more deep learning networks, wherein the one or more deep learning networks are trained to segment the selected medical image frames to identify markers and measure the identified markers. In a fourth example of the method, optionally including one or more or each of the first to third examples, an image quality threshold is learned by one or more deep learning networks during training. In a fifth example of the method, optionally including one or more or each of the first to fourth examples, to learn the image quality threshold and to output image quality, one or more deep learning networks are trained using a training dataset comprising multiple sets of medical image frames, each set of medical image frames including high-quality image frames and a subset of image frames generated from the high-quality image frames, with the quality of each subset of image frames gradually decreasing. In a sixth example of the method, which optionally includes one or more or each of the first to fifth examples, the liver disease is non-alcoholic fatty liver disease, wherein the marker is visceral fat surrounding the liver, and wherein a measurement model is trained to measure the thickness of the visceral fat. In a seventh example of the method, which optionally includes one or more or each of the first to sixth examples, the method further includes: determining the risk of a subject having liver disease or the current progression of the subject's liver disease based on the measurement of the marker; and outputting a visual representation of the risk or the progression for display.

[0072] This disclosure also provides support for a system comprising: a display device and an image processing system configured in a non-transitory memory with instructions that, when executed, cause the image processing system to: determine whether an ultrasound image frame of a subject includes a target anatomical view and whether the image quality of the ultrasound image frame is higher than a threshold image quality; if the ultrasound image frame includes a target anatomical view and has an image quality higher than the threshold image quality, input the ultrasound image frame into a measurement model trained to measure the thickness of visceral fat present in the ultrasound image frame; and output the measurement of the visceral fat thickness and an assessment of liver disease of the subject determined based on the measurement of visceral fat for display on the display device. In a first example of the system, the threshold image quality is learned by a deep learning network during training of the deep learning network, and determining whether the image quality of the ultrasound image frame is higher than the threshold image quality includes: inputting the ultrasound image frame as input into the deep learning network, wherein the deep learning network is trained to output the image quality. In a second example of the system, optionally including the first example, a deep learning network is trained using a training dataset comprising multiple sets of ultrasound image frames to learn threshold image quality and to output image quality. Each set of ultrasound image frames includes high-quality image frames and a subset of image frames generated from the high-quality image frames, with the quality of each subset of image frames progressively decreasing. In a third example of the system, optionally including one or both of the first and second examples, determining whether an ultrasound image frame includes a target anatomical view includes feeding the ultrasound image frame as input to a deep learning network, wherein the deep learning network is trained to output an anatomical view of the ultrasound image frame and a confidence score that the anatomical view of the ultrasound image frame is a target anatomical view. In a fourth example of the system, optionally including one or more or each of the first to third examples, instructions can be executed to adjust the threshold image quality based on the confidence score. In a fifth example of the system, optionally including one or more or each of the first to fourth examples, the assessment of liver disease includes a graphical indication of the risk of having liver disease or the current progression of liver disease, wherein the liver disease is non-alcoholic fatty liver disease.

[0073] This disclosure also provides support for a method for assessing liver disease in a subject, the method comprising: in response to a request to assess liver disease, using a measurement model to determine that selected ultrasound images of the subject include a target anatomical view and have image quality sufficient to perform visceral fat measurements in the selected ultrasound images, and in response, using the measurement model to measure the thickness of visceral fat in the selected ultrasound images, wherein the measurement model is trained using a training dataset comprising multiple subsets of images to determine that the image quality is sufficient to perform visceral fat measurements, each subset of images comprising an initial image and one or more quality-reduced images generated based on the initial image; and outputting a measurement of the thickness of visceral fat for display on a display device. In a first example of the method, the measurement model is trained to output an image quality metric indicating the image quality of the selected ultrasound images, and the image quality is determined to be sufficient based on the image quality metric being greater than a sufficiency threshold, wherein the sufficiency threshold is learned by the measurement model using the training dataset. In a second example of the method, the second example optionally including the first example, the method further comprising: determining, based on the measurement, the risk of the subject having liver disease or the current progression of the subject's liver disease; and outputting a visual representation of the risk or the progression for display. In a third example of the method, the third example optionally includes one or both of the first and second examples, and the liver disease is non-alcoholic fatty liver disease. In a fourth example of the method, the fourth example optionally includes one or more or each of the first to third examples, and the measurement model includes one or more deep learning networks. In a fifth example of the method, the fifth example optionally includes one or more or each of the first to fourth examples, and measuring the thickness of visceral fat in a selected ultrasound image using the measurement model includes: inputting the selected ultrasound image as input to the measurement model, which is trained to segment the selected ultrasound image to identify visceral fat; placing measurement points on the boundary of the visceral fat; and determining the thickness by measuring the distance between the measurement points.

[0074] When describing elements of various embodiments of this disclosure, the terms “an,” “a,” and “the” are intended to refer to one or more of these elements. The terms “first,” “second,” etc., do not indicate any order, quantity, or importance, but are used to distinguish one element from another. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that additional elements may exist in addition to the listed elements. As used herein, the terms “connected to,” “linked to,” etc., indicate that an object (e.g., a material, element, structure, component, etc.) may be connected to or linked to another object, regardless of whether the object is directly connected to or linked to the other object, or whether one or more intervening objects exist between the object and the other. Furthermore, it should be understood that references to “an embodiment” or “an embodiment” of this disclosure are not intended to be construed as excluding the existence of additional embodiments that also include the referenced features.

[0075] In addition to any modifications previously indicated, those skilled in the art can devise many other variations and alternative arrangements without departing from the spirit and scope of this description, and the appended claims are intended to cover such modifications and arrangements. Therefore, although the information has been described above in a specific and detailed manner in conjunction with what is currently considered to be the most practical and preferred aspects, it will be apparent to those skilled in the art that many modifications can be made without departing from the principles and concepts set forth herein, including but not limited to changes in form, function, mode of operation, and use. Likewise, as used herein, in all respects, examples and embodiments are intended to be illustrative only and should not be construed as restrictive in any way.

Claims

1. A method for assessing liver disease in a subject, the method being implemented by a computing device, the method comprising: In response to a request to assess the liver disease, a measurement model is used to determine that selected medical image frames of the subject include a target anatomical view and have image quality above an image quality threshold, and in response, the measurement model is used to measure markers of the liver disease in the selected medical image frames; as well as The measurement of the marker is output for display on a display device. The measurement model includes one or more deep learning networks configured to take the medical image frame as input and generate multiple outputs, the multiple outputs including: a first output indicating whether the medical image frame contains the target anatomical view, a second output indicating the image quality of the medical image frame, and a third output including a measurement of the thickness of liver fat. Wherein, after the measurement model is trained to generate the third output, the measurement model is trained to output the second output, and The process of training the measurement model to generate the second output includes: inputting multiple training medical images of different qualities into the measurement model, determining the accuracy of the liver fat measured in each training medical image, and updating the measurement model based on the accuracy.

2. The method of claim 1, further comprising using the measurement model to determine that a previous medical image frame acquired prior to the selected medical image frame does not include the target anatomical view and / or does not have an image quality higher than the image quality threshold, and in response, rejecting the previous medical image frame such that the marker is not measured.

3. The method of claim 2, wherein determining that the selected medical image frame includes the target anatomical view and has an image quality higher than the image quality threshold includes: The selected medical image frame is input into one or more deep learning networks, wherein the one or more deep learning networks are trained to output an anatomical view of the selected medical image frame and the image quality of the selected medical image frame.

4. The method of claim 3, wherein measuring the markers of the liver disease comprises: The selected medical image frame is input into one or more deep learning networks, wherein the one or more deep learning networks are trained to segment the selected medical image frame in order to identify the markers and measure the identified markers.

5. The method of claim 3, wherein the image quality threshold is learned by the one or more deep learning networks during training.

6. The method of claim 5, wherein, in order to learn the image quality threshold and to output the image quality, the one or more deep learning networks are trained using a training dataset comprising a plurality of medical image frame sets, each medical image frame set comprising high-quality image frames and a subset of image frames generated based on the high-quality image frames, the quality of each subset of image frames gradually decreasing.

7. The method of claim 1, wherein the liver disease is non-alcoholic fatty liver disease, wherein the marker is visceral fat surrounding the liver, and wherein the measurement model is trained to measure the thickness of the visceral fat.

8. The method according to claim 1, further comprising: The risk of the subject having the liver disease or the current progression of the subject's liver disease is determined based on the measurement of the markers; And output a visual representation of the risk or the progress for display.

9. A system for assessing liver disease in a subject, the system comprising: Display devices; and An image processing system having instructions configured in a non-transitory memory, the instructions causing the image processing system, when executed, to: Determine whether the ultrasound image frame of the subject includes a target anatomical view and whether the image quality of the ultrasound image frame is higher than a threshold image quality; If the ultrasound image frame includes the target anatomical view and has an image quality higher than the threshold image quality, then the ultrasound image frame is input into a measurement model, which is trained to measure the thickness of visceral fat present in the ultrasound image frame; as well as The measurement of the thickness of the visceral fat and an assessment of liver disease determined in the subject based on the measurement of the visceral fat are output for display on the display device. The measurement model includes one or more deep learning networks configured to take the ultrasound image frame as input and generate multiple outputs, including: a first output indicating whether the ultrasound image frame contains the target anatomical view, a second output indicating the image quality of the ultrasound image frame, and a third output including a measurement of the thickness of the visceral fat. Wherein, after the measurement model is trained to generate the third output, the measurement model is trained to output the second output, and The process of training the measurement model to generate the second output includes: inputting multiple training ultrasound images of different qualities into the measurement model, determining the accuracy of the visceral fat measured in each training ultrasound image, and updating the measurement model based on the accuracy.

10. The system of claim 9, wherein the threshold image quality is learned by a deep learning network during training of the deep learning network, and wherein determining whether the image quality of the ultrasound image frame is higher than the threshold image quality comprises: The ultrasound image frame is fed as input into the deep learning network, which is trained to output the image quality.

11. The system of claim 10, wherein, in order to learn the threshold image quality and to output the image quality, the deep learning network is trained using a training dataset comprising a plurality of ultrasound image frame sets, each ultrasound image frame set comprising high-quality image frames and a subset of image frames generated based on the high-quality image frames, the quality of each subset of image frames gradually decreasing.

12. The system of claim 9, wherein determining whether the ultrasound image frame includes the target anatomical view comprises: The ultrasound image frame is fed as input into a deep learning network, wherein the deep learning network is trained to output an anatomical view of the ultrasound image frame and a confidence score of the anatomical view of the ultrasound image frame as the target anatomical view.

13. The system of claim 12, wherein the instructions are executable to adjust the threshold image quality based on the confidence score.

14. The system of claim 9, wherein the assessment of the liver disease includes a graphical indication of the risk of having the liver disease or the current progression of the liver disease, wherein the liver disease is non-alcoholic fatty liver disease.

15. A method for assessing liver disease in a subject, the method being implemented by a computing device, the method comprising: In response to a request to assess the liver disease, a measurement model is used to determine that selected ultrasound images of the subject include a target anatomical view and have sufficient image quality to perform visceral fat measurements in the selected ultrasound images, and in response, the thickness of the visceral fat in the selected ultrasound images is measured using the measurement model, wherein the measurement model is trained using a training dataset comprising multiple image subsets to determine that the image quality is sufficient to perform the measurement of the visceral fat, each image subset comprising an initial image and one or more degraded images generated based on the initial image; as well as The measurement of the thickness of the visceral fat is output for display on a display device. The measurement model includes one or more deep learning networks configured to take the ultrasound image as input and generate multiple outputs, including: a first output indicating whether the ultrasound image contains the target anatomical view, a second output indicating the image quality of the ultrasound image, and a third output including a measurement of the thickness of the visceral fat. Wherein, after the measurement model is trained to generate the third output, the measurement model is trained to output the second output, and The process of training the measurement model to generate the second output includes: inputting multiple training ultrasound images of different qualities into the measurement model, determining the accuracy of the visceral fat measured in each training ultrasound image, and updating the measurement model based on the accuracy.

16. The method of claim 15, wherein the measurement model is trained to output an image quality metric indicating the image quality of the selected ultrasound image, and determines that the image quality is sufficient based on the image quality metric being greater than a sufficiency threshold, wherein the sufficiency threshold is learned by the measurement model using the training dataset.

17. The method according to claim 15, further comprising: The risk of the subject having the liver disease or the current progression of the subject's liver disease is determined based on the measurement. And output a visual representation of the risk or the progress for display.

18. The method of claim 15, wherein the liver disease is non-alcoholic fatty liver disease.

19. The method of claim 15, wherein the measurement model comprises one or more deep learning networks.

20. The method of claim 15, wherein measuring the thickness of the visceral fat in the selected ultrasound image using the measurement model comprises: The selected ultrasound image is input into the measurement model, which is trained to segment the selected ultrasound image to identify the visceral fat, place measurement points on the boundary of the visceral fat, and determine the thickness by measuring the distance between the measurement points.

Images