Bioassay measurements and quality assessment

Abstract

The present disclosure describes an imaging system configured to determine the accuracy of anatomical measurements obtained from image data. The system can include an ultrasound transducer configured to acquire echo signals in response to ultrasound pulses transmitted toward a target region. The system can also include a graphical user interface configured to display a biometric tool widget, such as a caliper, for acquiring measurements of anatomical features within the target region from at least one image frame generated from the ultrasound echoes. The system can also include one or more processors configured to determine a confidence metric indicating the accuracy of the measurement. The processor can also be configured to cause the graphical user interface to display a graphical indicator corresponding to the confidence metric. The processor can implement one or more neural networks and can derive additional information, such as gestational age or weight, from the acquired anatomical measurements.

Description

Technical Field

[0001] The present disclosure relates to ultrasound systems and methods for measuring anatomical features via ultrasound imaging and determining the associated measurement quality using at least one neural network. Specific implementations relate to a system configured to generate a probability-based confidence level for each measurement obtained via an ultrasound imaging system equipped with one or more biometric tools. Background Art

[0002] Ultrasound fetal biometry is commonly used to estimate gestational age and growth trajectory for pregnancy management and is a primary diagnostic tool for potential fetal health defects. Fetal diseases are often identified based on the difference between the actual and expected relationships of specific anatomical measurements at a given gestational age. The accuracy of fetal disease identification is highly dependent on the sonographer's skill in ultrasound image acquisition and measurement extraction, which may require accurately finding the correct imaging plane for specific anatomical measurements and using virtual instruments (e.g., calipers) to obtain measurements. Intra- and inter-observer variability in ultrasound imaging and assessment often leads to incorrect estimates of fetal size and growth, which leads to unnecessary repeated examinations, increased costs, and unnecessary stress for prospective parents. In addition, the shortage of trained sonographers and the increase in ultrasound examination prescriptions have increased the pressure on health care providers to use less trained sonographers and reduced the number of repeated examinations and the length of each patient's stay. Therefore, there is a need for new technologies that are configured to obtain and analyze ultrasound images of the fetus or other anatomical features with improved accuracy and consistency in less time. Summary of the Invention

[0003] The present disclosure describes systems and methods for obtaining and analyzing ultrasound images of various anatomical objects by employing at least one deep learning neural network. Although the examples herein are specifically directed to prenatal assessment of a fetus, it will be understood by those skilled in the art that the disclosed systems and methods are described for fetal assessment for illustrative purposes only, and that anatomical measurements can be performed on various objects within a patient at a range of time points, including, for example, but not limited to, the heart and lungs. In some embodiments, the system can be configured to improve the accuracy, efficiency, and automation of prenatal ultrasound scans or ultrasound scanning protocols associated with other clinical applications (e.g., heart, liver, breast, etc.). The system can reduce ultrasound examination errors by determining the quality of the obtained measurements in light of the current measurement set as a whole, the anatomical features, and / or the patient's prior measurements, and known health risks. The exemplary system implements a deep learning method to generate a probability that each measurement obtained from the ultrasound data belongs to a unique set, thereby providing a confidence metric that can be displayed to the user. In some embodiments, the neural network can be trained using expert data that explains the extensive relationship between anatomical measurements and natural population variability. The results can be used to guide the user (e.g., ultrasound physician) to redo a specific measurement.

[0004] According to some examples of the present disclosure, an ultrasound imaging system may include an ultrasound transducer configured to acquire echo signals in response to ultrasound pulses transmitted toward a target region. The system may also include a graphical user interface configured to display a biometric tool widget for acquiring measurements of anatomical features within the target region from at least one image frame generated from the ultrasound echoes. The system may also include one or more processors in communication with the ultrasound transducer and configured to: determine a confidence metric indicating the accuracy of the measurement; and cause the graphical user interface to display a graphical indicator corresponding to the confidence metric.

[0005] In some examples, the processor is configured to determine the confidence metric by inputting the at least one image frame into a first neural network trained using imaging data including an anatomical feature. In some embodiments, the processor is further configured to determine the confidence metric by inputting patient demographics, a priori measurements of the anatomical feature, derived measurements based on the prior measurements, a probability that an image frame contains an anatomical landmark associated with the anatomical feature, a quality level of the image frame, ultrasound transducer settings, or a combination thereof into the first neural network. In some examples, the probability that the image frame contains an anatomical landmark indicates whether a correct imaging plane has been obtained for measuring the anatomical feature. In some embodiments, an indication of the probability that the image frame contains an anatomical landmark is displayed on the graphical user interface. In some examples, the derived measurements include age-adjusted risk of gestational age or a chromosomal abnormality. In some embodiments, the patient demographics include maternal age, patient weight, patient height, or a combination thereof. In some examples, the quality level of the image frame is based on a distance of an anatomical feature from the ultrasound transducer, an orientation of a biometric tool widget relative to the ultrasound transducer, a distance of a beam focus region to the anatomical feature, a noise estimate obtained via frequency analysis, or a combination thereof. In some examples, the graphical user interface is not physically coupled to the ultrasound transducer.

[0006] In some embodiments, the processor is further configured to: apply a threshold to the confidence metric to determine whether the measurement should be reacquired; and cause the graphical user interface to display an indication of whether the measurement should be reacquired. In some examples, the biometric tool widget includes: a caliper, a trace tool, an ellipse tool, a curve tool, an area tool, a volume tool, or a combination thereof. In some examples, the anatomical feature is a feature associated with a fetus or a uterus. In some embodiments, the processor is further configured to determine a gestational age and / or weight estimate based on the measurement. In some examples, the first neural network includes: a multilayer perceptron network configured to perform supervised learning with stochastic dropout; or an autoencoder network configured to generate a compressed representation of the image frame and the measurement, and compare the compressed representation to a manifold of population-based data.

[0007] According to some examples of the present disclosure, a method of ultrasound imaging may include: acquiring, by a transducer operably coupled to an ultrasound system, an echo signal in response to an ultrasound pulse transmitted into a target region. The method may also include: displaying a biometric tool widget for acquiring a measurement of an anatomical feature within the target region from at least one image frame generated from the ultrasound echoes. The method may also include: determining a confidence metric indicating the accuracy of the measurement; and causing the graphical user interface to display a graphical indicator corresponding to the confidence metric.

[0008] In some examples, determining the confidence metric includes inputting the at least one image frame into a first neural network trained using imaging data including the anatomical feature. In some embodiments, the method may further include inputting patient demographics, prior measurements of the anatomical feature, derived measurements based on the prior measurements, a probability that the image frame contains an anatomical landmark associated with the anatomical feature, a quality level of the image frame, ultrasound transducer settings, or a combination thereof into the first neural network. In some embodiments, the patient demographics include maternal age, patient weight, patient height, or a combination thereof. In some examples, the derived measurements include gestational age or a gestational age-adjusted risk of a chromosomal abnormality. In some embodiments, the method may further include determining a gestational age and / or weight estimate based on the measurements.

[0009] Any of the methods described herein, or steps thereof, may be embodied in a non-transitory computer-readable medium comprising executable instructions that, when executed, cause a processor of a medical imaging system to perform the method or steps embodied herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a block diagram of an ultrasound system according to the principles of the present disclosure.

[0011] Figure 2 is a block diagram of an operational arrangement of system components implemented according to the principles of the present disclosure.

[0012] Figure 3 is a diagram of an autoencoder network implemented according to the principles of the present disclosure.

[0013] Figure 4 It shows Figure 1 A diagram of additional components of an ultrasound system.

[0014] Figure 5 is a flow chart of a method of ultrasound imaging performed according to the principles of the present disclosure. DETAILED DESCRIPTION

[0015] The following descriptions of specific embodiments are merely exemplary in nature and are in no way intended to limit the invention or its application or use. In the following detailed description of the embodiments of the present system and method, reference is made to the accompanying drawings that form a part thereof, and these drawings are shown by way of illustrative specific embodiments in which the described systems and methods can be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the currently disclosed systems and methods, and it should be understood that other embodiments may be utilized and that structural and logical changes may be made without departing from the spirit and scope of the present system. In addition, for the purpose of clarity, detailed descriptions of specific features will not be discussed when they are obvious to those skilled in the art, so as not to obscure the description of the present system. Therefore, the following detailed description should not be regarded as having a limiting meaning, and the scope of the present system is limited only by the appended claims.

[0016] Fetal size and growth trajectory are important indicators of fetal health. For example, fetal growth disorders are often identified based on the discrepancy between actual and expected biometric measurements for a given gestational age. Due in part to frequent human error, such discrepancies are often attributed to inaccurate anatomical measurements obtained via ultrasound imaging, resulting in false-positive results. Similarly, erroneous measurements can fail to detect true differences, resulting in false-negative results. Therefore, accurate anatomical assessment of additional anatomical features of the fetus or patient, determining which measurements are accurate and which are inaccurate, is crucial. The systems and methods herein can improve ultrasound image acquisition and assessment techniques configured to measure various anatomical features by distinguishing between accurate and inaccurate anatomical measurements and / or by reducing or eliminating the acquisition of inaccurate measurements. The systems herein can be configured to quantify the accuracy of specific measurements by determining a confidence level for the measurements. Specific implementations involve acquiring ultrasound images of the fetus and obtaining various anatomical measurements therefrom. Based on the obtained measurements, one or more derived measurements, such as gestational age, fetal weight, and / or the presence of anatomical abnormalities, can be determined. By determining a confidence level associated with the obtained measurements, the systems herein can reduce misinterpretation of fetal images, thereby reducing the likelihood of false positives and false negatives regarding abnormality detection and, for example, improving the accuracy of population-based growth comparisons. Specific implementations can be configured to increase the confidence level associated with obtained anatomical measurements by identifying the correct imaging plane required to obtain each measurement. Image quality can also be improved through enhanced acoustic coupling and automatic selection of the optimal image settings required to obtain a particular image. Such improvements can be particularly significant when examining difficult-to-image patients (e.g., obese patients).

[0017] Ultrasound systems according to the present disclosure can utilize neural networks, such as deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), autoencoder neural networks, etc., to determine the quality of measurements of anatomical features. In some examples, a neural network can also be employed to determine the quality of an ultrasound image from which the measurements were initially obtained. Image quality can encompass whether the image is visually easy or difficult to interpret, or whether the image contains specific necessary landmark features necessary to obtain accurate and consistent measurements of a particular target feature. In various examples, any of a variety of currently known or later developed learning techniques can be used to train (one or more) neural networks to obtain a neural network (e.g., a trained algorithm or a hardware-based node system) that is configured to analyze input data in the form of ultrasound image frames, measurements, and / or statistical data and determine the quality of the measurements, which can be reflected in a confidence level output by the network for each measurement.

[0018] An ultrasound system according to the principles of the present invention may include or be operably coupled to an ultrasound transducer configured to transmit ultrasound pulses into a medium (e.g., a human body or a specific portion thereof) and generate echo signals in response to the ultrasound pulses. The ultrasound system may include: a beamformer configured to perform transmit and / or receive beamforming, and a display configured to, in some examples, display an ultrasound image generated by an ultrasound imaging system. The ultrasound imaging system may include one or more processors and at least one model of a neural network, which may be implemented in hardware and / or software components. The neural network can be trained to assess the accuracy of anatomical measurements obtained via a biometric tool. In some examples, one or additional neural networks can be trained to assess the quality and content adequacy of images used to obtain the measurements. The neural network can be communicatively coupled or integrated into a multi-layer network.

[0019] Neural networks implemented according to the present disclosure can be hardware-based (e.g., neurons are represented by physical components) or software-based (e.g., neurons and pathways are implemented in a software application) and can utilize a variety of topologies and learning algorithms for training the neural network to produce a desired output. For example, a software-based neural network can be implemented using a processor configured to execute instructions (e.g., a single-core or multi-core CPU, a single GPU or GPU cluster, or multiple processors arranged for parallel processing), the instructions being stored in a computer-readable medium, and the instructions, when executed, causing the processor to execute a trained algorithm for assessing image and / or measurement quality. The ultrasound system can include a display or graphics processor operable to arrange an ultrasound image (2D, 3D, 4D, etc.) and / or additional graphical information (which may include annotations, confidence metrics, user instructions, tissue information, patient information, indicators, color coding, highlights, and other graphical components) in a display window for display on a user interface of the ultrasound system. In some embodiments, the ultrasound images and associated measurements can be provided to a storage device and / or memory device, such as a picture archiving and communication system (PACS), for post-examination review, reporting purposes, or future training (e.g., to continue to enhance the performance of the neural network), particularly for generating images with measurements associated with a high confidence level. The display can be remotely located and can be interacted with by users other than the ultrasound physician performing real-time imaging or asynchronous imaging. In some examples, the ultrasound images and / or associated measurements obtained during the scan may not be displayed to the user operating the ultrasound system, but may instead be analyzed by the system for potential anatomical abnormalities or measurement errors when performing the ultrasound scan. Depending on such an implementation, the images and / or measurements can be extracted in a generated report for review by a user, such as a ultrasound physician, obstetrician, or clinician.

[0020] Figure 1An exemplary ultrasound system according to the principles of the present disclosure is shown. The ultrasound system 100 may include an ultrasound data acquisition unit 110. The ultrasound data acquisition unit 110 may include an ultrasound probe including an ultrasound sensor array 112 configured to transmit ultrasound pulses 114 to a region 116 of a subject, such as the abdomen, and to receive ultrasound echoes 118 in response to the transmitted pulses. As shown, the region 116 may include a developing fetus, or various other anatomical objects, such as the heart or lungs. As further shown, the ultrasound data acquisition unit 110 may include a beamformer 120 and a signal processor 122, which may be configured to generate a stream of discrete ultrasound image frames 124 based on the ultrasound echoes 118 received at the array 112. In addition, one or more image biometric tool widgets 123 (e.g., a caliper, a tracking tool, and / or an ellipse tool, a curve tool, an area tool, a volume tool, etc.) may be configured to obtain one or more measurements 125 of anatomical features visible within the image frames 124. Tool widget measurements may be manual (requiring user input) or automatic. The image frame 124 and / or associated measurements 125 can be transmitted to a data processor 126, such as a computing module or circuit, which is configured to determine the accuracy of the measurements. In some examples, the data processor 126 can be configured to determine the accuracy of the measurements by implementing at least one neural network (such as neural network 128), which can be trained to estimate the accuracy of the measurements obtained from the ultrasound image. The data processor 126 can also be configured to implement an image classification network 144 and / or an image quality network 148, and in some embodiments, the outputs of these networks can be input into the neural network 128 to improve the accuracy of the network 128. In various examples, the data processor 126 can also be communicatively or otherwise coupled to a database 127, which is configured to store various data types, including training data and newly acquired patient-specific data.

[0021] The ultrasound data acquisition unit 110 can be configured to acquire ultrasound data from one or more regions of interest 116, which can include a fetus, a uterus, and features thereof. The ultrasound sensor array 112 can include at least one transducer array configured to transmit and receive ultrasound energy. The configuration of the ultrasound sensor array 112 can be preset for performing prenatal scans of the fetus and, in embodiments, can be adjustable during a particular scan. Various transducer arrays can be used, such as linear arrays, convex arrays, or phased arrays. In different examples, the number and arrangement of transducer elements included in the transducer array 112 can vary. For example, the ultrasound sensor array 112 can include a 1D array or a 2D array of transducer elements, corresponding to a linear array probe and a matrix array probe, respectively. The 2D matrix array can be configured to electronically scan in elevation and azimuth dimensions (via phased array beamforming) for 2D or 3D imaging. In addition to B-mode imaging, imaging modalities implemented according to the disclosure herein can also include, for example, shear wave and / or Doppler. Various users can process and operate the ultrasound data acquisition unit 110 to perform the methods described herein. In some examples, the user may be an inexperienced novice ultrasound operator who is unable to accurately identify every anatomical feature of the fetus required in a given scan. In some cases, the data acquisition unit 110 is controlled (positioned, set up, etc.) by a robot and can perform the methods described herein in place of a human operator. For example, the data acquisition unit 110 can be configured to utilize findings obtained by the data processor 126 to refine one or more image planes and / or anatomical measurements obtained therefrom. According to such an example, the data acquisition unit 110 can be configured to operate in an automated manner by adjusting one or more parameters of a transducer, signal processor, or beamformer in response to feedback received from the data processor.

[0022] The data acquisition unit 110 may also include a beamformer 120, for example, including a microbeamformer or a combination of a microbeamformer and a main beamformer, which is coupled to the ultrasound sensor array 112. The beamformer 120 can control the transmission of ultrasound energy, for example, by forming ultrasound pulses into focused beams. The beamformer 120 can also be configured to control the reception of ultrasound signals so that discernible image data is generated and processed with the aid of other system components. The role of the beamformer 120 can vary in different types of ultrasound probes. In some embodiments, the beamformer 120 may include two separate beamformers: a transmit beamformer configured to receive and process a pulse sequence of ultrasound energy for transmission into a subject; and a separate receive beamformer configured to amplify, delay, and / or sum received ultrasound echo signals. In some embodiments, the beamformer 120 may include a microbeamformer operating on groups of transducer elements for interferometric transmit and receive beamforming, coupled to a main beamformer operating on group inputs and outputs for transmit and receive beamforming, respectively.

[0023] The signal processor 122 may be communicatively, operatively, and / or physically coupled to the sensor array 112 and / or the beamformer 120. Figure 1 In the example shown in , the signal processor 122 is included as an integral component of the data acquisition unit 110, but in other examples, the signal processor 122 can be a separate component. In some examples, the signal processor can be housed in a housing along with the sensor array 112, or it can be physically separate from the sensor array 112 but communicatively coupled (e.g., via a wired or wireless connection). The signal processor 122 can be configured to receive unfiltered and unresolved ultrasound data representing ultrasound echoes 118 received at the sensor array 112. As the user scans the fetal region 116, the signal processor 122 can continuously generate a plurality of ultrasound image frames 124 based on the data.

[0024] In certain embodiments, the neural network 128 may comprise a deep learning network that is trained via imaging data to generate a probability that each measurement 125 obtained via the biometric tool widget 123 belongs to a unique set of measurements. An associated confidence level based on or equal to the probability is then generated for each measurement, thereby providing the user with a real-time assessment of the accuracy of the measurement and, in some examples, indicating whether one or more measurements should be reacquired. As described below with respect to Figure 2 As explained, neural network 128 can process a number of different inputs, including outputs generated by image classification network 144 and image quality network 148 .

[0025] Figure 2 An exemplary operational arrangement of components implemented according to system 100 is shown, including various inputs and outputs that can be received and generated, respectively, by data processor 126. As shown, data processor 126 can be configured to implement a neural network 128, which can be configured to receive one or more inputs 130. Inputs 130 can vary. For example, inputs 130 can include current fetal measurements and corresponding ultrasound images 130a obtained substantially in real time during an ultrasound examination. Fetal measurements obtained via system 100 can include, but are not limited to, crown-to-rump length, head circumference, biparietal diameter, gestational sac diameter, occipito-frontal diameter, femur length, humeral length, abdominal circumference, interocular distance, and / or interocular distance; as well as functional imaging such as heart rate, cardiac volume, and / or fetal movement. In some examples, additional measurements of other anatomical features can also be obtained, such as cross-sectional diameters of organs such as the heart or lungs, as well as additional parameters such as the angle between two measurements. Inputs can also include various maternal demographics 130b, including maternal weight, height, age, race, etc. A priori fetal measurements 130c for a given fetus can also be input. The inputs 130 can also include one or more derived measurements, such as a gestational age estimate 130d, which is based on direct fetal measurements, such as femur length. In some embodiments, the derived measurements can include ultrasound markers that indicate an increased risk of potential fetal aneuploidy or non-chromosomal abnormalities adjusted for gestational age. A neural network 128 can receive one or more of the above-mentioned inputs 130, and the neural network 128 is configured to analyze the inputs 130 and generate one or more outputs 132 based on the inputs. Such outputs 132 can include one or more fetal measurements and an associated confidence level 132a for each measurement, a gestational age estimate and an associated confidence level 132b, and a fetal weight estimate 132c.

[0026] The data processor 126 can apply a confidence threshold 134 to each output 132 of the neural network 128 to determine whether the quality of a given measurement result is satisfactory or whether re-measurement is required. The threshold can be tuned based on experience or can be set directly by a user or reviewer. According to some embodiments, the threshold results can be communicated in the form of one or more notifications 140. For example, the data processor 126 can be configured to generate a "take again" notification 136 for measurement results that do not meet the threshold, and an "all is well" notification 138 for measurement results that do meet the threshold 134. The specific manner in which the notification is communicated can vary. For example, the notification can include displayed text, as described below with reference to Figure 4The data processor 126 can be configured to generate a report 142 that may include all or selected measurements and associated confidence levels determined by the neural network 128. An exemplary report 142 may include outputs 132a, 132b, and / or 132c obtained during a given scan and any notifications associated therewith.

[0027] In some examples, system 100 can be configured to implement a second neural network to further improve the accuracy of image acquisition and assessment by evaluating the ultrasound probe position relative to the target anatomy. This can be performed before implementing neural network 128. Specifically, image classification network 144, which may include a CNN, can be trained to determine whether a given ultrasound image contains the necessary anatomical landmarks for obtaining a particular measurement. For example, if a transthalamic view is not obtained with the ultrasound probe, biparietal diameter and head circumference measurements may be erroneous. In a transthalamic view, both the thalamus and the diaphragm should be visible. Similarly, abdominal circumference measurements may be erroneous if the stomach, umbilical vein, and two ribs on each side of the abdomen are not visible. Therefore, when seeking biparietal diameter and head circumference measurements, image classification network 144 can be configured to determine whether the thalamus and diaphragm are included in the current image frame. Similarly, when seeking abdominal circumference, image classification network 144 can be configured to determine whether the stomach, umbilical vein, and two ribs on each side of the abdomen are included in the current image frame. By confirming the presence of one or more landmarks, image classification network 144 can confirm that the correct imaging plane has been obtained for the specified anatomical measurement, which allows biometric tool(s) 123 to measure the target features included in the image with a greater level of confidence. In some examples, in addition to or in lieu of image classification network 144, a segmentation processor can be implemented to perform automatic segmentation of target region 116.

[0028] The input 146 processed by the image classification network 144 can include a region within the image that is within a preselected circumference. By limiting the region to the preselected circumference, the total area over which the image classification network 144 searches for one or more anatomical landmarks is reduced, thereby reducing the amount of data required to train the network and further enhancing the processing efficiency of the system 100.

[0029] In various embodiments, the output 130e of the image classification network 144 can serve as another input source for processing by the neural network 128. The output 130e can include a numerical probability that a given image frame contains a landmark anatomical feature associated with a particular anatomical measurement, thereby providing an additional measure of confidence for assessing the quality of a particular image and the measurement obtained therefrom. For example, if the image classification network 144 is not implemented to filter the initial image frame, the likelihood that the final confidence level output(s) 132 are accurate can be reduced, as measurements that are consistent with the population-wide mean but obtained from a suboptimal image plane may be inaccurate. In some embodiments, the output 130e can be displayed for immediate assessment by a user performing an ultrasound scan, thereby enabling the user to discern whether the current probe position is sufficient to obtain a given measurement or whether the probe position, orientation, and / or settings need to be adjusted. For example, a suspect anatomical measurement and the corresponding image can be displayed to the user along with a notification 140 indicating that the image may or does lack one or more anatomical landmarks. By providing such instructions to the user prior to implementing neural network 128, futile processing steps may be avoided, thereby increasing the processing efficiency of system 100.

[0030] In additional or alternative embodiments, the system 100 can be configured to implement a third (or second) neural network to further improve the accuracy of image acquisition and assessment by evaluating the quality of ultrasound images obtained during a given scan. Specifically, the image quality network 148 can be trained to determine whether a given ultrasound image is of high, low, or medium quality. The input 150 received by the image quality network 148 can include an ultrasound image and / or image settings, such as frequency, gain, etc. The input 150 can also include, for example, an estimate of distortion that degrades image quality, the minimum, maximum, and average distances from the ultrasound transducer, the orientation of the measurement widget relative to the transducer, the distance from the measurement endpoint to the beam focus area, an estimated image resolution along the measurement axis, and / or a noise estimate obtained via frequency analysis in a region around the caliper-selected endpoint. The image quality network 148 can be trained using multiple images, each image associated with the aforementioned input 150 and labeled as having high, low, or medium quality.

[0031] In various examples, output 130f of image quality network 148 can be used as another input source for processing by neural network 128. Output 130f can include a numerical probability that a particular image used to obtain a measurement has the necessary quality. In some examples, output 130f can be generated and used substantially in real time during measurement acquisition to provide a user with an early indication of potential measurement errors. For example, upon selecting two measurement endpoints, a notification 140 of image quality, such as 50%, 75%, or 99%, can be generated and displayed, conveying the likelihood that a particular measurement can be accurately determined, such that a 10% image quality metric would convey a low likelihood that a measurement was accurately determined based on the image, while a 99% image quality metric would convey a high likelihood that a measurement was accurately determined based on the image. In some examples, image quality network 148 can be configured to allow a user to "go back" to input 150 to determine which particular input(s) contributed most to a particular image quality metric.

[0032] The neural network 128 can be configured to identify erroneous measurements or potentially erroneous measurements by implementing various supervised or unsupervised learning techniques specifically configured for the anatomical measurement applications described herein. Depending on the technology employed, the architecture of the neural network can also vary. For example, the neural network 128 can include a multilayer perceptron (MLP) network configured to perform supervised learning with random dropout. Although the specific architecture can vary, an MLP network typically includes an input layer, an output layer, and multiple hidden layers. Every neuron within a given layer (i) can be fully connected to every other neuron in the next layer (i+1), and neurons in one or more layers can be configured to implement a sigmoid or softmax activation function to classify each input. Instead of using the classification output of the MLP network, random dropout can be implemented to predict measurement uncertainty. Specifically, during processing, a predetermined percentage of randomly selected nodes within the MLP can be temporarily omitted or ignored. To predict the correct uncertainty for a given measurement obtained during an examination, multiple feed-forward iterations of the model can be run, and during each run, one or more nodes are randomly dropped from the network. The change in the predictions generated by the MLP for a single patient after multiple iterations can be used as an indicator of uncertainty. For example, a high change in the predictions obtained after multiple iterations can indicate high measurement uncertainty and, therefore, a greater likelihood of measurement error. Similarly, a low change after multiple iterations can indicate low measurement uncertainty. To train the MLP, medical expert annotations of various fetal images and / or measurements, as well as corresponding fetal outcomes, such as birth weight, normal birth, or abnormal birth, can be used.

[0033] In additional embodiments, unsupervised learning can be achieved via autoencoder-based phenotypic stratification and outlier identification. Specifically, autoencoder-based phenotypic stratification or restricted Boltzmann machines (RBMs) can be used to discover the underlying structure in the raw input data (embodied in input 130) without human input. Figure 3 An exemplary autoencoder-based operation is illustrated in FIG. As shown, the neural network 128 can include an autoencoder network that can be configured to receive a plurality (e.g., thousands or more) of sparse codes 152 representing the various inputs 130 described above. The autoencoder 128 learns to generate a compressed vector 154 of the sparse code, which can be compared to a population-based training data set constituting a manifold 156 of known data points to determine whether the combination of new measurement data is similar to the training data. If not, the data may be an outlier, which may indicate that a rare anomaly has been detected. The anomaly can represent a true anatomical difference or an incorrect measurement. In either case, a signal can be sent to the user to re-evaluate the outlier to confirm whether the data actually indicates an anatomical anomaly or whether the initial measurement result is simply inaccurate. In an example, the compressed vector 154 is processed via a clustering algorithm such as a t-distributed stochastic neighbor embedding (t-SNE). According to such an example, the distance of the new measurement data can be compared to the population-based distribution data represented in the manifold 156 to determine the degree to which the new data differs from the training data.

[0034] In some examples, after identifying a set of measurement results as ambiguous or uncertain, domain knowledge in the form of a rule-based graph can be applied to evaluate the results. The rule-based graph can be used to identify which measurement result(s) appear to be outliers. In some examples, incorrect measurements can be identified by iteratively excluding one of the measurement results from a particular data set analyzed by the neural network 128, and then selecting which measurement result contributes most to the measurement result uncertainty via the processor 126 or the user.

[0035] In various embodiments, the neural network 128, the image classification network 144, and / or the image quality network 148 may be implemented at least in part in a computer-readable medium comprising executable instructions executed by a processor (e.g., the data processor 126). To train the neural network 128, 144, and / or 148, a training set comprising a plurality of examples of input arrays and output classifications may be presented to a training algorithm(s) for the neural network(s) (e.g., an AlexNet training algorithm, as described in “ImageNet Classification with Deep Convolutional Neural Networks” by Krizhevsky, A., Sutskever, I., and Hinton GE, NIPS 2012 or its successors).

[0036] In some examples, the neural network training algorithm associated with the neural networks 128, 144, and / or 148 can be presented with tens of thousands or even millions of training data sets in order to train the neural network to determine a confidence level for each measurement obtained from a particular ultrasound image. In various examples, the number of ultrasound images used to train the neural network(s) can range from approximately 50,000 to 200,000 or more. If a greater number of different anatomical features are to be identified, or a greater variety of patient variations (e.g., weight, height, age, etc.) are to be accommodated, the number of images used to train the network(s) may be increased. The number of training images may vary for different anatomical features and may depend on the variability of the appearance of the particular feature. Training the network(s) to assess the quality of measurements associated with features that have a higher overall variability in the population, for example, may require a larger number of training images.

[0037] The results of the ultrasound scan, including the quality of the obtained measurements as reflected in one or more confidence level outputs 132, can be displayed to the user via one or more components of the system 100. Figure 4As shown in FIG, such components can include a display processor 158 communicatively coupled to the data processor 126. The display processor 158 is also coupled to a user interface 160, such that the display processor 158 can link the data processor 126 (and therefore the one or more neural networks operating thereon) to the user interface 160 so that the neural network output (e.g., measurements and confidence levels) can be displayed on the user interface. In an embodiment, the display processor 158 can be configured to generate an ultrasound image 162 based on the image frames 124 received at the data processor 126. In some examples, the user interface 160 can be configured to display the ultrasound image 162 in real time while the ultrasound scan is being performed, along with one or more notifications 140 that may be overlaid on the image. The notifications 140 can include the measurements and associated confidence levels in the form of annotations, color maps, percentages, bars, and auditory, voice, or tactile renderings, which can be organized in a report 142. Additionally, in some embodiments, an indication of whether a particular measurement meets a given threshold can be included in the notification 140, along with one or more instructions for directing the user to reacquire the particular measurement.

[0038] The user interface 160 can also be configured to receive user input 166 at any time before, during, or after the ultrasound scan. For example, the user interface 160 can be interactive, receiving user input 166 indicating confirmation that the anatomical feature has been accurately measured or that the measurement should be retaken. In some examples, the input 166 can include an instruction to increase or decrease the threshold 134 or adjust one or more image acquisition settings. As further shown, the user interface 160 can be configured to display a biometric tool widget 123 for obtaining measurements of the anatomical feature.

[0039] exist Figure 4 Together Figure 1 The configuration of the components shown in can vary. For example, the system 100 can be portable or fixed. Various portable devices (e.g., laptop computers, tablet computers, smart phones, remote displays and interfaces, etc.) can be used to implement one or more functions of the system 100. Some or all data processing can be performed remotely (e.g., in the cloud). In examples that incorporate such devices, the ultrasound sensor array 112 can be connectable via a USB interface, for example. In some examples, Figures 1 to 4 The various components shown in FIG1 may be combined. For example, neural network 128 may be combined with image classification network 144 and / or image quality network 148. According to such an embodiment, the outputs generated by networks 144 and / or 148 may still be input into neural network 128, but the three networks may constitute subcomponents of, for example, a larger hierarchical network.

[0040] Figure 5 is a flow chart of an ultrasound imaging method performed in accordance with the principles of the present disclosure. Exemplary method 500 illustrates steps that may be utilized by the systems and / or devices described herein, in any sequence, for determining the quality of one or more anatomical measurements during a fetal scan performed, for example, by a novice user and / or by a robotic ultrasound device following instructions generated by the system. Method 500 may be performed by an ultrasound imaging system such as system 100 or other systems including, for example, a mobile system such as LUMIFY manufactured by Koninklijke Philips NV (“Philips”). Additional exemplary systems may include SPARQ and / or EPIQ, also produced by Philips.

[0041] In the illustrated embodiment, the method 500 begins at block 502 by acquiring, by a transducer operatively coupled to an ultrasound system, an echo signal in response to an ultrasound pulse transmitted into a target region.

[0042] At block 504 , the method involves “displaying a biometric tool widget for obtaining measurements of anatomical features within a target region from at least one image frame generated from ultrasound echoes.”

[0043] At block 506, the method involves determining a confidence metric indicative of the accuracy of the measurement.

[0044] At block 508 , the method involves causing a graphical user interface to display a graphical indicator corresponding to the confidence metric.

[0045] In various embodiments in which programmable devices, such as computer-based systems or programmable logic, are used to implement components, systems, and / or methods, it should be appreciated that the systems and methods described above can be implemented using any of a variety of known or later developed programming languages, such as "C," "C++," "FORTRAN," "Pascal," "VHDL," and the like. Accordingly, various storage media, such as computer disks, optical disks, electronic memories, and the like, can be prepared that can contain information capable of directing a device, such as a computer, to implement the systems and / or methods described above. Once the information and programs contained on the storage media are accessible to an appropriate device, the storage media can provide the information and programs to the device, thereby enabling the device to perform the functions of the systems and / or methods described herein. For example, if a computer disk containing appropriate material, such as source files, object files, executable files, and the like, is provided to a computer, the computer can receive the information, configure itself appropriately, and perform the functions of the various systems and methods outlined in the figures and flow charts above to implement the various functions. That is, the computer may receive portions of the information related to the different elements of the systems and / or methods described above from the disk, implement the individual systems and / or methods, and coordinate the functionality of the individual systems and / or methods described above.

[0046] In view of the present disclosure, it should be noted that the various methods and devices described herein can be implemented in hardware, software, and firmware. In addition, the various methods and parameters are included only by way of example and without any limiting meaning. In view of the present disclosure, one of ordinary skill in the art can implement the present teachings while determining his or her own techniques and the required equipment that affects these techniques, while still remaining within the scope of the present invention. The functions of one or more of the processors described herein may be combined into a smaller number or a single processing unit (e.g., a CPU) and may be implemented using an application specific integrated circuit (ASIC) or a general-purpose processing circuit that is programmed in response to executable instructions to perform the functions described herein.

[0047] Although the present system may have been described with particular reference to ultrasound imaging systems, it is also contemplated that the present system can be extended to other medical imaging systems that obtain one or more images in a systematic manner. Thus, the present system can be used to obtain and / or record image information about, but not limited to, the kidneys, testicles, breasts, ovaries, uterus, thyroid, liver, lungs, musculoskeletal, spleen, heart, arterial and vascular systems, as well as other imaging applications related to ultrasound-guided interventions. In addition, the present system may also include one or more programs that can be used with conventional imaging systems so that they can provide the features and advantages of the present system. Certain additional advantages and features of the present disclosure will be apparent to those skilled in the art upon studying the present disclosure, or may be experienced by those employing the novel systems and methods of the present disclosure. Another advantage of the present system and method may be that conventional medical imaging systems can be easily upgraded to incorporate the features and advantages of the present systems, devices and methods.

[0048] Of course, it should be understood that any of the examples, embodiments, or processes described herein may be combined with one or more other examples, embodiments, and / or processes, or may be separated and / or performed in separate devices or device portions according to the present systems, devices, and methods.

[0049] Finally, the above discussion is intended to be merely illustrative of the present system and should not be construed as limiting the appended claims to any particular embodiment or group of embodiments. Thus, while the present system has been described in detail with reference to exemplary embodiments, it should be understood that numerous modifications and alternative embodiments may be devised by those skilled in the art without departing from the broader and intended spirit and scope of the present system as set forth in the claims that follow. The specification and drawings should, therefore, be viewed in an illustrative manner and are not intended to limit the scope of the appended claims.

Claims

1. An ultrasound imaging system, comprising: an ultrasound transducer configured to acquire an echo signal in response to an ultrasound pulse sent toward a target region of the patient; a graphical user interface configured to display a biometric tool widget for obtaining measurement results of anatomical features within the target region from at least one image frame generated based on the echo signal; as well as one or more processors in communication with the ultrasound transducer and configured to: determining a confidence metric indicative of accuracy of the measurement of the anatomical feature obtained by the biometric tool widget using a first neural network by inputting the measurement of the anatomical feature obtained by the biometric tool widget and the at least one image frame into the first neural network, wherein the first neural network is trained using imaging data including the anatomical feature; and The graphical user interface is caused to display a graphical indicator corresponding to the confidence metric.

2. The ultrasound imaging system according to claim 1, wherein: The processor is further configured to determine the confidence metric by inputting into the first neural network patient statistics, a priori measurements of the anatomical feature, derived measurements based on the a priori measurements of the anatomical feature, a probability that the image frame contains an anatomical landmark associated with the anatomical feature, a quality level of the image frame, a setting of the ultrasound transducer, or a combination thereof.

3. The ultrasound imaging system according to claim 2, wherein: The probability that the image frame contains the anatomical landmark indicates whether a correct imaging plane has been obtained for measuring the anatomical feature.

4. The ultrasound imaging system according to claim 1, wherein: The graphical user interface is not physically coupled to the ultrasound transducer.

5. The ultrasound imaging system according to claim 2, wherein: The anatomical feature is a feature associated with the fetus or uterus, and the derived measure comprises gestational age or a gestational age-adjusted risk of a chromosomal abnormality.

6. The ultrasound imaging system according to claim 2, wherein: The patient statistical data includes: maternal age, patient weight, patient height or a combination thereof.

7. The ultrasound imaging system according to claim 2, wherein: The quality level of the image frame is based on a distance of the anatomical feature from the ultrasound transducer, an orientation of the biometric tool widget relative to the ultrasound transducer, a distance of a beam focus area to the anatomical feature, a noise estimate obtained via frequency analysis, or a combination thereof.

8. The ultrasound imaging system according to claim 1, wherein: The processor is further configured to: applying a threshold to the confidence metric to determine whether measurements of the anatomical feature should be reacquired; and The graphical user interface is caused to display an indication of whether measurements of the anatomical feature should be reacquired.

9. The ultrasound imaging system according to claim 1, wherein: The biometric tool widget includes: a caliper, a tracking tool, an ellipse tool, a curve tool, an area tool, a volume tool, or a combination thereof.

10. The ultrasound imaging system according to claim 1, wherein: The anatomical feature is a feature associated with the fetus or uterus.

11. The ultrasound imaging system according to claim 10, wherein: The processor is further configured to determine a gestational age and / or weight estimate based on the measurements of the anatomical features.

12. The ultrasound imaging system according to claim 1, wherein: The first neural network comprises: a multilayer perceptron network configured to perform supervised learning with random dropout; or an autoencoder network configured to generate a compressed representation of the image frames and the measurements of the anatomical features, and compare the compressed representation to a manifold of population-based data.

13. A method of ultrasound imaging, comprising: acquiring, by a transducer operatively coupled to the ultrasound system, echo signals in response to ultrasound pulses transmitted into a target region of the patient; displaying a biometric tool widget, the biometric tool widget being configured to obtain measurement results of anatomical features within the target region from at least one image frame generated based on the echo signal; determining a confidence metric indicative of accuracy of the measurement of the anatomical feature obtained by the biometric tool widget using a first neural network by inputting the measurement of the anatomical feature obtained by the biometric tool widget and the at least one image frame into the first neural network, wherein the first neural network is trained using imaging data including the anatomical feature; and A graphical user interface is caused to display a graphical indicator corresponding to the confidence metric.

14. The method according to claim 13, further comprising: Patient statistics, a priori measurements of the anatomical feature, derived measurements based on the a priori measurements of the anatomical feature, a probability that the image frame contains an anatomical landmark associated with the anatomical feature, a quality level of the image frame, settings of the transducer, or a combination thereof are input into the first neural network.

15. The method according to claim 14, wherein The patient statistical data includes: maternal age, patient weight, patient height or a combination thereof.

16. The method according to claim 14, wherein The anatomical feature is a feature associated with the fetus or uterus, and the derived measure comprises gestational age or a gestational age-adjusted risk of a chromosomal abnormality.

17. The method according to claim 13, wherein: The anatomical feature is a feature associated with a fetus or uterus, and the method further comprises determining a gestational age and / or weight estimate based on measurements of the anatomical feature.

18. A non-transitory computer readable medium comprising executable instructions which, when executed, cause a processor of a medical imaging system to perform the method according to any one of claims 13-17.

Images