System and method for detecting image quality degradation of ultrasound imaging system

Abstract

A method for detecting degradation of an ultrasound imaging system (140) includes receiving ultrasound images of a heart (S311); automatically determining a cardiac view of the heart by applying at least one ultrasound image to a trained machine learning algorithm (S312); automatically selecting an image quality (IQ) model from multiple IQ models based on the determined cardiac view of the heart (S313); determining IQ metrics associated with each ultrasound image by applying each ultrasound image to the selected IQ model (S314); aggregating the IQ metrics to provide an aggregated IQ score associated with each ultrasound image to create a time series of aggregated IQ scores (S315); estimating a level of degradation of the ultrasound imaging system based on the time series of the aggregated IQ scores associated with the ultrasound images over time using a degradation detection machine learning algorithm (S316); and notifying a user of the estimated level of degradation (S317).

Description

SYSTEM AND METHOD FOR DETECTING IMAGE QUALITY DEGRADATION OF ULTRASOUND IMAGING SYSTEMBACKGROUND

[0001] The quality of images provided by ultrasound imaging systems (image quality (IQ)) degrades over time. This is particularly true with regard to ultrasound transducer probes of the ultrasound imaging systems. An acute equipment problem or failure typically causes a sudden drop in IQ, which is immediately discernable by the user, resulting in the need for repairs and possibly extended down time for the ultrasound imaging system.

[0002] However, potentially detectable degradation of IQ often starts long before equipment failure and / or sudden drops in IQ. It would be beneficial for degradation in image quality of an ultrasound imaging system to be detected before an acute equipment problem or failure, and for the user of the ultrasound imaging system to be timely notified of the same, so that the equipment involved may be exchanged and / or repaired proactively, without interrupting current exams or scheduling. This improves overall quality of ultrasound images within the technical field of medical imaging.SUMMARY

[0003] In a representative embodiment, a method is provided for detecting degradation of an ultrasound imaging system. The method includes receiving multiple ultrasound images of a heart of a subject acquired by the ultrasound imaging system; automatically determining a cardiac view of the heart from at least one ultrasound image of the multiple ultrasound images by applying the at least one ultrasound image to a trained first machine learning algorithm, where the cardiac view of the heart is one of multiple predetermined cardiac views of the heart; automatically selecting an image quality (IQ) model from multiple IQ models based on the determined cardiac view of the heart, wherein the multiple IQ models correspond to the plurality of predetermined cardiac views of the heart, respectively, and where the selected IQ model comprises a trained second machine learning algorithm; determining multiple IQ metrics associated with each ultrasound image of the multiple ultrasound images by applying eachultrasound image to the selected IQ model; aggregating the multiple IQ metrics to provide an aggregated IQ score associated with each ultrasound image of the multiple ultrasound images; predicting a time series trend of image quality based on a time series of aggregated IQ scores, including the aggregated IQ scores associated with the multiple ultrasound images over time; estimating a level of degradation of the ultrasound imaging system by analyzing the time series trend of the aggregated IQ scores associated with the multiple ultrasound images over time with a degradation detection machine learning algorithm; and notifying a user when the estimated level of degradation based on a predetermined degradation threshold.

[0004] In another representative embodiment, a system is provided for detecting degradation of an ultrasound imaging system. The system includes a display; a processor; and a non-transitory memory that stores instructions. When executed by the processor, the instructions cause the processor receive multiple ultrasound images of a heart of a subject acquired by the ultrasound imaging system; automatically determine a cardiac view of the heart from at least one ultrasound image of the multiple ultrasound images by applying the at least one ultrasound image to a trained cardiac view machine learning algorithm, where the cardiac view of the heart is one of multiple predetermined cardiac views of the heart; automatically select an IQ model from multiple IQ models based on the determined cardiac view of the heart, where the multiple IQ models correspond to the multiple predetermined cardiac views of the heart, respectively, and where the selected IQ model includes a trained IQ machine learning algorithm; determine multiple IQ metrics associated with each ultrasound image of the multiple ultrasound images by applying each ultrasound image to the selected IQ model; aggregate the multiple IQ metrics to provide an aggregated IQ score associated with each ultrasound image of the multiple ultrasound images to create a time series of aggregated IQ scores; estimate a level of degradation of the ultrasound imaging system based on the time series of the aggregated IQ scores associated with the multiple ultrasound images over time using a degradation detection machine learning algorithm; and cause a notification regarding the estimated level of degradation to be displayed on the display.

[0005] In another representative embodiment, a non-transitory computer readable medium stores instructions for detecting degradation of an ultrasound imaging system. When executed by a processor, the instructions cause the processor to receive multiple ultrasound images of a heart ofa subject acquired by the ultrasound imaging system; automatically determine a cardiac view of the heart from at least one ultrasound image of the multiple ultrasound images by applying the at least one ultrasound image to a trained cardiac view machine learning algorithm, where the cardiac view of the heart is one of multiple predetermined cardiac views of the heart; automatically select an IQ model from multiple IQ models based on the determined cardiac view of the heart, where the multiple IQ models correspond to the multiple predetermined cardiac views of the heart, respectively, and where the selected IQ model includes a trained IQ machine learning algorithm; determine multiple IQ metrics associated with each ultrasound image of the multiple ultrasound images by applying each ultrasound image to the selected IQ model; aggregate the multiple IQ metrics to provide an aggregated IQ score associated with each ultrasound image of the multiple ultrasound images to create a time series of aggregated IQ scores; estimate a level of degradation of the ultrasound imaging system based on the time series of the aggregated IQ scores associated with the multiple ultrasound images over time using a degradation detection machine learning algorithm; and cause a notification regarding the estimated level of degradation to be displayed to a user on the display.BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The example embodiments are best understood from the following detailed description when read with the accompanying drawing figures. It is emphasized that the various features are not necessarily drawn to scale. In fact, the dimensions may be arbitrarily increased or decreased for clarity of discussion. Wherever applicable and practical, like reference numerals refer to like elements.

[0007] FIG. 1 is a simplified block diagram of a system for detecting degradation of an ultrasound imaging system, according to a representative embodiment.

[0008] FIG. 2A is a chart showing a predicted time series trend of an ultrasound imaging system filtered by body mass index (BMI) of the patients, according to a representative embodiment.

[0009] FIG. 2B is a chart showing a predicted time series trend of an ultrasound imaging system filtered by transducer probe, according to a representative embodiment.

[0010] FIG. 3 is a flow diagram showing a method of detecting degradation of an ultrasound imaging system, according to a representative embodiment.DETAILED DESCRIPTION

[0011] In the following detailed description, for the purposes of explanation and not limitation, representative embodiments disclosing specific details are set forth in order to provide a thorough understanding of an embodiment according to the present teachings. Descriptions of known systems, devices, materials, methods of operation and methods of manufacture may be omitted so as to avoid obscuring the description of the representative embodiments. Nonetheless, systems, devices, materials and methods that are within the purview of one of ordinary skill in the art are within the scope of the present teachings and may be used in accordance with the representative embodiments. It is to be understood that the terminology used herein is for purposes of describing particular embodiments only and is not intended to be limiting. The defined terms are in addition to the technical and scientific meanings of the defined terms as commonly understood and accepted in the technical field of the present teachings.

[0012] It will be understood that, although the terms first, second, third, etc. may be used herein to describe various elements or components, these elements or components should not be limited by these terms. These terms are only used to distinguish one element or component from another element or component. Thus, a first element or component discussed below could be termed a second element or component without departing from the teachings of the inventive concept.

[0013] The terminology used herein is for purposes of describing particular embodiments only and is not intended to be limiting. As used in the specification and appended claims, the singular forms of terms “a,” “an” and “the” are intended to include both singular and plural forms, unless the context clearly dictates otherwise. Additionally, the terms “comprises,” “comprising,” and / or similar terms specify the presence of stated features, elements, and / or components, but do not preclude the presence or addition of one or more other features, elements, components, and / or groups thereof. As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items.

[0014] Unless otherwise noted, when an element or component is said to be “connected to,” “coupled to,” or “adjacent to” another element or component, it will be understood that the element or component can be directly connected or coupled to the other element or component, or intervening elements or components may be present. That is, these and similar termsencompass cases where one or more intermediate elements or components may be employed to connect two elements or components. However, when an element or component is said to be “directly connected” to another element or component, this encompasses only cases where the two elements or components are connected to each other without any intermediate or intervening elements or components.

[0015] The present disclosure, through one or more of its various aspects, embodiments and / or specific features or sub-components, is thus intended to bring out one or more of the advantages as specifically noted below. For purposes of explanation and not limitation, example embodiments disclosing specific details are set forth in order to provide a thorough understanding of an embodiment according to the present teachings. However, other embodiments consistent with the present disclosure that depart from specific details disclosed herein remain within the scope of the appended claims. Moreover, descriptions of well-known apparatuses and methods may be omitted so as to not obscure the description of the example embodiments. Such methods and apparatuses are within the scope of the present disclosure.

[0016] Generally, the various embodiments described herein provide a system and method for detecting degradation of an ultrasound imaging system. The embodiments provide for a processing pipeline that derives image quality (IQ) scores from ultrasound images acquired by the ultrasound imaging system being monitored. The pipeline includes a cardiac view model that identifies the cardiac view of the acquired ultrasound images, and selects an IQ model that is specific to that cardiac view. The selected IQ model determines IQ metrics of each ultrasound image, and the output of the IQ model is aggregated into an IQ metric per ultrasound image. The aggregated IQ metrics are continuously monitored and projected over time by a degradation detection model to enable detection of early signs of IQ degradation.

[0017] The detected IQ degradation may be tracked back to various features of operating the ultrasound imaging system, such as age, the number of times used, the total length of time of use, and the like. In various embodiments, IQ degradation may be tracked back to features not associated with the age or usage of the ultrasound imaging system. For example, the IQ metrics may be grouped and filtered by different criteria, such as transducer probe in use, system settings, the sonographer who acquired the images, or body mass index (BMI) of the patient. One or more machine learning algorithms may support the IQ analysis over time by providingestimates of whether the IQ degradation is occurring for a certain filter option.

[0018] FIG. 1 is a simplified block diagram of a system for detecting degradation of an ultrasound imaging system, according to a representative embodiment.

[0019] Referring to FIG. 1, system 100 includes a workstation 105 for implementing and / or managing the processes described herein with regard to detecting degradation of ultrasound images from an ultrasound imaging system 140. The workstation 105 includes one or more processors indicated by processor 120, one or more memories indicated by memory 130, a user interface 122 and a display 124. The processor 120 communicates with the ultrasound imaging system 140 through an imaging interface (not shown). The ultrasound imaging system 140 includes an ultrasound controller (base system) 143 and a transducer probe 145 operable by an operator to obtain ultrasound images of a portion of a subject (patient) 150. The transducer probe 145 may be manipulated manually by the operator (e.g., sonographer, physician), automatically by a robot under control of a robot controller (not shown), or a combination of both.

[0020] The transducer probe 145 may include a 2D matrix array of transducer elements, capable of scanning in two or three dimensions, for example, for emitting ultrasound waves into the body of the subject 150 and receiving echo signals in response. The transducer elements may include capacitive micromachined ultrasonic transducers (CMUTs) or piezoelectric transducers formed of materials such as lead zirconate titanate (PZT) or polyvinylidene difluoride (PVDF), for example, although other types of transducer material may be incorporated without departing from the scope of the present teachings. The transducer array may be coupled to a microbeamformer in the ultrasound transducer probe 145, which controls transmission and reception of signals by the transducer elements.

[0021] The transducer probe 145 is connected to the ultrasound controller 143 via a probe cable 147. The ultrasound controller 143 is configured to control the ultrasound imaging process, and includes known elements for performing ultrasound imaging, such as a transmit / receive (T / R) switch configured to switch between transmission and reception modes, and a main beamformer configured to provide final beamforming. One of the functions performed by the ultrasound controller 143 is the direction in which beams are steered and focused. For example, beams may be steered straight ahead from (orthogonal to) the transducer array of the transducer probe 145, or at different angles for a wider field of view. Generally, the transmitting of ultrasound signalsand the receiving and processing of echo signals in response is known, and therefore additional detail in this regard is not included herein. In various embodiments, all or part of the functionality of the ultrasound controller 143 may be implemented by the processor 120.

[0022] The memory 130 stores instructions executable by the processor 120. When executed, the instructions cause the processor 120 to implement one or more processes for performing a sarcopenia evaluation of the subject 150 using ultrasound images acquired by the ultrasound imaging system 140. The ultrasound images may be provided from the ultrasound imaging system 140 in real-time or near real-time during the scanning procedure, or may be retrieved from storage following the scanning procedure. For purposes of illustration, the memory 130 is shown to include software modules, each of which includes the instructions, executable by the processor 120, corresponding to an associated capability of the system 100.

[0023] The processor 120 is representative of one or more processing devices, and may be implemented by a general purpose computer, a central processing unit (CPU), a digital signal processor (DSP), a graphical processing unit, a computer processor, a microprocessor, a state machine, programmable logic device, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or combinations thereof, using any combination of hardware, software, firmware, hard-wired logic circuits, or combinations thereof. Any processor or processing unit herein may include multiple processors, parallel processors, or both. Multiple processors may be included in, or coupled to, a single device or multiple devices. The term “processor” as used herein encompasses an electronic component able to execute a program or machine executable instruction. A processor may also refer to a collection of processors within a single computer system or distributed among multiple computer systems, such as in a cloudbased or other multi-site application. Programs have software instructions performed by one or multiple processors that may be within the same computing device or which may be distributed across multiple computing devices.

[0024] The memory 130 may include main memory and / or static memory, where such memories may communicate with each other and the processor 120 via one or more buses. The memory 130 may be implemented by any number, type and combination of random access memory (RAM) and read-only memory (ROM), for example, and may store various types of information, such as software algorithms, artificial intelligence (Al) machine learning models, and computerprograms, all of which are executable by the processor 120. The various types of ROM and RAM may include any number, type and combination of computer readable storage media, such as a disk drive, flash memory, an electrically programmable read-only memory (EPROM), an electrically erasable and programmable read only memory (EEPROM), registers, a hard disk, a removable disk, tape, compact disk read only memory (CD-ROM), digital versatile disk (DVD), floppy disk, Blu-ray disk, a universal serial bus (USB) drive, or any other form of storage medium. The memory 130 is a tangible storage medium for storing data and executable software instructions, and is non-transitory during the time software instructions are stored therein. As used herein, the term “non-transitory” is to be interpreted not as an eternal characteristic of a state, but as a characteristic of a state that will last for a period. The term “non-transitory” specifically disavows fleeting characteristics such as characteristics of a carrier wave or signal or other forms that exist only transitorily in any place at any time. The memory 130 may store software instructions and / or computer readable code that enable performance of various functions. The memory 130 may be secure and / or encrypted, or unsecure and / or unencrypted.

[0025] The system 100 may also include a database 112 for storing information that may be used by the various software modules of the memory 130. For example, the database 112 may include image data from previously obtained ultrasound images of the subject 150 and / or of other similarly situated subjects. The stored image data may be used for training an Al machine learning model, such as a neural network model, for example, as discussed below. The database 112 may be implemented by any number, type and combination of RAM and ROM, for example. The various types of ROM and RAM may include any number, type and combination of computer readable storage media, such as a disk drive, flash memory, EPROM, EEPROM, registers, a hard disk, a removable disk, tape, CD-ROM, DVD, floppy disk, Blu-ray disk, USB drive, or any other form of storage medium known in the art. The database 112 comprises tangible storage mediums for storing data and executable software instructions and is non- transitory during the time data and software instructions are stored therein. The database 112 may be secure and / or encrypted, or unsecure and / or unencrypted. For purposes of illustration, the database 112 is shown as a separate storage medium, although it is understood that it may be combined with and / or included in the memory 130, without departing from the scope of the present teachings.

[0026] The processor 120 may include or have access to an artificial intelligence (Al) engine, which may be implemented as software that provides artificial intelligence (e.g., neutral network models) and applies machine learning described herein. The Al engine may reside in any of various components in addition to or other than the processor 120, such as the memory 130, an external server, and / or the cloud, for example. When the Al engine is implemented in a cloud, such as at a data center, for example, the Al engine may be connected to the processor 120 via the internet or other communication network using one or more wired and / or wireless connection(s). In various embodiments, all or part of the processes provided by first, second and third machine learning algorithms, discussed below, may be implemented by the Al engine, for example. The first, second and third machine learning algorithms cannot practically be performed in the human mind.

[0027] The user interface 122 is configured to provide information and data output by the processor 120, the memory 130 and / or the ultrasound imaging system 140 to the user and / or for receiving information and data input by the user. That is, the user interface 122 enables the user to enter data and to control or manipulate aspects of the processes described herein, and also enables the processor 120 to indicate the effects of the user’s input, which may include control or manipulation of the transducer probe 145 when the user / operator. All or a portion of the user interface 122 may be implemented by a graphical user interface (GUI), such as GUI 128 viewable on the display 124, discussed below. The user interface 122 may include one or more interface devices, such as a mouse, a keyboard, a trackball, a joystick, a microphone, a video camera, a touchpad, a touchscreen, voice or gesture recognition captured by a microphone or video camera, for example.

[0028] The display 124 may be a monitor such as a computer monitor, a television, a liquid crystal display (LCD), an organic light emitting diode (OLED), a flat panel display, a solid-state display, or a cathode ray tube (CRT) display, or an electronic whiteboard, for example. The display 124 includes a screen 126 for viewing ultrasound images of the subject 150, along with various features described herein to communicate to the user the degree of image degradation, if any, as well as the GUI 128 to enable the user to interact with the displayed images and features. In an embodiment, the ultrasound imaging system 140 may include a separate dedicated display for acquiring the ultrasound images, where dedicated display is also represented by the display124.

[0029] Referring to the memory 130, the various modules store sets of data and instructions executable by the processor 120 to detect image degradation, as mentioned above. Ultrasound image module 131 is configured to receive and process ultrasound images of a region of interest in the subject 150 acquired by the ultrasound imaging system 140, including the transducer probe 145. For purposes of illustration and not limitation, it may be assumed that the region of interest is the heart of the subject 150, and that the ultrasound images are two-dimensional (2D) cardiac ultrasound images or 2D slices of three-dimensional (3D) cardiac ultrasound images. Of course, effectively the same process described herein may be applied to other regions of interest and / or types of ultrasound imaging, such as fetal imaging or general ultrasound imaging, for example, without departing from the scope of the present teachings. The ultrasound image module 131 may also store data associated with the ultrasound images, such as time and date of image acquisition, identification of the ultrasound imaging system 140, and identification of the operator (e.g., sonographer, physician) operating the ultrasound imaging system 140 when the ultrasound images are acquired.

[0030] The ultrasound images may be displayed on the display 124. The ultrasound images may be received in real-time or near real-time from the ultrasound imaging system 140, e.g., during a contemporaneous imaging session of the subject 150. The display of real-time images, in particular, enables the operator to visualize the anatomy of the subject 150 while manipulating the transducer probe 145. Alternatively, or in addition, the ultrasound images may be previously acquired (historic) images, obtained during previous imaging session(s), which have been retrieved from storage (e.g., database 112), as mentioned above.

[0031] Cardiac view module 132 is configured to automatically determine a cardiac view of the heart from at least one ultrasound image of the ultrasound images received by the ultrasound image module 131. The cardiac view module 132 may include a cardiac view (first) machine learning algorithm that inputs the at least one ultrasound image, compares the at least one ultrasound image to multiple predetermined cardiac views with which it has been trained, and outputs the cardiac view that is most similar (closest match) to the cardiac view provided by the input ultrasound image. Examples of different cardiac views identifiable by the cardiac view module 132 include an apical two-chamber view, an apical three-chamber view, an apical four-chamber view, a parasternal short axis view, a parasternal long axis view, and a subcostal view.

[0032] The cardiac view machine learning algorithm may be implemented as any suitable type of trainable machine learning algorithm or model, such as a convolutional neural network (CNN), an artificial neural network (ANN), a vision transformer, or a U-net model, for example. The cardiac view machine learning algorithm may be trained in a supervised fashion using historic ultrasound images of hearts from various cardiac views as input and manually annotated views as targets. This provides a training data set that associates cardiac features of the historic ultrasound images of hearts with the various cardiac views of the hearts. The various cardiac views may correspond to different angles of image acquisition, for example. The training data set may be stored in the database 112, for example. After being trained on the historic ultrasound images and view targets, the cardiac view machine learning algorithm does not necessarily need additional historic data, as it is able to process new ultrasound images and predict corresponding cardiac views.

[0033] IQ models module 133 is configured to automatically select an IQ model from among multiple previously provided IQ models based on the cardiac view of the heart as determined by the cardiac view module 132, and to determine one or more IQ metrics associated with the ultrasound image using the selected IQ model. IQ metrics are objective measures of quality that correlate well with subjective perception of image quality by a human observer, while also rating unperceived errors, as would be apparent to one skilled in the art. Examples for IQ metrics include sharpness, signal-to-noise ratio (SNR), visibility of anatomical structures (e.g. cardiac walls, valves), view plane correctness, and others mentioned below.

[0034] The multiple IQ models respectively correspond to the multiple predetermined cardiac views of hearts that make up the training data set for the cardiac view machine learning algorithm of the cardiac view module 132, discussed above. Each of the IQ models includes a trained IQ (second) machine learning algorithm that predicts the one or more IQ metrics of an input ultrasound image at the corresponding cardiac view in order to determine image quality. Each of the IQ machine learning algorithms may be implemented as any suitable type of trainable machine learning algorithm or model, such as a CNN, an ANN, a vision transformer, or a U-net model, for example. Each of the IQ machine learning algorithms may be trained in a supervised fashion using historic ultrasound images of hearts from corresponding cardiac viewsas input and manually annotated views showing the IQ metrics as targets. This provides a training data set that associates image quality indicators in the ultrasound images, such as sharpness, SNR, visibility of anatomical structures, view plane correctness, as well as contrast, noise, wall visibility, and / or valve visibility, with the IQ metrics. IQ metrics for apical 2D views may include visibility of the walls in the left ventricle (LV), visibility of the right wall of the LV, and visibility of the LV apex, for example, although other types of IQ metrics may be included without departing from the scope of the present teachings. A similar approach can be pursued for valve visibility where annotators mark whether a valve is properly visible in an image or not. The training data set may be stored in the database 112, for example.

[0035] IQ time series module 134 is configured to provide a time series of image quality. In an embodiment, the IQ time series module 134 receives the IQ metrics output by the selected IQ model of the cardiac view module 132, and aggregates the IQ metrics into a single value for each time in the time series. For example, the IQ metrics of the same image at the same time may be averaged to provide the aggregated value. The aggregation of the IQ metrics may also include weighted averaging, where more weight is given to certain IQ metrics that are deemed more important. The IQ time series module 134 may also score the aggregated IQ metrics associated with the ultrasound images to provide aggregated IQ scores, where higher values of aggregated IQ metrics receive higher scores.

[0036] Degradation detection module 135 is configured to estimate the level of degradation of the ultrasound imaging system 140 based on the time series of image quality from the IQ time series module 134. The level of degradation may be estimated by a degradation detection (third) machine learning algorithm according to various embodiments. In an illustrative first embodiment, the degradation detection machine learning algorithm estimates the current level of degradation of the ultrasound imaging system directly from the time series of aggregated IQ scores. In an illustrative second embodiment, the degradation detection machine learning algorithm estimates (predicts or forecasts) a future level of degradation of the ultrasound imaging system 140 based on a time series trend of image quality that is estimated from the from the time series of aggregated IQ scores, for example, using extrapolation. The degradation detection machine learning algorithm may be implemented as any suitable type of trainable machine learning algorithm or model, such as a transformer-based network, a recurrent neural network(RNN), a long short-term memory (LSTM) network, or a gated recurrent unit (GRU), for example, discussed below. In terms of architecture, the degradation detection machine learning algorithm may be a recurrent model that includes one or several recurrent layers, followed by a fully-connected output layer.

[0037] With regard to the first embodiment, the degradation detection machine learning algorithm may be trained in a supervised fashion using historic ultrasound imaging system data from multiple ultrasound imaging systems and associated historic failure data that provide levels of degradation based on corresponding time series of image quality. The multiple ultrasound imaging systems may or may not include the ultrasound imaging system 140, for example. The levels of degradation indicated by the time series may include failure of the ultrasound imaging system, which would be 100 percent degradation. The levels of degradation may also include specific hallmarks short of failure, such as 25 percent or 50 percent degradation. In other words, failure of the ultrasound imaging system is considered complete degradation and other levels of degradation short of failure are measured or otherwise quantified relative to this complete degradation.

[0038] The target for training the degradation detection machine learning algorithm may be labeled based on historic data that indicates whether the time series being fed to the degradation detection machine learning algorithm was followed by hardware failure of the corresponding ultrasound imaging system. That is, the degradation detection machine learning algorithm is trained to predict hardware failure from the time series input. In other words, the target of the degradation detection machine learning algorithm is when the time-series results in hardware failure, which requires manual annotations and historic failure data. Percentage levels of degradation may also be annotated based on this target. When the training data does not include hardware failure, then the training data may be annotated to identify an unacceptable level of image quality as the target. A degradation threshold may be assigned to the image quality that corresponds to the hardware failure or the unacceptable level of image quality, for example. The degradation detection machine learning algorithm may then estimate the level of degradation of the ultrasound imaging system 140 by analyzing the time series of image quality from the IQ time series module 134 in the context of historic ultrasound imaging system failures. An alert may be issued to the user, e.g., via the display 124 or a networked display, when a hardwarefailure is predicted.

[0039] The time series may be tracked to provide internal quality checks for the user (e.g., operator, administrator and / or service engineer) through a dashboard, for example, shown on the display 124. The dashboard may display additional information relating to the time series of image quality, thereby providing a more complete picture. For example, the time series may be filtered according to filtering criteria, which may be extracted from meta information, for example. The filtering criterion may include date of scan, a type of transducer probe of the ultrasound imaging system, a type of ultrasound base system of the ultrasound imaging system, an identity of the operator (e.g., sonographer), a physical characteristic of the subject (e.g., BMI), and / or system settings (e.g., gain). The experience and skill of the sonographer, for example, is an important parameter that directly affects image quality, and the BMI of the subject influences the difficulty of the scans. The date of the scan is important for temporal analysis. A data source for the dashboard may be a relational database or table that contains filtering criteria, stored for example in the database 112. Alternatively, the degradation detection module 135 may provide one machine learning algorithm that has been trained for each filtering criterion, such as one algorithm for each sonographer, one algorithm for subject BMI, and the like.

[0040] In an embodiment, available filtering criteria may be provided to the user via the GUI 128 on the screen 126. The GUI 128 may display selectable elements or fields that correspond to the different filters, where the different filters have different filtering criteria, respectively. For example, the GUI 128 may provide a list of filters with associated selectable check boxes, or a drop down list of filters, so that the user may simply select, e.g., using a mouse or touchscreen, each of the filters to be applied to the time series of image quality. Alternatively, or in addition, the GUI 128 may provide a text field in which the user may type in the desired filter. In response, the selected filter is applied to the time series of image quality by the processor 120. Other parameters may be provided by the GUI 128 to be set by the user, such as length of time of the time series or a degradation threshold at which the GUI 128 generates a notification or alert. The degradation threshold may be set to correspond to imminent failure of the ultrasound imaging system or to some predetermined time prior to failure, for example.

[0041] The filtered time series and / or IQ metrics may be used to create meaningful charts to be shown on the display 124. For example, a chart may be created that shows the aggregated IQmetric over time for a certain sonographer, or that displays the aggregated IQ metric over time for patients with BMIs greater than 30. The display of such charts enables more detailed analysis of quality problems by the operator, administrator, or service engineer. Examples of time series filtered according to patient BMI and transducer probe are shown in FIGs. 2 A and 2B, respectively, discussed below.

[0042] With regard to the second embodiment, the degradation detection machine learning algorithm may be a time series forecasting algorithm with self-supervised training, without manual annotations. The training uses historic ultrasound imaging system data, including IQ metrics, over time. In this case, the degradation detection machine learning algorithm predicts the time series trend and associated degradation over a predetermined time in the future, which may be a relatively short time segment. The degradation detection machine learning algorithm may predict the time series trend of image quality by extrapolating the time series provided by the IQ time series module 134 over the predetermined time.

[0043] The predicted time series trend may be compared to a predetermined degradation threshold, which may be set manually or automatically as part of the degradation detections machine learning algorithm, in order to determine the level of degradation of the ultrasound imaging system 140 at the future time. The length of time in the future that may be addressed by the prediction may be a function of time scale. For example, if the time series from the IQ time series module 134 spans days, the degradation detections machine learning algorithm may predict the level of degradation roughly a day in advance. If the time series from the IQ time series module 134 spans weeks, the degradation detections machine learning algorithm may predict the level of degradation roughly a week in advance, or about +10 percent of the current time horizon.

[0044] In this embodiment, the degradation detection machine learning algorithm may initiate notification or alert to the user, e.g., via the display 124 or a networked display, when the estimated level of degradation falls below or approaches the predetermined degradation threshold. The target of the degradation detection machine learning algorithm is the future IQ value itself, and does not require manual annotations and historic failure data, as mentioned above.

[0045] FIG. 2A is a chart showing a time series of an ultrasound imaging system filtered by BMIof the patients, and FIG. 2B is a chart showing a time series of an ultrasound imaging system filtered by transducer probe, according to representative embodiments.

[0046] Referring to FIG. 2 A, trace 201 depicts a time series of image quality for ultrasound images over a predetermined time period. The time series is filtered according to patients having BMIs over 30. In this case, the time series as provided by the degradation detection module 135 shows no degradation of the ultrasound imaging system over the predetermined time period. Thus, no action is recommended or taken by the user.

[0047] Referring to FIG. 2B, trace 202 depicts a time series of image quality for ultrasound images over the same predetermined time period. The time series is filtered according to a specific transducer probe (US Probe ID12). In this case, the time series as provided by the degradation detection module 135 shows possible degradation of the ultrasound imaging system toward the end of the predetermined time period, where the estimated degradation is highlighted by dashed box 212. The left edge of the box 212 may be located at the time when the image quality drops below a predetermined threshold T, indicated on the image quality axis. As a result, the degradation detection module 135 may recommend and / or the user may take action to remove the particular transducer probe prior to the box 212.

[0048] FIG. 3 is a flow diagram of a method of detecting degradation of an ultrasound imaging system, according to a representative embodiment. The method may be implemented at least in part using instructions stored in memory 130 and executable by the processor 120 in the system 100, for example.

[0049] Referring to FIG. 3, multiple ultrasound images of a heart of a subject are received in block S311. The ultrasound images are acquired by the ultrasound imaging system, and may be received directly from the ultrasound system, e.g., during patient examinations, or may be received from a database of previously acquired ultrasound images.

[0050] In block S312, a cardiac view of the heart is automatically determined from at least one ultrasound image of the multiple ultrasound images acquired by the ultrasound imaging system. The cardiac view may be determined, for example, by applying the at least one ultrasound image to a trained cardiac view (first) machine learning algorithm. The cardiac view of the heart is one of multiple predetermined cardiac views of the heart. The cardiac view machine learning algorithm is configured to compare the at least one ultrasound image to the multiplepredetermined cardiac views, and to identify the closest match as the determined cardiac view of the heart.

[0051] In block S313, an IQ model is automatically selected from among multiple previously provided IQ models based on the determined cardiac view of the heart from block S312. The multiple IQ models correspond to the multiple predetermined cardiac views of the heart, respectively. Each of the IQ models, including the selected IQ model, includes a trained IQ (second) machine learning algorithm.

[0052] In block S314, IQ metrics associated with each ultrasound image of the multiple ultrasound images are determined. The IQ metrics may be determined by applying each ultrasound image of the multiple ultrasound images received in block S311 to the selected IQ model selected in block S313. The IQ metrics are indicative of the quality of the ultrasound image.

[0053] In block S315, the IQ metrics associated with each ultrasound image are aggregated to provide an aggregated IQ score associated with that ultrasound image. For example, an average value or a mean value may be determined to provide the aggregated IQ score. The aggregated IQ scores are collected over time to provide a time series of aggregated IQ scores.

[0054] In block S316, a level of degradation of the ultrasound imaging system is estimated by analyzing the time series trend of the aggregated IQ scores associated with the ultrasound images over time. The level of degradation is estimated also using the degradation detection machine learning algorithm.

[0055] In an embodiment, the estimated level of degradation is the current level of degradation of the ultrasound imaging system, which is estimated directly from the time series of aggregated IQ scores. In this case, the degradation detection machine learning algorithm is previously trained using historic ultrasound imaging system data, including IQ metrics, and associated historic failure data. The trained degradation detection machine learning algorithm is applied to the time series from block S315 to estimate the current level of degradation of the ultrasound imaging system. The level of degradation may be provided as a percentage of a total failure (100 percent). For example, an estimated level of degradation equal to 75 would indicate a 75 percent level of failure.

[0056] In another embodiment, the estimated level of degradation is a predicted level ofdegradation of the ultrasound imaging system based on forecasting a trend of the times series of aggregated IQ scores to a time in the future, and comparing the trend to a predetermined threshold, for example. In this case, the degradation detection machine learning algorithm is previously trained using historic ultrasound imaging system data, including IQ metrics, over a period of time. The trained degradation detection machine learning algorithm is applied to the time series of the aggregated IQ scores from block S315, and predicts a corresponding time series trend of image quality to a future time based on the aggregated IQ scores, e.g., by extrapolation the time series trend. The level of degradation of the ultrasound imaging system is estimated for the future time by analyzing the predicted time series trend of image quality. The estimated level of degradation may be based on a predetermined degradation threshold, where the predicted aggregated IQ score at the future time according to the time series trend is compared to the predetermined degradation threshold. The proximity to the predetermined degradation threshold indicates the estimated level of degradation, and passing the threshold indicates predicted failure of the ultrasound imaging system at the future time.

[0057] In block S317, the user is notified of the estimated level of degradation. In an embodiment, the user may be notified based on a predetermined degradation threshold. For example, the user may be notified when the estimated level of degradation drops below the threshold, or when the estimated level of degradation approaches the threshold so that the user may take action before the threshold is reached. The user may be visually notified via a display, although other types of notification may be provided, such as audio tones or flashing lights, without departing from the scope of the present teachings.

[0058] In accordance with various embodiments of the present disclosure, the methods described herein may be implemented using a hardware computer system that executes software programs stored on non-transitory storage mediums. Further, in an exemplary, non-limited embodiment, implementations can include distributed processing, component / object distributed processing, and parallel processing. Virtual computer system processing may implement one or more of the methods or functionalities as described herein, and a processor described herein may be used to support a virtual processing environment.

[0059] Although evaluating quality of an ultrasound imaging system has been described with reference to exemplary embodiments, it is understood that the words that have been used arewords of description and illustration, rather than words of limitation. Changes may be made within the purview of the appended claims, as presently stated and as amended, without departing from the scope and spirit of the embodiments. Also, although evaluating quality of an ultrasound imaging system has been described with reference to particular means, materials and embodiments, it is not intended to be limited to the particulars disclosed; rather evaluating quality of an ultrasound imaging system extends to all functionally equivalent structures, methods, and uses such as are within the scope of the appended claims.

[0060] The illustrations of the embodiments described herein are intended to provide a general understanding of the structure of the various embodiments. The illustrations are not intended to serve as a complete description of all of the elements and features of the disclosure described herein. Many other embodiments may be apparent to those of skill in the art upon reviewing the disclosure. Other embodiments may be utilized and derived from the disclosure, such that structural and logical substitutions and changes may be made without departing from the scope of the disclosure. Additionally, the illustrations are merely representational and may not be drawn to scale. Certain proportions within the illustrations may be exaggerated, while other proportions may be minimized. Accordingly, the disclosure and the figures are to be regarded as illustrative rather than restrictive.

[0061] One or more embodiments of the disclosure may be referred to herein, individually and / or collectively, by the term “invention” merely for convenience and without intending to voluntarily limit the scope of this application to any particular invention or inventive concept. Moreover, although specific embodiments have been illustrated and described herein, it should be appreciated that any subsequent arrangement designed to achieve the same or similar purpose may be substituted for the specific embodiments shown. This disclosure is intended to cover any and all subsequent adaptations or variations of various embodiments. Combinations of the above embodiments, and other embodiments not specifically described herein, will be apparent to those of skill in the art upon reviewing the description.

[0062] The Abstract of the Disclosure is provided to comply with 37 C.F.R. § 1.72(b) and is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, various features may be grouped together or described in a single embodiment for the purpose of streamlining thedisclosure. This disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter may be directed to less than all of the features of any of the disclosed embodiments. Thus, the following claims are incorporated into the Detailed Description, with each claim standing on its own as defining separately claimed subject matter.

[0063] The preceding description of the disclosed embodiments is provided to enable any person skilled in the art to practice the concepts described in the present disclosure. As such, the above disclosed subject matter is to be considered illustrative, and not restrictive, and the appended claims are intended to cover all such modifications, enhancements, and other embodiments which fall within the true spirit and scope of the present disclosure. Thus, to the maximum extent allowed by law, the scope of the present disclosure is to be determined by the broadest permissible interpretation of the following claims and their equivalents and shall not be restricted or limited by the foregoing detailed description.

Claims

CLAIMS:

1. A method for detecting degradation of an ultrasound imaging system (140), the method comprising: receiving a plurality of ultrasound images of a heart of a subject (150) acquired by the ultrasound imaging system (S311); automatically determining a cardiac view of the heart from at least one ultrasound image of the plurality of ultrasound images by applying the at least one ultrasound image to a trained cardiac view machine learning algorithm, wherein the cardiac view of the heart is one of a plurality of predetermined cardiac views of the heart (S312); automatically selecting an image quality (IQ) model from a plurality of IQ models based on the determined cardiac view of the heart, wherein the plurality of IQ models correspond to the plurality of predetermined cardiac views of the heart, respectively, and wherein the selected IQ model comprises a trained IQ machine learning algorithm (S313); determining a plurality of IQ metrics associated with each ultrasound image of the plurality of ultrasound images by applying each ultrasound image to the selected IQ model (S314); aggregating the plurality of IQ metrics to provide an aggregated IQ score associated with each ultrasound image of the plurality of ultrasound images to create a time series of aggregated IQ scores (S315); estimating a level of degradation of the ultrasound imaging system based on the time series of the aggregated IQ scores associated with the plurality of ultrasound images over time using a degradation detection machine learning algorithm (S316); and notifying a user of the estimated level of degradation (S317).

2. The method of claim 1 , wherein estimating the level of degradation of the ultrasound imaging system comprises: training the degradation detection machine learning algorithm in a supervised fashion using historic ultrasound imaging system data, including IQ metrics, from a plurality of ultrasound imaging systems and associated historic failure data;receiving the time series of aggregated IQ scores as input to the trained degradation detection machine learning algorithm; and estimating the level of degradation of the ultrasound imaging system based on the time series of aggregated IQ scores.

3. The method of claim 1 wherein estimating the level of degradation of the ultrasound imaging system comprises: training the degradation detection machine learning algorithm in a self-supervised fashion, without manual annotations, using historic ultrasound imaging system data over time, including IQ metrics, from a plurality of ultrasound imaging systems; receiving the time series of the aggregated IQ scores as input to the trained degradation detection machine learning algorithm; predicting a time series trend of image quality to a future time based on the aggregated IQ scores; estimating the level of degradation of the ultrasound imaging system at the future time by analyzing the predicted time series trend of image quality; and notifying the user of the estimated level of degradation based on a predetermined degradation threshold.

4. The method of claim 3, wherein predicting the time series trend comprises extrapolating the aggregated IQ scores in the time series to the future time.

5. The method of claim 2, further comprising: filtering the aggregated IQ scores using a filtering criterion, wherein the time series of the aggregated IQ scores is based on the filtered aggregated IQ scores associated with the plurality of ultrasound images.

6. The method of claim 5, wherein the filtering criterion comprises one of a type of transducer probe of the ultrasound imaging system, a type of ultrasound base system of theultrasound imaging system, an identity of the sonographer, or a physical characteristic of the subject.

7. The method of claim 5, further comprising: displaying on a graphical user interface a plurality of filters on a display; receiving a selection by the user of a filter from the plurality of filters to be applied to the time series of the aggregated IQ scores, wherein the selected filter includes the filtering criterion; performing the filtering of the aggregated IQ scores using the filtering criterion in response to the selection; and displaying the filtered aggregated IQ scores over time on the display.

8. The method of claim 1, wherein the plurality of IQ metrics include at least one of wall visibility or valve visibility.

9. A system for detecting degradation of an ultrasound imaging system (140), the system comprising: a display (124); a processor (120); and a non-transitory memory (130) storing instructions that, when executed by the processor, cause the processor to: receive a plurality of ultrasound images of a heart of a subject acquired by the ultrasound imaging system; automatically determine a cardiac view of the heart from at least one ultrasound image of the plurality of ultrasound images by applying the at least one ultrasound image to a trained cardiac view machine learning algorithm, wherein the cardiac view of the heart is one of a plurality of predetermined cardiac views of the heart; automatically select an image quality (IQ) model from a plurality of IQ models based on the determined cardiac view of the heart, wherein the plurality of IQ models correspond to the plurality of predetermined cardiac views of the heart, respectively, and wherein the selected IQ model comprises a trained IQ machine learning algorithm;determine a plurality of IQ metrics associated with each ultrasound image of the plurality of ultrasound images by applying each ultrasound image to the selected IQ model; aggregate the plurality of IQ metrics to provide an aggregated IQ score associated with each ultrasound image of the plurality of ultrasound images to create a time series of aggregated IQ scores; estimate a level of degradation of the ultrasound imaging system based on the time series of the aggregated IQ scores associated with the plurality of ultrasound images over time using a degradation detection machine learning algorithm; and cause a notification regarding the estimated level of degradation to be displayed on the display.

10. The system of claim 9, wherein the instructions cause the processor to estimate the level of degradation of the ultrasound imaging system by: training the degradation detection machine learning algorithm in a supervised fashion using historic ultrasound imaging system data, including IQ metrics, from a plurality of ultrasound imaging systems and associated historic failure data; receiving the time series of aggregated IQ scores as input to the trained degradation detection machine learning algorithm; and estimating the level of degradation of the ultrasound imaging system based on the time series of aggregated IQ scores.

11. The system of claim 9, wherein the instructions cause the processor to estimate the level of degradation of the ultrasound imaging system by: training the degradation detection machine learning algorithm in a self-supervised fashion, without manual annotations, using historic ultrasound imaging system data over time, including IQ metrics, from a plurality of ultrasound imaging systems; receiving the time series of the aggregated IQ scores as input to the trained degradation detection machine learning algorithm;predicting a time series trend of image quality to a future time based on the aggregated IQ scores; estimating the level of degradation of the ultrasound imaging system at the future time by analyzing the predicted time series trend of image quality; and notifying a user of the estimated level of degradation based on a predetermined degradation threshold.

12. The system of claim 11, wherein predicting the time series trend comprises extrapolating the aggregated IQ scores in the time series to the future time.

13. The system of claim 9, wherein the trained cardiac view machine learning algorithm comprises a convolutional neural network (CNN), an artificial neural network (ANN), or a U-net model that has been trained in a supervised fashion using a plurality of historic ultrasound images that have been manually annotated to identify corresponding cardiac views of the heart as a first training data set.

14. The system of claim 9, wherein the trained IQ machine learning algorithm comprises a CNN, an ANN, or a U-net model that has been trained in a supervised fashion using a plurality of historic ultrasound images that have been manually annotated to grade quality of at least one parameter in each of the plurality of historic ultrasound images as a second training data.

15. The system of claim 9, wherein the trained degradation detection machine learning algorithm comprises a transformer-based network, a recurrent neural network (RNN), a long short-term memory (LSTM) network, or a gated recurrent unit (GRU).

16. The system of claim 10, further comprising: a graphical user interface (GUI) displayable on the display, wherein the GUI is configured to receive a selection by a user of a filter from a plurality of filters to be applied to the time series of the aggregated IQ scores, wherein the selected filter includes a corresponding filtering criterion,wherein, in response to the selection of the filter, the instructions further cause the processor to: filter the aggregated IQ scores using the corresponding filtering criterion, wherein the time series of the aggregated IQ scores is based on the filtered aggregated IQ scores associated with the plurality of ultrasound images; and display the filtered aggregated IQ scores over time on the display.

17. The system of claim 16, wherein the filtering criterion comprises one of a type of transducer probe of the ultrasound imaging system, a type of ultrasound base system of the ultrasound imaging system, an identity of the sonographer, or a physical characteristic of the subject.

18. A non- transitory computer readable medium storing instructions for detecting degradation of an ultrasound imaging system that, when executed by a processor, cause the processor to: receive a plurality of ultrasound images of a heart of a subject acquired by the ultrasound imaging system; automatically determine a cardiac view of the heart from at least one ultrasound image of the plurality of ultrasound images by applying the at least one ultrasound image to a trained cardiac view machine learning algorithm, wherein the cardiac view of the heart is one of a plurality of predetermined cardiac views of the heart; automatically select an image quality (IQ) model from a plurality of IQ models based on the determined cardiac view of the heart, wherein the plurality of IQ models correspond to the plurality of predetermined cardiac views of the heart, respectively, and wherein the selected IQ model comprises a trained IQ machine learning algorithm; determine a plurality of IQ metrics associated with each ultrasound image of the plurality of ultrasound images by applying each ultrasound image to the selected IQ model; aggregate the plurality of IQ metrics to provide an aggregated IQ score associated with each ultrasound image of the plurality of ultrasound images to create a time series of aggregated IQ scores;estimate a level of degradation of the ultrasound imaging system based on the time series of the aggregated IQ scores associated with the plurality of ultrasound images over time using a degradation detection machine learning algorithm; and cause a notification regarding the estimated level of degradation to be displayed to a user on the display.

19. The non-transitory computer readable medium of claim 18, wherein the instructions cause the processor to estimate the level of degradation of the ultrasound imaging system by: training the degradation detection machine learning algorithm in a supervised fashion using historic ultrasound imaging system data, including IQ metrics, from a plurality of ultrasound imaging systems and associated historic failure data; receiving the time series of aggregated IQ scores as input to the trained degradation detection machine learning algorithm; and estimating the level of degradation of the ultrasound imaging system based on the time series of aggregated IQ scores.

20. The non-transitory computer readable medium of claim 18, wherein the instructions cause the processor to estimate the level of degradation of the ultrasound imaging system by: training the degradation detection machine learning algorithm in a self-supervised fashion, without manual annotations, using historic ultrasound imaging system data over time, including IQ metrics, from a plurality of ultrasound imaging systems; receiving the time series of the aggregated IQ scores as input to the trained degradation detection machine learning algorithm; predicting a time series trend of image quality to a future time based on the aggregated IQ scores; estimating the level of degradation of the ultrasound imaging system at the future time by analyzing the predicted time series trend of image quality; and notifying the user of the estimated level of degradation based on a predetermined degradation threshold.

Images