Ultrasound guidance method and system

Abstract

The method and system are for assisting a user in positioning an ultrasound probe relative to a subject's body. In use, a user moves the probe through a series of positions, with ultrasound image data being acquired for each position, the data at each position corresponding to a particular imaging view of the anatomical region being imaged. An evaluation procedure (16) is operable to process the ultrasound data for the different imaging views and determine, for the data for each view, an evaluation that represents the suitability of the data for the view for observing one or more anatomical features or for deriving one or more clinical parameters from the data. Guidance feedback is generated for the user to assist them in positioning the probe. The guidance feedback is based on a history of imaging view quality (26) over time over the course of the imaging procedure.

Description

[Technical field]

[0001] The present invention relates to a method and system for assisting a user in positioning an ultrasound probe of an ultrasound imaging system. [Background technology]

[0002] Ultrasound imaging is widely used in medicine to observe anatomical regions of interest and to derive measurements or estimates of certain important physiological parameters or anatomical features. For example, the dimensions of the imaged anatomical features can be determined, which in some cases can be performed automatically, for example, using a segmentation procedure. Also, in some cases, physiological parameters, such as cardiac output, can be derived based on monitoring of a relevant anatomical region (e.g., the left ventricle) over a period of time. Again, this can in some cases be performed automatically, for example, using segmentation or other forms of image analysis.

[0003] One form of ultrasound imaging is transesophageal echocardiography (TEE).

[0004] The use of transesophageal echocardiography (TEE) to monitor cardiac function, for example, during surgery, is increasing clinically. To view the left ventricle, the TEE probe is navigated to a mid-esophageal position. If the clinician wishes to view the heart at any point during surgery, the clinician manipulates the probe as necessary to ensure that the correct view is captured by the probe transducer.

[0005] The use of TEE ultrasound for continuous monitoring of the heart has been proposed, but little development has been done in this field. Continuous monitoring refers to the case where the probe is held in place for an extended period of time (e.g., the duration of a surgical procedure) and ultrasound data is captured throughout.

[0006] In this context, TEE ultrasound can be used not only for qualitative observation of cardiac anatomical features, e.g., dynamic motion of the heart in the captured US images, but also to perform quantitative measurements. Such measurements can include, for example, left ventricular (LV) volume, stroke volume (SV), and cardiac output (CO). In some cases, model-based segmentation (MBS) can be applied to derive such measurements or parameters. Automatic segmentation may, for example, be performed for one or more chambers of the heart. Other methods of deriving quantitative measurements can include hybrid approaches, such as performing a full MBS on the captured end-diastolic (ED) frame, followed by 2D or 3D tracing of the segmentation boundary over other frames of the cardiac cycle.

[0007] Long-term transesophageal echocardiographic (TEE) monitoring (e.g., over several hours) can be difficult because the probe position can drift over time. Furthermore, measurements may be desired even when a specialized operator (e.g., a cardiologist, intensivist, or other specialist) is not present (e.g., in an intensive care unit). This creates problems because the drift in the probe position interrupts the acquisition of accurate ultrasound observations and measurements, and potentially, no specialized operator is present to correct the position.

[0008] Therefore, a means to allow a non-expert operator to properly position the probe would be of value. Summary of the Invention [Problem to be solved by the invention]

[0009] The inventors of the present invention have previously proposed a new positioning assistance system that can provide real-time feedback as to whether an image view from a current probe position results in an accurate measurement of a given parameter or parameters.

[0010] However, static images often do not provide sufficient information because the motion of the probe is coupled with the motion of the heart. Thus, while a user may momentarily arrive at the best possible probe position, a static image-based method may not be able to identify that position as optimal (e.g., due to a lack of sufficient image features) before the user moves the probe again, and thus to a different view. Alternatively, the user may be close to the optimal view, but may lack sufficient information to know how to ultimately adjust the probe to achieve the optimal view.

[0011] Similar problems arise with respect to the positioning of other types of ultrasound probes, such as body ultrasound probes, and thus the problem is not limited to transesophageal imaging alone, or even to cardiac imaging alone.

[0012] Therefore, improved approaches to assist a user in positioning an ultrasound probe to obtain optimal data for observing one or more features or for deriving one or more parameters would be of value. [Means for solving the problem]

[0013] The invention is defined by the claims.

[0014] According to an example according to one aspect of the present invention, there is provided a method of guiding a user of an ultrasound system during ultrasound imaging of a body of a subject, the ultrasound system having an ultrasound probe having a movable position relative to the subject, the method comprising: receiving ultrasound data from the probe during an ultrasound imaging procedure, the ultrasound data representing a series of ultrasound imaging views, each view corresponding to data acquired from a respective position of the probe; At each point during the procedure, applying an evaluation procedure to the ultrasound data, the procedure operable to evaluate a quality of the ultrasound imaging views, the quality relating to the suitability of the ultrasound data for different imaging views for observing one or more anatomical features or for deriving one or more clinical parameters from the data; generating guidance feedback to assist a user in positioning the probe based on the evaluation procedure, the guidance feedback being based on a history of quality of imaging views over time during the imaging procedure; A method is provided comprising:

[0015] Thus, embodiments of the present invention are based on using the time history of image views captured by the probe and issuing feedback to the user based thereon to assist the user in determining how the probe can be moved to achieve optimal imaging views for observing a particular feature or for deriving one or more clinical measurements. The feedback may simply be a representation of the history of image view quality over time. This allows the user to observe whether the latest path along which the probe is moved is improving or deteriorating quality. This alone provides useful feedback to guide how to continue the movement of the probe. For example, if the quality is improving, continue moving along the same path, if not, change the path.

[0016] A quality assessment may be explicitly calculated for each imaging view indicating the suitability of the data for deriving clinical parameters or for observing anatomical features, or data for multiple views may be assessed collectively and quality trends across a series of views may be evaluated and output in a single operation. In either case, the method implicitly takes into account a quality assessment for each imaging view and provides feedback to the user based on the history of changes in the quality of the views over time.

[0017] Unlike more complex probe positioning guidance techniques, the method does not require in preferred embodiments that the actual physical position of the probe be measured and tracked, but can simply be based on image quality and changes in this quality over time as the probe is moved through different views. For example, the trend of image quality may be used to determine whether image quality is increasing or decreasing over time as the probe is moved. A positive trend (increasing image quality) may indicate that movement of the probe over the course of the trend period has brought the probe closer to an optimal imaging position, or at least that movement in that direction is improving image quality. Feedback can then be generated recommending maintaining the current movement trajectory. If the trend indicates a decrease in image quality, the feedback may include a recommendation to stop or change the current movement trajectory. These are just examples. Other examples are described in more detail below.

[0018] The approach proposed in embodiments of the present invention is therefore much simpler, both in terms of software and hardware, than approaches that rely on physical tracking of the probe. It also allows for a very intuitive feedback approach: it effectively emulates the cognitive process of a highly skilled user who tries to position the probe based on a qualitative assessment of the changing quality of the image views as he adjusts the probe.

[0019] Furthermore, by providing feedback based on historical quality across a sequence of image views, this allows the feedback to take into account temporal features (e.g., changes in features over time) as well as purely static spatial features. This provides an improvement over approaches based on assessing the quality of each single still image, for example, since the quality trend over a recent period provides greater guidance as to how motion should be continued thereafter to improve image quality.

[0020] The quality assessment mentioned above may be for assessing the quality of a view to observe a particular anatomical feature or body or parameter, or to derive or measure one or more parameters.

[0021] The ultrasound data preferably represents temporally consecutive image views, and may be a data sequence or a data stream.

[0022] The method is preferably performed in real time during the imaging procedure (scan). It should be understood by those skilled in the art in the context of this application that the term imaging is not limited to procedures, data, or systems that can provide an image perceived by the subject or user. Image or scan should be interpreted in the broader sense of any data acquisition suitable for evaluating internal subject anatomy (such as raw ultrasound data before its reconstruction). The real-time nature of the method can be beneficial, as it relaxes the constraint that the actual extraction of one or more clinical parameters should be performed in real time. This is because the method guides the user to the optimal data acquisition position for deriving the clinical parameters, which means that it is no longer necessary to calculate the clinical parameters in real time for each and every position of the probe as it is moved. This can therefore enable the use of highly accurate (but potentially slower running) quantification algorithms such as model-based segmentation (MBS) in some examples.

[0023] The method is a computer-implemented method, which may be implemented, for example, by one or more processors or computers.

[0024] As described above, the method can include determining, for each imaging view, a quality assessment indicative of a suitability of the ultrasound data for the imaging view for observing one or more anatomical features or deriving one or more clinical parameters from the data.

[0025] As also mentioned above, the method may include a step of determining a trend of temporal quality, and the feedback is based on said trend. In other words, the feedback is generated based on changes in image view ratings over time. This may include, for example, determining a trend of the temporally calculated image quality ratings. Alternatively, it may include determining a trend of the quality of the image views over time, without explicitly calculating a quality rating for each individual image view.

[0026] The feedback may possibly indicate the trend, for example, may simply represent the trend, where the change in time quality is explicitly displayed or otherwise communicated, such that the user can know whether they are moving towards or away from an optimal position.

[0027] For example, the feedback may be a sensory output that represents a trend as a function of time.

[0028] As mentioned above, the quality trend indicates the pattern of change in quality, from which it can be more easily derived how the probe position should be further modified to improve the quality.

[0029] In some examples, the method may include constructing a data set representing a history of image view quality over time during the imaging procedure based on the evaluation procedure.

[0030] In this case, the method includes a separate step of determining a history of image view quality (e.g., image view quality assessment). In alternative examples, the history may be taken into account implicitly, e.g., the method may include deriving trends in image view quality over time, e.g., using a machine learning algorithm.

[0031] In accordance with one or more embodiments, the feedback may be visual feedback indicating a history of the image visual quality as a function of time.

[0032] For example, the feedback may be the formation of a plot or graph of image view quality as a function of time.

[0033] From this, the user can quickly determine whether the quality is increasing or decreasing and therefore the user can quickly decide whether to continue along the current travel path or to change the travel path.

[0034] According to one or more embodiments, the method may include determining a trend in time quality, the feedback being indicative of the trend.

[0035] Here, the change in time quality is explicitly displayed or otherwise communicated, so that the user knows whether they are moving toward or away from an optimal position.

[0036] According to one or more embodiments, the feedback can include feedback indicating a proposed movement of the probe position to capture image data of improved quality, such as guidance for movement to move toward an optimal probe position. For example, the feedback can include an indication of a direction and / or distance to move the probe.

[0037] This can be based on determining a trend in the time quality, and feedback can be based on this trend, for example if the quality is improving the feedback can be to continue the travel path the user is following, and if the quality is deteriorating the feedback can be to change the travel path.

[0038] Thus, in some cases, the guidance feedback includes a recommendation to continue the current path of movement of the probe or to discontinue the current path of movement of the probe depending on whether the quality assessment trend is positive or negative.

[0039] In some further examples, guidance feedback may indicate a suggested distance and / or speed of movement, and a suggested direction of movement may also be provided, although this may require the provision of a positioning sensor for the probe to achieve.

[0040] There are various ways in which the evaluation procedure processes the ultrasound image data to derive image quality information.

[0041] According to one set of embodiments, the procedure may be applied imaging view by imaging view, ie, data relating to one imaging view or frame or snippet at a time is processed with the procedure.

[0042] Thus, now an assessment procedure may be applied in turn to the ultrasound data for each image view, and to each quality assessment generated for each image view.

[0043] Thus, the evaluation procedure may be applied to data for different image views one at a time, and an evaluation may be generated individually and in turn for each image view. According to an alternative set of embodiments, the evaluation procedure may take as input ultrasound data for an entire series of imaging views and generate an evaluation output for the series of image views. Thus, here, the evaluation procedure is applied to ultrasound data for multiple ultrasound image views at once. The multiple imaging views may be consecutive in time. In some instances, they may be ordered in time, i.e., according to the order in which they are acquired during a scan, but may include only every mth image view of a series of N acquired image views. For example, the evaluation procedure may be applied to an input set of image views that includes every other image view from a previous set of N acquired image views.

[0044] The evaluation procedure may include explicitly determining a quality assessment for each imaging view, or may implicitly take this into account and generate output quality information indicative of trends or changes in temporal quality over the sequence of images. For example, a machine learning algorithm may be used that is first trained on ultrasound image data corresponding to the entire sequence of imaging views and trained to determine quality trends, where the algorithm implicitly takes into account the quality of each image view but does not explicitly calculate it. Alternatively, a non-machine learning algorithm may explicitly determine a quality assessment for each imaging view.

[0045] In either case, by considering multiple temporally consecutive image views at once, the assessment decision can take into account the temporal context. For example, many anatomical features exhibit cyclical motion (e.g., heart, lungs), and their suitability for deriving clinical parameters is high during one phase of their duration and low during another. Evaluating a series of image views at once allows such cyclical features to be taken into account in order to avoid erroneously designating a particular image view as low quality when in fact it is of high quality but varies periodically.

[0046] According to one or more examples, the method may include applying an ultrasound imaging procedure to a complete set of ultrasound data captured during an imaging procedure, the data representing the complete set of imaging views encompassed by the imaging procedure, where in each execution of the evaluation procedure, the most recently captured image view and all of the previous image views (or at least for a defined previous time window) are processed, which may be evaluated all at once or separately, depending on which of the two approaches outlined above is used.

[0047] In some examples, the evaluation procedure may be applied iteratively to the complete set of image data upon capture of the data for each new imaging view.

[0048] In this way, the assessment is re-performed each time a new view is acquired (e.g., each time the probe is moved to a new position), allowing image quality trends to be continually updated.

[0049] In some examples, a history of image view quality (eg, image quality assessment) over time is determined as part of the procedure, the procedure being configured to determine a trend in image view quality as a function of time.

[0050] Here, all the analysis steps are performed as part of a single integrated evaluation procedure, which may for example be a machine learning or artificial intelligence procedure.

[0051] According to one or more embodiments, the procedure may utilize a machine learning procedure, which may include the use of one or more machine learning algorithms.

[0052] In an alternative example, a non-machine learning algorithm can be used to perform the evaluation procedure. For example, a model-based segmentation procedure can be applied to identify anatomical features present in the data of the captured view, and a quality assessment can be determined, for example, based on the presence or absence of certain features. According to a particular example, the evaluation procedure can include the use of a convolutional neural network, CNN.

[0053] The CNN is trained with training data. The training data may include ultrasound data for different image views, and the data is labeled with a quality assessment. The quality assessment is generated in advance by a human being to be trained, and the data is labeled accordingly. In another example, the training data may include a set of ultrasound data for different sequences of imaging views, each of which is labeled with a trend of the quality assessment as a function of time. In another example, the training data may be partially labeled or unlabeled, and the labels or annotations are determined during the training process itself, for example, by an automatic method such as segmentation or object detection. For example, if the segmentation is successful (i.e., if the anatomical object of interest is identified in the image), the quality of the image may be labeled as good. Otherwise, the image quality may be labeled as poor. In another example, the training data may be partially labeled, and unlabeled training instances may be incorporated into the training using an active learning approach.

[0054] In one or more embodiments, the evaluation procedure may include identifying or detecting the presence of one or more anatomical features in the ultrasound data for different image views, and a quality (e.g., a quality assessment) of the image views is determined based on the detection.

[0055] For example, the quality determination may be based on the presence or absence of one or more anatomical features.

[0056] In some examples, the determined quality (e.g., a quality rating) may indicate or relate to the degree to which an image view contains a particular anatomical feature. For example, if only half of the anatomical feature of interest is present in the image, the quality rating may be 50%, while if none of the features are present, the quality rating is 0%. For example, the quality may be related to the percentage of the anatomical feature or region of interest that is included in the image view.

[0057] In some examples, the quality assessment may relate to the presence and intensity of ultrasound image artifacts in combination with anatomical features. Image artifacts may be present, for example, due to poor acoustic coupling, anatomical abnormalities (calcifications, enlarged ventricles), or suboptimal image settings. Examples of image artifacts include speckle noise, reverberation, side lobes, or acoustic shadowing artifacts.

[0058] In some examples, the evaluation procedure may include evaluating the motion of one or more anatomical features across a sequence of image views.

[0059] For example, if the procedure receives an entire series of image views as input, the procedure may be configured to explicitly or implicitly track the movement of anatomical features across the series of views, e.g., to determine trends in temporal image quality for the series of image views.

[0060] An example according to a further aspect of the present invention provides a computer program product comprising code means configured to, when executed on a processor, cause the processor to perform a method according to any of the embodiments or examples outlined above or described herein, or according to any claim of the present application.

[0061] An example according to a further aspect of the present invention also provides a guidance system for guiding a user of an ultrasound imaging system during ultrasound imaging of a body of a subject, comprising: the ultrasound system includes an ultrasound probe having a movable position relative to the subject, and the guidance system is movably coupleable to the ultrasound system during use; The guidance system comprises: a receive module configured to receive ultrasound data representing a plurality of ultrasound imaging views from the probe during an ultrasound imaging procedure, each view corresponding to data acquired from a respective position of the probe; At each point during the procedure, an evaluation module configured to apply an evaluation procedure to the ultrasound data, the procedure evaluating a quality of the ultrasound imaging views, the quality relating to a suitability of the ultrasound data for different imaging views for observing one or more anatomical features or for deriving one or more clinical parameters from the data; a feedback module configured to generate guidance feedback to assist a user in positioning the probe based on the evaluation procedure, the guidance feedback being based on a history of quality of imaging views over time during the imaging procedure; The present invention provides a guidance system comprising:

[0062] The modules may be separate processing components or may represent the functionality of a single integrated processing unit. Each module may comprise one or more processors, or the functionality of all the modules may be performed by a single integrated processor or processing configuration.

[0063] An embodiment of the present invention also comprises an ultrasound imaging probe including one or more ultrasound transducers for acquiring ultrasound data; A guidance system according to any of the claims hereinbefore or hereinafter outlined. 1. An ultrasound imaging system comprising: an ultrasound imaging probe communicatively coupled to a receiving module of the guidance system for transmitting captured ultrasound data; An ultrasound imaging system is provided.

[0064] According to one or more embodiments, the ultrasound imaging probe may be a transesophageal echocardiography (TEE) probe.

[0065] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

[0066] For a better understanding of the present invention and to show more clearly how it may be carried into effect, reference will now be made, by way of example only, to the accompanying drawings in which: [Brief description of the drawings]

[0067] [Figure 1] 1 illustrates a block diagram outlining an exemplary method in accordance with one or more embodiments. [Diagram 2] 1 illustrates a schematic workflow of an exemplary method according to one or more embodiments. [Diagram 3] 1 further illustrates, in a schematic manner, the workflow of an exemplary method according to one group of embodiments. [Figure 4] 13 illustrates a schematic workflow of an exemplary method according to an embodiment of a further group; [Diagram 5] 1 illustrates a schematic workflow during an exemplary implementation of a method according to one or more embodiments. [Figure 6] 1 illustrates a schematic workflow during an exemplary implementation of a method according to one or more embodiments. [Figure 7] 1 shows an example set of captured ultrasound image views and associated quality scores. [Figure 8] 1 illustrates generally in block diagram form an exemplary guidance system in accordance with one or more embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0068] The present invention will now be described with reference to the drawings.

[0069] It should be understood that the detailed description and specific examples, while indicating exemplary embodiments of the devices, systems, and methods, are intended for purposes of illustration only and are not intended to limit the scope of the invention. These and other features, aspects, and advantages of the devices, systems, and methods of the present invention will become better understood from the following description, appended claims, and accompanying drawings. It should be understood that the drawings are merely schematic and are not drawn to scale. It should also be understood that the same reference numerals are used throughout the drawings to denote the same or similar parts.

[0070] The present invention provides a method and system for assisting a user in positioning an ultrasound probe relative to a subject's body. In use, a user moves the probe through a series of positions, for each position ultrasound image data is captured, the data at each position corresponding to a particular imaging view of the anatomical region being imaged. An evaluation procedure is operable to process the ultrasound data for the different image views and determine for the data for each view a quality rating representative of the suitability of the data for that view for observing one or more anatomical features or for deriving one or more clinical parameters from the data. Guidance feedback is generated to the user to assist the user in positioning the probe, the guidance feedback being based on a history of image viewing rates over time over the course of the imaging procedure. For example, a trend in image view ratings over time can be determined and output to the user, from which the user can quickly tell if they are moving towards (increasing the rating) or away from (decreasing the rating) an optimal imaging position.

[0071] This embodiment of the invention is based on using image features from the recent time history of the probe's image data to provide insightful feedback to the user, who can assist in adjusting the probe positioning to achieve an optimal view of the anatomical body of interest, for example to enable optimal acquisition of a particular clinical parameter or measurement. This helps to ensure that optimal probe positions are achieved faster by the operator and are not overlooked during their exploration.

[0072] One or more embodiments also propose a user interface operable to display guidance on how the user should move the probe based on the user's recent probe movements, which may be, for example, an indication to continue the previous path of the probe movement, or, for example, to change the path.

[0073] One or more embodiments are described below that utilize artificial intelligence (AI), e.g., machine learning, approaches, however, other conventional algorithmic approaches may also be used in further examples.

[0074] An exemplary guidance method according to one or more embodiments is generally outlined in block diagram form in FIG.

[0075] Method 10 is a method for guiding a user of an ultrasound imaging system during ultrasound imaging of a subject's body, where the ultrasound imaging system includes an ultrasound probe having a movable position relative to the subject's body.

[0076] The method 10 includes receiving 14 ultrasound data representing a series of ultrasound image views from a probe during an ultrasound imaging procedure, each view corresponding to data captured from a respective position of the probe and corresponding to a respective time during the procedure.

[0077] The method 10 further includes applying 16 a procedure to the ultrasound data that is operable to determine, for each image view, a quality assessment indicative of the suitability of the ultrasound data for that image view for observing one or more anatomical features or for deriving one or more clinical parameters from the data.

[0078] The method 10 further includes generating guidance feedback 18 to assist a user in positioning the probe based on the evaluation procedure, the guidance feedback being based on a history of image viewing over time during the imaging procedure.

[0079] The ultrasound data for each imaging view may correspond to image data for a single 2D or 3D image (i.e., an image frame), however, in alternative cases the image data may correspond to image data for a series of frames, e.g., small video snippets captured for a particular position of the probe, i.e., a particular imaging view.

[0080] The method is a computer-implemented method, e.g., may be executed on one or more processors or computers, which may be operatively coupled to and positioned in use with an ultrasound probe or ultrasound imaging system according to one or more embodiments.

[0081] In one or more embodiments, the assessment procedure may include identifying or detecting the presence of one or more anatomical features in the ultrasound data for the different image views, and the quality assessment is based on said detection.

[0082] For example, the quality assessment may be based on the presence or absence of one or more anatomical features.

[0083] In some examples, the quality assessment may indicate or relate to the degree to which an image view contains a particular anatomical feature. For example, if only half of the anatomical feature of interest is present in the image, the quality assessment may be 50%, while if none of the features are present, the quality assessment is 0%. For example, the assessment may be related to the percentage of the anatomical feature or region of interest that is included in the image view.

[0084] In some examples, the evaluation procedure can include evaluating the motion of one or more anatomical features across a sequence of image views. For example, if the procedure receives an entire sequence of image views as input, the procedure may be configured to explicitly or implicitly track the motion of the anatomical features across the sequence of views, e.g., to determine a trend in the temporal image quality evaluation for the sequence of image views.

[0085] Exemplary clinical parameters that may be extracted or derived or measured from ultrasound image data may include dimensions of one or more anatomical features, such as the external or internal dimensions of a section of the heart, such as the left or right ventricle. Derived clinical parameters may include, by way of non-limiting example, left ventricular (LV) volume, stroke volume (SV), and cardiac output (CO). In some cases, model-based segmentation (MBS) may be applied to derive such measurements or parameters. Automated segmentation may be performed, for example, for one or more chambers of the heart. Other methods of deriving quantitative measurements may include hybrid approaches, such as performing a full MBS on the captured end-diastolic (ED) frame, followed by 2D or 3D tracing of the segmentation boundary across other frames of the cardiac cycle.

[0086] The evaluation procedure is configured to evaluate the suitability of a given achieved image view for extracting one or more clinical parameters or measurements using a given extraction technique or techniques, or for observing one or more anatomical features, on which a quality assessment can be derived. This assessment may be based on the detection of one or more anatomical features in the imaging view (as described above), or on another image analysis technique or procedure. Those skilled in the art will readily appreciate various image processing and analysis techniques for achieving this evaluation function.

[0087] According to one preferred set of embodiments, the evaluation procedure may use one or more machine learning algorithms.

[0088] A machine learning algorithm is any self-learning algorithm that processes input data to generate or predict output data, where the input data can include ultrasound imaging data corresponding to different imaging views, and the output data can include quality assessments of the imaging data (as described above), or possibly trends in the temporal quality assessments of a series of imaging views. These options are described in more detail below.

[0089] Suitable machine learning algorithms for use in the present invention will be apparent to those skilled in the art. Examples of suitable machine learning algorithms include decision tree algorithms and artificial neural networks. Other machine learning algorithms such as logistic regression, support vector machines, or naive Bayes models are suitable alternatives.

[0090] The structure of artificial neural networks (or simply neural networks) has been inspired by the human brain. A neural network consists of layers, each layer containing multiple neurons. Each neuron contains a mathematical operation. In particular, each neuron may contain different weighted combinations of a single type of transformation (e.g., the same type of transformation, sigmoidal, etc., but with different weightings). In the process of processing input data, the mathematical operations of each neuron are performed on the input data to generate a numerical output, and the output of each layer in the neural network is fed sequentially to the next layer. The final layer provides the output.

[0091] Methods for training machine learning algorithms are well known. Typically, such methods include obtaining a training data set that includes training input data entries and corresponding training output data entries. An initialized machine learning algorithm is applied to each input data entry to generate a predicted output data entry. The error between the predicted output data entry and the corresponding training output data entry is used to correct the machine learning algorithm. This process can be repeated until the error converges and the predicted output data entry is sufficiently similar to the training output data entry (e.g., ±1%). This is commonly known as a supervised learning technique.

[0092] For example, if a machine learning algorithm is formed from a neural network, the mathematical operations (weights) of each neuron may be modified until the error converges. Known methods for modifying neural networks include gradient descent, backpropagation algorithms, etc.

[0093] The training input data entries in this case correspond to example ultrasound data sets for different image views or multiple image views. The training output data entries correspond, for example, to quality assessments of ultrasound data for observing one or more anatomical features or for determining one or more clinical parameters. They may correspond, in a further example, to a time trend of the quality assessments when the input data entries correspond to ultrasound data of a series of imaging views.

[0094] The machine learning algorithm is only one exemplary approach for implementing the evaluation procedure. Other possibilities include, for example, the use of non-machine learning algorithms, for example, using an algorithm programmed or coded in one or more processors to determine a quality assessment based on one or more image analysis procedures. According to one or more examples, a model-based segmentation algorithm can be used. For example, a segmentation algorithm can be used to identify or detect a particular anatomical feature of interest in each imaging view. The quality assessment of each imaging view can be based on the results of the segmentation algorithm, for example, based on the presence or absence of the anatomical feature of interest, or based on the degree to which the feature is completely contained within the imaging view.

[0095] In another example, a detection algorithm can be applied that is configured to identify and locate a particular anatomical feature, such as the center of the mitral valve or the septum, and to define a linear trajectory (directional vector) between the center of the current field of view and said feature. A quality assessment may be defined according to the length of the vector, for example proportional to the length of the vector. The further the feature is from the center of the field of view, the lower the quality assessment.

[0096] To illustrate the concepts of an embodiment of the present invention, an example workflow is shown generally in FIG.

[0097] In use, the ultrasound imaging probe 22 is placed at a first position 23a relative to a subject's body and ultrasound image data acquisition is initiated. For example, an ultrasound imaging system may be used to couple the ultrasound imaging probe 22 and configure it to control an ultrasound transducer of the probe to capture ultrasound data.

[0098] While in the first position, the ultrasound probe captures ultrasound data representative of a first imaging view 32a of a particular anatomical region. For purposes of illustration, the image view is shown as capturing a view of the left ventricle of the heart.

[0099] The assessment procedure 16 is operable to process the ultrasound data of the first imaging view 32a and determine a quality assessment or quality indicator. This is illustrated diagrammatically in Fig. 2 by an exemplary graph showing the quality assessment of the imaging views. The quality assessment of the first imaging view 32a is shown below the frame of the first image view.

[0100] As discussed above, the quality assessment indicates the suitability of the ultrasound data for that image view for observing one or more anatomical features or for deriving one or more clinical parameters from the data.

[0101] In the course of adjusting the probe position, the user moves the probe through a series of different positions 23a, 23b, 23c relative to the body. At each new position 23b, 23c, ultrasound data is acquired that represents an imaging view 32b, 32c that corresponds to the new position. The evaluation procedure is operable to determine a quality evaluation of the ultrasound data for each new image view.

[0102] As a result, a history 26 of data quality assessments for different ultrasound image views can be built. This is shown diagrammatically in the form of a plot or graph in Figure 2. In some examples, a quality assessment is explicitly determined for each imaging view, and a data set is built that stores a history of quality assessments for the imaging views over the course of the imaging procedure. In other examples, quality assessments of a series of image views are implicitly considered as part of the assessment procedure, and feedback is generated based on the history.

[0103] In either case, feedback is generated for the user based on the history of quality assessments for the image views over the course of the ultrasound scan. In some examples, a graph or plot of the time quality assessment history, such as that shown in Figure 2, may be output as feedback to the user. For example, the plot may be visually displayed on a display device.

[0104] As briefly described above, there are two main approaches to assessing captured image views to determine user feedback: one is based on assessing data views one at a time when the data for them are captured data views by imaging view approach, the other is based on assessing the whole set of image views, preferably temporally consecutive image views as a single block, and assessing the trend or change of quality assessment of the imaging views over time in the assessed sequence.

[0105] For example, in a first set of embodiments, an artificial intelligence (machine learning) module can be used and trained to estimate or evaluate incoming ultrasound data for different imaging views for each image view. The output evaluation can take the form of a qualitative evaluation of the view, such as, for example, "good" (quantification of a given parameter is likely to be successful), "medium" (quantification of a given parameter may or may not be accurate), or "poor" (quantification of a given parameter is unlikely to be successful). In other examples, the module can be configured to generate a quantitative evaluation, where this may be from a continuous scale or spectrum, or may be from a discrete set of quantitative evaluation values, for example, a scale of 1 to 5.

[0106] The quality assessment results of each imaging view may be stored in a data set (or "time vector"). This data set may be cumulatively built up one image view at a time as ultrasound data for each new image view is received. In an example, feedback may be provided to the user in the form of a sensory (e.g., visual) representation of at least a temporal portion of this data set. For example, the past n quality assessment results may be displayed to the user in an easy-to-understand manner. The user can then check based on this feedback whether the user is moving in the right direction (i.e., if the assessment is increasing, the user is moving in the right direction and vice versa). The number n of previous assessment results output to the user may be predefined or, in other examples, user adjustable. In this way, the results are displayed only for the most recent image views (the most recent time window) captured by the probe. This makes the results more useful for guiding ongoing or future movements of the probe, as observable trends in the results related to the most recent movement path. Thus, if the user is changing the movement path quite frequently, the displayed results can react to this and the user can be presented with quality results for the most recent movement path of the probe.

[0107] As an example, in a second set of embodiments, an evaluation procedure (e.g., a machine learning module) may be provided with ultrasound data for a set of n recent ultrasound image views as input. Based thereon, the evaluation procedure may generate an output indicating whether a user-provided probe movement is approaching an optimal position, e.g., based on a history of changes in image features. This may result in trend data being output. Feedback may in some examples be in the form of a simple suggestion or recommendation (e.g., text or audio) to the user based on trends in quality results.

[0108] These two approaches are now described in more detail, starting with the first approach.

[0109] According to a first set of embodiments, the assessment procedure is applied to the ultrasound data for each imaging view separately, i.e., the assessment procedure is applied to the data for each respective imaging view in a serial or sequential manner, with a respective quality assessment being generated for each imaging view in turn.

[0110] An exemplary workflow according to this set of embodiments is shown generally in FIG.

[0111] With the probe in a given (current) position relative to the subject's body, ultrasound data is captured by the probe 22 corresponding to that position, the ultrasound data representing a particular ultrasound imaging view 32a from that position. The ultrasound data from the currently captured view 32a is provided in real time to an evaluation procedure 16 (e.g., an evaluation module), which processes the data for that imaging view and derives a quality assessment indicative of the suitability of the ultrasound data for that imaging view 32a for observing one or more anatomical features or for deriving one or more clinical parameters from the data. For example, in the example shown in FIG. 3, the anatomical feature of interest is the left ventricle (LV). The evaluation procedure derives the quality assessment based on detection of the LV in the image. For example, the evaluation procedure determines the extent to which the LV is present in the ultrasound data for the imaging view 32a. To do this, the evaluation procedure can use a segmentation algorithm configured to detect at least the LV in the ultrasound data. Alternatively, a machine learning algorithm can be used and trained to recognize at least the LV in the ultrasound data and classify image views according to whether or to what extent the LV is present.

[0112] By way of example, in the example of FIG. 3, the current imaging view 32a has a left ventricle (LV) that is only partially visible, with a portion of the LV being occluded.

[0113] The quality assessment is output by the assessment procedure 16 for the current imaging view. This is then added to a dataset that stores a cumulative history 26 of image view quality assessments over time during the imaging procedure. This is shown in Figure 3 in the form of a graph displaying the assessment (y-axis) as a function of time (x-axis). However, the dataset can take the form of any data structure for storing the results.

[0114] The guidance feedback is generated for (e.g., sensory) output to the user based on the history of temporal quality assessment results 26. In the example of Fig. 3, the feedback takes the form of a sensory representation of the history of temporal quality assessment results. For example, the feedback takes the form of a visually displayed plot or graph of the history of quality assessment results 26. In some examples, results of the n most recent imaging frames may be output for feedback, as described above.

[0115] Based on the feedback, the user can see whether the current movement path of the probe is improving the quality results (thus moving them closer to the optimal imaging position) or degrading the results (thus moving them further away from the optimal position). Based on this, the user can adjust or adapt the movement of the probe to try and improve the quality results, moving it towards the optimal position.

[0116] In other words, the feedback presents the user with information showing the trend of the imaging view quality results over time (e.g., the trend of the n most recent results) so that the user can ascertain from this trend how the user should continue or modify the probe movement path from there.

[0117] For example, if view quality is trending upwards, the user can aggressively reinforce their actions and continue the same manipulation until a good view is reached. If view quality is instead trending downwards, the user can modify the probe manipulation until it starts to trend upwards.

[0118] At each new position, ultrasound data for that new image view is captured and provided to the evaluation procedure 16 for determination of a quality assessment of the image view. The method steps then continue cyclically until the user reaches an optimal imaging view 32c position where anatomical features are optimally visible or clinical parameters can be optimally derived based on the image position. The evaluation procedure in this example preferably uses one or more machine learning algorithms.

[0119] For example, the evaluation procedure in this example preferably employs the use of a Convolutional Neural Network (CNN).

[0120] The CNN may be trained, for example, with the training task of predicting a quality assessment of ultrasound data for each of a set of input imaging views (e.g., data for each imaging view corresponding to a single 2D or 3D ultrasound image frame).

[0121] For example, CNNs may, in some instances, take the form of a network for multi-class classification, which may allow, for example, classification into one of a set of distinct quality assessments (e.g., "bad," "medium," and "good"), which may be trained by minimizing cross-entropy loss.

[0122] Another approach could be a binary classification network such as a CNN. This network may be trained to classify images into two categories (e.g., "bad" - 0, "good" - 1), for example using cross-entropy loss. The predicted likelihood (between 0 and 1) may be output as a continuous quality score.

[0123] Another approach may be to train a network configured to regress quality scores, which may be classified according to one of a spectrum of quality ratings, which may be a continuous spectrum or a discrete spectrum (e.g., numbers 1 to 5). The network may be trained, for example, by minimizing a root-mean-square (RMS) error loss.

[0124] In either case, the network architecture may consist of a single convolutional layer, or multiple convolutional layers followed by spatial pooling (global, max pooling) and one or more fully connected layers with softmax (multi-class classification), sigmoid (binary classification), or linear (regression) activation functions in the final layer.

[0125] Each convolutional layer may include convolution operations, nonlinear activation, batch normalization, dropout, and spatial pooling. The implementation of each of these operations is well known and understood by those skilled in the art, and examples are known in the art (e.g., ResNet, DenseNet, AlexNet).

[0126] Ian Goodfellow, Yoshua Bengio, Aaron Courville's paper (2016, Deep Learning, MIT Press) provides, for example, details of suitable convolutional neural networks that may be implemented in accordance with embodiments of the present invention.

[0127] The network is preferably trained on a relatively large clinical dataset containing 2D or 3D echocardiographic images that are evenly distributed across classes (i.e., have evenly distributed quality scores). The dataset can, for example, contain a wide variety of different cases, patient sizes, and typical cardiac anomalies.

[0128] Each 2D or 3D ultrasound image is pre-labeled by an expert sonographer with a precise quality assessment (e.g., "poor," "fair," "good," or a quality score (1-5)) that indicates the confidence that the ultrasound data is suitable for observing one or more anatomical features of interest, or for deriving or quantifying one or more clinical parameters from the data. Some examples of clinical parameters that may be derived from the ultrasound data include, for example, the volume of one or more heart chambers, or the flow rate through one or more blood vessels.

[0129] 2D or 3D ultrasound images may be labeled by automatic algorithms such as model-based segmentation algorithms. For example, if a particular anatomical feature represented by a segmentation mask is present, the image may be labeled as "good" or reliable. If the segmentation mask is only partially present (60%), the image may be labeled as "moderate" or slightly quantifiable. The labeling schemes described herein are application specific and depend on the type of quantification performed. Also, the quality assessment may be a continuous assessment rather than a separate qualitative descriptor, as discussed in this example.

[0130] According to a second set of embodiments, instead of assessing each imaging view one at a time, the assessment procedure may be applied to ultrasound data for multiple ultrasound image views at once, the multiple imaging views being preferably consecutive in time. For example, the assessment procedure receives as input ultrasound data for a series of consecutive image views in time, e.g., n most recent image views. In some examples, it may generate as output an indication of a trend or change in image quality assessment over time, over the course of the series of images, or over at least a portion of the series of images, e.g., a recent time window.

[0131] An exemplary workflow according to this set of embodiments is shown generally in FIG.

[0132] In this example, the evaluation procedure 16 is configured to receive as input the ultrasound data of the n (or n+1) most recent image views. For example, at each new position of the probe as the user moves the probe, the system provides the ultrasound data of the new (current) imaging view 32n+1 to the evaluation procedure in addition to the ultrasound data of the previous n imaging views 32a-32n. The evaluation procedure is configured to process the ultrasound data for these series of recent image views and determine information indicative of a trend in the quality evaluation of the image views over time over the course of the series of n image views. For example, the evaluation procedure 16 evaluates whether the user-imposed probe movement is moving the probe closer to an optimal position based on a history of changes in the quality of the images or away from an optimal position.

[0133] The assessment procedure may be based on the detection or evaluation of one or more anatomical features in the data for each imaging view. For example, it may be based on the presence or absence of one or more anatomical features of interest. It may also be based on the degree to which one or more anatomical features are entirely present within an image view. For example, in the example shown in FIG. 4, the assessment procedure is based on the detection of the degree to which the left ventricle (LV) is present and centrally located within the image view.

[0134] The assessment procedure is configured to recognize from the ultrasound data for a sequence of image views the changing position of the LV across the image views to determine time trends or changes in the quality assessment, for example using an AI module that is trained to recognize this change in LV position.

[0135] The evaluation procedure is configured to generate feedback guidance 18 based on the evaluation of the series of images, the feedback guidance indicating suggested or suggested movements of the probe to improve the quality evaluation score of the ultrasound data, i.e., to move to a new position having a more optimal imaging view.

[0136] The feedback may take the form of, for example, continuing along the current motion path if the quality rating is increasing, or a recommendation to change the motion course if the quality rating is decreasing.

[0137] Each time, the last n image views provided to the evaluation procedure may correspond to a relatively short advance time window, such that the guidance feedback is related to the most recent movement direction of the probe. For example, n may be less than 10, such as less than 5. This means that the feedback can react quickly to changes in the movement of the ultrasound probe by the user and give recommendation feedback regarding their most recent movement path.

[0138] The number n may be predefined and / or user adjustable in some examples.

[0139] The feedback can guide the user to a probe position that includes the LV in a better or more acceptable location (eg, typically around the center of the image).

[0140] The approach according to this second set of embodiments can also take into account temporal features in the quality results that are not evident in a single still image alone. For example, the evaluation procedure can take into account cyclic variations of image features by taking as input ultrasound data for a series of image views. For example, many anatomical features exhibit cyclic motion (e.g., heart, lungs) and their suitability for deriving clinical parameters is high during one phase of their duration and low during another phase. By evaluating a series of image views at once, it becomes possible to take such cyclic features into account in order to avoid erroneously designating certain image views as low quality when in fact they are of high quality but vary periodically. In some cases, the system can suggest workarounds for poor quality views resulting from cardiac motion as part of the feedback.

[0141] For example, cardiac motion may cause the left ventricle (LV) to partially exit the ultrasound field of view (FOV) during end diastole (ED) while being completely contained in the FOV during end systole (ES). If no time trends are considered here, images may be alternately classified as "bad" and "good", respectively, even if the general probe position is satisfactory for extraction of clinical parameters or measurements related to the left ventricle, e.g., by a segmentation algorithm.

[0142] The evaluation procedure in this example preferably uses one or more machine learning algorithms, for example, the evaluation procedure in this example may use a convolutional neural network (CNN).

[0143] For example, the assessment procedure may apply a CNN that includes the same elements as outlined above with respect to the example of Figure 3 for the first set of embodiments. Alternatively, the assessment procedure may use a fully convolutional neural network (CNN) that is trained with the task of segmenting the LV in the ultrasound data for each of a plurality of ultrasound image sequences. Methods for performing such detection are known in the art.

[0144] The CNN may be trained to identify the LV in each individual imaging view of the input imaging sequence and to determine a quality assessment for each input image view separately. From this, a trend in temporal imaging view quality can be determined. Alternatively, the CNN may be trained to identify motion in the LV across the sequence of images based on consideration of the data set as a whole, or to determine a trend in quality assessment based on data for the entire sequence of images. Thus, in this case, an individual quality assessment is not determined for each imaging view, and the quality assessment for different views is merely implicitly considered.

[0145] In some examples, the shape and size of the LV mask applied to each imaging view can at least partially define the quality assessment of the imaging view(s). Alternatively, the feature embeddings generated by the decoder component of the U-Net architecture can be further used to regress the quality scores (e.g., multi-task learning (segmentation and classification)).

[0146] As an example, during the training phase of an AI procedure module (e.g., CNN), the procedure may be trained using multiple images (current live image + "n" previous images) that are input to the module via multiple channels.

[0147] Each such training group of n+1 image views is labeled (ground truth label) with a true indication of trends in quality assessment, or alternatively, relevant anatomical features, e.g., left ventricular (LV) motion trends, across the set of image views.

[0148] The ground truth labels can, for example, indicate the progression of relevant image features (in this case LVs) through multiple images and how they relate to, for example, what action the user should take.

[0149] Ground truth labels for each sequence of image views are pre-assigned by experts.

[0150] The evaluation module (e.g., AI module or network) is actually intended to emulate the analytical thinking or behavior of a human evaluating ultrasound images during probe manipulation. It is well known that humans incorporate temporal information into their decision making when evaluating US images. In this way, the feedback is more intuitively understandable for the operator.

[0151] According to this set of embodiments, various options are possible, both for the evaluation procedure and for the feedback to the user, which may be applied in addition to or as an alternative to any of the features mentioned above.

[0152] According to one or more examples, the evaluation procedure 16 can be provided with ultrasound data of the current (live) image view along with n previous image views as input and can be configured to determine and output a quality assessment of the current live image view. This can be, for example, a separate assessment of the form good / fair / poor. These quality assessments for each new "current" image view are cumulatively stored in a data set to build a history of image view quality assessments over time. This history of assessments can be presented (e.g., displayed) to the user as at least part of the feedback in some examples.

[0153] According to one or more examples, the evaluation procedure may be provided with ultrasound data for a current image view and n previous image views as input. The procedure may be configured to determine and output a quality assessment for the current live image view, as well as an indication of the trend of the quality assessment over the course of the n+1 image views (e.g., "improvement," "consistency," or "deterioration"). Both pieces of information may be presented to the user as part of the feedback. This may be presented, for example, in addition to a representation of the time quality assessment history 26.

[0154] According to one or more examples, the evaluation procedure may be provided as inputs of ultrasound data for a current image view and n previous image views. The evaluation procedure may be configured to determine a quality assessment for each of the imaging views. These may be stored in an array or data set. The evaluation procedure may provide this set of quality assessments for the image views as output. The evaluation procedure or feedback module may determine a trend in the temporal quality assessments over the course of the imaging views based on the array of quality assessments derived for each individual imaging view. Information indicative of this trend may be displayed to the user as at least part of the feedback.

[0155] As each subsequent image is acquired, the evaluation procedure may again derive a quality assessment for each imaging view of the updated set of n+1 imaging views. In some examples, a temporal filter may be applied to ensure consistency of quality assessment decisions for n overlapping frames in a temporally consecutive set of n+1 image views. This provides more stable feedback to the user over time.

[0156] According to one or more examples, instead of the input to the evaluation procedure 16 being the data of the n most recent imaging frames, a different set m of previous images can be provided. For example, a longer time history can be taken into account without increasing the computational requirements by inputting every other image view across the previous 2n image views, so that every mth image view is provided to the evaluation procedure. In further examples, any other pattern or selection of image views can alternatively be used.

[0157] According to one or more examples, the system may be configured to detect when an optimal or near-optimal image view has been reached and communicate this to the user as at least part of the feedback.

[0158] In particular, if the user is already close to the optimal image view, moving in any direction will tend to "degrade." Such a response in the historical data or other feedback should indicate to the user that they should stop attempting to adjust the probe. However, this can also be automatically flagged by the system and conveyed to the user by relevant feedback, e.g., a recommendation ("Undo the most recent operation" / "Return to previous view," etc.).

[0159] 5 and 6 show schematic diagrams of exemplary workflows during two exemplary implementations of the method according to the second set of embodiments described above. In the first exemplary workflow (FIG. 5), the user starts in a suboptimal position and slowly improves the quality of the image view. In the second example (FIG. 6), the user starts in a reasonable position but inadvertently worsens the imaging view quality before correcting based on feedback.

[0160] More specifically, and referring to Fig. 5, the probe 22 begins in a suboptimal position (view 32a). By way of example, a quality rating for each imaging view is generated based on the extent to which the anatomical features of the left ventricle (LV) are present and clearly visible in the imaging view. Fig. 5 is a graph showing a history 26 of quality ratings (y-axis) for a series of imaging views 32a-32f as a function of time (x-axis). A schematic diagram of each of the imaging views 32 is shown aligned above its respective rating on the graph.

[0161] For the first imaging view 32a, the image view has a low quality rating.

[0162] The user initiates repositioning of the probe while multiple images (the current live image and 'n' previous images) are continuously fed to an evaluation procedure 16 (eg, an AI module).

[0163] When the evaluation procedure recognizes a pattern progression in the images (ie, an upward or downward trend in the quality ratings), the evaluation procedure outputs feedback to the user in the form of recommendations regarding the currently executing probe steering.

[0164] In this example, the LV is detected to be moving more centrally in the image as the probe motion continues, resulting in an increased quality rating. Therefore, the recommendation is to "continue" the current probe operation.

[0165] As the user continues the current probe operation, the view quality further increases (i.e., the LV continues to move towards the center of the FOV), so the same output is maintained by the AI ​​module.

[0166] Eventually, the user reaches a good LV central view (imaging view 32f), at which point the evaluation procedure suggests to the user to stop further probe manipulation as a good view is achieved. For example, the evaluation procedure may determine that the rate of increase in imaging view quality as a function of time is slowing, which may be taken as an indication that an optimal view is being approached or is near.

[0167] In another example, referring to FIG. 6, the probe starts with imaging view 32a having a reasonably good ("medium") quality rating. In an attempt to improve the view, the user steers the probe. However, in this example, the movement moves the probe to a lower quality imaging view, resulting in imaging view 32c with a very low quality rating. The evaluation procedure detects a downward trend in the quality rating and generates feedback in the form of a recommendation to change the course of movement of the probe. The user then begins to modify the probe manipulation. This results in a slow improvement in the quality rating of the image view (e.g., the LV begins to move back into the FOV), with imaging view 32e being of medium quality and imaging view 32f eventually being of high quality. The evaluation procedure again detects this upward trend in the quality rating and generates feedback recommending that the user stop moving when the quality rating is high or the image is detected to be close to an optimal position.

[0168] Note that in both the examples of Figures 5 and 6, cardiac motion is present so the progression or motion of the left ventricle over time will not be as smooth as shown in the figures. In particular, there are typically two different motion frequencies exhibited in the motion: the cardiac (periodic) frequency and the actual probe motion frequency.

[0169] Furthermore, while the examples of Figs. 5 and 6 show varying image view quality with respect to the movement of an anatomical feature (in this case, the LV) toward or away from the center of the image view, it should be noted that more generally image view quality may relate to other factors. In particular, the degree to which the anatomical body of interest is present and clearly visible, e.g., blurred or sharply depicted. This is illustrated diagrammatically in Fig. 7 with an example of image view image quality assessment, showing a set of image views with respective labels. Image view (a) shows, for example, the left ventricle, which is centered and sharply depicted. It has a high quality score rating of 0.99. Image view (b) shows a less central location of the left ventricle, but still sharply depicted. It has a slightly lower quality score rating of 0.95. Image view (c) shows a centrally located left ventricle, but blurred or faded. It has a relatively low quality score rating of 0.40. Image view (d) shows a key portion of the heart (the left ventricle) that was missing from the captured ultrasound image data. Here, the image view rating is the lowest among the four example image views, with a rating score of 0.30.

[0170] According to any of the above-mentioned embodiments, it may be optimal to move at different speeds depending on the quality assessment of the image view that the user is currently generating. For example, if the user is currently in a position corresponding to an image view with a low quality assessment and is moving the probe in an attempt to improve the quality, a relatively fast movement may be acceptable. In contrast, if the user is attempting to reach an image view with a relatively high quality and achieve an optimal image view, a slower movement may be preferred to avoid overshooting the optimal position.

[0171] To address this, according to one or more examples, the system may be configured to output steering advice as part of the feedback to the user. This may be in addition to, for example, information about the current quality rating of the current image view, and / or information about the history of image view ratings over time, and / or information about trends in image view ratings. For example, the steering advice may be to move faster if the quality is poor, to move slower if the quality is medium, to maintain a constant speed if the quality is high, or to move slower at a faster speed. The system may use a simple lookup table to achieve this, i.e., move fast from poor quality, slow down from medium quality, and move steadily from good quality.

[0172] Additionally or alternatively, a motion detection algorithm may be provided to run in the background. Ultrasound-based motion detection algorithms are known in the art and include, for example, speckle tracking algorithms, which allow for detecting the speed of movement based on the speed at which speckle patterns in the ultrasound image data are detected to be moving. This algorithm can assess the magnitude of the movement and alert the user if the maneuver pace exceeds predefined thresholds or ranges for the current image view quality, which thresholds or ranges correspond to predefined optimal speeds for different image view qualities.

[0173] In accordance with one or more advantageous embodiments, the method according to any of the above embodiments can be implemented as part of a personalized learning tool. For example, for a particular user, the system can memorize patterns of probe movement during an imaging procedure and then provide specific recommendations based on how the user manipulated the probe (e.g., adjusted the probe position most efficiently, achieved an optimal imaging view). The user can then review these patterns offline, potentially learning their "preferred" manipulations and optimizing them to reach the desired view in the most efficient way possible.

[0174] Although the particular example above proposes using machine learning algorithms to perform the procedure, in further embodiments the procedure may use non-machine learning techniques.

[0175] As an example, a model-based segmentation (MBS) algorithm may be used according to one or more examples. For example, the segmentation algorithm may be used to identify or detect a particular anatomical feature of interest within each imaging view. A quality assessment of each imaging view may be based on the results of the segmentation algorithm, for example, based on the presence or absence of the anatomical feature of interest, or based on the extent to which the feature is completely contained within the imaging view.

[0176] In some examples, the quality assessment may indicate or relate to the degree to which an image view contains a particular anatomical feature. For example, if only half of the anatomical features of interest are detected as being present in the image by a segmentation (or other) algorithm, the quality assessment may be 50%, while if none of the features are present, the quality assessment may be 0%.

[0177] For example, the assessment may be related to a proportion of the anatomical feature or region of interest that is included in the image view.An example according to a further aspect of the present invention provides a computer program product comprising code means configured to, when executed on a processor, cause the processor to perform a method according to any of the embodiments or examples outlined above or described herein, or alternatively according to any claim of the present application.

[0178] The processor in use may be communicatively coupled to the ultrasound probe, or alternatively, the processor may be arranged to receive ultrasound data from different sources, for example from the ultrasound imaging system, from a central server, or in some examples from a memory store.

[0179] An example according to a further aspect of the present invention also provides a system for guiding a user of an ultrasound imaging system during ultrasound imaging of a subject's body, the ultrasound imaging system including an ultrasound probe having a movable position relative to the subject's body.

[0180] An exemplary system 90 is shown generally in FIG.

[0181] The system is shown in use communicatively coupled to an ultrasound imaging probe 22 for receiving ultrasound image data. In other examples, the system may be communicatively or operatively coupled to an ultrasound imaging system, such as a diagnostic ultrasound system. An example according to a further aspect provides an ultrasound imaging system including the ultrasound probe 22 and including a guidance system 90.

[0182] Returning to the guidance system 90 of FIG. 8, the system includes a receiving module 92 adapted to receive ultrasound data representing a plurality of ultrasound image views during an ultrasound imaging procedure, each view corresponding to data captured from a respective position of the probe and at a respective time during the procedure.

[0183] The system 90 further includes an evaluation module 94 configured to apply a procedure to the ultrasound data that is operable to evaluate a quality of the ultrasound image views, a quality related to the suitability of the ultrasound data for different imaging views for observing one or more anatomical features, or for deriving one or more clinical parameters from the data.

[0184] The system further includes a feedback module 96 configured to generate guidance feedback to assist a user in positioning the probe based on the evaluation procedure, the guidance feedback being based on a history of image view evaluations over time during the imaging procedure.

[0185] 8 is shown operatively coupled to a display unit 102 for outputting to display visual guide feedback for the user. This is just one example. In other examples, the feedback unit may generate different forms of sensory feedback, such as auditory feedback, or even haptic feedback in some examples, such as vibrations of an intensity or frequency or vibration pattern configured to represent a history of the quality assessment over time.

[0186] The implementation options and details for each of the above system features can be understood and interpreted in accordance with the descriptions and explanations provided above with respect to the method aspects of the present invention.

[0187] Any of the features or details of the examples, options or embodiments described above with respect to the guidance method may be applied to, combined with or incorporated in the present system aspects of the present invention.

[0188] Although the receiving module 92, the evaluation module 94, and the feedback module 96 are shown and described as separate components in the above-described exemplary system 90, this is not required. Their associated functionality may be distributed in different manners among one or more components. For example, in some examples, the functionality of the different modules may be integrated and performed by a single element, e.g., a single controller or processor, or alternatively, the functionality may be distributed in other manners among one or more elements or components. Alternatively, the functionality of each module may be performed by a separate controller or processor within the system 90.

[0189] Embodiments of the invention may find particularly advantageous application in transesophageal echocardiography (TEE), e.g., for purposes of TEE monitoring over extended and / or continuous periods. For example, the ultrasound probe may be a transesophageal echocardiography (TEE) probe.

[0190] However, TEE represents just one exemplary application area, and embodiments of the invention may be advantageously applied to any other type of ultrasound examination. The embodiments may be particularly advantageous for long-term or continuous or ongoing ultrasound examinations, i.e. ultrasound monitoring applications, where the position of the ultrasound probe may drift in time and may need to be corrected iteratively. The guidance system according to embodiments of the invention allows even non-expert operators to accurately and quickly adjust the position of the probe.

[0191] As mentioned above, the embodiments may utilize a processor to perform data processing. The processor may be implemented in various ways using software and / or hardware to perform the various functions required. The processor typically uses one or more microprocessors that can be programmed using software (e.g., microcode) to perform the necessary functions. The processor may be implemented as a combination of dedicated hardware to perform some functions and one or more programmed microprocessors and associated circuitry to perform other functions.

[0192] Examples of circuitry that may be employed in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs).

[0193] In various implementations, the processor may be associated with one or more storage media, such as volatile and non-volatile computer memory, such as RAM, PROM, EPROM, and EEPROM. The storage media may be encoded with one or more programs that, when executed on the one or more processors and / or controllers, perform the necessary functions.

[0194] The various storage media may be fixed within the processor or controller, or may be transportable such that one or more programs stored thereon can be loaded into the processor. Variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality.

[0195] A single processor or other unit may fulfill the functions of several items (eg, several modules) recited in the claims.

[0196] The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0197] The computer program may be stored / distributed on a suitable medium, such as an optical storage medium or a solid-state medium, supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems.

[0198] It is noted that when the term "adapted" is used in the claims or description, it is intended to be equivalent to the term "configured."

[0199] Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. 1. A computer-implemented method executed by an ultrasound guidance system for guiding a user of an ultrasound system during ultrasound imaging of a subject's body, the method comprising: receiving, at respective times during an imaging procedure, ultrasound data representing a series of ultrasound imaging views, each view corresponding to data obtained from a respective position of an ultrasound probe; applying an evaluation procedure to the ultrasound data, the evaluation procedure being operable to evaluate a quality of the ultrasound imaging views, the quality relating to a suitability indicative of whether the ultrasound data is suitable for observing one or more anatomical features in the different ultrasound imaging views or for deriving one or more clinical parameters from the data; generating guidance feedback to assist a user in positioning the ultrasound probe based on the evaluation procedure, the guidance feedback being based on a history showing trends of changes in quality of ultrasound imaging views over time during the imaging procedure; The method comprising:

2. 2. The method of claim 1, comprising determining a quality assessment for each of the ultrasound imaging views, the quality assessment indicating a suitability of the ultrasound data for observing one or more anatomical features in the ultrasound imaging view or for deriving one or more clinical parameters from the data.

3. The method of claim 1 , further comprising determining a trend in a quality of the imaging views over time, and wherein the guidance feedback is based on the trend.

4. 4. The method according to claim 1 , further comprising: building a historical data set based on the evaluation procedure, the data set representing a trend of changes in quality of the ultrasound imaging views over time during the imaging procedure.

5. The method of claim 1 , wherein the guidance feedback is visual feedback indicative of the history of quality of imaging views as a function of time.

6. the guidance feedback includes feedback indicating a suggested movement of a position of the ultrasound probe to acquire ultrasound data of improved quality; The method of claim 1 , wherein the guidance feedback comprises a recommendation to continue a current path of movement of the ultrasound probe or a recommendation to abort a current path of movement of the ultrasound probe.

7. 7. The method of claim 1, wherein the assessment procedure is applied sequentially to the ultrasound data for each imaging view and sequentially to each quality assessment generated for each imaging view.

8. The method of claim 1 , wherein the evaluation procedure is applied collectively to ultrasound data for a plurality of ultrasound imaging views.

9. the imaging views are contiguous in time; and / or The method of claim 8 , wherein the multiple imaging views are ordered in time.

10. said evaluation procedure utilizing a machine learning algorithm; The evaluation procedure comprises the use of a convolutional neural network.

10. The method according to any one of claims 1 to 9.

11. 11. The method of claim 1, wherein the evaluation procedure comprises identifying or detecting the presence of one or more anatomical features in the ultrasound data for the different imaging views, and the quality assessment is based on the detection.

12. A computer program product comprising code means configured, when executed on a processor, to cause said processor to carry out a method according to any one of claims 1 to 11.

13. 1. A guidance system for guiding a user of an ultrasound system during ultrasound imaging of a subject's body, comprising: The guidance system comprises: a receive module configured to receive, at respective points during an imaging procedure, ultrasound data representing a plurality of ultrasound imaging views, each view corresponding to data obtained from a respective position of an ultrasound probe; an evaluation module configured to apply an evaluation procedure to the ultrasound data, the evaluation procedure evaluating a quality of the ultrasound imaging views, the quality relating to a suitability of the ultrasound data indicative of whether it is suitable for observing one or more anatomical features in the different ultrasound imaging views or for deriving one or more clinical parameters from the data; a feedback module configured to generate guidance feedback to assist a user in positioning the ultrasound probe based on the evaluation procedure, the guidance feedback being based on a history showing trends of changes in quality of ultrasound imaging views over time during the imaging procedure; A guidance system comprising:

14. an ultrasound probe having one or more ultrasound transducers for acquiring ultrasound data; 14. The guidance system of claim 13, communicatively coupled to the receiving module of the guidance system for transferring acquired ultrasound data; 1. An ultrasound system comprising:

15. 15. The ultrasound system of claim 14, wherein the ultrasound probe is a transesophageal echocardiography probe.

Images