Ultrasonic imaging in distributed systems

Abstract

A computer-implemented method is provided for facilitating collection of ultrasound imaging data over an extended period of time with improved consistency of views of captured images. A first ultrasound imaging data set is collected and stored in a data store, the first data set including one or more images representing one or more views of a patient's anatomy. The image data is used as reference data for subsequent image collection. Subsequent image data is compared in real time to the reference image data to obtain a measure of agreement between the depicted views, which can be used to guide the collection of new image data to be consistent with previous image data. For example, an alert message can be issued to a user when a view represented in the newly collected image data sufficiently matches a view of the reference data.

Description

[Technical field]

[0001] The present invention relates to a method for ultrasound imaging in a distributed system, and in particular to a method for supporting longitudinal ultrasound imaging of the same patient. [Background technology]

[0002] Ultrasound imaging can be used to evaluate patients in a variety of clinical settings (eg, in intensive care units, operating rooms, general wards, etc.).

[0003] Ultrasound image acquisition is performed on patients at multiple time points, either during a single short-term monitoring period (e.g., during surgery) or over longer periods. In this context, it is important to collect consistent image views of the anatomical structures of interest to enable reliable clinical comparisons over time.

[0004] As an illustration, two ultrasound imaging modes used to evaluate patients with ultrasound are transthoracic echocardiography (TTE) and transesophageal echocardiography (TEE). TTE has a wider range of use than TEE because it is non-invasive. TTE can be used outside of emergency settings. TTE also poses less clinical risk to the patient (compared to TEE) and can be performed by operators with lower skill levels due to its non-invasive nature. However, it is more difficult to gather the correct view of a given anatomical structure using TTE than TEE because the operator must precisely place the probe in the correct location in the chest (compared to TEE where probe placement is greatly limited by the esophagus).

[0005] Ultrasound imaging can be used for cardiac function monitoring during surgery. For example, monitoring cardiac function using TEE is becoming increasingly popular in clinical settings, especially when other modes such as TTE are difficult to perform due to patient habits, chest trauma, other external medical devices, or patient dressings. For example, to view the left ventricle, the TEE probe is navigated to a mid-esophageal position. If the clinician requires a view of the heart at any time during surgery, the probe is manually manipulated to collect the desired view.

[0006] The use of ultrasound for continuous monitoring of the heart (e.g., in intensive care settings or during surgery) has been proposed in the medical community and is an area of ​​current technological development. Ultrasound imaging in this context is useful not only to obtain and view images of the patient's anatomy (e.g., the dynamic motion of the heart), but also to obtain quantitative measurements of physiological parameters (hemodynamic parameters such as left ventricular (LV) volume, stroke volume (SV), cardiac output (CO), and left ventricular ejection fraction (LVEF)). Trend patterns of these parameters can be tracked over time to inform the patient's treatment.

[0007] For example, it is known to apply model-based segmentation to acquired ultrasound images to obtain quantitative measurements. For example, automated segmentation of one or more chambers of the heart can be performed. Measurements of hemodynamic parameters are obtained by applying model-based segmentation to an end-diastolic (ED) ultrasound image frame to detect chamber boundaries, followed by 2D or 3D tracking of the segmentation boundaries across other frames of the cardiac cycle.

[0008] In 2D imaging, geometric assumptions can be made to extrapolate from collected single-plane or bi-plane information to generate measurements of 3D objects or features (such as cardiac chamber volumes. Example algorithms include modified Simpson's method, an ellipsoid model using single-plane, bi-plane, or one-dimensional data, or a hemisphere-cylinder model based on bi-plane data.

[0009] For more details on these segmentation methods see: Kosaraju A, Makaryus AN. Left Ventricular Ejection Fraction. In: StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing; 2019 Jan-. Available at: https: / / www.ncbi.nlm.nih.gov / books / NBK459131 / . Summary of the Invention [Problem to be solved by the invention]

[0010] Long-term ultrasound monitoring of a patient over a continuous period of time (e.g., several hours) can be difficult because the probe may become misaligned over time. In an intensive care environment, measurements may be taken by multiple clinicians or specialists at different times. Measurements may also be desired when a cardiologist, intensivist, or other specialist is not present. Furthermore, for example, TTE imaging presents its own challenges since the same correct view must be obtained from scratch every time.

[0011] Furthermore, when generating long-term trends for a given measurement (such as SV in the case of cardiac imaging), it is important to ensure the consistency of the image data provided to the measurement tool. For example, if the image data provided varies with the particular view depicted, it may impede the interpretation of long-term trends due to measurement errors associated with inconsistent image acquisition. This may result in inappropriate additions or changes to the medical intervention provided to the patient (e.g., changes in fluid status, vasopressors, inotropes, etc.). This is especially true in 2D imaging approaches where the possibility of foreshortening (the acquired ultrasound plane does not pass through the true apex of the left ventricle) is a significant challenge.

[0012] It would be advantageous to provide a solution that allows for more consistent image data to be collected over time. [Means for solving the problem]

[0013] The invention is defined by the claims.

[0014] According to an embodiment of the present invention, a computer-implemented method for execution in a distributed computing system is provided, the method comprising: acquiring, at a first ultrasound imaging device, first ultrasound imaging data of a subject, the first ultrasound imaging data including at least one image depicting at least one view of an anatomical structure of the subject; storing the first ultrasound imaging data in a data store as reference ultrasound image data tagged with the subject identifier information; obtaining, at a second ultrasound imaging device, reference ultrasound image data from the data store based on querying the data store with the patient identifier information, the second ultrasound imaging device being the same or different from the first ultrasound imaging device; acquiring live ultrasound image data at a second ultrasound imaging device; iteratively generating a live similarity measure indicative of a similarity between a view represented in the reference ultrasound image data and a view represented in the live ultrasound image data; generating a user output alert when the live similarity measure matches at least one predefined criterion; capturing and storing one or more frames of live ultrasound image data tagged with patient identifier information; Includes.

[0015] Thus, the method provides a means to assist in maintaining a consistent ultrasound imaging acquisition view over time, even when acquisitions are made by different operators with different skill levels. This objective is achieved by storing a first set of image views that can be accessed by another user at a later time and used as a reference for acquisition of subsequent image data of the same patient. Through active comparison of the reference image data with the newly acquired live image data and generation of user guidance alerts based thereon, the consistency of the acquired views at the two time points is ensured. Thus, the operator acquiring the second image data is guided to acquire image data that matches the first image data in terms of the view being captured. This concept can be reproduced for any number of image acquisition events or periods during a patient's stay at a medical institution, or even across multiple stays at multiple facilities or care groups. Each time ultrasound image data is acquired, at any of the multiple ultrasound imaging devices that make up the overall system, the previous reference image data is retrieved from a centrally accessible data source and used as a reference guide for the acquisition of new image data that corresponds to the previous data in terms of the view being captured.

[0016] By "view" is meant a particular viewing angle and field of view (acquisition window) of images relative to the anatomy imaged. For a particular anatomy, there is a set of standard imaging views that are typically acquired. For example, in cardiac imaging, this includes parasternal long and short axis views, apical four-chamber views, and subxiphoid views. In general, the imaging views will correspond to a particular positioning of the probe relative to the subject and to the anatomy being imaged. For example, in cardiac imaging, a parasternal view is obtained with the probe placed just to the left of the sternum in the third or fourth intercostal space. An apical four-chamber view of the heart is obtained with the probe placed just below the nipple line at the heart's maximum beat point. A subxiphoid four-chamber view is obtained with the probe pointed toward the left shoulder from a position just below the tip of the xiphoid process of the sternum.

[0017] As an example, the first ultrasound image data and the live ultrasound image data may each be processed with image segmentation to detect anatomical structures within an image field of each image data set to determine a similarity measure. For example, a relevant image is selected from the first ultrasound image data, e.g., preselected by a first operator. This image is processed with image segmentation. A live image frame from the live ultrasound image data is also processed with image segmentation. The similarity measure may be derived by comparing the segmentation results of the first image and the live image frame. For example, the similarity measure may be derived by evaluating the degree of spatial overlap of one or more anatomical structures (e.g., template matching or object matching) or by comparing the binary presence or absence of one or more anatomical structures in each of the first image and the live image frame.

[0018] In some embodiments, the method includes continuously or iteratively generating a similarity measure and capturing an image frame when the similarity measure exceeds a predefined threshold.

[0019] When generating a user alert based on the similarity measure to guide image collection, the predefined criteria include a predefined threshold value of the similarity measure.

[0020] In some embodiments, the data store holding the reference image data constitutes a patient monitoring subsystem. Thus, the reference ultrasound image data is obtained from a data store that constitutes the patient monitoring subsystem. The patient monitoring subsystem may also receive additional patient measurement data, including vital signs data.

[0021] In some embodiments, at least one of the live ultrasound imaging data and the reference ultrasound image data is collected using a transesophageal echocardiography (TEE) ultrasound probe or a transthoracic echocardiography (TTE) probe.

[0022] In some embodiments, the method further comprises capturing and storing at least one frame of the live ultrasound image data only if the similarity measure satisfies at least one predefined criterion, i.e., the capture of new second image data is constrained to only capturing image views that sufficiently match the image views represented in the reference image.

[0023] In some embodiments, the method includes capturing and storing at least one frame of live image data only if the similarity measure meets a predefined criterion and in response to a capture command from a user input device. That is, the user actively controls when image frames are captured for storage, but the time period during which the user can do so is constrained to the time period during which the real-time generated similarity measure meets a predefined threshold. In this way, a certain level of view conformance is enforced by the system.

[0024] In some embodiments, the data store stores a set of reference images depicting different views of the anatomy, each tagged according to the view depicted, and in some more specific examples, the data store further stores view recommendations indicating a recommended next view of the anatomy to capture, and the captured reference image is one of the set of images depicting the recommended next view.

[0025] First ultrasound imaging data of the subject is acquired at a first time point t1. The step of acquiring reference ultrasound image data from the data store in the second ultrasound imaging device includes acquiring reference ultrasound image data at a subsequent time point t n It is held.

[0026] In some embodiments, in addition to collecting the image data, the first user collecting the image data may also input one or more imaging recommendations at the first imaging device. Each imaging recommendation indicates one or more recommended views or images to be collected at a subsequent time point. The recommendations may include recommendations for one or more future time points t n may be specified, or the recommendation may specify a more general future time point, such as the next image acquisition event, or no time point may be specified.

[0027] In some embodiments, the method includes storing the one or more view recommendations in the data store along with the acquired first image data, in addition to storing the first ultrasound imaging data as reference ultrasound image data in the data store, e.g., for storage in a data record associated with the patient, e.g., the image data and the recommendations are each tagged with the same patient identifier information.

[0028] In some embodiments, at the second ultrasound imaging device, obtaining reference ultrasound image data from a data store further includes obtaining one or more view recommendations.

[0029] As described above, in some embodiments, the method includes using one or more view recommendations to determine which of a plurality of different image views stored in a data store associated with the patient are to be transferred to a second ultrasound imaging device.

[0030] For example, the data store stores a set of reference images depicting different views of the anatomical structure, each tagged according to the view depicted, and the data store further stores the aforementioned view recommendation indicating a recommended next view of the anatomical structure to capture, and the reference image acquired and transmitted to the second ultrasound imaging device is one of the set of images depicting the recommended next view.

[0031] Additionally, in one or more embodiments, a first user acquiring the first ultrasound image data inputs at the first ultrasound imaging device a selection of the acquired first ultrasound image to be subsequently used as a reference image for subsequent image data acquisition, and this information is transferred to the data store for storage. This information can be input by the first user at the first ultrasound imaging device or can be input at one or more additional devices or nodes of the system (by the same first user or one or more additional users).

[0032] In some embodiments, a first user collecting the first ultrasound image data and / or another user of the system inputs a recommendation for a time when subsequent ultrasound image data should be collected. The method includes receiving the recommendation, for example, via a user interface. The user interface may be, for example, a user interface of a patient monitoring subsystem. The user interface may be, for example, a user interface of another device, such as a mobile computing device. The method includes receiving an input command signal at the user interface, and a future ultrasound imaging event is scheduled, commanded, or recommended, including a specific time for the ultrasound imaging collection, based on the command signal.

[0033] In some embodiments, the method further includes generating probe positioning guidance based on the similarity metric, the probe position guidance for guiding a user's movement of the probe to improve the similarity metric, and outputting the guidance information to a user interface device, such that in this case the user is not only alerted when a view sufficiently corresponds with the reference image view, but is also provided with active guidance to move the positioning of the probe relative to the patient to approach the target image view.

[0034] In some embodiments, the method further includes receiving, at the second ultrasound imaging device, the annotation data for storage in the data store with the reference image data.

[0035] In some embodiments, the method includes processing at least one captured frame of live image data with an anatomical measurement algorithm to derive one or more anatomical and / or physiological measurements. For example, in cardiac imaging, one or more hemodynamic parameters are generated. Additionally or alternatively, one or more measurement algorithms may be applied to the first ultrasound imaging data. In either case, the collected measurement data is stored in a data store along with the patient's image data.

[0036] A further aspect of the present invention provides a processing device, the processing device including an input / output unit and one or more processors, the one or more processors being: obtaining, at the input / output, from a data store, reference ultrasound image data depicting at least one view of an anatomical structure of the subject based on querying the data store with the patient identifier information; The input / output unit receives live ultrasound image data, iteratively generating a live similarity measure indicative of a similarity between a view represented in the reference ultrasound image data and a view represented in the live ultrasound image data; generating a user output alert when the live similarity measure matches at least one predefined criterion; One or more frames of live ultrasound image data are captured and stored tagged with patient identifier information.

[0037] A control signal is provided to the input / output for controlling the user interface to generate a user output alert.

[0038] Another aspect of the present invention provides an ultrasound imaging device including a processing device according to any example or embodiment outlined in this disclosure or as described in any claim of this application, and an ultrasound imaging probe for collecting live ultrasound imaging data.

[0039] Another aspect of the present invention provides a system including an ultrasound imaging device according to any example or embodiment described in this disclosure or as set forth in any claim of this application, and a data store for storing reference ultrasound image data.

[0040] Another aspect of the present invention provides a computer program product including code means for execution by a processor, the code means for a processor communicatively linked to an ultrasound imaging device and communicatively linked to a data store, the code means causing the processor to: receiving, from a data store, reference ultrasound image data depicting at least one view of an anatomical structure of the subject based on querying the data store with the patient identifier information; receiving live ultrasound image data from an ultrasound imaging device; iteratively generating a live similarity measure indicative of a similarity between a view represented in the reference ultrasound image data and a view represented in the live ultrasound image data; generating a user output alert when the live similarity measure matches at least one predefined criterion; capturing and storing one or more frames of live ultrasound image data tagged with patient identifier information; Have them carry out the procedure.

[0041] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. [Brief description of the drawings]

[0042] For a better understanding of the present invention and to show more clearly how it may be carried into effect, reference will now be made, by way of example only, to the accompanying drawings in which:

[0043] [Figure 1] FIG. 1 illustrates an example system and data flow in accordance with one or more embodiments. [Diagram 2] FIG. 2 outlines in greater detail an exemplary system and data flow in accordance with one or more embodiments. [Diagram 3] FIG. 3 illustrates a display of any exemplary patient monitoring device that may comprise a system in accordance with certain embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0044] The present invention will now be described with reference to the drawings.

[0045] It should be understood that the detailed description and specific examples, while indicating exemplary embodiments of the devices, systems, and methods, are for purposes of illustration only and are not intended to limit the scope of the invention. These and other features, aspects, and advantages of the devices, systems, and methods of the present invention will become better understood from the following description, the appended claims, and the accompanying drawings. It should be understood that the figures are schematic representations only and are not drawn to scale. It should also be understood that the same reference numerals are used throughout the figures to indicate the same or similar parts.

[0046] The present invention provides a computer-implemented method for facilitating longitudinal collection of ultrasound image data that is consistent with the views of captured images. A first ultrasound imaging data set is collected and stored in a data store, the first data set including one or more images representing one or more views of a patient's anatomy. The image data is used as reference data for subsequent image collections. Subsequent image data is compared in real-time with the reference image data to obtain a measure of correspondence between the depicted views, which can be used to guide the collection of new image data to be consistent with previous image data. For example, an alert message can be issued to a user when a view represented in the newly collected image data sufficiently matches a view of the reference data.

[0047] FIG. 1 illustrates a schematic of components of an exemplary system according to one embodiment of the present invention.

[0048] The system includes one or more ultrasound imaging devices 14, 22. In this example, two ultrasound imaging devices 14, 22 are shown, however other embodiment options include more than two or only one. The system also includes a data store 18 for storing ultrasound imaging data for multiple patients.

[0049] The system is a distributed computing system, i.e., different components or nodes of the system are physically separate from one another but are functionally connected in a network for joint data collection, analysis, storage, and user presentation.

[0050] The system is for implementing a computer-implemented method for supporting longitudinal collection of ultrasound imaging data with consistent views of acquired ultrasound images, which method forms a further aspect of the invention. Exemplary computer-implemented method steps according to one or more embodiments of the invention are outlined in more detail below with reference to the system components of FIG.

[0051] The exemplary method includes acquiring ultrasound imaging data 16 of a subject 12 at a first ultrasound imaging device 14. The ultrasound imaging data 16 includes at least one image depicting at least one view of an anatomical structure of the subject.

[0052] The method further includes storing the first ultrasound imaging data 16, tagged with the subject identifier information, in a data store 18 as reference image data. The data store, in some embodiments, is a data store constituting a patient monitoring subsystem, whereby the patient monitoring subsystem acts as a central hub of the system. At the same time, the patient monitoring system may receive biological sensor data about the patient 12, which is displayed on a display of a user interface. The display of the user interface may also be controlled to display the collected first ultrasound imaging data. In some examples, the patient monitoring subsystem, or a separate processing component connected to the patient monitoring subsystem, processes the ultrasound imaging data to derive therefrom one or more quantitative anatomical and / or physiological measurements, such as hemodynamic measurements. The data store may instead be located elsewhere. The data store may be a separate dedicated data storage node, such as a storage server centrally located in a medical institution. The data store may be a cloud-based data storage means accessible via an internet connection.

[0053] The method further includes retrieving, at the second ultrasound imaging device 22, reference ultrasound image data from the data store 18 based on querying the data store with the patient identifier information. For example, an operator of the second ultrasound imaging device 22 may enter patient identifier information into the second ultrasound imaging device 22 in a set-up or start-up step, prompting retrieval of associated ultrasound image data from the data store 18. The second imaging device 22 may actually be the same as the first imaging device (i.e., there is only one imaging device), but is operated at a later time by a different operator than the operator who collected the first ultrasound imaging data 16. Alternatively, the second ultrasound imaging device 22 may be a different imaging device at a later time, for example, transported on a trolley to the same patient bed and placed in the same physical location, or may be a different imaging device in a different physical location, for example, where the patient has been moved to a different ward or treatment unit and imaging at the new location is recommended.

[0054] The method further includes acquiring live ultrasound image data of the patient's anatomy with a second ultrasound imaging device 22 .

[0055] The method further includes iteratively generating a live similarity measure indicative of the similarity between the view represented in the reference image data and the view represented in the live ultrasound image data, i.e., while an operator of the second ultrasound imaging device is acquiring images of the patient, a similarity measure is simultaneously calculated in the background, the similarity measure indicative of the degree to which the view being captured matches the view of the reference ultrasound data acquired by the operator of the first ultrasound imaging device 14, with the goal of maximizing the consistency of the views of the ultrasound image data acquired at two different times.

[0056] The method further includes generating a user output alert when the similarity measure matches at least one predefined criterion, e.g., when the similarity meets or exceeds a minimum similarity threshold, which indicates to the user that they have found the correct view and that they can stop adjusting the probe position. In other examples, the similarity measure is simply communicated to the user via a user interface device, such as by being displayed on a display device of the ultrasound imaging device 22.

[0057] The method further includes capturing and storing one or more frames of live ultrasound image data tagged with the patient identifier information, which are stored in data store 18 in association with the patient identifier information.

[0058] The computation of the similarity measure involves the application of an image matching operation, in which a reference image from the reference image data is compared with at least one frame from the live image data through a process where one image is matched with another image. The output is a metric that indicates how well the images match. Object detection is one good way to perform such image matching. Image template matching is another good way to perform image matching. Various methods for comparing and matching images, including object detection and cross-correlation, are known in the art and will be apparent to one skilled in the art.

[0059] An example of a suitable template matching using a cross-correlation approach is outlined in the following paper: Briechle, K. and Hanebeck, UD, "Template matching using fast normalized cross correlation," Proc. SPIE4387, Optical Pattern Recognition XII, (March 20, 2001).

[0060] Another example of using template matching is outlined in the following online paper: https: / / www.pyimagesearch.com / 2021 / 03 / 22 / opencv-template-matching-cv2-matchtemplate / . Copy of this retrieved on September 28, 2021.

[0061] To further illustrate the concepts of the present invention, an exemplary system and computer-implemented method in accordance with at least one set of embodiments will now be outlined in detail with reference to FIG.

[0062] In this embodiment, the system includes a patient monitor 32, and the aforementioned data store 18 constitutes the patient monitor 32. The patient monitor also includes a user interface .

[0063] The system allows for bidirectional communication and information flow between each of the one or more ultrasound imaging devices 14, 22 that make up the system and the patient monitor. This facilitates imaging, analysis, and patient monitoring workflows involving multiple users (e.g., including clinical professionals and non-professionals) and multiple ultrasound platforms (e.g., cart-based ultrasound imaging consoles, mobile ultrasound imaging probes, or smartphone or tablet computer-based devices). That is, multiple different types and classes of node devices can be connected to the distributed computing system and contribute to a common patient workflow. The system of this set of embodiments enables the multi-stage workflows described below. However, for purposes of illustration, the system is described with particular reference to cardiac imaging, and even more particularly to TTE and TEE probe imaging, although these concepts are equally applicable to imaging of other anatomical structures using other types of imaging devices. In general, the principles of the computer-implemented methodology performed are most advantageously applicable when longitudinal monitoring is required (e.g., lung monitoring or monitoring of any other anatomical structure). Longitudinal means over an extended period of time, which may be continuous (e.g., several hours of monitoring) or with one or more interruptions (e.g., when a patient is moved between different care units). Generally, it refers to monitoring over a period of time that is longer than a single ultrasound event.

[0064] The data flow of the method according to the embodiment of Fig. 2 is more particularly as follows: This data flow can be understood to consist of two phases: The first phase is an initial patient assessment by a first user (such as an expert) of a first ultrasound imaging device, and the second phase is a subsequent patient assessment by a second user, who may be a non-expert user of the ultrasound imaging device and / or a user with less clinical experience than the first user.

[0065] Referring to FIG. 2, in the first phase, the work flow and data flow are as follows:

[0066] When a patient 12 undergoes an ultrasound evaluation for the first time (at time t1), it is assumed that an ultrasound expert user does so. The expert user evaluates the patient using a first ultrasound imaging device 14 (such as TTE or TEE imaging). The user collects all views known to be clinically relevant and necessary. The user selects the views that provide the most clinically useful images for that patient and marks or tags these images or deletes other images that are not part of the most clinically useful subset. The image data is annotated, edited, or pre-processed in one or more other ways by the first expert user. The user may label one or more images or a subset of images according to the view they represent. That is, the user curates the images. The user generates one or more quantitative measurements from the images, either manually or using an automated algorithm, as previously described in this disclosure.

[0067] The curated images and / or measurements collected by the first user at the first ultrasound imaging device 14 are transferred to a patient monitor subsystem 32 and stored in a data store 18 included in the patient monitor subsystem. The patient monitor subsystem includes one or more client patient monitor devices (e.g., one or more bedside patient monitor devices) and a central patient monitor console to which each client device is connected. The data store may be stored in the central monitoring console. The first image data 16 is displayed on a display of a user interface 34 of the patient monitor subsystem. At the same time, the patient monitoring system may optionally receive biological sensor data about the patient 12 and display it on the display of the user interface.

[0068] The display of the user interface may also be controlled to display the acquired first ultrasound imaging data. For purposes of illustration, an exemplary user interface 34 of a patient monitoring subsystem is shown in Figure 3, where the first ultrasound image data is displayed in a display window 42, and other physiological information parameter data (such as vital signs) including trend data are shown in other areas of the display.

[0069] In an alternative embodiment, instead of transferring the image data 16 to the patient monitor subsystem for storage, a bridge processing module (not shown) is provided that connects to the patient monitor subsystem 32. The bridge processing module includes a data store 18. For example, the bridge processing module has communicative access to both the patient monitor subsystem and each of the one or more ultrasound imaging devices 14, 22 included in the system.

[0070] In addition to acquiring image data, the first user may also input one or more imaging recommendations at the first imaging device. Each imaging recommendation indicates one or more recommended views or images to be acquired at a specified subsequent time point. The recommendations may be general in terms of their timing, i.e., the recommendation is simply for the next imaging acquisition event. Alternatively, the timing may be specific, e.g., for one or more specific future time points t n , where k is a set of recommended views for each of the views. The recommendations are based on the expert user's experience during this first collection, e.g., with regard to the best available views, acoustic windows, details of the patient's anatomy, etc.

[0071] The generated recommendations are transferred to the data store 18 along with the first image data collected and stored in a data record associated with the patient. For example, the image data and recommendations may each be tagged with patient identifier information so that they can be later accessed by querying the database with the patient identifier information.

[0072] The second phase is a subsequent ultrasound examination by a further user, which may be a non-expert user less familiar with ultrasound.

[0073] At a later time when it is determined that ultrasound imaging of the patient is necessary, a second user of the system images the patient using any second ultrasound device 22 connected to the system (which may be multiple and in different physical locations). The ultrasound imaging device 22 used to image the patient on the second occasion may be the same as or different from the ultrasound imaging device 14 used to collect the first image data.

[0074] As an example, a user of the second imaging device 22 can input patient identifier information to initiate the device for use with the patient. The ultrasound device then communicates with the data store 18, either automatically or prompted by the user via input at the user interface, to retrieve stored image-referenced image data for the patient. For example, the data store is queried using the patient identifier information. In this case, the first image data 16 stored in the data store 18 by a professional user is found and retrieved. All image data in the data store 18 can be retrieved, or only a subset of it. For example, the user can specify a particular examination or monitoring protocol to be performed and retrieve the optimal images for that purpose. The retrieved images are transferred from the data store 18 to the second ultrasound imaging device 22.

[0075] If the first user provided recommended views, these are also forwarded to the second ultrasound imaging device 22. In some examples, these may be communicated to the second user using a user interface 24 of the ultrasound imaging device. In some examples, the view recommendations may be utilized to determine which of a number of different image views stored in the data store 18 associated with the patient are to be forwarded to the second ultrasound imaging device.

[0076] For example, the data store 18 stores a set of reference images depicting different views of the anatomy, each tagged according to the view depicted, and further stores the aforementioned view recommendations indicating a recommended next view of the anatomy to capture. The reference image acquired and transmitted to the second ultrasound imaging device 22 is one of the set of images depicting the recommended next view.

[0077] The second ultrasound imaging device 22 then uses the ultrasound image data obtained from the data store 18 as reference ultrasound image data to use in guiding the acquisition of a second ultrasound data set. For example, the ultrasound imaging device 22 provides acquisition assistance to the second user via a user interface to match the current acquisition to the reference image.

[0078] In some examples, acquisition assistance can be in the form of view recognition, where live ultrasound image data acquired by a second user is analyzed in real time with reference to reference image data to determine a live similarity measure between views represented in the reference image data and views represented in the live ultrasound image data, and a user interface 24 of the second ultrasound imaging device 22 is controlled to generate a user output alert when the similarity measure matches at least one predefined criterion (e.g., when the similarity measure meets a particular threshold).

[0079] Additionally or alternatively, acquisition assistance may be in the form of active probe guidance, where a processor of the ultrasound imaging device generates user guidance instructions to guide the movement of the probe to improve a similarity measure between views in the live image data and views represented in the reference images.

[0080] The method further includes capturing one or more frames of live ultrasound image data and storing the imaging data in a data store, tagged with, for example, patient identifier information and data acquisition time. In some cases, ultrasound image data is captured from the second imaging device 22 only if the similarity measure meets a predefined criterion. In some cases, the method includes capturing and storing at least one frame of live image data only if the similarity measure meets a predefined criterion and only in response to a capture command input by the user from the user input device 24. That is, the user controls the exact timing of image data capture, but the possible timings are constrained according to a similarity measure between the imaging device's live view and a view of the reference imaging data that meets the predefined criterion.

[0081] Each of the above-mentioned second ultrasound imaging device 14 and second ultrasound imaging device 22, as well as the patient monitor subsystem, includes a processing unit including one or more processors, which perform one or more of the steps of the above-mentioned methods.

[0082] In some cases, the method further includes processing at least one captured frame of live image data with an anatomical measurement algorithm to obtain one or more anatomical or physiological measurements, which may occur within the ultrasound imaging device 22 itself or elsewhere, such as the patient monitor subsystem 32.

[0083] In summary, the workflow of the embodiment of FIG. 2 can be outlined as follows: First ultrasound image data 16 for a given patient n is collected at a first time t1, e.g., by an expert user, at one ultrasound imaging device of the system. The user can annotate and curate the images. Measurements are generated from the images. The curated images and / or measurements from the initial expert user collection are preferably collected at a next time point, i.e., at a specified time point in the future, t, for the same patient n. n The image is then transferred to the patient monitor 32 along with a recommendation of the desired views and / or images that should be acquired at the current ultrasound system in use. At a later point in time, for example when it is determined that ultrasound imaging is required for the patient or according to timing specified in the recommendation, the system retrieves a first image 16 of the patient from the database in the patient monitor data store 18 and transfers it as a reference image to the current ultrasound system in use. The system then uses the transferred image as the reference image to provide acquisition assistance to the user with respect to the reference image.

[0084] In any of the examples or embodiments of the systems and computer-implemented methods described above, the system may include additional components or devices, for example, the system may further include one or more mobile computing devices (such as a smartphone or tablet computer) that are communicatively connected, linked, or networked with other components of the system.

[0085] In any of the above system and computer-implemented method examples or embodiments, there are various options regarding data flow between components of the system. In general, the methods are intended for implementation by a distributed computing system, and thus it will be understood that functions or method steps described as being performed by one particular component may in fact be performed by different components in another embodiment.

[0086] For example, in the above-described embodiments, annotation data (e.g., notes, labels, tags) is shown to be entered by a first user at the first ultrasound imaging device 14 and transmitted along with the first ultrasound imaging data 16 to the data store 18 for storage. However, in addition or alternatively, in further embodiments, annotation data associated with the first image data 16 may be entered by the first user and / or by one or more additional users at a different device or node of the system and transmitted to the data store 18 for storage. For example, this information is entered at a user interface 34 of the patient monitor subsystem 32 rather than at the first ultrasound imaging device 14. For example, the patient monitor subsystem may include a central console with a user interface and the annotation or measurement information is entered at the central patient monitor console device. Additionally or alternatively, the patient monitor may include one or more peripheral (bedside) monitoring devices and the annotation and / or measurement information is entered at a user interface of one or more of these peripheral devices. In some embodiments, the system further includes one or more mobile computing devices (such as a smartphone or tablet computer) communicatively coupled or linked to the data store 18, and the annotations and / or measurement data are entered by one or more users at the one or more mobile computing devices for transfer to the data store 18, where these one or more users may or may not include the first user (e.g., another expert user providing annotations).

[0087] Additionally, annotations or measurement information can be entered simultaneously with image data collection or at a different time (e.g., later on the mobile device of the first user or a different user) and transferred to a data store for storage.

[0088] Additionally, in one or more embodiments, a first user acquiring the first ultrasound image data inputs at the first ultrasound imaging device a selection of the acquired first ultrasound image to be subsequently used as a reference image for subsequent image data acquisition, and this information is transferred to the data store 18 for storage. This information can be input by the first user at the first ultrasound imaging device 14 or can be input at one or more additional devices or nodes of the system (by the same first user or one or more additional users).

[0089] Furthermore, the system includes functionality that allows the first user acquiring the first ultrasound image data 16 and / or another user of the system to input a recommendation for the time when subsequent ultrasound image data should be acquired. For example, illustratively, an expert clinician in direct contact with the patient reviews the patient's existing ultrasound images (which are retrieved from the data store 18), for example using the user interface 34 of the patient monitor subsystem 32, and determines that another ultrasound acquisition within two hours is clinically valuable. At the user interface of the patient monitor device (or another device of the system, such as a mobile computing device), the user can input an instruction signal that schedules, commands, or recommends a future ultrasound imaging event, including a specific time for the ultrasound imaging acquisition. Optionally, a message that a new ultrasound image acquisition has been commanded may be generated and pushed to one or more other devices of the system to change the user of the one or more additional devices. Any user (expert or non-expert) can then use the guidance functionality described above to acquire the recommended subsequent ultrasound imaging data at the specified time.

[0090] The above description is merely an exemplary set of embodiments, not all features of which are essential to the inventive concept.

[0091] For example, instead of using the patient monitor to store a database of patient image data, annotation data, measurement data, and / or further data, the data can be stored in a separate module, such as an interface module (which includes a data store) connected between one or more of the ultrasound imaging devices and the patient monitor device.

[0092] Generally, the various devices and components included in the system are spatially separated from one another, e.g., distributed at various locations throughout a medical institution. They are communicatively connected, linked, or networked to one another via one or more network connections. Their communication with one another is facilitated by one or more components acting as a network hub or server. Their communication between one another is facilitated, for example, via a local or wide area network and / or using an Internet connection. For example, each device is operable to connect to a common Internet web portal via a login function, through which interconnection between the devices is facilitated.

[0093] It should be noted that although references to a "first" image data and a "first" user are made above, this is done for ease of reference and brevity only. The ultrasound data referred to as the "first" ultrasound data need not in fact be the very first ultrasound data collected for a given patient n. The above description is intended merely to reflect the concept that a set of image data may be collected at any one of a plurality of ultrasound imaging devices of a system, and one or more images from this data may be used as a reference for use in collecting subsequent image data for the same patient at another or the same of the plurality of ultrasound imaging devices.

[0094] In various embodiments described above, an anatomical measurement algorithm is used to generate and process the ultrasound image data to derive one or more anatomical or physiological measurements, such as hemodynamic measurements. For example, one or more embodiments include processing at least one frame of captured live image data using an anatomical measurement algorithm. In one or more embodiments, a first image captured by the first ultrasound imaging device 14 is processed using an anatomical measurement algorithm to derive one or more anatomical or physiological measurements.

[0095] For example, the measurement algorithm may use model-based segmentation (MBS) to segment the boundaries of one or more anatomical structures and derive dimensional measurements and / or functional measurements such as movement patterns and fluid flow measurements.

[0096] For example, in cardiac imaging, ultrasound imaging is used to obtain quantitative measurements of hemodynamic parameters such as left ventricular (LV) volume, stroke volume (SV), cardiac output (CO), left ventricular ejection fraction (LVEF), etc. Trend patterns of these parameters can be tracked over time to inform patient treatment.

[0097] For example, it is known to apply model-based segmentation to acquired ultrasound images to obtain quantitative measurements. For example, automated segmentation of one or more chambers of the heart can be performed. Measurements of hemodynamic parameters are obtained by applying model-based segmentation to an end-diastolic (ED) ultrasound image frame to detect chamber boundaries, followed by 2D or 3D tracking of the segmentation boundaries across other frames of the cardiac cycle.

[0098] In 2D imaging, geometric assumptions can be made to extrapolate from collected single-plane or bi-plane information to generate measurements of 3D objects or features (such as cardiac chamber volumes. Example algorithms include modified Simpson's method, an ellipsoid model using single-plane, bi-plane, or one-dimensional data, or a hemisphere-cylinder model based on bi-plane data.

[0099] For more details on these segmentation methods, see: Kosaraju A, Makaryus AN. Left Ventricular Ejection Fraction. In: StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing; 2019 Jan-. Available at: https: / / www.ncbi.nlm.nih.gov / books / NBK459131 / .

[0100] As a summary of the description of at least one set of embodiments of the present invention, one advantageous system and method according to the present invention is summarized below.

[0101] The proposed system and method encompasses bidirectional communication between one or more ultrasound imaging devices and a data store (optionally included in a patient monitor) to enable replication of ultrasound image views that resulted in an initial high-quality image data set collected by an expert clinician. This allows other clinical staff (with no or lesser skills in ultrasound image collection) to collect follow-up images over time that match or correspond to the initial image views, improving patient monitoring. The proposed solution can optionally leverage the patient monitor for data storage, facilitating the use of multiple ultrasound devices during a patient's stay at a medical facility.

[0102] For example, the workflow and data flow is as follows: When a professional has acquired a satisfactory image (using a TTE or TEE imaging probe, or other non-cardiac imaging probe), the image is transferred to and stored in a database on the system's data store along with a timestamp, patient identifier information, optionally the operator's name, and optionally one or more measurements generated from the image. The data store may be contained in the patient monitor. The data store may be contained in an auxiliary device such as a storage module, or simply in a personal computer.

[0103] When another image and / or measurement at a later time is desired, the image corresponding to the previous measurement (and optionally the even earlier measurement) is retrieved from the data store and loaded onto the screen of the user interface of the ultrasound imaging device used to collect the new image data and used as a reference for the user. The system calculates a similarity measure between the live ultrasound image being collected and the reference image, and alerts the user when a match is found.

[0104] In some embodiments, anatomical features of the live image data are detected and annotations or labelings are generated by the similarity measure calculation or another processing operation. For example, one or more frames of the live image data can be labeled according to anatomical features present in the FOV, and the view depicted by the image can be identified based on the detected anatomical features. Also, image view metadata is generated that provides a description or labeling of the depicted image view (e.g., a LV4-chamber view label having four chambers and valves is identified). The system compares the detected image views with the target (reference) image views for acquisition, and an alert is issued if the second user is acquiring images in the wrong anatomical context (e.g., orthogonal to the desired plane) and at the wrong position.

[0105] Another aspect of the present invention also provides a processing device, the processing device including an input / output unit and one or more processors, the one or more processors performing the following steps: obtaining, at the input / output, from a data store, reference ultrasound image data depicting at least one view of an anatomical structure of the subject based on querying the data store with patient identifier information; receiving live ultrasound imaging data at an input / output unit; iteratively generating a live similarity measure indicative of a similarity between a view represented in the reference image data and a view represented in the live ultrasound image data; generating a user output alert when the similarity measure matches at least one predefined criterion; capturing and storing one or more frames of live image data tagged with patient identifier information; Do the following.

[0106] The processing unit is, for example, included in an ultrasound imaging apparatus, i.e. an ultrasound imaging device that collects the aforementioned ultrasound imaging data, for example the second ultrasound imaging device 22 described above with reference to Figures 1 and 2. The ultrasound imaging device forms a node in a distributed computing system that also includes a data store to which the processing unit connects. An ultrasound imaging apparatus including a processing unit and further including an ultrasound imaging probe for collecting live ultrasound imaging data forms another aspect of the present invention.

[0107] A processing device generally includes a single processor or multiple processors. The processing device may be located in a single containing device, structure, or unit, or distributed among multiple different devices, structures, or units. Thus, a reference to a processing device being adapted or configured to perform a particular step or task corresponds to that step or task being performed by any one or more of multiple processing components, either alone or in combination. Those skilled in the art will understand how such a distributed processing device can be implemented. The processing device includes a communication module or input / output for receiving data and outputting data to further components.

[0108] The one or more processors of the processing device can be implemented in various ways using software and / or hardware to perform the various functions required. Typically, the processor employs one or more microprocessors that are programmed using software (e.g., microcode) to perform the required functions. The processor can be implemented as a combination of dedicated hardware to perform some functions and one or more programmed microprocessors and associated circuitry to perform other functions. The instructions of the present invention can be in any interpretable or executable code mechanism, including, but not limited to, scripts, interpretable programs, dynamic link libraries (DLLs), or Java classes. The instructions can be provided as a complete executable program, a partial executable program, a modification (e.g., an update) of an existing program, or an extension (e.g., a plug-in) of an existing program. Additionally, parts of the processing of the present invention can be distributed across multiple computers or processors.

[0109] Examples of circuitry that may be used in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs).

[0110] In various implementations, the processor may be associated with one or more storage media, such as volatile and non-volatile computer memory, such as RAM, PROM, EPROM, and EEPROM. The storage media may be encoded with one or more programs that, when executed on the one or more processors and / or controllers, perform the necessary functions. The various storage media may be fixed within the processor or controller, or may be transportable such that one or more programs stored thereon may be loaded into the processor.

[0111] In further embodiments, the processing device may be a separate processing unit, for example a unit connected as a bridge or interface between the patient monitoring device and the ultrasound imaging device, or more generally a separate processing module forming a component of the system and communicatively connected, linked or networked with other devices or components of the system.

[0112] Another aspect of the present invention provides an ultrasound imaging apparatus including a processing device as described above and an ultrasound imaging probe for acquiring live ultrasound imaging data.

[0113] Another aspect of the present invention provides a system (e.g., a distributed computing system) according to any of the above examples or embodiments. For example, the system described in relation to Figure 1 may be provided as an embodiment of the present invention. Also, the system described in relation to Figure 2 may be provided as an embodiment of the present invention, including any of the optional features, examples, and variations described.

[0114] In one simple example, an embodiment of the present invention may provide a system including an ultrasound imaging device or apparatus including a processing apparatus according to any of the above or claims of the present application, and a data store for storing reference ultrasound image data.

[0115] Another aspect of the present invention also provides a computer program product including code means executable by a processor or processors for execution by one or more processors communicatively linked to an ultrasound imaging device and communicatively linked to a data store.

[0116] The code is then passed to your computer, which takes the following steps: receiving, from a data store, reference ultrasound image data depicting at least one view of an anatomical structure of the subject based on querying the data store with patient identifier information; receiving live ultrasound image data from an ultrasound imaging device; iteratively generating a live similarity measure indicative of a similarity between a view represented in the reference image data and a view represented in the live ultrasound image data; generating a user output alert when the similarity measure matches at least one predefined criterion; capturing and storing one or more frames of live image data tagged with patient identifier information; Have them carry out the procedure.

[0117] The code further causes the processor to output the captured frames of tagged image data to a data store for storage.

[0118] A user alert is generated by controlling a user interface connected to the processor, the user interface being the user interface of the ultrasound imaging device or a separate user interface.

[0119] In some examples, the code executes on a processor included in the ultrasound imaging device that acquired the ultrasound imaging data. For example, with reference to the embodiment of Figure 2, the code is executed by a processor of the second ultrasound imaging device 22.

[0120] Variations of the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and singular elements do not exclude a plurality.

[0121] A single processor or other unit may fulfill the functions of several items recited in the claims.

[0122] The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0123] The computer program may be stored / distributed on any suitable medium, such as an optical storage medium or a solid-state medium, supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless communication systems.

[0124] It should be noted that when the term "adapted to" is used in the claims or description, it is intended to be equivalent to the term "configured to."

[0125] Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. 1. A computer-implemented method for execution in a distributed computing system, the computer-implemented method comprising: acquiring, at a first ultrasound imaging device, first ultrasound imaging data of the subject, the first ultrasound imaging data including at least one image depicting at least one view of an anatomical structure of the subject; storing the first ultrasound imaging data in a data store as reference ultrasound image data tagged with subject identifier information; acquiring, at a second ultrasound imaging device, the reference ultrasound image data from the data store based on querying the data store using patient identifier information, the second ultrasound imaging device being the same as or different from the first ultrasound imaging device; acquiring live ultrasound image data at the second ultrasound imaging device; iteratively generating a live similarity measure indicative of a similarity between a view represented in the reference ultrasound image data and a view represented in the live ultrasound image data; generating a user output alert when the live similarity measure matches at least one predefined criterion; capturing and storing one or more frames of the live ultrasound image data tagged with the patient identifier information; 20. A computer-implemented method comprising:

2. The computer-implemented method of claim 1 , wherein the at least one predefined criterion comprises a predefined threshold of the similarity measure.

3. The computer-implemented method of claim 1 , wherein the reference ultrasound image data is obtained from a data store included in a patient monitoring subsystem.

4. 2. The computer-implemented method of claim 1, comprising capturing and storing at least one frame of the live ultrasound image data only if the similarity measure meets the at least one predefined criterion.

5. only if the similarity measure satisfies the at least one predefined criterion, and in response to a capture command from a user input device; The computer-implemented method of claim 4 , further comprising capturing and storing the at least one frame of the live ultrasound image data.

6. The computer-implemented method of claim 1 , wherein the data store stores a set of reference images depicting different views of the anatomy, each tagged according to the view it depicts.

7. 7. The computer-implemented method of claim 6, wherein the data store further stores view recommendations indicating a recommended next view of the anatomical structure to capture, and the acquired reference image is one of the set of images depicting the recommended next view.

8. generating probe positioning guidance based on the similarity metric, the probe positioning guidance for guiding a user in moving a probe to improve the similarity metric; outputting guidance information to a user interface device; The computer-implemented method of claim 1 further comprising:

9. The computer-implemented method of claim 1 , further comprising receiving annotation data along with the reference ultrasound image data for storage in the data store.

10. The computer-implemented method of claim 1 , wherein the second ultrasound imaging device is different from the first ultrasound imaging device.

11. The computer-implemented method of claim 1 , further comprising processing the at least one captured frame of the live ultrasound image data using an anatomical measurement algorithm.

12. A processing device including an input / output unit and one or more processors, The one or more processors: receiving, at the input / output unit, from a data store, reference ultrasound image data depicting at least one view of an anatomical structure of a subject based on querying the data store with patient identifier information; receiving live ultrasound image data at the input / output unit; iteratively generating a live similarity measure indicative of a similarity between a view represented in the reference ultrasound image data and a view represented in the live ultrasound image data; generating a user output alert when the live similarity measure matches at least one predefined criterion; and capturing and storing one or more frames of the live ultrasound image data tagged with the patient identifier information.

13. The processing device according to claim 12; an ultrasound imaging probe for collecting live ultrasound imaging data; 1. An ultrasound imaging device comprising:

14. The ultrasound imaging device of claim 13; A data store storing reference ultrasound image data. Including, the system.

15. 1. A computer program comprising code means for execution by a processor, said code means comprising: the processor is communicatively linked to an ultrasound imaging device and to a data store; The code means causes the processor to: receiving, from the data store, reference ultrasound image data depicting at least one view of an anatomical structure of a subject based on querying the data store with patient identifier information; receiving live ultrasound image data from the ultrasound imaging device; iteratively generating a live similarity measure indicative of a similarity between a view represented in the reference ultrasound image data and a view represented in the live ultrasound image data; generating a user output alert when the live similarity measure matches at least one predefined criterion; capturing and storing one or more frames of the live ultrasound image data tagged with the patient identifier information.

Images