Methods and systems for improved free fluid detection using ultrasound

Abstract

A method (100) for free fluid assessment of a patient, comprising: receiving (120) ultrasound imaging data of the patient's body; analyzing (130) the ultrasound imaging data to identify candidate free fluid in the patient's body, and to further detect one or more of the patient's organs in the patient's body; characterizing (140) the candidate free fluid by comparing the location of the candidate free fluid with the location of the one or more of the patient's organs, wherein the system characterizes candidate free fluid as confirmed free fluid when the location of the candidate free fluid does not overlap with the location of the one or more of the patient's organs, and wherein the system characterizes candidate free fluid as rejected free fluid when the location of the candidate free fluid overlaps with the location of the one or more of the patient's organs; displaying (150) a location of confirmed free fluid.

Description

METHODS AND SYSTEMS FOR IMPROVED FREE FLUID DETECTION USING ULTRASOUNDGovernment Interest

[0001] This invention was made with United States government support awarded by the United States Department of Health and Human Services under the grant number HHS / ASPR / BARDA 75A50120C00097. The United States has certain rights in this invention.Field of the Disclosure

[0002] The present disclosure is directed generally to methods and systems for trauma assessment of a patient, and more specifically to improved detection of free fluid using ultrasound.Background

[0003] Trauma is one of the leading causes of death. Often with trauma, there is internal bleeding in the abdomen due to injury to an organ. Severe internal bleeding can cause hemorrhagic shock or death if proper medical treatment isn’t provided quickly. Free fluid is a hypoechoic region between organ boundaries, and can be indicative of internal bleeding. The Focused Assessment with Sonography in Trauma (FAST) exam using ultrasound is regularly used as a rapid bedside imaging tool to screen for internal bleeding in the abdomen or the heart, especially after trauma. The FAST exam by ultrasound focuses on identifying free intraperitoneal or pericardial fluid in trauma examination by imaging the Right Upper Quadrant (RUQ), left upper quadrant (LUQ), suprapubic, and cardiac regions.

[0004] While the utility of the FAST exam has been widely acknowledged, interpretation by a well-trained clinician is often necessary to read the ultrasound before the plan of treatment can be defined. Artificial Intelligence (Al) applied to medical images may help bridge this expertise gap. Various classification and detection models have been developed to classify images and detect regions of interest (ROI) in them, which can be applied to the FAST Exam. However, the output of learning models used to detect free fluid can suffer from false positives due to hypoechoic regions within solid organs such as blood vessels within the liver. Other examples of failure include perinephric fat that is hypoechoic but lies within the kidney. While training with a large amountof data may overcome this limitation, such data and annotations are not always available. Indeed, the accuracy of these models often falls short of the professional standards due to the lack of sufficient data required for deep learning models. In medical imaging, achieving high sensitivity is crucial, particularly in cases where the detection of ROIs can impact patient mortality and influence clinical decision-making. In the case of FAST exam, some hypoechoic regions such as vessels may be falsely detected as free fluid, leading to lower accuracy.Summary of the Disclosure

[0005] Accordingly, there is a continued need for methods and systems for improved trauma assessment of a patient, and more specifically to the automated detection of free fluid using ultrasound. Various embodiments and implementations herein are directed to an ultrasound system configured for trauma assessment of a patient. The system receives ultrasound imaging data of a patient, and analyzes that data using a trained free fluid detection algorithm. The algorithm is trained to identify candidate free fluid in the patient’s body, and to identify one or more of the patient’s organs. The free fluid detection algorithm is further trained to characterize the candidate free fluid by comparing the location of the identified candidate free fluid with the location of the one or more of the patient’s organs. The system is further configured to display, on a user interface, a location of confirmed free fluid in the patient’s body.

[0006] Generally, in one aspect, a method for free fluid assessment of a patient is provided. The method includes: receiving ultrasound imaging data of at least a portion of the patient’s body; analyzing, with a trained free fluid detection algorithm, the received ultrasound imaging data to identify candidate free fluid in the patient’s body, including a location of the identified candidate free fluid, and to further detect one or more of the patient’s organs in the patient’s body, including a location of each of the one or more of the patient’s organs; characterizing, with the trained free fluid detection algorithm, the candidate free fluid by comparing the location of the identified candidate free fluid with the location of the one or more of the patient’s organs, wherein the trained free fluid detection algorithm characterizes candidate free fluid as confirmed free fluid when the location of the candidate free fluid does not overlap with the location of the one or more of the patient’s organs, and wherein the trained free fluid detection algorithm characterizes candidate free fluid as rejected free fluid when the location of the candidate free fluid overlaps with the locationof the one or more of the patient’s organs; displaying, on a user interface, a location of confirmed free fluid in the patient’s body.

[0007] According to an embodiment, the method further includes segmenting, using an automated segmentation algorithm, the detected one or more of the patient’s organs. According to an embodiment, the automated segmentation algorithm performs a pixel-by-pixel segmentation of at least some of the received ultrasound imaging data.

[0008] According to an embodiment, characterizing the candidate free fluid comprises comparing, on a pixel-by-pixel basis, the location of the identified candidate free fluid with segmented location of the one or more of the patient’s organs.

[0009] According to an embodiment, whether the location of the candidate free fluid overlaps or does not overlap the location of the one or more of the patient’s organs is characterized by the trained free fluid detection algorithm using a predetermined overlap threshold.

[0010] According to an embodiment, the method further includes receiving, via a user interface, input from a user regarding the one or more of the patient’s organs in the chest and / or abdomen.

[0011] According to an embodiment, the input comprises information about a location or identification of the one or more of the patient’s organs.

[0012] According to an embodiment, the method further includes diagnosing, based on the displayed location of confirmed free fluid in the patient’s body, the patient as comprising free fluid.

[0013] According to another aspect is an ultrasound system configured for free fluid assessment of a patient. The system includes: ultrasound imaging data of at least a portion of the patient’s body; a processor comprising a trained free fluid detection algorithm, wherein the free fluid detection algorithm is trained to: (i) analyze the received ultrasound imaging data to identify candidate free fluid in the patient’s body, including a location of the identified candidate free fluid, and to further detect one or more of the patient’s organs, including a location of each of the one or more of the patient’s organs; and (ii) characterize the candidate free fluid by comparing the location of the identified candidate free fluid with the location of the one or more of the patient’s organs, wherein the trained free fluid detection algorithm characterizes candidate free fluid as confirmed free fluid when the location of the candidate free fluid does not overlap with the location of the one or more of the patient’s organs, and wherein the trained free fluid detection algorithm characterizes candidate free fluid as rejected free fluid when the location of the candidate free fluidoverlaps with the location of the one or more of the patient’s organs; and a user interface configured to display a location of confirmed free fluid in the patient’s body.

[0014] According to an embodiment, the user interface is further configured to receive input from a user regarding the one or more of the patient’s organs.

[0015] It should be appreciated that all combinations of the foregoing concepts and additional concepts discussed in greater detail below (provided such concepts are not mutually inconsistent) are contemplated as being part of the inventive subject matter disclosed herein. In particular, all combinations of claimed subject matter appearing at the end of this disclosure are contemplated as being part of the inventive subject matter disclosed herein. It should also be appreciated that terminology explicitly employed herein that also may appear in any disclosure incorporated by reference should be accorded a meaning most consistent with the particular concepts disclosed herein.

[0016] These and other aspects of the various embodiments will be apparent from and elucidated with reference to the embodiment(s) described hereinafter.Brief Description of the Drawings

[0017] In the drawings, like reference characters generally refer to the same parts throughout the different views. The figures showing features and ways of implementing various embodiments and are not to be construed as being limiting to other possible embodiments falling within the scope of the attached claims. Also, the drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the various embodiments.

[0018] FIG. 1 is a flowchart of a method for trauma assessment of a patient, in accordance with an embodiment.

[0019] FIG. 2 is a schematic representation of a trauma assessment system, in accordance with an embodiment.

[0020] FIG. 3A is an ultrasound image comprising detection of abdominal organs, in accordance with an embodiment.

[0021] FIG. 3B is an ultrasound image comprising detection of abdominal organs, in accordance with an embodiment.

[0022] FIG. 3C is an ultrasound image comprising detection of abdominal organs, in accordance with an embodiment.

[0023] FIG. 4 is a method for training a free fluid detection algorithm, in accordance with an embodiment.

[0024] FIG. 5A is an annotated ultrasound image comparing candidate free fluid with an identified organ, in accordance with an embodiment.

[0025] FIG. 5B is an annotated ultrasound image comparing candidate free fluid with an identified organ, in accordance with an embodiment.Detailed Description of Embodiments

[0026] The present disclosure describes various embodiments of a system and method configured for detection of free fluid in a patient, using ultrasound. More generally, Applicant has recognized and appreciated that it would be beneficial to provide improved methods and systems for automated detection of free fluid in a patient. Accordingly, an ultrasound system receives or obtains ultrasound imaging data of a patient, and analyzes that data using a trained free fluid detection algorithm. The algorithm is trained to identify candidate free fluid in the patient’s body, and to identify one or more of the patient’s organs. The free fluid detection algorithm is further trained to characterize the candidate free fluid by comparing the location of the identified candidate free fluid with the location of the one or more of the patient’s organs. The system is further configured to display, on a user interface, a location of confirmed free fluid in the patient’s body.

[0027] Accordingly, the methods and systems described or otherwise envisioned herein improve free fluid detection using information gained from concurrent methods that detect the presence of solid organs or other features. The system uses anatomical features detected in ultrasound imagery, and the knowledge that free fluid will occur between organ interfaces and not within organs in order, to improve the robustness of free fluid detection methods. The system can build on organ detection methods that are already utilized to provide scan completeness guidance to the user, and thus does not add additional burden to the development process or computational effort. The systems and methods will thus be useful for providing more accurate detection of free fluid in patients.

[0028] According to an embodiment, the systems and methods described or otherwise envisioned herein can, in some non-limiting embodiments, be implemented as a component of or an extension of an ultrasound system such as the Philips® Lumify® system (available from Koninklijke Philips NV, the Netherlands), or as an element for a commercial product for patient analysis or monitoring, or any suitable system. However, the disclosure is not limited to these devices or systems, and thus disclosure and embodiments disclosed herein can encompass any system that may utilize or benefit from the analysis and visualization described or otherwise envisioned herein.

[0029] Referring to FIG. 1 , in one embodiment is a flowchart of a method 100 for providing a free fluid assessment of a patient using an ultrasound system. The methods described in connection with the figures are provided as examples only, and shall be understood to not limit the scope of the disclosure. The ultrasound system can be any of the systems described or otherwise envisioned herein. The ultrasound system can be a single system or multiple different systems.

[0030] At step 110 of the method, an ultrasound system 200 is provided. Referring to an embodiment of an ultrasound system 200 as depicted in FIG. 2, for example, the system comprises one or more of a processor 220, memory 230, user interface 240, communications interface 250, and storage 260, interconnected via one or more system buses 212. It will be understood that FIG. 2 constitutes, in some respects, an abstraction and that the actual organization of the components of the system 200 may be different and more complex than illustrated. Additionally, ultrasound system 200 can be any of the systems described or otherwise envisioned herein. Other elements and components of ultrasound system 200 are disclosed and / or envisioned elsewhere herein.

[0031] According to an embodiment, the ultrasound system 200 is or comprises one or more ultrasound devices 270 capable of acquiring the required ultrasound images and / or performing the analyses as described or otherwise envisioned herein. According to another embodiment, the ultrasound system 200 is in wired and / or wireless communication with a local or remote ultrasound device 270 capable of acquiring the required ultrasound images.

[0032] At step 120 of the method, the ultrasound system receives ultrasound data of the patient. The patient may be any patient for which the ultrasound analyses described or otherwise envisioned herein might be useful or applicable. The patient may be any patient in a local or remote care setting, emergency setting, at a remote location receiving emergency care, or any other patient.According to an embodiment, the patient is a trauma victim. For example, the patient may have experienced trauma such as a motor vehicle accident, fall, impact, shooting, or any other trauma that could lead to free fluid. According to another embodiment, the patient is undergoing an inpatient procedure or analysis, and a free fluid analysis is useful for that in-patient procedure or analysis. Many other options and situations are possible.

[0033] Notably, the patient may be an animal other than a human being. For example, the patient may be an animal, such as a mammal other than a human being, that is being analyzed for the presence of free fluid. Examples of animals that may be the patient include horses, bovines, dogs, cats, and many other animals.

[0034] According to an embodiment, the ultrasound data is of the any portion of the patient’s body where free fluid may be found. For example, during a typical FAST exam, the pericardium and three potential spaces within the peritoneal cavity are analyzed for pathologic fluid. The right upper quadrant (RUQ) visualizes the hepatorenal recess, also known as Morrison’s pouch, the right paracolic gutter, the hepato-diaphragmatic area, and the caudal edge of the left liver lobe. Subxiphoid (or subcostal) views are obtained to evaluate the pericardial space. The left upper quadrant (LUQ) is imaged to inspect the splenorenal recess, the subphrenic space, and the left paracolic gutter. Other regions can also be imaged during a FAST exam. Additionally, the imaging data can be obtained from many other regions of the body.

[0035] According to an embodiment, the ultrasound image data may be obtained using any ultrasound device or system, such as ultrasound system 270, which may be any device or system suitable to obtain or otherwise receive the ultrasound image data of the patient utilized by system 200. The ultrasound image data may be obtained as 2D or 3D data. The ultrasound image data may be obtained as video data. One or more parameters of the ultrasound device can be set, adjusted, preprogrammed, or otherwise determined by a healthcare professional. According to an embodiment, the ultrasound device or system comprises an ultrasound transducer probe configured to obtain the ultrasound images.

[0036] For example, according to one embodiment, the ultrasound data is obtained from a patient using an ultrasound transducer probe of an ultrasound device, and that ultrasound device further comprises a processor or other components necessary to carry out the remainder of the method and analysis. Thus, ultrasound system 200 is a single unit. According to anotherembodiment, the ultrasound data is obtained from a patient using an ultrasound transducer probe of an ultrasound device, and that ultrasound device is remote from a processor or other components necessary to carry out the remainder of the method and analysis. Thus, ultrasound system may comprise the processor and other components to carry out the remainder of the method and analysis, and may optionally comprise the ultrasound device or may otherwise be in communication with the ultrasound device utilized to obtain the ultrasound data.

[0037] Once the ultrasound data of the patient is received or obtained by the ultrasound system 200, that ultrasound data may be utilized immediately, or may be stored in local or remote storage for use in further steps of the method. The ultrasound data of the patient may optionally be pre- processed at any point. For example, the ultrasound system may comprise a data pre-processor or similar component or algorithm configured to process the received ultrasound data. For example, the data pre-processor analyzes the ultrasound data to remove noise, errors, and / or other potential issues. Many other forms of ultrasound data pre-processing are possible.

[0038] At step 130 of the method, the ultrasound system 200 analyzes the received ultrasound imaging data to identify candidate free fluid in the patient’s body, specifically the portion of the patient’s body that was imaged by the ultrasound device, and / or the portion of the patient’s body that could potentially comprise free fluid. According to an embodiment, analyzing the received ultrasound imaging data to identify candidate free fluid in the patient’s body further includes determining the location of the identified candidate free fluid within the patient’s body.

[0039] Also at step 130 of the method, the ultrasound system 200 analyzes the received ultrasound imaging data to identify or detect one or more of the patient’s organs in the patient’s body. According to an embodiment, the patient’s organ may be any organ in the body, such as the liver, heart, spleen, bladder, uterus, kidney(s), and / or bone(s), among many other organs. Additionally, for purposes of the methods and systems described or otherwise envisioned herein, an organ may further include a shadow or artifact created by the organ during imaging. According to an embodiment, analyzing the received ultrasound imaging data to identify or detect one or more of the patient’s organs further includes determining the location of the one or more of the patient’s organs.

[0040] According to an embodiment, the ultrasound system uses an algorithm to analyze the received ultrasound imaging data to identify or detect one or more of the patient’s organs in thepati ent’s body. For example, the system may utilize a segmentation approach to outline the contour of the organs. The segmentation can be implemented using either a deep learning methodology such as a U-net based architecture, or an image processing method, or any other method for segmentation. In another embodiment, a deep learning based object detection model can be used to detect the bounding boxes for these objects.

[0041] Referring to FIGS. 3A-3C are examples of abdominal organs being detected using this approach, although this is provided as an example only and thus does not limit the scope of the methods or systems described or otherwise envisioned herein. FIG. 3A comprises an LUQ Image with the spleen, diaphragm, and the spleen tip detected in agreement with ground truth labels. FIG. 3B comprises an RUQ image with the liver, diaphragm and kidney detected in agreement with ground truth labels. FIG. 3C comprises free fluid detection in the hepatorenal space and other organs, showing good agreement between the ground truth and the prediction.

[0042] According to an embodiment, the ultrasound system uses an algorithm to analyze the received ultrasound imaging data to identify candidate free fluid in the patient’s body. For example, the ultrasound system may use an algorithm to perform free fluid detection, segmentation, and / or classification. This can be accomplished, for example, using object detection, image classification, or a segmentation model. According to one embodiment, the system uses U-net based architectures for achieving pixel-by-pixel segmentation to determine candidate regions for free fluid. However, other embodiments and methods are possible.

[0043] According to an embodiment, the ultrasound system 200 uses a trained free fluid detection algorithm to analyze the received ultrasound imaging data (as input to the model) to identify the candidate free fluid in the patient’s body, and to identify or detect the one or more of the patient’s organs in the patient’s body (as output of the model). The trained free fluid detection algorithm can be any model that can be trained to utilize the input to generate the output, as described or otherwise envisioned herein. For example, the free fluid detection algorithm can be a neural network or other trained machine learning model, and / or fluid detection can be performed using object detection, image classification, or a segmentation model. Thus, according to an embodiment, the ultrasound system comprises a trained free fluid detection algorithm that receives the input data (i.e., the received ultrasound imaging data) and outputs data regarding candidate freefluid and one or more organs in the patient’s body, including the location of the candidate free fluid and the one or more organs in the patient’s body.

[0044] The free fluid detection algorithm can be trained in a variety of different ways. According to one embodiment, the free fluid detection algorithm is trained in an unsupervised manner or in a supervised manner. Referring to FIG. 4, in one embodiment, is a flowchart of a method 400 for training the free fluid detection algorithm of the ultrasound system 200. This method may be performed by the ultrasound system, or may be performed by another system such as a specialized machine learning model training system.

[0045] At step 410 of the method, the training system receives training data which will be used to train the model. The training data can be any data sufficient to train the model to utilize the described input data to generate the described output. For example, the training data may comprise ultrasound imaging for each of a plurality of patients and procedures, including with ground truth optimization. This training data, which could be utilized in a supervised or unsupervised manner, can comprise ultrasound imaging for 100s or 1000s of patients and / or procedures, and can optionally be updated with new imaging. The training data may also comprise other information. This training data may be obtained and curated by an expert such as a clinician, or it may be obtained and curated under the supervision of a clinician, or it may be obtained and utilized without curation. The training data may be received from any source. For example, the training data may be received from the electronic medical record database or system, or any other component of the ultrasound system or a training system. According to an embodiment, the ultrasound system 200 comprises or is in direct or indirect communication with an imaging database which comprises some or all of the training data set.

[0046] According to an embodiment, the training system may comprise a data pre-processor or similar component or algorithm configured to process the received training data. For example, the data pre-processor analyzes the training data to remove noise, bias, errors, and other potential issues. The data pre-processor may also analyze the input data to remove low quality data. Many other forms of data pre-processing or data point identification and / or extraction are possible.

[0047] At step 420 of the method, the training system trains the free fluid detection algorithm, using the training data, to identify the candidate free fluid in the patient’s body, and to identify or detect the one or more of the patient’s organs in the patient’s body, including the locations of thecandidate free fluid and the one or more organs. The free fluid detection algorithm is trained using any method for training such a model. The trained free fluid detection algorithm is a unique model based on the training data used to train the model. Following training, the system comprises a free fluid detection algorithm.

[0048] Thus, following training, the free fluid detection algorithm is a specialized model configured to receive the input (namely, the ultrasound imaging of the patient’s body) and generate the very specific output, namely the identification of candidate free fluid in the patient’s body, and the identification or detection of the one or more of the patient’s organs in the patient’s body, including the locations of the candidate free fluid and the one or more organs.

[0049] At step 430 of the method, the trained free fluid detection algorithm is stored for future use. According to an embodiment, the trained free fluid detection algorithm may be stored in local or remote storage.

[0050] Returning to method 100 in FIG. 1, at optional step 132 of the method, the system may receive input from a user, provided via a user interface of the ultrasound system, regarding the identification of the one or more organs in the patient’s body. For example, the system may be configured such that a user may input seed points to aid in the segmentation or detection methods. Other options for receiving user input are possible.

[0051] At step 140 of the method, the ultrasound system 200 analyzes the identified candidate free fluid and the one or more organs, including their identified locations, to determine if there is or is not location overlap between the candidate free fluid and the one or more identified or detected organs in the patient’s body. According to an embodiment, the ultrasound system compares the location of the identified candidate free fluid with the location of the one or more of the patient’s organs to determine whether there is overlap.

[0052] According to an embodiment, the trained free fluid detection algorithm characterizes candidate free fluid as rejected free fluid when the location of the candidate free fluid overlaps with the location of the one or more of the patient’s organs. In other words, since there cannot be confirmed free fluid within the patient’s organ, candidate free fluid is rejected as being free fluid if there is overlap between the location of candidate free fluid and the location of an organ.

[0053] According to an embodiment, the trained free fluid detection algorithm characterizes candidate free fluid as confirmed free fluid when the location of the candidate free fluid does not overlap with the location of the one or more of the patient’s organs. In other words, since free fluid is not found within the patient’s organ, candidate free fluid is confirmed as being free fluid if there is no overlap between the location of candidate free fluid and the location of an organ.

[0054] According to an embodiment, the ultrasound system can perform geometric calculations to determine whether the candidate free fluid region is substantially within one of the identified or detected patient organs. This can be accomplished by looking at each pixel in the candidate free fluid region and determining whether it is within the region bounded by one of the organs as shown in FIGS. 5A-5B where the free fluid region can be easily discarded as it falls inside a segmented organ such as liver (FIG. 5A, showing an RUQ image with segmented liver and outlined diaphragm along with false positive prediction of free fluid (arrow) in the liver region), and inside the bladder (FIG. 5B, showing an SP image with the segmented bladder and false positive prediction of free fluid (arrow) inside the bladder region). The determination can be repeated for all pixels in the candidate free fluid region. If the percentage of free fluid pixel candidates lying within organ boundaries exceeds a threshold, then the candidate free fluid region is discarded and is not shown to the user. The procedure is repeated for all candidate free fluid regions. Remaining free fluid candidate regions, if any, that do not meet the criteria above can be shown as free fluid regions on the ultrasound frame.

[0055] The threshold for determining whether the percentage of free fluid pixel candidates lying within organ boundaries constitutes overlap or lack of overlap can be determined or set in a variety of different ways. For example, the threshold may be set by a clinician. As another example, the threshold may be experimentally determined. As yet another example, the threshold may be determined by the trained free fluid detection algorithm.

[0056] At step 150 of the method, the ultrasound system generates and displays, on a user interface of the system, a visualization of the location of the confirmed free fluid in the patient’s body. The location of the confirmed free fluid may be presented using any method for providing ultrasound imaging data, and optionally annotation of that data. The display and / or visualization may further comprise other information, such as the location of the identified or characterized one or more organs, rejected free fluid and associated locations, and other information. Thevisualization may be any visualization that is currently utilized for ultrasound systems and ultrasound visualizations, among other possible visualizations.

[0057] The visualization may be generated using any method for generating ultrasound visualizations. According to an embodiment, the ultrasound system may comprise software or an algorithm that is programmed or trained to generate a visualization of ultrasound data. Thus, the generated visualization may be provided via any mechanism for visualizing ultrasound data, such as via an annotated ultrasound image or video, or other mechanism.

[0058] The generated visualization of the location of the confirmed free fluid in the patient’s body may be displayed to a medical professional or other user via the user interface of the system. The generated visualization may be provided to a user via any mechanism for display, visualization, or otherwise providing information via a user interface. According to an embodiment, the information may be communicated by wired and / or wireless communication to a user interface and / or to another device. For example, the system may communicate the information to a mobile phone, computer, laptop, wearable device, and / or any other device configured to allow display and / or other communication of the report. The user interface can be any device or system that allows information to be conveyed and / or received, and may include a display, a mouse, and / or a keyboard for receiving user commands. As just one non-limiting example, the user interface may be a component of a patient monitoring system, an ultrasound system, or any other system.

[0059] According to an embodiment, the display may further comprise patient information such as monitoring information about the patient, demographic information about the patient, a diagnosis for the patient, medical history of the patient, and / or any other information.

[0060] According to an embodiment, some or all of the displayed information may be manipulatable, in response to user input provided via the user interface.

[0061] At optional step 170 of the method, the medical professional or clinician that receives and reviews the generated and provided visualization of the location of the confirmed free fluid in the patient’s body can decide to reject or accept the confirmation of the free fluid in the patient’s body. This can be based in part or in whole on the generated and provided visualization, and / or can be supplemented with experience of the medical professional or clinician. Thus, according to an embodiment, the method can comprise diagnosing, based on the displayed location ofconfirmed free fluid in the patient’s body, the patient as comprising the free fluid at the displayed location. This can then trigger a treatment protocol to resolve the free fluid and / or the cause of the free fluid.

[0062] Thus, at optional step 180 of the method, the medical professional or clinician initiates a treatment protocol to resolve the free fluid and / or the cause of the free fluid, based on the diagnosis from step 170 of the method (which in turn is based on the generated and provided visualization of the location of the confirmed free fluid in the patient’s body). The treatment protocol can include, among other things, additional imaging to confirm the free fluid (such as CT), therapeutic laparotomy, diagnostic laparoscopy, surgery, IV fluids, and other treatments. For example, in the instance of abdominal free fluid, the treatment may be abdominal imaging, abdominal laparotomy, abdominal surgery, and / or other treatments.

[0063] Referring to FIG. 2 is a schematic representation of an ultrasound system 200. System 200 may be any of the systems described or otherwise envisioned herein, and may comprise any of the components described or otherwise envisioned herein. It will be understood that FIG. 2 constitutes, in some respects, an abstraction and that the actual organization of the components of the system 200 may be different and more complex than illustrated.

[0064] According to an embodiment, system 200 comprises a processor 220 capable of executing instructions stored in memory 230 or storage 260 or otherwise processing data to, for example, perform one or more steps of the method. Processor 220 may be formed of one or multiple modules. Processor 220 may take any suitable form, including but not limited to a microprocessor, microcontroller, multiple microcontrollers, circuitry, field programmable gate array (FPGA), application-specific integrated circuit (ASIC), a single processor, or plural processors.

[0065] Memory 230 can take any suitable form, including a non-volatile memory and / or RAM. The memory 230 may include various memories such as, for example LI, L2, or L3 cache or system memory. As such, the memory 230 may include static random access memory (SRAM), dynamic RAM (DRAM), flash memory, read only memory (ROM), or other similar memory devices. The memory can store, among other things, an operating system. The RAM is used by the processor for the temporary storage of data. According to an embodiment, an operating system may contain code which, when executed by the processor, controls operation of one or morecomponents of system 200. It will be apparent that, in embodiments where the processor implements one or more of the functions described herein in hardware, the software described as corresponding to such functionality in other embodiments may be omitted.

[0066] User interface 240 may include one or more devices for enabling communication with a user. The user interface can be any device or system that allows information to be conveyed and / or received, and may include a display, a mouse, and / or a keyboard for receiving user commands. In some embodiments, user interface 240 may include a command line interface or graphical user interface that may be presented to a remote terminal via communication interface 250. The user interface may be located with one or more other components of the system, or may located remote from the system and in communication via a wired and / or wireless communications network.

[0067] Communication interface 250 may include one or more devices for enabling communication with other hardware devices. For example, communication interface 250 may include a network interface card (NIC) configured to communicate according to the Ethernet protocol. Additionally, communication interface 250 may implement a TCP / IP stack for communication according to the TCP / IP protocols. Various alternative or additional hardware or configurations for communication interface 250 will be apparent.

[0068] Storage 260 may include one or more machine-readable storage media such as readonly memory (ROM), random-access memory (RAM), magnetic disk storage media, optical storage media, flash-memory devices, or similar storage media. In various embodiments, storage 260 may store instructions for execution by processor 220 or data upon which processor 220 may operate. For example, storage 260 may store an operating system 261 for controlling various operations of system 200.

[0069] It will be apparent that various information described as stored in storage 260 may be additionally or alternatively stored in memory 230. In this respect, memory 230 may also be considered to constitute a storage device and storage 260 may be considered a memory. Various other arrangements will be apparent. Further, memory 230 and storage 260 may both be considered to be non-transitory machine-readable media. As used herein, the term non-transitory will be understood to exclude transitory signals but to include all forms of storage, including both volatile and non-volatile memories.

[0070] While system 200 is shown as including one of each described component, the various components may be duplicated in various embodiments. For example, processor 220 may include multiple microprocessors that are configured to independently execute the methods described herein or are configured to perform steps or subroutines of the methods described herein such that the multiple processors cooperate to achieve the functionality described herein. Further, where one or more components of system 200 is implemented in a cloud computing system, the various hardware components may belong to separate physical systems. For example, processor 220 may include a first processor in a first server and a second processor in a second server. Many other variations and configurations are possible.

[0071] According to an embodiment, the system comprises one or more ultrasound devices 270 capable of acquiring the required ultrasound images or analysis. According to another embodiment, ultrasound system 200 is in wired and / or wireless communication with a local or remote ultrasound device 270 capable of acquiring the required ultrasound images or analysis.

[0072] According to an embodiment, storage 260 of system 200 may store one or more algorithms, modules, and / or instructions to carry out one or more functions or steps of the methods described or otherwise envisioned herein. For example, the system may comprise, among other instructions or data, ultrasound imaging data 262, a trained free fluid detection algorithm 263, and visualization and reporting instructions 264, among other possible instructions.

[0073] According to an embodiment, ultrasound imaging data 262 is data obtained from the patient for which the ultrasound analysis described or otherwise envisioned herein will be performed. The patient may be any patient for which the ultrasound analyses described or otherwise envisioned herein might be useful or applicable, and can be a human being or an animal other than a human being. The patient may be any patient in a local or remote care setting, emergency setting, at a remote location receiving emergency care, or any other patient. According to an embodiment, the patient is a trauma victim. For example, the patient may have experienced trauma such as a motor vehicle accident, fall, impact, shooting, or any other trauma that could lead to free fluid. According to another embodiment, the patient has a medical condition - such as cirrhosis of the liver among many other conditions - that results in free fluid. According to another embodiment, the patient is undergoing an in-patient procedure or analysis, and a free fluid analysis is useful for that in-patient procedure or analysis. Many other options and situations are possible.According to an embodiment, the ultrasound data is of the any portion of the patient’s body where free fluid may be found. According to an embodiment, the ultrasound image data may be obtained using any ultrasound device or system, such as ultrasound system 270, which may be any device or system suitable to obtain or otherwise receive the ultrasound image data of the patient utilized by system 200. Once the ultrasound data of the patient is received or obtained by the ultrasound system 200, that ultrasound data may be utilized immediately, or may be stored in local or remote storage for use in further steps of the method.

[0074] According to an embodiment, free fluid detection algorithm 263 is trained according to the methods and systems described or otherwise envisioned herein to utilize the received ultrasound imaging data (as input to the model) to identify the candidate free fluid in the patient’s body, and to identify or detect the one or more of the patient’s organs in the patient’s body (as output of the model). The trained free fluid detection algorithm can be any model that can be trained to utilize the input to generate the output, as described or otherwise envisioned herein. For example, the free fluid detection algorithm can be a neural network or other trained machine learning model. Thus, according to an embodiment, the ultrasound system comprises a trained free fluid detection algorithm that receives the input data (i.e., the received ultrasound imaging data) and outputs data regarding candidate free fluid and one or more organs in the patient’s body, including the location of the candidate free fluid and the one or more organs in the patient’s body.

[0075] According to an embodiment, free fluid detection algorithm 263 is trained according to the methods and systems described or otherwise envisioned herein to analyze the identified candidate free fluid and the one or more organs, including their identified locations, to determine if there is or is not location overlap between the candidate free fluid and the one or more identified or detected organs in the patient’s body. According to an embodiment, the ultrasound system compares the location of the identified candidate free fluid with the location of the one or more of the patient’s organs to determine whether there is overlap. The free fluid detection algorithm is trained to characterize candidate free fluid as rejected free fluid when the location of the candidate free fluid overlaps with the location of the one or more of the patient’s organs. In other words, since there cannot be confirmed free fluid within the patient’s organ, candidate free fluid is rejected as being free fluid if there is overlap between the location of candidate free fluid and the location of an organ. The free fluid detection algorithm is further trained to characterize candidate free fluid as confirmed free fluid when the location of the candidate free fluid does not overlap with thelocation of the one or more of the patient’s organs. In other words, since free fluid is not found within the patient’s organ, candidate free fluid is confirmed as being free fluid if there is no overlap between the location of candidate free fluid and the location of an organ.

[0076] According to an embodiment, visualization and reporting instructions 264 direct the system to generate a visualization of the location of the confirmed free fluid in the patient’s body. The visualization may be any visualization that is currently utilized for ultrasound systems and ultrasound visualizations, among other possible visualizations. The visualization may be generated using any method for generating ultrasound visualizations. According to an embodiment, the ultrasound system may comprise software or an algorithm that is programmed or trained to generate a visualization of ultrasound data.

[0077] According to an embodiment, visualization and reporting instructions 264 further direct the system to provide the generated visualization of the location of the confirmed free fluid in the patient’s body. The location of the confirmed free fluid may be presented using any method for providing ultrasound imaging data, and optionally annotation of that data. The display and / or visualization may further comprise other information, such as the location of the identified or characterized one or more organs, rejected free fluid and associated locations, and other information. The visualization may be any visualization that is currently utilized for ultrasound systems and ultrasound visualizations, among other possible visualizations. The generated visualization may be provided to a user via any mechanism for display, visualization, or otherwise providing information via a user interface. According to an embodiment, the information may be communicated by wired and / or wireless communication to a user interface and / or to another device. For example, the system may communicate the information to a mobile phone, computer, laptop, wearable device, and / or any other device configured to allow display and / or other communication of the report. The user interface can be any device or system that allows information to be conveyed and / or received, and may include a display, a mouse, and / or a keyboard for receiving user commands.

[0078] According to an embodiment, the ultrasound system is configured to process many thousands or millions of datapoints in the input data used to train the free fluid detection algorithm, as well as to process and analyze the received ultrasound data to identify free fluid and patient organ(s). For example, generating a functional and skilled trained algorithm using an automatedprocess such as feature identification and extraction and subsequent training requires processing of millions of datapoints from input data and the generated features. This can require millions or billions of calculations to generate a novel trained detection algorithm from those millions of datapoints and millions or billions of calculations. As a result, each trained algorithm is novel and distinct based on the input data and parameters of the machine learning algorithm, and thus improves the functioning of the infection detection system. Thus, generating a functional and skilled trained classifier comprises a process with a volume of calculation and analysis that a human brain cannot accomplish in a lifetime, or multiple lifetimes.

[0079] In addition, the ultrasound system can be configured to continually receive ultrasound data, perform the analysis, and provide periodic or continual updates via the report provided to a clinician. This requires the analysis of thousands or millions of datapoints on a continual basis to optimize the reporting, requiring a volume of calculation and analysis that a human brain cannot accomplish in a lifetime.

[0080] By providing improved free fluid detection, this novel ultrasound system has an enormous positive effect on patient care compared to prior art systems. As just one example, by providing a system that can improve healthcare outcomes by detection free fluid, the system can facilitate early treatment decisions and improve survival outcomes, thereby leading to saved lives.

[0081] The present invention may be a system, a method, and / or a computer program product. The computer program product may include a non-transitory computer readable storage medium (or media) having computer readable program instructions thereon for causing a system or processor to carry out aspects of the present invention. The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination of the foregoing, among other possibilities. Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the internet, a local area network, and / or a wireless network. Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus, systems, andcomputer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0082] All definitions, as defined and used herein, should be understood to control over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0083] The indefinite articles “a” and “an,” as used herein in the specification and in the claims, unless clearly indicated to the contrary, should be understood to mean “at least one.”

[0084] The phrase “and / or,” as used herein in the specification and in the claims, should be understood to mean “either or both” of the elements so conjoined, i.e., elements that are conjunctively present in some cases and disjunctively present in other cases. Multiple elements listed with “and / or” should be construed in the same fashion, i.e., “one or more” of the elements so conjoined. Other elements may optionally be present other than the elements specifically identified by the “and / or” clause, whether related or unrelated to those elements specifically identified.

[0085] As used herein in the specification and in the claims, “or” should be understood to have the same meaning as “and / or” as defined above. For example, when separating items in a list, “or” or “and / or” shall be interpreted as being inclusive, i.e., the inclusion of at least one, but also including more than one, of a number or list of elements, and, optionally, additional unlisted items. Only terms clearly indicated to the contrary, such as “only one of’ or “exactly one of,” or, when used in the claims, “consisting of,” will refer to the inclusion of exactly one element of a number or list of elements. In general, the term “or” as used herein shall only be interpreted as indicating exclusive alternatives (i.e. “one or the other but not both”) when preceded by terms of exclusivity, such as “either,” “one of,” “only one of,” or “exactly one of.”

[0086] As used herein in the specification and in the claims, the phrase “at least one,” in reference to a list of one or more elements, should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but not necessarily including at least one of each and every element specifically listed within the list of elements and not excluding any combinations of elements in the list of elements. This definition also allows thatelements may optionally be present other than the elements specifically identified within the list of elements to which the phrase “at least one” refers, whether related or unrelated to those elements specifically identified.

[0087] It should also be understood that, unless clearly indicated to the contrary, in any methods claimed herein that include more than one step or act, the order of the steps or acts of the method is not necessarily limited to the order in which the steps or acts of the method are recited.

[0088] In the claims, as well as in the specification above, all transitional phrases such as “comprising,” “including,” “carrying,” “having,” “containing,” “involving,” “holding,” “composed of,” and the like are to be understood to be open-ended, i.e., to mean including but not limited to. Only the transitional phrases “consisting of’ and “consisting essentially of’ shall be closed or semi-closed transitional phrases, respectively.

[0089] While several inventive embodiments have been described and illustrated herein, those of ordinary skill in the art will readily envision a variety of other means and / or structures for performing the function and / or obtaining the results and / or one or more of the advantages described herein, and each of such variations and / or modifications is deemed to be within the scope of the inventive embodiments described herein. More generally, those skilled in the art will readily appreciate that all parameters, dimensions, materials, and configurations described herein are meant to be exemplary and that the actual parameters, dimensions, materials, and / or configurations will depend upon the specific application or applications for which the inventive teachings is / are used. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, many equivalents to the specific inventive embodiments described herein. It is, therefore, to be understood that the foregoing embodiments are presented by way of example only and that, within the scope of the appended claims and equivalents thereto, inventive embodiments may be practiced otherwise than as specifically described and claimed. Inventive embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and / or method described herein. In addition, any combination of two or more such features, systems, articles, materials, kits, and / or methods, if such features, systems, articles, materials, kits, and / or methods are not mutually inconsistent, is included within the inventive scope of the present disclosure.

Claims

ClaimsWhat is claimed is:

1. A method (100) for free fluid assessment of a patient, comprising: receiving (120) ultrasound imaging data of at least a portion of the patient’s body; analyzing (130), with a trained free fluid detection algorithm, the received ultrasound imaging data to identify candidate free fluid in the patient’s body, including a location of the identified candidate free fluid, and to further detect one or more of the patient’s organs in the patient’s body, including a location of each of the one or more of the patient’s organs; characterizing (140), with the trained free fluid detection algorithm, the candidate free fluid by comparing the location of the identified candidate free fluid with the location of the one or more of the patient’s organs, wherein the trained free fluid detection algorithm characterizes candidate free fluid as confirmed free fluid when the location of the candidate free fluid does not overlap with the location of the one or more of the patient’s organs, and wherein the trained free fluid detection algorithm characterizes candidate free fluid as rejected free fluid when the location of the candidate free fluid overlaps with the location of the one or more of the patient’s organs; displaying (150), on a user interface, a location of confirmed free fluid in the patient’s body.

2. The method of claim 1, further comprising segmenting, using an automated segmentation algorithm, the detected one or more of the patient’s organs.

3. The method of claim 2, wherein the automated segmentation algorithm performs a pixel-by-pixel segmentation of at least some of the received ultrasound imaging data.

4. The method of claim 1, wherein characterizing the candidate free fluid comprises comparing, on a pixel-by-pixel basis, the location of the identified candidate free fluid with segmented location of the one or more of the patient’s organs.

5. The method of claim 1, wherein whether the location of the candidate free fluid overlaps or does not overlap with the location of the one or more of the patient’s organs is characterized by the trained free fluid detection algorithm using a predetermined overlap threshold.

6. The method of claim 5, wherein the predetermined overlap threshold is determined by a user.

7. The method of claim 1, further comprising the step of receiving (132), via a user interface, input from a user regarding the one or more of the patient’s organs in the chest and / or abdomen.

8. The method of claim 7, wherein the input comprises information about a location or identification of the one or more of the patient’s organs.

9. The method of claim 1 , further comprising diagnosing (170), based on the displayed location of confirmed free fluid in the patient’s body, the patient as comprising free fluid.

10. An ultrasound system (200) for free fluid assessment of a patient, comprising: ultrasound imaging data (262) of at least a portion of the patient’s body; a processor (220) comprising a trained free fluid detection algorithm (263), wherein the free fluid detection algorithm is trained to: (i) analyze the received ultrasound imaging data to identify candidate free fluid in the patient’s body, including a location of the identified candidate free fluid, and to further detect one or more of the patient’s organs, including a location of each of the one or more of the patient’s organs; and (ii) characterize the candidate free fluid by comparing the location of the identified candidate free fluid with the location of the one or more of the patient’s organs, wherein the trained free fluid detection algorithm characterizes candidate free fluid as confirmed free fluid when the location of the candidate free fluid does not overlap with the location of the one or more of the patient’s organs, and wherein the trained free fluid detection algorithm characterizes candidate free fluid as rejected free fluid when the location of the candidate free fluid overlaps with the location of the one or more of the patient’s organs; anda user interface (240) configured to display a location of confirmed free fluid in the patient’s body.

11. The system of claim 10, wherein the processor further comprises an automated segmentation algorithm configured to segment the detected one or more of the patient’s organs.

12. The system of claim 11, wherein the automated segmentation algorithm performs a pixel-by-pixel segmentation of at least some of the received ultrasound imaging data.

13. The system of claim 10, wherein characterizing the candidate free fluid comprises comparing, on a pixel-by-pixel basis, the location of the identified candidate free fluid with segmented location of the one or more of the patient’s organs.

14. The system of claim 10, wherein whether the location of the candidate free fluid overlaps or does not overlap with the location of the one or more of the patient’s organs is characterized by the trained free fluid detection algorithm using a predetermined overlap threshold.

15. The system of claim 10, wherein the user interface is further configured to receive input from a user regarding the one or more of the patient’s organs.

Images