Method and system for clinical scoring of lung ultrasound examinations - Patents.com

Abstract

A method 100 for generating an ultrasound examination score includes (i) receiving a temporal sequence of ultrasound image data for each of a plurality of distinct zones of one or both lungs of a patient (104), (ii) identifying one or more lung features for each zone using a feature identification algorithm (106), (iii) determining a confidence score for each identified feature (108), (iv) determining a composite score for each zone using a composite score algorithm (110) that is indicative of a severity of a disease or condition or a diagnosis, (v) determining a lung score using a lung score algorithm (112) that is indicative of a severity of a pulmonary disease or condition or a diagnosis, and (vi) providing, via a user interface (114), one or more of the lung score, the composite score for one or more of the plurality of distinct zones, and the confidence score for one or more of the identified feature(s).

Description

[Technical field]

[0001] Government Interests This invention was made with United States Government support awarded by the Department of Health and Human Services under Grant No. HHS / ASPR / BARDA75A50120C00097. The United States has certain rights in this invention.

[0002] The present disclosure is generally directed to methods and systems for generating an ultrasound examination score. [Background technology]

[0003] Lung ultrasound imaging is an important tool for screening, monitoring, diagnostic support, and management of disease. Important clinical features (e.g., B-lines, combined B-lines, pleural line changes, consolidations, and pleural effusions, among others) can be identified using lung ultrasound. The combined presence of these features is predictive of various pulmonary diseases and infections, including COVID-19 pneumonia. However, identifying clinical features using lung ultrasound is a difficult skill to master, and success typically requires extensive specialized training and experience.

[0004] During an ultrasound exam, images or videos are collected from multiple lung zones (up to 14 zones per exam). These zones are then examined separately and the results are compiled to determine an overall diagnosis or severity for the entire exam. Proper interpretation of the lung ultrasound features in each video, and overall interpretation at the exam level, requires specialized training and expertise.

[0005] Automated quantification tools offer the potential to simplify and standardize image interpretation tasks, including ultrasound analysis. Studies have shown correlations between automated lung ultrasound features and expert assessments, as well as with gold standard measures such as blood tests and chest CT. Automated analysis can also be used to diagnose conditions such as COVID-19 pneumonia. However, automated quantification tools have significant limitations. One limitation of these tools is their black box nature, which makes the tool's decision-making process difficult to understand or interpret. This results in challenges to clinical adoption and increases the burden of approval for regulatory agencies. This lack of transparency is particularly problematic in the case of lung ultrasound, as it involves aggregating results across multiple videos. The lack of interpretation at the video level makes it difficult for clinicians to arrive at a holistically meaningful exam-level decision.

[0006] US Pat. No. 1,0667,793 B2 discloses detecting B-lines in each image of an ultrasound video clip and assigning a score to each image of the video clip based on the number of B-lines detected. Summary of the Invention [Problem to be solved by the invention]

[0007] Therefore, there is a need for automated lung ultrasound examination quantification tools that can generate and present exam-level information in an easy-to-understand, interpretable manner. [Means for solving the problem]

[0008] The present disclosure is directed to an inventive method and system for analysis of ultrasound lung imaging. Various embodiments and implementations herein are directed to an ultrasound analysis system including an ultrasound device configured to acquire ultrasound images of a patient's lungs including a time sequence of ultrasound image data for each of a plurality of different zones of the patient's lungs. The system uses a feature identification algorithm to identify one or more features of the lungs for each of the plurality of different zones from the time sequence of ultrasound image data. A confidence score for each identified feature is then determined. A composite score is determined for each of the plurality of different zones using a composite score algorithm, the identified features, and the determined confidence score. A lung score is determined using a lung score algorithm that analyzes the plurality of determined composite scores. A user interface is used to provide information to a user, possibly including one or more of the lung score, the composite score for one or more of the plurality of different zones, and a confidence score for one or more of the identified one or more features.

[0009] In general, in one aspect, a method for generating an ultrasound score is provided. The method includes the steps of: (i) receiving a temporal sequence of ultrasound image data for each of a plurality of different zones of one or both lungs of a patient; (ii) identifying one or more lung features for each of the plurality of different zones from the temporal sequence of ultrasound image data using a feature identification algorithm; (iii) determining a confidence score for each of the identified one or more features; (iv) determining a composite score for each of the plurality of different zones using a composite score algorithm that analyzes the identified features and the determined confidence scores, the composite score for the zone being indicative of either a severity of a disease or condition for the zone or a diagnosis for the zone; (v) determining a pulmonary score using a pulmonary score algorithm that analyzes the plurality of determined composite scores, the pulmonary score being indicative of either a severity of a pulmonary disease or condition or a pulmonary diagnosis; and (vi) providing, via a user interface, one or more of the pulmonary score, the composite score for one or more of the plurality of different zones, and the confidence score for one or more of the identified one or more features.

[0010] According to one embodiment, the temporal sequence of ultrasound image data includes a plurality of frames for each of a plurality of zones, and further, the confidence score is determined based on an aggregation of per-frame confidence scores determined for each frame of the plurality of frames.

[0011] According to one embodiment, identifying one or more features of the lungs includes determining the presence or absence of the features.

[0012] According to one embodiment, the composite score algorithm determines a composite score for a zone by aggregating two or more composite scores for the zone.

[0013] According to an embodiment, the method further comprises weighting the one or more composite scores, and the pulmonary score algorithm utilizes the weighted composite scores when determining the pulmonary score.

[0014] According to an embodiment, the method further includes receiving, via a user interface, a request for additional information regarding the user-selected lung zone; and providing, via the user interface, the requested additional information regarding the user-selected lung zone, the additional information including one or more of a temporal sequence of ultrasound image data for the user-selected lung zone, a composite score for the user-selected lung zone, and a confidence score for the user-selected lung zone.

[0015] According to one embodiment, a composite score for each of a number of different zones is determined by aggregating the identified characteristics of each zone.

[0016] According to one embodiment, a composite score for each of a number of different zones is determined by aggregating the same features identified across the multiple zones.

[0017] According to another aspect, a method for generating an ultrasound examination score includes: (i) receiving a temporal sequence of ultrasound image data for each of a plurality of distinct zones of one or both lungs of a patient; (ii) determining a composite score for each of the plurality of distinct zones using a composite score algorithm that analyzes the ultrasound image data, the composite score for the zone being indicative of either a severity of a disease or condition of the zone or a diagnosis of the zone; (iii) determining a pulmonary score using a pulmonary score algorithm that analyzes the plurality of determined composite scores, the pulmonary score being indicative of either a severity of a disease or condition of the zone or a diagnosis of the zone; and (iv) providing, via a user interface, one or more of the pulmonary score, the composite score for one or more of the plurality of distinct zones, and a confidence score for one or more of the identified one or more features.

[0018] According to another aspect, an ultrasound analysis system configured to generate an ultrasound examination score. The system includes a processor configured to perform: (ii) a temporal sequence of ultrasound image data for each of a plurality of different zones of one or both lungs of the patient; (ii) identifying one or more lung features for each of the plurality of different zones from the temporal sequence of ultrasound image data using a feature identification algorithm; (ii) determining a confidence score for each of the identified one or more features; (iii) determining a composite score for each of the plurality of different zones using a composite score algorithm that analyzes the identified features and the determined confidence scores, wherein the composite score for the zone is indicative of either (a) a severity of a disease or condition of the zone or (b) a diagnosis of the zone; and (iv) determining a pulmonary score using a pulmonary score algorithm that analyzes the plurality of determined composite scores, wherein the pulmonary score is indicative of either (a) a severity of a disease or condition of the lung or (b) a diagnosis of the zone; and a user interface configured to provide one or more of the pulmonary score, the composite score for one or more of the plurality of different zones, and the confidence score for one or more of the identified one or more features.

[0019] It should be understood that all combinations of the foregoing concepts and additional concepts detailed below (provided those concepts are not mutually inconsistent) are contemplated as part of the inventive subject matter disclosed herein, and in particular, all combinations of claimed subject matter appearing at the end of this disclosure are contemplated as part of the inventive subject matter disclosed herein.

[0020] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter.

[0021] In the drawings, like reference characters generally refer to the same parts throughout the different views. Also, the drawings are not necessarily to scale, emphasis generally being placed upon illustrating the principles of the invention. [Brief description of the drawings]

[0022] [Figure 1] 1 is a flowchart of a method for generating an ultrasound examination score using an ultrasound analysis system, according to an embodiment. [Diagram 2] 1 is a schematic diagram of an ultrasound analysis system, in accordance with an embodiment. [Diagram 3] 1 is a schematic diagram of a composite severity score derived by an ultrasound analysis system, according to an embodiment. [Figure 4] FIG. 1 is a schematic diagram of reporting of a composite severity score and a pulmonary score by an ultrasound analysis system, according to an embodiment. [Diagram 5] 1 is a flowchart of a method for generating a final pulmonary score using an ultrasound analysis system, according to an embodiment. [Figure 6] 1 is a flowchart of a method for generating a final pulmonary score using an ultrasound analysis system, according to an embodiment. [Figure 7A] 1 is a flowchart of a method for generating a final pulmonary score using an ultrasound analysis system, according to an embodiment. [Figure 7B] 1 is a flowchart of a method for generating a final pulmonary score using an ultrasound analysis system, according to an embodiment. [Figure 8] 1 is a flowchart of a method for generating a final pulmonary score using an ultrasound analysis system, according to an embodiment. [Figure 9] 1 is a flowchart of a method for generating a final pulmonary score using an ultrasound analysis system, according to an embodiment. [Figure 10] 1 is a flowchart of a method for generating a final pulmonary score using an ultrasound analysis system, according to an embodiment. [Figure 11]FIG. 1 is a schematic diagram of reporting lung scores by an ultrasound analysis system, according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] This disclosure describes various embodiments of ultrasound analysis systems and methods. More generally, applicants have recognized and appreciated that it would be beneficial to provide an ultrasound analysis that automatically generates a lung test score or diagnosis. For example, an ultrasound analysis system receives or acquires ultrasound image data for each of a plurality of different zones of a patient's lungs. The system uses a feature identification algorithm to identify one or more lung features for each of the plurality of different zones from the time sequence of ultrasound image data. A confidence score for each identified feature is then determined. A composite score is determined for each of the plurality of different zones using a composite score algorithm, the identified features, and the determined confidence score. A lung score is determined using a lung score algorithm that analyzes the plurality of determined composite scores. A user interface is used to provide information to a user, possibly including one or more of the lung score, the composite score for one or more of the plurality of different zones, and a confidence score for one or more of the identified one or more features.

[0024] The ultrasound analysis systems and methods disclosed or otherwise contemplated herein provide many advantages over the prior art: Providing ultrasound analysis systems and methods that enable automated detection and diagnosis of pulmonary diseases or conditions in an easy-to-understand, interpretable manner can prevent serious pulmonary damage, improve patient outcomes, and potentially save lives.

[0025] Referring to Figure 1, in one embodiment, there is shown a flow chart of a method 100 for generating an ultrasound test score using an ultrasound analysis system. It should be understood that the method described in connection with the figure is provided by way of example only and is not intended to limit the scope of the present disclosure. The ultrasound analysis system may be any of the systems described or otherwise contemplated herein. The ultrasound analysis system may be a single system or multiple different systems.

[0026] In step 102 of the method, an ultrasound analysis system 200 is provided. Referring to the embodiment of ultrasound analysis system 200 shown in FIG. 2, for example, the system includes one or more of a processor 220, a memory 230, a user interface 240, a communication interface 250, a storage 260, and an ultrasound device 270, interconnected via one or more system buses 212. It will be understood that FIG. 2 constitutes an abstraction in some respects, and the actual organization of the components of system 200 may be different and more complex than that shown. Moreover, ultrasound analysis system 200 may be any of the systems described or otherwise contemplated herein. Other elements and components of system 200 are disclosed and / or contemplated elsewhere herein.

[0027] In step 104 of the method, ultrasound image data is transmitted, acquired, or otherwise received by the system. The ultrasound image data includes a temporal sequence of ultrasound image data, such as, for example, a video including multiple frames. The ultrasound image data may be acquired for one region or zone of the patient's lungs, or for multiple different zones. For example, ultrasound image data is acquired for one, two, or more zones. Some exams may include 14 or more zones, although more or fewer zones are possible. The ultrasound image data may be received by the system in real time, or may be stored in local and / or remote memory and received by the system at a future time.

[0028] The ultrasound image data may be acquired using any ultrasound device or system, which may be any device or system suitable for acquiring or otherwise receiving ultrasound image data of a patient. One or more parameters of the ultrasound device may be set, adjusted, pre-programmed, or otherwise determined by medical personnel. The ultrasound device or system may be remote from, local to, or a component of ultrasound analysis system 200.

[0029] In step 106 of the method, the ultrasound analysis system uses the ultrasound image data to identify one or more features of the lungs (also referred to as clinical features). According to an embodiment, a feature is any recognizable aspect of the lungs. A feature may be a normal or abnormal aspect of the lungs. A feature is indicative of a healthy lung, a diseased or disordered lung, or a lung with any type of lung-related condition. Thus, a feature may be, for example, anything that can be identified in or from the ultrasound image data. Examples of features include A-lines, B-lines, combined B-lines, pleural line abnormalities, consolidation, pleural effusion, and many others.

[0030] According to an embodiment, the ultrasound analysis system uses a feature identification algorithm to identify one or more features of the lungs from the ultrasound image data. The feature identification algorithm is any algorithm capable of identifying features from the ultrasound image data. The feature identification algorithm may be trained, for example, using a set of feature training data. The feature identification algorithm may include a single algorithm configured or trained to identify any type of feature, or the feature identification algorithm may include different algorithms or aspects each configured or trained to identify a particular feature. According to an embodiment, the feature identification algorithm utilizes a feature detection or segmentation algorithm to identify the location of the feature in each frame, such as by drawing a bounding box. Each detected feature, such as each bounding box, may include a predicted confidence score, as described herein.

[0031] According to one embodiment, the ultrasound analysis system uses a feature identification algorithm to identify pulmonary features from the ultrasound image data based on presence or absence. The system can identify the features on a frame-by-frame basis.

[0032] According to one embodiment, the ultrasound analysis system uses a feature identification algorithm to identify lung features from the ultrasound image data across a single video for a lung zone, which may be based on presence or absence, for example. By way of example only, the feature identification algorithm may identify features using only the video and not individual frames. For example, the system may include a deep neural network model that takes an entire video clip as input and outputs an identification of features. As another example, the feature identification algorithm may identify features for one or more frames in the video and aggregate the identified features across multiple frames.

[0033] According to one embodiment, an ultrasound analysis system identifies lung features from ultrasound image data for each imaged zone of a plurality of different imaged zones, the system may identify features in each zone sequentially or may identify features in two or more zones simultaneously.

[0034] In method step 108, the ultrasound analysis system determines a confidence score for each of the identified feature or features. For example, if the ultrasound analysis system identifies lung features from the ultrasound image data using a feature identification algorithm based on presence or absence, the identification includes a confidence score between 0 and 1.

[0035] The confidence score indicates the likelihood that a feature identified by the feature identification algorithm was correctly identified.

[0036] According to an embodiment in which an ultrasound analysis system uses a feature identification algorithm to identify lung features from ultrasound image data across a single video of a region of the lung, the algorithm can use the entire video clip as input and output a confidence score for each feature. As another example, the feature identification algorithm can identify features and confidence scores for one or more frames in the video and aggregate the identified features and confidence scores across multiple frames. For example, for each feature, the system can calculate a total video-level confidence score based on the sum or average of the confidence scores per frame. Alternatively, the system can use the size of the feature instead of the confidence score. Alternatively, the system can identify features using a combination of these embodiments, such as, for example, (size of feature) x (confidence score of feature), or any other method or equation for aggregation or combination. The system can determine the confidence scores of identified features sequentially or can determine the confidence scores of two or more identified features simultaneously.

[0037] In step 110 of the method, the ultrasound analysis system determines one or more composite scores for each imaged zone. According to an embodiment, the system includes a composite score algorithm that utilizes one or more of the identified features and the determined confidence scores as inputs. The one or more composite scores determined for a zone include or are otherwise indicative of the severity of the disease or condition of that zone. Additionally or alternatively, the composite score determined for a zone includes or is otherwise indicative of the diagnosis of the disease or condition of that zone. The composite score determined for a zone may include other information about the zone.

[0038] According to an embodiment, the ultrasound analysis system determines multiple composite scores for the imaged zones. For example, a composite score algorithm may utilize the input of a given zone to generate a first composite score for B-lines indicative of the severity or diagnosis of the B-lines, a second composite score for pleural lines, etc.

[0039] Thus, according to an embodiment, a composite score algorithm derives a composite severity score and / or diagnosis for each frame or video for each zone. This may be a rule-based algorithm or a predictive model such as a machine learning based algorithm. The composite score may be calculated for each individual frame and then aggregated, or calculated directly for the video. Typically, one video is collected for each lung zone scanned, which results in a different severity score for each lung zone.

[0040] According to an embodiment, the output of the composite score algorithm includes a binary yes / no, a continuous measurement between 0 and 1, an integer severity scale (such as 0 to 3), or any other informative or useful output. The output may be a triage recommendation or outcome prediction.

[0041] According to an embodiment, the composite score may be based on a combined panel of two or more lung ultrasound features. One example of a panel of features is A-lines, B-lines, combined B-lines, pleural line abnormalities, consolidation, and pleural effusion. Other panels with more, fewer, and / or other features are possible.

[0042] According to an embodiment, the composite score can be derived using a rules-based algorithm. The rules can be obtained or derived from established guidelines or international consensus recommendations. In the case of lung ultrasound for respiratory disease and infection management, such guidelines have been reported in the literature. The advantage of this approach is that the algorithm has already been validated and accepted within the wider community.

[0043] According to an embodiment, instead of rule-based algorithms, composite score prediction models can be used, including machine learning based approaches such as decision trees, support vector machines, neural networks, among other possibilities. One advantage is that the model can be optimized for a particular diagnostic or severity assessment task.

[0044] According to an embodiment, the composite score is derived directly from the raw ultrasound video data of the zone. This can be achieved, for example, by feeding the entire video clip to the system and the composite score algorithm, which predicts the overall severity as a single number. Typically, one video is collected for each lung zone scanned, resulting in a different severity score for each lung zone. According to an embodiment, the composite score is calculated on a frame-by-frame basis and then aggregated.

[0045] 3, in one embodiment, is a summary of possible composite severity scores that can be derived by a composite score algorithm for a zone using a rule-based algorithm and a scale of 0 to 3, with 0 being least severe (i.e., probably healthy) and 3 being most severe. For example, a video level severity score based on a scale of 0 to 3 can be determined from a combination of features present in the video.

[0046] As shown in Figure 3, the ultrasound analysis system is configured to analyze multiple features, including but not limited to lung sliding, A-lines, B-lines, pleural lines, and consolidation. The composite score algorithm uses the identified features and / or feature confidence scores for each feature and for each zone to determine a composite score. For example, for the B-line feature, with 2 or less B-lines, the severity score is 0, with 3 or more B-lines, the severity score is 1, with less than 50% combined B-lines, the severity score is 2, and with 50% or more combined B-lines, the severity score is 3. Figure 3 represents only a single possible embodiment of the composite score algorithm analysis and resulting composite score. Many other embodiments are possible.

[0047] In method step 112, the ultrasound analysis system determines an overall lung score. According to an embodiment, the system includes a lung score algorithm that utilizes as input a plurality of composite scores determined for a plurality of imaged lung zones. The determined lung score includes or is otherwise indicative of information regarding the severity of the patient's lung disease or condition. Additionally or alternatively, the determined lung score includes or is otherwise indicative of information regarding the diagnosis of the patient's lung disease or condition. Additionally or alternatively, the determined lung score may include other information regarding the lungs.

[0048] An ultrasound exam typically includes multiple imaged zones, including 14 or more zones, although more or less are possible. The lung score may be the sum or average of the composite scores determined for each lung zone, although other approaches to deriving an overall exam-level lung score are possible, and several of these embodiments are described or otherwise contemplated herein.

[0049] Referring to FIG. 4, in one embodiment, an example of an overall lung score (in this example, a severity score) generated from multiple composite scores. The ultrasound image was analyzed by an ultrasound analysis system to identify features of multiple zones (i.e., eight zones, R1-R4 and L1-L4) using a feature identification algorithm. A composite score algorithm was then utilized to determine a composite score for each of the multiple different zones, as shown in Table 1. A final lung score was determined using a lung score algorithm that analyzes the multiple determined composite scores. In this example, the composite scores of the eight imaged zones were summed to generate a lung score of 8 (0+0+0+0+3+2+1+2=8). As shown in FIG. 4 and further described herein, the composite score and overall score are presented to the user via a diagram including the lungs with the corresponding composite score (e.g., "R1" and composite score "0") and overall score ("overall score 8").

[0050] [Table 1]

[0051] According to one embodiment, a lung scoring algorithm is configured or trained to determine feature-specific scores for each lung zone, aggregate scores for all zones, and combine features at the exam level.

[0052] According to an embodiment, some lung zones may be more important than others. To take this into account, the overall score may be weighted based on the relative importance of each lung zone. Thus, in an optional step 111 of the method, the ultrasound analysis system weights one or more of the determined composite scores based on the importance of the corresponding lung zone. The importance of a lung zone may be pre-determined by a user of the system, flagged or otherwise indicated during or after the lung examination. In step 112 of the method, the lung score algorithm uses the one or more weighted composite scores in determining the lung score. The lung score algorithm may increase the final lung score when adding or otherwise combining or analyzing the composite scores as a result of utilizing the one or more weighted composite scores. By way of example, if L2 is a particularly important zone for or in the test, any composite score generated for L2 may be weighted. If the composite score for L2 is 0, the score is not weighted. Alternatively, if the L2 composite score is greater than 0, indicating a possible problem, the composite score may be automatically weighted to reflect a predetermined importance of the L2 zone. As another example, if B-line abnormalities are of particular concern or importance in an exam, a composite score greater than 0 due to B-line abnormalities may be weighted by the system to increase or otherwise influence or impact the determined overall lung score.

[0053] According to an embodiment, the pulmonary score may also be determined based on multiple examinations of the same patient. This occurs when a patient undergoes examinations at different times during a hospital stay. For example, ultrasound imaging data from two or more examinations may be stored in a database and accessed or otherwise received by the system for analysis according to the methods described or otherwise contemplated herein. A pulmonary score is calculated for each examination and then combined or otherwise aggregated. Differences in the pulmonary scores, such as increases or decreases in the score, are also reported to the user to show changes over time.

[0054] According to an embodiment, if additional information beyond ultrasound is required for overall diagnosis or severity assessment, this may be provided as a secondary input to the final algorithm or model.

[0055] 5, in one embodiment, a flow chart of a method 500 for generating a final pulmonary score using an ultrasound analysis system, which is an embodiment of method 100. According to this embodiment, features are identified or determined at the frame 510, video 520, or study 530 level to generate a prediction of a set of pulmonary ultrasound (LUS) features 540 for the study, such as B-lines, connected B-lines, etc. The set of features generated from the frame, video, and / or study are used in downstream steps of the method.

[0056] According to an embodiment, data from the generated or extracted features is combined to generate a final pulmonary score 550, which indicates the severity of the disease and / or condition and / or the diagnosis of the condition or disease.

[0057] Referring to FIG. 6, in one embodiment, a flow chart of a method 600 for generating a final lung score using an ultrasound analysis system, which is an embodiment of method 100. According to this embodiment, features are identified or determined at the frame level. That is, features may be identified by the system individually for a single zone in each of multiple different frames and repeated for subsequent zones. To generate a video level identification or prediction of features, the system may combine the frame level feature determinations in various ways, such as a confidence score or an averaging scheme based on feature detection bounding box size. For example, using approach 1, the system may combine the confidence scores (pred) determined for different features. K ) can be averaged (pred1+pred2+pred3+pred4+pred5···pred KUsing approach 2, the system can estimate the size of the different features. K ) can be averaged (size1+size2+size3+size4+size5···size K Using approach 3, the system can generate a final lung score from both the confidence score and the feature size, for example, using the following formula:

number

[0058] 7A, in one embodiment, a flow chart of a method for generating a final pulmonary score using an ultrasound analysis system, which is an embodiment of method 100. According to this embodiment, features are identified or determined at the video level. Thus, for each zone (e.g., Zone1), a feature prediction (pred1) is made and the predictions are averaged to generate an exam-level prediction using the following formula:

number

[0059] 7B, in one embodiment, a flow chart of a method for generating a final pulmonary score using an ultrasound analysis system, which is an embodiment of method 100. According to this embodiment, features are identified or determined at a frame level. Thus, for each zone (e.g., Zone1), a feature prediction (e.g., pred1) is made for each frame, and the predictions are combined to generate a test-level prediction using the following formula:

number

[0060] Referring to FIG. 8, in one embodiment, an embodiment of the method 100 is to use an ultrasound analysis system to measure lung severity. Final1 is a flow chart of a method for generating a severity score for a lung zone video. According to this embodiment, features are identified or determined at the video level. The video level predictions for each feature are combined to generate an overall severity score for the lung zone video. ZoneK ) to generate an overall severity score for the entire test. Final ) is derived from the score per video (per lung zone). It can be, for example, the sum of the severity scores of multiple lung zones. Final ) may be a severity rating or may be a diagnosis (such as a binary yes / no) instead of or in addition to an overall severity rating.

[0061] Referring to FIG. 9, in one embodiment, an embodiment of the method 100 is to measure the lung score (Pred) using an ultrasound analysis system. Final ) according to this embodiment, features are identified or determined at the video level for each feature of each zone (Pred in B lines). Zone1 …Pred ZoneK Pred in abnormalities of the pleural line Zone1 …Pred ZoneK ), which are then used to generate a confidence score. Thus, the video-level predictions for each feature are aggregated across all lung zones to produce a test-level prediction (Pred for B lines) per feature across all zones. Exam Pred in abnormalities of the pleural line Exam (Pred, etc.) is generated, which involves summing predictions, among other aggregation methods. The test-level feature predictions are fed into the final model. For example, if there are 5 features, the final model will take and use 5 inputs. The output (Pred Final ) can be an overall severity and / or a diagnosis (such as a binary yes / no).

[0062] Referring to FIG. 10, in one embodiment, an embodiment of a method 100 is to measure the lung score (Pred) using an ultrasound analysis system. Final) according to this embodiment, features are specified or determined at the video level for each feature of each zone (Pred in B lines). Zone1 …Pred ZoneK Pred in abnormalities of the pleural line Zone1 …Pred ZoneK The zone predictions are then fed directly into the final model or algorithm to generate a lung score. For example, if 5 features are analyzed, the final model will use 5*K inputs to generate a lung score.

[0063] Returning to method 100 shown in FIG. 1, in method step 114, the ultrasound analysis system provides information from the analysis to a user via a user interface. This information may include one or more of the determined lung score, a composite score of one or more features for one or more of the imaged lung zones, and / or a confidence score for one or more of the identified features. Other information is also possible, including, but not limited to, patient identity, patient demographic information, diagnostic or treatment information, and a variety of other possible information. The information may be provided via the user interface using any method for communicating or displaying information, and the user interface may be any device, interface, or mechanism for providing communicated or displayed information.

[0064] Referring to FIG. 11, in one embodiment, an example of information provided to a user via a user interface. This information includes a visual dashboard that displays a composite score for each lung zone (e.g., "L1" is "3") along with an overall exam score ("Overall Score" is "8"). The user can click on each lung zone to see more detailed information, such as frame / video level detection of each feature within that lung zone. Similarly, clicking on a lung zone can play the lung ultrasound video collected in that lung zone along with the individual detected features.

[0065] According to an embodiment, the user interface can summarize information across multiple tests (i.e., multiple time points). In one implementation, a summary dashboard for each time point is displayed. In a second implementation, the overall test-level scores for each time point are plotted on a single graph, such as a disease progression plot that charts pulmonary severity scores over time.

[0066] Thus, the methods and systems described or otherwise contemplated herein provide many advantages over the prior art. For example, the system improves interpretability of ultrasound imaging over prior art systems by allowing clinicians to better evaluate individual features that provide a final lung score. Importantly, by detecting and visualizing relevant lung features and providing intermediate results at the frame / video / exam level, users can interpret ultrasound findings together with other patient medical information to make a more informed final decision, thereby improving patient outcomes.

[0067] According to certain embodiments, the methods and systems described or otherwise contemplated herein have numerous applications. For example, the systems can be utilized in pre-hospital settings, as an initial evaluation in emergency rooms, for post-treatment follow-up, and in many other settings. The methods are applicable to all ultrasound imaging systems, particularly in point-of-care applications. The methods and systems can be used in a variety of settings, such as ambulances, ER or critical care, or surgical settings (including acute respiratory and chest emergencies).

[0068] In method step 116, the ultrasound analysis system receives a request for additional information regarding one or more lung zones via a user interface. For example, in method step 114, information regarding one or more of the determined lung scores, the composite score of one or more features for one or more of the imaged lung zones, and / or the confidence score for one or more of the identified features is displayed in the user interface. In step 116, the user can use the user interface to identify one or more lung zones for the extended information. The user can, for example, click or touch or otherwise select a lung zone, a composite score, or other aspect of the display to indicate a request for additional information.

[0069] For example, referring to Figure 11, the user clicks, touches, or otherwise selects zone L4 for additional information. The user interface interprets this selection as a request for more information about L4 (the lung zone selected by the user).

[0070] In method step 118, the ultrasound analysis system, in response to a request to receive additional information regarding one or more lung zones, provides some or all of the requested information via a user interface. According to an embodiment, the additional information includes one or more of a temporal sequence of ultrasound image data for the user-selected lung zone, a composite score for the user-selected lung zone, a confidence score for the user-selected lung zone, and / or other information regarding the lung zone.

[0071] For example, referring again to Figure 11, the user clicks, touches, or otherwise selects zone L4 for additional information. The ultrasound analysis system provides the user, via the user interface, with additional information shown in an expansion panel on the right side of the figure. The additional information includes frame / video level detections of each feature within that lung zone, such as B-line predictions and a composite score (somewhere between 1 and 2). The predictions or composite scores may be integers or a finer breakdown.

[0072] Referring to Figure 2, there is shown a schematic diagram of an ultrasound analysis system 200. System 200 may be any of the systems described or otherwise contemplated herein and may include any of the components described or otherwise contemplated herein. It will be appreciated that Figure 2 constitutes an abstraction in some respects, and that the actual organization of the components of system 200 may be different and more complex than that depicted.

[0073] In one embodiment, system 200 includes a processor 220 that can execute instructions stored in memory 230 or storage 260 or otherwise process data to, for example, perform one or more steps of a method. Processor 220 may be formed of one or more modules. Processor 220 may take any suitable form including, but not limited to, a microprocessor, a microcontroller, multiple microcontrollers, a circuit, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a single processor, or multiple processors.

[0074] The memory 230 may take any suitable form, including non-volatile memory and / or RAM. The memory 230 may include various memories, such as, for example, an L1, L2, or L3 cache, or a system memory. As such, the memory 230 may include a static random access memory (SRAM), a dynamic RAM (DRAM), a flash memory, a read-only memory (ROM), or other similar memory devices. The memory may store, among other things, an operating system. The RAM is used by the processor for temporary storage of data. According to an embodiment, the operating system may include code that, when executed by the processor, controls the operation of one or more components of the system 200. In embodiments in which the processor implements one or more of the functions described herein in hardware, it will be apparent that in other embodiments, software described as corresponding to such functions may be omitted.

[0075] User interface 240 may include one or more devices for enabling user interaction. A user interface is any device or system that enables the sending and receiving of information and may include a display, a mouse, and / or a keyboard for receiving user commands. In some embodiments, user interface 240 includes a command line interface or a graphical user interface that may be presented to a remote terminal via communications interface 250. A user interface may be located with one or more other components of the system or may be located remotely from the system and communicate via a wired and / or wireless communications network.

[0076] Communications interface 250 may include one or more devices for enabling communication with other hardware devices. For example, communications interface 250 may include a network interface card (NIC) configured to communicate according to an Ethernet protocol. Additionally, communications interface 250 may implement a TCP / IP stack for communicating according to a TCP / IP protocol. Various alternative or additional hardware or configurations for communications interface 250 will be apparent.

[0077] Storage 260 may include one or more machine-readable storage media, such as read-only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, or similar storage media. In various embodiments, storage 260 may store instructions for execution by processor 220 or data on which processor 220 operates. For example, storage 260 stores operating system 261 for controlling various operations of system 200.

[0078] It will be apparent that various information described as being stored in storage 260 may additionally or alternatively be stored in memory 230. In this regard, memory 230 may also be considered to constitute a storage device, and storage 260 may be considered a memory. Various other arrangements will be apparent. Moreover, both memory 230 and storage 260 may be considered to be non-transitory machine-readable media. As used herein, the term "non-transitory" is meant to exclude transitory signals, but to include all forms of storage, including volatile and non-volatile memory.

[0079] Although system 200 is shown as including one of each of the described components, various components may overlap in various embodiments. For example, processor 220 may include multiple microprocessors configured to independently execute the methods described herein or to perform steps or subroutines of the methods described herein, such that the multiple processors cooperate to accomplish the functions described herein. Furthermore, when one or more components of system 200 are implemented in a cloud computing system, various hardware components may belong to separate physical systems. For example, processor 220 may include a first processor located in a first server and a second processor located in a second server. Many other variations and configurations are possible.

[0080] In an embodiment, storage 260 of system 200 may store one or more algorithms, modules, and / or instructions for performing one or more functions or steps of the methods described or otherwise contemplated herein. For example, the system may include ultrasound imaging data 262, feature identification algorithms 263, composite scoring algorithms 264, lung scoring algorithms 265, and / or reporting instructions 266, among other instructions and data, among many other possible instructions and / or data.

[0081] According to an embodiment, ultrasound imaging data 262 is any ultrasound imaging data transmitted to, acquired by, or otherwise received by the system. Ultrasound image data includes a temporal sequence of ultrasound image data, such as, for example, a video including multiple frames. Ultrasound image data may be acquired for one region or zone of the patient's lungs, or for multiple different zones. For example, ultrasound image data is acquired for one, two, or more zones. Some exams may include 14 or more zones, although more or fewer zones are possible. The ultrasound image data may be received by the system in real time, or may be stored in local and / or remote memory and received by the system at a future time. The ultrasound image data may be acquired using any ultrasound device or system, which may be any device or system suitable for acquiring or otherwise receiving ultrasound image data of a patient. The ultrasound device or system may be remote from, local to, or a component of the ultrasound analysis system 200.

[0082] According to an embodiment, the feature identification algorithm 263 is any model or algorithm trained or configured to use ultrasound image data to identify lung features, which may be anything that can be identified in or from the ultrasound image data. Examples of features include A-lines, B-lines, joined B-lines, pleural line abnormalities, consolidations, pleural effusions, and many others. According to an embodiment, the feature identification algorithm 263 is trained or configured to identify features from individual frames or videos.

[0083] According to an embodiment, the composite score algorithm 264 is any model or algorithm trained or configured to determine a composite score for the imaged zone, using one or more of the identified features and the determined confidence scores as inputs. The composite score determined for a zone includes or is otherwise indicative of the severity of the disease or condition in that zone. Additionally or alternatively, the composite score determined for a zone includes or is otherwise indicative of the diagnosis of the disease or condition in that zone. The composite score determined for a zone may include other information about the zone. According to an embodiment, the composite score algorithm derives a composite severity score and / or diagnosis for each frame or video of each zone. This may be a predictive model, such as a rule-based algorithm or a machine learning based algorithm. According to an embodiment, the output of the composite score algorithm includes a binary yes / no, a continuous measurement between 0 and 1, an integer severity scale (e.g., 0-3), or any other informative or useful output. The output may be a triage recommendation or an outcome prediction.

[0084] According to an embodiment, the lung score algorithm 265 is any model or algorithm trained or configured to determine a lung score, using as input a plurality of composite scores determined for a plurality of imaged lung zones. The determined lung score includes or is otherwise indicative of the severity of the patient's lung disease or condition. Additionally or alternatively, the determined lung score includes or is otherwise indicative of the diagnosis of the patient's lung disease or condition. Additionally or alternatively, the determined lung score may include other information about the lungs. Ultrasound exams typically include a plurality of imaged zones. This includes 14 or more zones, although more or less zones are possible. The lung score may be the sum or average of the composite scores determined for each lung zone. However, other approaches to deriving an overall exam-level lung score are possible, some of which are described or otherwise contemplated herein.

[0085] According to an embodiment, the reporting instructions 265 instruct the system to generate and provide a report or visualization to the user via the user interface 240 of the ultrasound analysis system 200. The report or visualization may include information regarding one or more of the following: the determined lung score, a composite score of one or more features for one or more of the imaged lung zones, and / or a confidence score for one or more of the identified features. Other information is also possible, including but not limited to the patient's identity, patient demographic information, diagnostic or treatment information, and a variety of other possible information. The information may be provided via the user interface using any method for communicating or displaying information, and the user interface may be any device, interface, or mechanism for providing communicated or displayed information. According to an embodiment, the instructions may instruct the system to display the information on a user interface or display of the system. The report may be communicated to another device by wired and / or wireless communication. For example, the system may communicate the report to a mobile phone, a computer, a laptop, a wearable device, and / or other device configured to enable display of the report or other communication.

[0086] A confidence score is output by a feature identification algorithm. Many feature identification algorithms output a confidence score for their output. The confidence score indicates the "confidence" the algorithm has in the output. In other words, the confidence score indicates the likelihood that the identified feature is in fact the desired feature. Of course, other algorithms may be used to determine the confidence score. The confidence score may be based, for example, on the quality of the acquired image (e.g., features identified in a blurry image will have a relatively low confidence).

[0087] The confidence score generally provides a score indicative of the features of a given zone. The confidence score may also be referred to as a per-zonal feature prediction. Of course, the feature confidence score may be based on many per-frame per-feature confidence scores (i.e., each feature identified at each frame in the temporal sequence is given a per-frame per-feature confidence score). The per-frame per-feature confidence scores for a given zone may then be combined to determine a per-zonal feature confidence score. The per-zonal per-feature confidence scores for all features within the zone may then be combined to generate a per-zonal composite score. Of course, the per-zonal composite scores for each zone may also be further combined to determine a final lung score.

[0088] Note that the confidence scores for all zones (per feature per zone) corresponding to a feature can also be combined to determine the final feature score for each feature.

[0089] Also note that the confidence scores for all zones and all features [per feature per zone] can be combined to generate a final lung score.

[0090] Therefore, many confidence scores (eg, the scores for each feature in each frame) can be reduced to a single final lung score (or a relatively small set of final feature scores).

[0091] The size of the identified feature may be the size of the feature relative to the image itself. In general, the larger the size of the identified feature, the more likely the feature is present. Thus, the size of the identified feature provides a pseudo-confidence score for the identified feature. Given that many current feature identification algorithms output a bounding box for the identified feature, the size may also be the size of the bounding box.

[0092] It will be appreciated that the above method and system can also be used in other types of ultrasound examinations (other than lung ultrasound examinations). For example, cardiac and abdominal ultrasound examinations often require imaging of different zones of an organ. Hence, the confidence scores [per feature per zone] of these organs can be found and combined to generate a final cardiac / abdominal score.

[0093] All definitions and those used herein should be understood to take precedence over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0094] The singular terms "a," "an," and "the" as used in the specification and claims should be understood to mean "at least one."

[0095] The term "and / or" as used in the specification and claims should be understood to mean "either or both" of the elements so conjoined (i.e., elements that may be conjunctive or disjunctive). Multiple elements listed with "and / or" should be construed in the same manner, i.e., "one or more" of the elements so conjoined. Other elements other than those specifically identified by the "and / or" clause may optionally be present, whether or not related to the specifically identified element.

[0096] As used herein and in the claims, "or" should be understood to have the same meaning as "and / or" as defined above. For example, when separating items in a list, "or" or "and / or" should be interpreted as inclusive; that is, the inclusion of at least one of an element or list of elements, but including more than one element, and optionally additional unlisted items. Only terms clearly indicating the contrary (e.g., "only one of" or "exactly one of," or, when used in the claims, "consisting of") mean the inclusion of exactly one element of an element or list of elements. In general, as used herein, the term "or" should only be interpreted as indicating exclusive alternatives (i.e., "either one of," "one of," "only one of," or "exactly one of") when preceded by terms of exclusivity (e.g., "either one of," "one of," "only one of," or "exactly one of").

[0097] As used in this specification and claims, the phrase "at least one" in reference to a list of one or more elements should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but does not require the inclusion of at least one of each element specifically listed in the list of elements, nor does it exclude any combination of elements in the list of elements. By this definition, elements other than those identified in the list of elements to which the phrase "at least one" refers may optionally be present, whether related to the identified elements or not.

[0098] It should also be understood that, unless otherwise specified, in any method claimed herein that includes two or more steps or actions, the order of the method steps or actions is not necessarily limited to the order in which the method steps or actions are described.

[0099] In the above specification, as well as in the claims, all transitional phrases such as "comprises," "includes," "carries," "has," "contains," "accompanying," "holds," "consisting of," and the like, are to be understood to be open ended, meaning "including, but not limited to." Only the transitional phrases "consisting of" and "consisting essentially of" are closed or semi-closed transitional phrases, as set forth in Section 2111.03 of the United States Patent Office Manual of Patent Examining Procedures.

[0100] Although several inventive embodiments have been described and illustrated herein, those skilled in the art will readily envision numerous other means and structures for performing the functions and obtaining the results and / or advantages of one or more of the embodiments described herein. Each of these variations and modifications is deemed to be within the scope of the inventive embodiments described herein. More generally, those skilled in the art will readily appreciate that all parameters, dimensions, materials, and configurations described herein are exemplary, and that the actual parameters, dimensions, materials, and / or configurations will depend on the particular application in which the teachings of the present invention are being used. Those skilled in the art will recognize or be able to ascertain, using no more than routine experimentation, many equivalents to the specific inventive embodiments described herein. Thus, the foregoing embodiments are presented by way of example only, and it should be understood that, within the scope of the appended claims and equivalents, the inventive embodiments may be practiced otherwise than as specifically described and claimed. Inventive embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and / or method described herein. Furthermore, any combination of two or more such features, systems, articles, materials, kits, and / or methods is within the inventive scope of the present disclosure, unless such features, systems, articles, materials, kits, and / or methods are mutually inconsistent.

Claims

1. 1. A method for generating an ultrasound score, comprising: receiving a temporal sequence of ultrasound image data having a plurality of frames acquired from an ultrasound examination of a patient, for each of a plurality of different zones of one or both lungs of the patient; identifying one or more features of the lung for each of the plurality of different zones from the temporal sequence of the ultrasound image data using a feature identification algorithm; determining a confidence score for each of the one or more identified features, the confidence score for the identified feature indicating the likelihood that the identified feature was correctly identified by the feature identification algorithm; determining a composite score for each of the plurality of different zones by combining the determined confidence scores corresponding to each zone, wherein the composite score for a zone indicates either (i) the severity of the disease or condition of the zone, or (ii) a diagnosis of the zone; determining a pulmonary score by combining a plurality of the determined composite scores, the pulmonary score indicating either (i) the severity of the pulmonary disease or condition, or (ii) the pulmonary diagnosis; providing, via a user interface, one or more of the lung score, the composite score for one or more of the plurality of different zones, and the confidence score for one or more of the identified one or more features; A method comprising:

2. The method of claim 1 , wherein determining the composite score is further based on a determined size of the identified feature relative to a size of the frame in which the feature was identified.

3. The method of claim 1 , further comprising determining a score for each identified feature by combining the confidence scores corresponding to each identified feature.

4. The method described in claim 1, wherein the reliability score is determined based on an aggregation of frame-by-frame reliability scores determined for each frame of the plurality of frames.

5. The method of claim 1 , wherein identifying one or more features of the lung comprises determining the presence or absence of the features.

6. 10. The method of claim 1, further comprising weighting one or more of the composite scores, wherein a pulmonary score algorithm utilizes the weighted composite scores when determining the pulmonary score.

7. receiving, via the user interface, a request for additional information regarding a user-selected lung zone; providing, via the user interface, the requested additional information regarding the user-selected lung zone, the additional information including one or more of the temporal sequence of ultrasound image data for the user-selected lung zone, the composite score for the user-selected lung zone, and the confidence score for the user-selected lung zone; The method of claim 1 further comprising:

8. 1. A method for generating an ultrasound score, comprising: receiving a temporal sequence of ultrasound image data acquired from an ultrasound examination of a patient for each of a plurality of different zones of one or both lungs of the patient; identifying one or more features of the lung for each of the plurality of different zones from the temporal sequence of the ultrasound image data using a feature identification algorithm; determining a confidence score for each of the one or more identified features, the confidence score for the identified feature indicating the likelihood that the identified feature was correctly identified by the feature identification algorithm; determining a pulmonary score by combining the determined confidence scores for all zones, the pulmonary score indicating either (i) the severity of the pulmonary disease or condition, or (ii) the pulmonary diagnosis; providing, via a user interface, one or more of the pulmonary scores and the confidence scores for one or more of the identified one or more features; A method comprising:

9. 1. A method for generating an ultrasound score, comprising: receiving a temporal sequence of ultrasound image data having a plurality of frames acquired from an ultrasound examination of a patient, for each of a plurality of different zones of one or both lungs of the patient; identifying one or more features of the lung for each of the plurality of different zones from the temporal sequence of the ultrasound image data using a feature identification algorithm; for each of the one or more identified features, determining a size of the identified feature relative to a size of the frame in which the feature is identified; determining a pulmonary score by combining the determined sizes for all zones, the pulmonary score indicating either (i) the severity of the pulmonary disease or condition, or (ii) the pulmonary diagnosis; providing, via a user interface, one or more of the pulmonary score and the confidence score for one or more of the identified one or more features; A method comprising:

10. 1. An ultrasound analysis system for generating an ultrasound examination score, comprising: a temporal sequence of ultrasound image data having a plurality of frames for each of a plurality of different zones of one or both lungs of a patient; a processor configured to: (i) identify one or more features of the lung for each of the plurality of different zones from the time sequence of the ultrasound image data using a feature identification algorithm; (ii) determine a confidence score for each of the one or more identified features, wherein the confidence score for the identified feature indicates a likelihood that the identified feature was correctly identified by the feature identification algorithm; (iii) determine a composite score for each of the plurality of different zones by combining the determined confidence scores corresponding to each zone, wherein the composite score for a zone indicates either (a) a severity of a disease or condition of the zone, or (b) a diagnosis of the zone; and (iv) determine a pulmonary score by combining a plurality of the determined composite scores, wherein the pulmonary score indicates either (a) a severity of the disease or condition of the lung, or (b) a diagnosis of the lung; a user interface that provides one or more of the lung score, the composite score for one or more of the plurality of different zones, and the confidence score for one or more of the identified features; An ultrasound analysis system comprising:

11. The ultrasound analysis system of claim 10 , wherein the processor determines the composite score further based on a determined size of the identified feature relative to a size of the frame in which the feature was identified.

12. The ultrasound analysis system of claim 10 , wherein the processor further determines a score for each identified feature by combining the confidence scores corresponding to each identified feature.

13. The ultrasound analysis system of claim 10, wherein the reliability score is determined based on an aggregation of frame-by-frame reliability scores determined for each frame of the plurality of frames.

14. 11. The ultrasound analysis system of claim 10, wherein the processor further weights one or more of the composite scores, and a pulmonary scoring algorithm utilizes the weighted composite scores when determining the pulmonary score.

15. 11. The ultrasound analysis system of claim 10, wherein the processor is further configured to receive a request for additional information regarding a user-selected lung zone via the user interface and to provide the requested additional information regarding the user-selected lung zone via the user interface, the additional information including one or more of the temporal sequence of ultrasound image data for the user-selected lung zone, the composite score for the user-selected lung zone, and the confidence score for the user-selected lung zone.

Images