Drone assisted rapid triage

Abstract

Different technologies related to the identification and triage of injured persons are disclosed. Drones and other unmanned autonomous vehicles are used to collect information about the physiological status of each injured person. That information is sent to a control station, which creates a triaged casualty list using one or more algorithms. The triage list is then communicated to and used by first responders.

Description

DRONE ASSISTED RAPID TRIAGECROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Patent Application Serial No. 63 / 537,991 , filed on September 12, 2023, which is incorporated by reference in its entirety.BACKGROUND

[0002] The present disclosure relates to care, triage, and assessment of injured persons, to improved triage and delivery of life-saving intervention (LSI) in mass casualty incidents (MCls) such as battlefields, natural disasters such as hurricanes or earthquakes, other types of disasters such as high-rise building collapse, and / or so forth.

[0003] Mass casualty incidents (MCI) are events where the number of patients exceeds the amount of healthcare resources available, especially in the number of personnel. Triage and delivery of medical assistance at disaster scenes or military battlefields is challenging, due to factors such as the unpredictable location of such events and the potential for mass casualties with a wide range of possible injury types, such as blunt trauma, gunshot, burns, traumatic brain injuries, inhalation of toxic chemical and / or biohazardous biological agents, and / or so forth.

[0004] Although triage protocols specific to mass casualty incidents (MCls) have been adopted, such as Sort, Assess, Lifesaving Interventions, Treatment / Transport (SALT), these practices can be inadequate for the secondary period since casualties assumed to be stable are often left unmonitored, due to the need to allocate those scarce medical personnel to more immediate priorities. Even in situations where personnel can be allocated to monitor these patients, those that perform this task often can only depend on physiological signatures, such as heart rate, respiration, oxygen saturation, or temperature, to assess each patient’s state when attempting to observe many casualties.

[0005] Trained first responders may not have the bandwidth to monitor many individuals simultaneously for subtle changes in physiological signatures indicating the need for an LSI. The sheer volume of information from multiple data streams from each casualty can oversaturate medical personnel attempting to monitor many differentindividuals, and it is difficult to track trends across different time scales. Alarms on monitoring instruments are employed to alert monitoring personnel once a single value moves outside an acceptable range. However, these signals fail to account for the holistic integration of all data streams and are often ignored in practice by medical personnel due to frequent false positives (i.e. , alarm fatigue). These challenges are compounded by the potential chaos of an MCI environment, such as a battlefield or disaster area, or the tasks of monitoring patients during transport to a higher level of care. These factors may evolve across time, creating high-stress, emotional, and even disorienting circumstances that impair objective decision making.

[0006] To maximize health outcomes, the patient should be monitored during initial assessment, evacuation, while in transit, upon arrival at field hospitals or medical centers, during treatment, and in recovery. Continuous monitoring enables rapid response interventions following unexpected events and patient health deterioration. Even at higher levels of care, hospitals can become overwhelmed with casualties, and resources can be misallocated leading to delayed or absent LSIs.

[0007] The present disclosure proposes solutions to overcome these problems and others.BRIEF DESCRIPTION

[0008] Multiple different technologies are described herein for addressing such issues in the civilian and military context related to treatment of multiple persons and leveraging technology to provide timely information to improve care. Besides the battlefield, these may be useful for injuries in other hazardous conditions, such as fires, floods, hurricanes, earthquakes, etc.

[0009] Various embodiments relate to methods for performing triage, comprising: capturing physiological signatures of casualties using standoff measurement sensors mounted on one or more unmanned aerial vehicles; sending the physiological signatures of the casualties from the one or more unmanned aerial vehicles to a control station for prioritizing medical response; creating a triaged casualty list using an algorithm; and communicating the triaged casualty list to first responders.

[0010] Also disclosed are various methods for performing triage, comprising: capturing vital signs of one or more persons using a monitoring system; sending the vital signs to a control station; producing a triage report using an algorithm; and displaying the triage report.

[0011] These and other non-limiting aspects and / or objects of the disclosure are more particularly described below.BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The following is a brief description of the drawings, which are presented for the purposes of illustrating the exemplary embodiments disclosed herein and not for the purposes of limiting the same. Various features are not drawn to scale, and their dimensions may be arbitrarily increased or reduced for clarity of discussion.

[0013] FIG. 1 is an illustrative depiction of a mass casualty incident scene and various components of the DART system, in accordance with some embodiments of the present disclosure.

[0014] FIG. 2A and FIG. 2B together form a flow chart illustrating a first method for performing triage, in accordance with some embodiments of the present disclosure.

[0015] FIGS. 3A-3C illustrate three different search strategies which could be used by drones to locate casualties.

[0016] FIG. 4 is a flow chart illustrating a model for searching and locating casualties, in accordance with some embodiments of the present disclosure.

[0017] FIG. 5 is an example architecture for one or more control stations used to control drones, in accordance with some embodiments of the present disclosure.

[0018] FIG. 6 is an example illustrating showing a method for detecting blood loss from an image, in accordance with some embodiments of the present disclosure.

[0019] FIG. 7 is an illustration showing a heart rate determined from an image of a face, in accordance with some embodiments of the present disclosure.

[0020] FIG. 8 is a flow chart illustrating an algorithm for determining a triage status of a casualty, in accordance with some embodiments of the present disclosure.DETAILED DESCRIPTION

[0021] A more complete understanding of the processes and apparatuses disclosed herein can be obtained by reference to the accompanying drawings. These figures are merely schematic representations based on convenience and the ease of demonstrating the existing art and / or the present development, and are, therefore, not intended to indicate relative size and dimensions of the assemblies or components thereof.

[0022] Although specific terms are used in the following description for the sake of clarity, these terms are intended to refer only to the particular structure of the embodiments selected for illustration in the drawings, and are not intended to define or limit the scope of the disclosure. In the drawings and the following description below, it is to be understood that like numeric designations refer to components of like function.

[0023] Spatially relative terms, such as “beneath,” “below,” “lower,” “above,” “upper” and the like, are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. The devices may be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein may likewise be interpreted accordingly.

[0024] Numerical values in the specification and claims of this application should be understood to include numerical values which are the same when reduced to the same number of significant figures and numerical values which differ from the stated value by less than the experimental error of conventional measurement technique of the type described in the present application to determine the value. All ranges disclosed herein are inclusive of the recited endpoint.

[0025] The modifier "about" used in connection with a quantity is inclusive of the stated value and has the meaning dictated by the context (for example, it includes at least the degree of error associated with the measurement of the particular quantity). When used with a specific value, it should also be considered as disclosing that value. For example, the term “about 2” also discloses the value “2” and the range “from about 2 to about 4” also discloses the range “from 2 to 4.” The term “about” may refer to plus or minus 10% of the indicated number.

[0026] The term “drone” is used herein to refer to a self-propelled robot which can travel either through the air or on the ground. Examples of drones include unmanned aerial vehicles (LIAVs) and unmanned ground vehicles (UGVs). A drone may be capableof autonomous propulsion, and may accept periodic human commands to perform various tasks.

[0027] The present disclosure relates to Drone Assisted Rapid Triage (DART) systems that support first responder triage decisions following MCls by collecting as much information as possible and utilizing that data to support resource-optimized decisionmaking. Autonomous systems are used to search for, localize, and assess casualties. Standoff sensors on uncrewed ground and aerial autonomous systems will capture the physiological signatures of casualties during an MCI, and through sensor fusion will make an assessment of each casualty’s underlying physiological state. Detection of hard-to- find or obscured casualties can be more easily accomplished through the coordination of multiple sensor delivery platforms, enabled by autonomous control. Information regarding casualties can be synthesized, screened for the need for life saving interventions (LSIs), and presented to first responders in a clear and concise format to maximize medical resources and expedite triage decision making.

[0028] Initially, FIG. 1 diagrammatically illustrates a scene 100 of a mass casualty incident (MCI) undergoing initial triage using drones such as unmanned aerial vehicles (UAVs) 102 or unmanned ground vehicles (UGVs) 104. The MCI scene 100 could be the scene of a natural disaster such as an earthquake, hurricane landfall, flood, tsunami landfall, or the like. The MCI scene 100 could alternatively be the scene of a disaster in an urban environment such as a large-scale fire, a building collapse, or so forth. The MCI scene 100 could alternatively be a battlefield. These are merely some nonlimiting illustrative examples.

[0029] As diagrammatically indicated in the upper enlarged UAV image, each UAV 102 includes sensors 106, 108, 110 that provide various sensor signals for locating casualties 112 in the MCI scene 100 and for performing triage of such casualties. Although not shown, each UGV 104 could also include such sensors. By way of some nonlimiting illustrative examples, the sensor 106 may be an acoustic sensor such as a microphone, a phased acoustic array operating in the audible frequency range, a phased ultrasound (US) array operating in ultrasonic frequencies, various combinations thereof, or so forth. The sensor 108 may be a radar, such as an ultrawide-band (UWB) radar. Thesensor 110 may be an imaging device such as a video camera, electro-optical (EO) sensor, an infrared (IR) sensor, or combination thereof.

[0030] The sensor data are wirelessly transmitted to a control station 120, which in the illustrative example is a pair of mobile control stations deployed in respective vans or other emergency response vehicles. One or more algorithms are used to perform triage. Operators 114, such as emergency response personnel, can use user interfacing devices 122 to interact with the control station 120. For example, inset 116 illustrates an electronic tablet or cellphone 122.

[0031] FIG. 2 is a flow chart illustrating an illustrative method 200 for performing triage using a DART system. It is noted that not all steps described in the flow chart are required, and other steps may occur that are not described in the flow chart.

[0032] It is contemplated that the triage is needed due to a mass casualty incident (MCI). In step 202 of FIG. 2A, a search strategy is developed. This may include steps such as updating maps (step 204) and identifying areas of high casualty likelihood (step 206). As a result, a computerized map containing known data is created and continuously updated during the triage process.

[0033] Prior information / data concerning the location of the MCI can be obtained. This may include, for example, a general description of the disaster and any known geographic or demographic data (e.g., topographical GIS data, maps, weather conditions, population). The DART system can then digest this information to develop an optimized search strategy. For example, a list of zones within the search area can be developed to be initially prioritized. As one example, casualties of a large fire are more likely to be found distributed throughout the entire search area, whereas casualties of a flood are more likely to be found in locations downstream of the water source. As another example, a location with many tall trees is more likely to need to be searched by UGVs rather than UAVs whose flight paths could be interdicted or whose cameras might not penetrate through foliage. As a third example, a large open park may only need one drone to cover the area, whereas a collapsed building structure may require multiple drones for complete coverage.

[0034] In step 208 of FIG. 2A, search routes are planned for each drone. This may include creating / setting waypoints for each drone to follow along its search route. Thesearch strategy is used to determine the number of UAVs and UGVs to deploy for specific MCls, and how they should be deployed. Waypoints may be weighted based on the likelihood of finding one or multiple casualties. Waypoints could be sorted into three categories: Waypoints Accessible Only by UGVs, Waypoints Accessible Only by UAVs, and Waypoints Accessible by Any Sensor Delivery Platform. This differentiation will allow for the optimization of path planning when routing specific types of drone platforms to specific waypoints. It is noted that steps 202-208 can be performed in an automated manner using computer algorithms along with manual operator input and feedback.

[0035] FIGS. 3A-3C illustrate three different search methods which could be used for locating casualties. FIG. 3A illustrates a Standard search method. Drones 101 form a line and move from waypoint 124 to waypoint 124. FIG. 3B illustrates a Spiral search method, in which a drone 101 starts at the center of a point of interest and follows a spiral path (dashed line) outwards for a predetermined distance. This permits in-depth analysis of the point of interest. For example, the point of interest could be a location or cluster where high numbers of casualties are expected to be located, and the Spiral search method ensures all casualties in that area are localized. The other two drones are illustrated following a Standard search path. FIG. 3C illustrates a Scatter search method. Here, the drones 101 are each sent in a pseudorandom path. Each drone then “repels” each other, to avoid areas previously searched by another drone during execution of the search method. It is noted that UAVs and UGVs may be partially or completely decoupled from each other with respect to their search routes, or in other words their routes may or may not overlap. For example, UAVs may be tasked to search areas not easily accessed by UGVs, such as upper floors of a building structure or areas separated by water.

[0036] FIG. 4 is a flow chart illustrating a model 270 for searching and locating casualties, in accordance with some embodiments of the present disclosure. In this model, one or more UAVs are programmed to act as “oversight” or “Eye in the Sky” (EITS) platforms. It is contemplated that such oversight platforms will climb to a high operating altitude and orbit the search area, and will have a view of a large area. These EITS platform(s) can update any maps using onboard sensors, and populate physical changes to the MCI area indicative of high casualty likelihood. For example, they may be able to identify collapsed buildings, a plane that crash landed, a car wreck, a field that has caughtfire, etc. Individual casualties and clusters of casualties may also be identified. This information can be used to populate waypoints for “seeker” platforms to explore further, prioritizing regions with high casualty likelihood. Seeker platforms will include both UAVs and LIGVs that are capable of efficiently navigating close to a casualty to assess their physiological state and any injuries using multi-modal standoff sensors. This model allows for efficient coverage of a large area as well as navigating difficult terrain and obstacles to quickly reach casualties.

[0037] In step 216 of FIG. 4, drones are initialized. This is discussed in further detail herein, with respect to FIG. 2A. In step 202 of FIG. 4, an initial search strategy is developed. As indicated in step 208, a drone deployment strategy is developed, in which routes are planned for each drone. In step 218, drones are launched. As indicated in step 272, EITS platforms will orbit the search area. In step 274, the data obtained from their overwatch is used to generate or modify the waypoints initially selected in steps 202 and 208. In step 276, the search paths of the seeker platforms are dynamically modified based on the updated information from the EITS platforms. In step 278, casualties are searched for and localized. Following initial waypoint population, EITS platforms will continue to orbit and update mapping and waypoints to account for dynamic activities. This feedback is also indicated with arrow 219 in FIG. 2A.

[0038] For example, UAV seeker platforms will execute on paths generated from Waypoints Accessible Only by UAVs and Waypoints Accessible by Any Sensor Delivery Platform. They may start with the Standard search method of FIG. 3A until casualties are localized. If more than one casualty is discovered en route to a waypoint, a casualty cluster may be declared, and multiple UAVs can be diverted to immediately scan the region and assess individual casualties. Once all casualties immediately identified have been assessed, the first available UAV will execute the Spiral search method of FIG. 3B around the casualty cluster while the remaining UAVs will resume the Standard search method. Once all waypoints are assessed, UAVs can execute the Scatter search method of FIG. 3C. This process can repeat as new waypoints are identified by the EITS platforms.

[0039] UGV seeker platforms will execute similar behavior to UAVs, but follow paths generated from the Waypoints Accessible Only by UGVs and Waypoints Accessible byAny Sensor Delivery Platform. Waypoints only accessible by land would be prioritized, and casualties and casualty clusters would be assessed during the UGV traversal of their planned path.

[0040] Continuing, as indicated in step 210 of FIG. 2A, equipment is physically deployed to the area of the MCI. Referring back to FIG. 1 , such equipment may include, for example, drones such as UAVs 102 and UGVs 104, personnel 114, control stations 120, and user interfaces 122. In optional step 212 of FIG. 2A, one or more standalone Position I Navigation / Timing (PNT) beacons may be deployed to define a search area for the drone(s), and / or to be used as known location references in the event other PNT systems (such as GPS) are not usable.

[0041] In step 214 of FIG. 2A, one or more control stations 120 are set up. FIG. 5 illustrates one embodiment of a potential control system architecture. As illustrated here, each UAV 102 and UGV 104 has a dedicated control station 126, along with other support equipment. It is contemplated the dedicated control station for each drone will collect telemetry and standoff sensor data, and run data processing and detection algorithms specific to the type and role of its linked drone. Each dedicated control station 126 communicates with a central control station 128, which can also communicate with user interface(s) 122. The central control station 128 will collate the information from each dedicated control station, and run data processing and casualty triage algorithms. This architecture permits the gathering and processing of data both to and from the drones. This node-based architecture also reduces technical risk by segmenting data management and algorithm functions, allowing for parallel development.

[0042] Generally speaking, each control station will include a processor which can execute processing instructions stored in memory connected to the processor. The processor can be variously embodied, such as by a single core processor, a dual core processor (or more generally by a multiple core processor), a digital processor and cooperating math coprocessor, a digital controller, or the like. The memory may be any type of non-transitory computer readable medium such as random access memory (RAM), read only memory (ROM), magnetic disk or tape, optical disk, flash memory, or holographic memory. The processor and memory may be combined in a single chip.Communications devices such as radio controllers, network interfaces, or wireless routers may be used as well.

[0043] In step 216 of FIG. 2A, each drone is initialized. This may include, for example, ensuring communication with its dedicated ground station, performing sensor calibration, and uploading its search route. The drones are then ready for deployment. It is noted that steps 202-216 could be performed in any order at any desired time.

[0044] Continuing, then, in step 218 of FIG. 2A, each drone is deployed to begin searching for casualties. For purposes of this disclosure, the term “casualty” is used to refer to any person who may be located within the search area, regardless of whether the person is actually injured or not.

[0045] Each drone desirably has performance characteristics suitable for search and localization. For example, each drone includes a GPS transmitter for tracking its own location, which permits the location of casualties to be marked on a map as well. Each drone should have a minimum operating lifetime, for example of at least one hour. Each drone should have a minimum operating range, both in distance from its control station and possibly in height (for AGVs). For example, an operating range of greater than one kilometer or two kilometers or up to five kilometers or even 10 kilometers is desirable, as is an operating ceiling of up to 300 meters or higher. A high payload capability may be desirable too, for dropping off supplies if desired. Each drone should also include a communications capability, and can be controlled remotely or operate autonomously. UGVs may use treads or wheels for mobility as desired.

[0046] Depending on the operating environment, the drone itself may be built for survivability. For example, in a military theater of operations, the drone might include advanced materials such as reactive armor or other lightweight protection. It may include a booster system to exit the combat area quickly. Advanced countermeasures might be included to defeat radar-guided, heat-guided, or other types of guided munitions. The drone may be designed for continued flight after taking some specified amount of damage. Flight time and Time on Station might be improved with the use of solar arrays that would allow for charging in flight. Drones could be deployed from a “mother ship” that releases and recovers smaller drones.

[0047] Generally speaking, as illustrated in FIG. 1 , each drone carries one or more sensors, such as an acoustic sensor 106, a radar 108, and an imaging device 110. Each sensor is appropriate for different measurements. For example, a camera can provide high spatial and temporal resolution over long ranges. An IR sensor is less sensitive to lighting conditions, has long range, and can directly image heat sources like body heat and / or breaths. Acoustic sensors such as speakers and microphones can generate sounds that can penetrate barriers, and enable direct communication with the casualty. Radar can also penetrate barriers, and is relatively insensitive to environmental conditions. These sensors can also provide data that can be processed to determine various physiological signatures, as will be discussed further herein.

[0048] In step 220 of FIG. 2A, casualties are identified. This may be done using onboard EO, IR, radar, and / or phased acoustic array sensors that provide data to realtime object detection models. For example, a real-time object detection model (such as YOLO) that is trained to identify people in an image could be used to detect potential casualties in EO and IR data streams. Thermal signatures could also be used to identify humans. Phased acoustic array sensors could identify sounds associated with human activity and distress. Radar can be used to detect breathing and heartbeats. It is contemplated that each casualty will be assigned a de-identifiable code and associated information will be stored in a database and used to prevent duplicative casualty assessment. It is generally contemplated that the standoff sensors on each drone may be most useful when used within a distance of about 15 feet.

[0049] Identification by multiple different sensors increases the confidence that a potential casualty is identified (versus being a “false positive”). In addition, images from multiple perspectives I angles can further increase confidence in casualty identification. For example, a given casualty might be obscured from one side, but visible from the other side. If image processing flags a potential casualty, the path of a drone may be changed to obtain an image from another angle, which would provide further information to more definitively determine whether a casualty is present or not. It is contemplated that false positives would be flagged geospatially to prevent repeated interrogation I effort.

[0050] As each casualty is identified, s / he is assessed and triaged. In step 230 of FIG. 2B, data, such as vital signs, are captured for each casualty. Steps 232-252 describeseveral different types of data and physiological characteristics that can be captured and determined. It is noted that not all of these steps must be performed. It is contemplated that desirably, the time needed for assessment of each casualty may be reduced to a time of, for example, about 30 seconds to about 60 seconds, or up to about 2 minutes.

[0051] In step 232 of FIG. 2B, the location of the casualty is identified. The location information is logged and used to update maps.

[0052] In optional step 234 of FIG. 2B, the casualty may be engaged to determine their responsiveness. For example, signatures of responsiveness will be measured using arm movement, blink, and speech detection. Signatures of mobility can be used to evaluate whether the casualty is mobile using real-time pose estimation algorithms. This information can help determine if further assessment is required. EO and IR sensors carried by the drone can be used to perform motion capture, and a speaker and microphone can be used to perform audio-based communication. The drone can use an introductory dialogue with the casualty to humanize itself and build trust. After introducing itself as a medical assistant, the drone can ask a series of response-based questions, such as move your arm if you are okay, blink if you understand, are you injured or in pain?, etc. Visual and / or auditory responses (or lack thereof) can be computationally analyzed. For example, a motion capture I pose estimation algorithm can be used to interpret arm movement and blink responses. A speech-to-text based natural language processing (NLP) algorithm can be used to understand verbal responses. Where faces are exposed and detectable, EO and IR can be used to measure blink responses and eyelid movement. Language models can be used on recorded audio to extract selfevaluating injury descriptions from the casualty being assessed, for example using open- source pretrained language models like Bidirectional Encoder Representation from Transformers. First responders and system controllers could also be able to perform two- way auditory interactions with the casualty using the microphones and speakers if desired.

[0053] In step 236 of FIG. 2B, the body temperature of the casualty can be measured. This may be done, for example, directly by a thermal or IR sensor.

[0054] In step 238 of FIG. 2B, the casualty is tracked to maintain their identity and their location. This may be done, for example, using an EITS platform as indicated inFIG. 4. Tracking of all casualties and non-casualty personnel is important. First, ambulatory casualties tend to move and / or congregate during an MCI and maintaining confident identification of previously assessed casualties avoids re-assessment of already-assessed casualties. Secondly, as first responders begin to simultaneously provide treatment, they will need to be identified and tracked to avoid diverting drones to assess them as potential casualties. This data capture is continuously performed, as indicated by arrow 239 of FIG. 2B.

[0055] Some physiological characteristics may be determined by processing of visual images and / or other captured data. It is contemplated that this processing may be performed at a control station 126, 128. In step 240 of FIG. 2B, the data captured for each casualty is sent to a control station. In step 242 of FIG. 2B, the control station processes the data to determine one or more physiological signatures of a given casualty.

[0056] Some physiological signatures of interest include those in the MARCH acronym. Those include Massive hemorrhage, Airway management, Respiratory, Circulation, and Hypothermia / Head injuries. These signatures indicate a need for immediate LSIs that medics are trained and equipped to deliver in the field.

[0057] In step 244 of FIG. 2B, whether the casualty is hemorrhaging or has hemorrhaged is determined. This may also include an estimate of how much blood loss has occurred. Referring to FIG. 6, signatures of blood loss can be measured via EO and IR imaging and used to evaluate if the casualty is experiencing a massive hemorrhage. Using computer vision techniques, an anatomical region of interest can be defined around the casualty. Instance segmentation can be performed on EO and IR images to identify and mask the casualty. Instance segmentation can be performed using open-source computer vision software. Reflectance spectroscopy is then performed on the masked anatomical region to detect regions of blood. As indicated in FIG. 6, blood has a unique spectral signature in the IR spectrum. With that information, a blood area percentage can be computed to infer percent blood loss and if a massive hemorrhage occurred. This analysis can be performed across multiple frames in time (provided by EO and IR sensors) to determine if blood loss is increasing or remaining stable.

[0058] In step 246 of FIG. 2B, the respiration rate and / or chest movement of the casualty is measured. Video captured by EO and IR sensors can be broken down into individual frames. This corresponds to the AR steps of the MARCH acronym.

[0059] Signatures of respiration can be measured via both chest movement and breath to ensure an accurate measurement independent of environmental conditions. Features are extracted from each frame, and then boosted to increase the signal. Chest movement can be measured using both EO and IR based pixel motion and UWB based time-of-flight (ToF) or Doppler measurements. A region of interest (Rol) model is used to identify the chest. Oscillations in chest movement can then be used to calculate respiration rate (RR). Such methods generally perform well regardless of differences in lighting, clothing, background noise, and / or body position.

[0060] Breath detection can be measured using both IR and acoustic phased arrays. Breath oscillations can also be used to calculate RR. IR images can show the mouth or nostrils, where thermal deviations occur during respiration. Acoustic sensors may permit fixation on audio near the mouth, where exhale / inhale sounds can be processed to extract breath frequency and thus determine the RR. Wheezing could be identified in the frequency spectrum of decomposed audio recordings of breathing. Sucking chest wounds could be identified using similar IR oscillation methods imaged at the thoracic region to identify thermal signatures of cavity airflow in and out of the chest.

[0061] In step 248 of FIG. 2B, the heart rate of the casualty is measured. This corresponds to the C step of the MARCH acronym. Signatures of circulation can be measured via remote photoplethysmography (rPPG) and heart movement. Heart rate (HR) can be extracted using both techniques. As illustrated in FIG. 7, rPPG can be measured using EO and IR imaging of the patient’s face (in scenarios of facial exposure). The face can be identified using a facial detection algorithm, and rPPG can be extracted using pixel modulation analysis of the RGB values.

[0062] In step 250 of FIG. 2B, estimates of wound severity is made. For example, convolutional neural network (CNN) models can be trained on wound images to classify specific superficial injuries (e.g., wounds, burns), and such models can then be used on images obtained from the drones. This corresponds to the H step of the MARCH acronym.

[0063] In step 252 of FIG. 2B, an estimate of whether the casualty is suffering from hypothermia is made. This also corresponds to the H step of the MARCH acronym. Signatures of hypothermia can be measured via body temperature, physical tremors, and audible shivering. Thermal signatures at the corner of the eye and forehead can be obtained using IR sensors. Rapid movement of the limbs can be detected using EO and IR based pose-estimation methods. Audible shivering at the mouth may be detectable using acoustic phased array-based frequency decomposition methods. These can be input into an algorithm that provides an estimate of whether or not the casualty is suffering hypothermia.

[0064] In step 253 of FIG. 2B, the physiological signatures determined by the control station(s) from the processed data is sent to the triage algorithm. This seamlessly collates the data from all drone sensors into a centralized algorithm that continually evaluates and prioritizes casualty response needs as the situation evolves. Again, such data processing occurs continuously, as indicated by arrow 257 in FIG. 2B.

[0065] In step 254 of FIG. 2B, available resources are inputted into the triage system. Maximizing the distribution of such resources is one purpose of triage, with the goal of saving as many lives as possible. Such resources may also change over time, for example as more first responders / equipment arrive. Examples of such resources may include, for example, the number of first responders; the type of first responder (e.g. EMT vs. paramedic vs. MD); numbers of particular types of equipment (e g. numbers of portable X-ray units, ambulances, units of blood, etc.); and / or the location of equipment.

[0066] In step 260 of FIG. 2B, the triage status of the casualty is determined. Based on the physiological signatures extracted from individual casualties localized during the casualty search, a primary triage algorithm will categorize and prioritize care for the first responders. A core static triage model with an additional dynamic elements layer allows contextual responses with improved accuracy. The DART system’s individual assessment procedures leverage parallel sensor data acquisition, data processing, and understanding of MCI context. By integrating this information into a dynamic resource allocation model, triage can be streamlined, allowing drones and first responders to reach a greater number of casualties per unit of time.

[0067] FIG. 8 is a flowchart indicating one embodiment of a triage algorithm 300. Not all steps need to be used. This particular triage algorithm utilizes the same base triage categories as START and SALT: MINOR, DELAYED, IMMEDIATE, and EXPECTANT / DEAD. Each category may be designated by a specific color designation for easy recognition by a system operator when viewing a user interface. The triage algorithm sort casualties into these categories as quickly as possible. At minimum, each assessed casualty will be designated as falling within one of the triage categories. Time, resourcing, and situation permitting, the drones may continue to monitor casualties and acquire additional data to refine assessments, predict precise injury classifications, and increase triage certainties.

[0068] The system will use established criteria commonly accepted across MCI field triage guidelines to assign categories. Primary physiological signatures, (i.e. , ambulation, mental alertness, respiratory rate, massive hemorrhage, and pulse) will be assessed immediately. Secondary physiological signatures (e.g., body temperature, severe wounds) will also be used to further inform the triage algorithm. Assessments of physiological signatures may be modified by dynamic adjustments. Signatures such as mental alertness (responsiveness) and respiratory rate will be used as references for triage determinations where established quantitative field triage guidelines exist. Signatures such as major hemorrhage can also be quantified.

[0069] When a primary signature is deemed acceptable or out of bounds, a triage category can be assigned according to the logic used in the triage algorithm. Triage categories are assigned when a definitive category can be assigned (e.g., the casualty is walking and thus has minimal injuries according to established triage protocols). In cases where primary signatures are inconclusive, secondary physiological signatures and dynamic triage elements are used to complete the categorization.

[0070] Many dynamic elements may impede autonomous MCI triage. Sensor data may be affected by environmental factors such as changes in lighting conditions, smoke or fog, and obscuring obstacles. Dynamic obstacles and situational factors such as the presence of first responders may introduce noise and off-target readings from casualties. Lastly, MCI context such as the number and severity of casualties, the type of MCI (e.g.,chemical agent, explosion, natural disaster), time since the start of the MCI, and the available response resources all factor into resource management strategies.

[0071] The triage algorithm can take into account such dynamic elements. These parameters can be balanced to optimize triage efficiency. For instance, there is an inverse relationship between the time spent per causality assessment and the total number of casualties that can be assessed per unit of time. While longer assessment periods allow for more data collection and thus increased triage categorization confidence, they may inadvertently postpone urgent assessments for immediate casualties. Additionally, the types of sensors, data processing, and modeling of secondary physiological signatures can be down-selected to emphasize signatures relevant to MCI context or to efficiently expend energy according to onboard power storage and power consumption constraints. Lastly, signature bounds may need adjusted depending on casualty demographics. For instance, some triage guidelines differ depending on casualty age, and some sensors may be sensitive to anatomical differences such as skin color. Tuning these tradeoffs properly will allow for efficient triage that both minimizes the time required per casualty for an accurate assessment and maximizes operational time.

[0072] Two major elements of dynamic triage are contemplated: (1 ) Triage Confidence Filtering and (2) Triage Resource Allocation. The first element, Triage Confidence Filtering, is important for tuning the time / accuracy tradeoff. A static time window to assess each causality may not be appropriate especially when acquiring noisy signals or monitoring an evolving process. Thus, each primary signature is gated until a confidence threshold is achieved. This threshold will decay over time such that the most confident decisions can be made quickly whereas more ambiguous cases will be allotted more time for further data acquisition before a decision is made. The initial rate of decay parameter can be determined by fitting models to the data provided to the triage algorithm, and they can be further modified by contextual factors such as the number of identified casualties that have yet to be assessed.

[0073] Extracting confidence values for each primary physiological signature is dependent on the processing method. For instance, neural network classifiers output class probabilities during prediction, whereas a power spectrum analysis model for predicting heart rate may not. A statistical process control (SPC) algorithm can be usedto set bounds for different sensor signals to verify stable processes. Beyond monitoring individual sensors for noisy or aberrant values indicative of low confidence predictions, this permits monitoring of correlations among sensors that capture redundant signatures. This process is also amenable to confidence filtering as the control limits can expand or decay across time as the situation demands. Control limits for each signature can be guided by established triage guidelines and the analysis of the data being acquired by the drones.

[0074] The second dynamic element, Triage Resource Allocation, will manage data acquisition for secondary physiological signatures. Simultaneous collecting, processing, and modeling data streams from many signatures compounded with power costs for telemetry can delay subsequent assessments. To minimize extraneous processing, signatures will be split into primary (mandatory) and secondary (optional) processes. However, depending on specific injury profiles, the type of MCI, and the necessary resolution of triage (e.g., predicting the need for specific LSIs) secondary sensors may supply valuable information. Spending additional time and resources beyond that required for categorization into primary triage class will likely be context dependent and be determined by the confidence of any primary signature deviations.

[0075] This triage architecture allows for flexible substitution of primary or secondary physiological signatures as they are developed. For instance, the shock index, the ratio of heart rate to systolic blood pressure which has shown to be a strong predictor of shock following trauma may be used in the triage algorithm as well.

[0076] Referring now to FIG. 8, the triage algorithm 300 illustrates how data for a given casualty may be used to assign a triage status to the casualty. The method starts at step 302. Decision box 304 asks whether the casualty is walking or not. If yes, then at step 306 the casualty can be assigned a triage status of MINOR. If the answer to step 304 is no, then the method passes to step 308, where primary physiological signatures are evaluated. Those primary physiological signatures may include, for example, the respiratory rate, whether hemorrhage is present, and / or the pulse (i.e. heart rate) of the casualty. Those primary signatures inform decision box 310, which asks whether the casualty is responsive. Regardless of the answer, decision box 312 asks whether all of the primary signatures are within set bounds or not. If the answer is yes, the algorithmcontinues to decision box 314. If the answer to decision box 312 is no, the algorithm continues to decision box 316.

[0077] Decision box 314 asks whether further assessment is needed for the casualty whose primary signatures are all within bounds. If the answer is no, then in step 320 the casualty is assigned a triage status of DELAYED. This can mean the casualty still has serious medical issues, but is expected to survive until further care can be provided. Put another way, the casualty does not need to be immediately treated to survive. If the answer is yes, then the algorithm continues to box 326.

[0078] Decision box 316 asks whether the casualty is likely to live based on the primary signatures. If the answer is no, then in step 322 the casualty is assigned a triage status of EXPECTANT. This means the casualty is not expected to live, even if life-saving interventions (LSIs) are immediately provided. If the answer is yes, the algorithm continues to decision box 318.

[0079] Decision box 318 asks whether further assessment is needed for the casualty whose primary signatures are out of bounds. If the answer is no, then in step 320 the casualty is assigned a triage status of IMMEDIATE. This means the casualty has a good chance of surviving if (LSIs) are immediately provided by first responders. If the answer is yes, then the algorithm continues to box 326.

[0080] In box 326, secondary physiological signatures are evaluated. Those secondary physiological signatures may include, for example, the presence and / or severity of hypothermia, major wounds, severe burns, and / or sucking chest wounds. Arrow 328 indicates the secondary physiological signatures are then further considered by the triage algorithm to determine whether a triage status of DELAYED, IMMEDIATE or EXPECTANT is applied to the casualty. Again, the triage status for each casualty can be continuously assessed, and their status can be changed as appropriate.

[0081] Continuing, in step 262 of FIG. 2B, a triaged casualty list or triage report is created. This list I report sorts all of the identified casualties based on their triage status. Applying autonomous data collection and artificial intelligence decision making to MCls reduces or eliminates the cognitive, communication, and manpower limitations to situational assessments and casualty triage and allows first responders to prioritize high risk casualties that need an LSI.

[0082] With information such as the number of first responders and the location of each casualty and their triage status, the casualty list or triage report can take several different forms. For example, the triage report can be in the form of one list that identifies all casualties with IMMEDIATE status, then DELAYED status, then MINOR status, then EXPECTANT status. Alternatively, the triage report could be in the form of several lists, one for each first responder / team, arranged by triage status and location. Thus, for example, first responder team A would receive a casualty list for an area A that sorts the casualties in that area by triage status, first responder team B would receive a different casualty list for area B, etc.

[0083] In step 264 of FIG. 2B, the triaged casualty list or triage report is communicated to one or more operators. This could be done, for example, using printed handouts or other user interface, such as a cellphone, tablet, heads-up display, etc. For example, the Android Team Awareness Kit (ATAK) app can act as the core of the user interface (Ul). The ATAK architecture has a plug-in functionality to integrate data streams into a top- down spatial representation that will give first responders a birds-eye representation of the mass casualty environment. Data streams from the onboard processing on the drones and the off-board processing done at the control stations can be processed and represented by ATAK as different icons for, for example, casualties, non-casualty humans, potential environmental threats, or notable objects in the mass casualty environment. This provides users with categorization (e.g., casualty that needs LSI vs. stable casualty), with the ability for users to select an individual casualty to get an in-depth readout of the underlying physiological signatures. Whenever the algorithm detects a need for an LSI, that casualty location can be reported immediately to the user with a pop-up describing the requisite LSI and urgency. The resulting user interface can combine existing GPS and geospatial map data with an overlay for real-time situational awareness of ongoing events.

[0084] In optional step 266 of FIG. 2B, an operator, such as a first responder, may apply a wearable monitoring system to a casualty. This wearable monitoring system may then provide additional data to the control station, as indicated by arrow 267. As nonlimiting example, the wearable monitoring system could be a heart rate monitor, or a pulse oximeter, an ECG / EKG monitor, a blood pressure cuff, or generally any device that canmeasure a physiological parameter. This information can be transmitted to a ground station and incorporated into the triage algorithm to provide better information for determining an appropriate triage status.

[0085] The DART systems improve MCI response in three primary areas. First, information is centralized through the integration of autonomous systems for casualty search, localization, assessment, and triage. Given the complexity and demands of an MCI, effective information integration improves response, assessment, and monitoring. Second, casualty identification and interaction is performed using stand-off sensors. This reduces the need to have a medical responder in close physical contact with each casualty, and increases the speed of initial casualty assessments. Third, an algorithmbased decision-making triage approach minimizes response times for those that need and would benefit the most from immediate help from a first responder. Overall, this should greatly increase the speed and accuracy of triage decisions, and also reduce the cognitive and mental load of first responders. Both clinical and non-clinical factors unique to the MCI can be accounted for. Triage decisions can also be dynamically adjusted as injuries progress and / or as medical resources are made available. The system also assists with intelligent first responder routes and intervention mapping to maximize resource usage. This serves as a force multiplier.

[0086] The present disclosure has been described with reference to several different embodiments. Obviously, modifications and alterations will occur to others upon reading and understanding the preceding detailed description. It is intended that the present disclosure be construed as including all such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims

CLAIMS:1 . A method for performing triage, comprising: identifying at least one casualty using one or more standoff sensors mounted on one or more drones; capturing data on the at least one casualty using the one or more standoff sensors; sending the data on the at least one casualty from the one or more drones to a control station for prioritizing medical response; creating a triaged casualty list using an algorithm; and communicating the triaged casualty list to at least one operator.

2. The method of claim 1 , further comprising developing a search strategy; and planning a search route for each of the one or more drones.

3. The method of claim 1 , further comprising identifying areas of high casualty likelihood.

4. The method of claim 1 , further comprising establishing and maintaining a track for each casualty to maintain identification and location of the casualty.

5. The method of claim 1 , further comprising engaging the at least one casualty using the one or more drones.

6. The method of claim 5, wherein the at least one casualty is engaged using at least one speaker mounted on the one or more drones.

7. The method of claim 1 , wherein the one or more standoff sensors comprise one or more of a camera, an infrared sensor, an acoustic sensor, a radar, and a microphone.

8. The method of claim 1 , wherein the data captured using the standoff sensors includes visual images; and wherein the control station processes the visual images to determine at least one physiological signature.

9. The method of claim 8, wherein the at least one physiological signature determined by processing of the visual images comprises a chest movement status; a blood loss estimate; a wound estimate; or a hypothermia estimate.

10. The method of claim 1 , wherein the one or more drones comprise an unmanned aerial vehicle (UAV) or an unmanned ground vehicle (UGV).

11. The method of claim 1 , further comprising identifying locations of the respective identified casualties and sending the locations of the respective identified casualties from the one or more drones to the control station.

12. The method of claim 1 , further comprising deploying one or more standalone Position I Navigation / Timing (PNT) beacons to define a search area for the one or more drones.

13. The method of claim 1 , wherein the algorithm considers one or more physiological signatures selected from the group consisting of a body temperature; a respiration status; a respiration rate; a heart rate; a chest movement status; a responsiveness status; a blood loss estimate; an amputation estimate; a wound estimate; and a hypothermia estimate.

14. The method of claim 1 , wherein the triaged casualty list is communicated to the at least one operator using a heads-up display of a helmet.

15. The method of claim 1 , wherein the triaged casualty list includes, for each casualty, one or more of a casualty triage status, a casualty location, and a physiological signature.

16. A method for performing triage, comprising: capturing vital signs of one or more persons using a monitoring system; sending the vital signs to a control station; producing a triage report using an algorithm; and displaying the triage report.

17. The method of claim 16, wherein the triage report is displayed on a headsup display of a medic’s helmet.

18. The method of claim 16, wherein:- the one or more persons is a plurality of persons, and- the monitoring system comprises wearable monitoring systems worn by the respective persons, and- the capturing comprises wirelessly receiving the vital signs of the respective persons from the respective wearable monitoring systems, and- the triage report comprises a triage report for each respective person produced by applying the algorithm to the vital signs of the respective person wirelessly received from the wearable monitoring system worn by the respective person, and- the method further includes assigning priorities to the respective persons based on the triage reports of the respective persons.

19. The method of claim 16, further comprising: receiving location information for the respective persons; and displaying the locations and the priorities of the respective persons of the plurality of persons.

20. The method of claim 16, wherein the algorithm considers one or more physiological signatures selected from the group consisting of a respiration rate and a heart rate.

Images