Generating emergency spatial analysis results

Abstract

Devices, systems, and methods for generating emergency spatial analysis results are described herein. In some examples, one or more embodiments include a controller including a memory and a processor executing instructions stored in the memory to receive visual input from a first sensor and spatial input from a second sensor, wherein the first sensor and the second sensor are located on an unmanned aerial vehicle (UAV); generating an emergency spatial analysis result based on the visual input and the spatial input; and displaying the emergency spatial analysis result via a user interface.

Description

Technical Field

[0001] The present disclosure relates to devices, systems and methods for generating emergency event spatial analysis results. Background Art

[0002] A first responder may be a person who is designated and / or trained to respond to an emergency. Such emergencies may include, for example, accidents, disasters, medical emergencies, fires, crimes, and other emergencies. A first responder may be specially trained and may be one of the first people to arrive at the scene of an emergency in order to provide assistance and / or resolve the incident. First responders may include firefighters, paramedics, emergency medical technicians (EMTs), law enforcement officers, and the like. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] Figure 1 is an example of a system for generating emergency event spatial analysis results according to one or more embodiments of the present disclosure.

[0004] Figure 2 is an example of a user interface of a mobile device showing points of interest for generating emergency spatial analysis results according to one or more embodiments of the present disclosure.

[0005] Figure 3 is an example of a user interface of a mobile device showing objects in emergency spatial analysis results according to one or more embodiments of the present disclosure.

[0006] Figure 4 is an example of a controller for generating emergency event spatial analysis results according to one or more embodiments of the present disclosure. DETAILED DESCRIPTION

[0007] The present invention describes an apparatus, system, and method for generating an emergency spatial analysis result. In some examples, one or more embodiments include a controller including a memory and a processor that executes instructions stored in the memory to receive visual input from a first sensor and spatial input from a second sensor, wherein the first sensor and the second sensor are located on an unmanned aerial vehicle (UAV); generate an emergency spatial analysis result based on the visual input and the spatial input; and display the emergency spatial analysis result via a user interface.

[0008] During an emergency, first responders may respond to the location of the emergency. Finding information about an emergency can be challenging. Because emergencies can be very chaotic, it can be difficult to get information about the type of incident, the scope of the incident, whether there may be trapped or injured people, etc.

[0009] Therefore, first responders may perform their duties without the assistance of such information. For example, firefighters may enter a building without knowing the precise location of an ongoing fire, the intensity of the fire, whether there are trapped persons in the building, the best path to the fire location and / or to trapped persons, the best evacuation route, etc. As another example, first responders arriving at the scene of an earthquake may not know whether some of the rubble caused by the earthquake is stable or unstable, whether there are trapped persons, the location of the trapped persons, etc. Instead, such decisions about how and where to go are often made on the fly. However, such an approach may put the lives of first responders at risk.

[0010] According to the present disclosure, generating emergency spatial analysis results can utilize unmanned aerial vehicles (UAVs) with various sensors to collect information about emergency events for first responders. Information from sensors can be used to generate emergency spatial analysis results so that first responders can make informed decisions about how and where to go during an emergency. Compared with previous methods, such methods can reduce the risks faced by first responders responding to emergency events.

[0011] In the following Detailed Description, reference is made to the accompanying drawings which form part of the present Detailed Description. The drawings show, by way of illustration, the manner in which one or more embodiments of the present disclosure may be practiced.

[0012] These embodiments are described in sufficient detail to enable one of ordinary skill in the art to practice one or more embodiments of the present disclosure. It is to be understood that other embodiments may be utilized and process, electrical and / or structural changes may be made without departing from the scope of the present disclosure.

[0013] It should be understood that the elements shown in the various embodiments herein may be added, exchanged, combined and / or eliminated to provide multiple additional embodiments of the present disclosure. The proportions and relative sizes of the elements provided in the drawings are intended to illustrate embodiments of the present disclosure and should not be limiting.

[0014] The figures herein follow the following numbering convention: one or more first digits correspond to the figure number, and the remaining digits identify the element or component in the figure. Similar elements or components between different figures can be identified by using similar numerals. For example, 104 can be in Figure 1 The component is referenced as "04" in the Figure 4 Referenced as 404.

[0015] As used herein, "a" or "a number of" things may refer to one or more such things, and "a plurality" of things may refer to more than one such thing. For example, "a number of components" may refer to one or more components, and "a plurality of components" may refer to more than one component.

[0016] Figure 1 is an example of a system 100 for generating emergency spatial analysis results according to one or more embodiments of the present disclosure. The system 100 may include mobile devices 102-1, 102-2, 102-3, 102-M, and a space 108 in which a UAV 110 and objects 116-1, 116-2, 116-N may be located.

[0017] As described above, first responders may respond to an incident scene. Such an incident may occur in a space 108. Space 108 may be a location where an emergency is occurring. Space 108 may be a building, a portion of a building, an outdoor location, etc.

[0018] For example, the first responder may be a firefighter who responds to a report of a fire occurring in space 108. When the first responder arrives, the first responder may not understand the location of the fire inside space 108, the intensity of the fire, whether there are trapped people in space 108, etc.

[0019] Thus, first responders may utilize UAV 110. As used herein, the term "UAV" refers to an aircraft without a human pilot on board. For example, a UAV may be an aircraft that can be operated autonomously and / or by remote control. For example, UAV 110 may be a single-rotor UAS or a multi-rotor UAS such as a three-rotor aircraft, a quad-rotor aircraft, a six-rotor aircraft, an eight-rotor aircraft, etc. In some embodiments, UAV 110 may include multi-rotor positioning, including other examples of rotor positioning such as quadrotor I, quadrotor X, six-rotor I, six-rotor V, six-rotor Y, six-rotor IY, eight-rotor X, eight-rotor I, eight-rotor V, etc.

[0020] The UAV 110 may be controlled by the mobile device 102-1. As used herein, a mobile device may include a device that is carried and / or worn by a user (or may be carried and / or worn by a user). For example, the mobile device may be a phone (e.g., a smart phone), a tablet computer, a personal digital assistant (PDA), smart glasses, and / or a wrist-worn device (e.g., a smart watch), as well as other types of mobile devices. For example, the mobile device 102-1 may be a phone that may control the UAV 110 via a wireless connection to the UAV 110.

[0021] Using the mobile device 102-1, the first responder may control the UAV 110 so as to direct the UAV 110 into the space 108. Using the information transmitted from the UAV 110, the first responder may determine the location of the emergency within the space 108, the intensity level of the emergency, objects of interest within the space 108, ranging information about the space 108, proposed evacuation routes 115, and other information that may assist the first responder in resolving the emergency in a safer manner than previously possible, as further described herein.

[0022] like Figure 1 As illustrated, the UAV 110 may include a first sensor and a second sensor. The first sensor may be an infrared (IR) camera 112. The IR camera 112 may be a thermal imaging device that captures IR wavelength images. The second sensor may be a light detection and ranging (LIDAR) sensor 114. A LIDAR sensor is a device that captures ranging information by transmitting light to reflect from an object and measuring the time it takes for the reflected light to return to the device. The IR camera 112 and the LIDAR sensor 114 are located on the UAV 110, as further described herein.

[0023] As described above, IR camera 112 may capture visual information and generate visual input in space 108. Visual input may include, for example, thermal imaging data regarding space 108. For example, visual input may be thermal imaging data regarding objects 116-1, 116-2, 116-N located in space 108.

[0024] Additionally, the LIDAR sensor 114 may capture spatial information of the space 108 and generate a spatial input. The spatial input may include, for example, time-of-flight distance and / or dimensional data regarding the space 108 captured by the LIDAR sensor 114.

[0025] Visual input from the IR camera 112 and spatial input from the LIDAR sensor 114 may be transmitted to the controller 104. For example, although not shown in the figure for clarity and ease of understanding of the embodiments of the present disclosure, Figure 1 , but UAV 110 may include a transceiver capable of transmitting and / or receiving information wirelessly. Visual input and spatial input may be transmitted wirelessly from UAV 110 to controller 104 of mobile device 102-1 (e.g., via a transceiver).

[0026] Thus, the controller 104 can generate an emergency spatial analysis result based on the visual input and the spatial input. As used herein, the emergency spatial analysis result can be a collection of evaluated and focused information related to the emergency. First responders can use such a collection of evaluated and focused information to effectively identify objects 116-1, 116-2, 116-N, and safely enter and exit the space 108 when responding to the emergency, as further described herein.

[0027] In some examples, objects 116-1, 116-2, 116-N may be objects that emit thermal energy in space 108. For example, object 116-1 may be a person, and object 116-2 may be a dog. Objects 116-1 and 116-2 may be people trapped in space 108 due to an ongoing emergency. Therefore, controller 104 may generate emergency spatial analysis results by generating images of objects 116-1 and 116-2. For example, controller 104 may generate images of objects 116-1, 116-2 that emit thermal energy in space 108 using thermal imaging data included in the visual input.

[0028] The controller 104 may cause the emergency spatial analysis results to be displayed via the user interface 106. For example, the controller may cause images of the objects 116-1, 116-2 to be displayed on the user interface 106 of the mobile device 102-1. Generating images of the objects 116-1, 116-2 using the thermal energy emitted by the objects 116-1, 116-2 may allow first responders to view the objects 116-1, 116-2 via their thermal signatures through the user interface 106. Such an approach may allow first responders to determine that the objects 116-1, 116-2 are in the space 108 when normal visibility may be obstructed, such as by smoke when the emergency is a fire.

[0029] In some examples, the first responder may use the mobile device 102-1 to guide the UAV 110 while also passing through the space 108. Therefore, the first responder can identify the trapped personnel and dog by viewing the user interface 106. However, the embodiments of the present disclosure are not limited to this. For example, a user who may be far away from the space 108 may guide the UAV 110 via a different mobile device (e.g., mobile device 102-2), and the first responder who enters the space 108 carrying the mobile device 102-1 may view the emergency spatial analysis results via the user interface 106. In other words, in some examples, the first responder with the mobile device 102-1 may guide the UAV 110 and view the emergency spatial analysis results at the same time, and in some examples, the first responder with the mobile device 102-1 may view the emergency spatial analysis results while another different user guides the UAV 110.

[0030] In some examples, generating emergency spatial analysis results may include determining the temperature of objects 116-1, 116-2, 116-N. The controller 104 may utilize thermal imaging data included in the visual input to determine the temperature of objects 116-1, 116-2, 116-N that emit thermal energy in the space 108. The controller 104 may then cause the temperature of the objects 116-1, 116-2, 116-N to be displayed on the user interface 106 of the mobile device 102-1. Such an approach may allow a first responder to determine how close he or she can get to the objects 116-1, 116-2, 116-N. For example, if the object 116-N is a fire, the first responder may determine the temperature of the fire and how close he or she can get to the fire.

[0031] In some examples, the controller 104 may generate the emergency spatial analysis results by determining ranging information for the space 108 in which the UAV 110 is located. For example, the controller 104 may utilize the time-of-flight distance and / or dimensional data included in the spatial input to determine the ranging information for the space 108. The ranging information may include the dimensions of the space 108 and / or the distances from the LIDAR sensor 114 of the UAV 110 to the objects 116-1, 116-2, 116-N in the space, as further described herein.

[0032] As described above, in some examples, the ranging information may include the dimensions of the space 108 in which the UAV 110 is located. For example, the space 108 may be the lobby of an office building. Using the ranging information from the LIDAR sensor 114, the controller 104 may determine the dimensions of the lobby (e.g., 12 meters by 15 meters).

[0033] Controller 104 may display the emergency space analysis results by causing the dimensions of the space to be displayed on user interface 106. Using the dimensions of space 108, first responders may determine how large the space is. Such information may be used to determine how best to fight a fire, enter / exit space 108, and / or determine any other useful information for responding to any other emergency event.

[0034] Although the dimensions of the space 108 are described above as the dimensions of the ground space, embodiments of the present disclosure are not limited thereto. For example, the dimensions of the space 108 may include the dimensions of any entrance and / or exit (e.g., door, window, etc.) of the space 108, the dimensions of the objects 116-1, 116-2, 116-N in the space 108, etc.

[0035] In some examples, the ranging information may include the distance from the LIDAR sensor 114 to objects 116-1, 116-2, 116-N in the space 108 in which the UAV 110 is located. For example, the space 108 may be a lobby of an office building, and the object 116-1 may be a human, the object 116-2 may be a chair, and the object 116-N may be a table. Using the ranging information from the LIDAR sensor 114, the controller 104 may determine the distance from the LIDAR sensor 114 to the objects 116-1, 116-2, 116-N (e.g., as well as the distance from the UAV 110 to the objects 116-1, 116-2, 116-N). For example, the controller 104 may determine that the distance from the LIDAR sensor 114 to the object 116-1 is 5 meters, the distance from the LIDAR sensor 114 to the object 116-2 is 3 meters, and the distance from the LIDAR sensor 114 to the object 116-N is 4 meters. As described above, the distance from the LIDAR sensor 114 to an object emitting energy in the space 108 in which the UAV is located (e.g., the distance between the LIDAR sensor 114 and the object 116-1, which in some examples may be a human) may be determined.

[0036] The controller 104 may display the emergency spatial analysis results by causing the distance between the LIDAR sensor 114 and the objects 116-1, 116-2, 116-N to be displayed on the user interface 106. Using the distance between the LIDAR sensor 114 and the objects 116-1, 116-2, 116-N, a user may determine how far they are from the objects 116-1, 116-2, 116-N (e.g., by knowing their position relative to the position of the UAV 110). Furthermore, in conjunction with knowing the dimensions of the space 108, the user may determine the position of the objects 116-1, 116-2, 116-N in the space 108 with greater accuracy than previously possible. Such an approach may allow a user to determine additional information than previously possible in order to more safely plan an emergency response.

[0037] Although the controller 104 is described above as determining the distance between the LIDAR sensor 114 and the objects 116-1, 116-2, 116-N, embodiments of the present disclosure are not limited thereto. For example, the controller may determine the distance between the LIDAR sensor 114 and other points of interest in the space 108 in which the UAV 110 is located. Points of interest may include entry locations / exit locations, emergency equipment (e.g., fire alarms, pull-type fire alarm boxes, door locks, etc.), etc. The controller 104 may cause such distances between the LIDAR sensor 114 and other points of interest in the space 108 to be displayed via the user interface 106.

[0038] Generating the emergency spatial analysis results may also include generating a three-dimensional (3D) image of the space 108 based on the spatial input from the LIDAR sensor 114. For example, using ranging information (e.g., from the time-of-flight distance and / or dimensional data), the controller 104 may generate a 3D image of the space 108. The 3D image of the space 108 may include a 3D model of the space 108, as well as 3D models of the objects 116-1, 116-2, 116-N in the space, any other points of interest in the space, entry / exit points, etc.

[0039] The controller 104 may display the emergency spatial analysis results by causing a 3D image of the space 108 to be displayed on the user interface 106. The 3D image of the space 108 may include any and / or all of the information also described above. For example, the 3D image of the space 108 may include further information overlaid thereon, including any images of objects 116-1, 116-2, 116-N that may emit thermal energy in the space 108, the temperature of such objects 116-1, 116-2, 116-N, the dimensions of the space 108, and / or the distance from the LIDAR sensor 114 to the objects 116-1, 116-2, 116-N and / or any other points of interest in the space 108. Utilizing a 3D image of the space may allow first responders to get a full picture of events that may occur in the space 108 during an emergency, even if such space 108 may not be accessible, vision is obstructed (e.g., due to smoke or other particles), etc.

[0040] Using the 3D image of the space 108, the controller 104 may generate an evacuation route out of the space 108. For example, using the visual input from the IR camera 112 and the spatial input from the LIDAR sensor 114, the controller 104 may determine the entry / exit locations in the space 108, and based on the locations of the objects 116-1, 116-2, 116-N, generate an evacuation route through the entry / exit locations while avoiding "hot spots" (e.g., locations in the space 108 where the temperature exceeds a threshold temperature). Such an evacuation route may be displayed on the user interface 106, and may provide a route out of the space 108 to first responders, direct any trapped personnel out of the space 108, etc.

[0041] Thus, as described above, the controller 104 may cause emergency spatial analysis results to be displayed via the user interface 106. Such emergency spatial analysis results may include objects 116-1, 116-2, 116-N that may emit thermal energy, temperatures of such objects 116-1, 116-2, 116-N, ranging information about the space 108, and / or a 3D image of the space 108.

[0042] In addition, the controller 104 can cause the emergency spatial analysis results to be displayed on the user interfaces of other mobile devices 102-2, 102-3, 102-M. For example, multiple first responders can respond to the emergency. Other user interfaces that mirror the emergency spatial analysis results to other mobile devices 102-2, 102-3, 102-M can allow users of such mobile devices 102-2, 102-3, 102-M to also be aware of the situation in the space 108.

[0043] Figure 2 2 is an example of a user interface 206 of a mobile device showing points of interest 218 for generating emergency spatial analysis results according to one or more embodiments of the present disclosure. The controller can generate emergency spatial analysis results for display on the user interface 206.

[0044] As previously described above, the controller can utilize spatial input from the LIDAR sensor to generate ranging information about the space. Within the space, there may be objects and / or other points of interest. In addition, the controller can utilize visual input from the IR camera to generate images of objects and / or other points of interest in the space. The ranging information may include the dimensions of the space and / or the distance to the objects and / or other points of interest in the space in which the UAV is located.

[0045] For example, Figure 2 As illustrated, object 216 may be a cat emitting heat energy in a space. During an emergency, the cat may be trapped in the space and have its normal view obstructed by smoke. Figure 2 As illustrated, the controller may cause the display of the dimensions of the space (e.g., 8 meters from the back wall) and the distance to a point of interest in the space (e.g., object 216, a cat, which is 4 meters from the location of the LIDAR sensor on the UAV). In addition, as described above, the cat may radiate energy in the space in which the UAV is located, and therefore, the distance to the object radiating energy may be displayed (e.g., 4 meters).

[0046] Figure 3 is an example of a user interface 306 of a mobile device showing an object 316 in an emergency spatial analysis result according to one or more embodiments of the present disclosure. The controller may generate the emergency spatial analysis result for display on the user interface 306.

[0047] As previously described above, the controller can utilize spatial input from the LIDAR sensor to generate ranging information about the space. Within the space, there may be objects and / or other points of interest. In addition, the controller can utilize visual input from the IR camera to generate images of objects and / or other points of interest in the space. The ranging information may include the dimensions of the space and / or the distance to the objects and / or other points of interest in the space in which the UAV is located.

[0048] like Figure 3 As shown, the controller may utilize the ranging information and the visual input to cause object 316 to be displayed as part of the emergency spatial analysis results. It may be determined that the temperature of object 316 is 106°F, and the distance from the sensor on the UAV to object 316 may be 4 meters.

[0049] like Figure 3 As illustrated, object 316 appears to be a human. The controller may determine whether the object is a human based on the image of object 316. For example, the controller may compare the image of object 316 with a predetermined reference image to determine whether object 316 has the shape of a human. Such reference images may be stored locally at the controller (e.g., via a remote computing device / cloud computing environment) and / or may be remote from the controller and accessible via a wireless network relationship between a mobile device and the remote computing device / cloud computing environment.

[0050] In response to object 316 being determined to be a human, the controller may cause the color of object 316 to be set differently from other objects in the space. This different color setting may cause first responders to notice that object 316 in the space is a human, and that object 316 should be noted because the human may be trapped in the space due to an emergency. First responders may use the above emergency space analysis results to determine a plan for locating and rescuing the human.

[0051] Figure 4 is an example of a controller for generating emergency spatial analysis results according to one or more embodiments of the present disclosure. Figure 4 As illustrated, the controller 404 may include a memory 432 and a processor 430 for generating emergency spatial analysis results according to the present disclosure.

[0052] The memory 432 may be any type of storage medium that can be accessed by the processor 430 to perform various examples of the present disclosure. For example, the memory 432 may be a non-transitory computer-readable medium having computer-readable instructions (e.g., executable instructions / computer program instructions) stored thereon, which instructions may be executed by the processor 430 for generating emergency spatial analysis results according to the present disclosure.

[0053] The memory 432 may be a volatile memory or a non-volatile memory. The memory 432 may also be a removable (e.g., portable) memory or a non-removable (e.g., internal) memory. For example, the memory 432 may be a random access memory (RAM) (e.g., a dynamic random access memory (DRAM) and / or a phase change random access memory (PCRAM)), a read-only memory (ROM) (e.g., an electrically erasable programmable read-only memory (EEPROM) and / or a compact disc read-only memory (CD-ROM)), a flash memory, a laser disc, a digital versatile disc (DVD), or other optical storage devices and / or magnetic media (such as a cassette, a tape, or a disk), and other types of memory.

[0054] In addition, although the memory 432 is illustrated as being located within the controller 404, the embodiments of the present disclosure are not limited thereto. For example, the memory 432 may also be located within another computing resource (e.g., enabling computer readable instructions to be downloaded via the Internet or another wired or wireless connection).

[0055] Processor 430 may be a central processing unit (CPU), a semiconductor-based microprocessor, and / or other hardware device suitable for retrieving and executing machine-readable instructions stored in memory 432 .

[0056] Although specific implementations have been illustrated and described herein, those of ordinary skill in the art will appreciate that any arrangement calculated to achieve the same technique may be substituted for the specific implementations shown. This disclosure is intended to cover any and all modifications or variations of various implementations of the present disclosure.

[0057] It should be understood that the above description is given in an illustrative rather than limiting manner.By reading the above description, combinations of the above embodiments, as well as other embodiments not specifically described herein, will be apparent to those skilled in the art.

[0058] The scope of the various embodiments of the present disclosure includes any other applications using the above-described structures and methods.The scope of the various embodiments of the present disclosure should therefore be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.

[0059] In the above detailed description, various features are grouped together in the exemplary embodiments shown in the drawings for the purpose of simplifying the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the present disclosure require more features than are expressly recited in each claim.

[0060] Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment.Thus the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment.

Claims

1. A controller (104, 404) for generating emergency event spatial analysis results, the controller comprising: Memory (432); and a processor (430) configured to execute executable instructions stored in the memory (432) to: receiving visual input from a first sensor and spatial input from a second sensor, wherein the first sensor and the second sensor are located on an unmanned aerial vehicle (UAV) (110); generating emergency event spatial analysis results based on the visual input and the spatial input; as well as The emergency event spatial analysis result is displayed via a user interface (106, 206, 306).

2. A controller (104, 404) according to claim 1, wherein the processor (430) is configured to execute the instructions to generate the emergency event spatial analysis results by generating an image of an object (216, 316) emitting thermal energy in a space (108) in which the UAV (110) is located based on the visual input.

3. The controller (104, 404) of claim 1, wherein the first sensor is an infrared (IR) camera that generates and transmits the visual input.

4. The controller (104, 404) of claim 1, wherein the processor (430) is configured to execute the instructions to generate the emergency event spatial analysis result by determining ranging information of a space (108) in which the UAV (110) is located based on the spatial input.

5. The controller (104, 404) of claim 4, wherein the ranging information includes dimensions of the space (108) in which the UAV (110) is located.

6. The controller (104, 404) of claim 4, wherein the ranging information includes a distance from the second sensor to an object (116-1, 116-2, 116-N) in the space (108) in which the UAV (110) is located.

7. The controller (104, 404) of claim 1, wherein the second sensor is a light detection and ranging (LIDAR) sensor that generates and transmits the spatial input.

8. The controller (104, 404) of claim 1, wherein the processor (430) is configured to execute the instructions to display the emergency event spatial analysis results by displaying on the user interface (106, 206, 306) objects (216, 316) emitting thermal energy in the space (108) in which the UAV (110) is located.

9. The controller (104, 404) of claim 1, wherein the processor (430) is configured to execute the instructions to display the emergency event spatial analysis results by displaying the dimensions of the space (108) in which the UAV (110) is located on the user interface (106, 206, 306).

10. The controller (104, 404) of claim 9, wherein the processor (430) is configured to execute the instructions to display the emergency spatial analysis results by displaying on the user interface (106, 206, 306) the distance to a point of interest (218) in the space (108) in which the UAV (110) is located.

Images