Indoor personnel evacuation route guide method using portable device augmented reality and system therefor

Abstract

The present invention provides a method for guiding an indoor person to an evacuation route using a portable device and a system therefor, the evacuation route guide system including: a portable device on which an evacuation guide application program for guiding an evacuation route from a current position of an indoor person to a shelter is installed and which is carried by the indoor person; and a server for providing a plurality of evacuation route information through the portable device, the portable device executing the evacuation guide application program to display an evacuation guide interface, the portable device displaying a photographed landscape image on an image display area of the evacuation guide interface and displaying one of the plurality of evacuation routes set based on specific information related to a disaster type, a user's movement ability, and individual circumstances on a map display area of the evacuation guide interface, and the portable device displaying guide information indicating a direction of movement from the current position on the image display area overlaid with the landscape image along the evacuation route displayed on the map display area.

Description

Technical Field

[0001] This invention relates to a method and system for guiding people to refuge routes indoors using portable devices and augmented reality. Background Technology

[0002] With rapid industrial development, social progress, and population growth, medium and large-sized buildings, including cultural, commercial, and residential facilities, are becoming increasingly larger and more multifunctional, leading to more complex interiors and a rise in cases of users getting lost. This is especially true during disasters such as fires, gas leaks, earthquakes, and terrorist attacks, which exacerbate the confusion and disorientation of residents during evacuation and rescue. Therefore, reliable evacuation route guidance systems are needed to address disasters and emergencies within complex medium and large-sized buildings. Recently, the development of evacuation-related technologies in various fields has enabled the design of more proactive evacuation route guidance systems to respond to disasters and emergencies. Current evacuation-related technologies include communication technologies for tracking the location of evacuees within buildings, sensor technologies for detecting disasters and emergencies, sensor technologies for detecting building collapse risks, and technologies for monitoring disasters and their spread. Simultaneously, by integrating smart building technologies based on Building Information Modeling (BIM) for building facility management with these evacuation-related technologies, the internal conditions of buildings can be understood, allowing for the design of novel evacuation route guidance systems. Traditional technologies focus on predicting disaster situations indoors, thus limiting their ability to provide appropriate evacuation routes to evacuees in chaotic situations during a disaster. Recently, to address this issue, Korean Patent No. 10-2124097 describes a technology that uses IoT sensors to detect smoke and flames and guides evacuees in real-time via their smartphones. However, when fires or other disasters cause IoT sensors to malfunction or become damaged, the ability to guide appropriate evacuation routes is limited due to the constantly changing disaster situation at multiple locations along the evacuation path. Additionally, Korean Patent Publication No. 10-2019-0138222 describes a technology that guides evacuation routes within buildings using smartphones based on augmented reality; however, it is limited to providing pre-set evacuation routes and is limited in its ability to identify and respond to emergencies along the evacuation path in advance.

[0003] [Preliminary Technology Documents]

[0004] [Patent Documents]

[0005] (Patent Document 001) Korean Patent Registration No. 10-2124097

[0006] (Patent Document 002) Korean Patent Publication No. 10-2019-0138222

[0007] (Patent Document 003) European Patent Registration No. 3149716 Summary of the Invention

[0008] The problem to be solved

[0009] The purpose of this invention is to provide a method and system for guiding indoor occupants to safety using portable augmented reality devices, characterized in that it enables indoor occupants in a building to escape quickly and safely in the event of a disaster.

[0010] Furthermore, the present invention provides a method and system for guiding indoor evacuation routes using portable devices and augmented reality, characterized in that: it can understand the emergency situations at multiple locations along the evacuation route, enabling indoor occupants to escape via detours.

[0011] Furthermore, the present invention provides a method and system for guiding indoor evacuation routes using portable devices and augmented reality, characterized in that: it provides location information and usage methods of various emergency items that correspond to the types of disasters along the evacuation route, enabling indoor occupants to evacuate safely.

[0012] Furthermore, the present invention provides a method and system for guiding indoor evacuation routes for personnel using portable devices and augmented reality, characterized in that: a machine learning approach using deep learning is adopted to analyze images taken at multiple locations along the evacuation route, determine whether personnel can move to the analyzed locations, and quickly determine whether evacuees can escape via a detour route.

[0013] Furthermore, this invention provides a method and system for guiding indoor evacuation routes using portable devices and augmented reality. The system is characterized by: installing multiple evacuation guidance auxiliary devices on the ceiling or passageways inside the building; in the event of disasters such as fires or earthquakes, these devices guide people to emergency exits or escape areas; reflecting the individual circumstances or current environmental conditions of evacuees; and outputting messages through audible and visual means, enabling disabled evacuees or children to accurately receive evacuation-related information.

[0014] Problem Solution

[0015] An embodiment provides an augmented reality evacuation route guidance system using a portable device, comprising: a portable device equipped with an evacuation guidance application that provides evacuation routes from the current location of an indoor occupant to a shelter in the event of a disaster, carried by the occupant; and a server that provides multiple evacuation route information via the portable device. The portable device executes the evacuation guidance application to display an evacuation guidance interface. The portable device displays captured landscape images in the image display area of ​​the evacuation guidance interface and displays one of the multiple evacuation routes, set based on disaster type, user mobility, and individual circumstances, in the map display area of ​​the evacuation guidance interface. The portable device overlays the landscape images on the image display area, thereby displaying directional information along the evacuation routes displayed in the map display area, the directional information indicating the direction of movement from the current location.

[0016] On the other hand, an augmented reality evacuation route guidance system using a portable device can be provided, wherein the server sets a priority order for the multiple evacuation routes based on specific information related to the disaster type, the user's mobility, and individual circumstances; and the portable device displays any selected evacuation route on the map display area based on the priority order of the multiple evacuation routes received from the server.

[0017] On the other hand, an evacuation route guidance system using a portable device to augment reality can be provided, wherein the portable device receives location information of at least one emergency item related to the type of disaster from the server and displays it on an evacuation route in the map display area.

[0018] On the other hand, an evacuation route guidance system using a portable device to augment reality can be provided, characterized in that: when the portable device is located on an evacuation route that matches the location of the emergency supplies and is displayed on the map display area, the portable device analyzes the captured image and determines whether the emergency supplies are detected in the captured image. If the portable device detects the emergency supplies in the captured image, it marks them to facilitate easy identification of the emergency supplies and displays a label including information on the usage method of the detected emergency supplies.

[0019] On the other hand, an augmented reality evacuation route guidance system using portable devices can be provided, characterized in that: when the portable device is located on an evacuation route that matches the location of the emergency supplies and is displayed on the map display area, the portable device analyzes the captured image and determines whether the emergency supplies are detected in the captured image. If the portable device fails to detect the emergency supplies in the captured image, it transmits the information that the emergency supplies have been used and the location information of the used emergency supplies to the server. The server then transmits the information that the emergency supplies have been used and the location information of the used emergency supplies to the portable devices of all indoor personnel communicating with the server, so as to clear the location of the used emergency supplies displayed on the evacuation route.

[0020] On the other hand, an augmented reality evacuation route guidance system using portable instruments can be provided, characterized in that: the portable instruments receive from the server video images taken by several other portable instruments at multiple different locations along the remaining path from the current location to the evacuation point, and display them, wherein the video images are waypoint status images.

[0021] On the other hand, an evacuation route guidance system using a portable device to augment reality can be provided, wherein the portable device responds to the displayed waypoint status image and the route reset function availability option, resetting the evacuation route that detours the location on the selected waypoint status image and displaying it on the map display area.

[0022] On the other hand, an evacuation route guidance system using a portable device to augment reality can be provided, characterized in that: as the location of the portable device changes, the status images of several waypoints displayed on the portable device are updated periodically.

[0023] On the other hand, an augmented reality refuge path guidance system using portable devices can be provided, characterized in that: the number of waypoint status images transmitted by the server to the user's portable device is determined based on the remaining distance from the current position of the portable device to the refuge, the number of portable devices of other users on the remaining path from the current position to the refuge, and the spacing between the portable devices of other users.

[0024] On the other hand, an augmented reality evacuation route guidance system using portable devices can be provided, wherein the server receives: video images of the same location taken by multiple users moving along the same evacuation route, analyzes the data of the multiple video images, detects the environmental state information of the same location, and, when it determines that the same location is an unsuitable refuge location, transmits the evacuation route reset request, refuge location information, and video image data of the refuge location to the portable devices of the multiple users moving along the same evacuation route.

[0025] On the other hand, an augmented reality evacuation route guidance system utilizing a portable device can be provided, characterized in that: the portable device determines whether a preset distance has been traveled along the evacuation route, calculates the average speed of traveling the preset distance and the battery consumption of the portable device during the travel of the preset distance, and determines whether the estimated battery consumption required to travel the remaining distance exceeds the current battery charge level. If not, it executes a real-time disaster alert mode and a disaster battery consumption control mode; the portable device extracts usage records from a messenger application with usage history within a given time period and extracts data from within the messenger application for use within the preset time period. The device displays the information of the main conversation partner, responds to the selection of the displayed main conversation partner, and transmits the evacuation route and the current location information of the user to the selected main conversation partner in real time. When a response is received from the selected main conversation partner within a preset time, the device executes the dialogue mode with the selected main conversation partner. The portable device determines whether it senses the pre-registered user biometric information. If the biometric information is sensed, the portable device keeps its display on and transmits the image of the user and the image of the front to the selected main conversation partner. If the biometric information is not sensed, the display is turned off and evacuation information is given via voice prompt.

[0026] Invention Effects

[0027] The embodiment provides a method and system for guiding indoor occupants to evacuation routes using portable devices and augmented reality, which can enable indoor occupants in a building to evacuate quickly and safely in the event of a disaster.

[0028] The embodiment provides a method for guiding indoor evacuation routes using portable devices and augmented reality, and the system for that method can understand emergencies at multiple locations along the evacuation route, enabling indoor occupants to escape via alternative routes.

[0029] The embodiment can provide location information and usage instructions for various emergency supplies located along evacuation routes and conforming to the disaster type. This allows people indoors to quickly learn the location of emergency supplies during evacuation and update the location information based on whether the emergency supplies are used. This solves the problem of wasting time in emergency situations where every second counts, and prevents the waste of time in obtaining used emergency supplies.

[0030] The implementation plan can take into account the type of disaster, the user's mobility, and individual special circumstances, and set multiple evacuation routes and their priorities to provide evacuation routes that are suitable for the current environment and the user.

[0031] The embodiment allows users to understand the actual situation of multiple locations on the remaining path through video images while moving along the evacuation route. It enables them to quickly determine the current situation of the points they pass through and whether they are accessible. When there are locations that are expected to be inaccessible, it sets up detour evacuation routes to avoid those locations.

[0032] The embodiment uses deep learning machine learning to analyze multiple video images of any location on the same evacuation route, determine whether it is safe to go to that location, and notify the user of the result. This allows the user to quickly determine whether the current evacuation route is safe and whether it needs to be reset.

[0033] The implementation can output messages in a form that conforms to the current situation and the user's individual circumstances, based on the disaster type, the user's mobility, individual circumstances, and relevant specific information, through an evacuation guide assistive device paired with the user's portable device, so that disabled evacuees or children can also accurately receive evacuation-related information. Attached Figure Description

[0034] Figure 1 This is a block diagram illustrating the configuration of an evacuation and guidance system according to one embodiment of the present invention.

[0035] Figures 2 to 6 This roughly illustrates the multiple evacuation guide interfaces displayed on the portable device to help users safely travel along evacuation routes to the shelter after running the evacuation guide application on the portable device.

[0036] Figure 7 The method for displaying the status image of the waypoints is shown.

[0037] Figure 8 It roughly shows how to reset the evacuation route and the related evacuation guide interface.

[0038] Figure 9The system of an embodiment is shown in general, which describes a method for a server to receive video image data from multiple portable instruments and analyze environmental state information.

[0039] Figure 10 The evacuation guide interface is roughly shown, which relates to the display of shelter locations and the indication of reset paths using guide labels.

[0040] Figure 11 An example of an evacuation guide interface is shown, which describes a method for detecting the removal status of emergency supplies and clearing the location information of emergency supplies along the evacuation route.

[0041] Figures 12 to 14 The flowchart illustrates the execution of a real-time disaster situation alert mode and a disaster situation battery consumption control mode in an indoor evacuation route guidance method using portable devices and augmented reality.

[0042] Figure 15 and Figure 16 The evacuation guide interface is shown when the real-time disaster alert mode is activated.

[0043] Symbol Explanation

[0044] 10: Escape guidance system; 20: Portable terminal; 30: Smart glass; 100: Portable instrument; 100a: Escape guidance application; 110: Escape guidance interface; 111: Image display area; 112: Map display area; 200: Server; 300: Escape guidance auxiliary device. Detailed Implementation

[0045] This invention can be implemented in various ways and has many embodiments. Specific embodiments will be described below with examples and further detailed explanations. The effects, features, and implementation methods of this invention can be clearly understood by referring to the accompanying drawings and the following embodiments. However, this invention is not limited to the following embodiments and can take many forms. In the following embodiments, terms such as "first" and "second" do not have limiting meanings; their use is to distinguish one component from another. Furthermore, unless the context clearly indicates a difference in meaning, singular expressions include plural expressions. Also, terms such as "including" or "having" indicate the presence of the features or components described in the specification, which does not preclude the possibility of adding more than one other feature or component. Furthermore, in the drawings, for ease of description, the size of several components may be enlarged or reduced. For example, in the drawings, for ease of description, the size and thickness of each component are shown; this invention is not necessarily limited to the illustrations in the drawings.

[0046] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. When describing the invention with reference to the accompanying drawings, the same or corresponding components will be referred to by the same reference numerals, and further descriptions thereof will be omitted.

[0047] Figure 1 This is a block diagram illustrating the configuration of an evacuation and guidance system according to one embodiment of the present invention.

[0048] In one embodiment, the evacuation guidance system 10 can be implemented using a portable device 100 and a server 200. In various embodiments, the evacuation guidance system 10 can be implemented using a portable device 100, a server 200, and an evacuation guidance aid 300.

[0049] The portable instrument 100 and the server 200 are connected via communication lines such as the Internet.

[0050] The portable device 100 can be a portable terminal 20, such as a smartphone. In various embodiments, the portable device 100 can be a wearable computing device, such as smart glass 30 or a head-mounted display. The smart glass 30 type portable device 100 can include a display system that transmits light and displays virtual content (in this embodiment, various information based on augmented reality, etc.) in the user's field of vision, so that the user can see the glass of the surrounding physical space while wearing it. The portable device 100 can receive user input. The portable terminal 20 type portable device 100 can include: physical buttons that can receive physical input, a touch display that can receive touch input, and a microphone system that can receive voice commands. In addition, the smart glass 30 type portable device 100 can include a body language recognition system that can recognize user gestures and other body language, allowing the user to input command signals to the smart glass 30 through body language.

[0051] To implement the evacuation guidance system 10, the portable device 100 is equipped with an evacuation guidance application 100a. In various embodiments, a response can be made to the determination that the portable device 100 is located within a predetermined building, and a message can be transmitted regarding whether the evacuation guidance application 100a has been downloaded and installed on the portable device 100. The user can use the information or link in the message to download and install the evacuation guidance application 100a onto the portable device 100.

[0052] Evacuation Guide App 100a is a software that prompts users (indoor occupants) for evacuation routes from their current location to a shelter in the event of various disasters such as fires or earthquakes within a building.

[0053] Users can activate the evacuation guide application 100a through pre-defined operations. By installing this evacuation guide application 100a on the portable device 100, users can easily reach safe places such as shelters when they suddenly encounter a disaster.

[0054] The portable instrument 100 includes at least one processor, a memory, a camera device, a display device, various sensors, and communication devices. The processor executes an evacuation guide program stored in the memory, providing the user with the functionality of an evacuation guide application 100a. With the execution of the evacuation guide application 100a, the camera device is activated to capture images of the scenery.

[0055] The portable device 100 receives multiple evacuation route information from the server 200 and displays the evacuation route that matches the user through a display device.

[0056] Figures 2 to 6 This roughly illustrates the multiple evacuation guide interfaces displayed on the portable device to help users safely travel along evacuation routes to the shelter after running the evacuation guide application on the portable device.

[0057] like Figure 2 As shown, when a user executes the evacuation guide application 100a on the portable device 100, the evacuation guide interface 110 can be displayed. Then, the portable device 100's camera takes a picture, and the captured landscape image is displayed on the display device. Furthermore, a map including the user's current location can be displayed on the portable device 100.

[0058] The user's current location can be the location of the portable device 100 carried by the user. The location of the portable device 100 can be estimated through one of the following methods: a method of estimating the location by analyzing the sensing information of the geomagnetic sensor built into the portable device 100; a wireless communication technology (Wi-Fi) positioning method that estimates the location using surrounding wireless communication technology (Wi-Fi); a fingerprinting-based location determination method that uses Wi-Fi / Bluetooth-based indoor location determination technology; a trilateration-based location determination method; a BLE-based location recognition method; a location determination method using the Earth's magnetic field; or a camera-based location recognition method. However, the present invention is not limited to the above-mentioned location estimation methods.

[0059] The captured landscape images are displayed in the image display area 111, and the map is displayed in the map display area 112. The image display area 111 and the map display area 112 can be displayed separately. The attached figure illustrates the state of the portable device 100 as a portable terminal 20, but the present invention is not limited thereto. When it is a smart glass 30, the image display area 111 and the map display area 112 can be displayed in different positions in various forms.

[0060] The map display area 112 can display the current location information (cl) of the portable device 100. Furthermore, as the location of the portable device 100 changes, the location information (cl) on the map displayed in the map display area 112 also changes accordingly. In response to movement of the portable device 100, the location information (cl) also moves.

[0061] like Figure 2 and Figure 3 As shown, the evacuation guide interface 110 overlaps with a portion of the landscape image displayed on the image display area 111, displaying multiple status input function options ia1, ia2, and ia3. The evacuation guide interface 110 responds to one of the multiple status input function options ia1, ia2, and ia3 displayed (e.g., touch input) and displays multiple specific information input function options (sa) that match the selected function option ia1.

[0062] One of the multiple status input function options ia1, ia2, and ia3 can be a button used to input the disaster type and display specific information. Another of the multiple status input function options ia1, ia2, and ia3 can be a button related to the user's mobility and display specific information. Yet another of the multiple status input function options ia1, ia2, and ia3 can be a button related to the user's individual circumstances and display specific information.

[0063] Regarding the type of disaster, the specific information input function can be a button that allows input of information such as earthquakes, fires, and hazardous material leaks.

[0064] Regarding the user's mobility, the specific information input option could be a button for inputting information such as "unable to walk," "unable to climb," "unable to run," or "leg injury." Furthermore, regarding the user's individual circumstances, the specific information input option could be a button for inputting information related to individual special circumstances such as "children under a certain age," "deaf or disabled persons," "children accompanying adults," "wheelchairs," or "strollers." The user can choose one of several displayed specific information input options (sa) to input specific information related to the type of disaster currently occurring, their current mobility, and their individual special circumstances.

[0065] The portable device 100 can transmit specific information related to the type of disaster, the user's mobility, individual circumstances, and the current location information of the portable device 100 to the server 200.

[0066] Based on specific information related to the type of disaster received, the user's mobility, individual circumstances, and the current location information of the portable device 100, the server 200 sets multiple evacuation routes from the current location of the portable device 100 to the shelter.

[0067] Server 200 sets appropriate evacuation routes based on specific information related to the type of disaster, the user's mobility, and individual circumstances.

[0068] For example, in the event of a fire, the system can pre-register the best evacuation routes to pre-registered shelters. Furthermore, if a user has a leg injury, is in a wheelchair, or is pushing a stroller, it is considered an inability to use stairs, and evacuation routes that avoid stairs are pre-set. Additionally, if the user is a child under a certain age or a deaf or disabled person, further auxiliary evacuation guidance information is required, and evacuation routes via the area where the evacuation guide assist device 300 is located are pre-set.

[0069] Server 200 can generate multiple evacuation route information and set their priority order. Server 200 can set the priority order of multiple evacuation route information based on specific information related to disaster type, user mobility, and individual circumstances.

[0070] Server 200 can provide multiple evacuation routes to portable device 100.

[0071] The evacuation guide interface 110 responds to the selection of the displayed route setting function availability (ra) and overlays the highest priority evacuation route among the multiple evacuation routes received from the server 200 onto the pre-displayed map on the map display area 112 to represent the evacuation route.

[0072] In various embodiments, the evacuation guide interface 110 may display additional secondary evacuation routes besides the evacuation routes shown when the evacuation route setting function availability (ra) is selected. Therefore, the user can select the path setting function availability (ra) to change the displayed evacuation route.

[0073] In various embodiments, the evacuation guide interface 110 may further display priority information of the displayed evacuation routes.

[0074] like Figure 4 As shown, the refuge guide interface 110 displays the movement direction guidance information of the landscape image superimposed on the image display area 111 based on the position information and tilt angle information of the portable instrument 100.

[0075] For example, arrows or other means can be used to show the path from the current location of the portable device 100 along the evacuation route to guide the user's direction of movement.

[0076] The evacuation guide interface 110 can display the image captured by the portable device 100 on the image display area 111, and overlay it to display evacuation path guide information showing the direction and distance of movement along the evacuation path from the current position. Based on the user's position and tilt information, the portable device 100 appropriately adjusts the evacuation path guide information to guide the user to move along the evacuation path in the correct direction.

[0077] like Figure 5 As shown, the evacuation guide interface 110 can display the location information of at least one emergency item located on the evacuation route on the map display area 112. In various embodiments, once the evacuation route information to be displayed on the map display area 112 is determined, the portable device 100 can receive from the server 200 the location information of the emergency item located on the evacuation route to be displayed. Furthermore, if the evacuation route information to be displayed on the map display area 112 is changed, the portable device 100 can receive from the server 200 the location information of the emergency item located on the changed evacuation route.

[0078] For example, when the input fire type is fire, the evacuation guide interface 110 receives the location information of multiple emergency items on the map from the server 200, and only filters out the emergency items located on the evacuation route displayed on the map display area 112, and displays them on the map display area 112.

[0079] In various embodiments, the evacuation guide interface 110 can display emergency item icons (esi) at the location of the emergency item, which are shaped in a way that makes it easy to identify the type of emergency item from the map display area 112.

[0080] When the portable device 100 determines that an emergency item is located on a path matching its configuration location, the evacuation guide interface 110 can analyze the captured image to determine whether the emergency item is detected within the image. If the portable device 100 detects an emergency item within the captured image, the evacuation guide interface 110 can further display various markers that easily identify the emergency item (es) (as shown in the example, a quadrilateral border surrounding the emergency item can be displayed). Furthermore, the evacuation guide interface 110 can display a label (tg) on ​​the image display area 111, including information on how to use the emergency item (es). The label (tg) can be displayed around the detected emergency item (es).

[0081] like Figure 6As shown, server 200 can receive evacuation route information currently displayed on portable device 100 from portable device 100. Furthermore, server 200 can periodically receive current location information of portable device 100 from portable device 100. Additionally, server 200 can periodically receive images captured by portable device 100 and location information of the image capture location from portable device 100. In various embodiments, for rapid data transmission and reception, the images transmitted from portable device 100 to server 200 can be image data.

[0082] Server 200 can transmit to portable device 100 image data representing the status of transit points captured by other portable devices at multiple locations along the refuge route from the current location of portable device 100 to the shelter. In various embodiments, the images captured by server 200 and provided to portable device 100 may be image data.

[0083] Figure 7 The method for displaying the status image of the waypoints is shown.

[0084] like Figure 6 and Figure 7 As shown, by collecting image data captured by multiple portable devices along the evacuation path and referring to relevant data, server 200 can receive images a, b, and c of the images captured by these devices from the following components: several other portable devices 100ap1, 100ap2, and 100ap3 located at different positions along the evacuation path p from the starting point SP to the shelter (SH), from the current location of the user's portable device 100p to the shelter (SH). Furthermore, server 200 can transmit the status image data of the path points for the received images a, b, and c to the user's portable device 100p. As shown in the example, server 200 will transmit images received from three other portable devices 100ap1, 100ap2, and 100ap3 to user's portable device 100p. However, not limited to this, the number of images transmitted by server 200 to user's portable device 100p will vary based on the remaining distance from user's portable device 100p's current location to the shelter (SH), the number of other portable devices on the remaining path, and the spacing between other portable devices.

[0085] The portable instrument 100 can receive waypoint status image data from the server 200 and display the waypoint status images. In various embodiments, the location information of multiple waypoint status images captured can be displayed on the map display area 112.

[0086] Users can view video images of multiple locations along the remaining evacuation route on the portable device 100. Furthermore, the video images of these locations can be periodically updated as the portable device 100 changes position. Users can use the video images to assess the status of evacuees at these locations, the surrounding environment, and safety information. If a user deems a location on the evacuation route unsuitable, they can select the route setting availability (ra) to reset the evacuation route, thereby changing the displayed evacuation route and using an alternative evacuation route.

[0087] Figure 8 This roughly shows the evacuation wizard interface related to the evacuation route reset method.

[0088] like Figure 8 As shown, in various embodiments, the evacuation guide application 100a can respond to the selection of one of the video images of multiple locations on the displayed remaining path and the availability of the path setting function (ra), and can change to an evacuation path (cp) that detours to the location on the selected video image and display it. Specifically, the evacuation guide application 100a extracts evacuation path information from multiple evacuation path information that does not include the location captured in the selected video image, and displays the evacuation path (cp) with the highest priority among the extracted evacuation paths. Therefore, when the user confirms that the video images of multiple locations on the displayed remaining path are not suitable as a movement path, they select the path setting function availability (ra) of the selected video image, thereby resetting the evacuation path that detours to the selected location, and detours to the reset evacuation path to achieve evacuation.

[0089] Figure 9 The system of an embodiment is shown in general, illustrating a method by which a server receives video image data from multiple portable devices and analyzes environmental state information.

[0090] like Figure 9 As shown, server 200 can receive video image data taken by multiple users moving along the same evacuation route. Server 200 can periodically receive video image data taken at multiple locations along the evacuation route. After receiving video image data taken at one of the multiple locations multiple times, server 200 analyzes the multiple video images to detect the environmental status information at that location. This environmental status information may be about whether a fire has occurred at that location, the density of people, but is not limited to these; it may be information used to determine whether it is suitable for evacuees to go to that location.

[0091] To analyze multiple video images and determine whether a fire has occurred at a specific location, server 200 can use a CNN (Convolutional Neural Network) to extract features from smoke and spark images to determine whether a fire has occurred, but it is not limited to this method. Any method that can determine whether a fire has occurred at a location in a video image or whether fire victims can move through video image analysis can be used. Furthermore, server 200 can use a deep-learning network to analyze multiple video images and detect the population density information at locations in the video images.

[0092] As described above, when server 200 determines that the analyzed location is unsuitable for personnel movement and is an immobile refuge location, it can transmit the refuge route reset request, the location information of the refuge location, and video image data of the refuge location to: several portable devices that have identified locations unsuitable or immobile for movement by multiple users as refuge locations and display the refuge routes associated with them.

[0093] Figure 10 The interface for the evacuation guide is roughly shown, which displays guide labels that mark refuge locations and reset routes.

[0094] like Figure 10 As shown, the portable device 100 can display the received evacuation location (dp) on the map display area 112. Furthermore, the portable device 100 can display the received evacuation location video image (dpi) and guide labels (e.g., the location marked on the map is the site of a fire; please reset the evacuation route, etc.).

[0095] The system checks the video image (dpi) of the displayed refuge location and, taking into account the user's physical condition, determines whether the user can traverse the designated refuge location. If the system determines that the user cannot traverse the refuge location, it resets the path so that the user can reach refuge through a detour.

[0096] Re-reference Figure 1 It is understood that, in various embodiments, at least one evacuation guide auxiliary device 300 may be provided on the evacuation route.

[0097] Multiple evacuation guide assist devices 300 can be installed in ceilings or passageways inside buildings to guide trapped personnel to emergency exits or escape routes in the event of disasters such as fires or earthquakes.

[0098] The evacuation guide assist device 300 can be paired with at least one portable device 100 located within a preset radius. When a user moves along an evacuation path and arrives at a point adjacent to the evacuation guide assist device 300, the evacuation guide assist device 300 can automatically pair with the portable device 100 carried by the user. The evacuation guide assist device 300 can receive specific information related to the disaster type, the user's mobility ability, and individual circumstances from the paired portable device 100. For example, if the user is deaf or disabled, visual information can be output regarding the measures to be taken while moving along the evacuation path or according to the disaster situation. Furthermore, if the user is a child under a certain age, the evacuation guide assist device 300 can display text or images easily understood by children regarding the measures to be taken while moving along the evacuation path or according to the disaster situation.

[0099] The evacuation guide assist device 300 is paired with the user's portable instrument 100 located within a preset radius. Taking into account the type of disaster, the user's mobility, and the user's special circumstances, it provides messages that are appropriate to the user's status and current situation through audible and / or visual means, thereby helping the user to evacuate safely and quickly.

[0100] Figure 11 An example of an evacuation guide interface is shown, which describes a method for detecting when emergency items have been removed and clearing the location information of emergency items on the evacuation route.

[0101] like Figure 5 and Figure 11 As shown, when the evacuation guide interface 110 determines that the portable instrument 100 is located on a path that matches the location of the emergency supplies, it can analyze the captured images to determine whether the emergency supplies can be detected from the captured images.

[0102] Suppose a user uses emergency supplies configured on an evacuation route. At this time, the evacuation guide interface of portable devices 100 belonging to other users following behind the user will determine that the portable device is located on a path matching the location of the emergency supplies and analyze the captured image. However, since the emergency supplies have already been used, they cannot be detected in the captured image. At this point, the portable device transmits the information about the emergency supplies being used and their location to the server 200. Upon receiving this information, the server 200 transmits the used emergency supplies and their location to the portable devices of all indoor personnel communicating with the server 200.

[0103] like Figure 11As shown by the dashed circle on the map of area 112, the portable device that receives information on used emergency supplies and their locations can clear the emergency supply icon (esi) displayed on the evacuation route to which the location of the used emergency supplies belongs and update the location information of the emergency supplies.

[0104] Figures 12 to 14 The flowchart illustrates a method for guiding indoor evacuation routes using portable devices and augmented reality, including a real-time disaster situation alert mode and a disaster situation battery consumption control mode. Furthermore, Figure 15 and Figure 16 The evacuation guide interface is shown when the real-time disaster alert mode is activated.

[0105] like Figure 12 As shown, the method for guiding indoor evacuation routes using a portable device in augmented reality, including the execution of a real-time disaster situation alert mode and a disaster situation battery consumption control mode (S100), may include the following steps: First, the portable device 100 may respond to a request from the evacuation guide application 100a by executing the application 100a (S110) and determining whether the evacuation route (p) has been displayed (S120). When the portable device 100 determines that the evacuation route (p) has been displayed, it may periodically determine whether the user has moved a preset distance along the evacuation route (p) based on the location information of the portable device 100 (S130). When the portable device 100 determines that the user has moved a preset distance along the evacuation route (p), it may calculate the average moving speed over the preset distance and calculate the battery consumption of the portable device 100 during the preset distance movement (S140). The portable instrument 100 can calculate an estimated value of battery consumption when moving the remaining path based on the average moving speed over the calculated preset distance and the battery consumption of the portable instrument 100 when moving the preset distance (S140). That is, the portable instrument 100 can calculate an estimated value of battery consumption when moving the remaining path on the refuge path (p).

[0106] Secondly, the portable instrument 100 can determine whether the estimated value of battery consumption when moving the remaining path exceeds the current battery charge (S150).

[0107] When the estimated battery consumption exceeds the current battery charge level while moving the remaining path, the portable instrument 100 can return to the step of determining whether the preset distance has been moved from the current position along the evacuation path (p) (S130).

[0108] During the user's movement to the evacuation route (p), the above process will be repeated periodically. This allows monitoring to determine whether the portable device 100's battery charge is sufficient to keep it activated until the evacuation site is reached.

[0109] When the portable device 100 determines that the estimated battery consumption is lower than the current battery charge level when moving the remaining path, it can execute the real-time disaster alert mode and the disaster battery consumption control mode (S160, S170).

[0110] like Figure 13 , Figure 15 and Figure 16 As shown, in the real-time disaster alert mode (S160) of the portable instrument 100, the following steps can be performed.

[0111] When the first set of applications is executed in background mode, the portable instrument 100 can forcibly terminate the first set of applications to reduce battery consumption (S161). The first set of applications refers to all applications other than the additional applications and messenger applications required for the operation of the evacuation guide application 100a.

[0112] The portable device 100 can execute a second set of applications to extract information about the main dialogue parties engaged in a conversation within a preset time period (S162). The second set of applications can be a messenger application with usage records from the current time point to the given time period.

[0113] The portable instrument 100 extracts information from multiple main dialogue parties, expanding the user's selection of dialogue parties suitable for providing current disaster information.

[0114] like Figure 15As shown, the portable instrument 100 can overlay the icons (m.app) of the second set of applications on the image display area 111 and display them separately from the applications: the main conversational functional availability (app1.c.u1, app1.c.u2, app1.c.u3, app2.c.u1, app2.c.u2, app2.c.u3) extracted from the messenger application corresponding to each icon (m.app). The portable instrument 100 can determine whether to select one of the displayed main conversational functional availability (app1.c.u1, app1.c.u2, app1.c.u3, app2.c.u1, app2.c.u2, app2.c.u3) (e.g., touch input) (S164). The portable device 100 can respond to a selection of one of several functional availabilityes (app1.c.u1, app1.c.u2, app1.c.u3, app2.c.u1, app2.c.u2, app2.c.u3) from the displayed main dialogue party, and forcibly terminate the remaining messenger applications except for the messenger application corresponding to the selected main dialogue party's functional availability app1.c.u2 (S165). The portable device 100 can transmit map accuracy and evacuation route information displayed on the map display area 112, as well as the user's current location information, to the selected main dialogue party through the messenger application (S166).

[0115] The portable instrument 100 can determine whether a response has been received from the selected main dialogue party within a preset time (S167). If a response is not received from the selected main dialogue party within the preset time, the portable instrument 100 can return to step S162 to extract information from the other main dialogue parties besides those executing the second set of applications and the selected main dialogue party. If a response is received from the selected main dialogue party within the preset time, such as... Figure 16 As shown, the portable instrument 100 can overlay the dialogue mode window (cw) between itself and the selected main dialogue party on the image display area 111, and execute the dialogue mode between itself and the selected main dialogue party to display the message received from the selected main dialogue party (S168).

[0116] like Figure 14As shown, the portable device 100 can determine whether it has detected the user's biometric information (S171) during the execution of the disaster situation battery consumption control mode (S170). The biometric information can be the user's facial information or iris information pre-registered for unlocking the portable device 100 using its omnidirectional camera. When the user's biometric information is detected, the portable device 100 can keep the display on (S173). Furthermore, the portable device 100 can periodically transmit information about the user's image captured by the omnidirectional camera and images taken from the front to the selected main dialogue party (S174). When the user's biometric information is not detected, to reduce battery consumption, the portable device 100 can turn off the display and provide evacuation information via voice prompt. In some embodiments, the portable device 100 can detect the level of external noise and adjust the volume of the evacuation information voice prompt output. The portable device 100 can continuously sense the user's biometric information (S171). The display is kept on only when the user looks at the portable device 100. When the user does not look at the portable device 100 during the evacuation, the display can be turned off to reduce battery consumption and provide evacuation information via voice prompts.

[0117] As described above, embodiments of the present invention are implemented in the form of program instructions executable by various computer components and stored on a computer-readable medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., individually or in combination. The program instructions recorded on the computer-readable recording medium may be specifically designed and configured for the present invention, or may be known and used by those skilled in the art of computer software. Examples of computer-readable recording media include: magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floppy disks; and specialized hardware devices for storing and executing program instructions, such as ROMs, RAMs, and flash memory. Examples of program instructions include not only machine language code compiled by a compiler, but also high-level language code that can be executed by a computer using an interpreter, etc. To execute the present invention, the hardware device can be changed to one or more software modules, and vice versa.

[0118] The specific implementation described in this invention is one embodiment, and no method is intended to limit the scope of the invention. For the sake of simplicity and clarity, descriptions of conventional electronic components, control systems, software, and other functional aspects of said systems may be omitted. Furthermore, the linear connections or connectors between components shown in the drawings exemplify functional connections and / or physical or loop connections; in actual devices, various alternative or additional functional connections, physical connections, or loop connections may exist. Moreover, components not specifically mentioned as "essential" or "important" may not be essential for the application of this invention.

[0119] Furthermore, it should be understood that although the specific description of the present invention has been illustrated with reference to preferred embodiments, various modifications and alterations can be made by those skilled in the art or those with common knowledge in the art without departing from the inventive concept and technical scope set forth in the appended claims. The technical scope of the present invention is not limited by the contents of the specification but should be defined by the claims.

Claims

1. An evacuation route guidance system utilizing portable instruments to augment reality, characterized in that: include: Portable devices equipped with an evacuation guide application that displays evacuation routes from the current location of occupants to a shelter in the event of a disaster, and are carried by occupants; and The server provides information on multiple evacuation routes via the portable device. The portable device executes the evacuation guide application and displays the evacuation guide interface. The portable device displays the captured landscape images in the image display area of ​​the evacuation guide interface, and displays one of the multiple evacuation routes set based on specific information such as disaster type, user mobility, and individual circumstances on the map display area of ​​the evacuation guide interface. The portable instrument overlays the landscape image onto the image display area, thereby displaying directional information along the evacuation route shown on the map display area. This directional information indicates the direction of movement from the current location. The portable device receives location information of at least one emergency item related to the type of disaster from the server and displays it on the evacuation routes in the map display area. When the portable device is located on an evacuation route that matches the location of the emergency supplies and is displayed on the map display area, the portable device analyzes the captured image and determines whether the emergency supplies are detected in the captured image. If the portable device fails to detect emergency supplies in the captured image, it will transmit information about the emergency supplies' use-out status and their location to the server. The server transmits the used emergency supplies and their location information to the portable devices of all indoor personnel communicating with the server, so as to clear the location of the used emergency supplies displayed on the evacuation route.

2. The evacuation route guidance system using portable instruments and augmented reality as described in claim 1, characterized in that: The server prioritizes the multiple evacuation routes based on specific information related to the disaster type, the user's mobility, and individual circumstances. The portable device displays any selected refuge route on the map display area based on the priority order of multiple refuge routes received from the server.

3. The evacuation route guidance system using portable instruments and augmented reality as described in claim 1, characterized in that: When the portable device is located on an evacuation route that matches the location of the emergency supplies and is displayed on the map display area, the portable device analyzes the captured image and determines whether the emergency supplies are detected in the captured image. If the portable device detects emergency items in the captured image, it marks them to facilitate easy identification of the emergency items and displays labels including information on how to use the detected emergency items.

4. The evacuation route guidance system using portable instruments and augmented reality as described in claim 1, characterized in that: The portable device receives from the server video images captured by several other portable devices at multiple different locations along the remaining path from the current location to the shelter, and displays these video images as status images of the waypoints.

5. The evacuation route guidance system using portable instruments and augmented reality as described in claim 4, characterized in that: The portable instrument responds to the displayed waypoint status image and the route reset function options, resetting the refuge route to the location on the selected waypoint status image and displaying it on the map display area.

6. The evacuation route guidance system using portable instruments and augmented reality as described in claim 4, characterized in that: As the location of the portable instrument changes, the status images of several waypoints displayed on the portable instrument are updated periodically.

7. The evacuation route guidance system using portable instruments and augmented reality as described in claim 6, characterized in that: Based on the remaining distance from the current location of the portable device to the shelter, the remaining path from the current location to the shelter, the number of portable devices of other users, and the spacing between the portable devices of other users, the number of path point status images transmitted by the server to the user's portable device is determined.

8. The evacuation route guidance system using portable instruments and augmented reality as described in claim 1, characterized in that: The server receives data of video images taken by multiple users who moved along the same evacuation route at the same location. By analyzing data from multiple captured video images, detecting environmental status information at the same location, and determining that the same location is an unsuitable refuge location, the refuge route reset request, refuge location information, and video image data of the refuge location are transmitted to the portable devices of multiple users moving along the same refuge route.

9. The evacuation route guidance system using portable instruments and augmented reality according to claim 1, characterized in that: The portable device determines whether it has moved a preset distance along the evacuation path, calculates the average speed of moving the preset distance and the battery consumption of the portable device during the movement of the preset distance, and determines whether the estimated battery consumption required to move the remaining distance exceeds the current battery charge level. If not, it executes the real-time disaster alert mode and the disaster battery consumption control mode. The portable device extracts and displays information about the main conversation partners within a preset time frame from messenger applications with usage records within a given time period and from within those applications. It responds to selections of the displayed main conversation partner information, transmits evacuation routes and the current user's location information to the selected main conversation partner in real time, and executes a dialogue mode with the selected main conversation partner upon receiving a response from them within the preset time frame. The portable device determines whether it senses the pre-registered user's biometric information. If the biometric information is sensed, the portable device keeps its display on and transmits the image of the user and the image of the front to the selected main dialogue party. If the biometric information is not sensed, the display is turned off and an evacuation message is given via voice.

Images