System

Abstract

A system is provided.SOLUTION: A system comprising: means for acquiring, pre-processing, and analyzing real-time video from a live camera; means for acquiring, integrating, and analyzing hazard map data; means for calculating an optimal evacuation route based on the acquired and analyzed information; and means for reflecting the calculated evacuation route in a map application and displaying it to a user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, natural disasters have become more frequent, necessitating quick and safe evacuation. However, in reality, many people are often exposed to dangerous situations due to not knowing the appropriate evacuation route. Travelers and people unfamiliar with the area often face difficulties when a disaster occurs, unable to determine which route to take to safely evacuate. Furthermore, conventional evacuation information systems have limited functionality for providing real-time road conditions and disaster information, which can result in delays in providing appropriate evacuation routes. Given this background, there is an urgent need to provide optimal evacuation routes in real time when a disaster occurs and support safe evacuation behavior. [Means for solving the problem]

[0005] To solve the above-mentioned problems, the present invention provides the following means. First, the system includes a means for acquiring real-time video from a live camera and performing preprocessing and analysis using an image recognition algorithm and a machine learning model. This means can quickly determine whether roads are passable and whether there are obstacles. Next, the system includes a means for acquiring hazard map data and performing analysis using integrated and different analysis models corresponding to each type of disaster. This means can accurately grasp the extent of disaster impact and dangerous areas. Furthermore, the system includes a means for calculating an optimal evacuation route based on the acquired and analyzed information using the Dijkstra algorithm, A algorithm, or the like. This means allows users to obtain a safe and efficient evacuation route that is updated in real time. Finally, the system includes a means for reflecting the calculated evacuation route in a map application and displaying it visually to the user. This means allows users to easily confirm the optimal route from their current location to an evacuation site and take safe evacuation action.

[0006] A "live camera" is a camera that captures video in real time and distributes that video to remote locations via the Internet or a network.

[0007] "Real-time video" refers to video that is currently being shot and distributed almost immediately, with very little time lag.

[0008] "Preprocessing" refers to performing preprocessing such as noise removal, resizing, and normalization on the original data to make video and data analysis easier.

[0009] An "image recognition algorithm" is a computational method that allows a computer to automatically recognize and classify specific objects or patterns from image data.

[0010] A "machine learning model" is a mathematical model that learns patterns based on data and uses them to make predictions and classifications.

[0011] "Hazard map data" is map-format data that visualizes the risk and danger of disasters in a specific area.

[0012] "Integration" refers to bringing together multiple data sets or information sources and treating them in a consistent manner.

[0013] The "optimal evacuation route" is the shortest and least dangerous route that allows users to safely evacuate from a disaster.

[0014] "Dijkstra's algorithm" is a geometric algorithm that is used to find the shortest path from a specific vertex to all other vertices.

[0015] The "A algorithm" is a graph search algorithm that is a method for finding the shortest path from the starting point to the goal point, and improves search efficiency by using heuristic functions.

[0016] A "map application" is application software that provides map information on an electronic device and provides route guidance to a destination. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6]FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention is a system that supports safe evacuation in the event of a disaster, integrating live cameras, hazard maps, and map applications to provide optimal evacuation routes in real time. The specific operation and program processing of this system are described below.

[0039] System Overview

[0040] This system consists of multiple servers and user terminals. The servers have the means to collect and analyze video data from live cameras. They also acquire and analyze hazard map data to assess the impact of disasters. They also have the means to calculate optimal evacuation routes and provide the results to users. Meanwhile, the user terminals use a map application to receive and visually display evacuation route information.

[0041] What the program does

[0042] The program processing of the system will be explained in natural language below.

[0043] Collection and analysis of live camera footage

[0044] Server: Real-time video is acquired from multiple live cameras installed in the surrounding area. This video is first pre-processed and then analyzed using image recognition algorithms and machine learning models to determine whether roads are passable and whether there are any obstacles.

[0045] Hazard map data integration and analysis

[0046] Server: Acquires hazard map data provided by government agencies and public institutions. This data is integrated into a GIS database and subjected to analytical models to identify the extent of disaster impact and risk areas. Different analytical models are applied for different types of disasters (e.g., floods, fires, earthquakes, etc.).

[0047] Calculating the optimal evacuation route

[0048] Server: Integrates the analysis results of live camera footage with hazard map data to calculate the optimal evacuation route based on the departure and destination points specified by the user. Path-finding algorithms such as Dijkstra's algorithm and A algorithm are used. The calculated evacuation route information is immediately saved in a database.

[0049] Route information provided by map applications

[0050] Device: The user launches a map application on their smartphone or tablet and inputs their current location and evacuation destination. The application sends this information to the server, from which it receives information on the optimal evacuation route.

[0051] Server: Receives user requests and returns optimal evacuation route information in real time.

[0052] Device: The received evacuation route information is displayed on a map application, allowing the user to check the information and evacuate safely.

[0053] Specific examples

[0054] Below are some examples of specific scenarios that use this system.

[0055] Disaster scenario

[0056] User B is a traveler staying in a flood-hit area and is looking for a route to a suitable evacuation site.

[0057] 1. Collection and analysis of live camera footage

[0058] Server: Acquires and analyzes real-time video from multiple live cameras. From the acquired video data, it is possible to determine whether major roads are flooded.

[0059] 2. Integration and analysis of hazard map data

[0060] Server: Obtains the latest hazard map data and identifies the area affected by the flood. Based on this data, the server understands the status of the disaster affecting User B's location.

[0061] 3. Calculating the optimal evacuation route

[0062] Server: Integrates the analysis results of live camera footage with hazard map data to calculate the optimal route from User B's current location to the nearest evacuation site. Using Dijkstra's algorithm, the safest and most efficient route is derived and the information is stored in a database.

[0063] 4. Route information provided by map applications

[0064] Device: User B launches a map application and inputs their current location and evacuation destination. The application sends a request to the server and receives information on the optimal evacuation route.

[0065] Server: Receives User B's request and returns the optimal route information calculated in real time.

[0066] Device: The map application displays the received information, and User B can visually check the route.

[0067] 5. Supporting users in evacuation

[0068] User: User B follows the instructions of the application and safely travels to the evacuation site via the designated route.

[0069] In this way, the present invention allows users to obtain appropriate evacuation routes based on real-time information when a disaster occurs, enabling safe and rapid evacuation.

[0070] The processing flow will be explained below.

[0071] Step 1: Acquire live camera footage

[0072] Server: Accesses live cameras installed in the specified area and acquires video in real time. Collects data periodically using the endpoint URL and authentication information.

[0073] Step 2: Preprocessing the video data

[0074] Server: Preprocesses the captured live camera footage. Specifically, it resizes the footage, removes noise, corrects color, etc., and converts it into a state suitable for analysis.

[0075] Step 3: Analyzing the video data

[0076] Server: Analyzes the pre-processed video data using image recognition algorithms and machine learning models, extracting information such as road passability, obstacles, and flood damage.

[0077] Step 4: Save the analysis results

[0078] Server: Stores the analysis results in a database, including location information, traffic conditions, and details of detected obstacles.

[0079] Step 5: Obtaining hazard map data

[0080] Server: Obtains the latest hazard map data provided by government agencies and public institutions using API and FTP.

[0081] Step 6: Preprocessing and integration of hazard map data

[0082] Server: Performs preprocessing to integrate acquired hazard map data into the GIS database, including data format conversion, spatial correction, and filtering.

[0083] Step 7: Analyze hazard map data

[0084] Server: Analyzes the integrated hazard map data to identify the extent of disaster impacts and risk areas. Different analytical models are used for each disaster, such as floods, fires, and earthquakes.

[0085] Step 8: Calculate the optimal evacuation route

[0086] Server: Based on the results of live camera analysis and hazard map data, the server calculates the optimal evacuation route for the departure and destination points specified by the user. It uses Dijkstra's algorithm and A algorithm to derive a safe and efficient route.

[0087] Step 9: Save the evacuation route data

[0088] Server: The calculated optimal evacuation route information is stored in a database so that subsequent client requests can be handled promptly.

[0089] Step 10: Accepting a user request

[0090] Device: The user launches a map application and inputs their current location and evacuation destination. This information is sent as a request to the server.

[0091] Step 11: Send evacuation route information

[0092] Server: Receives user requests, generates optimal evacuation route information based on the latest analysis data and route calculation results, and returns the generated information to the device.

[0093] Step 12: Display evacuation route information

[0094] Device: The received evacuation route information is displayed in a map application, showing the route and important points (obstacles, evacuation points, etc.) to make it easier for the user to understand visually.

[0095] Step 13: Start evacuation and check the situation

[0096] User: While looking at the map application, the user begins evacuation by following the optimal route displayed. While on the move, the user can check the latest analysis data through the application to ensure safety.

[0097] Step 14: Notification of evacuation completion

[0098] User: Upon reaching the evacuation location, the application will notify the user that evacuation is complete. If necessary, the user will be able to check the recalculated route.

[0099] This series of processes allows users to evacuate quickly and safely based on information updated in real time.

[0100] Example 1

[0101] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0102] When a disaster occurs, it is important to provide safe and effective evacuation routes in real time. However, conventional evacuation support systems have difficulty collecting real-time information and calculating optimal evacuation routes based on that information. Furthermore, they do not perform sufficient analysis according to different disaster types, making it impossible to quickly provide optimal evacuation routes for users. This leads to evacuation delays and the selection of inappropriate evacuation routes, which can compromise user safety.

[0103] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0104] In this invention, the server includes means for acquiring real-time video from a live camera and analyzing it using preprocessing and image recognition algorithms and machine learning models, means for acquiring hazard map data from public institutions, integrating it into a GIS database, and analyzing it using different analysis models depending on the disaster, means for calculating the optimal evacuation route from the departure point and destination using a route search algorithm based on the acquired and analyzed video data and hazard map data, and means for saving the calculated evacuation route in the database and reflecting it in a map application and displaying it to the user upon user request. This makes it possible to quickly provide the optimal and safest evacuation route based on information collected in real time when a disaster occurs.

[0105] A "live camera" is an image capturing device installed to capture real-time images of the surrounding area.

[0106] "Preprocessing" refers to the process of performing processes such as noise removal and frame resizing to make the acquired video data easier to analyze.

[0107] An "image recognition algorithm" is a computational method for detecting specific objects or patterns within video data.

[0108] A "machine learning model" is a collection of algorithms that automatically learn from data and make predictions and classifications.

[0109] "Hazard map data" is geographic information that indicates the risk and impact of disasters.

[0110] "Public institutions" are public data providers such as government agencies and local governments.

[0111] A "GIS database" is a database system that manages geographic information in an integrated manner.

[0112] A "route search algorithm" is a computational method for finding the optimal route between a specified starting point and destination.

[0113] A "database" is a system for systematically storing information and efficiently retrieving it when needed.

[0114] A "user terminal" is an electronic device, such as a smartphone or tablet, that runs a map application.

[0115] A "map application" is software that displays a user's current location, destination, and route information.

[0116] This invention is a system for supporting safe evacuation in the event of a disaster, integrating live cameras, hazard maps, and map applications to provide optimal evacuation routes in real time. This system is composed of multiple servers and user terminals, and a specific embodiment is shown below.

[0117] System Overview

[0118] The system mainly uses a server, live cameras, user devices, map applications, and hazard map data. Below, we will explain the details of each component and how they work.

[0119] Collection and analysis of live camera footage

[0120] Server: The server acquires real-time video from multiple live cameras installed in the surrounding area. The video is transferred to the server at a fixed frame rate and received using a streaming protocol (e.g., RTSP). The video data is pre-processed and analyzed using a machine learning model (e.g., the TENSORFLOW® model).

[0121] Specifically, the server connects to each live camera and captures video data in real time. This data undergoes preprocessing such as noise removal and frame resizing, and is then analyzed using image recognition algorithms and machine learning models. This allows the system to determine whether the road is passable and whether there are any obstacles.

[0122] Acquisition and analysis of hazard map data

[0123] Server: The server periodically retrieves hazard map data provided by government agencies and public institutions. This data is retrieved through APIs and stored in a GIS database. The retrieved data is analyzed using analytical models (e.g., ArcGIS) to identify the extent of disaster impact and risk areas. In this case, analytical models appropriate for different disasters, such as floods, fires, and earthquakes, are applied.

[0124] Calculating the optimal evacuation route

[0125] Server: The server integrates the analysis results of live camera footage with hazard map data and calculates the optimal evacuation route based on the departure and destination points specified by the user. A path-finding algorithm (e.g., Dijkstra algorithm, A algorithm) is used, and the calculated evacuation route information is stored in a database.

[0126] Route information provided by map applications

[0127] Device: The user launches a map application (e.g., Google (registered trademark) Maps, Apple Maps) on their smartphone or tablet and enters their current location and evacuation destination. The application sends this information to a server, from which it receives information on the optimal evacuation route. The received evacuation route information is displayed on the map application, allowing the user to check it and safely evacuate.

[0128] Specific examples

[0129] Disaster scenario

[0130] User B is staying in a flood-hit area and is searching for a suitable evacuation route. The specific steps are as follows:

[0131] 1. Server: Acquires real-time video from live cameras, performs pre-processing and analysis, and checks whether major roads are submerged from the acquired video data.

[0132] 2. Server: Obtains the latest hazard map data and identifies the extent of flood impact.

[0133] 3. Server: Integrates live camera footage and hazard map data, and calculates the optimal route from User B's current location to the nearest evacuation site using Dijkstra's algorithm.

[0134] 4. Device: User B launches the map application and inputs their current location and evacuation destination. The application sends a request to the server and receives information on the optimal evacuation route.

[0135] 5. Server: Returns the optimal route information calculated in real time to the user device.

[0136] 6. Device: The map application displays the received information, and User B visually checks the route and proceeds with the evacuation safely.

[0137] Examples of prompt statements

[0138] Below is an example of a prompt sentence to input to the generative AI model.

[0139] Prompt: Explain how a real-time evacuation assistance system works in the event of a disaster.

[0140] Collecting and analyzing live camera footage

[0141] Integrating and analyzing hazard map data obtained from government agencies

[0142] Calculate the optimal evacuation route based on your current location and evacuation destination

[0143] Providing route information to users through a map application

[0144] In this way, the present invention can quickly provide optimal and safe evacuation routes based on information collected in real time when a disaster occurs.

[0145] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0146] Step 1:

[0147] Collection of live camera footage

[0148] Server: Obtains real-time video data from live cameras. The input is the IP address of each live camera, and the output is raw video data. The server connects to the live cameras using the RTSP streaming protocol and stores the video data in a buffer at a specific frame rate.

[0149] Specific behavior:

[0150] 1. The server connects to the IP address of each live camera and starts the RTSP stream.

[0151] 2. Video data is stored in a buffer in real time.

[0152] Step 2:

[0153] Video data preprocessing and analysis

[0154] Server: Preprocesses the acquired video data and analyzes it using image recognition algorithms and machine learning models. The input is raw video data, and the output is analysis results indicating passability and the presence or absence of obstacles. Preprocessing involves noise removal and frame resizing.

[0155] Specific behavior:

[0156] 1. Noise is removed from the video data.

[0157] 2. Resize the frame to a size that is easy to analyze.

[0158] 3. Use a machine learning model (e.g., a TensorFlow model) to analyze each frame and identify obstacles and impassable areas.

[0159] Step 3:

[0160] Obtaining hazard map data

[0161] Server: Obtains hazard map data provided by public institutions. The input is the API endpoint, and the output is raw hazard map data. The server periodically issues API requests to obtain the latest data.

[0162] Specific behavior:

[0163] 1. The server issues an API request to obtain the latest hazard map data.

[0164] 2. The acquired data is stored in a GIS database.

[0165] Step 4:

[0166] Analysis of hazard map data

[0167] Server: Stores the acquired hazard map data in a GIS database and performs analysis to identify the extent of disaster impact and risk areas. The input is raw hazard map data, and the output is the analysis results.

[0168] Specific behavior:

[0169] 1. Hazard map data stored in a GIS database is input into the analytical model.

[0170] 2. Apply different analytical models to various disasters (e.g., flood, fire, earthquake) to identify risk areas.

[0171] Step 5:

[0172] Calculating the optimal evacuation route

[0173] Server: Integrates the analysis results of live camera footage and hazard map data, and calculates the optimal evacuation route based on the departure and destination points specified by the user. The input is the analysis results and departure and destination information, and the output is the optimal evacuation route.

[0174] Specific behavior:

[0175] 1. Based on composite data (camera image analysis results and hazard map data), information on the departure and destination points is combined.

[0176] 2. Calculate the optimal evacuation route using a pathfinding algorithm (e.g., Dijkstra's algorithm, A algorithm).

[0177] 3. Save the calculation results in the database.

[0178] Step 6:

[0179] Processing user requests

[0180] Device: The user launches a map application on their smartphone or tablet and inputs their current location and evacuation destination. The input is the user's current location and evacuation destination information, and the output is a request sent to the server.

[0181] Specific behavior:

[0182] 1. Provide an interface for inputting current location and evacuation destination from a map application.

[0183] 2. The entered data is sent to the server as an HTTP request.

[0184] Step 7:

[0185] Providing evacuation route information

[0186] Server: Receives user requests and returns optimal evacuation route information calculated in real time. The output is optimal evacuation route information.

[0187] Terminal: Displays the received evacuation route information in a map application. The input is the optimal evacuation route information sent from the server, and the output is a visual display for the user.

[0188] Specific behavior (server side):

[0189] 1. Receives a request and retrieves the optimal route information from the database.

[0190] 2. The acquired route information is returned to the user's device in JSON format.

[0191] Specific operations (terminal side):

[0192] 1. The route information returned from the server is displayed in the map application.

[0193] 2. The user proceeds with evacuation while referring to the map application.

[0194] (Application example 1)

[0195] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0196] Currently, many regions require systems to ensure safe evacuation in the event of a disaster. However, there are still limited systems that can respond to changing situations in real time and quickly provide optimal evacuation routes. This makes it difficult for autonomous vehicles to operate appropriately and safely, especially when transportation infrastructure is disrupted. Furthermore, there are no systems that can integrate information from different sources, such as live cameras and hazard maps, and provide optimal evacuation information in real time. For this reason, there is a need to develop a system that can reliably guide autonomous vehicles along safe routes and provide visual evacuation information to passengers in the event of a disaster.

[0197] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0198] In this invention, the server includes means for acquiring real-time images from live cameras and performing preprocessing and analysis, means for acquiring hazard map data and performing integration and analysis, means for calculating optimal evacuation routes based on the acquired and analyzed information, means for reflecting the calculated evacuation routes in the control system of the autonomous vehicle and updating the driving route in real time, and means for displaying evacuation route information on an in-vehicle display to provide it visually to the user. This makes it possible to provide optimal evacuation routes to autonomous vehicles in real time and achieve safe and efficient driving in the event of a disaster.

[0199] A "live camera" is a camera device that captures images in real time and transmits the data.

[0200] "Real-time video" refers to video data that shows the current situation in real time.

[0201] "Preprocessing" refers to the initial stage of processing performed on video data acquired by a live camera, and includes processes such as noise removal and image standardization.

[0202] "Analysis" is the process of extracting necessary information and making decisions based on preprocessed data.

[0203] "Hazard map data" is map information showing disasters and dangerous areas, and is data provided by government agencies and public institutions.

[0204] "Integration" is the process of bringing together multiple different data sources into one system.

[0205] An "optimal evacuation route" is the route that will allow you to reach your evacuation destination most safely and efficiently in the event of a disaster.

[0206] "Calculation" is the process of deriving a specific result using an algorithm based on given data.

[0207] An "autonomous vehicle" is a vehicle that has the ability to operate automatically without the need for a driver.

[0208] "Control system" means an electronic or mechanical system for managing and directing the operation of a vehicle.

[0209] The "travel route" is the route that a vehicle takes to reach its destination.

[0210] An "in-vehicle display" is a screen installed inside a vehicle to display information.

[0211] "User" means a person who uses the system or vehicle.

[0212] "Visually presented" means that the information is presented in a visual form.

[0213] "Real-time updates" refers to revising the system information as needed in response to changes in background data and conditions.

[0214] MODE FOR CARRYING OUT THE INVENTION

[0215] An example of this application is a system that enables autonomous vehicles to safely evacuate in the event of a disaster, integrating live cameras, hazard maps, and real-time calculation of optimal evacuation routes.

[0216] Overall structure

[0217] This system consists of multiple servers and user terminals. The servers have the means to collect and analyze video data from live cameras. They also acquire and analyze hazard map data to assess the impact of disasters. They also have the means to calculate optimal evacuation routes and provide the results to the autonomous vehicle. Meanwhile, the user terminals receive the evacuation route information and visually display it on an in-vehicle display.

[0218] Hardware and software used

[0219] Hardware:

[0220] Live cameras: Multiple cameras installed on a vehicle.

[0221] Server: A computing device used to collect, process, and store data.

[0222] In-vehicle display: A screen installed inside the vehicle to display information.

[0223] software:

[0224] OpenCV: Used to collect and preprocess camera footage.

[0225] Machine learning models (e.g. TensorFlow, PyTorch): Road condition analysis.

[0226] requests library: A library for sending HTTP requests and retrieving hazard map data.

[0227] Pathfinding algorithms (e.g., Dijkstra, A): Used to calculate optimal evacuation routes.

[0228] Program processing description

[0229] The server acquires real-time video footage from live cameras and performs preprocessing using OpenCV. The acquired video footage is analyzed using a machine learning model to determine whether roads are passable and whether there are any obstacles. Hazard map data is then acquired from government agencies and public institutions and integrated into a GIS database. Based on this data, the server calculates the optimal evacuation route. This calculation uses route search algorithms such as Dijkstra's algorithm and A algorithm. The calculated evacuation route information is reflected in the autonomous vehicle's control system, and the driving route is updated in real time.

[0230] The terminals then display the received evacuation route information on the in-car display, providing a visual representation to passengers, allowing users to evacuate safely while checking the information in real time.

[0231] Specific examples

[0232] For example, the process for updating route information when a disaster occurs is as follows:

[0233] 1. Collection and analysis of live camera footage

[0234] The server acquires real-time images from live cameras installed in vehicles, processes them, and analyzes them. For example, it uses OpenCV to remove noise and uses models using TensorFlow and PyTorch to determine whether a road is passable.

[0235] 2. Integration and analysis of hazard map data

[0236] The server uses the requests library to retrieve the latest hazard map data from government agencies and integrate it into a GIS database. This data is then processed with different analytical models (e.g., flood, fire).

[0237] 3. Calculating the optimal evacuation route

[0238] Based on the acquired and analyzed data, the server uses Dijkstra's algorithm and A algorithm to calculate the optimal route from the user's current location to the evacuation site.

[0239] 4. Real-time route updates

[0240] The evacuation route calculated by the server is sent to the autonomous vehicle's control system, which uses this information to update its route in real time.

[0241] 5. Providing visual information through in-car displays

[0242] The terminal displays the calculated evacuation route information on the in-vehicle display, allowing users to visually confirm it.

[0243] Prompt Sentence Examples

[0244] The following prompts are used to encourage a generative AI model to perform a specific task.

[0245] text

[0246] To provide safe evacuation routes in the event of a disaster, implement an optimal route calculation system that analyzes live camera footage and integrates it with hazard maps. Specifically, create an application that allows an autonomous vehicle to calculate the optimal evacuation route in real time based on its current location and destination, and display that information on an in-car display.

[0247] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0248] Step 1: Collecting live camera footage

[0249] The server acquires video data in real time from live cameras installed in the vehicle. The input is video data from multiple cameras, and the output is preprocessed video data. Specifically, it uses OpenCV to capture video data and performs preprocessing such as noise removal and frame alignment.

[0250] Step 2: Analyzing the video data

[0251] The server inputs the preprocessed video data into a machine learning model to analyze whether the road is passable and whether there are any obstacles. The input is the preprocessed video data, and the output is information on whether the road is passable or not. Specifically, the server uses a trained model using TensorFlow and PyTorch to determine the road conditions from the video data.

[0252] Step 3: Obtaining hazard map data

[0253] The server uses the requests library to obtain the latest hazard map data from government agencies and public institutions. The input is an HTTP request, and the output is hazard map data in JSON format. Specifically, it downloads the data from the specified URL and converts it into an analyzable format.

[0254] Step 4: Analyze hazard map data

[0255] The server analyzes the acquired hazard map data and identifies dangerous areas and areas of impact. The input is the hazard map data, and the output is the analyzed dangerous area information. Specifically, it applies analytical models corresponding to different types of disasters (e.g., floods, fires, earthquakes) and determines safe routes.

[0256] Step 5: Calculate the optimal evacuation route

[0257] The server integrates the video analysis results with hazard map data to calculate the optimal route from the user's current location to the evacuation destination. The input is road condition data and dangerous area information, and the output is the optimal evacuation route. Specifically, it uses Dijkstra's algorithm and A algorithm to derive the safest route.

[0258] Step 6: Real-time updates of routes

[0259] The server sends the calculated optimal evacuation route to the autonomous vehicle's control system and updates the driving route in real time. The input is the optimal evacuation route information, and the output is the updated driving route. Specifically, the server sends data to the vehicle's control system using a communication protocol.

[0260] Step 7: Display evacuation route information

[0261] The terminal displays the received evacuation route information on the in-car display, providing a visual presentation to passengers. The input is the optimal evacuation route information, and the output is the visual information on the display. Specifically, the information is displayed using a GUI (Graphical User Interface).

[0262] Step 8: Support users in evacuation

[0263] The user follows the route information displayed on the screen and safely travels along the designated route to the evacuation site. The input is the visually provided route information, and the output is the user's evacuation behavior. Specific actions include selecting a safe route based on the application's instructions and traveling by manual or automatic driving.

[0264] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0265] This invention is a system that supports safe evacuation in the event of a disaster, and provides an optimal evacuation route that takes into account the user's emotional state in addition to collecting and analyzing information in real time. The specific operation and program processing of this system are described below.

[0266] System Overview

[0267] This system consists of multiple servers and user terminals. The servers have the means to collect and analyze video data from live cameras. They also acquire hazard map data and assess the impact of disasters. They also have an emotion engine that recognizes the user's emotional state and calculates the optimal evacuation route. The user terminals receive evacuation route information using a map application and visually display it.

[0268] What the program does

[0269] The program processing of the system will be explained in natural language below.

[0270] Collection and analysis of live camera footage

[0271] Server: Acquires real-time video feeds from multiple live cameras installed in the surrounding area. These feeds are pre-processed and analyzed using image recognition algorithms and machine learning models. The analysis results include information on road availability and the presence of obstacles.

[0272] Hazard map data integration and analysis

[0273] Server: The latest hazard map data provided by government agencies and public institutions is obtained via API and FTP. The obtained data is integrated into a GIS database and subjected to analytical models to identify the extent of disaster impact and risk areas.

[0274] Emotion Engine Operation

[0275] Server: Collects voice data, camera footage, and input text data from the user's device and analyzes them with an emotion engine. The server uses techniques such as voice analysis, facial expression analysis, and text analysis to determine the user's emotional state.

[0276] Calculating the optimal evacuation route

[0277] Server: Integrates live camera analysis results, hazard map data, and the user's emotional state to calculate the optimal evacuation route. Utilizing Dijkstra's algorithm and A algorithm, it derives a safe and efficient route.

[0278] Emotion-based user interface adjustment

[0279] Server: Based on the analysis results of the emotion engine, the server adjusts the evacuation route guidance method. For example, if the user is feeling stressed, it provides detailed step-by-step instructions. Conversely, if the user is feeling less anxious, it provides simpler instructions.

[0280] Route information provided by map applications

[0281] Device: The user launches a map application on their smartphone or tablet and inputs their current location and evacuation destination. The application then sends a request to the server, which then receives information on the optimal evacuation route.

[0282] Server: Receives user requests, generates optimal evacuation route information based on the latest analysis data and route calculation results, and returns it to the terminal.

[0283] Device: The received evacuation route information is displayed on a map application, allowing the user to check the information and evacuate safely.

[0284] Specific examples

[0285] Below are some examples of specific scenarios that utilize this system.

[0286] Disaster scenario

[0287] User C is a traveler staying in a flood-hit area and is looking for a route to an appropriate evacuation site. User C feels anxious and needs detailed evacuation instructions.

[0288] 1. Collection and analysis of live camera footage

[0289] Server: Acquires and analyzes real-time video from multiple live cameras. From the acquired video data, it is possible to determine whether major roads are flooded.

[0290] 2. Integration and analysis of hazard map data

[0291] Server: Obtains the latest hazard map data and identifies the area affected by the flood. Based on this data, the server understands the status of the disaster affecting User C's location.

[0292] 3. Operation of the Emotion Engine

[0293] Server: Analyzes audio data and camera footage collected from User C's smartphone to assess his / her anxiety and stress levels. It turns out that User C is experiencing high levels of anxiety.

[0294] 4. Calculating the optimal evacuation route

[0295] Server: Calculates the optimal evacuation route by taking into account the live camera analysis results, hazard map data, and the emotional state of User C. Generates route information including detailed guidance steps.

[0296] 5. Route information provided by map applications

[0297] Device: User C launches a map application and inputs their current location and evacuation destination. The application sends a request to the server and receives information on the optimal evacuation route.

[0298] Server: Receives user C's request and returns the optimal route information calculated in real time.

[0299] Device: The map application displays the received information in detail and guides User C to evacuate without worry.

[0300] 6. Supporting users in evacuation

[0301] User C follows the instructions of the application and begins evacuation along the optimal route displayed. While on the move, he checks the latest analysis data to ensure his safety.

[0302] In this way, the system of the present invention can support safer and more secure evacuation behavior by taking into account the user's emotional state. Users can evacuate safely based on information updated in real time.

[0303] The processing flow will be explained below.

[0304] Step 1: Acquire live camera footage

[0305] Server: Accesses live cameras installed in the specified area and acquires video in real time. Collects data periodically using the endpoint URL and authentication information.

[0306] Step 2: Preprocessing the video data

[0307] Server: Preprocesses the captured live camera footage. Specifically, it resizes the footage, removes noise, corrects color, etc., and converts it into a state suitable for analysis.

[0308] Step 3: Analyzing the video data

[0309] Server: Analyzes the pre-processed video data using image recognition algorithms and machine learning models, extracting information such as road passability, obstacles, and flood damage.

[0310] Step 4: Save the analysis results

[0311] Server: Stores the analysis results in a database, including location information, traffic conditions, and details of detected obstacles.

[0312] Step 5: Obtaining hazard map data

[0313] Server: Obtains the latest hazard map data provided by government agencies and public institutions using API and FTP.

[0314] Step 6: Preprocessing and integration of hazard map data

[0315] Server: Performs preprocessing to integrate acquired hazard map data into the GIS database, including data format conversion, spatial correction, and filtering.

[0316] Step 7: Analyze hazard map data

[0317] Server: Analyzes the integrated hazard map data to identify the extent of disaster impacts and risk areas. Different analytical models are used for each disaster, such as floods, fires, and earthquakes.

[0318] Step 8: Obtaining Emotion Data

[0319] Device: The user provides audio data and camera footage via a smartphone or tablet, which then collects data on the user's emotional state.

[0320] Step 9: Analyze the sentiment data

[0321] Server: Analyzes collected voice, facial expression, and text data using an emotion engine to evaluate the user's emotional state (anxiety, stress, calm, etc.).

[0322] Step 10: Storing Emotion Data

[0323] Server: The analysis results are saved in a database. This data is used as auxiliary data for later evacuation route calculations.

[0324] Step 11: Calculate the optimal evacuation route

[0325] Server: Integrates live camera analysis results, hazard map data, and the user's emotional state to calculate the optimal evacuation route between the departure and destination points specified by the user. A safe and efficient route is derived using Dijkstra's algorithm and the A algorithm.

[0326] Step 12: Save the evacuation route data

[0327] Server: The calculated optimal evacuation route information is stored in a database so that subsequent client requests can be handled promptly.

[0328] Step 13: Accepting User Requests

[0329] Terminal: The user launches the map application and inputs their current location and evacuation destination. After inputting the information, a request for an evacuation route is sent to the server.

[0330] Step 14: Send evacuation route information

[0331] Server: Receives user requests, generates optimal evacuation route information based on the latest analysis data and route calculation results, and returns the generated information to the device.

[0332] Step 15: Display evacuation route information

[0333] Device: The received evacuation route information is displayed in a map application, showing the route and important points (obstacles, evacuation points, etc.) to make it easier for the user to understand visually.

[0334] Step 16: Start evacuation and check the situation

[0335] User: While looking at the map application, the user begins evacuation by following the optimal route displayed. The user also checks the latest analysis data while on the move to ensure safety.

[0336] Step 17: Guidance adjustment based on emotional state

[0337] Server: Based on the analysis results of the emotion engine, the server adjusts the evacuation route guidance method. For example, if the user is feeling stressed, it provides detailed step-by-step instructions to reassure the user.

[0338] Step 18: Notification of Evacuation Completion

[0339] User: Upon reaching the evacuation location, the application will notify the user that evacuation is complete. If necessary, the user will be able to check the recalculated route.

[0340] This series of processes allows users to obtain an appropriate evacuation route based on real-time updated information and emotional state, allowing them to take evacuation action safely and with peace of mind.

[0341] Example 2

[0342] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0343] In modern society, rapid and safe evacuation in the event of a disaster is a crucial issue. However, conventional evacuation support systems lack real-time information collection and analysis capabilities, making it difficult to provide evacuation support that is tailored to the user's individual emotional state. As a result, it is not possible to provide optimal evacuation routes in situations where users are feeling stressed or anxious, and issues remain regarding the safety and efficiency of evacuation behavior.

[0344] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0345] In this invention, the server includes means for acquiring real-time video from a live camera and performing preprocessing and analysis, means for acquiring hazard map data and performing integration and analysis, means for analyzing audio, image, and text data acquired from a user terminal and determining the user's emotional state, means for adjusting the evacuation route guidance method based on the user's emotional state, means for calculating an optimal evacuation route based on the acquired and analyzed information, and means for reflecting the calculated evacuation route in a map application and displaying it to the user. This makes it possible to provide an optimal evacuation route that takes into account the emotional state of each individual user based on information updated in real time.

[0346] A "live camera" is a camera device that captures video in real time and transmits it as digital data.

[0347] "Preprocessing" refers to processes such as noise removal and brightness adjustment that are performed to make the acquired video data easier to analyze.

[0348] "Analysis" is the process of analyzing acquired data using specific algorithms or models to extract useful information.

[0349] "Hazard map data" is map data that shows the risk of disasters occurring and the extent of their impact in a specific area.

[0350] "Integration" is the process of combining different types of data into a single dataset.

[0351] A "user terminal" is an electronic device such as a computer, smartphone, or tablet that is directly operated by a user.

[0352] "Voice data" refers to data in which the user's voice is recorded in digital format.

[0353] "Image data" refers to digital data of still or moving images captured by a camera or other photographic device.

[0354] "Text data" is data that stores characters or sentences in digital format.

[0355] "Emotional state" refers to the user's psychological state, and includes emotions such as anxiety, stress, and fear.

[0356] An "evacuation route" is a route that a user should follow to evacuate safely.

[0357] A "map application" is software that displays digital maps and provides navigation functions.

[0358] "Display" means visually presenting data or information to a user.

[0359] An "image recognition algorithm" is an algorithm for detecting specific patterns or features from input image data.

[0360] A "machine learning model" is a computational model that learns patterns from large amounts of data and performs predictions and classifications.

[0361] An "analytic model" is a set of rules or algorithms for performing analysis on a particular type of data.

[0362] "Evacuation route calculation" is the process of deriving a safe and efficient route based on acquired data.

[0363] "Adjustment based on the user's emotional state" refers to an operation of changing the information to be presented or the method of guidance in consideration of the user's psychological state.

[0364] The present invention is a system for supporting safe evacuation in the event of a disaster, collecting and analyzing information in real time and providing an optimal evacuation route taking into account the emotional state of the user. This system is composed of multiple servers and user terminals. Detailed embodiments of this system are described below.

[0365] Collection and analysis of live camera footage

[0366] Server: Images are acquired in real time from multiple live cameras installed in the surrounding area. The image data is first streamed to the server, where it is pre-processed using OpenCV. Noise removal and brightness adjustment are applied.

[0367] Server: The preprocessed video data is passed through image recognition algorithms such as TensorFlow and PyTorch to analyze whether roads are passable and whether there are any obstacles.

[0368] Examples:

[0369] The server acquires images from cameras in urban areas, preprocesses them using OpenCV, and then uses a TensorFlow model to analyze road congestion and flooding conditions.

[0370] Hazard map data integration and analysis

[0371] Server: The latest hazard map data provided by government agencies and public institutions is periodically obtained via API or FTP. The obtained data is stored in a GIS database such as PostGIS and updated in real time.

[0372] Server: This data is used to identify the extent of the disaster impact and risk areas and highlight them on a map.

[0373] Examples:

[0374] The server retrieves the latest flood hazard data from the Ministry of Land, Infrastructure, Transport and Tourism's API, imports it into a PostGIS database, and then identifies the flood-affected area and highlights it in red on the map.

[0375] Emotion Engine Operation

[0376] User: Enter voice messages, facial expressions, and text from a smartphone or tablet.

[0377] Terminal: Voice and facial expression data are collected on the user's terminal and sent to the server.

[0378] Server: Analyzes the received data using IBM Watson (registered trademark) or Google Cloud Natural Language API to identify the user's emotional state.

[0379] Examples:

[0380] The user speaks to their smartphone, saying, "I'm very anxious." The device records the voice data and sends it to the server, which then analyzes the voice data using IBM Watson to evaluate the user's anxiety level.

[0381] Calculating the optimal evacuation route

[0382] Server: Calculates evacuation routes by integrating live camera analysis results, hazard map data, and the user's emotional state. Uses Google Maps API to obtain local geographic information and Algorithm A to calculate optimal evacuation routes.

[0383] Server: Generates evacuation route information based on the calculation results, including detailed instructions according to the user's emotional state.

[0384] Examples:

[0385] The server applies algorithm A based on the user's current location to calculate the optimal evacuation route, for example, generating a route that avoids areas impassable due to flooding and includes detailed instructions such as "turn right at the next intersection, then go straight."

[0386] Providing an emotion-based user interface

[0387] Server: Considers the user's emotional state and customizes the evacuation route guidance method.

[0388] Server: Provides detailed step-by-step guides if the user is unsure, or generates simplified instructions if simple instructions are appropriate.

[0389] Examples:

[0390] The server provides detailed "audio directions" and "step-by-step visual directions" in a map application based on the user's emotional state.

[0391] Route information provided by map applications

[0392] User: When evacuation is necessary, the user launches the map application on their smartphone or tablet and enters their current location and evacuation destination.

[0393] Terminal: Sends user requests to the server and receives optimal evacuation route information.

[0394] Terminal: The received evacuation route information is visually displayed on a map application.

[0395] Examples:

[0396] The user opens a map application and inputs their current location and evacuation destination. The device sends a request to the server and receives information on the optimal evacuation route. The route is displayed on a map so the user can follow it safely.

[0397] Prompt Sentence Examples

[0398] Below are some example prompts to input to the generative AI model:

[0399] "When a user needs to evacuate due to heavy rain, the system sends a notification such as, 'Please enter your current location and evacuation destination.' At that time, the system calculates and displays the optimal evacuation route based on live camera footage and hazard map data. It also takes into account the user's emotional state and provides detailed step-by-step instructions if necessary."

[0400] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0401] Step 1:

[0402] Collection of live camera footage

[0403] Server: Acquires real-time images from multiple live cameras installed in the surrounding area.

[0404] Input: Real-time video sent from a live camera.

[0405] Output: Video data streamed to the server.

[0406] Specific operation: The server streams video from multiple live cameras in urban areas and acquires it as data.

[0407] Step 2:

[0408] Video data preprocessing

[0409] Server: Preprocesses the acquired video data. Uses OpenCV to remove noise and adjust brightness.

[0410] Input: The video data streamed in step 1.

[0411] Output: Pre-processed and clean video data.

[0412] Specific operation: The server processes the video data using OpenCV, performing noise removal and brightness adjustment.

[0413] Step 3:

[0414] Video data analysis

[0415] Server: The preprocessed video data is run through an image recognition algorithm (TensorFlow or PyTorch) to analyze whether the road is passable and whether there are any obstacles.

[0416] Input: Preprocessed video data.

[0417] Output: Analysis results include data on road traversability and the presence of obstacles.

[0418] Specific operation: The server uses TensorFlow to analyze the level of road congestion and the presence of obstacles from video data.

[0419] Step 4:

[0420] Hazard map data acquisition and integration

[0421] Server: The latest hazard map data is periodically obtained from government agencies and public institutions via API and FTP.

[0422] Input: Hazard map data provided by government agencies and public institutions.

[0423] Output: Hazard map data stored in a GIS database such as PostGIS.

[0424] Specific operation: The server retrieves the latest flood hazard data from the Ministry of Land, Infrastructure, Transport and Tourism's API and imports it into a PostGIS database.

[0425] Step 5:

[0426] Analysis of hazard map data

[0427] Server: Analyzes the acquired hazard map data and identifies the extent of the disaster's impact and dangerous areas.

[0428] Input: Hazard map data stored in a PostGIS database.

[0429] Output: Data on the extent of the disaster impact and risk areas.

[0430] Specific operation: The server analyzes hazard map data and highlights the affected areas and dangerous areas of floods and other disasters on the map.

[0431] Step 6:

[0432] Collecting Emotional Data

[0433] User: Enter voice messages, facial expressions, and text from a smartphone or tablet.

[0434] Input: Audio, image, and text input data.

[0435] Output: Emotion data recorded on a smartphone or tablet.

[0436] Specific operation: The user speaks to the smartphone, "I'm very anxious." The device records the voice data.

[0437] Step 7:

[0438] Sending and analyzing emotional data

[0439] Terminal: Sends emotion data to the server.

[0440] Server: Analyzes the received data using IBM Watson or Google Cloud Natural Language API to identify the user's emotional state.

[0441] Input: Emotion data sent from the user device.

[0442] Output: User's emotional state (e.g., anxiety, stress).

[0443] Specific operation: The device sends the recorded voice data to the server, which then analyzes it using IBM Watson.

[0444] Step 8:

[0445] Calculating the optimal evacuation route

[0446] Server: Calculates the optimal evacuation route by integrating live camera analysis results, hazard map data, and the user's emotional state. Uses the Google Maps API to obtain local geographic information and algorithm A.

[0447] Input: Analysis results (live camera data, hazard map data, emotional state).

[0448] Output: The calculated optimal evacuation route.

[0449] Specific operation: The server executes algorithm A based on the user's current location and calculates a safe and efficient evacuation route.

[0450] Step 9:

[0451] Coordination of evacuation route guidance

[0452] Server: Customize evacuation route guidance based on the user's emotional state. Provide detailed guidance if anxiety is high, and simple guidance if anxiety is low.

[0453] Input: The user's emotional state.

[0454] Output: Customized evacuation route guidance.

[0455] Specific behavior: If the server is concerned about the user, it generates a detailed step-by-step guide.

[0456] Step 10:

[0457] Providing evacuation route information

[0458] User: When evacuation is necessary, the user launches the map application on their smartphone or tablet and enters their current location and evacuation destination.

[0459] Terminal: Sends user input information to the server as a request and receives optimal evacuation route information.

[0460] Server: Generates optimal evacuation route information and sends it back to the terminal.

[0461] Input: User's current location and evacuation destination.

[0462] Output: Optimal evacuation route displayed on the user's terminal.

[0463] Specific operation: The user inputs the evacuation destination using a map application, and the server sends the calculation results to the device, which displays them on a map.

[0464] (Application example 2)

[0465] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0466] Safe evacuation during disasters requires real-time information collection and analysis, and complex data integration is required to provide appropriate evacuation routes. Psychological factors also need to be taken into account if evacuees are feeling stressed or anxious. Previous systems have not adequately considered emotional states when proposing evacuation routes, which has led to issues with ensuring the safety and security of evacuees.

[0467] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0468] In this invention, the server includes means for acquiring real-time images from a live camera and performing preprocessing and analysis, means for acquiring hazard map data and performing integration and analysis, means for calculating an optimal evacuation route based on the acquired and analyzed information, means for reflecting the calculated evacuation route in a map application and displaying it to the user, means for collecting the user's voice data and camera images and analyzing their emotional state, and means for adjusting the user interface according to their emotional state, thereby making it possible to provide an optimal evacuation route that takes the user's emotional state into consideration.

[0469] A "live camera" is a camera that captures images in real time and monitors the surrounding situation.

[0470] "Real-time video" is video that records the current situation instantly and is transmitted with almost no delay.

[0471] "Preprocessing" refers to the initial data processing step of organizing and converting collected raw data into a form that is easier to analyze.

[0472] "Hazard map data" is map data that shows dangerous areas due to natural disasters, accidents, etc.

[0473] "Integration" is the process of bringing together multiple pieces of data into one system and correlating them.

[0474] The "optimal evacuation route" is a route that avoids danger and allows you to reach your destination safely and quickly.

[0475] A "map application" is software that displays digital maps and provides location information and route guidance.

[0476] "Voice data" refers to data that records a user's voice and saves it in an analyzable format.

[0477] "Camera video" refers to video data captured by a camera.

[0478] "Emotional state" is an indicator of the user's psychological state, including anxiety and stress.

[0479] "Analysis" is a method for examining data and extracting information.

[0480] "Adjusting the user interface" means changing the operation screen and guidance method according to the user's situation and emotions.

[0481] This invention is a system that supports safe evacuation behavior in the event of a disaster. This system acquires and analyzes live camera images, integrates and analyzes the latest hazard map data, analyzes the user's emotional state, and calculates and presents the optimal evacuation route. Each processing step is described in detail below.

[0482] Collection and analysis of live camera footage

[0483] The server receives real-time video from multiple live cameras, and the video data is pre-processed and analyzed using image recognition algorithms and machine learning models to determine whether roads are passable and whether there are any obstacles.

[0484] Hazard map data integration and analysis

[0485] The server retrieves the latest hazard map data provided by government agencies and public institutions via API and FTP. The retrieved data is integrated into a GIS database and analyzed to identify the extent of disaster impact and risk areas.

[0486] Emotion Engine Operation

[0487] The server collects the user's voice data and camera footage and analyzes them with an emotion engine. Using techniques such as voice analysis, facial expression analysis, and text analysis, the server determines the user's emotional state, assessing whether the user is feeling stressed or anxious.

[0488] Calculating the optimal evacuation route

[0489] The server integrates live camera analysis results, hazard map data, and the user's emotional state to calculate the optimal evacuation route. Safe and efficient routes are derived using Dijkstra's algorithm and A algorithm. It is also possible to include detailed guidance steps according to the user's emotional state.

[0490] Emotion-based user interface adjustment

[0491] The server then adjusts the evacuation route guidance based on the emotion engine's analysis: if the user is feeling stressed, it provides more detailed, step-by-step instructions, and if the user is feeling less anxious, it provides simpler instructions.

[0492] Route information provided by map applications

[0493] Users launch a map application on their smartphone or tablet and input their current location and evacuation destination. The device then sends a request to the server and receives information on the optimal evacuation route. The server returns the optimal route information calculated in real time, and the device displays this information in detail on the map application. Users can check this information to evacuate safely.

[0494] Specific examples

[0495] In the event of a disaster, if a user is evacuating in their vehicle, the vehicle's infotainment system will use this system to provide the optimal evacuation route in real time. The guidance method will automatically adjust according to the user's emotional state, reducing the user's anxiety.

[0496] Prompt Sentence Examples

[0497] Visual Recognition Prompt: "Analyze real-time video from the camera to detect road conditions and obstacles."

[0498] Hazard Map Integration Prompt: "Get the latest hazard map data and identify the extent of the impact of a disaster."

[0499] Emotion Analysis Prompt: "Analyze the user's voice and facial expressions to determine their emotional state."

[0500] Route Optimization Prompt: "Calculate the optimal evacuation route, taking into account real-time data and emotional state."

[0501] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0502] Step 1:

[0503] The server acquires real-time video from the live camera. The acquired video data is first pre-processed to remove noise and adjust the resolution. This pre-processed data is then analyzed using image recognition algorithms and machine learning models. As a result of the analysis, it determines whether the road is passable and whether there are any obstacles, and outputs this as numerical data.

[0504] Step 2:

[0505] The server retrieves the latest hazard map data provided by government agencies and public institutions via API and FTP. The retrieved data is integrated into a GIS database and analyzed to identify the extent of disaster impact and risk areas. This generates and outputs coordinate data for risk areas.

[0506] Step 3:

[0507] The server collects voice data and camera footage from the user's smartphone and vehicle infotainment system. This data is then analyzed using an emotion engine to perform voice analysis and facial expression analysis. The analysis results are output as numerical data representing the user's stress and anxiety levels.

[0508] Step 4:

[0509] The server integrates the road condition data obtained in step 1, the hazard map data obtained in step 2, and the user's emotional state data obtained in step 3. It calculates the optimal evacuation route using Dijkstra's algorithm and the A algorithm. As a result of the calculation, it outputs a specific route and detailed guidance steps.

[0510] Step 5:

[0511] The server adjusts the evacuation route guidance method based on the analysis results of the emotion engine. For example, if the user is feeling stressed, it provides detailed step-by-step guidance. If the user is feeling less anxious, it provides simple guidance. This generates and outputs data for a customized guidance method.

[0512] Step 6:

[0513] The device sends a request to the server based on the user's input. The server returns optimal route information calculated in real time to the device. The device displays this received route information in a map application, providing a visual representation to the user. The user can then begin evacuation actions based on the displayed route information.

[0514] Step 7:

[0515] Users can follow the evacuation route displayed on the device's map application and check real-time updates to safely evacuate. The system continues to collect and analyze data while on the move, recalculating and updating evacuation routes as needed.

[0516] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0517] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0518] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0519] [Second embodiment]

[0520] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0521] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0522] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0523] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0524] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0525] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0526] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0527] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0528] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0529] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0530] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0531] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0532] This invention is a system that supports safe evacuation in the event of a disaster, integrating live cameras, hazard maps, and map applications to provide optimal evacuation routes in real time. The specific operation and program processing of this system are described below.

[0533] System Overview

[0534] This system consists of multiple servers and user terminals. The servers have the means to collect and analyze video data from live cameras. They also acquire and analyze hazard map data to assess the impact of disasters. They also have the means to calculate optimal evacuation routes and provide the results to users. Meanwhile, the user terminals use a map application to receive and visually display evacuation route information.

[0535] What the program does

[0536] The program processing of the system will be explained in natural language below.

[0537] Collection and analysis of live camera footage

[0538] Server: Real-time video is acquired from multiple live cameras installed in the surrounding area. This video is first pre-processed and then analyzed using image recognition algorithms and machine learning models to determine whether roads are passable and whether there are any obstacles.

[0539] Hazard map data integration and analysis

[0540] Server: Acquires hazard map data provided by government agencies and public institutions. This data is integrated into a GIS database and subjected to analytical models to identify the extent of disaster impact and risk areas. Different analytical models are applied for different types of disasters (e.g., floods, fires, earthquakes, etc.).

[0541] Calculating the optimal evacuation route

[0542] Server: Integrates the analysis results of live camera footage with hazard map data to calculate the optimal evacuation route based on the departure and destination points specified by the user. Path-finding algorithms such as Dijkstra's algorithm and A algorithm are used. The calculated evacuation route information is immediately saved in a database.

[0543] Route information provided by map applications

[0544] Device: The user launches a map application on their smartphone or tablet and inputs their current location and evacuation destination. The application sends this information to the server, from which it receives information on the optimal evacuation route.

[0545] Server: Receives user requests and returns optimal evacuation route information in real time.

[0546] Device: The received evacuation route information is displayed on a map application, allowing the user to check the information and evacuate safely.

[0547] Specific examples

[0548] Below are some examples of specific scenarios that use this system.

[0549] Disaster scenario

[0550] User B is a traveler staying in a flood-hit area and is looking for a route to a suitable evacuation site.

[0551] 1. Collection and analysis of live camera footage

[0552] Server: Acquires and analyzes real-time video from multiple live cameras. From the acquired video data, it is possible to determine whether major roads are flooded.

[0553] 2. Integration and analysis of hazard map data

[0554] Server: Obtains the latest hazard map data and identifies the area affected by the flood. Based on this data, the server understands the status of the disaster affecting User B's location.

[0555] 3. Calculating the optimal evacuation route

[0556] Server: Integrates the analysis results of live camera footage with hazard map data to calculate the optimal route from User B's current location to the nearest evacuation site. Using Dijkstra's algorithm, the safest and most efficient route is derived and the information is stored in a database.

[0557] 4. Route information provided by map applications

[0558] Device: User B launches a map application and inputs their current location and evacuation destination. The application sends a request to the server and receives information on the optimal evacuation route.

[0559] Server: Receives User B's request and returns the optimal route information calculated in real time.

[0560] Device: The map application displays the received information, and User B can visually check the route.

[0561] 5. Supporting users in evacuation

[0562] User: User B follows the instructions of the application and safely travels to the evacuation site via the designated route.

[0563] In this way, the present invention allows users to obtain appropriate evacuation routes based on real-time information when a disaster occurs, enabling safe and rapid evacuation.

[0564] The processing flow will be explained below.

[0565] Step 1: Acquire live camera footage

[0566] Server: Accesses live cameras installed in the specified area and acquires video in real time. Collects data periodically using the endpoint URL and authentication information.

[0567] Step 2: Preprocessing the video data

[0568] Server: Preprocesses the captured live camera footage. Specifically, it resizes the footage, removes noise, corrects color, etc., and converts it into a state suitable for analysis.

[0569] Step 3: Analyzing the video data

[0570] Server: Analyzes the pre-processed video data using image recognition algorithms and machine learning models, extracting information such as road passability, obstacles, and flood damage.

[0571] Step 4: Save the analysis results

[0572] Server: Stores the analysis results in a database, including location information, traffic conditions, and details of detected obstacles.

[0573] Step 5: Obtaining hazard map data

[0574] Server: Obtains the latest hazard map data provided by government agencies and public institutions using API and FTP.

[0575] Step 6: Preprocessing and integration of hazard map data

[0576] Server: Performs preprocessing to integrate acquired hazard map data into the GIS database, including data format conversion, spatial correction, and filtering.

[0577] Step 7: Analyze hazard map data

[0578] Server: Analyzes the integrated hazard map data to identify the extent of disaster impacts and risk areas. Different analytical models are used for each disaster, such as floods, fires, and earthquakes.

[0579] Step 8: Calculate the optimal evacuation route

[0580] Server: Based on the results of live camera analysis and hazard map data, the server calculates the optimal evacuation route for the departure and destination points specified by the user. It uses Dijkstra's algorithm and A algorithm to derive a safe and efficient route.

[0581] Step 9: Save the evacuation route data

[0582] Server: The calculated optimal evacuation route information is stored in a database so that subsequent client requests can be handled promptly.

[0583] Step 10: Accepting a user request

[0584] Device: The user launches a map application and inputs their current location and evacuation destination. This information is sent as a request to the server.

[0585] Step 11: Send evacuation route information

[0586] Server: Receives user requests, generates optimal evacuation route information based on the latest analysis data and route calculation results, and returns the generated information to the device.

[0587] Step 12: Display evacuation route information

[0588] Device: The received evacuation route information is displayed in a map application, showing the route and important points (obstacles, evacuation points, etc.) to make it easier for the user to understand visually.

[0589] Step 13: Start evacuation and check the situation

[0590] User: While looking at the map application, the user begins evacuation by following the optimal route displayed. While on the move, the user can check the latest analysis data through the application to ensure safety.

[0591] Step 14: Notification of evacuation completion

[0592] User: Upon reaching the evacuation location, the application will notify the user that evacuation is complete. If necessary, the user will be able to check the recalculated route.

[0593] This series of processes allows users to evacuate quickly and safely based on information updated in real time.

[0594] Example 1

[0595] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0596] When a disaster occurs, it is important to provide safe and effective evacuation routes in real time. However, conventional evacuation support systems have difficulty collecting real-time information and calculating optimal evacuation routes based on that information. Furthermore, they do not perform sufficient analysis according to different disaster types, making it impossible to quickly provide optimal evacuation routes for users. This leads to evacuation delays and the selection of inappropriate evacuation routes, which can compromise user safety.

[0597] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0598] In this invention, the server includes means for acquiring real-time video from a live camera and analyzing it using preprocessing and image recognition algorithms and machine learning models, means for acquiring hazard map data from public institutions, integrating it into a GIS database, and analyzing it using different analysis models depending on the disaster, means for calculating the optimal evacuation route from the departure point and destination using a route search algorithm based on the acquired and analyzed video data and hazard map data, and means for saving the calculated evacuation route in the database and reflecting it in a map application and displaying it to the user upon user request. This makes it possible to quickly provide the optimal and safest evacuation route based on information collected in real time when a disaster occurs.

[0599] A "live camera" is an image capturing device installed to capture real-time images of the surrounding area.

[0600] "Preprocessing" refers to the process of performing processes such as noise removal and frame resizing to make the acquired video data easier to analyze.

[0601] An "image recognition algorithm" is a computational method for detecting specific objects or patterns within video data.

[0602] A "machine learning model" is a collection of algorithms that automatically learn from data and make predictions and classifications.

[0603] "Hazard map data" is geographic information that indicates the risk and impact of disasters.

[0604] "Public institutions" are public data providers such as government agencies and local governments.

[0605] A "GIS database" is a database system that manages geographic information in an integrated manner.

[0606] A "route search algorithm" is a computational method for finding the optimal route between a specified starting point and destination.

[0607] A "database" is a system for systematically storing information and efficiently retrieving it when needed.

[0608] A "user terminal" is an electronic device, such as a smartphone or tablet, that runs a map application.

[0609] A "map application" is software that displays a user's current location, destination, and route information.

[0610] This invention is a system for supporting safe evacuation in the event of a disaster, integrating live cameras, hazard maps, and map applications to provide optimal evacuation routes in real time. This system is composed of multiple servers and user terminals, and a specific embodiment is shown below.

[0611] System Overview

[0612] The system mainly uses a server, live cameras, user devices, map applications, and hazard map data. Below, we will explain the details of each component and how they work.

[0613] Collection and analysis of live camera footage

[0614] Server: The server acquires real-time video from multiple live cameras installed in the surrounding area. The video is transferred to the server at a fixed frame rate and received using a streaming protocol (e.g., RTSP). The video data is pre-processed and analyzed using a machine learning model (e.g., TensorFlow model).

[0615] Specifically, the server connects to each live camera and captures video data in real time. This data undergoes preprocessing such as noise removal and frame resizing, and is then analyzed using image recognition algorithms and machine learning models. This allows the system to determine whether the road is passable and whether there are any obstacles.

[0616] Acquisition and analysis of hazard map data

[0617] Server: The server periodically retrieves hazard map data provided by government agencies and public institutions. This data is retrieved through APIs and stored in a GIS database. The retrieved data is analyzed using analytical models (e.g., ArcGIS) to identify the extent of disaster impact and risk areas. In this case, analytical models appropriate for different disasters, such as floods, fires, and earthquakes, are applied.

[0618] Calculating the optimal evacuation route

[0619] Server: The server integrates the analysis results of live camera footage with hazard map data and calculates the optimal evacuation route based on the departure and destination points specified by the user. A path-finding algorithm (e.g., Dijkstra algorithm, A algorithm) is used, and the calculated evacuation route information is stored in a database.

[0620] Route information provided by map applications

[0621] Device: The user launches a map application (e.g., Google Maps, Apple Maps) on their smartphone or tablet and enters their current location and evacuation destination. The application sends this information to the server, which then receives information on the optimal evacuation route. The received evacuation route information is displayed on the map application, allowing the user to check it and safely evacuate.

[0622] Specific examples

[0623] Disaster scenario

[0624] User B is staying in a flood-hit area and is searching for a suitable evacuation route. The specific steps are as follows:

[0625] 1. Server: Acquires real-time video from live cameras, performs pre-processing and analysis, and checks whether major roads are submerged from the acquired video data.

[0626] 2. Server: Obtains the latest hazard map data and identifies the extent of flood impact.

[0627] 3. Server: Integrates live camera footage and hazard map data, and calculates the optimal route from User B's current location to the nearest evacuation site using Dijkstra's algorithm.

[0628] 4. Device: User B launches the map application and inputs their current location and evacuation destination. The application sends a request to the server and receives information on the optimal evacuation route.

[0629] 5. Server: Returns the optimal route information calculated in real time to the user device.

[0630] 6. Device: The map application displays the received information, and User B visually checks the route and proceeds with the evacuation safely.

[0631] Examples of prompt statements

[0632] Below is an example of a prompt sentence to input to the generative AI model.

[0633] Prompt: Explain how a real-time evacuation assistance system works in the event of a disaster.

[0634] Collecting and analyzing live camera footage

[0635] Integrating and analyzing hazard map data obtained from government agencies

[0636] Calculate the optimal evacuation route based on your current location and evacuation destination

[0637] Providing route information to users through a map application

[0638] In this way, the present invention can quickly provide optimal and safe evacuation routes based on information collected in real time when a disaster occurs.

[0639] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0640] Step 1:

[0641] Collection of live camera footage

[0642] Server: Obtains real-time video data from live cameras. The input is the IP address of each live camera, and the output is raw video data. The server connects to the live cameras using the RTSP streaming protocol and stores the video data in a buffer at a specific frame rate.

[0643] Specific behavior:

[0644] 1. The server connects to the IP address of each live camera and starts the RTSP stream.

[0645] 2. Video data is stored in a buffer in real time.

[0646] Step 2:

[0647] Video data preprocessing and analysis

[0648] Server: Preprocesses the acquired video data and analyzes it using image recognition algorithms and machine learning models. The input is raw video data, and the output is analysis results indicating passability and the presence or absence of obstacles. Preprocessing involves noise removal and frame resizing.

[0649] Specific behavior:

[0650] 1. Noise is removed from the video data.

[0651] 2. Resize the frame to a size that is easy to analyze.

[0652] 3. Use a machine learning model (e.g., a TensorFlow model) to analyze each frame and identify obstacles and impassable areas.

[0653] Step 3:

[0654] Obtaining hazard map data

[0655] Server: Obtains hazard map data provided by public institutions. The input is the API endpoint, and the output is raw hazard map data. The server periodically issues API requests to obtain the latest data.

[0656] Specific behavior:

[0657] 1. The server issues an API request to obtain the latest hazard map data.

[0658] 2. The acquired data is stored in a GIS database.

[0659] Step 4:

[0660] Analysis of hazard map data

[0661] Server: Stores the acquired hazard map data in a GIS database and performs analysis to identify the extent of disaster impact and risk areas. The input is raw hazard map data, and the output is the analysis results.

[0662] Specific behavior:

[0663] 1. Hazard map data stored in a GIS database is input into the analytical model.

[0664] 2. Apply different analytical models to various disasters (e.g., flood, fire, earthquake) to identify risk areas.

[0665] Step 5:

[0666] Calculating the optimal evacuation route

[0667] Server: Integrates the analysis results of live camera footage and hazard map data, and calculates the optimal evacuation route based on the departure and destination points specified by the user. The input is the analysis results and departure and destination information, and the output is the optimal evacuation route.

[0668] Specific behavior:

[0669] 1. Based on composite data (camera image analysis results and hazard map data), information on the departure and destination points is combined.

[0670] 2. Calculate the optimal evacuation route using a pathfinding algorithm (e.g., Dijkstra's algorithm, A algorithm).

[0671] 3. Save the calculation results in the database.

[0672] Step 6:

[0673] Processing user requests

[0674] Device: The user launches a map application on their smartphone or tablet and inputs their current location and evacuation destination. The input is the user's current location and evacuation destination information, and the output is a request sent to the server.

[0675] Specific behavior:

[0676] 1. Provide an interface for inputting current location and evacuation destination from a map application.

[0677] 2. The entered data is sent to the server as an HTTP request.

[0678] Step 7:

[0679] Providing evacuation route information

[0680] Server: Receives user requests and returns optimal evacuation route information calculated in real time. The output is optimal evacuation route information.

[0681] Terminal: Displays the received evacuation route information in a map application. The input is the optimal evacuation route information sent from the server, and the output is a visual display for the user.

[0682] Specific behavior (server side):

[0683] 1. Receives a request and retrieves the optimal route information from the database.

[0684] 2. The acquired route information is returned to the user's device in JSON format.

[0685] Specific operations (terminal side):

[0686] 1. The route information returned from the server is displayed in the map application.

[0687] 2. The user proceeds with evacuation while referring to the map application.

[0688] (Application example 1)

[0689] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0690] Currently, many regions require systems to ensure safe evacuation in the event of a disaster. However, there are still limited systems that can respond to changing situations in real time and quickly provide optimal evacuation routes. This makes it difficult for autonomous vehicles to operate appropriately and safely, especially when transportation infrastructure is disrupted. Furthermore, there are no systems that can integrate information from different sources, such as live cameras and hazard maps, and provide optimal evacuation information in real time. For this reason, there is a need to develop a system that can reliably guide autonomous vehicles along safe routes and provide visual evacuation information to passengers in the event of a disaster.

[0691] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0692] In this invention, the server includes means for acquiring real-time images from live cameras and performing preprocessing and analysis, means for acquiring hazard map data and performing integration and analysis, means for calculating optimal evacuation routes based on the acquired and analyzed information, means for reflecting the calculated evacuation routes in the control system of the autonomous vehicle and updating the driving route in real time, and means for displaying evacuation route information on an in-vehicle display to provide it visually to the user. This makes it possible to provide optimal evacuation routes to autonomous vehicles in real time and achieve safe and efficient driving in the event of a disaster.

[0693] A "live camera" is a camera device that captures images in real time and transmits the data.

[0694] "Real-time video" refers to video data that shows the current situation in real time.

[0695] "Preprocessing" refers to the initial stage of processing performed on video data acquired by a live camera, and includes processes such as noise removal and image standardization.

[0696] "Analysis" is the process of extracting necessary information and making decisions based on preprocessed data.

[0697] "Hazard map data" is map information showing disasters and dangerous areas, and is data provided by government agencies and public institutions.

[0698] "Integration" is the process of bringing together multiple different data sources into one system.

[0699] An "optimal evacuation route" is the route that will allow you to reach your evacuation destination most safely and efficiently in the event of a disaster.

[0700] "Calculation" is the process of deriving a specific result using an algorithm based on given data.

[0701] An "autonomous vehicle" is a vehicle that has the ability to operate automatically without the need for a driver.

[0702] "Control system" means an electronic or mechanical system for managing and directing the operation of a vehicle.

[0703] The "travel route" is the route that a vehicle takes to reach its destination.

[0704] An "in-vehicle display" is a screen installed inside a vehicle to display information.

[0705] "User" means a person who uses the system or vehicle.

[0706] "Visually presented" means that the information is presented in a visual form.

[0707] "Real-time updates" refers to revising the system information as needed in response to changes in background data and conditions.

[0708] MODE FOR CARRYING OUT THE INVENTION

[0709] An example of this application is a system that enables autonomous vehicles to safely evacuate in the event of a disaster, integrating live cameras, hazard maps, and real-time calculation of optimal evacuation routes.

[0710] Overall structure

[0711] This system consists of multiple servers and user terminals. The servers have the means to collect and analyze video data from live cameras. They also acquire and analyze hazard map data to assess the impact of disasters. They also have the means to calculate optimal evacuation routes and provide the results to the autonomous vehicle. Meanwhile, the user terminals receive the evacuation route information and visually display it on an in-vehicle display.

[0712] Hardware and software used

[0713] Hardware:

[0714] Live cameras: Multiple cameras installed on a vehicle.

[0715] Server: A computing device used to collect, process, and store data.

[0716] In-vehicle display: A screen installed inside the vehicle to display information.

[0717] software:

[0718] OpenCV: Used to collect and preprocess camera footage.

[0719] Machine learning models (e.g. TensorFlow, PyTorch): Road condition analysis.

[0720] requests library: A library for sending HTTP requests and retrieving hazard map data.

[0721] Pathfinding algorithms (e.g., Dijkstra, A): Used to calculate optimal evacuation routes.

[0722] Program processing description

[0723] The server acquires real-time video footage from live cameras and performs preprocessing using OpenCV. The acquired video footage is analyzed using a machine learning model to determine whether roads are passable and whether there are any obstacles. Hazard map data is then acquired from government agencies and public institutions and integrated into a GIS database. Based on this data, the server calculates the optimal evacuation route. This calculation uses route search algorithms such as Dijkstra's algorithm and A algorithm. The calculated evacuation route information is reflected in the autonomous vehicle's control system, and the driving route is updated in real time.

[0724] The terminals then display the received evacuation route information on the in-car display, providing a visual representation to passengers, allowing users to evacuate safely while checking the information in real time.

[0725] Specific examples

[0726] For example, the process for updating route information when a disaster occurs is as follows:

[0727] 1. Collection and analysis of live camera footage

[0728] The server acquires real-time images from live cameras installed in vehicles, processes them, and analyzes them. For example, it uses OpenCV to remove noise and uses models using TensorFlow and PyTorch to determine whether a road is passable.

[0729] 2. Integration and analysis of hazard map data

[0730] The server uses the requests library to retrieve the latest hazard map data from government agencies and integrate it into a GIS database. This data is then processed with different analytical models (e.g., flood, fire).

[0731] 3. Calculating the optimal evacuation route

[0732] Based on the acquired and analyzed data, the server uses Dijkstra's algorithm and A algorithm to calculate the optimal route from the user's current location to the evacuation site.

[0733] 4. Real-time route updates

[0734] The evacuation route calculated by the server is sent to the autonomous vehicle's control system, which uses this information to update its route in real time.

[0735] 5. Providing visual information through in-car displays

[0736] The terminal displays the calculated evacuation route information on the in-vehicle display, allowing users to visually confirm it.

[0737] Prompt Sentence Examples

[0738] The following prompts are used to encourage a generative AI model to perform a specific task.

[0739] text

[0740] To provide safe evacuation routes in the event of a disaster, implement an optimal route calculation system that analyzes live camera footage and integrates it with hazard maps. Specifically, create an application that allows an autonomous vehicle to calculate the optimal evacuation route in real time based on its current location and destination, and display that information on an in-car display.

[0741] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0742] Step 1: Collecting live camera footage

[0743] The server acquires video data in real time from live cameras installed in the vehicle. The input is video data from multiple cameras, and the output is preprocessed video data. Specifically, it uses OpenCV to capture video data and performs preprocessing such as noise removal and frame alignment.

[0744] Step 2: Analyzing the video data

[0745] The server inputs the preprocessed video data into a machine learning model to analyze whether the road is passable and whether there are any obstacles. The input is the preprocessed video data, and the output is information on whether the road is passable or not. Specifically, the server uses a trained model using TensorFlow and PyTorch to determine the road conditions from the video data.

[0746] Step 3: Obtaining hazard map data

[0747] The server uses the requests library to obtain the latest hazard map data from government agencies and public institutions. The input is an HTTP request, and the output is hazard map data in JSON format. Specifically, it downloads the data from the specified URL and converts it into an analyzable format.

[0748] Step 4: Analyze hazard map data

[0749] The server analyzes the acquired hazard map data and identifies dangerous areas and areas of impact. The input is the hazard map data, and the output is the analyzed dangerous area information. Specifically, it applies analytical models corresponding to different types of disasters (e.g., floods, fires, earthquakes) and determines safe routes.

[0750] Step 5: Calculate the optimal evacuation route

[0751] The server integrates the video analysis results with hazard map data to calculate the optimal route from the user's current location to the evacuation destination. The input is road condition data and dangerous area information, and the output is the optimal evacuation route. Specifically, it uses Dijkstra's algorithm and A algorithm to derive the safest route.

[0752] Step 6: Real-time updates of routes

[0753] The server sends the calculated optimal evacuation route to the autonomous vehicle's control system and updates the driving route in real time. The input is the optimal evacuation route information, and the output is the updated driving route. Specifically, the server sends data to the vehicle's control system using a communication protocol.

[0754] Step 7: Display evacuation route information

[0755] The terminal displays the received evacuation route information on the in-car display, providing a visual presentation to passengers. The input is the optimal evacuation route information, and the output is the visual information on the display. Specifically, the information is displayed using a GUI (Graphical User Interface).

[0756] Step 8: Support users in evacuation

[0757] The user follows the route information displayed on the screen and safely travels along the designated route to the evacuation site. The input is the visually provided route information, and the output is the user's evacuation behavior. Specific actions include selecting a safe route based on the application's instructions and traveling by manual or automatic driving.

[0758] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0759] This invention is a system that supports safe evacuation in the event of a disaster, and provides an optimal evacuation route that takes into account the user's emotional state in addition to collecting and analyzing information in real time. The specific operation and program processing of this system are described below.

[0760] System Overview

[0761] This system consists of multiple servers and user terminals. The servers have the means to collect and analyze video data from live cameras. They also acquire hazard map data and assess the impact of disasters. They also have an emotion engine that recognizes the user's emotional state and calculates the optimal evacuation route. The user terminals receive evacuation route information using a map application and visually display it.

[0762] What the program does

[0763] The program processing of the system will be explained in natural language below.

[0764] Collection and analysis of live camera footage

[0765] Server: Acquires real-time video feeds from multiple live cameras installed in the surrounding area. These feeds are pre-processed and analyzed using image recognition algorithms and machine learning models. The analysis results include information on road availability and the presence of obstacles.

[0766] Hazard map data integration and analysis

[0767] Server: The latest hazard map data provided by government agencies and public institutions is obtained via API and FTP. The obtained data is integrated into a GIS database and subjected to analytical models to identify the extent of disaster impact and risk areas.

[0768] Emotion Engine Operation

[0769] Server: Collects voice data, camera footage, and input text data from the user's device and analyzes them with an emotion engine. The server uses techniques such as voice analysis, facial expression analysis, and text analysis to determine the user's emotional state.

[0770] Calculating the optimal evacuation route

[0771] Server: Integrates live camera analysis results, hazard map data, and the user's emotional state to calculate the optimal evacuation route. Utilizing Dijkstra's algorithm and A algorithm, it derives a safe and efficient route.

[0772] Emotion-based user interface adjustment

[0773] Server: Based on the analysis results of the emotion engine, the server adjusts the evacuation route guidance method. For example, if the user is feeling stressed, it provides detailed step-by-step instructions. Conversely, if the user is feeling less anxious, it provides simpler instructions.

[0774] Route information provided by map applications

[0775] Device: The user launches a map application on their smartphone or tablet and inputs their current location and evacuation destination. The application then sends a request to the server, which then receives information on the optimal evacuation route.

[0776] Server: Receives user requests, generates optimal evacuation route information based on the latest analysis data and route calculation results, and returns it to the terminal.

[0777] Device: The received evacuation route information is displayed on a map application, allowing the user to check the information and evacuate safely.

[0778] Specific examples

[0779] Below are some examples of specific scenarios that utilize this system.

[0780] Disaster scenario

[0781] User C is a traveler staying in a flood-hit area and is looking for a route to an appropriate evacuation site. User C feels anxious and needs detailed evacuation instructions.

[0782] 1. Collection and analysis of live camera footage

[0783] Server: Acquires and analyzes real-time video from multiple live cameras. From the acquired video data, it is possible to determine whether major roads are flooded.

[0784] 2. Integration and analysis of hazard map data

[0785] Server: Obtains the latest hazard map data and identifies the area affected by the flood. Based on this data, the server understands the status of the disaster affecting User C's location.

[0786] 3. Operation of the Emotion Engine

[0787] Server: Analyzes audio data and camera footage collected from User C's smartphone to assess his / her anxiety and stress levels. It turns out that User C is experiencing high levels of anxiety.

[0788] 4. Calculating the optimal evacuation route

[0789] Server: Calculates the optimal evacuation route by taking into account the live camera analysis results, hazard map data, and the emotional state of User C. Generates route information including detailed guidance steps.

[0790] 5. Route information provided by map applications

[0791] Device: User C launches a map application and inputs their current location and evacuation destination. The application sends a request to the server and receives information on the optimal evacuation route.

[0792] Server: Receives user C's request and returns the optimal route information calculated in real time.

[0793] Device: The map application displays the received information in detail and guides User C to evacuate without worry.

[0794] 6. Supporting users in evacuation

[0795] User C follows the instructions of the application and begins evacuation along the optimal route displayed. While on the move, he checks the latest analysis data to ensure his safety.

[0796] In this way, the system of the present invention can support safer and more secure evacuation behavior by taking into account the user's emotional state. Users can evacuate safely based on information updated in real time.

[0797] The processing flow will be explained below.

[0798] Step 1: Acquire live camera footage

[0799] Server: Accesses live cameras installed in the specified area and acquires video in real time. Collects data periodically using the endpoint URL and authentication information.

[0800] Step 2: Preprocessing the video data

[0801] Server: Preprocesses the captured live camera footage. Specifically, it resizes the footage, removes noise, corrects color, etc., and converts it into a state suitable for analysis.

[0802] Step 3: Analyzing the video data

[0803] Server: Analyzes the pre-processed video data using image recognition algorithms and machine learning models, extracting information such as road passability, obstacles, and flood damage.

[0804] Step 4: Save the analysis results

[0805] Server: Stores the analysis results in a database, including location information, traffic conditions, and details of detected obstacles.

[0806] Step 5: Obtaining hazard map data

[0807] Server: Obtains the latest hazard map data provided by government agencies and public institutions using API and FTP.

[0808] Step 6: Preprocessing and integration of hazard map data

[0809] Server: Performs preprocessing to integrate acquired hazard map data into the GIS database, including data format conversion, spatial correction, and filtering.

[0810] Step 7: Analyze hazard map data

[0811] Server: Analyzes the integrated hazard map data to identify the extent of disaster impacts and risk areas. Different analytical models are used for each disaster, such as floods, fires, and earthquakes.

[0812] Step 8: Obtaining Emotion Data

[0813] Device: The user provides audio data and camera footage via a smartphone or tablet, which then collects data on the user's emotional state.

[0814] Step 9: Analyze the sentiment data

[0815] Server: Analyzes collected voice, facial expression, and text data using an emotion engine to evaluate the user's emotional state (anxiety, stress, calm, etc.).

[0816] Step 10: Storing Emotion Data

[0817] Server: The analysis results are saved in a database. This data is used as auxiliary data for later evacuation route calculations.

[0818] Step 11: Calculate the optimal evacuation route

[0819] Server: Integrates live camera analysis results, hazard map data, and the user's emotional state to calculate the optimal evacuation route between the departure and destination points specified by the user. A safe and efficient route is derived using Dijkstra's algorithm and the A algorithm.

[0820] Step 12: Save the evacuation route data

[0821] Server: The calculated optimal evacuation route information is stored in a database so that subsequent client requests can be handled promptly.

[0822] Step 13: Accepting User Requests

[0823] Terminal: The user launches the map application and inputs their current location and evacuation destination. After inputting the information, a request for an evacuation route is sent to the server.

[0824] Step 14: Send evacuation route information

[0825] Server: Receives user requests, generates optimal evacuation route information based on the latest analysis data and route calculation results, and returns the generated information to the device.

[0826] Step 15: Display evacuation route information

[0827] Device: The received evacuation route information is displayed in a map application, showing the route and important points (obstacles, evacuation points, etc.) to make it easier for the user to understand visually.

[0828] Step 16: Start evacuation and check the situation

[0829] User: While looking at the map application, the user begins evacuation by following the optimal route displayed. The user also checks the latest analysis data while on the move to ensure safety.

[0830] Step 17: Guidance adjustment based on emotional state

[0831] Server: Based on the analysis results of the emotion engine, the server adjusts the evacuation route guidance method. For example, if the user is feeling stressed, it provides detailed step-by-step instructions to reassure the user.

[0832] Step 18: Notification of Evacuation Completion

[0833] User: Upon reaching the evacuation location, the application will notify the user that evacuation is complete. If necessary, the user will be able to check the recalculated route.

[0834] This series of processes allows users to obtain an appropriate evacuation route based on real-time updated information and emotional state, allowing them to take evacuation action safely and with peace of mind.

[0835] Example 2

[0836] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0837] In modern society, rapid and safe evacuation in the event of a disaster is a crucial issue. However, conventional evacuation support systems lack real-time information collection and analysis capabilities, making it difficult to provide evacuation support that is tailored to the user's individual emotional state. As a result, it is not possible to provide optimal evacuation routes in situations where users are feeling stressed or anxious, and issues remain regarding the safety and efficiency of evacuation behavior.

[0838] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0839] In this invention, the server includes means for acquiring real-time video from a live camera and performing preprocessing and analysis, means for acquiring hazard map data and performing integration and analysis, means for analyzing audio, image, and text data acquired from a user terminal and determining the user's emotional state, means for adjusting the evacuation route guidance method based on the user's emotional state, means for calculating an optimal evacuation route based on the acquired and analyzed information, and means for reflecting the calculated evacuation route in a map application and displaying it to the user. This makes it possible to provide an optimal evacuation route that takes into account the emotional state of each individual user based on information updated in real time.

[0840] A "live camera" is a camera device that captures video in real time and transmits it as digital data.

[0841] "Preprocessing" refers to processes such as noise removal and brightness adjustment that are performed to make the acquired video data easier to analyze.

[0842] "Analysis" is the process of analyzing acquired data using specific algorithms or models to extract useful information.

[0843] "Hazard map data" is map data that shows the risk of disasters occurring and the extent of their impact in a specific area.

[0844] "Integration" is the process of combining different types of data into a single dataset.

[0845] A "user terminal" is an electronic device such as a computer, smartphone, or tablet that is directly operated by a user.

[0846] "Voice data" refers to data in which the user's voice is recorded in digital format.

[0847] "Image data" refers to digital data of still or moving images captured by a camera or other photographic device.

[0848] "Text data" is data that stores characters or sentences in digital format.

[0849] "Emotional state" refers to the user's psychological state, and includes emotions such as anxiety, stress, and fear.

[0850] An "evacuation route" is a route that a user should follow to evacuate safely.

[0851] A "map application" is software that displays digital maps and provides navigation functions.

[0852] "Display" means visually presenting data or information to a user.

[0853] An "image recognition algorithm" is an algorithm for detecting specific patterns or features from input image data.

[0854] A "machine learning model" is a computational model that learns patterns from large amounts of data and performs predictions and classifications.

[0855] An "analytic model" is a set of rules or algorithms for performing analysis on a particular type of data.

[0856] "Evacuation route calculation" is the process of deriving a safe and efficient route based on acquired data.

[0857] "Adjustment based on the user's emotional state" refers to an operation of changing the information to be presented or the method of guidance in consideration of the user's psychological state.

[0858] The present invention is a system for supporting safe evacuation in the event of a disaster, collecting and analyzing information in real time and providing an optimal evacuation route taking into account the emotional state of the user. This system is composed of multiple servers and user terminals. Detailed embodiments of this system are described below.

[0859] Collection and analysis of live camera footage

[0860] Server: Images are acquired in real time from multiple live cameras installed in the surrounding area. The image data is first streamed to the server, where it is pre-processed using OpenCV. Noise removal and brightness adjustment are applied.

[0861] Server: The preprocessed video data is passed through image recognition algorithms such as TensorFlow and PyTorch to analyze whether roads are passable and whether there are any obstacles.

[0862] Examples:

[0863] The server acquires images from cameras in urban areas, preprocesses them using OpenCV, and then uses a TensorFlow model to analyze road congestion and flooding conditions.

[0864] Hazard map data integration and analysis

[0865] Server: The latest hazard map data provided by government agencies and public institutions is periodically obtained via API or FTP. The obtained data is stored in a GIS database such as PostGIS and updated in real time.

[0866] Server: This data is used to identify the extent of the disaster impact and risk areas and highlight them on a map.

[0867] Examples:

[0868] The server retrieves the latest flood hazard data from the Ministry of Land, Infrastructure, Transport and Tourism's API, imports it into a PostGIS database, and then identifies the flood-affected area and highlights it in red on the map.

[0869] Emotion Engine Operation

[0870] User: Enter voice messages, facial expressions, and text from a smartphone or tablet.

[0871] Terminal: Voice and facial expression data are collected on the user's terminal and sent to the server.

[0872] Server: Analyzes the received data using IBM Watson or Google Cloud Natural Language API to identify the user's emotional state.

[0873] Examples:

[0874] The user speaks to their smartphone, saying, "I'm very anxious." The device records the voice data and sends it to the server, which then analyzes the voice data using IBM Watson to evaluate the user's anxiety level.

[0875] Calculating the optimal evacuation route

[0876] Server: Calculates evacuation routes by integrating live camera analysis results, hazard map data, and the user's emotional state. Uses Google Maps API to obtain local geographic information and Algorithm A to calculate optimal evacuation routes.

[0877] Server: Generates evacuation route information based on the calculation results, including detailed instructions according to the user's emotional state.

[0878] Examples:

[0879] The server applies algorithm A based on the user's current location to calculate the optimal evacuation route, for example, generating a route that avoids areas impassable due to flooding and includes detailed instructions such as "turn right at the next intersection, then go straight."

[0880] Providing an emotion-based user interface

[0881] Server: Considers the user's emotional state and customizes the evacuation route guidance method.

[0882] Server: Provides detailed step-by-step guides if the user is unsure, or generates simplified instructions if simple instructions are appropriate.

[0883] Examples:

[0884] The server provides detailed "audio directions" and "step-by-step visual directions" in a map application based on the user's emotional state.

[0885] Route information provided by map applications

[0886] User: When evacuation is necessary, the user launches the map application on their smartphone or tablet and enters their current location and evacuation destination.

[0887] Terminal: Sends user requests to the server and receives optimal evacuation route information.

[0888] Terminal: The received evacuation route information is visually displayed on a map application.

[0889] Examples:

[0890] The user opens a map application and inputs their current location and evacuation destination. The device sends a request to the server and receives information on the optimal evacuation route. The route is displayed on a map so the user can follow it safely.

[0891] Prompt Sentence Examples

[0892] Below are some example prompts to input to the generative AI model:

[0893] "When a user needs to evacuate due to heavy rain, the system sends a notification such as, 'Please enter your current location and evacuation destination.' At that time, the system calculates and displays the optimal evacuation route based on live camera footage and hazard map data. It also takes into account the user's emotional state and provides detailed step-by-step instructions if necessary."

[0894] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0895] Step 1:

[0896] Collection of live camera footage

[0897] Server: Acquires real-time images from multiple live cameras installed in the surrounding area.

[0898] Input: Real-time video sent from a live camera.

[0899] Output: Video data streamed to the server.

[0900] Specific operation: The server streams video from multiple live cameras in urban areas and acquires it as data.

[0901] Step 2:

[0902] Video data preprocessing

[0903] Server: Preprocesses the acquired video data. Uses OpenCV to remove noise and adjust brightness.

[0904] Input: The video data streamed in step 1.

[0905] Output: Pre-processed and clean video data.

[0906] Specific operation: The server processes the video data using OpenCV, performing noise removal and brightness adjustment.

[0907] Step 3:

[0908] Video data analysis

[0909] Server: The preprocessed video data is run through an image recognition algorithm (TensorFlow or PyTorch) to analyze whether the road is passable and whether there are any obstacles.

[0910] Input: Preprocessed video data.

[0911] Output: Analysis results include data on road traversability and the presence of obstacles.

[0912] Specific operation: The server uses TensorFlow to analyze the level of road congestion and the presence of obstacles from video data.

[0913] Step 4:

[0914] Hazard map data acquisition and integration

[0915] Server: The latest hazard map data is periodically obtained from government agencies and public institutions via API and FTP.

[0916] Input: Hazard map data provided by government agencies and public institutions.

[0917] Output: Hazard map data stored in a GIS database such as PostGIS.

[0918] Specific operation: The server retrieves the latest flood hazard data from the Ministry of Land, Infrastructure, Transport and Tourism's API and imports it into a PostGIS database.

[0919] Step 5:

[0920] Analysis of hazard map data

[0921] Server: Analyzes the acquired hazard map data and identifies the extent of the disaster's impact and dangerous areas.

[0922] Input: Hazard map data stored in a PostGIS database.

[0923] Output: Data on the extent of the disaster impact and risk areas.

[0924] Specific operation: The server analyzes hazard map data and highlights the affected areas and dangerous areas of floods and other disasters on the map.

[0925] Step 6:

[0926] Collecting Emotional Data

[0927] User: Enter voice messages, facial expressions, and text from a smartphone or tablet.

[0928] Input: Audio, image, and text input data.

[0929] Output: Emotion data recorded on a smartphone or tablet.

[0930] Specific operation: The user speaks to the smartphone, "I'm very anxious." The device records the voice data.

[0931] Step 7:

[0932] Sending and analyzing emotional data

[0933] Terminal: Sends emotion data to the server.

[0934] Server: Analyzes the received data using IBM Watson or Google Cloud Natural Language API to identify the user's emotional state.

[0935] Input: Emotion data sent from the user device.

[0936] Output: User's emotional state (e.g., anxiety, stress).

[0937] Specific operation: The device sends the recorded voice data to the server, which then analyzes it using IBM Watson.

[0938] Step 8:

[0939] Calculating the optimal evacuation route

[0940] Server: Calculates the optimal evacuation route by integrating live camera analysis results, hazard map data, and the user's emotional state. Uses the Google Maps API to obtain local geographic information and algorithm A.

[0941] Input: Analysis results (live camera data, hazard map data, emotional state).

[0942] Output: The calculated optimal evacuation route.

[0943] Specific operation: The server executes algorithm A based on the user's current location and calculates a safe and efficient evacuation route.

[0944] Step 9:

[0945] Coordination of evacuation route guidance

[0946] Server: Customize evacuation route guidance based on the user's emotional state. Provide detailed guidance if anxiety is high, and simple guidance if anxiety is low.

[0947] Input: The user's emotional state.

[0948] Output: Customized evacuation route guidance.

[0949] Specific behavior: If the server is concerned about the user, it generates a detailed step-by-step guide.

[0950] Step 10:

[0951] Providing evacuation route information

[0952] User: When evacuation is necessary, the user launches the map application on their smartphone or tablet and enters their current location and evacuation destination.

[0953] Terminal: Sends user input information to the server as a request and receives optimal evacuation route information.

[0954] Server: Generates optimal evacuation route information and sends it back to the terminal.

[0955] Input: User's current location and evacuation destination.

[0956] Output: Optimal evacuation route displayed on the user's terminal.

[0957] Specific operation: The user inputs the evacuation destination using a map application, and the server sends the calculation results to the device, which displays them on a map.

[0958] (Application example 2)

[0959] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0960] Safe evacuation during disasters requires real-time information collection and analysis, and complex data integration is required to provide appropriate evacuation routes. Psychological factors also need to be taken into account if evacuees are feeling stressed or anxious. Previous systems have not adequately considered emotional states when proposing evacuation routes, which has led to issues with ensuring the safety and security of evacuees.

[0961] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0962] In this invention, the server includes means for acquiring real-time images from a live camera and performing preprocessing and analysis, means for acquiring hazard map data and performing integration and analysis, means for calculating an optimal evacuation route based on the acquired and analyzed information, means for reflecting the calculated evacuation route in a map application and displaying it to the user, means for collecting the user's voice data and camera images and analyzing their emotional state, and means for adjusting the user interface according to their emotional state, thereby making it possible to provide an optimal evacuation route that takes the user's emotional state into consideration.

[0963] A "live camera" is a camera that captures images in real time and monitors the surrounding situation.

[0964] "Real-time video" is video that records the current situation instantly and is transmitted with almost no delay.

[0965] "Preprocessing" refers to the initial data processing step of organizing and converting collected raw data into a form that is easier to analyze.

[0966] "Hazard map data" is map data that shows dangerous areas due to natural disasters, accidents, etc.

[0967] "Integration" is the process of bringing together multiple pieces of data into one system and correlating them.

[0968] The "optimal evacuation route" is a route that avoids danger and allows you to reach your destination safely and quickly.

[0969] A "map application" is software that displays digital maps and provides location information and route guidance.

[0970] "Voice data" refers to data that records a user's voice and saves it in an analyzable format.

[0971] "Camera video" refers to video data captured by a camera.

[0972] "Emotional state" is an indicator of the user's psychological state, including anxiety and stress.

[0973] "Analysis" is a method for examining data and extracting information.

[0974] "Adjusting the user interface" means changing the operation screen and guidance method according to the user's situation and emotions.

[0975] This invention is a system that supports safe evacuation behavior in the event of a disaster. This system acquires and analyzes live camera images, integrates and analyzes the latest hazard map data, analyzes the user's emotional state, and calculates and presents the optimal evacuation route. Each processing step is described in detail below.

[0976] Collection and analysis of live camera footage

[0977] The server receives real-time video from multiple live cameras, and the video data is pre-processed and analyzed using image recognition algorithms and machine learning models to determine whether roads are passable and whether there are any obstacles.

[0978] Hazard map data integration and analysis

[0979] The server retrieves the latest hazard map data provided by government agencies and public institutions via API and FTP. The retrieved data is integrated into a GIS database and analyzed to identify the extent of disaster impact and risk areas.

[0980] Emotion Engine Operation

[0981] The server collects the user's voice data and camera footage and analyzes them with an emotion engine. Using techniques such as voice analysis, facial expression analysis, and text analysis, the server determines the user's emotional state, assessing whether the user is feeling stressed or anxious.

[0982] Calculating the optimal evacuation route

[0983] The server integrates live camera analysis results, hazard map data, and the user's emotional state to calculate the optimal evacuation route. Safe and efficient routes are derived using Dijkstra's algorithm and A algorithm. It is also possible to include detailed guidance steps according to the user's emotional state.

[0984] Emotion-based user interface adjustment

[0985] The server then adjusts the evacuation route guidance based on the emotion engine's analysis: if the user is feeling stressed, it provides more detailed, step-by-step instructions, and if the user is feeling less anxious, it provides simpler instructions.

[0986] Route information provided by map applications

[0987] Users launch a map application on their smartphone or tablet and input their current location and evacuation destination. The device then sends a request to the server and receives information on the optimal evacuation route. The server returns the optimal route information calculated in real time, and the device displays this information in detail on the map application. Users can check this information to evacuate safely.

[0988] Specific examples

[0989] In the event of a disaster, if a user is evacuating in their vehicle, the vehicle's infotainment system will use this system to provide the optimal evacuation route in real time. The guidance method will automatically adjust according to the user's emotional state, reducing the user's anxiety.

[0990] Prompt Sentence Examples

[0991] Visual Recognition Prompt: "Analyze real-time video from the camera to detect road conditions and obstacles."

[0992] Hazard Map Integration Prompt: "Get the latest hazard map data and identify the extent of the impact of a disaster."

[0993] Emotion Analysis Prompt: "Analyze the user's voice and facial expressions to determine their emotional state."

[0994] Route Optimization Prompt: "Calculate the optimal evacuation route, taking into account real-time data and emotional state."

[0995] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0996] Step 1:

[0997] The server acquires real-time video from the live camera. The acquired video data is first pre-processed to remove noise and adjust the resolution. This pre-processed data is then analyzed using image recognition algorithms and machine learning models. As a result of the analysis, it determines whether the road is passable and whether there are any obstacles, and outputs this as numerical data.

[0998] Step 2:

[0999] The server retrieves the latest hazard map data provided by government agencies and public institutions via API and FTP. The retrieved data is integrated into a GIS database and analyzed to identify the extent of disaster impact and risk areas. This generates and outputs coordinate data for risk areas.

[1000] Step 3:

[1001] The server collects voice data and camera footage from the user's smartphone and vehicle infotainment system. This data is then analyzed using an emotion engine to perform voice analysis and facial expression analysis. The analysis results are output as numerical data representing the user's stress and anxiety levels.

[1002] Step 4:

[1003] The server integrates the road condition data obtained in step 1, the hazard map data obtained in step 2, and the user's emotional state data obtained in step 3. It calculates the optimal evacuation route using Dijkstra's algorithm and the A algorithm. As a result of the calculation, it outputs a specific route and detailed guidance steps.

[1004] Step 5:

[1005] The server adjusts the evacuation route guidance method based on the analysis results of the emotion engine. For example, if the user is feeling stressed, it provides detailed step-by-step guidance. If the user is feeling less anxious, it provides simple guidance. This generates and outputs data for a customized guidance method.

[1006] Step 6:

[1007] The device sends a request to the server based on the user's input. The server returns optimal route information calculated in real time to the device. The device displays this received route information in a map application, providing a visual representation to the user. The user can then begin evacuation actions based on the displayed route information.

[1008] Step 7:

[1009] Users can follow the evacuation route displayed on the device's map application and check real-time updates to safely evacuate. The system continues to collect and analyze data while on the move, recalculating and updating evacuation routes as needed.

[1010] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1011] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1012] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1013] [Third embodiment]

[1014] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1015] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1016] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1017] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1018] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1019] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1020] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1021] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1022] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1023] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1024] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1025] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1026] This invention is a system that supports safe evacuation in the event of a disaster, integrating live cameras, hazard maps, and map applications to provide optimal evacuation routes in real time. The specific operation and program processing of this system are described below.

[1027] System Overview

[1028] This system consists of multiple servers and user terminals. The servers have the means to collect and analyze video data from live cameras. They also acquire and analyze hazard map data to assess the impact of disasters. They also have the means to calculate optimal evacuation routes and provide the results to users. Meanwhile, the user terminals use a map application to receive and visually display evacuation route information.

[1029] What the program does

[1030] The program processing of the system will be explained in natural language below.

[1031] Collection and analysis of live camera footage

[1032] Server: Real-time video is acquired from multiple live cameras installed in the surrounding area. This video is first pre-processed and then analyzed using image recognition algorithms and machine learning models to determine whether roads are passable and whether there are any obstacles.

[1033] Hazard map data integration and analysis

[1034] Server: Acquires hazard map data provided by government agencies and public institutions. This data is integrated into a GIS database and subjected to analytical models to identify the extent of disaster impact and risk areas. Different analytical models are applied for different types of disasters (e.g., floods, fires, earthquakes, etc.).

[1035] Calculating the optimal evacuation route

[1036] Server: Integrates the analysis results of live camera footage with hazard map data to calculate the optimal evacuation route based on the departure and destination points specified by the user. Path-finding algorithms such as Dijkstra's algorithm and A algorithm are used. The calculated evacuation route information is immediately saved in a database.

[1037] Route information provided by map applications

[1038] Device: The user launches a map application on their smartphone or tablet and inputs their current location and evacuation destination. The application sends this information to the server, from which it receives information on the optimal evacuation route.

[1039] Server: Receives user requests and returns optimal evacuation route information in real time.

[1040] Device: The received evacuation route information is displayed on a map application, allowing the user to check the information and evacuate safely.

[1041] Specific examples

[1042] Below are some examples of specific scenarios that use this system.

[1043] Disaster scenario

[1044] User B is a traveler staying in a flood-hit area and is looking for a route to a suitable evacuation site.

[1045] 1. Collection and analysis of live camera footage

[1046] Server: Acquires and analyzes real-time video from multiple live cameras. From the acquired video data, it is possible to determine whether major roads are flooded.

[1047] 2. Integration and analysis of hazard map data

[1048] Server: Obtains the latest hazard map data and identifies the area affected by the flood. Based on this data, the server understands the status of the disaster affecting User B's location.

[1049] 3. Calculating the optimal evacuation route

[1050] Server: Integrates the analysis results of live camera footage with hazard map data to calculate the optimal route from User B's current location to the nearest evacuation site. Using Dijkstra's algorithm, the safest and most efficient route is derived and the information is stored in a database.

[1051] 4. Route information provided by map applications

[1052] Device: User B launches a map application and inputs their current location and evacuation destination. The application sends a request to the server and receives information on the optimal evacuation route.

[1053] Server: Receives User B's request and returns the optimal route information calculated in real time.

[1054] Device: The map application displays the received information, and User B can visually check the route.

[1055] 5. Supporting users in evacuation

[1056] User: User B follows the instructions of the application and safely travels to the evacuation site via the designated route.

[1057] In this way, the present invention allows users to obtain appropriate evacuation routes based on real-time information when a disaster occurs, enabling safe and rapid evacuation.

[1058] The processing flow will be explained below.

[1059] Step 1: Acquire live camera footage

[1060] Server: Accesses live cameras installed in the specified area and acquires video in real time. Collects data periodically using the endpoint URL and authentication information.

[1061] Step 2: Preprocessing the video data

[1062] Server: Preprocesses the captured live camera footage. Specifically, it resizes the footage, removes noise, corrects color, etc., and converts it into a state suitable for analysis.

[1063] Step 3: Analyzing the video data

[1064] Server: Analyzes the pre-processed video data using image recognition algorithms and machine learning models, extracting information such as road passability, obstacles, and flood damage.

[1065] Step 4: Save the analysis results

[1066] Server: Stores the analysis results in a database, including location information, traffic conditions, and details of detected obstacles.

[1067] Step 5: Obtaining hazard map data

[1068] Server: Obtains the latest hazard map data provided by government agencies and public institutions using API and FTP.

[1069] Step 6: Preprocessing and integration of hazard map data

[1070] Server: Performs preprocessing to integrate acquired hazard map data into the GIS database, including data format conversion, spatial correction, and filtering.

[1071] Step 7: Analyze hazard map data

[1072] Server: Analyzes the integrated hazard map data to identify the extent of disaster impacts and risk areas. Different analytical models are used for each disaster, such as floods, fires, and earthquakes.

[1073] Step 8: Calculate the optimal evacuation route

[1074] Server: Based on the results of live camera analysis and hazard map data, the server calculates the optimal evacuation route for the departure and destination points specified by the user. It uses Dijkstra's algorithm and A algorithm to derive a safe and efficient route.

[1075] Step 9: Save the evacuation route data

[1076] Server: The calculated optimal evacuation route information is stored in a database so that subsequent client requests can be handled promptly.

[1077] Step 10: Accepting a user request

[1078] Device: The user launches a map application and inputs their current location and evacuation destination. This information is sent as a request to the server.

[1079] Step 11: Send evacuation route information

[1080] Server: Receives user requests, generates optimal evacuation route information based on the latest analysis data and route calculation results, and returns the generated information to the device.

[1081] Step 12: Display evacuation route information

[1082] Device: The received evacuation route information is displayed in a map application, showing the route and important points (obstacles, evacuation points, etc.) to make it easier for the user to understand visually.

[1083] Step 13: Start evacuation and check the situation

[1084] User: While looking at the map application, the user begins evacuation by following the optimal route displayed. While on the move, the user can check the latest analysis data through the application to ensure safety.

[1085] Step 14: Notification of evacuation completion

[1086] User: Upon reaching the evacuation location, the application will notify the user that evacuation is complete. If necessary, the user will be able to check the recalculated route.

[1087] This series of processes allows users to evacuate quickly and safely based on information updated in real time.

[1088] Example 1

[1089] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1090] When a disaster occurs, it is important to provide safe and effective evacuation routes in real time. However, conventional evacuation support systems have difficulty collecting real-time information and calculating optimal evacuation routes based on that information. Furthermore, they do not perform sufficient analysis according to different disaster types, making it impossible to quickly provide optimal evacuation routes for users. This leads to evacuation delays and the selection of inappropriate evacuation routes, which can compromise user safety.

[1091] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1092] In this invention, the server includes means for acquiring real-time video from a live camera and analyzing it using preprocessing and image recognition algorithms and machine learning models, means for acquiring hazard map data from public institutions, integrating it into a GIS database, and analyzing it using different analysis models depending on the disaster, means for calculating the optimal evacuation route from the departure point and destination using a route search algorithm based on the acquired and analyzed video data and hazard map data, and means for saving the calculated evacuation route in the database and reflecting it in a map application and displaying it to the user upon user request. This makes it possible to quickly provide the optimal and safest evacuation route based on information collected in real time when a disaster occurs.

[1093] A "live camera" is an image capturing device installed to capture real-time images of the surrounding area.

[1094] "Preprocessing" refers to the process of performing processes such as noise removal and frame resizing to make the acquired video data easier to analyze.

[1095] An "image recognition algorithm" is a computational method for detecting specific objects or patterns within video data.

[1096] A "machine learning model" is a collection of algorithms that automatically learn from data and make predictions and classifications.

[1097] "Hazard map data" is geographic information that indicates the risk and impact of disasters.

[1098] "Public institutions" are public data providers such as government agencies and local governments.

[1099] A "GIS database" is a database system that manages geographic information in an integrated manner.

[1100] A "route search algorithm" is a computational method for finding the optimal route between a specified starting point and destination.

[1101] A "database" is a system for systematically storing information and efficiently retrieving it when needed.

[1102] A "user terminal" is an electronic device, such as a smartphone or tablet, that runs a map application.

[1103] A "map application" is software that displays a user's current location, destination, and route information.

[1104] This invention is a system for supporting safe evacuation in the event of a disaster, integrating live cameras, hazard maps, and map applications to provide optimal evacuation routes in real time. This system is composed of multiple servers and user terminals, and a specific embodiment is shown below.

[1105] System Overview

[1106] The system mainly uses a server, live cameras, user devices, map applications, and hazard map data. Below, we will explain the details of each component and how they work.

[1107] Collection and analysis of live camera footage

[1108] Server: The server acquires real-time video from multiple live cameras installed in the surrounding area. The video is transferred to the server at a fixed frame rate and received using a streaming protocol (e.g., RTSP). The video data is pre-processed and analyzed using a machine learning model (e.g., TensorFlow model).

[1109] Specifically, the server connects to each live camera and captures video data in real time. This data undergoes preprocessing such as noise removal and frame resizing, and is then analyzed using image recognition algorithms and machine learning models. This allows the system to determine whether the road is passable and whether there are any obstacles.

[1110] Acquisition and analysis of hazard map data

[1111] Server: The server periodically retrieves hazard map data provided by government agencies and public institutions. This data is retrieved through APIs and stored in a GIS database. The retrieved data is analyzed using analytical models (e.g., ArcGIS) to identify the extent of disaster impact and risk areas. In this case, analytical models appropriate for different disasters, such as floods, fires, and earthquakes, are applied.

[1112] Calculating the optimal evacuation route

[1113] Server: The server integrates the analysis results of live camera footage with hazard map data and calculates the optimal evacuation route based on the departure and destination points specified by the user. A path-finding algorithm (e.g., Dijkstra algorithm, A algorithm) is used, and the calculated evacuation route information is stored in a database.

[1114] Route information provided by map applications

[1115] Device: The user launches a map application (e.g., Google Maps, Apple Maps) on their smartphone or tablet and enters their current location and evacuation destination. The application sends this information to the server, which then receives information on the optimal evacuation route. The received evacuation route information is displayed on the map application, allowing the user to check it and safely evacuate.

[1116] Specific examples

[1117] Disaster scenario

[1118] User B is staying in a flood-hit area and is searching for a suitable evacuation route. The specific steps are as follows:

[1119] 1. Server: Acquires real-time video from live cameras, performs pre-processing and analysis, and checks whether major roads are submerged from the acquired video data.

[1120] 2. Server: Obtains the latest hazard map data and identifies the extent of flood impact.

[1121] 3. Server: Integrates live camera footage and hazard map data, and calculates the optimal route from User B's current location to the nearest evacuation site using Dijkstra's algorithm.

[1122] 4. Device: User B launches the map application and inputs their current location and evacuation destination. The application sends a request to the server and receives information on the optimal evacuation route.

[1123] 5. Server: Returns the optimal route information calculated in real time to the user device.

[1124] 6. Device: The map application displays the received information, and User B visually checks the route and proceeds with the evacuation safely.

[1125] Examples of prompt statements

[1126] Below is an example of a prompt sentence to input to the generative AI model.

[1127] Prompt: Explain how a real-time evacuation assistance system works in the event of a disaster.

[1128] Collecting and analyzing live camera footage

[1129] Integrating and analyzing hazard map data obtained from government agencies

[1130] Calculate the optimal evacuation route based on your current location and evacuation destination

[1131] Providing route information to users through a map application

[1132] In this way, the present invention can quickly provide optimal and safe evacuation routes based on information collected in real time when a disaster occurs.

[1133] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1134] Step 1:

[1135] Collection of live camera footage

[1136] Server: Obtains real-time video data from live cameras. The input is the IP address of each live camera, and the output is raw video data. The server connects to the live cameras using the RTSP streaming protocol and stores the video data in a buffer at a specific frame rate.

[1137] Specific behavior:

[1138] 1. The server connects to the IP address of each live camera and starts the RTSP stream.

[1139] 2. Video data is stored in a buffer in real time.

[1140] Step 2:

[1141] Video data preprocessing and analysis

[1142] Server: Preprocesses the acquired video data and analyzes it using image recognition algorithms and machine learning models. The input is raw video data, and the output is analysis results indicating passability and the presence or absence of obstacles. Preprocessing involves noise removal and frame resizing.

[1143] Specific behavior:

[1144] 1. Noise is removed from the video data.

[1145] 2. Resize the frame to a size that is easy to analyze.

[1146] 3. Use a machine learning model (e.g., a TensorFlow model) to analyze each frame and identify obstacles and impassable areas.

[1147] Step 3:

[1148] Obtaining hazard map data

[1149] Server: Obtains hazard map data provided by public institutions. The input is the API endpoint, and the output is raw hazard map data. The server periodically issues API requests to obtain the latest data.

[1150] Specific behavior:

[1151] 1. The server issues an API request to obtain the latest hazard map data.

[1152] 2. The acquired data is stored in a GIS database.

[1153] Step 4:

[1154] Analysis of hazard map data

[1155] Server: Stores the acquired hazard map data in a GIS database and performs analysis to identify the extent of disaster impact and risk areas. The input is raw hazard map data, and the output is the analysis results.

[1156] Specific behavior:

[1157] 1. Hazard map data stored in a GIS database is input into the analytical model.

[1158] 2. Apply different analytical models to various disasters (e.g., flood, fire, earthquake) to identify risk areas.

[1159] Step 5:

[1160] Calculating the optimal evacuation route

[1161] Server: Integrates the analysis results of live camera footage and hazard map data, and calculates the optimal evacuation route based on the departure and destination points specified by the user. The input is the analysis results and departure and destination information, and the output is the optimal evacuation route.

[1162] Specific behavior:

[1163] 1. Based on composite data (camera image analysis results and hazard map data), information on the departure and destination points is combined.

[1164] 2. Calculate the optimal evacuation route using a pathfinding algorithm (e.g., Dijkstra's algorithm, A algorithm).

[1165] 3. Save the calculation results in the database.

[1166] Step 6:

[1167] Processing user requests

[1168] Device: The user launches a map application on their smartphone or tablet and inputs their current location and evacuation destination. The input is the user's current location and evacuation destination information, and the output is a request sent to the server.

[1169] Specific behavior:

[1170] 1. Provide an interface for inputting current location and evacuation destination from a map application.

[1171] 2. The entered data is sent to the server as an HTTP request.

[1172] Step 7:

[1173] Providing evacuation route information

[1174] Server: Receives user requests and returns optimal evacuation route information calculated in real time. The output is optimal evacuation route information.

[1175] Terminal: Displays the received evacuation route information in a map application. The input is the optimal evacuation route information sent from the server, and the output is a visual display for the user.

[1176] Specific behavior (server side):

[1177] 1. Receives a request and retrieves the optimal route information from the database.

[1178] 2. The acquired route information is returned to the user's device in JSON format.

[1179] Specific operations (terminal side):

[1180] 1. The route information returned from the server is displayed in the map application.

[1181] 2. The user proceeds with evacuation while referring to the map application.

[1182] (Application example 1)

[1183] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1184] Currently, many regions require systems to ensure safe evacuation in the event of a disaster. However, there are still limited systems that can respond to changing situations in real time and quickly provide optimal evacuation routes. This makes it difficult for autonomous vehicles to operate appropriately and safely, especially when transportation infrastructure is disrupted. Furthermore, there are no systems that can integrate information from different sources, such as live cameras and hazard maps, and provide optimal evacuation information in real time. For this reason, there is a need to develop a system that can reliably guide autonomous vehicles along safe routes and provide visual evacuation information to passengers in the event of a disaster.

[1185] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1186] In this invention, the server includes means for acquiring real-time images from live cameras and performing preprocessing and analysis, means for acquiring hazard map data and performing integration and analysis, means for calculating optimal evacuation routes based on the acquired and analyzed information, means for reflecting the calculated evacuation routes in the control system of the autonomous vehicle and updating the driving route in real time, and means for displaying evacuation route information on an in-vehicle display to provide it visually to the user. This makes it possible to provide optimal evacuation routes to autonomous vehicles in real time and achieve safe and efficient driving in the event of a disaster.

[1187] A "live camera" is a camera device that captures images in real time and transmits the data.

[1188] "Real-time video" refers to video data that shows the current situation in real time.

[1189] "Preprocessing" refers to the initial stage of processing performed on video data acquired by a live camera, and includes processes such as noise removal and image standardization.

[1190] "Analysis" is the process of extracting necessary information and making decisions based on preprocessed data.

[1191] "Hazard map data" is map information showing disasters and dangerous areas, and is data provided by government agencies and public institutions.

[1192] "Integration" is the process of bringing together multiple different data sources into one system.

[1193] An "optimal evacuation route" is the route that will allow you to reach your evacuation destination most safely and efficiently in the event of a disaster.

[1194] "Calculation" is the process of deriving a specific result using an algorithm based on given data.

[1195] An "autonomous vehicle" is a vehicle that has the ability to operate automatically without the need for a driver.

[1196] "Control system" means an electronic or mechanical system for managing and directing the operation of a vehicle.

[1197] The "travel route" is the route that a vehicle takes to reach its destination.

[1198] An "in-vehicle display" is a screen installed inside a vehicle to display information.

[1199] "User" means a person who uses the system or vehicle.

[1200] "Visually presented" means that the information is presented in a visual form.

[1201] "Real-time updates" refers to revising the system information as needed in response to changes in background data and conditions.

[1202] MODE FOR CARRYING OUT THE INVENTION

[1203] An example of this application is a system that enables autonomous vehicles to safely evacuate in the event of a disaster, integrating live cameras, hazard maps, and real-time calculation of optimal evacuation routes.

[1204] Overall structure

[1205] This system consists of multiple servers and user terminals. The servers have the means to collect and analyze video data from live cameras. They also acquire and analyze hazard map data to assess the impact of disasters. They also have the means to calculate optimal evacuation routes and provide the results to the autonomous vehicle. Meanwhile, the user terminals receive the evacuation route information and visually display it on an in-vehicle display.

[1206] Hardware and software used

[1207] Hardware:

[1208] Live cameras: Multiple cameras installed on a vehicle.

[1209] Server: A computing device used to collect, process, and store data.

[1210] In-vehicle display: A screen installed inside the vehicle to display information.

[1211] software:

[1212] OpenCV: Used to collect and preprocess camera footage.

[1213] Machine learning models (e.g. TensorFlow, PyTorch): Road condition analysis.

[1214] requests library: A library for sending HTTP requests and retrieving hazard map data.

[1215] Pathfinding algorithms (e.g., Dijkstra, A): Used to calculate optimal evacuation routes.

[1216] Program processing description

[1217] The server acquires real-time video footage from live cameras and performs preprocessing using OpenCV. The acquired video footage is analyzed using a machine learning model to determine whether roads are passable and whether there are any obstacles. Hazard map data is then acquired from government agencies and public institutions and integrated into a GIS database. Based on this data, the server calculates the optimal evacuation route. This calculation uses route search algorithms such as Dijkstra's algorithm and A algorithm. The calculated evacuation route information is reflected in the autonomous vehicle's control system, and the driving route is updated in real time.

[1218] The terminals then display the received evacuation route information on the in-car display, providing a visual representation to passengers, allowing users to evacuate safely while checking the information in real time.

[1219] Specific examples

[1220] For example, the process for updating route information when a disaster occurs is as follows:

[1221] 1. Collection and analysis of live camera footage

[1222] The server acquires real-time images from live cameras installed in vehicles, processes them, and analyzes them. For example, it uses OpenCV to remove noise and uses models using TensorFlow and PyTorch to determine whether a road is passable.

[1223] 2. Integration and analysis of hazard map data

[1224] The server uses the requests library to retrieve the latest hazard map data from government agencies and integrate it into a GIS database. This data is then processed with different analytical models (e.g., flood, fire).

[1225] 3. Calculating the optimal evacuation route

[1226] Based on the acquired and analyzed data, the server uses Dijkstra's algorithm and A algorithm to calculate the optimal route from the user's current location to the evacuation site.

[1227] 4. Real-time route updates

[1228] The evacuation route calculated by the server is sent to the autonomous vehicle's control system, which uses this information to update its route in real time.

[1229] 5. Providing visual information through in-car displays

[1230] The terminal displays the calculated evacuation route information on the in-vehicle display, allowing users to visually confirm it.

[1231] Prompt Sentence Examples

[1232] The following prompts are used to encourage a generative AI model to perform a specific task.

[1233] text

[1234] To provide safe evacuation routes in the event of a disaster, implement an optimal route calculation system that analyzes live camera footage and integrates it with hazard maps. Specifically, create an application that allows an autonomous vehicle to calculate the optimal evacuation route in real time based on its current location and destination, and display that information on an in-car display.

[1235] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1236] Step 1: Collecting live camera footage

[1237] The server acquires video data in real time from live cameras installed in the vehicle. The input is video data from multiple cameras, and the output is preprocessed video data. Specifically, it uses OpenCV to capture video data and performs preprocessing such as noise removal and frame alignment.

[1238] Step 2: Analyzing the video data

[1239] The server inputs the preprocessed video data into a machine learning model to analyze whether the road is passable and whether there are any obstacles. The input is the preprocessed video data, and the output is information on whether the road is passable or not. Specifically, the server uses a trained model using TensorFlow and PyTorch to determine the road conditions from the video data.

[1240] Step 3: Obtaining hazard map data

[1241] The server uses the requests library to obtain the latest hazard map data from government agencies and public institutions. The input is an HTTP request, and the output is hazard map data in JSON format. Specifically, it downloads the data from the specified URL and converts it into an analyzable format.

[1242] Step 4: Analyze hazard map data

[1243] The server analyzes the acquired hazard map data and identifies dangerous areas and areas of impact. The input is the hazard map data, and the output is the analyzed dangerous area information. Specifically, it applies analytical models corresponding to different types of disasters (e.g., floods, fires, earthquakes) and determines safe routes.

[1244] Step 5: Calculate the optimal evacuation route

[1245] The server integrates the video analysis results with hazard map data to calculate the optimal route from the user's current location to the evacuation destination. The input is road condition data and dangerous area information, and the output is the optimal evacuation route. Specifically, it uses Dijkstra's algorithm and A algorithm to derive the safest route.

[1246] Step 6: Real-time updates of routes

[1247] The server sends the calculated optimal evacuation route to the autonomous vehicle's control system and updates the driving route in real time. The input is the optimal evacuation route information, and the output is the updated driving route. Specifically, the server sends data to the vehicle's control system using a communication protocol.

[1248] Step 7: Display evacuation route information

[1249] The terminal displays the received evacuation route information on the in-car display, providing a visual presentation to passengers. The input is the optimal evacuation route information, and the output is the visual information on the display. Specifically, the information is displayed using a GUI (Graphical User Interface).

[1250] Step 8: Support users in evacuation

[1251] The user follows the route information displayed on the screen and safely travels along the designated route to the evacuation site. The input is the visually provided route information, and the output is the user's evacuation behavior. Specific actions include selecting a safe route based on the application's instructions and traveling by manual or automatic driving.

[1252] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1253] This invention is a system that supports safe evacuation in the event of a disaster, and provides an optimal evacuation route that takes into account the user's emotional state in addition to collecting and analyzing information in real time. The specific operation and program processing of this system are described below.

[1254] System Overview

[1255] This system consists of multiple servers and user terminals. The servers have the means to collect and analyze video data from live cameras. They also acquire hazard map data and assess the impact of disasters. They also have an emotion engine that recognizes the user's emotional state and calculates the optimal evacuation route. The user terminals receive evacuation route information using a map application and visually display it.

[1256] What the program does

[1257] The program processing of the system will be explained in natural language below.

[1258] Collection and analysis of live camera footage

[1259] Server: Acquires real-time video feeds from multiple live cameras installed in the surrounding area. These feeds are pre-processed and analyzed using image recognition algorithms and machine learning models. The analysis results include information on road availability and the presence of obstacles.

[1260] Hazard map data integration and analysis

[1261] Server: The latest hazard map data provided by government agencies and public institutions is obtained via API and FTP. The obtained data is integrated into a GIS database and subjected to analytical models to identify the extent of disaster impact and risk areas.

[1262] Emotion Engine Operation

[1263] Server: Collects voice data, camera footage, and input text data from the user's device and analyzes them with an emotion engine. The server uses techniques such as voice analysis, facial expression analysis, and text analysis to determine the user's emotional state.

[1264] Calculating the optimal evacuation route

[1265] Server: Integrates live camera analysis results, hazard map data, and the user's emotional state to calculate the optimal evacuation route. Utilizing Dijkstra's algorithm and A algorithm, it derives a safe and efficient route.

[1266] Emotion-based user interface adjustment

[1267] Server: Based on the analysis results of the emotion engine, the server adjusts the evacuation route guidance method. For example, if the user is feeling stressed, it provides detailed step-by-step instructions. Conversely, if the user is feeling less anxious, it provides simpler instructions.

[1268] Route information provided by map applications

[1269] Device: The user launches a map application on their smartphone or tablet and inputs their current location and evacuation destination. The application then sends a request to the server, which then receives information on the optimal evacuation route.

[1270] Server: Receives user requests, generates optimal evacuation route information based on the latest analysis data and route calculation results, and returns it to the terminal.

[1271] Device: The received evacuation route information is displayed on a map application, allowing the user to check the information and evacuate safely.

[1272] Specific examples

[1273] Below are some examples of specific scenarios that utilize this system.

[1274] Disaster scenario

[1275] User C is a traveler staying in a flood-hit area and is looking for a route to an appropriate evacuation site. User C feels anxious and needs detailed evacuation instructions.

[1276] 1. Collection and analysis of live camera footage

[1277] Server: Acquires and analyzes real-time video from multiple live cameras. From the acquired video data, it is possible to determine whether major roads are flooded.

[1278] 2. Integration and analysis of hazard map data

[1279] Server: Obtains the latest hazard map data and identifies the area affected by the flood. Based on this data, the server understands the status of the disaster affecting User C's location.

[1280] 3. Operation of the Emotion Engine

[1281] Server: Analyzes audio data and camera footage collected from User C's smartphone to assess his / her anxiety and stress levels. It turns out that User C is experiencing high levels of anxiety.

[1282] 4. Calculating the optimal evacuation route

[1283] Server: Calculates the optimal evacuation route by taking into account the live camera analysis results, hazard map data, and the emotional state of User C. Generates route information including detailed guidance steps.

[1284] 5. Route information provided by map applications

[1285] Device: User C launches a map application and inputs their current location and evacuation destination. The application sends a request to the server and receives information on the optimal evacuation route.

[1286] Server: Receives user C's request and returns the optimal route information calculated in real time.

[1287] Device: The map application displays the received information in detail and guides User C to evacuate without worry.

[1288] 6. Supporting users in evacuation

[1289] User C follows the instructions of the application and begins evacuation along the optimal route displayed. While on the move, he checks the latest analysis data to ensure his safety.

[1290] In this way, the system of the present invention can support safer and more secure evacuation behavior by taking into account the user's emotional state. Users can evacuate safely based on information updated in real time.

[1291] The processing flow will be explained below.

[1292] Step 1: Acquire live camera footage

[1293] Server: Accesses live cameras installed in the specified area and acquires video in real time. Collects data periodically using the endpoint URL and authentication information.

[1294] Step 2: Preprocessing the video data

[1295] Server: Preprocesses the captured live camera footage. Specifically, it resizes the footage, removes noise, corrects color, etc., and converts it into a state suitable for analysis.

[1296] Step 3: Analyzing the video data

[1297] Server: Analyzes the pre-processed video data using image recognition algorithms and machine learning models, extracting information such as road passability, obstacles, and flood damage.

[1298] Step 4: Save the analysis results

[1299] Server: Stores the analysis results in a database, including location information, traffic conditions, and details of detected obstacles.

[1300] Step 5: Obtaining hazard map data

[1301] Server: Obtains the latest hazard map data provided by government agencies and public institutions using API and FTP.

[1302] Step 6: Preprocessing and integration of hazard map data

[1303] Server: Performs preprocessing to integrate acquired hazard map data into the GIS database, including data format conversion, spatial correction, and filtering.

[1304] Step 7: Analyze hazard map data

[1305] Server: Analyzes the integrated hazard map data to identify the extent of disaster impacts and risk areas. Different analytical models are used for each disaster, such as floods, fires, and earthquakes.

[1306] Step 8: Obtaining Emotion Data

[1307] Device: The user provides audio data and camera footage via a smartphone or tablet, which then collects data on the user's emotional state.

[1308] Step 9: Analyze the sentiment data

[1309] Server: Analyzes collected voice, facial expression, and text data using an emotion engine to evaluate the user's emotional state (anxiety, stress, calm, etc.).

[1310] Step 10: Storing Emotion Data

[1311] Server: The analysis results are saved in a database. This data is used as auxiliary data for later evacuation route calculations.

[1312] Step 11: Calculate the optimal evacuation route

[1313] Server: Integrates live camera analysis results, hazard map data, and the user's emotional state to calculate the optimal evacuation route between the departure and destination points specified by the user. A safe and efficient route is derived using Dijkstra's algorithm and the A algorithm.

[1314] Step 12: Save the evacuation route data

[1315] Server: The calculated optimal evacuation route information is stored in a database so that subsequent client requests can be handled promptly.

[1316] Step 13: Accepting User Requests

[1317] Terminal: The user launches the map application and inputs their current location and evacuation destination. After inputting the information, a request for an evacuation route is sent to the server.

[1318] Step 14: Send evacuation route information

[1319] Server: Receives user requests, generates optimal evacuation route information based on the latest analysis data and route calculation results, and returns the generated information to the device.

[1320] Step 15: Display evacuation route information

[1321] Device: The received evacuation route information is displayed in a map application, showing the route and important points (obstacles, evacuation points, etc.) to make it easier for the user to understand visually.

[1322] Step 16: Start evacuation and check the situation

[1323] User: While looking at the map application, the user begins evacuation by following the optimal route displayed. The user also checks the latest analysis data while on the move to ensure safety.

[1324] Step 17: Guidance adjustment based on emotional state

[1325] Server: Based on the analysis results of the emotion engine, the server adjusts the evacuation route guidance method. For example, if the user is feeling stressed, it provides detailed step-by-step instructions to reassure the user.

[1326] Step 18: Notification of Evacuation Completion

[1327] User: Upon reaching the evacuation location, the application will notify the user that evacuation is complete. If necessary, the user will be able to check the recalculated route.

[1328] This series of processes allows users to obtain an appropriate evacuation route based on real-time updated information and emotional state, allowing them to take evacuation action safely and with peace of mind.

[1329] Example 2

[1330] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1331] In modern society, rapid and safe evacuation in the event of a disaster is a crucial issue. However, conventional evacuation support systems lack real-time information collection and analysis capabilities, making it difficult to provide evacuation support that is tailored to the user's individual emotional state. As a result, it is not possible to provide optimal evacuation routes in situations where users are feeling stressed or anxious, and issues remain regarding the safety and efficiency of evacuation behavior.

[1332] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1333] In this invention, the server includes means for acquiring real-time video from a live camera and performing preprocessing and analysis, means for acquiring hazard map data and performing integration and analysis, means for analyzing audio, image, and text data acquired from a user terminal and determining the user's emotional state, means for adjusting the evacuation route guidance method based on the user's emotional state, means for calculating an optimal evacuation route based on the acquired and analyzed information, and means for reflecting the calculated evacuation route in a map application and displaying it to the user. This makes it possible to provide an optimal evacuation route that takes into account the emotional state of each individual user based on information updated in real time.

[1334] A "live camera" is a camera device that captures video in real time and transmits it as digital data.

[1335] "Preprocessing" refers to processes such as noise removal and brightness adjustment that are performed to make the acquired video data easier to analyze.

[1336] "Analysis" is the process of analyzing acquired data using specific algorithms or models to extract useful information.

[1337] "Hazard map data" is map data that shows the risk of disasters occurring and the extent of their impact in a specific area.

[1338] "Integration" is the process of combining different types of data into a single dataset.

[1339] A "user terminal" is an electronic device such as a computer, smartphone, or tablet that is directly operated by a user.

[1340] "Voice data" refers to data in which the user's voice is recorded in digital format.

[1341] "Image data" refers to digital data of still or moving images captured by a camera or other photographic device.

[1342] "Text data" is data that stores characters or sentences in digital format.

[1343] "Emotional state" refers to the user's psychological state, and includes emotions such as anxiety, stress, and fear.

[1344] An "evacuation route" is a route that a user should follow to evacuate safely.

[1345] A "map application" is software that displays digital maps and provides navigation functions.

[1346] "Display" means visually presenting data or information to a user.

[1347] An "image recognition algorithm" is an algorithm for detecting specific patterns or features from input image data.

[1348] A "machine learning model" is a computational model that learns patterns from large amounts of data and performs predictions and classifications.

[1349] An "analytic model" is a set of rules or algorithms for performing analysis on a particular type of data.

[1350] "Evacuation route calculation" is the process of deriving a safe and efficient route based on acquired data.

[1351] "Adjustment based on the user's emotional state" refers to an operation of changing the information to be presented or the method of guidance in consideration of the user's psychological state.

[1352] The present invention is a system for supporting safe evacuation in the event of a disaster, collecting and analyzing information in real time and providing an optimal evacuation route taking into account the emotional state of the user. This system is composed of multiple servers and user terminals. Detailed embodiments of this system are described below.

[1353] Collection and analysis of live camera footage

[1354] Server: Images are acquired in real time from multiple live cameras installed in the surrounding area. The image data is first streamed to the server, where it is pre-processed using OpenCV. Noise removal and brightness adjustment are applied.

[1355] Server: The preprocessed video data is passed through image recognition algorithms such as TensorFlow and PyTorch to analyze whether roads are passable and whether there are any obstacles.

[1356] Examples:

[1357] The server acquires images from cameras in urban areas, preprocesses them using OpenCV, and then uses a TensorFlow model to analyze road congestion and flooding conditions.

[1358] Hazard map data integration and analysis

[1359] Server: The latest hazard map data provided by government agencies and public institutions is periodically obtained via API or FTP. The obtained data is stored in a GIS database such as PostGIS and updated in real time.

[1360] Server: This data is used to identify the extent of the disaster impact and risk areas and highlight them on a map.

[1361] Examples:

[1362] The server retrieves the latest flood hazard data from the Ministry of Land, Infrastructure, Transport and Tourism's API, imports it into a PostGIS database, and then identifies the flood-affected area and highlights it in red on the map.

[1363] Emotion Engine Operation

[1364] User: Enter voice messages, facial expressions, and text from a smartphone or tablet.

[1365] Terminal: Voice and facial expression data are collected on the user's terminal and sent to the server.

[1366] Server: Analyzes the received data using IBM Watson or Google Cloud Natural Language API to identify the user's emotional state.

[1367] Examples:

[1368] The user speaks to their smartphone, saying, "I'm very anxious." The device records the voice data and sends it to the server, which then analyzes the voice data using IBM Watson to evaluate the user's anxiety level.

[1369] Calculating the optimal evacuation route

[1370] Server: Calculates evacuation routes by integrating live camera analysis results, hazard map data, and the user's emotional state. Uses Google Maps API to obtain local geographic information and Algorithm A to calculate optimal evacuation routes.

[1371] Server: Generates evacuation route information based on the calculation results, including detailed instructions according to the user's emotional state.

[1372] Examples:

[1373] The server applies algorithm A based on the user's current location to calculate the optimal evacuation route, for example, generating a route that avoids areas impassable due to flooding and includes detailed instructions such as "turn right at the next intersection, then go straight."

[1374] Providing an emotion-based user interface

[1375] Server: Considers the user's emotional state and customizes the evacuation route guidance method.

[1376] Server: Provides detailed step-by-step guides if the user is unsure, or generates simplified instructions if simple instructions are appropriate.

[1377] Examples:

[1378] The server provides detailed "audio directions" and "step-by-step visual directions" in a map application based on the user's emotional state.

[1379] Route information provided by map applications

[1380] User: When evacuation is necessary, the user launches the map application on their smartphone or tablet and enters their current location and evacuation destination.

[1381] Terminal: Sends user requests to the server and receives optimal evacuation route information.

[1382] Terminal: The received evacuation route information is visually displayed on a map application.

[1383] Examples:

[1384] The user opens a map application and inputs their current location and evacuation destination. The device sends a request to the server and receives information on the optimal evacuation route. The route is displayed on a map so the user can follow it safely.

[1385] Prompt Sentence Examples

[1386] Below are some example prompts to input to the generative AI model:

[1387] "When a user needs to evacuate due to heavy rain, the system sends a notification such as, 'Please enter your current location and evacuation destination.' At that time, the system calculates and displays the optimal evacuation route based on live camera footage and hazard map data. It also takes into account the user's emotional state and provides detailed step-by-step instructions if necessary."

[1388] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1389] Step 1:

[1390] Collection of live camera footage

[1391] Server: Acquires real-time images from multiple live cameras installed in the surrounding area.

[1392] Input: Real-time video sent from a live camera.

[1393] Output: Video data streamed to the server.

[1394] Specific operation: The server streams video from multiple live cameras in urban areas and acquires it as data.

[1395] Step 2:

[1396] Video data preprocessing

[1397] Server: Preprocesses the acquired video data. Uses OpenCV to remove noise and adjust brightness.

[1398] Input: The video data streamed in step 1.

[1399] Output: Pre-processed and clean video data.

[1400] Specific operation: The server processes the video data using OpenCV, performing noise removal and brightness adjustment.

[1401] Step 3:

[1402] Video data analysis

[1403] Server: The preprocessed video data is run through an image recognition algorithm (TensorFlow or PyTorch) to analyze whether the road is passable and whether there are any obstacles.

[1404] Input: Preprocessed video data.

[1405] Output: Analysis results include data on road traversability and the presence of obstacles.

[1406] Specific operation: The server uses TensorFlow to analyze the level of road congestion and the presence of obstacles from video data.

[1407] Step 4:

[1408] Hazard map data acquisition and integration

[1409] Server: The latest hazard map data is periodically obtained from government agencies and public institutions via API and FTP.

[1410] Input: Hazard map data provided by government agencies and public institutions.

[1411] Output: Hazard map data stored in a GIS database such as PostGIS.

[1412] Specific operation: The server retrieves the latest flood hazard data from the Ministry of Land, Infrastructure, Transport and Tourism's API and imports it into a PostGIS database.

[1413] Step 5:

[1414] Analysis of hazard map data

[1415] Server: Analyzes the acquired hazard map data and identifies the extent of the disaster's impact and dangerous areas.

[1416] Input: Hazard map data stored in a PostGIS database.

[1417] Output: Data on the extent of the disaster impact and risk areas.

[1418] Specific operation: The server analyzes hazard map data and highlights the affected areas and dangerous areas of floods and other disasters on the map.

[1419] Step 6:

[1420] Collecting Emotional Data

[1421] User: Enter voice messages, facial expressions, and text from a smartphone or tablet.

[1422] Input: Audio, image, and text input data.

[1423] Output: Emotion data recorded on a smartphone or tablet.

[1424] Specific operation: The user speaks to the smartphone, "I'm very anxious." The device records the voice data.

[1425] Step 7:

[1426] Sending and analyzing emotional data

[1427] Terminal: Sends emotion data to the server.

[1428] Server: Analyzes the received data using IBM Watson or Google Cloud Natural Language API to identify the user's emotional state.

[1429] Input: Emotion data sent from the user device.

[1430] Output: User's emotional state (e.g., anxiety, stress).

[1431] Specific operation: The device sends the recorded voice data to the server, which then analyzes it using IBM Watson.

[1432] Step 8:

[1433] Calculating the optimal evacuation route

[1434] Server: Calculates the optimal evacuation route by integrating live camera analysis results, hazard map data, and the user's emotional state. Uses the Google Maps API to obtain local geographic information and algorithm A.

[1435] Input: Analysis results (live camera data, hazard map data, emotional state).

[1436] Output: The calculated optimal evacuation route.

[1437] Specific operation: The server executes algorithm A based on the user's current location and calculates a safe and efficient evacuation route.

[1438] Step 9:

[1439] Coordination of evacuation route guidance

[1440] Server: Customize evacuation route guidance based on the user's emotional state. Provide detailed guidance if anxiety is high, and simple guidance if anxiety is low.

[1441] Input: The user's emotional state.

[1442] Output: Customized evacuation route guidance.

[1443] Specific behavior: If the server is concerned about the user, it generates a detailed step-by-step guide.

[1444] Step 10:

[1445] Providing evacuation route information

[1446] User: When evacuation is necessary, the user launches the map application on their smartphone or tablet and enters their current location and evacuation destination.

[1447] Terminal: Sends user input information to the server as a request and receives optimal evacuation route information.

[1448] Server: Generates optimal evacuation route information and sends it back to the terminal.

[1449] Input: User's current location and evacuation destination.

[1450] Output: Optimal evacuation route displayed on the user's terminal.

[1451] Specific operation: The user inputs the evacuation destination using a map application, and the server sends the calculation results to the device, which displays them on a map.

[1452] (Application example 2)

[1453] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1454] Safe evacuation during disasters requires real-time information collection and analysis, and complex data integration is required to provide appropriate evacuation routes. Psychological factors also need to be taken into account if evacuees are feeling stressed or anxious. Previous systems have not adequately considered emotional states when proposing evacuation routes, which has led to issues with ensuring the safety and security of evacuees.

[1455] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1456] In this invention, the server includes means for acquiring real-time images from a live camera and performing preprocessing and analysis, means for acquiring hazard map data and performing integration and analysis, means for calculating an optimal evacuation route based on the acquired and analyzed information, means for reflecting the calculated evacuation route in a map application and displaying it to the user, means for collecting the user's voice data and camera images and analyzing their emotional state, and means for adjusting the user interface according to their emotional state, thereby making it possible to provide an optimal evacuation route that takes the user's emotional state into consideration.

[1457] A "live camera" is a camera that captures images in real time and monitors the surrounding situation.

[1458] "Real-time video" is video that records the current situation instantly and is transmitted with almost no delay.

[1459] "Preprocessing" refers to the initial data processing step of organizing and converting collected raw data into a form that is easier to analyze.

[1460] "Hazard map data" is map data that shows dangerous areas due to natural disasters, accidents, etc.

[1461] "Integration" is the process of bringing together multiple pieces of data into one system and correlating them.

[1462] The "optimal evacuation route" is a route that avoids danger and allows you to reach your destination safely and quickly.

[1463] A "map application" is software that displays digital maps and provides location information and route guidance.

[1464] "Voice data" refers to data that records a user's voice and saves it in an analyzable format.

[1465] "Camera video" refers to video data captured by a camera.

[1466] "Emotional state" is an indicator of the user's psychological state, including anxiety and stress.

[1467] "Analysis" is a method for examining data and extracting information.

[1468] "Adjusting the user interface" means changing the operation screen and guidance method according to the user's situation and emotions.

[1469] This invention is a system that supports safe evacuation behavior in the event of a disaster. This system acquires and analyzes live camera images, integrates and analyzes the latest hazard map data, analyzes the user's emotional state, and calculates and presents the optimal evacuation route. Each processing step is described in detail below.

[1470] Collection and analysis of live camera footage

[1471] The server receives real-time video from multiple live cameras, and the video data is pre-processed and analyzed using image recognition algorithms and machine learning models to determine whether roads are passable and whether there are any obstacles.

[1472] Hazard map data integration and analysis

[1473] The server retrieves the latest hazard map data provided by government agencies and public institutions via API and FTP. The retrieved data is integrated into a GIS database and analyzed to identify the extent of disaster impact and risk areas.

[1474] Emotion Engine Operation

[1475] The server collects the user's voice data and camera footage and analyzes them with an emotion engine. Using techniques such as voice analysis, facial expression analysis, and text analysis, the server determines the user's emotional state, assessing whether the user is feeling stressed or anxious.

[1476] Calculating the optimal evacuation route

[1477] The server integrates live camera analysis results, hazard map data, and the user's emotional state to calculate the optimal evacuation route. Safe and efficient routes are derived using Dijkstra's algorithm and A algorithm. It is also possible to include detailed guidance steps according to the user's emotional state.

[1478] Emotion-based user interface adjustment

[1479] The server then adjusts the evacuation route guidance based on the emotion engine's analysis: if the user is feeling stressed, it provides more detailed, step-by-step instructions, and if the user is feeling less anxious, it provides simpler instructions.

[1480] Route information provided by map applications

[1481] Users launch a map application on their smartphone or tablet and input their current location and evacuation destination. The device then sends a request to the server and receives information on the optimal evacuation route. The server returns the optimal route information calculated in real time, and the device displays this information in detail on the map application. Users can check this information to evacuate safely.

[1482] Specific examples

[1483] In the event of a disaster, if a user is evacuating in their vehicle, the vehicle's infotainment system will use this system to provide the optimal evacuation route in real time. The guidance method will automatically adjust according to the user's emotional state, reducing the user's anxiety.

[1484] Prompt Sentence Examples

[1485] Visual Recognition Prompt: "Analyze real-time video from the camera to detect road conditions and obstacles."

[1486] Hazard Map Integration Prompt: "Get the latest hazard map data and identify the extent of the impact of a disaster."

[1487] Emotion Analysis Prompt: "Analyze the user's voice and facial expressions to determine their emotional state."

[1488] Route Optimization Prompt: "Calculate the optimal evacuation route, taking into account real-time data and emotional state."

[1489] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1490] Step 1:

[1491] The server acquires real-time video from the live camera. The acquired video data is first pre-processed to remove noise and adjust the resolution. This pre-processed data is then analyzed using image recognition algorithms and machine learning models. As a result of the analysis, it determines whether the road is passable and whether there are any obstacles, and outputs this as numerical data.

[1492] Step 2:

[1493] The server retrieves the latest hazard map data provided by government agencies and public institutions via API and FTP. The retrieved data is integrated into a GIS database and analyzed to identify the extent of disaster impact and risk areas. This generates and outputs coordinate data for risk areas.

[1494] Step 3:

[1495] The server collects voice data and camera footage from the user's smartphone and vehicle infotainment system. This data is then analyzed using an emotion engine to perform voice analysis and facial expression analysis. The analysis results are output as numerical data representing the user's stress and anxiety levels.

[1496] Step 4:

[1497] The server integrates the road condition data obtained in step 1, the hazard map data obtained in step 2, and the user's emotional state data obtained in step 3. It calculates the optimal evacuation route using Dijkstra's algorithm and the A algorithm. As a result of the calculation, it outputs a specific route and detailed guidance steps.

[1498] Step 5:

[1499] The server adjusts the evacuation route guidance method based on the analysis results of the emotion engine. For example, if the user is feeling stressed, it provides detailed step-by-step guidance. If the user is feeling less anxious, it provides simple guidance. This generates and outputs data for a customized guidance method.

[1500] Step 6:

[1501] The device sends a request to the server based on the user's input. The server returns optimal route information calculated in real time to the device. The device displays this received route information in a map application, providing a visual representation to the user. The user can then begin evacuation actions based on the displayed route information.

[1502] Step 7:

[1503] Users can follow the evacuation route displayed on the device's map application and check real-time updates to safely evacuate. The system continues to collect and analyze data while on the move, recalculating and updating evacuation routes as needed.

[1504] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1505] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1506] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1507] [Fourth embodiment]

[1508] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1509] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1510] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1511] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1512] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1513] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1514] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1515] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1516] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1517] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1518] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1519] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1520] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1521] This invention is a system that supports safe evacuation in the event of a disaster, integrating live cameras, hazard maps, and map applications to provide optimal evacuation routes in real time. The specific operation and program processing of this system are described below.

[1522] System Overview

[1523] This system consists of multiple servers and user terminals. The servers have the means to collect and analyze video data from live cameras. They also acquire and analyze hazard map data to assess the impact of disasters. They also have the means to calculate optimal evacuation routes and provide the results to users. Meanwhile, the user terminals use a map application to receive and visually display evacuation route information.

[1524] What the program does

[1525] The program processing of the system will be explained in natural language below.

[1526] Collection and analysis of live camera footage

[1527] Server: Real-time video is acquired from multiple live cameras installed in the surrounding area. This video is first pre-processed and then analyzed using image recognition algorithms and machine learning models to determine whether roads are passable and whether there are any obstacles.

[1528] Hazard map data integration and analysis

[1529] Server: Acquires hazard map data provided by government agencies and public institutions. This data is integrated into a GIS database and subjected to analytical models to identify the extent of disaster impact and risk areas. Different analytical models are applied for different types of disasters (e.g., floods, fires, earthquakes, etc.).

[1530] Calculating the optimal evacuation route

[1531] Server: Integrates the analysis results of live camera footage with hazard map data to calculate the optimal evacuation route based on the departure and destination points specified by the user. Path-finding algorithms such as Dijkstra's algorithm and A algorithm are used. The calculated evacuation route information is immediately saved in a database.

[1532] Route information provided by map applications

[1533] Device: The user launches a map application on their smartphone or tablet and inputs their current location and evacuation destination. The application sends this information to the server, from which it receives information on the optimal evacuation route.

[1534] Server: Receives user requests and returns optimal evacuation route information in real time.

[1535] Device: The received evacuation route information is displayed on a map application, allowing the user to check the information and evacuate safely.

[1536] Specific examples

[1537] Below are some examples of specific scenarios that use this system.

[1538] Disaster scenario

[1539] User B is a traveler staying in a flood-hit area and is looking for a route to a suitable evacuation site.

[1540] 1. Collection and analysis of live camera footage

[1541] Server: Acquires and analyzes real-time video from multiple live cameras. From the acquired video data, it is possible to determine whether major roads are flooded.

[1542] 2. Integration and analysis of hazard map data

[1543] Server: Obtains the latest hazard map data and identifies the area affected by the flood. Based on this data, the server understands the status of the disaster affecting User B's location.

[1544] 3. Calculating the optimal evacuation route

[1545] Server: Integrates the analysis results of live camera footage with hazard map data to calculate the optimal route from User B's current location to the nearest evacuation site. Using Dijkstra's algorithm, the safest and most efficient route is derived and the information is stored in a database.

[1546] 4. Route information provided by map applications

[1547] Device: User B launches a map application and inputs their current location and evacuation destination. The application sends a request to the server and receives information on the optimal evacuation route.

[1548] Server: Receives User B's request and returns the optimal route information calculated in real time.

[1549] Device: The map application displays the received information, and User B can visually check the route.

[1550] 5. Supporting users in evacuation

[1551] User: User B follows the instructions of the application and safely travels to the evacuation site via the designated route.

[1552] In this way, the present invention allows users to obtain appropriate evacuation routes based on real-time information when a disaster occurs, enabling safe and rapid evacuation.

[1553] The processing flow will be explained below.

[1554] Step 1: Acquire live camera footage

[1555] Server: Accesses live cameras installed in the specified area and acquires video in real time. Collects data periodically using the endpoint URL and authentication information.

[1556] Step 2: Preprocessing the video data

[1557] Server: Preprocesses the captured live camera footage. Specifically, it resizes the footage, removes noise, corrects color, etc., and converts it into a state suitable for analysis.

[1558] Step 3: Analyzing the video data

[1559] Server: Analyzes the pre-processed video data using image recognition algorithms and machine learning models, extracting information such as road passability, obstacles, and flood damage.

[1560] Step 4: Save the analysis results

[1561] Server: Stores the analysis results in a database, including location information, traffic conditions, and details of detected obstacles.

[1562] Step 5: Obtaining hazard map data

[1563] Server: Obtains the latest hazard map data provided by government agencies and public institutions using API and FTP.

[1564] Step 6: Preprocessing and integration of hazard map data

[1565] Server: Performs preprocessing to integrate acquired hazard map data into the GIS database, including data format conversion, spatial correction, and filtering.

[1566] Step 7: Analyze hazard map data

[1567] Server: Analyzes the integrated hazard map data to identify the extent of disaster impacts and risk areas. Different analytical models are used for each disaster, such as floods, fires, and earthquakes.

[1568] Step 8: Calculate the optimal evacuation route

[1569] Server: Based on the results of live camera analysis and hazard map data, the server calculates the optimal evacuation route for the departure and destination points specified by the user. It uses Dijkstra's algorithm and A algorithm to derive a safe and efficient route.

[1570] Step 9: Save the evacuation route data

[1571] Server: The calculated optimal evacuation route information is stored in a database so that subsequent client requests can be handled promptly.

[1572] Step 10: Accepting a user request

[1573] Device: The user launches a map application and inputs their current location and evacuation destination. This information is sent as a request to the server.

[1574] Step 11: Send evacuation route information

[1575] Server: Receives user requests, generates optimal evacuation route information based on the latest analysis data and route calculation results, and returns the generated information to the device.

[1576] Step 12: Display evacuation route information

[1577] Device: The received evacuation route information is displayed in a map application, showing the route and important points (obstacles, evacuation points, etc.) to make it easier for the user to understand visually.

[1578] Step 13: Start evacuation and check the situation

[1579] User: While looking at the map application, the user begins evacuation by following the optimal route displayed. While on the move, the user can check the latest analysis data through the application to ensure safety.

[1580] Step 14: Notification of evacuation completion

[1581] User: Upon reaching the evacuation location, the application will notify the user that evacuation is complete. If necessary, the user will be able to check the recalculated route.

[1582] This series of processes allows users to evacuate quickly and safely based on information updated in real time.

[1583] Example 1

[1584] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1585] When a disaster occurs, it is important to provide safe and effective evacuation routes in real time. However, conventional evacuation support systems have difficulty collecting real-time information and calculating optimal evacuation routes based on that information. Furthermore, they do not perform sufficient analysis according to different disaster types, making it impossible to quickly provide optimal evacuation routes for users. This leads to evacuation delays and the selection of inappropriate evacuation routes, which can compromise user safety.

[1586] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1587] In this invention, the server includes means for acquiring real-time video from a live camera and analyzing it using preprocessing and image recognition algorithms and machine learning models, means for acquiring hazard map data from public institutions, integrating it into a GIS database, and analyzing it using different analysis models depending on the disaster, means for calculating the optimal evacuation route from the departure point and destination using a route search algorithm based on the acquired and analyzed video data and hazard map data, and means for saving the calculated evacuation route in the database and reflecting it in a map application and displaying it to the user upon user request. This makes it possible to quickly provide the optimal and safest evacuation route based on information collected in real time when a disaster occurs.

[1588] A "live camera" is an image capturing device installed to capture real-time images of the surrounding area.

[1589] "Preprocessing" refers to the process of performing processes such as noise removal and frame resizing to make the acquired video data easier to analyze.

[1590] An "image recognition algorithm" is a computational method for detecting specific objects or patterns within video data.

[1591] A "machine learning model" is a collection of algorithms that automatically learn from data and make predictions and classifications.

[1592] "Hazard map data" is geographic information that indicates the risk and impact of disasters.

[1593] "Public institutions" are public data providers such as government agencies and local governments.

[1594] A "GIS database" is a database system that manages geographic information in an integrated manner.

[1595] A "route search algorithm" is a computational method for finding the optimal route between a specified starting point and destination.

[1596] A "database" is a system for systematically storing information and efficiently retrieving it when needed.

[1597] A "user terminal" is an electronic device, such as a smartphone or tablet, that runs a map application.

[1598] A "map application" is software that displays a user's current location, destination, and route information.

[1599] This invention is a system for supporting safe evacuation in the event of a disaster, integrating live cameras, hazard maps, and map applications to provide optimal evacuation routes in real time. This system is composed of multiple servers and user terminals, and a specific embodiment is shown below.

[1600] System Overview

[1601] The system mainly uses a server, live cameras, user devices, map applications, and hazard map data. Below, we will explain the details of each component and how they work.

[1602] Collection and analysis of live camera footage

[1603] Server: The server acquires real-time video from multiple live cameras installed in the surrounding area. The video is transferred to the server at a fixed frame rate and received using a streaming protocol (e.g., RTSP). The video data is pre-processed and analyzed using a machine learning model (e.g., TensorFlow model).

[1604] Specifically, the server connects to each live camera and captures video data in real time. This data undergoes preprocessing such as noise removal and frame resizing, and is then analyzed using image recognition algorithms and machine learning models. This allows the system to determine whether the road is passable and whether there are any obstacles.

[1605] Acquisition and analysis of hazard map data

[1606] Server: The server periodically retrieves hazard map data provided by government agencies and public institutions. This data is retrieved through APIs and stored in a GIS database. The retrieved data is analyzed using analytical models (e.g., ArcGIS) to identify the extent of disaster impact and risk areas. In this case, analytical models appropriate for different disasters, such as floods, fires, and earthquakes, are applied.

[1607] Calculating the optimal evacuation route

[1608] Server: The server integrates the analysis results of live camera footage with hazard map data and calculates the optimal evacuation route based on the departure and destination points specified by the user. A path-finding algorithm (e.g., Dijkstra algorithm, A algorithm) is used, and the calculated evacuation route information is stored in a database.

[1609] Route information provided by map applications

[1610] Device: The user launches a map application (e.g., Google Maps, Apple Maps) on their smartphone or tablet and enters their current location and evacuation destination. The application sends this information to the server, which then receives information on the optimal evacuation route. The received evacuation route information is displayed on the map application, allowing the user to check it and safely evacuate.

[1611] Specific examples

[1612] Disaster scenario

[1613] User B is staying in a flood-hit area and is searching for a suitable evacuation route. The specific steps are as follows:

[1614] 1. Server: Acquires real-time video from live cameras, performs pre-processing and analysis, and checks whether major roads are submerged from the acquired video data.

[1615] 2. Server: Obtains the latest hazard map data and identifies the extent of flood impact.

[1616] 3. Server: Integrates live camera footage and hazard map data, and calculates the optimal route from User B's current location to the nearest evacuation site using Dijkstra's algorithm.

[1617] 4. Device: User B launches the map application and inputs their current location and evacuation destination. The application sends a request to the server and receives information on the optimal evacuation route.

[1618] 5. Server: Returns the optimal route information calculated in real time to the user device.

[1619] 6. Device: The map application displays the received information, and User B visually checks the route and proceeds with the evacuation safely.

[1620] Examples of prompt statements

[1621] Below is an example of a prompt sentence to input to the generative AI model.

[1622] Prompt: Explain how a real-time evacuation assistance system works in the event of a disaster.

[1623] Collecting and analyzing live camera footage

[1624] Integrating and analyzing hazard map data obtained from government agencies

[1625] Calculate the optimal evacuation route based on your current location and evacuation destination

[1626] Providing route information to users through a map application

[1627] In this way, the present invention can quickly provide optimal and safe evacuation routes based on information collected in real time when a disaster occurs.

[1628] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1629] Step 1:

[1630] Collection of live camera footage

[1631] Server: Obtains real-time video data from live cameras. The input is the IP address of each live camera, and the output is raw video data. The server connects to the live cameras using the RTSP streaming protocol and stores the video data in a buffer at a specific frame rate.

[1632] Specific behavior:

[1633] 1. The server connects to the IP address of each live camera and starts the RTSP stream.

[1634] 2. Video data is stored in a buffer in real time.

[1635] Step 2:

[1636] Video data preprocessing and analysis

[1637] Server: Preprocesses the acquired video data and analyzes it using image recognition algorithms and machine learning models. The input is raw video data, and the output is analysis results indicating passability and the presence or absence of obstacles. Preprocessing involves noise removal and frame resizing.

[1638] Specific behavior:

[1639] 1. Noise is removed from the video data.

[1640] 2. Resize the frame to a size that is easy to analyze.

[1641] 3. Use a machine learning model (e.g., a TensorFlow model) to analyze each frame and identify obstacles and impassable areas.

[1642] Step 3:

[1643] Obtaining hazard map data

[1644] Server: Obtains hazard map data provided by public institutions. The input is the API endpoint, and the output is raw hazard map data. The server periodically issues API requests to obtain the latest data.

[1645] Specific behavior:

[1646] 1. The server issues an API request to obtain the latest hazard map data.

[1647] 2. The acquired data is stored in a GIS database.

[1648] Step 4:

[1649] Analysis of hazard map data

[1650] Server: Stores the acquired hazard map data in a GIS database and performs analysis to identify the extent of disaster impact and risk areas. The input is raw hazard map data, and the output is the analysis results.

[1651] Specific behavior:

[1652] 1. Hazard map data stored in a GIS database is input into the analytical model.

[1653] 2. Apply different analytical models to various disasters (e.g., flood, fire, earthquake) to identify risk areas.

[1654] Step 5:

[1655] Calculating the optimal evacuation route

[1656] Server: Integrates the analysis results of live camera footage and hazard map data, and calculates the optimal evacuation route based on the departure and destination points specified by the user. The input is the analysis results and departure and destination information, and the output is the optimal evacuation route.

[1657] Specific behavior:

[1658] 1. Based on composite data (camera image analysis results and hazard map data), information on the departure and destination points is combined.

[1659] 2. Calculate the optimal evacuation route using a pathfinding algorithm (e.g., Dijkstra's algorithm, A algorithm).

[1660] 3. Save the calculation results in the database.

[1661] Step 6:

[1662] Processing user requests

[1663] Device: The user launches a map application on their smartphone or tablet and inputs their current location and evacuation destination. The input is the user's current location and evacuation destination information, and the output is a request sent to the server.

[1664] Specific behavior:

[1665] 1. Provide an interface for inputting current location and evacuation destination from a map application.

[1666] 2. The entered data is sent to the server as an HTTP request.

[1667] Step 7:

[1668] Providing evacuation route information

[1669] Server: Receives user requests and returns optimal evacuation route information calculated in real time. The output is optimal evacuation route information.

[1670] Terminal: Displays the received evacuation route information in a map application. The input is the optimal evacuation route information sent from the server, and the output is a visual display for the user.

[1671] Specific behavior (server side):

[1672] 1. Receives a request and retrieves the optimal route information from the database.

[1673] 2. The acquired route information is returned to the user's device in JSON format.

[1674] Specific operations (terminal side):

[1675] 1. The route information returned from the server is displayed in the map application.

[1676] 2. The user proceeds with evacuation while referring to the map application.

[1677] (Application example 1)

[1678] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1679] Currently, many regions require systems to ensure safe evacuation in the event of a disaster. However, there are still limited systems that can respond to changing situations in real time and quickly provide optimal evacuation routes. This makes it difficult for autonomous vehicles to operate appropriately and safely, especially when transportation infrastructure is disrupted. Furthermore, there are no systems that can integrate information from different sources, such as live cameras and hazard maps, and provide optimal evacuation information in real time. For this reason, there is a need to develop a system that can reliably guide autonomous vehicles along safe routes and provide visual evacuation information to passengers in the event of a disaster.

[1680] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1681] In this invention, the server includes means for acquiring real-time images from live cameras and performing preprocessing and analysis, means for acquiring hazard map data and performing integration and analysis, means for calculating optimal evacuation routes based on the acquired and analyzed information, means for reflecting the calculated evacuation routes in the control system of the autonomous vehicle and updating the driving route in real time, and means for displaying evacuation route information on an in-vehicle display to provide it visually to the user. This makes it possible to provide optimal evacuation routes to autonomous vehicles in real time and achieve safe and efficient driving in the event of a disaster.

[1682] A "live camera" is a camera device that captures images in real time and transmits the data.

[1683] "Real-time video" refers to video data that shows the current situation in real time.

[1684] "Preprocessing" refers to the initial stage of processing performed on video data acquired by a live camera, and includes processes such as noise removal and image standardization.

[1685] "Analysis" is the process of extracting necessary information and making decisions based on preprocessed data.

[1686] "Hazard map data" is map information showing disasters and dangerous areas, and is data provided by government agencies and public institutions.

[1687] "Integration" is the process of bringing together multiple different data sources into one system.

[1688] An "optimal evacuation route" is the route that will allow you to reach your evacuation destination most safely and efficiently in the event of a disaster.

[1689] "Calculation" is the process of deriving a specific result using an algorithm based on given data.

[1690] An "autonomous vehicle" is a vehicle that has the ability to operate automatically without the need for a driver.

[1691] "Control system" means an electronic or mechanical system for managing and directing the operation of a vehicle.

[1692] The "travel route" is the route that a vehicle takes to reach its destination.

[1693] An "in-vehicle display" is a screen installed inside a vehicle to display information.

[1694] "User" means a person who uses the system or vehicle.

[1695] "Visually presented" means that the information is presented in a visual form.

[1696] "Real-time updates" refers to revising the system information as needed in response to changes in background data and conditions.

[1697] MODE FOR CARRYING OUT THE INVENTION

[1698] An example of this application is a system that enables autonomous vehicles to safely evacuate in the event of a disaster, integrating live cameras, hazard maps, and real-time calculation of optimal evacuation routes.

[1699] Overall structure

[1700] This system consists of multiple servers and user terminals. The servers have the means to collect and analyze video data from live cameras. They also acquire and analyze hazard map data to assess the impact of disasters. They also have the means to calculate optimal evacuation routes and provide the results to the autonomous vehicle. Meanwhile, the user terminals receive the evacuation route information and visually display it on an in-vehicle display.

[1701] Hardware and software used

[1702] Hardware:

[1703] Live cameras: Multiple cameras installed on a vehicle.

[1704] Server: A computing device used to collect, process, and store data.

[1705] In-vehicle display: A screen installed inside the vehicle to display information.

[1706] software:

[1707] OpenCV: Used to collect and preprocess camera footage.

[1708] Machine learning models (e.g. TensorFlow, PyTorch): Road condition analysis.

[1709] requests library: A library for sending HTTP requests and retrieving hazard map data.

[1710] Pathfinding algorithms (e.g., Dijkstra, A): Used to calculate optimal evacuation routes.

[1711] Program processing description

[1712] The server acquires real-time video footage from live cameras and performs preprocessing using OpenCV. The acquired video footage is analyzed using a machine learning model to determine whether roads are passable and whether there are any obstacles. Hazard map data is then acquired from government agencies and public institutions and integrated into a GIS database. Based on this data, the server calculates the optimal evacuation route. This calculation uses route search algorithms such as Dijkstra's algorithm and A algorithm. The calculated evacuation route information is reflected in the autonomous vehicle's control system, and the driving route is updated in real time.

[1713] The terminals then display the received evacuation route information on the in-car display, providing a visual representation to passengers, allowing users to evacuate safely while checking the information in real time.

[1714] Specific examples

[1715] For example, the process for updating route information when a disaster occurs is as follows:

[1716] 1. Collection and analysis of live camera footage

[1717] The server acquires real-time images from live cameras installed in vehicles, processes them, and analyzes them. For example, it uses OpenCV to remove noise and uses models using TensorFlow and PyTorch to determine whether a road is passable.

[1718] 2. Integration and analysis of hazard map data

[1719] The server uses the requests library to retrieve the latest hazard map data from government agencies and integrate it into a GIS database. This data is then processed with different analytical models (e.g., flood, fire).

[1720] 3. Calculating the optimal evacuation route

[1721] Based on the acquired and analyzed data, the server uses Dijkstra's algorithm and A algorithm to calculate the optimal route from the user's current location to the evacuation site.

[1722] 4. Real-time route updates

[1723] The evacuation route calculated by the server is sent to the autonomous vehicle's control system, which uses this information to update its route in real time.

[1724] 5. Providing visual information through in-car displays

[1725] The terminal displays the calculated evacuation route information on the in-vehicle display, allowing users to visually confirm it.

[1726] Prompt Sentence Examples

[1727] The following prompts are used to encourage a generative AI model to perform a specific task.

[1728] text

[1729] To provide safe evacuation routes in the event of a disaster, implement an optimal route calculation system that analyzes live camera footage and integrates it with hazard maps. Specifically, create an application that allows an autonomous vehicle to calculate the optimal evacuation route in real time based on its current location and destination, and display that information on an in-car display.

[1730] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1731] Step 1: Collecting live camera footage

[1732] The server acquires video data in real time from live cameras installed in the vehicle. The input is video data from multiple cameras, and the output is preprocessed video data. Specifically, it uses OpenCV to capture video data and performs preprocessing such as noise removal and frame alignment.

[1733] Step 2: Analyzing the video data

[1734] The server inputs the preprocessed video data into a machine learning model to analyze whether the road is passable and whether there are any obstacles. The input is the preprocessed video data, and the output is information on whether the road is passable or not. Specifically, the server uses a trained model using TensorFlow and PyTorch to determine the road conditions from the video data.

[1735] Step 3: Obtaining hazard map data

[1736] The server uses the requests library to obtain the latest hazard map data from government agencies and public institutions. The input is an HTTP request, and the output is hazard map data in JSON format. Specifically, it downloads the data from the specified URL and converts it into an analyzable format.

[1737] Step 4: Analyze hazard map data

[1738] The server analyzes the acquired hazard map data and identifies dangerous areas and areas of impact. The input is the hazard map data, and the output is the analyzed dangerous area information. Specifically, it applies analytical models corresponding to different types of disasters (e.g., floods, fires, earthquakes) and determines safe routes.

[1739] Step 5: Calculate the optimal evacuation route

[1740] The server integrates the video analysis results with hazard map data to calculate the optimal route from the user's current location to the evacuation destination. The input is road condition data and dangerous area information, and the output is the optimal evacuation route. Specifically, it uses Dijkstra's algorithm and A algorithm to derive the safest route.

[1741] Step 6: Real-time updates of routes

[1742] The server sends the calculated optimal evacuation route to the autonomous vehicle's control system and updates the driving route in real time. The input is the optimal evacuation route information, and the output is the updated driving route. Specifically, the server sends data to the vehicle's control system using a communication protocol.

[1743] Step 7: Display evacuation route information

[1744] The terminal displays the received evacuation route information on the in-car display, providing a visual presentation to passengers. The input is the optimal evacuation route information, and the output is the visual information on the display. Specifically, the information is displayed using a GUI (Graphical User Interface).

[1745] Step 8: Support users in evacuation

[1746] The user follows the route information displayed on the screen and safely travels along the designated route to the evacuation site. The input is the visually provided route information, and the output is the user's evacuation behavior. Specific actions include selecting a safe route based on the application's instructions and traveling by manual or automatic driving.

[1747] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1748] This invention is a system that supports safe evacuation in the event of a disaster, and provides an optimal evacuation route that takes into account the user's emotional state in addition to collecting and analyzing information in real time. The specific operation and program processing of this system are described below.

[1749] System Overview

[1750] This system consists of multiple servers and user terminals. The servers have the means to collect and analyze video data from live cameras. They also acquire hazard map data and assess the impact of disasters. They also have an emotion engine that recognizes the user's emotional state and calculates the optimal evacuation route. The user terminals receive evacuation route information using a map application and visually display it.

[1751] What the program does

[1752] The program processing of the system will be explained in natural language below.

[1753] Collection and analysis of live camera footage

[1754] Server: Acquires real-time video feeds from multiple live cameras installed in the surrounding area. These feeds are pre-processed and analyzed using image recognition algorithms and machine learning models. The analysis results include information on road availability and the presence of obstacles.

[1755] Hazard map data integration and analysis

[1756] Server: The latest hazard map data provided by government agencies and public institutions is obtained via API and FTP. The obtained data is integrated into a GIS database and subjected to analytical models to identify the extent of disaster impact and risk areas.

[1757] Emotion Engine Operation

[1758] Server: Collects voice data, camera footage, and input text data from the user's device and analyzes them with an emotion engine. The server uses techniques such as voice analysis, facial expression analysis, and text analysis to determine the user's emotional state.

[1759] Calculating the optimal evacuation route

[1760] Server: Integrates live camera analysis results, hazard map data, and the user's emotional state to calculate the optimal evacuation route. Utilizing Dijkstra's algorithm and A algorithm, it derives a safe and efficient route.

[1761] Emotion-based user interface adjustment

[1762] Server: Based on the analysis results of the emotion engine, the server adjusts the evacuation route guidance method. For example, if the user is feeling stressed, it provides detailed step-by-step instructions. Conversely, if the user is feeling less anxious, it provides simpler instructions.

[1763] Route information provided by map applications

[1764] Device: The user launches a map application on their smartphone or tablet and inputs their current location and evacuation destination. The application then sends a request to the server, which then receives information on the optimal evacuation route.

[1765] Server: Receives user requests, generates optimal evacuation route information based on the latest analysis data and route calculation results, and returns it to the terminal.

[1766] Device: The received evacuation route information is displayed on a map application, allowing the user to check the information and evacuate safely.

[1767] Specific examples

[1768] Below are some examples of specific scenarios that utilize this system.

[1769] Disaster scenario

[1770] User C is a traveler staying in a flood-hit area and is looking for a route to an appropriate evacuation site. User C feels anxious and needs detailed evacuation instructions.

[1771] 1. Collection and analysis of live camera footage

[1772] Server: Acquires and analyzes real-time video from multiple live cameras. From the acquired video data, it is possible to determine whether major roads are flooded.

[1773] 2. Integration and analysis of hazard map data

[1774] Server: Obtains the latest hazard map data and identifies the area affected by the flood. Based on this data, the server understands the status of the disaster affecting User C's location.

[1775] 3. Operation of the Emotion Engine

[1776] Server: Analyzes audio data and camera footage collected from User C's smartphone to assess his / her anxiety and stress levels. It turns out that User C is experiencing high levels of anxiety.

[1777] 4. Calculating the optimal evacuation route

[1778] Server: Calculates the optimal evacuation route by taking into account the live camera analysis results, hazard map data, and the emotional state of User C. Generates route information including detailed guidance steps.

[1779] 5. Route information provided by map applications

[1780] Device: User C launches a map application and inputs their current location and evacuation destination. The application sends a request to the server and receives information on the optimal evacuation route.

[1781] Server: Receives user C's request and returns the optimal route information calculated in real time.

[1782] Device: The map application displays the received information in detail and guides User C to evacuate without worry.

[1783] 6. Supporting users in evacuation

[1784] User C follows the instructions of the application and begins evacuation along the optimal route displayed. While on the move, he checks the latest analysis data to ensure his safety.

[1785] In this way, the system of the present invention can support safer and more secure evacuation behavior by taking into account the user's emotional state. Users can evacuate safely based on information updated in real time.

[1786] The processing flow will be explained below.

[1787] Step 1: Acquire live camera footage

[1788] Server: Accesses live cameras installed in the specified area and acquires video in real time. Collects data periodically using the endpoint URL and authentication information.

[1789] Step 2: Preprocessing the video data

[1790] Server: Preprocesses the captured live camera footage. Specifically, it resizes the footage, removes noise, corrects color, etc., and converts it into a state suitable for analysis.

[1791] Step 3: Analyzing the video data

[1792] Server: Analyzes the pre-processed video data using image recognition algorithms and machine learning models, extracting information such as road passability, obstacles, and flood damage.

[1793] Step 4: Save the analysis results

[1794] Server: Stores the analysis results in a database, including location information, traffic conditions, and details of detected obstacles.

[1795] Step 5: Obtaining hazard map data

[1796] Server: Obtains the latest hazard map data provided by government agencies and public institutions using API and FTP.

[1797] Step 6: Preprocessing and integration of hazard map data

[1798] Server: Performs preprocessing to integrate acquired hazard map data into the GIS database, including data format conversion, spatial correction, and filtering.

[1799] Step 7: Analyze hazard map data

[1800] Server: Analyzes the integrated hazard map data to identify the extent of disaster impacts and risk areas. Different analytical models are used for each disaster, such as floods, fires, and earthquakes.

[1801] Step 8: Obtaining Emotion Data

[1802] Device: The user provides audio data and camera footage via a smartphone or tablet, which then collects data on the user's emotional state.

[1803] Step 9: Analyze the sentiment data

[1804] Server: Analyzes collected voice, facial expression, and text data using an emotion engine to evaluate the user's emotional state (anxiety, stress, calm, etc.).

[1805] Step 10: Storing Emotion Data

[1806] Server: The analysis results are saved in a database. This data is used as auxiliary data for later evacuation route calculations.

[1807] Step 11: Calculate the optimal evacuation route

[1808] Server: Integrates live camera analysis results, hazard map data, and the user's emotional state to calculate the optimal evacuation route between the departure and destination points specified by the user. A safe and efficient route is derived using Dijkstra's algorithm and the A algorithm.

[1809] Step 12: Save the evacuation route data

[1810] Server: The calculated optimal evacuation route information is stored in a database so that subsequent client requests can be handled promptly.

[1811] Step 13: Accepting User Requests

[1812] Terminal: The user launches the map application and inputs their current location and evacuation destination. After inputting the information, a request for an evacuation route is sent to the server.

[1813] Step 14: Send evacuation route information

[1814] Server: Receives user requests, generates optimal evacuation route information based on the latest analysis data and route calculation results, and returns the generated information to the device.

[1815] Step 15: Display evacuation route information

[1816] Device: The received evacuation route information is displayed in a map application, showing the route and important points (obstacles, evacuation points, etc.) to make it easier for the user to understand visually.

[1817] Step 16: Start evacuation and check the situation

[1818] User: While looking at the map application, the user begins evacuation by following the optimal route displayed. The user also checks the latest analysis data while on the move to ensure safety.

[1819] Step 17: Guidance adjustment based on emotional state

[1820] Server: Based on the analysis results of the emotion engine, the server adjusts the evacuation route guidance method. For example, if the user is feeling stressed, it provides detailed step-by-step instructions to reassure the user.

[1821] Step 18: Notification of Evacuation Completion

[1822] User: Upon reaching the evacuation location, the application will notify the user that evacuation is complete. If necessary, the user will be able to check the recalculated route.

[1823] This series of processes allows users to obtain an appropriate evacuation route based on real-time updated information and emotional state, allowing them to take evacuation action safely and with peace of mind.

[1824] Example 2

[1825] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1826] In modern society, rapid and safe evacuation in the event of a disaster is a crucial issue. However, conventional evacuation support systems lack real-time information collection and analysis capabilities, making it difficult to provide evacuation support that is tailored to the user's individual emotional state. As a result, it is not possible to provide optimal evacuation routes in situations where users are feeling stressed or anxious, and issues remain regarding the safety and efficiency of evacuation behavior.

[1827] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1828] In this invention, the server includes means for acquiring real-time video from a live camera and performing preprocessing and analysis, means for acquiring hazard map data and performing integration and analysis, means for analyzing audio, image, and text data acquired from a user terminal and determining the user's emotional state, means for adjusting the evacuation route guidance method based on the user's emotional state, means for calculating an optimal evacuation route based on the acquired and analyzed information, and means for reflecting the calculated evacuation route in a map application and displaying it to the user. This makes it possible to provide an optimal evacuation route that takes into account the emotional state of each individual user based on information updated in real time.

[1829] A "live camera" is a camera device that captures video in real time and transmits it as digital data.

[1830] "Preprocessing" refers to processes such as noise removal and brightness adjustment that are performed to make the acquired video data easier to analyze.

[1831] "Analysis" is the process of analyzing acquired data using specific algorithms or models to extract useful information.

[1832] "Hazard map data" is map data that shows the risk of disasters occurring and the extent of their impact in a specific area.

[1833] "Integration" is the process of combining different types of data into a single dataset.

[1834] A "user terminal" is an electronic device such as a computer, smartphone, or tablet that is directly operated by a user.

[1835] "Voice data" refers to data in which the user's voice is recorded in digital format.

[1836] "Image data" refers to digital data of still or moving images captured by a camera or other photographic device.

[1837] "Text data" is data that stores characters or sentences in digital format.

[1838] "Emotional state" refers to the user's psychological state, and includes emotions such as anxiety, stress, and fear.

[1839] An "evacuation route" is a route that a user should follow to evacuate safely.

[1840] A "map application" is software that displays digital maps and provides navigation functions.

[1841] "Display" means visually presenting data or information to a user.

[1842] An "image recognition algorithm" is an algorithm for detecting specific patterns or features from input image data.

[1843] A "machine learning model" is a computational model that learns patterns from large amounts of data and performs predictions and classifications.

[1844] An "analytic model" is a set of rules or algorithms for performing analysis on a particular type of data.

[1845] "Evacuation route calculation" is the process of deriving a safe and efficient route based on acquired data.

[1846] "Adjustment based on the user's emotional state" refers to an operation of changing the information to be presented or the method of guidance in consideration of the user's psychological state.

[1847] The present invention is a system for supporting safe evacuation in the event of a disaster, collecting and analyzing information in real time and providing an optimal evacuation route taking into account the emotional state of the user. This system is composed of multiple servers and user terminals. Detailed embodiments of this system are described below.

[1848] Collection and analysis of live camera footage

[1849] Server: Images are acquired in real time from multiple live cameras installed in the surrounding area. The image data is first streamed to the server, where it is pre-processed using OpenCV. Noise removal and brightness adjustment are applied.

[1850] Server: The preprocessed video data is passed through image recognition algorithms such as TensorFlow and PyTorch to analyze whether roads are passable and whether there are any obstacles.

[1851] Examples:

[1852] The server acquires images from cameras in urban areas, preprocesses them using OpenCV, and then uses a TensorFlow model to analyze road congestion and flooding conditions.

[1853] Hazard map data integration and analysis

[1854] Server: The latest hazard map data provided by government agencies and public institutions is periodically obtained via API or FTP. The obtained data is stored in a GIS database such as PostGIS and updated in real time.

[1855] Server: This data is used to identify the extent of the disaster impact and risk areas and highlight them on a map.

[1856] Examples:

[1857] The server retrieves the latest flood hazard data from the Ministry of Land, Infrastructure, Transport and Tourism's API, imports it into a PostGIS database, and then identifies the flood-affected area and highlights it in red on the map.

[1858] Emotion Engine Operation

[1859] User: Enter voice messages, facial expressions, and text from a smartphone or tablet.

[1860] Terminal: Voice and facial expression data are collected on the user's terminal and sent to the server.

[1861] Server: Analyzes the received data using IBM Watson or Google Cloud Natural Language API to identify the user's emotional state.

[1862] Examples:

[1863] The user speaks to their smartphone, saying, "I'm very anxious." The device records the voice data and sends it to the server, which then analyzes the voice data using IBM Watson to evaluate the user's anxiety level.

[1864] Calculating the optimal evacuation route

[1865] Server: Calculates evacuation routes by integrating live camera analysis results, hazard map data, and the user's emotional state. Uses Google Maps API to obtain local geographic information and Algorithm A to calculate optimal evacuation routes.

[1866] Server: Generates evacuation route information based on the calculation results, including detailed instructions according to the user's emotional state.

[1867] Examples:

[1868] The server applies algorithm A based on the user's current location to calculate the optimal evacuation route, for example, generating a route that avoids areas impassable due to flooding and includes detailed instructions such as "turn right at the next intersection, then go straight."

[1869] Providing an emotion-based user interface

[1870] Server: Considers the user's emotional state and customizes the evacuation route guidance method.

[1871] Server: Provides detailed step-by-step guides if the user is unsure, or generates simplified instructions if simple instructions are appropriate.

[1872] Examples:

[1873] The server provides detailed "audio directions" and "step-by-step visual directions" in a map application based on the user's emotional state.

[1874] Route information provided by map applications

[1875] User: When evacuation is necessary, the user launches the map application on their smartphone or tablet and enters their current location and evacuation destination.

[1876] Terminal: Sends user requests to the server and receives optimal evacuation route information.

[1877] Terminal: The received evacuation route information is visually displayed on a map application.

[1878] Examples:

[1879] The user opens a map application and inputs their current location and evacuation destination. The device sends a request to the server and receives information on the optimal evacuation route. The route is displayed on a map so the user can follow it safely.

[1880] Prompt Sentence Examples

[1881] Below are some example prompts to input to the generative AI model:

[1882] "When a user needs to evacuate due to heavy rain, the system sends a notification such as, 'Please enter your current location and evacuation destination.' At that time, the system calculates and displays the optimal evacuation route based on live camera footage and hazard map data. It also takes into account the user's emotional state and provides detailed step-by-step instructions if necessary."

[1883] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1884] Step 1:

[1885] Collection of live camera footage

[1886] Server: Acquires real-time images from multiple live cameras installed in the surrounding area.

[1887] Input: Real-time video sent from a live camera.

[1888] Output: Video data streamed to the server.

[1889] Specific operation: The server streams video from multiple live cameras in urban areas and acquires it as data.

[1890] Step 2:

[1891] Video data preprocessing

[1892] Server: Preprocesses the acquired video data. Uses OpenCV to remove noise and adjust brightness.

[1893] Input: The video data streamed in step 1.

[1894] Output: Pre-processed and clean video data.

[1895] Specific operation: The server processes the video data using OpenCV, performing noise removal and brightness adjustment.

[1896] Step 3:

[1897] Video data analysis

[1898] Server: The preprocessed video data is run through an image recognition algorithm (TensorFlow or PyTorch) to analyze whether the road is passable and whether there are any obstacles.

[1899] Input: Preprocessed video data.

[1900] Output: Analysis results include data on road traversability and the presence of obstacles.

[1901] Specific operation: The server uses TensorFlow to analyze the level of road congestion and the presence of obstacles from video data.

[1902] Step 4:

[1903] Hazard map data acquisition and integration

[1904] Server: The latest hazard map data is periodically obtained from government agencies and public institutions via API and FTP.

[1905] Input: Hazard map data provided by government agencies and public institutions.

[1906] Output: Hazard map data stored in a GIS database such as PostGIS.

[1907] Specific operation: The server retrieves the latest flood hazard data from the Ministry of Land, Infrastructure, Transport and Tourism's API and imports it into a PostGIS database.

[1908] Step 5:

[1909] Analysis of hazard map data

[1910] Server: Analyzes the acquired hazard map data and identifies the extent of the disaster's impact and dangerous areas.

[1911] Input: Hazard map data stored in a PostGIS database.

[1912] Output: Data on the extent of the disaster impact and risk areas.

[1913] Specific operation: The server analyzes hazard map data and highlights the affected areas and dangerous areas of floods and other disasters on the map.

[1914] Step 6:

[1915] Collecting Emotional Data

[1916] User: Enter voice messages, facial expressions, and text from a smartphone or tablet.

[1917] Input: Audio, image, and text input data.

[1918] Output: Emotion data recorded on a smartphone or tablet.

[1919] Specific operation: The user speaks to the smartphone, "I'm very anxious." The device records the voice data.

[1920] Step 7:

[1921] Sending and analyzing emotional data

[1922] Terminal: Sends emotion data to the server.

[1923] Server: Analyzes the received data using IBM Watson or Google Cloud Natural Language API to identify the user's emotional state.

[1924] Input: Emotion data sent from the user device.

[1925] Output: User's emotional state (e.g., anxiety, stress).

[1926] Specific operation: The device sends the recorded voice data to the server, which then analyzes it using IBM Watson.

[1927] Step 8:

[1928] Calculating the optimal evacuation route

[1929] Server: Calculates the optimal evacuation route by integrating live camera analysis results, hazard map data, and the user's emotional state. Uses the Google Maps API to obtain local geographic information and algorithm A.

[1930] Input: Analysis results (live camera data, hazard map data, emotional state).

[1931] Output: The calculated optimal evacuation route.

[1932] Specific operation: The server executes algorithm A based on the user's current location and calculates a safe and efficient evacuation route.

[1933] Step 9:

[1934] Coordination of evacuation route guidance

[1935] Server: Customize evacuation route guidance based on the user's emotional state. Provide detailed guidance if anxiety is high, and simple guidance if anxiety is low.

[1936] Input: The user's emotional state.

[1937] Output: Customized evacuation route guidance.

[1938] Specific behavior: If the server is concerned about the user, it generates a detailed step-by-step guide.

[1939] Step 10:

[1940] Providing evacuation route information

[1941] User: When evacuation is necessary, the user launches the map application on their smartphone or tablet and enters their current location and evacuation destination.

[1942] Terminal: Sends user input information to the server as a request and receives optimal evacuation route information.

[1943] Server: Generates optimal evacuation route information and sends it back to the terminal.

[1944] Input: User's current location and evacuation destination.

[1945] Output: Optimal evacuation route displayed on the user's terminal.

[1946] Specific operation: The user inputs the evacuation destination using a map application, and the server sends the calculation results to the device, which displays them on a map.

[1947] (Application example 2)

[1948] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1949] Safe evacuation during disasters requires real-time information collection and analysis, and complex data integration is required to provide appropriate evacuation routes. Psychological factors also need to be taken into account if evacuees are feeling stressed or anxious. Previous systems have not adequately considered emotional states when proposing evacuation routes, which has led to issues with ensuring the safety and security of evacuees.

[1950] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1951] In this invention, the server includes means for acquiring real-time images from a live camera and performing preprocessing and analysis, means for acquiring hazard map data and performing integration and analysis, means for calculating an optimal evacuation route based on the acquired and analyzed information, means for reflecting the calculated evacuation route in a map application and displaying it to the user, means for collecting the user's voice data and camera images and analyzing their emotional state, and means for adjusting the user interface according to their emotional state, thereby making it possible to provide an optimal evacuation route that takes the user's emotional state into consideration.

[1952] A "live camera" is a camera that captures images in real time and monitors the surrounding situation.

[1953] "Real-time video" is video that records the current situation instantly and is transmitted with almost no delay.

[1954] "Preprocessing" refers to the initial data processing step of organizing and converting collected raw data into a form that is easier to analyze.

[1955] "Hazard map data" is map data that shows dangerous areas due to natural disasters, accidents, etc.

[1956] "Integration" is the process of bringing together multiple pieces of data into one system and correlating them.

[1957] The "optimal evacuation route" is a route that avoids danger and allows you to reach your destination safely and quickly.

[1958] A "map application" is software that displays digital maps and provides location information and route guidance.

[1959] "Voice data" refers to data that records a user's voice and saves it in an analyzable format.

[1960] "Camera video" refers to video data captured by a camera.

[1961] "Emotional state" is an indicator of the user's psychological state, including anxiety and stress.

[1962] "Analysis" is a method for examining data and extracting information.

[1963] "Adjusting the user interface" means changing the operation screen and guidance method according to the user's situation and emotions.

[1964] This invention is a system that supports safe evacuation behavior in the event of a disaster. This system acquires and analyzes live camera images, integrates and analyzes the latest hazard map data, analyzes the user's emotional state, and calculates and presents the optimal evacuation route. Each processing step is described in detail below.

[1965] Collection and analysis of live camera footage

[1966] The server receives real-time video from multiple live cameras, and the video data is pre-processed and analyzed using image recognition algorithms and machine learning models to determine whether roads are passable and whether there are any obstacles.

[1967] Hazard map data integration and analysis

[1968] The server retrieves the latest hazard map data provided by government agencies and public institutions via API and FTP. The retrieved data is integrated into a GIS database and analyzed to identify the extent of disaster impact and risk areas.

[1969] Emotion Engine Operation

[1970] The server collects the user's voice data and camera footage and analyzes them with an emotion engine. Using techniques such as voice analysis, facial expression analysis, and text analysis, the server determines the user's emotional state, assessing whether the user is feeling stressed or anxious.

[1971] Calculating the optimal evacuation route

[1972] The server integrates live camera analysis results, hazard map data, and the user's emotional state to calculate the optimal evacuation route. Safe and efficient routes are derived using Dijkstra's algorithm and A algorithm. It is also possible to include detailed guidance steps according to the user's emotional state.

[1973] Emotion-based user interface adjustment

[1974] The server then adjusts the evacuation route guidance based on the emotion engine's analysis: if the user is feeling stressed, it provides more detailed, step-by-step instructions, and if the user is feeling less anxious, it provides simpler instructions.

[1975] Route information provided by map applications

[1976] Users launch a map application on their smartphone or tablet and input their current location and evacuation destination. The device then sends a request to the server and receives information on the optimal evacuation route. The server returns the optimal route information calculated in real time, and the device displays this information in detail on the map application. Users can check this information to evacuate safely.

[1977] Specific examples

[1978] In the event of a disaster, if a user is evacuating in their vehicle, the vehicle's infotainment system will use this system to provide the optimal evacuation route in real time. The guidance method will automatically adjust according to the user's emotional state, reducing the user's anxiety.

[1979] Prompt Sentence Examples

[1980] Visual Recognition Prompt: "Analyze real-time video from the camera to detect road conditions and obstacles."

[1981] Hazard Map Integration Prompt: "Get the latest hazard map data and identify the extent of the impact of a disaster."

[1982] Emotion Analysis Prompt: "Analyze the user's voice and facial expressions to determine their emotional state."

[1983] Route Optimization Prompt: "Calculate the optimal evacuation route, taking into account real-time data and emotional state."

[1984] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1985] Step 1:

[1986] The server acquires real-time video from the live camera. The acquired video data is first pre-processed to remove noise and adjust the resolution. This pre-processed data is then analyzed using image recognition algorithms and machine learning models. As a result of the analysis, it determines whether the road is passable and whether there are any obstacles, and outputs this as numerical data.

[1987] Step 2:

[1988] The server retrieves the latest hazard map data provided by government agencies and public institutions via API and FTP. The retrieved data is integrated into a GIS database and analyzed to identify the extent ...

Claims

1. A means for acquiring real-time images from a live camera and performing pre-processing and analysis; A means of acquiring, synthesizing and analyzing hazard map data; A means for calculating an optimal evacuation route based on the acquired and analyzed information; A means for reflecting the calculated evacuation route in a map application and displaying it to the user; A system including:

2. The system of claim 1 , wherein image recognition algorithms and machine learning models are used for pre-processing and analysis of live camera footage.

3. The system according to claim 1, wherein the hazard map data is analyzed using different analytical models depending on the disaster.

Images