system

Abstract

We provide the system. [Solution] Data collection means for collecting environmental data, Information processing means for analyzing collected environmental data, A route calculation means for generating evacuation routes based on analysis results, A data transmission means for distributing the generated evacuation routes to communication devices, A means of providing guidance to evacuate persons in communication equipment, A notification method for providing information to a device via push notifications, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In case of disasters, it is often difficult to evacuate safely and quickly due to traffic congestion on road networks and new dangerous areas. In particular, for disaster victims such as the elderly and disabled, it is difficult to obtain appropriate evacuation information and support, so evacuation may be delayed and they may be exposed to life-threatening risks. Under such circumstances, it is necessary to enable evacuees to know a safe evacuation route in real time and evacuate with confidence.

Means for Solving the Problems

[0005] The present invention solves the above problems by providing a system comprising data collection means for collecting environmental information, artificial intelligence analysis means for analyzing the collected information, route generation means for generating evacuation routes based on the analysis results, information distribution means for distributing the generated evacuation routes to communication terminals, and display means for guiding evacuees. Furthermore, by providing means for updating evacuation routes in real time and means for providing special support information to those requiring assistance, flexible evacuation guidance according to various situations becomes possible.

[0006] "Environmental information" refers to data that shows the local conditions during a disaster, and specifically includes information such as temperature, water level, and the presence or absence of smoke.

[0007] "Data collection means" refers to devices or programs that acquire environmental information using sensors or drones.

[0008] An "artificial intelligence analysis tool" is an artificial intelligence system that performs analysis based on collected environmental information to identify the progress of a disaster and dangerous areas.

[0009] A "route generation means" is an algorithm or device for designing the optimal evacuation route for each user based on the analysis results.

[0010] "Information distribution means" refers to a data transmission system for sending generated evacuation route information to communication terminals.

[0011] "Display means" refers to an interface on a communication terminal that visually or audibly indicates evacuation routes to evacuees.

[0012] A "device" refers to a communication device used by a user, such as a smartphone or wearable device.

[0013] "Persons requiring special assistance" refers to elderly people, people with disabilities, and others who require special consideration or support during a disaster. [Brief explanation of the drawing]

[0014] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

MODE FOR CARRYING OUT THE INVENTION

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0020] [[ID=In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention relates to a "real-time evacuation guidance system during disasters," which consists of multiple components that interact with each other to provide evacuees with safe and efficient evacuation routes.

[0036] The server uses drones and sensors to collect environmental information in real time. This includes specific data such as temperature, water level, and the presence or absence of smoke at disaster sites. Accurately understanding the rapidly changing situation is especially crucial during an ongoing disaster.

[0037] The collected data is analyzed by the server's artificial intelligence (AI) analysis system. The AI ​​detects patterns in the data and identifies the progression of the disaster and newly emerging danger areas. Based on this information, the server generates the optimal evacuation route from the user's current location. This is made possible by combining it with a Geographic Information System (GIS) to determine the shortest and safest route.

[0038] The generated evacuation routes are transmitted to the user's terminal via an information distribution system. The terminal processes the received information and provides evacuees with a visual map display and audio guidance. This allows evacuees to proceed safely in the correct direction even during a disaster. The evacuation routes are updated in real time according to the passage of time and environmental changes, ensuring that the most up-to-date information is always provided.

[0039] Furthermore, special support information is provided to those who require assistance. For example, information on barrier-free routes and areas where support is available along evacuation paths can be provided. In this way, evacuation support is provided that is tailored to individual needs.

[0040] As a concrete example, in the event of a flood, the server identifies flooded areas based on aerial data from drones. The artificial intelligence analysis system analyzes this information and instructs the user to take a safe detour route that avoids flooding, rather than the usual route. The terminal displays a map showing the route and provides voice guidance as needed. By following the terminal's instructions, the user can safely reach their evacuation destination.

[0041] By implementing these functions, the present invention can support rapid and safe evacuation during disasters and meet diverse user needs.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] The server collects environmental information from disaster sites through drones and sensors. Specifically, it acquires data such as temperature, water level, wind speed, and the presence or absence of smoke in real time and manages it centrally.

[0045] Step 2:

[0046] The server inputs the collected environmental information into an artificial intelligence analysis system. The AI ​​analyzes this data to understand the progression of the disaster, identify new hazardous areas, and predict potential problems.

[0047] Step 3:

[0048] Based on the analysis results, the server generates the optimal evacuation route from each user's current location to their destination. In doing so, it utilizes route generation methods to formulate routes while considering the safety of available roads and evacuation routes.

[0049] Step 4:

[0050] The server transmits the generated evacuation route to the user's terminal via an information distribution system. The transmitted information includes details of the evacuation route and important points to note.

[0051] Step 5:

[0052] The terminal provides users with visual and auditory guidance based on the received evacuation route information. It displays the route on a map and guides the user with voice instructions.

[0053] Step 6:

[0054] Users follow the instructions on their device and take the designated evacuation route. The device will display additional information and alerts as needed to ensure the user's safety.

[0055] Step 7:

[0056] The server updates evacuation routes in real time according to changing circumstances and sends the latest information back to the terminal. This ensures that users always have access to the most up-to-date safe routes.

[0057] Step 8:

[0058] The terminal displays special support information to those in need of assistance and, if necessary, notifies the server that rescue is required. This facilitates appropriate responses to users who need assistance.

[0059] (Example 1)

[0060] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0061] While evacuation guidance during disasters must be rapid and safe, conventional systems have difficulty responding to real-time environmental changes and accurately reflecting the needs of individual evacuees. Furthermore, insufficient data collection and analysis to accurately grasp the disaster situation make it difficult to improve the accuracy of evacuation routes.

[0062] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0063] In this invention, the server includes an information acquisition means for collecting environmental information, a data analysis means for analyzing the collected environmental information, and a route calculation means for generating evacuation routes based on the analysis results. This enables the provision of evacuation routes that respond immediately to environmental changes and the distribution of support information tailored to individual needs.

[0064] "Information acquisition means" refers to technical means for collecting environmental information during a disaster.

[0065] "Data analysis means" refers to technical means for analyzing collected environmental information to identify the progress of a disaster and dangerous areas.

[0066] A "route calculation means" is a technical means that generates the optimal evacuation route for evacuees based on analyzed data.

[0067] "Communication means" refers to the technical means for distributing the generated evacuation routes to terminal devices.

[0068] "Guidance means" refers to technical means that provide visual or audio guidance for guiding evacuees on terminal devices.

[0069] A "route updating method" is a technical means that makes it possible to change evacuation routes in real time.

[0070] "Individualized support measures" refer to technical means for providing additional support information tailored to the specific needs of evacuees.

[0071] "Remote photography means" refers to technical means for aerial photography and measurement of disaster situations.

[0072] "Safety measures" refer to technical means for identifying dangerous areas and constructing safe evacuation routes based on those areas.

[0073] This invention is an information system designed to support rapid and safe evacuation during disasters. This system consists of multiple components: a server, terminals, and users. The specific operation of each component is described below.

[0074] server

[0075] The server is designed to collect environmental information in real time. This includes weather sensors and aerial imaging equipment. For example, drones monitor disaster areas and transmit information such as temperature, water levels, and the presence of smoke to the server. The server processes this information using data analysis tools to identify the situation in the affected area and risk areas. Deep learning algorithms are used in this analysis, and generative AI models identify data patterns.

[0076] Based on the analysis results, the server uses Geographic Information System (GIS) data to calculate the optimal evacuation route. This route information is sent to terminals to enable evacuees to evacuate safely.

[0077] terminal

[0078] The terminal processes evacuation information received from the server and presents it to the user. It provides information that the user can intuitively understand through visual map displays and voice guidance. The terminal receives new information updated in real time from the server in response to changes in the situation and always provides the user with the optimal evacuation route.

[0079] User

[0080] Users can evacuate safely by receiving instructions from their devices. Furthermore, special support information is displayed for those requiring assistance. For example, the devices are designed to display information on barrier-free routes and areas where support staff are stationed.

[0081] Examples of specific cases and prompt statements

[0082] As a concrete example, in the event of a flood, the server identifies flooded areas from drone aerial data and guides users to safe detour routes that avoid the flooded areas. Users can then evacuate to a safe location while viewing a map on their device.

[0083] An example of a prompt message would be: "Please describe a system that generates the optimal evacuation route during a flood and displays it on a terminal, including accessibility information for people requiring assistance."

[0084] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0085] Step 1:

[0086] The server uses drones and sensors to collect environmental information in disaster areas. This information includes temperature, water level, and smoke. The input is raw data acquired from sensors and drones. This data is transmitted wirelessly to the server and stored in a database. The output is a dataset of collected environmental conditions.

[0087] Step 2:

[0088] The server analyzes the collected environmental data using artificial intelligence. Here, a deep learning algorithm is used to detect patterns in the data. The input is the dataset obtained in Step 1. The AI ​​model analyzes the data to identify the progression of the disaster and dangerous areas. The output is information identifying high-risk areas.

[0089] Step 3:

[0090] The server generates evacuation routes based on the analysis results. It utilizes GIS data to calculate the shortest and safest routes. The inputs are the analysis results from step 2 and the GIS data. The output is optimal route information from the user's current location to the evacuation destination. This provides an efficient evacuation route.

[0091] Step 4:

[0092] The server distributes the generated routing information to the terminal. It sends data to the terminal via a secure communication protocol. The input is the routing data generated in step 3. The output is the evacuation route information distributed to the terminal.

[0093] Step 5:

[0094] The terminal processes the received evacuation route information and provides the user with visual and audio guidance. The input is the route information sent to the terminal in step 4. The terminal displays a map on its screen, and a voice assistant provides route instructions. The output is the interface presented to the user.

[0095] Step 6:

[0096] The server continuously monitors environmental changes and updates evacuation routes in real time. The input is continuously collected environmental data. The server re-analyzes the data and generates new route information as needed. The output is the updated route information.

[0097] Step 7:

[0098] The server provides support information for vulnerable individuals. For example, it generates and sends barrier-free route information to terminals. The input is data related to the user's specific needs. The output is evacuation support information tailored to vulnerable individuals.

[0099] (Application Example 1)

[0100] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0101] In modern urban areas, there is a need for a system that allows large numbers of people to evacuate quickly and safely in the event of a disaster. However, real-time information is often lacking during disasters, making it difficult to provide appropriate evacuation routes. Furthermore, evacuation support tailored to individual needs is insufficient, posing particular challenges in providing assistance to vulnerable individuals.

[0102] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0103] In this invention, the server includes data collection means for collecting environmental data, information processing means for analyzing the collected environmental data, and data transmission means for distributing the generated evacuation routes to communication devices. This makes it possible to evaluate environmental changes in real time during a disaster and to quickly provide appropriate evacuation routes.

[0104] "Environmental data" refers to detailed information such as temperature, water level, and the presence or absence of smoke, which is necessary to understand the situation during a disaster.

[0105] "Data collection means" refers to functions for collecting environmental data using drones and sensors.

[0106] "Information processing means" refers to functions that analyze collected environmental data to identify the progress of a disaster and dangerous areas.

[0107] The "route calculation means" is a function for calculating the optimal evacuation route based on the analysis results.

[0108] "Data transmission means" refers to a function for transmitting the generated evacuation route to communication equipment.

[0109] "Information provision means" refers to a function in communication equipment that provides information to evacuees visually or audibly.

[0110] A "push notification method" is a function that notifies communication devices of new evacuation information in real time.

[0111] "Persons receiving protection" refers to people who require special assistance during a disaster.

[0112] This invention provides a system to support real-time evacuation guidance during disasters.

[0113] The server uses drones and environmental sensors to collect environmental data such as temperature, water level, and smoke presence in real time. The collected data is analyzed on an artificial intelligence platform through information processing tools. Specifically, Python and TENSORFLOW® are used to detect data patterns and identify hazardous areas.

[0114] Based on the analysis results, the server generates the optimal evacuation route using a route calculation system. This calculation utilizes GIS technology and takes into account the safety and efficiency of the evacuation route.

[0115] The generated evacuation routes are transmitted via data transmission to the user's communication device, specifically their smartphone. The smartphone application uses React Native and provides visual map display and voice guidance. Furthermore, push notifications are used to immediately inform the user of updates to disaster information.

[0116] Users can evacuate safely by following the instructions provided by the app. In addition, those under protection will receive special support information via push notifications, including information on available support centers and barrier-free routes along the evacuation route.

[0117] As a concrete example, in the event of a flood, the server identifies the flooded area based on video data from drones and provides users with visual and audio instructions for safe detours. Through this process, users can evacuate quickly and safely.

[0118] Examples of prompt statements to input into a generative AI model are as follows:

[0119] "You are an AI expert developing a real-time disaster evacuation app for smart cities, designing a model to provide safe routes to citizens during floods. How would you use AI to perform analysis and provide users with specific evacuation routes?"

[0120] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0121] Step 1:

[0122] The server collects environmental data using drones and sensors. Specifically, it acquires sensor data such as temperature, water level, and presence of smoke, and stores it in cloud-based data storage. The input for this step is raw data from sensors, and the output is structured environmental information stored in the cloud.

[0123] Step 2:

[0124] The server analyzes the collected environmental data on an artificial intelligence platform. It uses Python and TensorFlow to analyze data patterns and estimate the progression of a disaster. The input for this step is environmental data; it recognizes data patterns and generates a report. The output is the analysis results regarding the progression of the disaster and the dangerous areas.

[0125] Step 3:

[0126] Based on the analysis results, the server generates the optimal evacuation route using a route calculation tool. It utilizes GIS technology to calculate a safe and efficient route. The input is the analysis results, and the route is calculated based on the GIS data. The output is the evacuation route determined to be optimal.

[0127] Step 4:

[0128] The server sends the generated evacuation route to a communication device such as a smartphone. The React Native application receives this information on the device and initiates a visual map display and voice guidance. The input is the evacuation route information, and the data is sent to the user's device. The output is the state where visual and voice guidance are activated.

[0129] Step 5:

[0130] The user's terminal receives real-time push notifications based on evacuation routes and informs the user of the content. Whenever disaster information is updated, the new information is immediately conveyed to the user. The input is real-time updated disaster information, and the output is voice and notification information for the user.

[0131] Step 6:

[0132] Users follow instructions from the device to evacuate safely. Especially for those under protection, necessary special support information is provided through the device. Input is the evacuation route and support information from the device, and output is the user's actual evacuation actions.

[0133] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0134] This invention combines a "real-time evacuation guidance system during disasters" with an emotion engine that recognizes the user's emotional state, enabling flexible support tailored to the psychological state of evacuees.

[0135] In the initial stages, the server collects environmental information from drones and sensors at the disaster site. This includes information such as local temperature, humidity, and the presence of hazardous materials. The server processes this data using artificial intelligence analysis to conduct a detailed analysis of the current state of the disaster and its impact.

[0136] Based on the analyzed information, the server generates an optimal evacuation route for each individual user. In this process, in addition to the usual geographic information system, road safety and risks along the evacuation route are considered. The server sends the generated route to the terminal, allowing the user to receive the latest safety information in real time.

[0137] The emotion engine analyzes the user's voice, facial expressions, and input contextual information via the device to identify the user's emotional state, such as stress and anxiety levels. This emotional information is used to personalize evacuation guidance.

[0138] For example, if a user is feeling stressed, the device can adjust the guidance information to be concise and reassuring. Furthermore, based on analysis by the emotion engine, it can play relaxing music or encouraging messages for the user.

[0139] For those in need of assistance, special support information is provided via the device. For example, if a person is judged to have a high level of stress or anxiety, an alert is sent directly to the rescue team from the device, allowing for prompt assistance.

[0140] For example, if an evacuee shows extreme anxiety during evacuation guidance after an earthquake, the server receives feedback from the emotion engine and sends additional information to provide reassurance, such as advance notification of the evacuation route. In this way, flexible responses tailored to the psychological state of the evacuees become possible.

[0141] By implementing these functions, we can improve the safety and efficiency of evacuations during disasters, as well as provide psychological support to users.

[0142] The following describes the processing flow.

[0143] Step 1:

[0144] The server uses drones and sensors to collect environmental information from the disaster site. This information includes changes in terrain, weather conditions, and the presence of hazardous materials.

[0145] Step 2:

[0146] The server processes the collected environmental information using artificial intelligence analysis to identify dangerous areas and possible evacuation routes. Based on the analysis results, it generates the optimal evacuation route for evacuees.

[0147] Step 3:

[0148] The server sends the generated evacuation route information to the terminal. This information includes specific route directions and precautions to take during evacuation.

[0149] Step 4:

[0150] The terminal guides the user through the transmitted evacuation route using maps and visual displays, and provides voice guidance as needed.

[0151] Step 5:

[0152] The emotion engine built into the device analyzes the user's voice and facial expressions to detect their current emotional state. For example, it can assess the degree of stress or anxiety.

[0153] Step 6:

[0154] Based on the analysis results of the emotion engine, the device adjusts the content and method of evacuation guidance according to the user's emotional state and displays the information accordingly. For example, it might play relaxation music to reduce stress.

[0155] Step 7:

[0156] Users follow the instructions on their device and proceed along the designated evacuation route. If they feel anxious during the evacuation, the device will continue to provide support in real time.

[0157] Step 8:

[0158] The device provides support information specifically tailored to those in need of assistance and, when necessary, sends emergency notifications to rescue teams. These notifications are based on user state analysis using an emotion engine.

[0159] Step 9:

[0160] The server continuously updates evacuation routes based on changes in the disaster situation and new environmental information, and sends the latest information to terminals. This allows users to continue evacuating safely according to the situation.

[0161] (Example 2)

[0162] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0163] Conventional evacuation guidance systems have the challenge of not being able to provide sufficient safety and psychological stability because they struggle to respond flexibly and immediately based on the situation during a disaster and the emotional state of individual evacuees. Furthermore, they lack special consideration and support information for vulnerable individuals, and further improvements are needed.

[0164] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0165] In this invention, the server includes means for collecting environmental data, information analysis means for analyzing the collected environmental data, and route design means for generating routes based on the analysis results. This enables real-time information collection and analysis, and the generation of optimal evacuation routes during disasters. Furthermore, by providing emotion analysis means and adjustment means for individually responding to information according to the emotional state of the user, it is possible to enhance the psychological stability of evacuees and provide prompt and appropriate support to those in need of assistance.

[0166] "Environmental data" refers to information about disaster sites, such as temperature, humidity, and the presence of hazardous materials, collected using drones and sensors.

[0167] "Information analysis methods" refer to means of analyzing environmental data collected using artificial intelligence technology to understand the current state and impact of a disaster.

[0168] A "route design means" is a system that generates evacuation routes suitable for individual users based on analyzed information.

[0169] "Information transmission means" refers to a means of transmitting generated evacuation routes to the user's terminal and providing necessary information in real time.

[0170] A "display function" is a function on a terminal that visually presents information to the user to support evacuation.

[0171] "Emotional analysis tools" are means of analyzing a user's voice and facial expressions to recognize and evaluate their emotional state.

[0172] "Adjustment measures" refer to methods for optimizing evacuation guidance and information provision according to the emotional state of users, and for providing individualized support.

[0173] A "coordination system" refers to a system that provides special support information to those in need of assistance and notifies support organizations as needed.

[0174] This invention is a system that combines a real-time evacuation guidance system for disaster situations with an emotion analysis function that recognizes the emotional state of the user.

[0175] The server collects environmental data from drones and fixed sensors to understand the current situation at the disaster site. During this process, the server uses high-precision sensors and drones to obtain specific data such as temperature, humidity, and the presence of hazardous materials. The collected data is processed using artificial intelligence libraries such as TensorFlow and PyTorch to analyze the scale and extent of the disaster.

[0176] The server uses geographic information systems such as Google Maps API to generate the optimal evacuation route for each individual evacuee based on the analyzed information. The evacuation route design takes into account road safety and risks along the route. The generated route information is transmitted to the terminal in real time.

[0177] The device supplies voice and facial expressions from the user to an emotion analysis engine to check the user's emotional state. Specifically, the device uses libraries such as OpenCV and DeepFace to identify the emotional state. This allows it to determine the degree to which the user is experiencing stress or anxiety, and sends this data to the server.

[0178] Based on user emotion data, the device provides information best suited to the situation. For example, if the user is feeling stressed, the device adjusts the voice guidance to be calm and reassuring, and changes the music and messages to be encouraging. As an example of a prompt, the AI ​​generation model can be asked the question, "What message should be provided to reassure evacuees who are feeling anxious during an earthquake?" and an answer can be obtained.

[0179] This invention improves the safety and efficiency of providing information regarding evacuation during disasters, and also provides psychological support to users.

[0180] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0181] Step 1:

[0182] The server collects environmental data from drones and sensors. Inputs include on-site temperature, humidity, and the presence of hazardous materials. The server begins by receiving this data from various sources and storing it in a database. Specifically, it implements a system to periodically poll sensor data via an API and update it in real time. The output will be a basic dataset showing the current situation at the disaster site.

[0183] Step 2:

[0184] The server processes the collected environmental data using information analysis tools. The input is the dataset obtained in Step 1. Based on this, the server uses TensorFlow and PyTorch to perform anomaly detection and predict the extent of impact. Specific examples include simulations of temperature fluctuations and chemical diffusion. The output is the analysis results indicating hazardous areas and areas where evacuation is recommended.

[0185] Step 3:

[0186] The server generates the optimal evacuation route for each individual user based on the analyzed information. The input is the analysis results from step 2. The server utilizes the Google Maps API to calculate the optimal route while referencing road information and risk maps. Specifically, it quickly plots the route from the evacuation starting point to the safe zone on the map. The output is personalized evacuation route information for each user.

[0187] Step 4:

[0188] The device inputs the user's voice and facial features into an emotion analysis engine. Inputs include the user's voice data and facial imagery. Using this information, the device analyzes the emotional state using OpenCV and DeepFace, quantifying stress and anxiety levels. Specifically, it evaluates facial data captured by the camera in real time and performs speech recognition. The output is quantitative data indicating the user's emotional state.

[0189] Step 5:

[0190] The device provides personalized information to the user based on the analyzed emotional data. The input is the emotional data from step 4. The device uses this information to optimize the guidance messages and music it outputs. For example, for a user identified as high-stress, it plays calming music or encouraging messages. The output is adjusted information tailored to the user's state.

[0191] Step 6:

[0192] The terminal sends an alert to the server if it detects a special condition in a person in need of assistance. The input is a specific emotional state obtained in step 5. The terminal implements a mechanism to quickly aggregate data and notify the server. Specifically, it sends an emergency alert, and the server generates instructions for rapid assistance. The output is real-time notification information to support organizations.

[0193] (Application Example 2)

[0194] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0195] During disaster evacuations, a lack of proper route guidance can lead to confusion among evacuees, potentially compromising their safety. Furthermore, insufficient support tailored to the evacuees' psychological state can amplify stress and fear, hindering their evacuation efforts. Additionally, inadequate information provision to individuals in particular need of assistance can make rapid evacuation support difficult.

[0196] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0197] In this invention, the server includes information gathering means for collecting environmental data, machine learning analysis means for analyzing the collected environmental data, and route creation means for creating evacuation routes based on the analysis results. This makes it possible to provide optimized evacuation routes in real time and appropriately personalize support information based on the user's emotional state. Furthermore, by providing individualized support information to individuals who particularly need assistance, it is possible to provide assistance quickly.

[0198] "Information gathering means" refers to methods and devices for acquiring environmental information, such as sensors and drones, which play a role in collecting data.

[0199] "Machine learning analysis methods" refer to processing methods that use artificial intelligence technology to analyze collected data and design appropriate evacuation routes.

[0200] A "route creation means" is a method or device that creates the optimal evacuation route based on analysis results, generating a safe and efficient route.

[0201] "Data distribution means" refers to technologies and devices that transmit created evacuation routes to communication devices so that users can receive them.

[0202] "Visualization means" refers to methods or devices that display evacuation routes on communication devices, enabling users to visually understand the route.

[0203] "Means of acquiring emotions" refer to methods or devices for detecting a user's emotional state, which determine emotions through voice and facial expression analysis.

[0204] "Support information generation means" refers to technologies and devices for generating support information tailored to an individual based on their acquired emotional state.

[0205] "Individualized support information provision means" refers to methods or devices for providing specialized information to individuals who need support, and are used to provide appropriate support.

[0206] This invention is a system for supporting evacuees during disasters, which includes collecting and analyzing environmental data, generating evacuation routes, acquiring emotional states, and providing support information. The system is implemented using multiple hardware and software components.

[0207] First, the server collects environmental data using sensors and drones. It can collect information such as temperature, humidity, and the presence of hazardous substances, and the "pandas" library is used for data management.

[0208] Next, the collected data is analyzed using machine learning analysis tools. This analysis utilizes "TensorFlow" to understand the disaster situation and its impact, thereby generating optimal evacuation routes. The created evacuation routes are then distributed to users' communication terminals via data distribution tools.

[0209] On communication terminals, evacuation routes are displayed on a map using visualization methods. Users can then evacuate safely based on this information.

[0210] Furthermore, the device analyzes the user's voice and facial expressions through emotion acquisition mechanisms. Here, "OpenCV" and "librosa" are utilized to acquire the user's emotional state. Based on the user's emotional state, personalized messages and music are generated by the support information generation mechanism and delivered through the individual support information provision mechanism.

[0211] For example, if an earthquake occurs and the user is feeling anxious, the system can generate a message such as, "It's okay, we're guiding you to the safest route. Please proceed slowly," and play relaxing music. The prompts given to the AI ​​model might include phrases like, "Analyze the user's voice and facial expressions, assess their current emotional state, and generate an evacuation guidance message. Also, select music to help reduce stress."

[0212] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0213] Step 1:

[0214] The server collects environmental data from sensors and drones. The input consists of data such as temperature, humidity, and the presence of hazardous materials, obtained from various sensors and drones. This data is organized using "pandas" and managed as structured information. The explicit output is a dataset representing the environment of a disaster site.

[0215] Step 2:

[0216] The server analyzes the collected environmental data using TensorFlow. The input is the environmental data obtained in Step 1. Based on this, the disaster situation and its scope of impact are evaluated, and the optimal evacuation route is generated. Specifically, the data processing involves evaluating environmental risks using a machine learning model. The output is the generated evacuation route information.

[0217] Step 3:

[0218] The server transmits the generated evacuation route to the user's communication terminal using a data distribution method. The input is the evacuation route information obtained from the analysis in step 2. The specific operation when executing data transmission is real-time data delivery from the cloud server to the user terminal. The output is a route map display on the user terminal.

[0219] Step 4:

[0220] The terminal displays the evacuation route on the screen using visualization means. The input is the evacuation route information received in step 3. Here, the route is visualized using a map application and provided in a user-friendly format. The output is an intuitive evacuation route that appears on the display.

[0221] Step 5:

[0222] The device analyzes the user's voice and facial expressions using emotion acquisition methods. The input is real-time data of the user captured by the device's camera and microphone. The user's emotional state is analyzed using "OpenCV" and "librosa". The output is evaluation data of stress levels and anxiety levels indicated by the current emotional state.

[0223] Step 6:

[0224] The server generates support information tailored to the user's emotional state. The input is the emotional assessment data obtained in step 5. Based on this, the generation AI model operates to select appropriate messages and relaxing music. The prompt used is: "Analyze the user's voice and facial expressions, assess their current emotional state, and generate an evacuation guidance message. Also, select recommended music to reduce stress." The output is the content of the emotionally tailored support information.

[0225] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0226] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0227] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0228] [Second Embodiment]

[0229] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0230] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0231] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0232] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0233] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0234] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0235] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0236] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0237] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0238] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0239] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0240] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0241] This invention relates to a "real-time evacuation guidance system during disasters," which consists of multiple components that interact with each other to provide evacuees with safe and efficient evacuation routes.

[0242] The server uses drones and sensors to collect environmental information in real time. This includes specific data such as temperature, water level, and the presence or absence of smoke at disaster sites. Accurately understanding the rapidly changing situation is especially crucial during an ongoing disaster.

[0243] The collected data is analyzed by the server's artificial intelligence (AI) analysis system. The AI ​​detects patterns in the data and identifies the progression of the disaster and newly emerging danger areas. Based on this information, the server generates the optimal evacuation route from the user's current location. This is made possible by combining it with a Geographic Information System (GIS) to determine the shortest and safest route.

[0244] The generated evacuation routes are transmitted to the user's terminal via an information distribution system. The terminal processes the received information and provides evacuees with a visual map display and audio guidance. This allows evacuees to proceed safely in the correct direction even during a disaster. The evacuation routes are updated in real time according to the passage of time and environmental changes, ensuring that the most up-to-date information is always provided.

[0245] Furthermore, special support information is provided to those who require assistance. For example, information on barrier-free routes and areas where support is available along evacuation paths can be provided. In this way, evacuation support is provided that is tailored to individual needs.

[0246] As a concrete example, in the event of a flood, the server identifies flooded areas based on aerial data from drones. The artificial intelligence analysis system analyzes this information and instructs the user to take a safe detour route that avoids flooding, rather than the usual route. The terminal displays a map showing the route and provides voice guidance as needed. By following the terminal's instructions, the user can safely reach their evacuation destination.

[0247] By implementing these functions, the present invention can support rapid and safe evacuation during disasters and meet diverse user needs.

[0248] The following describes the processing flow.

[0249] Step 1:

[0250] The server collects environmental information from disaster sites through drones and sensors. Specifically, it acquires data such as temperature, water level, wind speed, and the presence or absence of smoke in real time and manages it centrally.

[0251] Step 2:

[0252] The server inputs the collected environmental information into an artificial intelligence analysis system. The AI ​​analyzes this data to understand the progression of the disaster, identify new hazardous areas, and predict potential problems.

[0253] Step 3:

[0254] Based on the analysis results, the server generates the optimal evacuation route from each user's current location to their destination. In doing so, it utilizes route generation methods to formulate routes while considering the safety of available roads and evacuation routes.

[0255] Step 4:

[0256] The server transmits the generated evacuation route to the user's terminal via an information distribution system. The transmitted information includes details of the evacuation route and important points to note.

[0257] Step 5:

[0258] The terminal provides users with visual and auditory guidance based on the received evacuation route information. It displays the route on a map and guides the user with voice instructions.

[0259] Step 6:

[0260] Users follow the instructions on their device and take the designated evacuation route. The device will display additional information and alerts as needed to ensure the user's safety.

[0261] Step 7:

[0262] The server updates evacuation routes in real time according to changing circumstances and sends the latest information back to the terminal. This ensures that users always have access to the most up-to-date safe routes.

[0263] Step 8:

[0264] The terminal displays special support information to those in need of assistance and, if necessary, notifies the server that rescue is required. This facilitates appropriate responses to users who need assistance.

[0265] (Example 1)

[0266] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0267] While evacuation guidance during disasters must be rapid and safe, conventional systems have difficulty responding to real-time environmental changes and accurately reflecting the needs of individual evacuees. Furthermore, insufficient data collection and analysis to accurately grasp the disaster situation make it difficult to improve the accuracy of evacuation routes.

[0268] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0269] In this invention, the server includes an information acquisition means for collecting environmental information, a data analysis means for analyzing the collected environmental information, and a route calculation means for generating evacuation routes based on the analysis results. This enables the provision of evacuation routes that respond immediately to environmental changes and the distribution of support information tailored to individual needs.

[0270] "Information acquisition means" refers to technical means for collecting environmental information during a disaster.

[0271] "Data analysis means" refers to technical means for analyzing collected environmental information to identify the progress of a disaster and dangerous areas.

[0272] A "route calculation means" is a technical means that generates the optimal evacuation route for evacuees based on analyzed data.

[0273] "Communication means" refers to the technical means for distributing the generated evacuation routes to terminal devices.

[0274] "Guidance means" refers to technical means that provide visual or audio guidance for guiding evacuees on terminal devices.

[0275] A "route updating method" is a technical means that makes it possible to change evacuation routes in real time.

[0276] "Individualized support measures" refer to technical means for providing additional support information tailored to the specific needs of evacuees.

[0277] "Remote photography means" refers to technical means for aerial photography and measurement of disaster situations.

[0278] "Safety measures" refer to technical means for identifying dangerous areas and constructing safe evacuation routes based on those areas.

[0279] This invention is an information system designed to support rapid and safe evacuation during disasters. This system consists of multiple components: a server, terminals, and users. The specific operation of each component is described below.

[0280] server

[0281] The server is designed to collect environmental information in real time. This includes weather sensors and aerial-capable flying devices. For example, a drone monitors a disaster area and transmits information such as temperature, water level, and the presence of smoke to the server. The server processes this information using data analysis means to identify the situation of the disaster area and risk areas. Deep learning algorithms are utilized in this analysis, and the generated AI model identifies data patterns.

[0282] Based on the analysis results, the server uses Geographic Information System (GIS) data to calculate the optimal evacuation route. This route information is sent to the terminal to enable evacuees to evacuate safely.

[0283] Terminal

[0284] The terminal processes the evacuation information received from the server and presents it to the user. It provides information that the user can intuitively understand through visual map displays and voice guidance. The terminal receives new information updated in real time from the server according to changes in the situation and always provides the user with the optimal evacuation route.

[0285] User

[0286] The user can evacuate safely by receiving instructions from the terminal. Additionally, special support information is displayed for those in need of assistance. For example, it is designed such that information on barrier-free routes and areas where support staff are waiting is displayed on the terminal.

[0287] Examples of specific cases and prompt texts

[0288] As a specific example, when a flood occurs, the server discriminates the flooded areas from the aerial photography data of the drone and guides the user to a safe detour route that avoids the flooded areas. The user can evacuate to a safe place while looking at the map on the terminal.

[0289] An example of a prompt message would be: "Please describe a system that generates the optimal evacuation route during a flood and displays it on a terminal, including accessibility information for those requiring assistance."

[0290] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0291] Step 1:

[0292] The server uses drones and sensors to collect environmental information in disaster areas. This information includes temperature, water level, and smoke. The input is raw data acquired from sensors and drones. This data is transmitted wirelessly to the server and stored in a database. The output is a dataset of collected environmental conditions.

[0293] Step 2:

[0294] The server analyzes the collected environmental data using artificial intelligence. Here, a deep learning algorithm is used to detect patterns in the data. The input is the dataset obtained in Step 1. The AI ​​model analyzes the data to identify the progression of the disaster and dangerous areas. The output is information identifying high-risk areas.

[0295] Step 3:

[0296] The server generates evacuation routes based on the analysis results. It utilizes GIS data to calculate the shortest and safest routes. The inputs are the analysis results from step 2 and the GIS data. The output is optimal route information from the user's current location to the evacuation destination. This provides an efficient evacuation route.

[0297] Step 4:

[0298] The server distributes the generated routing information to the terminal. It sends data to the terminal via a secure communication protocol. The input is the routing data generated in step 3. The output is the evacuation route information distributed to the terminal.

[0299] Step 5:

[0300] The terminal processes the received evacuation route information and provides the user with visual and audio guidance. The input is the route information sent to the terminal in step 4. The terminal displays a map on its screen, and a voice assistant provides route instructions. The output is the interface presented to the user.

[0301] Step 6:

[0302] The server continuously monitors environmental changes and updates evacuation routes in real time. The input is continuously collected environmental data. The server re-analyzes the data and generates new route information as needed. The output is the updated route information.

[0303] Step 7:

[0304] The server provides support information for vulnerable individuals. For example, it generates and sends barrier-free route information to terminals. The input is data related to the user's specific needs. The output is evacuation support information tailored to vulnerable individuals.

[0305] (Application Example 1)

[0306] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0307] In modern urban areas, there is a need for a system that allows large numbers of people to evacuate quickly and safely in the event of a disaster. However, real-time information is often lacking during disasters, making it difficult to provide appropriate evacuation routes. Furthermore, evacuation support tailored to individual needs is insufficient, posing particular challenges in providing assistance to vulnerable individuals.

[0308] The specific processing by the specific processing unit 290 of the data processing apparatus 12 in Application Example 1 is realized by the following means respectively.

[0309] In this invention, the server includes a data collection means for collecting environmental data, an information processing means for analyzing the collected environmental data, and a data transmission means for distributing the generated evacuation route to communication devices. Thereby, it becomes possible to evaluate the environmental changes in real time during a disaster and quickly provide an appropriate evacuation route.

[0310] "Environmental data" refers to detailed information such as temperature, water level, presence or absence of smoke, etc., which is necessary to grasp the situation during a disaster.

[0311] "Data collection means" refers to a function for collecting environmental data using drones or sensors.

[0312] "Information processing means" refers to a function for analyzing the collected environmental data and identifying the progress of the disaster and dangerous areas.

[0313] "Route calculation means" refers to a function for calculating an optimal evacuation route based on the analysis results.

[0314] "Data transmission means" refers to a function for transmitting the generated evacuation route to communication devices.

[0315] "Guidance providing means" refers to a function for providing guidance visually or by voice to evacuees on communication devices.

[0316] "Push notification means" refers to a function for notifying communication devices of new evacuation information in real time.

[0317] "Protected persons" refer to people who require special assistance during a disaster.

[0318] This invention provides a system to support real-time evacuation guidance during disasters.

[0319] The server uses drones and environmental sensors to collect environmental data such as temperature, water level, and smoke presence in real time. The collected data is analyzed on an artificial intelligence platform through information processing tools. Specifically, Python and TensorFlow are used to detect data patterns and identify hazardous areas.

[0320] Based on the analysis results, the server generates the optimal evacuation route using a route calculation system. This calculation utilizes GIS technology and takes into account the safety and efficiency of the evacuation route.

[0321] The generated evacuation routes are transmitted via data transmission to the user's communication device, specifically their smartphone. The smartphone application uses React Native and provides visual map display and voice guidance. Furthermore, push notifications are used to immediately inform the user of updates to disaster information.

[0322] Users can evacuate safely by following the instructions provided by the app. In addition, those under protection will receive special support information via push notifications, including information on available support centers and barrier-free routes along the evacuation route.

[0323] As a concrete example, in the event of a flood, the server identifies the flooded area based on video data from drones and provides users with visual and audio instructions for safe detours. Through this process, users can evacuate quickly and safely.

[0324] Examples of prompt statements to input into a generative AI model are as follows:

[0325] "You are an AI expert developing a real-time disaster evacuation app for smart cities, designing a model to provide safe routes to citizens during floods. How would you use AI to perform analysis and provide users with specific evacuation routes?"

[0326] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0327] Step 1:

[0328] The server collects environmental data using drones and sensors. Specifically, it acquires sensor data such as temperature, water level, and presence of smoke, and stores it in cloud-based data storage. The input for this step is raw data from sensors, and the output is structured environmental information stored in the cloud.

[0329] Step 2:

[0330] The server analyzes the collected environmental data on an artificial intelligence platform. It uses Python and TensorFlow to analyze data patterns and estimate the progression of a disaster. The input for this step is environmental data; it recognizes data patterns and generates a report. The output is the analysis results regarding the progression of the disaster and the dangerous areas.

[0331] Step 3:

[0332] Based on the analysis results, the server generates the optimal evacuation route using a route calculation tool. It utilizes GIS technology to calculate a safe and efficient route. The input is the analysis results, and the route is calculated based on the GIS data. The output is the evacuation route determined to be optimal.

[0333] Step 4:

[0334] The server sends the generated evacuation route to a communication device such as a smartphone. The React Native application receives this information on the device and initiates a visual map display and voice guidance. The input is the evacuation route information, and the data is sent to the user's device. The output is the state where visual and voice guidance are activated.

[0335] Step 5:

[0336] The user's terminal receives real-time push notifications based on evacuation routes and informs the user of the content. Whenever disaster information is updated, the new information is immediately conveyed to the user. The input is real-time updated disaster information, and the output is voice and notification information for the user.

[0337] Step 6:

[0338] Users follow instructions from the device to evacuate safely. Especially for those under protection, necessary special support information is provided through the device. Input is the evacuation route and support information from the device, and output is the user's actual evacuation actions.

[0339] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0340] This invention combines a "real-time evacuation guidance system during disasters" with an emotion engine that recognizes the user's emotional state, enabling flexible support tailored to the psychological state of evacuees.

[0341] In the initial stages, the server collects environmental information from drones and sensors at the disaster site. This includes information such as local temperature, humidity, and the presence of hazardous materials. The server processes this data using artificial intelligence analysis to conduct a detailed analysis of the current state of the disaster and its impact.

[0342] Based on the analyzed information, the server generates an optimal evacuation route for each individual user. In this process, in addition to the usual geographic information system, road safety and risks along the evacuation route are considered. The server sends the generated route to the terminal, allowing users to receive the latest safety information in real time.

[0343] The emotion engine analyzes the user's voice, facial expressions, and input contextual information via the device to identify the user's emotional state, such as stress and anxiety levels. This emotional information is used to personalize evacuation guidance.

[0344] For example, if a user is feeling stressed, the device can adjust the guidance information to be concise and reassuring. Furthermore, based on analysis by the emotion engine, it can play relaxing music or encouraging messages for the user.

[0345] For those in need of assistance, special support information is provided via the device. For example, if a person is judged to have a high level of stress or anxiety, an alert is sent directly to the rescue team from the device, allowing for prompt assistance.

[0346] For example, if an evacuee shows extreme anxiety during evacuation guidance after an earthquake, the server receives feedback from the emotion engine and sends additional information to provide reassurance, such as advance notification of the evacuation route. In this way, flexible responses tailored to the psychological state of the evacuees become possible.

[0347] By implementing these functions, we can improve the safety and efficiency of evacuations during disasters, as well as provide psychological support to users.

[0348] The following describes the processing flow.

[0349] Step 1:

[0350] The server uses drones and sensors to collect environmental information from the disaster site. This information includes changes in terrain, weather conditions, and the presence of hazardous materials.

[0351] Step 2:

[0352] The server processes the collected environmental information using artificial intelligence analysis to identify dangerous areas and possible evacuation routes. Based on the analysis results, it generates the optimal evacuation route for evacuees.

[0353] Step 3:

[0354] The server sends the generated evacuation route information to the terminal. This information includes specific route directions and precautions to take during evacuation.

[0355] Step 4:

[0356] The terminal guides the user through the transmitted evacuation route using maps and visual displays, and provides voice guidance as needed.

[0357] Step 5:

[0358] The emotion engine built into the device analyzes the user's voice and facial expressions to detect their current emotional state. For example, it can assess the degree of stress or anxiety.

[0359] Step 6:

[0360] Based on the analysis results of the emotion engine, the device adjusts the content and method of evacuation guidance according to the user's emotional state and displays the information accordingly. For example, it might play relaxation music to reduce stress.

[0361] Step 7:

[0362] Users follow the instructions on their device and proceed along the designated evacuation route. If they feel anxious during the evacuation, the device will continue to provide support in real time.

[0363] Step 8:

[0364] The device provides support information specifically tailored to those in need of assistance and, when necessary, sends emergency notifications to rescue teams. These notifications are based on user state analysis using an emotion engine.

[0365] Step 9:

[0366] The server continuously updates evacuation routes based on changes in the disaster situation and new environmental information, and sends the latest information to terminals. This allows users to continue evacuating safely according to the situation.

[0367] (Example 2)

[0368] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0369] Conventional evacuation guidance systems have the challenge of not being able to provide sufficient safety and psychological stability because they struggle to respond flexibly and immediately based on the situation during a disaster and the emotional state of individual evacuees. Furthermore, they lack special consideration and support information for vulnerable individuals, and further improvements are needed.

[0370] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0371] In this invention, the server includes means for collecting environmental data, information analysis means for analyzing the collected environmental data, and route design means for generating routes based on the analysis results. This enables real-time information collection and analysis, and the generation of optimal evacuation routes during disasters. Furthermore, by providing emotion analysis means and adjustment means for individually responding to information according to the emotional state of the user, it is possible to enhance the psychological stability of evacuees and provide prompt and appropriate support to those in need of assistance.

[0372] "Environmental data" refers to information about disaster sites, such as temperature, humidity, and the presence of hazardous materials, collected using drones and sensors.

[0373] "Information analysis methods" refer to means of analyzing environmental data collected using artificial intelligence technology to understand the current state and impact of a disaster.

[0374] A "route design means" is a system that generates evacuation routes suitable for individual users based on analyzed information.

[0375] "Information transmission means" refers to a means of transmitting generated evacuation routes to the user's terminal and providing necessary information in real time.

[0376] A "display function" is a function on a terminal that visually presents information to the user to support evacuation.

[0377] "Emotional analysis tools" are means of analyzing a user's voice and facial expressions to recognize and evaluate their emotional state.

[0378] "Adjustment measures" refer to methods for optimizing evacuation guidance and information provision according to the emotional state of users, and for providing individualized support.

[0379] A "coordination method" refers to a system that provides special support information to those in need of assistance and notifies support organizations as needed.

[0380] This invention is a system that combines a real-time evacuation guidance system for disaster situations with an emotion analysis function that recognizes the emotional state of the user.

[0381] The server collects environmental data from drones and fixed sensors to understand the current situation at the disaster site. During this process, the server uses high-precision sensors and drones to obtain specific data such as temperature, humidity, and the presence of hazardous materials. The collected data is processed using artificial intelligence libraries such as TensorFlow and PyTorch to analyze the scale and extent of the disaster.

[0382] The server uses geographic information systems such as the Google Maps API to generate the optimal evacuation route for each individual evacuee based on the analyzed information. The evacuation route design takes into account road safety and risks along the route. The generated route information is transmitted to the terminal in real time.

[0383] The device supplies voice and facial expressions from the user to an emotion analysis engine to check the user's emotional state. Specifically, the device uses libraries such as OpenCV and DeepFace to identify the emotional state. This allows it to determine the degree to which the user is experiencing stress or anxiety, and sends this data to the server.

[0384] Based on user emotion data, the device provides information best suited to the situation. For example, if the user is feeling stressed, the device adjusts the voice guidance to be calm and reassuring, and changes the music and messages to be encouraging. As an example of a prompt, the AI ​​generation model can be asked the question, "What message should be provided to reassure evacuees who are feeling anxious during an earthquake?" and an answer can be obtained.

[0385] This invention improves the safety and efficiency of providing information regarding evacuation during disasters, and also provides psychological support to users.

[0386] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0387] Step 1:

[0388] The server collects environmental data from drones and sensors. Inputs include on-site temperature, humidity, and the presence of hazardous materials. The server begins by receiving this data from various sources and storing it in a database. Specifically, it implements a system to periodically poll sensor data via an API and update it in real time. The output will be a basic dataset showing the current situation at the disaster site.

[0389] Step 2:

[0390] The server processes the collected environmental data using information analysis tools. The input is the dataset obtained in Step 1. Based on this, the server uses TensorFlow and PyTorch to perform anomaly detection and predict the extent of impact. Specific examples include simulations of temperature fluctuations and chemical diffusion. The output is the analysis results indicating hazardous areas and areas where evacuation is recommended.

[0391] Step 3:

[0392] The server generates the optimal evacuation route for each individual user based on the analyzed information. The input is the analysis results from step 2. The server utilizes the Google Maps API to calculate the optimal route while referencing road information and risk maps. Specifically, it quickly plots the route from the evacuation starting point to the safe zone on the map. The output is personalized evacuation route information for each user.

[0393] Step 4:

[0394] The device inputs the user's voice and facial features into an emotion analysis engine. Inputs include the user's voice data and facial imagery. Using this information, the device analyzes the emotional state using OpenCV and DeepFace, quantifying stress and anxiety levels. Specifically, it evaluates facial data captured by the camera in real time and performs speech recognition. The output is quantitative data indicating the user's emotional state.

[0395] Step 5:

[0396] The device provides personalized information to the user based on the analyzed emotional data. The input is the emotional data from step 4. The device uses this information to optimize the guidance messages and music it outputs. For example, for a user identified as high-stress, it plays calming music or encouraging messages. The output is adjusted information tailored to the user's state.

[0397] Step 6:

[0398] The terminal sends an alert to the server if it detects a special condition in a person in need of assistance. The input is a specific emotional state obtained in step 5. The terminal implements a mechanism to quickly aggregate data and notify the server. Specifically, it sends an emergency alert, and the server generates instructions for rapid assistance. The output is real-time notification information to support organizations.

[0399] (Application Example 2)

[0400] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0401] During disaster evacuations, a lack of proper route guidance can lead to confusion among evacuees, potentially compromising their safety. Furthermore, insufficient support tailored to the evacuees' psychological state can amplify stress and fear, hindering their evacuation efforts. Additionally, inadequate information provision to individuals in particular need of assistance can make rapid evacuation support difficult.

[0402] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0403] In this invention, the server includes information gathering means for collecting environmental data, machine learning analysis means for analyzing the collected environmental data, and route creation means for creating evacuation routes based on the analysis results. This makes it possible to provide optimized evacuation routes in real time and appropriately personalize support information based on the user's emotional state. Furthermore, by providing individualized support information to individuals who particularly need assistance, it is possible to provide assistance quickly.

[0404] "Information gathering means" refers to methods and devices for acquiring environmental information, such as sensors and drones, which play a role in collecting data.

[0405] "Machine learning analysis methods" refer to processing methods that use artificial intelligence technology to analyze collected data and design appropriate evacuation routes.

[0406] A "route creation means" is a method or device that creates the optimal evacuation route based on analysis results, generating a safe and efficient route.

[0407] "Data distribution means" refers to technologies and devices that transmit created evacuation routes to communication devices so that users can receive them.

[0408] "Visualization means" refers to methods or devices that display evacuation routes on communication devices, enabling users to visually understand the route.

[0409] "Means of acquiring emotions" refer to methods or devices for detecting a user's emotional state, which determine emotions through voice and facial expression analysis.

[0410] "Support information generation means" refers to technologies and devices for generating support information tailored to an individual based on their acquired emotional state.

[0411] "Individualized support information provision means" refers to methods or devices for providing specialized information to individuals who need support, and are used to provide appropriate support.

[0412] This invention is a system for supporting evacuees during disasters, which includes collecting and analyzing environmental data, generating evacuation routes, acquiring emotional states, and providing support information. The system is implemented using multiple hardware and software components.

[0413] First, the server collects environmental data using sensors and drones. It can collect information such as temperature, humidity, and the presence of hazardous substances, and the "pandas" library is used for data management.

[0414] Next, the collected data is analyzed using machine learning analysis tools. This analysis utilizes "TensorFlow" to understand the disaster situation and its impact, thereby generating optimal evacuation routes. The created evacuation routes are then distributed to users' communication terminals via data distribution tools.

[0415] On communication terminals, evacuation routes are displayed on a map using visualization methods. Users can then evacuate safely based on this information.

[0416] Furthermore, the device analyzes the user's voice and facial expressions through emotion acquisition mechanisms. Here, "OpenCV" and "librosa" are utilized to acquire the user's emotional state. Based on the user's emotional state, personalized messages and music are generated by the support information generation mechanism and delivered through the individual support information provision mechanism.

[0417] For example, if an earthquake occurs and the user is feeling anxious, the system can generate a message such as, "It's okay, we're guiding you to the safest route. Please proceed slowly," and play relaxing music. The prompts given to the AI ​​model might include phrases like, "Analyze the user's voice and facial expressions, assess their current emotional state, and generate an evacuation guidance message. Also, select music to help reduce stress."

[0418] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0419] Step 1:

[0420] The server collects environmental data from sensors and drones. The input consists of data such as temperature, humidity, and the presence of hazardous materials, obtained from various sensors and drones. This data is organized using "pandas" and managed as structured information. The explicit output is a dataset representing the environment of a disaster site.

[0421] Step 2:

[0422] The server analyzes the collected environmental data using TensorFlow. The input is the environmental data obtained in Step 1. Based on this, the disaster situation and its scope of impact are evaluated, and the optimal evacuation route is generated. Specifically, the data processing involves evaluating environmental risks using a machine learning model. The output is the generated evacuation route information.

[0423] Step 3:

[0424] The server transmits the generated evacuation route to the user's communication terminal using a data distribution method. The input is the evacuation route information obtained from the analysis in step 2. The specific operation when executing data transmission is real-time data delivery from the cloud server to the user terminal. The output is a route map display on the user terminal.

[0425] Step 4:

[0426] The terminal displays the evacuation route on the screen using visualization means. The input is the evacuation route information received in step 3. Here, the route is visualized using a map application and provided in a user-friendly format. The output is an intuitive evacuation route that appears on the display.

[0427] Step 5:

[0428] The device analyzes the user's voice and facial expressions using emotion acquisition methods. The input is real-time data of the user captured by the device's camera and microphone. The user's emotional state is analyzed using "OpenCV" and "librosa". The output is evaluation data of stress levels and anxiety levels indicated by the current emotional state.

[0429] Step 6:

[0430] The server generates support information tailored to the user's emotional state. The input is the emotional assessment data obtained in step 5. Based on this, the generation AI model operates to select appropriate messages and relaxing music. The prompt used is: "Analyze the user's voice and facial expressions, assess their current emotional state, and generate an evacuation guidance message. Also, select recommended music to reduce stress." The output is the content of the emotionally tailored support information.

[0431] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0432] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0433] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0434] [Third Embodiment]

[0435] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0436] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0437] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0438] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0439] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0440] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0441] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0442] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0443] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0444] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0445] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0446] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0447] This invention relates to a "real-time evacuation guidance system during disasters," which consists of multiple components that interact with each other to provide evacuees with safe and efficient evacuation routes.

[0448] The server uses drones and sensors to collect environmental information in real time. This includes specific data such as temperature, water level, and the presence or absence of smoke at disaster sites. Accurately understanding the rapidly changing situation is especially crucial during an ongoing disaster.

[0449] The collected data is analyzed by the server's artificial intelligence (AI) analysis system. The AI ​​detects patterns in the data and identifies the progression of the disaster and newly emerging danger areas. Based on this information, the server generates the optimal evacuation route from the user's current location. This is made possible by combining it with a Geographic Information System (GIS) to determine the shortest and safest route.

[0450] The generated evacuation routes are transmitted to the user's terminal via an information distribution system. The terminal processes the received information and provides evacuees with a visual map display and audio guidance. This allows evacuees to proceed safely in the correct direction even during a disaster. The evacuation routes are updated in real time according to the passage of time and environmental changes, ensuring that the most up-to-date information is always provided.

[0451] Furthermore, special support information is provided to those who require assistance. For example, information on barrier-free routes and areas where support is available along evacuation paths can be provided. In this way, evacuation support is provided that is tailored to individual needs.

[0452] As a concrete example, in the event of a flood, the server identifies flooded areas based on aerial data from drones. The artificial intelligence analysis system analyzes this information and instructs the user to take a safe detour route that avoids flooding, rather than the usual route. The terminal displays a map showing the route and provides voice guidance as needed. By following the terminal's instructions, the user can safely reach their evacuation destination.

[0453] By implementing these functions, the present invention can support rapid and safe evacuation during disasters and meet diverse user needs.

[0454] The following describes the processing flow.

[0455] Step 1:

[0456] The server collects environmental information from disaster sites through drones and sensors. Specifically, it acquires data such as temperature, water level, wind speed, and the presence or absence of smoke in real time and manages it centrally.

[0457] Step 2:

[0458] The server inputs the collected environmental information into an artificial intelligence analysis system. The AI ​​analyzes this data to understand the progression of the disaster, identify new hazardous areas, and predict potential problems.

[0459] Step 3:

[0460] Based on the analysis results, the server generates the optimal evacuation route from each user's current location to their destination. In doing so, it utilizes route generation methods to formulate routes while considering the safety of available roads and evacuation routes.

[0461] Step 4:

[0462] The server transmits the generated evacuation route to the user's terminal via an information distribution system. The transmitted information includes details of the evacuation route and important points to note.

[0463] Step 5:

[0464] The terminal provides users with visual and auditory guidance based on the received evacuation route information. It displays the route on a map and guides the user with voice instructions.

[0465] Step 6:

[0466] Users follow the instructions on their device and take the designated evacuation route. The device will display additional information and alerts as needed to ensure the user's safety.

[0467] Step 7:

[0468] The server updates evacuation routes in real time according to changing circumstances and sends the latest information back to the terminal. This ensures that users always have access to the most up-to-date safe routes.

[0469] Step 8:

[0470] The terminal displays special support information to those in need of assistance and, if necessary, notifies the server that rescue is required. This facilitates appropriate responses to users who need assistance.

[0471] (Example 1)

[0472] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0473] While evacuation guidance during disasters must be rapid and safe, conventional systems have difficulty responding to real-time environmental changes and accurately reflecting the needs of individual evacuees. Furthermore, insufficient data collection and analysis to accurately grasp the disaster situation make it difficult to improve the accuracy of evacuation routes.

[0474] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0475] In this invention, the server includes an information acquisition means for collecting environmental information, a data analysis means for analyzing the collected environmental information, and a route calculation means for generating evacuation routes based on the analysis results. This enables the provision of evacuation routes that respond immediately to environmental changes and the distribution of support information tailored to individual needs.

[0476] "Information acquisition means" refers to technical means for collecting environmental information during a disaster.

[0477] "Data analysis means" refers to technical means for analyzing collected environmental information to identify the progress of a disaster and dangerous areas.

[0478] A "route calculation means" is a technical means that generates the optimal evacuation route for evacuees based on analyzed data.

[0479] "Communication means" refers to the technical means for distributing the generated evacuation routes to terminal devices.

[0480] "Guidance means" refers to technical means that provide visual or audio guidance for guiding evacuees on terminal devices.

[0481] A "route updating method" is a technical means that makes it possible to change evacuation routes in real time.

[0482] "Individualized support measures" refer to technical means for providing additional support information tailored to the specific needs of evacuees.

[0483] "Remote photography means" refers to technical means for aerial photography and measurement of disaster situations.

[0484] "Safety measures" refer to technical means for identifying dangerous areas and constructing safe evacuation routes based on those areas.

[0485] This invention is an information system designed to support rapid and safe evacuation during disasters. This system consists of multiple components: a server, terminals, and users. The specific operation of each component is described below.

[0486] server

[0487] The server is designed to collect environmental information in real time. This includes weather sensors and aerial imaging equipment. For example, drones monitor disaster areas and transmit information such as temperature, water levels, and the presence of smoke to the server. The server processes this information using data analysis tools to identify the situation in the affected area and risk areas. Deep learning algorithms are used in this analysis, and generative AI models identify data patterns.

[0488] Based on the analysis results, the server uses Geographic Information System (GIS) data to calculate the optimal evacuation route. This route information is sent to terminals to enable evacuees to evacuate safely.

[0489] terminal

[0490] The terminal processes evacuation information received from the server and presents it to the user. It provides information that the user can intuitively understand through visual map displays and voice guidance. The terminal receives new information updated in real time from the server in response to changes in the situation and always provides the user with the optimal evacuation route.

[0491] User

[0492] Users can evacuate safely by receiving instructions from their devices. Furthermore, special support information is displayed for those requiring assistance. For example, the devices are designed to display information on barrier-free routes and areas where support staff are stationed.

[0493] Examples of specific cases and prompt statements

[0494] As a concrete example, in the event of a flood, the server identifies flooded areas from drone aerial data and guides users to safe detour routes that avoid the flooded areas. Users can then evacuate to a safe location while viewing a map on their device.

[0495] An example of a prompt message would be: "Please describe a system that generates the optimal evacuation route during a flood and displays it on a terminal, including accessibility information for those requiring assistance."

[0496] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0497] Step 1:

[0498] The server uses drones and sensors to collect environmental information in disaster areas. This information includes temperature, water level, and smoke. The input is raw data acquired from sensors and drones. This data is transmitted wirelessly to the server and stored in a database. The output is a dataset of collected environmental conditions.

[0499] Step 2:

[0500] The server analyzes the collected environmental data using artificial intelligence. Here, a deep learning algorithm is used to detect patterns in the data. The input is the dataset obtained in Step 1. The AI ​​model analyzes the data to identify the progression of the disaster and dangerous areas. The output is information identifying high-risk areas.

[0501] Step 3:

[0502] The server generates evacuation routes based on the analysis results. It utilizes GIS data to calculate the shortest and safest routes. The inputs are the analysis results from step 2 and the GIS data. The output is optimal route information from the user's current location to the evacuation destination. This provides an efficient evacuation route.

[0503] Step 4:

[0504] The server distributes the generated routing information to the terminal. It sends data to the terminal via a secure communication protocol. The input is the routing data generated in step 3. The output is the evacuation route information distributed to the terminal.

[0505] Step 5:

[0506] The terminal processes the received evacuation route information and provides the user with visual and audio guidance. The input is the route information sent to the terminal in step 4. The terminal displays a map on its screen, and a voice assistant provides route instructions. The output is the interface presented to the user.

[0507] Step 6:

[0508] The server continuously monitors environmental changes and updates evacuation routes in real time. The input is continuously collected environmental data. The server re-analyzes the data and generates new route information as needed. The output is the updated route information.

[0509] Step 7:

[0510] The server provides support information for vulnerable individuals. For example, it generates and sends barrier-free route information to terminals. The input is data related to the user's specific needs. The output is evacuation support information tailored to vulnerable individuals.

[0511] (Application Example 1)

[0512] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0513] In modern urban areas, there is a need for a system that allows large numbers of people to evacuate quickly and safely in the event of a disaster. However, real-time information is often lacking during disasters, making it difficult to provide appropriate evacuation routes. Furthermore, evacuation support tailored to individual needs is insufficient, posing particular challenges in providing assistance to vulnerable individuals.

[0514] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0515] In this invention, the server includes data collection means for collecting environmental data, information processing means for analyzing the collected environmental data, and data transmission means for distributing the generated evacuation routes to communication devices. This makes it possible to evaluate environmental changes in real time during a disaster and to quickly provide appropriate evacuation routes.

[0516] "Environmental data" refers to detailed information such as temperature, water level, and the presence or absence of smoke, which is necessary to understand the situation during a disaster.

[0517] "Data collection means" refers to functions for collecting environmental data using drones and sensors.

[0518] "Information processing means" refers to functions that analyze collected environmental data to identify the progress of a disaster and dangerous areas.

[0519] The "route calculation means" is a function for calculating the optimal evacuation route based on the analysis results.

[0520] "Data transmission means" refers to a function for transmitting the generated evacuation route to communication equipment.

[0521] "Information provision means" refers to a function in communication equipment that provides information to evacuees visually or audibly.

[0522] A "push notification method" is a function that notifies communication devices of new evacuation information in real time.

[0523] "Persons receiving protection" refers to people who require special assistance during a disaster.

[0524] This invention provides a system to support real-time evacuation guidance during disasters.

[0525] The server uses drones and environmental sensors to collect environmental data such as temperature, water level, and smoke presence in real time. The collected data is analyzed on an artificial intelligence platform through information processing tools. Specifically, Python and TensorFlow are used to detect data patterns and identify hazardous areas.

[0526] Based on the analysis results, the server generates the optimal evacuation route using a route calculation system. This calculation utilizes GIS technology and takes into account the safety and efficiency of the evacuation route.

[0527] The generated evacuation routes are transmitted via data transmission to the user's communication device, specifically their smartphone. The smartphone application uses React Native and provides visual map display and voice guidance. Furthermore, push notifications are used to immediately inform the user of updates to disaster information.

[0528] Users can evacuate safely by following the instructions provided by the app. In addition, those under protection will receive special support information via push notifications, including information on available support centers and barrier-free routes along the evacuation route.

[0529] As a concrete example, in the event of a flood, the server identifies the flooded area based on video data from drones and provides users with visual and audio instructions for safe detours. Through this process, users can evacuate quickly and safely.

[0530] Examples of prompt statements to input into a generative AI model are as follows:

[0531] "You are an AI expert developing a real-time disaster evacuation app for smart cities, designing a model to provide safe routes to citizens during floods. How would you use AI to perform analysis and provide users with specific evacuation routes?"

[0532] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0533] Step 1:

[0534] The server collects environmental data using drones and sensors. Specifically, it acquires sensor data such as temperature, water level, and presence of smoke, and stores it in cloud-based data storage. The input for this step is raw data from sensors, and the output is structured environmental information stored in the cloud.

[0535] Step 2:

[0536] The server analyzes the collected environmental data on an artificial intelligence platform. It uses Python and TensorFlow to analyze data patterns and estimate the progression of a disaster. The input for this step is environmental data; it recognizes data patterns and generates a report. The output is the analysis results regarding the progression of the disaster and the dangerous areas.

[0537] Step 3:

[0538] Based on the analysis results, the server generates the optimal evacuation route using a route calculation tool. It utilizes GIS technology to calculate a safe and efficient route. The input is the analysis results, and the route is calculated based on the GIS data. The output is the evacuation route determined to be optimal.

[0539] Step 4:

[0540] The server sends the generated evacuation route to a communication device such as a smartphone. The React Native application receives this information on the device and initiates a visual map display and voice guidance. The input is the evacuation route information, and the data is sent to the user's device. The output is the state where visual and voice guidance are activated.

[0541] Step 5:

[0542] The user's terminal receives real-time push notifications based on evacuation routes and informs the user of the content. Whenever disaster information is updated, the new information is immediately conveyed to the user. The input is real-time updated disaster information, and the output is voice and notification information for the user.

[0543] Step 6:

[0544] Users follow instructions from the device to evacuate safely. Especially for those under protection, necessary special support information is provided through the device. Input is the evacuation route and support information from the device, and output is the user's actual evacuation actions.

[0545] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0546] This invention combines a "real-time evacuation guidance system during disasters" with an emotion engine that recognizes the user's emotional state, enabling flexible support tailored to the psychological state of evacuees.

[0547] In the initial stages, the server collects environmental information from drones and sensors at the disaster site. This includes information such as local temperature, humidity, and the presence of hazardous materials. The server processes this data using artificial intelligence analysis to conduct a detailed analysis of the current state of the disaster and its impact.

[0548] Based on the analyzed information, the server generates an optimal evacuation route for each individual user. In this process, in addition to the usual geographic information system, road safety and risks along the evacuation route are considered. The server sends the generated route to the terminal, allowing users to receive the latest safety information in real time.

[0549] The emotion engine analyzes the user's voice, facial expressions, and input contextual information via the device to identify the user's emotional state, such as stress and anxiety levels. This emotional information is used to personalize evacuation guidance.

[0550] For example, if a user is feeling stressed, the device can adjust the guidance information to be concise and reassuring. Furthermore, based on analysis by the emotion engine, it can play relaxing music or encouraging messages for the user.

[0551] For those in need of assistance, special support information is provided via the device. For example, if a person is judged to have a high level of stress or anxiety, an alert is sent directly to the rescue team from the device, allowing for prompt assistance.

[0552] For example, if an evacuee shows extreme anxiety during evacuation guidance after an earthquake, the server receives feedback from the emotion engine and sends additional information to provide reassurance, such as advance notification of the evacuation route. In this way, flexible responses tailored to the psychological state of the evacuees become possible.

[0553] By implementing these functions, we can improve the safety and efficiency of evacuations during disasters, as well as provide psychological support to users.

[0554] The following describes the processing flow.

[0555] Step 1:

[0556] The server uses drones and sensors to collect environmental information from the disaster site. This information includes changes in terrain, weather conditions, and the presence of hazardous materials.

[0557] Step 2:

[0558] The server processes the collected environmental information using artificial intelligence analysis to identify dangerous areas and possible evacuation routes. Based on the analysis results, it generates the optimal evacuation route for evacuees.

[0559] Step 3:

[0560] The server sends the generated evacuation route information to the terminal. This information includes specific route directions and precautions to take during evacuation.

[0561] Step 4:

[0562] The terminal guides the user through the transmitted evacuation route using maps and visual displays, and provides voice guidance as needed.

[0563] Step 5:

[0564] The emotion engine built into the device analyzes the user's voice and facial expressions to detect their current emotional state. For example, it can assess the degree of stress or anxiety.

[0565] Step 6:

[0566] Based on the analysis results of the emotion engine, the device adjusts the content and method of evacuation guidance according to the user's emotional state and displays the information accordingly. For example, it might play relaxation music to reduce stress.

[0567] Step 7:

[0568] Users follow the instructions on their device and proceed along the designated evacuation route. If they feel anxious during the evacuation, the device will continue to provide support in real time.

[0569] Step 8:

[0570] The device provides support information specifically tailored to those in need of assistance and, when necessary, sends emergency notifications to rescue teams. These notifications are based on user state analysis using an emotion engine.

[0571] Step 9:

[0572] The server continuously updates evacuation routes based on changes in the disaster situation and new environmental information, and sends the latest information to terminals. This allows users to continue evacuating safely according to the situation.

[0573] (Example 2)

[0574] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0575] Conventional evacuation guidance systems have the challenge of not being able to provide sufficient safety and psychological stability because they struggle to respond flexibly and immediately based on the situation during a disaster and the emotional state of individual evacuees. Furthermore, they lack special consideration and support information for vulnerable individuals, and further improvements are needed.

[0576] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0577] In this invention, the server includes means for collecting environmental data, information analysis means for analyzing the collected environmental data, and route design means for generating routes based on the analysis results. This enables real-time information collection and analysis, and the generation of optimal evacuation routes during disasters. Furthermore, by providing emotion analysis means and adjustment means for individually responding to information according to the emotional state of the user, it is possible to enhance the psychological stability of evacuees and provide prompt and appropriate support to those in need of assistance.

[0578] "Environmental data" refers to information about disaster sites, such as temperature, humidity, and the presence of hazardous materials, collected using drones and sensors.

[0579] "Information analysis methods" refer to means of analyzing environmental data collected using artificial intelligence technology to understand the current state and impact of a disaster.

[0580] A "route design means" is a system that generates evacuation routes suitable for individual users based on analyzed information.

[0581] "Information transmission means" refers to a means of transmitting generated evacuation routes to the user's terminal and providing necessary information in real time.

[0582] A "display function" is a function on a terminal that visually presents information to the user to support evacuation.

[0583] "Emotional analysis tools" are means of analyzing a user's voice and facial expressions to recognize and evaluate their emotional state.

[0584] "Adjustment measures" refer to methods for optimizing evacuation guidance and information provision according to the emotional state of users, and for providing individualized support.

[0585] A "coordination method" refers to a system that provides special support information to those in need of assistance and notifies support organizations as needed.

[0586] This invention is a system that combines a real-time evacuation guidance system for disaster situations with an emotion analysis function that recognizes the emotional state of the user.

[0587] The server collects environmental data from drones and fixed sensors to understand the current situation at the disaster site. During this process, the server uses high-precision sensors and drones to obtain specific data such as temperature, humidity, and the presence of hazardous materials. The collected data is processed using artificial intelligence libraries such as TensorFlow and PyTorch to analyze the scale and extent of the disaster.

[0588] The server uses geographic information systems such as the Google Maps API to generate the optimal evacuation route for each individual evacuee based on the analyzed information. The evacuation route design takes into account road safety and risks along the route. The generated route information is transmitted to the terminal in real time.

[0589] The device supplies voice and facial expressions from the user to an emotion analysis engine to check the user's emotional state. Specifically, the device uses libraries such as OpenCV and DeepFace to identify the emotional state. This allows it to determine the degree to which the user is experiencing stress or anxiety, and sends this data to the server.

[0590] Based on user emotion data, the device provides information best suited to the situation. For example, if the user is feeling stressed, the device adjusts the voice guidance to be calm and reassuring, and changes the music and messages to be encouraging. As an example of a prompt, the AI ​​generation model can be asked the question, "What message should be provided to reassure evacuees who are feeling anxious during an earthquake?" and an answer can be obtained.

[0591] This invention improves the safety and efficiency of providing information regarding evacuation during disasters, and also provides psychological support to users.

[0592] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0593] Step 1:

[0594] The server collects environmental data from drones and sensors. Inputs include on-site temperature, humidity, and the presence of hazardous materials. The server begins by receiving this data from various sources and storing it in a database. Specifically, it implements a system to periodically poll sensor data via an API and update it in real time. The output will be a basic dataset showing the current situation at the disaster site.

[0595] Step 2:

[0596] The server processes the collected environmental data using information analysis tools. The input is the dataset obtained in Step 1. Based on this, the server uses TensorFlow and PyTorch to perform anomaly detection and predict the extent of impact. Specific examples include simulations of temperature fluctuations and chemical diffusion. The output is the analysis results indicating hazardous areas and areas where evacuation is recommended.

[0597] Step 3:

[0598] The server generates the optimal evacuation route for each individual user based on the analyzed information. The input is the analysis results from step 2. The server utilizes the Google Maps API to calculate the optimal route while referencing road information and risk maps. Specifically, it quickly plots the route from the evacuation starting point to the safe zone on the map. The output is personalized evacuation route information for each user.

[0599] Step 4:

[0600] The device inputs the user's voice and facial features into an emotion analysis engine. Inputs include the user's voice data and facial imagery. Using this information, the device analyzes the emotional state using OpenCV and DeepFace, quantifying stress and anxiety levels. Specifically, it evaluates facial data captured by the camera in real time and performs speech recognition. The output is quantitative data indicating the user's emotional state.

[0601] Step 5:

[0602] The device provides personalized information to the user based on the analyzed emotional data. The input is the emotional data from step 4. The device uses this information to optimize the guidance messages and music it outputs. For example, for a user identified as high-stress, it plays calming music or encouraging messages. The output is adjusted information tailored to the user's state.

[0603] Step 6:

[0604] The terminal sends an alert to the server if it detects a special condition in a person in need of assistance. The input is a specific emotional state obtained in step 5. The terminal implements a mechanism to quickly aggregate data and notify the server. Specifically, it sends an emergency alert, and the server generates instructions for rapid assistance. The output is real-time notification information to support organizations.

[0605] (Application Example 2)

[0606] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0607] During disaster evacuations, a lack of proper route guidance can lead to confusion among evacuees, potentially compromising their safety. Furthermore, insufficient support tailored to the evacuees' psychological state can amplify stress and fear, hindering their evacuation efforts. Additionally, inadequate information provision to individuals in particular need of assistance can make rapid evacuation support difficult.

[0608] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0609] In this invention, the server includes information gathering means for collecting environmental data, machine learning analysis means for analyzing the collected environmental data, and route creation means for creating evacuation routes based on the analysis results. This makes it possible to provide optimized evacuation routes in real time and appropriately personalize support information based on the user's emotional state. Furthermore, by providing individualized support information to individuals who particularly need assistance, it is possible to provide assistance quickly.

[0610] "Information gathering means" refers to methods and devices for acquiring environmental information, such as sensors and drones, which play a role in collecting data.

[0611] "Machine learning analysis methods" refer to processing methods that use artificial intelligence technology to analyze collected data and design appropriate evacuation routes.

[0612] A "route creation means" is a method or device that creates the optimal evacuation route based on analysis results, generating a safe and efficient route.

[0613] "Data distribution means" refers to technologies and devices that transmit created evacuation routes to communication devices so that users can receive them.

[0614] "Visualization means" refers to methods or devices that display evacuation routes on communication devices, enabling users to visually understand the route.

[0615] "Means of acquiring emotions" refer to methods or devices for detecting a user's emotional state, which determine emotions through voice and facial expression analysis.

[0616] "Support information generation means" refers to technologies and devices for generating support information tailored to an individual based on their acquired emotional state.

[0617] "Individualized support information provision means" refers to methods or devices for providing specialized information to individuals who need support, and are used to provide appropriate support.

[0618] This invention is a system for supporting evacuees during disasters, which includes collecting and analyzing environmental data, generating evacuation routes, acquiring emotional states, and providing support information. The system is implemented using multiple hardware and software components.

[0619] First, the server collects environmental data using sensors and drones. It can collect information such as temperature, humidity, and the presence of hazardous substances, and the "pandas" library is used for data management.

[0620] Next, the collected data is analyzed using machine learning analysis tools. This analysis utilizes "TensorFlow" to understand the disaster situation and its impact, thereby generating optimal evacuation routes. The created evacuation routes are then distributed to users' communication terminals via data distribution tools.

[0621] On communication terminals, evacuation routes are displayed on a map using visualization methods. Users can then evacuate safely based on this information.

[0622] Furthermore, the device analyzes the user's voice and facial expressions through emotion acquisition mechanisms. Here, "OpenCV" and "librosa" are utilized to acquire the user's emotional state. Based on the user's emotional state, personalized messages and music are generated by the support information generation mechanism and delivered through the individual support information provision mechanism.

[0623] For example, if an earthquake occurs and the user is feeling anxious, the system can generate a message such as, "It's okay, we're guiding you to the safest route. Please proceed slowly," and play relaxing music. The prompts given to the AI ​​model might include phrases like, "Analyze the user's voice and facial expressions, assess their current emotional state, and generate an evacuation guidance message. Also, select music to help reduce stress."

[0624] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0625] Step 1:

[0626] The server collects environmental data from sensors and drones. The input consists of data such as temperature, humidity, and the presence of hazardous materials, obtained from various sensors and drones. This data is organized using "pandas" and managed as structured information. The explicit output is a dataset representing the environment of a disaster site.

[0627] Step 2:

[0628] The server analyzes the collected environmental data using TensorFlow. The input is the environmental data obtained in Step 1. Based on this, the disaster situation and its scope of impact are evaluated, and the optimal evacuation route is generated. Specifically, the data processing involves evaluating environmental risks using a machine learning model. The output is the generated evacuation route information.

[0629] Step 3:

[0630] The server transmits the generated evacuation route to the user's communication terminal using a data distribution method. The input is the evacuation route information obtained from the analysis in step 2. The specific operation when executing data transmission is real-time data delivery from the cloud server to the user terminal. The output is a route map display on the user terminal.

[0631] Step 4:

[0632] The terminal displays the evacuation route on the screen using visualization means. The input is the evacuation route information received in step 3. Here, the route is visualized using a map application and provided in a user-friendly format. The output is an intuitive evacuation route that appears on the display.

[0633] Step 5:

[0634] The device analyzes the user's voice and facial expressions using emotion acquisition methods. The input is real-time data of the user captured by the device's camera and microphone. The user's emotional state is analyzed using "OpenCV" and "librosa". The output is evaluation data of stress levels and anxiety levels indicated by the current emotional state.

[0635] Step 6:

[0636] The server generates support information tailored to the user's emotional state. The input is the emotional assessment data obtained in step 5. Based on this, the generation AI model operates to select appropriate messages and relaxing music. The prompt used is: "Analyze the user's voice and facial expressions, assess their current emotional state, and generate an evacuation guidance message. Also, select recommended music to reduce stress." The output is the content of the emotionally tailored support information.

[0637] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0638] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0639] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0640] [Fourth Embodiment]

[0641] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0642] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0643] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0644] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0645] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0646] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0647] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0648] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0649] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0650] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0651] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0652] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0653] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0654] This invention relates to a "real-time evacuation guidance system during disasters," which consists of multiple components that interact with each other to provide evacuees with safe and efficient evacuation routes.

[0655] The server uses drones and sensors to collect environmental information in real time. This includes specific data such as temperature, water level, and the presence or absence of smoke at disaster sites. Accurately understanding the rapidly changing situation is especially crucial during an ongoing disaster.

[0656] The collected data is analyzed by the server's artificial intelligence (AI) analysis system. The AI ​​detects patterns in the data and identifies the progression of the disaster and newly emerging danger areas. Based on this information, the server generates the optimal evacuation route from the user's current location. This is made possible by combining it with a Geographic Information System (GIS) to determine the shortest and safest route.

[0657] The generated evacuation routes are transmitted to the user's terminal via an information distribution system. The terminal processes the received information and provides evacuees with a visual map display and audio guidance. This allows evacuees to proceed safely in the correct direction even during a disaster. The evacuation routes are updated in real time according to the passage of time and environmental changes, ensuring that the most up-to-date information is always provided.

[0658] Furthermore, special support information is provided to those who require assistance. For example, information on barrier-free routes and areas where support is available along evacuation paths can be provided. In this way, evacuation support is provided that is tailored to individual needs.

[0659] As a concrete example, in the event of a flood, the server identifies flooded areas based on aerial data from drones. The artificial intelligence analysis system analyzes this information and instructs the user to take a safe detour route that avoids flooding, rather than the usual route. The terminal displays a map showing the route and provides voice guidance as needed. By following the terminal's instructions, the user can safely reach their evacuation destination.

[0660] By implementing these functions, the present invention can support rapid and safe evacuation during disasters and meet diverse user needs.

[0661] The following describes the processing flow.

[0662] Step 1:

[0663] The server collects environmental information from disaster sites through drones and sensors. Specifically, it acquires data such as temperature, water level, wind speed, and the presence or absence of smoke in real time and manages it centrally.

[0664] Step 2:

[0665] The server inputs the collected environmental information into an artificial intelligence analysis system. The AI ​​analyzes this data to understand the progression of the disaster, identify new hazardous areas, and predict potential problems.

[0666] Step 3:

[0667] Based on the analysis results, the server generates the optimal evacuation route from each user's current location to their destination. In doing so, it utilizes route generation methods to formulate routes while considering the safety of available roads and evacuation routes.

[0668] Step 4:

[0669] The server transmits the generated evacuation route to the user's terminal via an information distribution system. The transmitted information includes details of the evacuation route and important points to note.

[0670] Step 5:

[0671] The terminal provides users with visual and auditory guidance based on the received evacuation route information. It displays the route on a map and guides the user with voice instructions.

[0672] Step 6:

[0673] Users follow the instructions on their device and take the designated evacuation route. The device will display additional information and alerts as needed to ensure the user's safety.

[0674] Step 7:

[0675] The server updates evacuation routes in real time according to changing circumstances and sends the latest information back to the terminal. This ensures that users always have access to the most up-to-date safe routes.

[0676] Step 8:

[0677] The terminal displays special support information to those in need of assistance and, if necessary, notifies the server that rescue is required. This facilitates appropriate responses to users who need assistance.

[0678] (Example 1)

[0679] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0680] While evacuation guidance during disasters must be rapid and safe, conventional systems have difficulty responding to real-time environmental changes and accurately reflecting the needs of individual evacuees. Furthermore, insufficient data collection and analysis to accurately grasp the disaster situation make it difficult to improve the accuracy of evacuation routes.

[0681] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0682] In this invention, the server includes an information acquisition means for collecting environmental information, a data analysis means for analyzing the collected environmental information, and a route calculation means for generating evacuation routes based on the analysis results. This enables the provision of evacuation routes that respond immediately to environmental changes and the distribution of support information tailored to individual needs.

[0683] "Information acquisition means" refers to technical means for collecting environmental information during a disaster.

[0684] "Data analysis means" refers to technical means for analyzing collected environmental information to identify the progress of a disaster and dangerous areas.

[0685] A "route calculation means" is a technical means that generates the optimal evacuation route for evacuees based on analyzed data.

[0686] "Communication means" refers to the technical means for distributing the generated evacuation routes to terminal devices.

[0687] "Guidance means" refers to technical means that provide visual or audio guidance for guiding evacuees on terminal devices.

[0688] A "route updating method" is a technical means that makes it possible to change evacuation routes in real time.

[0689] "Individualized support measures" refer to technical means for providing additional support information tailored to the specific needs of evacuees.

[0690] "Remote photography means" refers to technical means for aerial photography and measurement of disaster situations.

[0691] "Safety measures" refer to technical means for identifying dangerous areas and constructing safe evacuation routes based on those areas.

[0692] This invention is an information system designed to support rapid and safe evacuation during disasters. This system consists of multiple components: a server, terminals, and users. The specific operation of each component is described below.

[0693] server

[0694] The server is designed to collect environmental information in real time. This includes weather sensors and aerial imaging equipment. For example, drones monitor disaster areas and transmit information such as temperature, water levels, and the presence of smoke to the server. The server processes this information using data analysis tools to identify the situation in the affected area and risk areas. Deep learning algorithms are used in this analysis, and generative AI models identify data patterns.

[0695] Based on the analysis results, the server uses Geographic Information System (GIS) data to calculate the optimal evacuation route. This route information is sent to terminals to enable evacuees to evacuate safely.

[0696] terminal

[0697] The terminal processes evacuation information received from the server and presents it to the user. It provides information that the user can intuitively understand through visual map displays and voice guidance. The terminal receives new information updated in real time from the server in response to changes in the situation and always provides the user with the optimal evacuation route.

[0698] User

[0699] Users can evacuate safely by receiving instructions from their devices. Furthermore, special support information is displayed for those requiring assistance. For example, the devices are designed to display information on barrier-free routes and areas where support staff are stationed.

[0700] Examples of specific cases and prompt statements

[0701] As a concrete example, in the event of a flood, the server identifies flooded areas from drone aerial data and guides users to safe detour routes that avoid the flooded areas. Users can then evacuate to a safe location while viewing a map on their device.

[0702] An example of a prompt message would be: "Please describe a system that generates the optimal evacuation route during a flood and displays it on a terminal, including accessibility information for those requiring assistance."

[0703] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0704] Step 1:

[0705] The server uses drones and sensors to collect environmental information in disaster areas. This information includes temperature, water level, and smoke. The input is raw data acquired from sensors and drones. This data is transmitted wirelessly to the server and stored in a database. The output is a dataset of collected environmental conditions.

[0706] Step 2:

[0707] The server analyzes the collected environmental data using artificial intelligence. Here, a deep learning algorithm is used to detect patterns in the data. The input is the dataset obtained in Step 1. The AI ​​model analyzes the data to identify the progression of the disaster and dangerous areas. The output is information identifying high-risk areas.

[0708] Step 3:

[0709] The server generates evacuation routes based on the analysis results. It utilizes GIS data to calculate the shortest and safest routes. The inputs are the analysis results from step 2 and the GIS data. The output is optimal route information from the user's current location to the evacuation destination. This provides an efficient evacuation route.

[0710] Step 4:

[0711] The server distributes the generated routing information to the terminal. It sends data to the terminal via a secure communication protocol. The input is the routing data generated in step 3. The output is the evacuation route information distributed to the terminal.

[0712] Step 5:

[0713] The terminal processes the received evacuation route information and provides the user with visual and audio guidance. The input is the route information sent to the terminal in step 4. The terminal displays a map on its screen, and a voice assistant provides route instructions. The output is the interface presented to the user.

[0714] Step 6:

[0715] The server continuously monitors environmental changes and updates evacuation routes in real time. The input is continuously collected environmental data. The server re-analyzes the data and generates new route information as needed. The output is the updated route information.

[0716] Step 7:

[0717] The server provides support information for vulnerable individuals. For example, it generates and sends barrier-free route information to terminals. The input is data related to the user's specific needs. The output is evacuation support information tailored to vulnerable individuals.

[0718] (Application Example 1)

[0719] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0720] In modern urban areas, there is a need for a system that allows large numbers of people to evacuate quickly and safely in the event of a disaster. However, real-time information is often lacking during disasters, making it difficult to provide appropriate evacuation routes. Furthermore, evacuation support tailored to individual needs is insufficient, posing particular challenges in providing assistance to vulnerable individuals.

[0721] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0722] In this invention, the server includes data collection means for collecting environmental data, information processing means for analyzing the collected environmental data, and data transmission means for distributing the generated evacuation routes to communication devices. This makes it possible to evaluate environmental changes in real time during a disaster and to quickly provide appropriate evacuation routes.

[0723] "Environmental data" refers to detailed information such as temperature, water level, and the presence or absence of smoke, which is necessary to understand the situation during a disaster.

[0724] "Data collection means" refers to functions for collecting environmental data using drones and sensors.

[0725] "Information processing means" refers to functions that analyze collected environmental data to identify the progress of a disaster and dangerous areas.

[0726] The "route calculation means" is a function for calculating the optimal evacuation route based on the analysis results.

[0727] "Data transmission means" refers to a function for transmitting the generated evacuation route to communication equipment.

[0728] "Information provision means" refers to a function in communication equipment that provides information to evacuees visually or audibly.

[0729] A "push notification method" is a function that notifies communication devices of new evacuation information in real time.

[0730] "Persons receiving protection" refers to people who require special assistance during a disaster.

[0731] This invention provides a system to support real-time evacuation guidance during disasters.

[0732] The server uses drones and environmental sensors to collect environmental data such as temperature, water level, and smoke presence in real time. The collected data is analyzed on an artificial intelligence platform through information processing tools. Specifically, Python and TensorFlow are used to detect data patterns and identify hazardous areas.

[0733] Based on the analysis results, the server generates the optimal evacuation route using a route calculation system. This calculation utilizes GIS technology and takes into account the safety and efficiency of the evacuation route.

[0734] The generated evacuation routes are transmitted via data transmission to the user's communication device, specifically their smartphone. The smartphone application uses React Native and provides visual map display and voice guidance. Furthermore, push notifications are used to immediately inform the user of updates to disaster information.

[0735] Users can evacuate safely by following the instructions provided by the app. In addition, those under protection will receive special support information via push notifications, including information on available support centers and barrier-free routes along the evacuation route.

[0736] As a concrete example, in the event of a flood, the server identifies the flooded area based on video data from drones and provides users with visual and audio instructions for safe detours. Through this process, users can evacuate quickly and safely.

[0737] Examples of prompt statements to input into a generative AI model are as follows:

[0738] "You are an AI expert developing a real-time disaster evacuation app for smart cities, designing a model to provide safe routes to citizens during floods. How would you use AI to perform analysis and provide users with specific evacuation routes?"

[0739] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0740] Step 1:

[0741] The server collects environmental data using drones and sensors. Specifically, it acquires sensor data such as temperature, water level, and presence of smoke, and stores it in cloud-based data storage. The input for this step is raw data from sensors, and the output is structured environmental information stored in the cloud.

[0742] Step 2:

[0743] The server analyzes the collected environmental data on an artificial intelligence platform. It uses Python and TensorFlow to analyze data patterns and estimate the progression of a disaster. The input for this step is environmental data; it recognizes data patterns and generates a report. The output is the analysis results regarding the progression of the disaster and the dangerous areas.

[0744] Step 3:

[0745] Based on the analysis results, the server generates the optimal evacuation route using a route calculation tool. It utilizes GIS technology to calculate a safe and efficient route. The input is the analysis results, and the route is calculated based on the GIS data. The output is the evacuation route determined to be optimal.

[0746] Step 4:

[0747] The server sends the generated evacuation route to a communication device such as a smartphone. The React Native application receives this information on the device and initiates a visual map display and voice guidance. The input is the evacuation route information, and the data is sent to the user's device. The output is the state where visual and voice guidance are activated.

[0748] Step 5:

[0749] The user's terminal receives real-time push notifications based on evacuation routes and informs the user of the content. Whenever disaster information is updated, the new information is immediately conveyed to the user. The input is real-time updated disaster information, and the output is voice and notification information for the user.

[0750] Step 6:

[0751] Users follow instructions from the device to evacuate safely. Especially for those under protection, necessary special support information is provided through the device. Input is the evacuation route and support information from the device, and output is the user's actual evacuation actions.

[0752] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0753] This invention combines a "real-time evacuation guidance system during disasters" with an emotion engine that recognizes the user's emotional state, enabling flexible support tailored to the psychological state of evacuees.

[0754] In the initial stages, the server collects environmental information from drones and sensors at the disaster site. This includes information such as local temperature, humidity, and the presence of hazardous materials. The server processes this data using artificial intelligence analysis to conduct a detailed analysis of the current state of the disaster and its impact.

[0755] Based on the analyzed information, the server generates an optimal evacuation route for each individual user. In this process, in addition to the usual geographic information system, road safety and risks along the evacuation route are considered. The server sends the generated route to the terminal, allowing users to receive the latest safety information in real time.

[0756] The emotion engine analyzes the user's voice, facial expressions, and input contextual information via the device to identify the user's emotional state, such as stress and anxiety levels. This emotional information is used to personalize evacuation guidance.

[0757] For example, if a user is feeling stressed, the device can adjust the guidance information to be concise and reassuring. Furthermore, based on analysis by the emotion engine, it can play relaxing music or encouraging messages for the user.

[0758] For those in need of assistance, special support information is provided via the device. For example, if a person is judged to have a high level of stress or anxiety, an alert is sent directly to the rescue team from the device, allowing for prompt assistance.

[0759] For example, if an evacuee shows extreme anxiety during evacuation guidance after an earthquake, the server receives feedback from the emotion engine and sends additional information to provide reassurance, such as advance notification of the evacuation route. In this way, flexible responses tailored to the psychological state of the evacuees become possible.

[0760] By implementing these functions, we can improve the safety and efficiency of evacuations during disasters, as well as provide psychological support to users.

[0761] The following describes the processing flow.

[0762] Step 1:

[0763] The server uses drones and sensors to collect environmental information from the disaster site. This information includes changes in terrain, weather conditions, and the presence of hazardous materials.

[0764] Step 2:

[0765] The server processes the collected environmental information using artificial intelligence analysis to identify dangerous areas and possible evacuation routes. Based on the analysis results, it generates the optimal evacuation route for evacuees.

[0766] Step 3:

[0767] The server sends the generated evacuation route information to the terminal. This information includes specific route directions and precautions to take during evacuation.

[0768] Step 4:

[0769] The terminal guides the user through the transmitted evacuation route using maps and visual displays, and provides voice guidance as needed.

[0770] Step 5:

[0771] The emotion engine built into the device analyzes the user's voice and facial expressions to detect their current emotional state. For example, it can assess the degree of stress or anxiety.

[0772] Step 6:

[0773] Based on the analysis results of the emotion engine, the device adjusts the content and method of evacuation guidance according to the user's emotional state and displays the information accordingly. For example, it might play relaxation music to reduce stress.

[0774] Step 7:

[0775] Users follow the instructions on their device and proceed along the designated evacuation route. If they feel anxious during the evacuation, the device will continue to provide support in real time.

[0776] Step 8:

[0777] The device provides support information specifically tailored to those in need of assistance and, when necessary, sends emergency notifications to rescue teams. These notifications are based on user state analysis using an emotion engine.

[0778] Step 9:

[0779] The server continuously updates evacuation routes based on changes in the disaster situation and new environmental information, and sends the latest information to terminals. This allows users to continue evacuating safely according to the situation.

[0780] (Example 2)

[0781] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0782] Conventional evacuation guidance systems have the challenge of not being able to provide sufficient safety and psychological stability because they struggle to respond flexibly and immediately based on the situation during a disaster and the emotional state of individual evacuees. Furthermore, they lack special consideration and support information for vulnerable individuals, and further improvements are needed.

[0783] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0784] In this invention, the server includes means for collecting environmental data, information analysis means for analyzing the collected environmental data, and route design means for generating routes based on the analysis results. This enables real-time information collection and analysis, and the generation of optimal evacuation routes during disasters. Furthermore, by providing emotion analysis means and adjustment means for individually responding to information according to the emotional state of the user, it is possible to enhance the psychological stability of evacuees and provide prompt and appropriate support to those in need of assistance.

[0785] "Environmental data" refers to information about disaster sites, such as temperature, humidity, and the presence of hazardous materials, collected using drones and sensors.

[0786] "Information analysis methods" refer to means of analyzing environmental data collected using artificial intelligence technology to understand the current state and impact of a disaster.

[0787] A "route design means" is a system that generates evacuation routes suitable for individual users based on analyzed information.

[0788] "Information transmission means" refers to a means of transmitting generated evacuation routes to the user's terminal and providing necessary information in real time.

[0789] A "display function" is a function on a terminal that visually presents information to the user to support evacuation.

[0790] "Emotional analysis tools" are means of analyzing a user's voice and facial expressions to recognize and evaluate their emotional state.

[0791] "Adjustment measures" refer to methods for optimizing evacuation guidance and information provision according to the emotional state of users, and for providing individualized support.

[0792] A "coordination method" refers to a system that provides special support information to those in need of assistance and notifies support organizations as needed.

[0793] This invention is a system that combines a real-time evacuation guidance system for disaster situations with an emotion analysis function that recognizes the emotional state of the user.

[0794] The server collects environmental data from drones and fixed sensors to understand the current situation at the disaster site. During this process, the server uses high-precision sensors and drones to obtain specific data such as temperature, humidity, and the presence of hazardous materials. The collected data is processed using artificial intelligence libraries such as TensorFlow and PyTorch to analyze the scale and extent of the disaster.

[0795] The server uses geographic information systems such as the Google Maps API to generate the optimal evacuation route for each individual evacuee based on the analyzed information. The evacuation route design takes into account road safety and risks along the route. The generated route information is transmitted to the terminal in real time.

[0796] The device supplies voice and facial expressions from the user to an emotion analysis engine to check the user's emotional state. Specifically, the device uses libraries such as OpenCV and DeepFace to identify the emotional state. This allows it to determine the degree to which the user is experiencing stress or anxiety, and sends this data to the server.

[0797] Based on user emotion data, the device provides information best suited to the situation. For example, if the user is feeling stressed, the device adjusts the voice guidance to be calm and reassuring, and changes the music and messages to be encouraging. As an example of a prompt, the AI ​​generation model can be asked the question, "What message should be provided to reassure evacuees who are feeling anxious during an earthquake?" and an answer can be obtained.

[0798] This invention improves the safety and efficiency of providing information regarding evacuation during disasters, and also provides psychological support to users.

[0799] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0800] Step 1:

[0801] The server collects environmental data from drones and sensors. Inputs include on-site temperature, humidity, and the presence of hazardous materials. The server begins by receiving this data from various sources and storing it in a database. Specifically, it implements a system to periodically poll sensor data via an API and update it in real time. The output will be a basic dataset showing the current situation at the disaster site.

[0802] Step 2:

[0803] The server processes the collected environmental data using information analysis tools. The input is the dataset obtained in Step 1. Based on this, the server uses TensorFlow and PyTorch to perform anomaly detection and predict the extent of impact. Specific examples include simulations of temperature fluctuations and chemical diffusion. The output is the analysis results indicating hazardous areas and areas where evacuation is recommended.

[0804] Step 3:

[0805] The server generates the optimal evacuation route for each individual user based on the analyzed information. The input is the analysis results from step 2. The server utilizes the Google Maps API to calculate the optimal route while referencing road information and risk maps. Specifically, it quickly plots the route from the evacuation starting point to the safe zone on the map. The output is personalized evacuation route information for each user.

[0806] Step 4:

[0807] The device inputs the user's voice and facial features into an emotion analysis engine. Inputs include the user's voice data and facial imagery. Using this information, the device analyzes the emotional state using OpenCV and DeepFace, quantifying stress and anxiety levels. Specifically, it evaluates facial data captured by the camera in real time and performs speech recognition. The output is quantitative data indicating the user's emotional state.

[0808] Step 5:

[0809] The device provides personalized information to the user based on the analyzed emotional data. The input is the emotional data from step 4. The device uses this information to optimize the guidance messages and music it outputs. For example, for a user identified as high-stress, it plays calming music or encouraging messages. The output is adjusted information tailored to the user's state.

[0810] Step 6:

[0811] The terminal sends an alert to the server if it detects a special condition in a person in need of assistance. The input is a specific emotional state obtained in step 5. The terminal implements a mechanism to quickly aggregate data and notify the server. Specifically, it sends an emergency alert, and the server generates instructions for rapid assistance. The output is real-time notification information to support organizations.

[0812] (Application Example 2)

[0813] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0814] During disaster evacuations, a lack of proper route guidance can lead to confusion among evacuees, potentially compromising their safety. Furthermore, insufficient support tailored to the evacuees' psychological state can amplify stress and fear, hindering their evacuation efforts. Additionally, inadequate information provision to individuals in particular need of assistance can make rapid evacuation support difficult.

[0815] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0816] In this invention, the server includes information gathering means for collecting environmental data, machine learning analysis means for analyzing the collected environmental data, and route creation means for creating evacuation routes based on the analysis results. This makes it possible to provide optimized evacuation routes in real time and appropriately personalize support information based on the user's emotional state. Furthermore, by providing individualized support information to individuals who particularly need assistance, it is possible to provide assistance quickly.

[0817] "Information gathering means" refers to methods and devices for acquiring environmental information, such as sensors and drones, which play a role in collecting data.

[0818] "Machine learning analysis methods" refer to processing methods that use artificial intelligence technology to analyze collected data and design appropriate evacuation routes.

[0819] A "route creation means" is a method or device that creates the optimal evacuation route based on analysis results, generating a safe and efficient route.

[0820] "Data distribution means" refers to technologies and devices that transmit created evacuation routes to communication devices so that users can receive them.

[0821] "Visualization means" refers to methods or devices that display evacuation routes on communication devices, enabling users to visually understand the route.

[0822] "Means of acquiring emotions" refer to methods or devices for detecting a user's emotional state, which determine emotions through voice and facial expression analysis.

[0823] "Support information generation means" refers to technologies and devices for generating support information tailored to an individual based on their acquired emotional state.

[0824] "Individualized support information provision means" refers to methods or devices for providing specialized information to individuals who need support, and are used to provide appropriate support.

[0825] This invention is a system for supporting evacuees during disasters, which includes collecting and analyzing environmental data, generating evacuation routes, acquiring emotional states, and providing support information. The system is implemented using multiple hardware and software components.

[0826] First, the server collects environmental data using sensors and drones. It can collect information such as temperature, humidity, and the presence of hazardous substances, and the "pandas" library is used for data management.

[0827] Next, the collected data is analyzed using machine learning analysis tools. This analysis utilizes "TensorFlow" to understand the disaster situation and its impact, thereby generating optimal evacuation routes. The created evacuation routes are then distributed to users' communication terminals via data distribution tools.

[0828] On communication terminals, evacuation routes are displayed on a map using visualization methods. Users can then evacuate safely based on this information.

[0829] Furthermore, the device analyzes the user's voice and facial expressions through emotion acquisition mechanisms. Here, "OpenCV" and "librosa" are utilized to acquire the user's emotional state. Based on the user's emotional state, personalized messages and music are generated by the support information generation mechanism and delivered through the individual support information provision mechanism.

[0830] For example, if an earthquake occurs and the user is feeling anxious, the system can generate a message such as, "It's okay, we're guiding you to the safest route. Please proceed slowly," and play relaxing music. The prompts given to the AI ​​model might include phrases like, "Analyze the user's voice and facial expressions, assess their current emotional state, and generate an evacuation guidance message. Also, select music to help reduce stress."

[0831] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0832] Step 1:

[0833] The server collects environmental data from sensors and drones. The input consists of data such as temperature, humidity, and the presence of hazardous materials, obtained from various sensors and drones. This data is organized using "pandas" and managed as structured information. The explicit output is a dataset representing the environment of a disaster site.

[0834] Step 2:

[0835] The server analyzes the collected environmental data using TensorFlow. The input is the environmental data obtained in Step 1. Based on this, the disaster situation and its scope of impact are evaluated, and the optimal evacuation route is generated. Specifically, the data processing involves evaluating environmental risks using a machine learning model. The output is the generated evacuation route information.

[0836] Step 3:

[0837] The server transmits the generated evacuation route to the user's communication terminal using a data distribution method. The input is the evacuation route information obtained from the analysis in step 2. The specific operation when executing data transmission is real-time data delivery from the cloud server to the user terminal. The output is a route map display on the user terminal.

[0838] Step 4:

[0839] The terminal displays the evacuation route on the screen using visualization means. The input is the evacuation route information received in step 3. Here, the route is visualized using a map application and provided in a user-friendly format. The output is an intuitive evacuation route that appears on the display.

[0840] Step 5:

[0841] The device analyzes the user's voice and facial expressions using emotion acquisition methods. The input is real-time data of the user captured by the device's camera and microphone. The user's emotional state is analyzed using "OpenCV" and "librosa". The output is evaluation data of stress levels and anxiety levels indicated by the current emotional state.

[0842] Step 6:

[0843] The server generates support information tailored to the user's emotional state. The input is the emotional assessment data obtained in step 5. Based on this, the generation AI model operates to select appropriate messages and relaxing music. The prompt used is: "Analyze the user's voice and facial expressions, assess their current emotional state, and generate an evacuation guidance message. Also, select recommended music to reduce stress." The output is the content of the emotionally tailored support information.

[0844] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0845] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0846] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0847] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0848] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0849] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0850] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0851] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0852] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0853] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0854] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0855] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0856] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0857] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0858] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0859] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0860] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0861] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0862] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0863] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0864] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0865] The following is further disclosed regarding the embodiments described above.

[0866] (Claim 1)

[0867] Data collection methods for collecting environmental information,

[0868] An artificial intelligence analysis tool for analyzing collected environmental information,

[0869] Route generation means for generating evacuation routes based on analysis results,

[0870] Information distribution means for distributing generated evacuation routes to communication terminals,

[0871] A display means for guiding evacuees on a terminal,

[0872] A system that includes this.

[0873] (Claim 2)

[0874] The system according to claim 1, comprising means for updating evacuation routes in real time based on analysis results.

[0875] (Claim 3)

[0876] The system according to claim 1, comprising means for providing special support information to persons in need of assistance.

[0877] "Example 1"

[0878] (Claim 1)

[0879] Information acquisition methods for collecting environmental information,

[0880] A data analysis method for analyzing collected environmental information,

[0881] A route calculation means for generating evacuation routes based on analysis results,

[0882] A communication means for distributing the generated evacuation route to terminal devices,

[0883] A means of guiding evacuees on a terminal device,

[0884] A route update method for updating evacuation routes in real time,

[0885] Individualized support means to provide support information tailored to individual needs,

[0886] A system that includes this.

[0887] (Claim 2)

[0888] The system according to claim 1, comprising remote photography means for aerial photography and measurement of disaster conditions.

[0889] (Claim 3)

[0890] The system according to claim 1, comprising safety measures for identifying dangerous areas and constructing safe routes based on optimized information.

[0891] "Application Example 1"

[0892] (Claim 1)

[0893] Data collection means for collecting environmental data,

[0894] Information processing means for analyzing collected environmental data,

[0895] A route calculation means for generating evacuation routes based on analysis results,

[0896] A data transmission means for distributing the generated evacuation routes to communication devices,

[0897] A means of providing guidance to evacuate persons in communication equipment,

[0898] A notification method for providing information to a device via push notifications,

[0899] A system that includes this.

[0900] (Claim 2)

[0901] The system according to claim 1, comprising means for updating evacuation routes in real time based on analysis results and sending information via push notification.

[0902] (Claim 3)

[0903] The system according to claim 1, comprising means for providing special support information to a protected person.

[0904] "Example 2 of combining an emotion engine"

[0905] (Claim 1)

[0906] Means of collecting environmental data,

[0907] Information analysis means for analyzing collected environmental data,

[0908] A path design means for generating a path based on the analysis results,

[0909] Information transmission means for sending the generated route to the terminal,

[0910] Display functions to assist users on the device,

[0911] A means of analyzing emotions to recognize the emotional state of the user,

[0912] A means of adjusting information to suit the user's emotional state,

[0913] A system that includes this.

[0914] (Claim 2)

[0915] The system according to claim 1, comprising a function to optimize evacuation routes in real time based on analysis results and user sentiment information.

[0916] (Claim 3)

[0917] The system according to claim 1, comprising means for providing special support information to persons in need of assistance and for providing prompt notification to support organizations.

[0918] "Application example 2 when combining with an emotional engine"

[0919] (Claim 1)

[0920] Information gathering means for collecting environmental data,

[0921] A machine learning analysis method for analyzing collected environmental data,

[0922] A means for creating evacuation routes based on analysis results,

[0923] A data distribution means for distributing the generated evacuation routes to a communication device,

[0924] A means of visualization for guiding evacuees in a communication device,

[0925] A means of acquiring the user's emotional state,

[0926] A means for generating support information corresponding to an emotional state,

[0927] A system that includes this.

[0928] (Claim 2)

[0929] The system according to claim 1, comprising means for correcting evacuation routes in real time based on analysis results.

[0930] (Claim 3)

[0931] The system according to claim 1, comprising means for providing individualized support information to provide support information tailored to individuals in need of support. [Explanation of Symbols]

[0932] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Data collection means for collecting environmental data, Information processing means for analyzing collected environmental data, A route calculation means for generating evacuation routes based on analysis results, A data transmission means for distributing the generated evacuation routes to communication devices, A means of providing guidance to evacuate persons in communication equipment, A notification method for providing information to a device via push notifications, A system that includes this.

2. The system according to claim 1, comprising means for updating evacuation routes in real time based on analysis results and sending information via push notification.

3. The system according to claim 1, comprising means for providing special support information to a protected person.

Images