System

Abstract

A system is provided.SOLUTION: A system includes a means for detecting a disaster occurrence, a means for activating a plurality of unmanned aerial vehicles and collecting video and image data of a disaster area, a means for analyzing the collected video and image data and generating a three-dimensional map of a disaster situation, a means for visualizing the generated three-dimensional map and distributing a countermeasure instruction, a means for receiving information from residents of a shelter and rescue workers and extracting important information by natural language analysis, and a means for determining a rescue activity instruction and a transport route of support materials on the basis of the extracted information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When a disaster occurs, it is necessary to quickly and accurately grasp the extent of the damage and to carry out appropriate relief activities based on that information. However, with conventional methods, it takes time to gather information, which often delays relief activities. It is also not possible to adequately grasp the situation in dangerous areas where people cannot enter, or to gather information on needs at evacuation centers. To solve this problem, a system is needed that can quickly grasp the extent of the damage and support effective relief activities. [Means for solving the problem]

[0005] In order to solve the above problems, the present invention provides the following means.

[0006] The system is equipped with a means for detecting the occurrence of a disaster, a means for launching multiple unmanned aerial vehicles and collecting video and image data of the affected area, a means for analyzing the collected video and image data and generating a three-dimensional map of the damage situation, a means for visualizing the generated three-dimensional map and distributing guidance on countermeasures, a means for receiving information from residents of evacuation centers and relief workers and extracting important information through natural language analysis, and a means for issuing instructions for relief operations and determining routes for transporting relief supplies based on the extracted information.

[0007] This system will enable us to quickly and accurately grasp the damage situation immediately after a disaster occurs, enabling effective relief efforts and the distribution of appropriate relief supplies. In addition, by utilizing unmanned aerial vehicles and generative AI, we will be able to efficiently grasp the situation in areas where people cannot enter.

[0008] "Means for detecting the occurrence of a disaster" refers to a device or program that detects the occurrence of a natural disaster such as an earthquake, heavy rain, or flood, and uses that as a trigger to start various operations of the system.

[0009] An "unmanned aerial vehicle" is an aircraft that can be remotely operated or fly autonomously without a human on board, and is also known as a drone.

[0010] "Video and image data" refers to digital data of videos and still images taken by unmanned aerial vehicles.

[0011] "Means for collection" refers to a device or program for acquiring video and image data using a sensor such as a camera mounted on an unmanned aerial vehicle.

[0012] The "analyzing means" refers to a program or algorithm for processing the acquired video and image data and analyzing the damage situation.

[0013] A "three-dimensional map" is a digital map that shows the topography of the affected area and structures such as buildings in three dimensions.

[0014] The "means for visualization and delivery of countermeasure guidance" refers to a device or program for visually displaying the generated three-dimensional map and sending disaster prevention instructions based on it to each relevant party.

[0015] "Shelter residents and relief workers" refer to people living in shelters and those engaged in relief efforts on site when a disaster occurs.

[0016] The "means for receiving information and extracting important information through natural language analysis" refers to a program that receives text messages sent by evacuation center residents and relief workers, analyzes their contents using natural language processing technology, and extracts necessary information.

[0017] The "means for determining instructions for relief operations and routes for transporting relief supplies" is a device or program for calculating and determining optimal instructions for relief operations and routes for transporting relief supplies based on the analyzed information. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The disaster response system of the present invention includes various hardware and software components that enable rapid and efficient assessment of the damage situation and appropriate rescue operations when a disaster occurs. The main components of this system include a disaster detection means, an unmanned aerial vehicle (drone), a server, a generation AI, and a user interface.

[0040] 1. Disaster detection methods

[0041] The server collects information from weather data, earthquake sensors, flood monitoring sensors, etc. in order to monitor the occurrence of natural disasters such as earthquakes, heavy rains, and floods in real time. If it determines that a disaster has occurred based on this information, the entire system will immediately begin operating.

[0042] 2. Launching and flying unmanned aerial vehicles (drones)

[0043] When the server detects a disaster, it sends instructions to launch multiple drones in a designated area, which follow automated flight routes and use their cameras to collect video and image data of the affected area.

[0044] 3. Data collection and transmission

[0045] The drone (terminal) transmits high-resolution video and image data acquired during flight to a server in real time, enabling a rapid assessment of the damage situation.

[0046] 4. Data Analysis and 3D Map Generation

[0047] The server stores the video and image data sent from the drone in cloud storage and inputs it into the generation AI. The generation AI compares the data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. Based on the results of this analysis, it generates a 3D map of the affected area.

[0048] 5. Visualizing the damage situation and providing guidance on countermeasures

[0049] The server visualizes and displays the 3D map generated by the AI ​​on the terminals at the emergency response headquarters. This allows for a visual understanding of the damage situation and enables quick decision-making. Furthermore, it automatically generates countermeasure guidance based on the analysis results and distributes it to all parties involved.

[0050] 6. Collecting and analyzing information from the field

[0051] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server. The server receives the information, and the generative AI performs natural language analysis to extract important information.

[0052] 7. Directing relief operations and determining routes for transporting relief supplies

[0053] Based on the extracted information, the server updates the database with the status of relief supplies needed at each evacuation shelter, determines instructions for relief operations and routes for delivering relief supplies, and transmits the instructions and route information to relief workers on the ground in real time.

[0054] 8. Specific Examples

[0055] For example, if a large-scale earthquake occurs, the server immediately launches multiple drones to fly over areas with severe damage. The drones collect video and image data, which the server inputs into the generation AI in real time. The generation AI then creates a three-dimensional map of the affected area, identifying collapsed buildings and disrupted roads. At the same time, users submit requests for relief supplies from evacuation centers, and this information is used to determine the optimal route for quickly delivering the necessary supplies. The server then distributes this information to relief workers, ensuring that appropriate relief efforts are carried out.

[0056] In this way, this system can collect information quickly and accurately when a disaster occurs, and support effective relief efforts.

[0057] The processing flow will be explained below.

[0058] Step 1:

[0059] The server collects and monitors weather data, earthquake sensors, and flood monitoring sensors in real time to detect the occurrence of disasters. When a disaster is detected, it immediately issues instructions to start operating the entire system.

[0060] Step 2:

[0061] When the server detects a disaster, it sends activation instructions to multiple unmanned aerial vehicles (drones) located within a pre-defined area.

[0062] Step 3:

[0063] Once the drone (terminal) receives a start command from the server, it begins flying within the designated area according to an automatic flight route. The drone uses a camera to collect high-resolution video and image data.

[0064] Step 4:

[0065] The drone (terminal) transmits video and image data collected during flight to a server in real time.

[0066] Step 5:

[0067] The server stores the video and image data received from the drone in cloud storage.

[0068] Step 6:

[0069] The server inputs the video and image data stored in cloud storage into the generation AI, which analyzes the damage situation.

[0070] Step 7:

[0071] The generating AI compares data before and after the disaster, automatically detecting building damage, road disruption, and the extent of flooding, and generates a three-dimensional map of the affected area.

[0072] Step 8:

[0073] The server visualizes the generated three-dimensional map and analysis results and displays them on a terminal at the emergency response headquarters.

[0074] Step 9:

[0075] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server.

[0076] Step 10:

[0077] The server receives messages sent by users and uses generative AI to perform natural language analysis and extract important information.

[0078] Step 11:

[0079] Based on the analyzed information, the server updates the database with the status of relief supplies at each evacuation shelter.

[0080] Step 12:

[0081] The server uses generative AI based on the information in the database to determine rescue operation instructions and the optimal route for transporting relief supplies.

[0082] Step 13:

[0083] The server transmits the determined rescue operation instructions and information on the route for transporting relief supplies to rescue workers on the scene in real time.

[0084] Step 14:

[0085] The relief workers (users) carry out their activities on site in accordance with the received relief operation instructions and the delivery route of relief supplies.

[0086] This system will enable rapid and accurate collection of information in the event of a disaster, enabling appropriate and effective relief efforts to be carried out.

[0087] Example 1

[0088] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0089] The problem is that when a disaster occurs, there is a lack of means to collect and analyze information to quickly and accurately grasp the damage situation and carry out effective relief activities. With conventional methods, it takes time to accurately grasp the damage situation on the ground, and the accuracy of the information is limited, making it difficult to formulate appropriate relief operation plans.

[0090] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0091] In this invention, the server includes means for detecting the occurrence of a disaster, means for launching multiple unmanned aerial vehicles and collecting video and image data of the affected area, means for analyzing the collected video and image data and generating a three-dimensional map of the damage situation, means for visualizing the generated three-dimensional map and distributing countermeasure instructions, means for receiving information from evacuation shelter residents and relief workers and extracting important information through natural language analysis, means for determining rescue operation instructions and delivery routes for relief supplies based on the extracted information, means for automatically analyzing the damage situation and providing countermeasure instructions using a generative AI model, means for immediately recognizing the occurrence of a disaster and emergency activation of the system, and means for saving data captured by the unmanned aerial vehicle's camera in real time to cloud storage. This enables rapid and accurate information collection and analysis, supporting effective rescue operations.

[0092] "Means for detecting the occurrence of disasters" refers to devices or systems that monitor the occurrence of natural disasters such as earthquakes, heavy rains, and floods in real time, collect and analyze data, and determine whether a disaster has occurred.

[0093] "Multiple unmanned aerial vehicles" means an aircraft system that includes drones that are flown by remote control or autonomous pilot and used to monitor a specific area.

[0094] "Video and image data of the affected areas" refers to video and photographic data taken by unmanned aerial vehicles to show the current state of the areas affected by the disaster.

[0095] The "means of collection" is a communication device or protocol for transferring video and image data captured by the unmanned aerial vehicle to a specific server or cloud storage.

[0096] "Means for generating a three-dimensional map" refers to software or algorithms that analyze collected video and image data and recreate the extent and details of the damage in three-dimensional space.

[0097] The "means for visualization and delivery of countermeasure guidance" is a system that displays the generated three-dimensional map on a terminal display and notifies relevant parties of rescue operation instructions and countermeasures.

[0098] "Means for extracting important information through natural language analysis" refers to algorithms or software that analyze text data sent by evacuation shelter residents and relief workers and automatically extract important information necessary for providing assistance.

[0099] The "means for determining rescue operation instructions and transportation routes for relief supplies" is a system that uses the extracted information to create a list of relief supplies needed at each evacuation shelter and makes a plan for transporting them via the optimal route.

[0100] "Means for automatically analyzing damage situations and providing guidance on countermeasures using generative AI models" refers to machine learning models or AI systems that analyze damage situations based on collected data and automatically propose optimal countermeasures.

[0101] "Means for immediately recognizing the occurrence of a disaster and for emergency activation of the system" refers to a combination of hardware and software that immediately activates the entire system and initiates necessary measures when a disaster is detected.

[0102] "Data captured by the camera of an unmanned aerial vehicle" refers to visual information collected through a camera mounted on an unmanned aerial vehicle, and is a means of storing this information in cloud storage in real time.

[0103] The present invention is directed to a system that aims to quickly and accurately grasp the damage situation when a disaster occurs and to support effective relief efforts. The present invention is configured by combining various hardware and software components.

[0104] Disaster detection

[0105] To detect the occurrence of a disaster, the server collects and analyzes real-time data from weather data, earthquake sensors, flood monitoring sensors, etc. This is done using, for example, the Japan Meteorological Agency's API or specific sensor devices. When the server detects data that exceeds a certain threshold, it recognizes the occurrence of a disaster and issues a command to immediately start up the operation of the entire system. Specifically, it uses software that immediately recognizes the occurrence of a disaster and immediately starts up the system.

[0106] Launching and flying unmanned aerial vehicles

[0107] The server launches multiple unmanned aerial vehicles (drones) via drone control software in areas where a disaster has been confirmed. These drones follow automated flight routes and use cameras to collect video and image data of the affected area. The drones are equipped with high-resolution camera modules and fly autonomously along pre-programmed flight paths. Flight control is performed using the DroneKit API, for example.

[0108] Data collection and transmission

[0109] The drone (terminal) transmits high-resolution video and image data acquired during flight to a server in real time over a 5G network, and the data is stored in a cloud storage service, specifically Amazon S3.

[0110] Data analysis and 3D map generation

[0111] The server inputs the video and image data stored in cloud storage into a generative AI model. This generative AI model was developed using Tensorflow (registered trademark) and other software, and analyzes the extent and details of the damage. Based on the analysis results, a three-dimensional map of the affected area is generated. This map is saved for further detailed analysis and is linked to the emergency response headquarters terminal.

[0112] Visualization of damage status and delivery of countermeasure guidance

[0113] The server then displays the generated 3D map on the device's display, using a visualization tool such as Tableau. The automated guidance generated by the AI ​​model is then distributed to relevant parties via the SendGrid API.

[0114] Collecting and analyzing information from the field

[0115] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server. This application is a mobile app using Firebase. The server receives this information and performs natural language analysis using a generative AI model, which automatically extracts important information.

[0116] Directing relief operations and determining routes for transporting relief supplies

[0117] The server updates a database with the status of relief supplies needed at each evacuation center based on the information extracted through natural language analysis. It then calculates optimal delivery routes using the Google® Maps API and generates rescue operation instructions. This information is then sent in real time to rescue workers on the ground using Firebase Cloud Messaging.

[0118] Specific examples

[0119] For example, if a large-scale earthquake occurs, the server immediately recognizes the occurrence of a disaster based on data obtained from the Japan Meteorological Agency API and launches multiple drones using the DroneKit API. The drones store the acquired data in Amazon S3 via the 5G network, and this data is input into a generative AI model using TensorFlow. The analysis results are displayed in Tableau and distributed to relevant parties via the SendGrid API. Requests from evacuation centers are received through the Firebase app, and the information is analyzed using a natural language analysis model. The optimal route for delivering relief supplies is calculated using the Google Maps API and sent to relief workers via Firebase Cloud Messaging.

[0120] Prompt Sentence Examples

[0121] "Please tell me the detailed operation procedure of the system that quickly grasps the damage situation in an area where an earthquake has occurred and uses a generative AI model to plan rescue operations."

[0122] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0123] Step 1: Detecting the occurrence of a disaster

[0124] The server collects data in real time from sources such as weather data, earthquake sensors, and flood monitoring sensors. Specifically, it takes in data obtained from APIs and sensor devices every minute. Statistical analysis is performed based on this data, and if a certain threshold is exceeded, a disaster is recognized. Various disaster data is input, and a disaster occurrence signal is generated as output.

[0125] Step 2: Launch command for unmanned aerial vehicles

[0126] When the server confirms that a disaster has occurred, it sends instructions to launch multiple unmanned aerial vehicles (drones). This instruction is sent via drone control software (e.g., DroneKit API). The input is a disaster occurrence signal, and the output is a command to launch the drones. Specific operations include setting an automatic flight route based on GPS coordinates and activating cameras.

[0127] Step 3: Video and image data collection

[0128] The drone (terminal) collects high-resolution video and image data while flying along an automatic flight route. It uses a camera module (e.g., DJI Zenmuse X7) to capture images along the specified route. The input is the set flight route information, and the output is high-resolution video and image data.

[0129] Step 4: Sending data in real time

[0130] The drone transmits the collected video and image data to a server in real time. The communication method is a 5G network. The input is high-resolution video and image data, and the output is the data sent to the server. Specifically, the data is packetized and transmitted.

[0131] Step 5: Save your data to the cloud

[0132] The server stores the received video and image data in a cloud storage service (e.g., Amazon S3). The input is data sent in real time, and the output is data stored in the cloud storage. Specifically, the data is stream-processed and converted into files.

[0133] Step 6: Generate a 3D map

[0134] The server inputs the data stored in cloud storage into a generative AI model and analyzes the damage situation. Using the generative AI model (e.g., TensorFlow), it compares data before and after the disaster and generates a 3D map. The input is the stored video and image data, and the output is a 3D map. Specific operations include preprocessing the data, inputting it into the trained model, and outputting the analysis results.

[0135] Step 7: Visualize the damage situation

[0136] The server displays the generated 3D map on the emergency headquarters terminal, allowing the damage situation to be visually grasped. The input is the generated 3D map data, and the output is the map displayed on the terminal. Specifically, the 3D map data is rendered and displayed.

[0137] Step 8: Automatic generation and distribution of countermeasure guidance

[0138] The server uses a generative AI model to automatically generate countermeasure guidance based on the analysis results and distributes it to relevant parties. The input is the analysis results and a 3D map, and the output is the automatically generated countermeasure guidance and distributed guidance message. Specific operations involve the use of an email distribution system (e.g., SendGrid API).

[0139] Step 9: Gather information from the field

[0140] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for relief supplies to the server. The input is text, images, and audio data, and the output is the information sent to the server. Specific operations involve the use of a mobile app (e.g., Firebase app).

[0141] Step 10: Extracting important information through natural language analysis

[0142] The server uses a generative AI model to perform natural language analysis on the received local information and extract important information. The input is the transmitted text, images, and audio data, and the output is the extracted important information. The specific operation involves applying a natural language processing algorithm.

[0143] Step 11: Deciding on relief operation instructions and relief supply transport routes

[0144] The server updates the database based on the extracted information and determines rescue operation instructions and delivery routes for relief supplies. The input is important information and the current supply situation, and the output is optimized delivery routes and rescue instructions. Specifically, a route calculation algorithm (e.g., Google Maps API) is applied.

[0145] Step 12: Send instructions to relief workers

[0146] The server sends the determined instructions and transport route information to the rescue workers on the scene via push notification. The input is the rescue instructions and transport route, and the output is the information sent to the rescue workers' devices. The specific operation is to send a push notification (e.g., Firebase Cloud Messaging).

[0147] (Application example 1)

[0148] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0149] When a disaster occurs, it is important to quickly and accurately grasp the damage situation. To quickly and appropriately carry out relief efforts, it is necessary to collect and analyze information from various evacuation centers and relief activity sites. However, the current system does not efficiently coordinate the understanding of the damage situation and instructions for relief efforts, making it particularly difficult to optimize the transportation routes of supplies. Furthermore, logistics centers do not have the appropriate means to rapidly distribute relief supplies.

[0150] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0151] In this invention, the server includes means for detecting the occurrence of a disaster, means for launching multiple unmanned aerial vehicles to collect video and image data of the affected area, means for analyzing the collected video and image data and generating a 3D map of the damage situation, and means for displaying the generated 3D map on a terminal at the logistics center, and for quickly managing requests for relief supplies and optimizing delivery routes. This enables a detailed understanding of the damage situation in the event of a disaster and efficient distribution of relief supplies via the logistics center.

[0152] "Means for detecting the occurrence of disasters" refers to a system that monitors and notifies in real time the occurrence of natural disasters such as earthquakes, heavy rain, and floods.

[0153] An "unmanned aerial vehicle" is a device, such as a drone, that flies remotely or autonomously and collects video and image data of a specific area.

[0154] "Video and image data" refers to video and photographic data used to visually record the situation in the affected areas.

[0155] A "three-dimensional map" is a map that displays the state of the terrain and buildings before and after a disaster in three dimensions.

[0156] A "generative AI model" is an artificial intelligence system that analyzes collected data and generates new data and information.

[0157] The "means for delivering guidance on countermeasures" is a system that automatically notifies each relevant party of appropriate countermeasures and instructions based on the analyzed data.

[0158] "Means for receiving information from evacuation center residents and relief workers" refers to a system for receiving information on the local situation and requests for relief supplies from evacuation centers and local relief workers.

[0159] "Natural language analysis" is a technology for analyzing received information and extracting important information.

[0160] A "relief supply transport route" is a route for efficiently delivering necessary relief supplies to evacuation centers and disaster-stricken areas.

[0161] A "logistics center" is a facility for distributing and managing relief supplies.

[0162] A "terminal" is a computer system for displaying and manipulating information.

[0163] "Relief supply request management" is the process of receiving requests for relief supplies from evacuation centers and disaster-stricken areas and managing the necessary supplies.

[0164] "Route optimization" is the process of planning and executing the safe delivery of relief supplies via the shortest and most optimal route.

[0165] This invention provides a system for quickly and accurately assessing the damage situation when a disaster occurs, and for efficiently carrying out appropriate relief activities and transporting relief supplies. The system includes the following main hardware and software components:

[0166] 1. Means of detecting disasters

[0167] The server monitors disasters such as earthquakes, heavy rains, and floods in real time, collecting information from weather data, earthquake sensors, flood monitoring sensors, etc. If it determines that a disaster has occurred, it immediately starts operating the entire system.

[0168] 2. Starting and flying the drone

[0169] When the server detects a disaster, it sends instructions to launch multiple unmanned aerial vehicles in a designated area. The drones follow automated flight routes and use their onboard cameras to collect video and image data of the affected area.

[0170] 3. Data collection and transmission

[0171] The drones capture high-resolution video and image data during flight and transmit it to a server in real time, enabling a rapid assessment of the damage situation.

[0172] 4. Data Analysis and 3D Map Generation

[0173] The server stores the video and image data sent from the drone in cloud storage (AWS (registered trademark) S3) and inputs it into a generative AI model (OpenAI (registered trademark) API). The generative AI compares data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. Based on the results of this analysis, it generates a 3D map of the affected area.

[0174] 5. Visualizing the damage situation and providing guidance on countermeasures

[0175] The server visualizes and displays the 3D map generated by the AI ​​on terminals at the logistics center and the emergency response headquarters. This allows for a visual understanding of the damage situation, enabling quick decision-making. Furthermore, guidance on countermeasures based on the analysis results is distributed to all parties in real time.

[0176] 6. Collecting and analyzing information from the field

[0177] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server. The server receives the information, and the generative AI performs natural language analysis to extract important information.

[0178] 7. Directing relief operations and determining routes for transporting relief supplies

[0179] Based on the extracted information, the server updates the database with the status of relief supplies needed at each evacuation shelter, determines optimal instructions for relief efforts, and determines the optimal route for delivering relief supplies. The determined instructions and route information are sent in real time to relief workers on the ground.

[0180] Specific examples

[0181] For example, if a large-scale earthquake occurs, the server immediately launches multiple drones to fly over areas with severe damage. The drones collect video and image data, which the server inputs into the generation AI in real time. The generation AI generates a three-dimensional map of the affected area and identifies collapsed buildings and disrupted roads. At the same time, users submit requests for relief supplies from evacuation centers, and this information is used to determine the optimal route for quickly delivering the necessary supplies. The server then distributes this information to relief workers, ensuring that appropriate relief efforts are carried out.

[0182] Prompt Sentence Examples

[0183] I would like to use the "Disaster Logistics Manager" app to carry out appropriate relief efforts in the event of a large-scale earthquake. Collect data from drones, analyze the damage situation with AI, create a 3D map, manage requests for relief supplies, and generate optimal delivery routes.

[0184] This system will enable rapid and accurate information gathering in the event of a disaster and efficient distribution of relief supplies from logistics centers.

[0185] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0186] Step 1:

[0187] The server detects the occurrence of a disaster in real time. The main input data is weather data, information from earthquake sensors, and flood monitoring sensors. The server aggregates this data and determines whether a disaster has occurred. The output of this step is the determination result of "disaster occurrence." If a disaster occurs, the system begins operation.

[0188] Step 2:

[0189] When the server detects a disaster, it immediately sends instructions to launch multiple unmanned aerial vehicles (drones). These instructions include the flight area and automatic flight route. The drones receive the instructions and fly autonomously within the designated disaster area. The input for this step is instructions from the server, and the output is the launch and flight of the drones.

[0190] Step 3:

[0191] The drone uses its on-board camera to collect high-resolution video and image data of the affected area. This data is sent to the server in real time. The input is the aerial video and image data, and the output is the data sent to the server. In this step, the drone performs the specific operation of sending the captured data to the server.

[0192] Step 4:

[0193] The server stores the video and image data sent from the drone in cloud storage (AWS S3). Next, this data is input into a generative AI model (OpenAI API) to begin analyzing the data. During the analysis process, data before and after the disaster is compared to identify building damage, road disruption, the extent of flood damage, etc. The input for this step is the data stored in cloud storage, and the output is the analysis results.

[0194] Step 5:

[0195] The server generates a 3D map of the affected area based on the analysis results obtained from the generative AI model. The generated 3D map is then visualized and displayed by the server on terminals at the emergency response headquarters and logistics center. This 3D map allows the scale and location of the damage to be visually confirmed. The input to this step is the analysis results, and the output is a 3D map.

[0196] Step 6:

[0197] Users (refugee residents and relief workers) use a dedicated smartphone app or chatbot to send information about the local situation and requests for relief supplies to a server. The server then sends the received information to a generation AI, which performs natural language analysis to extract important information. The input for this step is information from the local area, and the output is the extracted important information.

[0198] Step 7:

[0199] The server updates the database with the required relief supplies based on the extracted information. At the same time, it determines appropriate instructions for relief operations and delivery routes for relief supplies. The determined instructions and route information are sent in real time to the logistics center and relief workers on the ground. The input of this step is the extracted important information, and the output is relief operation instructions and delivery route information.

[0200] This allows specific input data to be processed and calculated at each step to generate output, ultimately enabling fast and accurate relief operations and delivery of relief supplies.

[0201] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0202] The disaster response system of the present invention includes various hardware and software components that enable rapid and efficient assessment of the damage situation and appropriate rescue operations when a disaster occurs. The main components of this system include a disaster detection means, an unmanned aerial vehicle (drone), a server, a generative AI, a user interface, and an emotion engine.

[0203] 1. Disaster detection methods

[0204] The server collects information from weather data, earthquake sensors, flood monitoring sensors, etc. in order to monitor the occurrence of natural disasters such as earthquakes, heavy rains, and floods in real time. If it determines that a disaster has occurred based on this information, the entire system will immediately begin operating.

[0205] 2. Launching and flying unmanned aerial vehicles (drones)

[0206] When the server detects a disaster, it sends instructions to launch multiple drones in a designated area, which follow automated flight routes and use their cameras to collect video and image data of the affected area.

[0207] 3. Data collection and transmission

[0208] The drone (terminal) transmits high-resolution video and image data acquired during flight to a server in real time.

[0209] 4. Data Analysis and 3D Map Generation

[0210] The server stores the video and image data sent from the drone in cloud storage and inputs it into the generation AI. The generation AI compares the data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. Based on the results of this analysis, it generates a 3D map of the affected area.

[0211] 5. Visualizing the damage situation and providing guidance on countermeasures

[0212] The server visualizes and displays the 3D map generated by the AI ​​on the terminals at the emergency response headquarters. This allows for a visual understanding of the damage situation and enables quick decision-making. Furthermore, it automatically generates countermeasure guidance based on the analysis results and distributes it to all parties involved.

[0213] 6. Collecting information from the field and analyzing emotions

[0214] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server.

[0215] The server receives messages sent by users and analyzes the emotions in the messages using an emotion engine. The emotion engine analyzes the text data and determines what emotions the user is feeling.

[0216] 7. Natural Language Analysis and Extraction of Important Information

[0217] The server uses generative AI to perform natural language analysis of messages and extract important information, along with the emotional data recognized by the emotion engine. The emotional data is then added to the analysis to identify high-priority requests.

[0218] 8. Directing relief operations and determining routes for transporting relief supplies

[0219] Based on the analyzed information, the server updates a database with the status of relief supplies needed at each evacuation center, and uses generative AI to determine relief operation instructions and optimal delivery routes for relief supplies. It also takes into account the results of sentiment analysis, prioritizing requests with a high level of urgency.

[0220] 9. Distribution of rescue operation instructions

[0221] The server transmits the determined rescue operation instructions and information on the delivery route of relief supplies to rescue workers on the scene in real time, thereby enabling rescue operations to be carried out quickly and efficiently.

[0222] 10. Specific Examples

[0223] For example, if a large-scale earthquake occurs, the server immediately launches multiple drones to fly over areas with severe damage. The drones collect video and image data, which the server inputs into the generation AI in real time. The generation AI then creates a three-dimensional map of the affected area, identifying collapsed buildings and disrupted roads. At the same time, users submit requests for relief supplies from evacuation centers, and based on this information, the optimal route for quickly delivering the necessary supplies is determined. Furthermore, an emotion engine performs emotion analysis, and requests deemed to be of a high level of urgency are prioritized. The server then distributes this information to relief workers, ensuring that appropriate relief efforts are carried out.

[0224] In this way, this system can collect information quickly and accurately when a disaster occurs, and support effective relief efforts that also take into account the user's emotions.

[0225] The processing flow will be explained below.

[0226] Step 1:

[0227] The server collects and monitors weather data, earthquake sensors, and flood monitoring sensors in real time to detect the occurrence of disasters. When a disaster is detected, it immediately issues instructions to start operating the entire system.

[0228] Step 2:

[0229] When the server detects a disaster, it sends activation instructions to multiple unmanned aerial vehicles (drones) located within a pre-defined area.

[0230] Step 3:

[0231] Once the drone (terminal) receives a start command from the server, it begins flying within the designated area according to an automatic flight route. The drone uses a camera to collect high-resolution video and image data.

[0232] Step 4:

[0233] The drone (terminal) transmits video and image data collected during flight to a server in real time.

[0234] Step 5:

[0235] The server stores the video and image data received from the drone in cloud storage.

[0236] Step 6:

[0237] The server inputs the video and image data stored in cloud storage into the generation AI, which analyzes the damage situation.

[0238] Step 7:

[0239] The generating AI compares data before and after the disaster, automatically detecting building damage, road disruption, and the extent of flooding, and generates a three-dimensional map of the affected area.

[0240] Step 8:

[0241] The server visualizes the generated three-dimensional map and analysis results and displays them on a terminal at the emergency response headquarters.

[0242] Step 9:

[0243] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server.

[0244] Step 10:

[0245] The server receives messages sent by users.

[0246] Step 11:

[0247] The server inputs the received message into the emotion engine for emotion analysis. The emotion engine analyzes the text data and determines the emotion the user is feeling.

[0248] Step 12:

[0249] The server uses generative AI to perform natural language analysis of messages and extract important information, along with the emotional data recognized by the emotion engine. The emotional data is then added to the analysis to identify high-priority requests.

[0250] Step 13:

[0251] Based on the analyzed information, the server updates a database with the status of relief supplies needed at each evacuation center, and uses generative AI to determine relief operation instructions and optimal delivery routes for relief supplies. It also takes into account the results of sentiment analysis, prioritizing requests with a high level of urgency.

[0252] Step 14:

[0253] The server transmits the determined rescue operation instructions and information on the route for transporting relief supplies to rescue workers on the scene in real time.

[0254] Step 15:

[0255] The relief workers (users) carry out their activities on site in accordance with the received relief operation instructions and the delivery route of relief supplies.

[0256] This system will enable the rapid and accurate collection of information in the event of a disaster, and will support effective relief efforts that take into account the user's emotions.

[0257] Example 2

[0258] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0259] When a natural disaster occurs, the challenge is to quickly grasp the extent of damage at the scene and implement effective relief efforts. With conventional systems, it takes time to collect information after a disaster occurs, making it difficult to accurately grasp the extent of the damage. It is also difficult to respond appropriately, taking into account the urgency of the victims.

[0260] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0261] In this invention, the server includes means for detecting the occurrence of a disaster, means for launching multiple unmanned aerial vehicles and collecting video and image data of the affected area, means for analyzing the collected video and image data and generating a three-dimensional map of the damage situation, means for visualizing the generated three-dimensional map and distributing countermeasure guidance, means for receiving information from residents of evacuation centers and relief workers and extracting important information through natural language analysis including emotional data, and means for determining rescue operation instructions and delivery routes for relief supplies based on the extracted information. This enables rapid and accurate information collection when a disaster occurs, and enables effective rescue operations that take into account the emotions and urgency of the victims.

[0262] "Means for detecting the occurrence of disasters" refers to a system that collects information from data sources such as seismometers, meteorological data, and flood monitoring sensors, and detects the occurrence of natural disasters in real time.

[0263] An "unmanned aerial vehicle" is a device that flies automatically or remotely without a human on board and collects video and image data of the affected area.

[0264] "Means for collecting video and image data of the affected area" refers to a mechanism for collecting video and image data of the affected area using a camera mounted on an unmanned aerial vehicle.

[0265] "Means for analyzing collected video and image data and generating a three-dimensional map of the damage situation" refers to a system that uses image analysis technology to reconstruct the damage situation as a three-dimensional model based on the acquired data.

[0266] "Means for visualizing the generated 3D map and distributing countermeasure guidance" refers to a function that displays a 3D map on a terminal at the countermeasure headquarters, generates appropriate action guidelines according to the situation, and distributes them to relevant parties.

[0267] "Means for receiving information from evacuation center residents and relief workers and extracting important information through natural language analysis, including emotional data" refers to a system that receives messages sent from evacuation centers and on-site locations, analyzes them using natural language processing technology, including emotional data, and identifies information with a high level of urgency.

[0268] The "means for determining rescue operation instructions and transportation routes for relief supplies" is a system that, based on analyzed information, issues action instructions for efficiently providing necessary support and determines the optimal transportation route for relief supplies.

[0269] The disaster response system of the present invention is designed to quickly and efficiently assess the damage situation and carry out appropriate relief activities when a disaster occurs. The system includes various hardware and software components.

[0270] Hardware and Software Configuration

[0271] 1. Server:

[0272] The server collects information from weather data APIs, earthquake sensors, flood monitoring sensors, etc., and monitors disaster occurrences in real time. When it determines that a disaster has occurred, it starts operating the entire system.

[0273] 2. Unmanned Aerial Vehicles (Drones):

[0274] When a disaster occurs, the server sends instructions to launch multiple drones in a designated area. The drones follow autonomous flight routes and use their onboard cameras to collect high-resolution video and image data of the affected area.

[0275] 3. Data Collection and Transmission:

[0276] The drone (terminal) transmits the video and image data it acquires during flight to a server in real time using Wi-Fi or mobile networks.

[0277] 4. Data analysis and three-dimensional map generation:

[0278] The server stores the received data in cloud storage and inputs it into a generative AI model. The generative AI model compares data from before and after the disaster, analyzes building damage, road disruption, and the extent of flood damage, and generates a 3D map of the affected area. For example, it uses image recognition technology to identify damage and creates a 3D model based on this.

[0279] 5. Visualization of damage situation and guidance on countermeasures:

[0280] The server visualizes and displays the generated 3D map on the headquarters' terminal. This allows the headquarters to quickly grasp the damage situation and make decisions quickly. Furthermore, the generative AI model automatically generates countermeasure guidance based on the analysis results and distributes this to all parties involved. Specifically, the visualization tool is used to draw the 3D map, and the report generation function is used to create the guidance content.

[0281] 6. Collecting information from the field and analyzing sentiment:

[0282] Users (refugee residents and relief workers) request information about the local situation and necessary relief supplies through a dedicated app or chatbot. This request is sent to a server. The server passes the received message to an emotion engine, which analyzes the emotion in the message. For example, it can automatically determine the emotion of the message using a text analysis API.

[0283] 7. Natural Language Analysis and Key Information Extraction:

[0284] The server passes the emotion data recognized by the emotion engine to a generative AI model, which then performs natural language analysis of the message to extract important information. The emotion data is used as an indicator to identify high-priority requests. Specifically, the message is analyzed using a natural language processing API.

[0285] 8. Directing relief operations and determining routes for transporting relief supplies:

[0286] The server updates the database with the status of relief supplies needed at each evacuation center based on the analysis results. The generative AI model determines the optimal route for delivering relief supplies and generates instructions for relief operations. It also takes into account the results of sentiment analysis and prioritizes requests with high urgency. For example, it uses a route optimization algorithm to calculate efficient routes.

[0287] 9. Distribution of rescue operation instructions:

[0288] The server sends rescue operation instructions and delivery route information for relief supplies to rescue workers in real time, allowing rescue operations to proceed quickly and efficiently. Specifically, it sends instructions to relevant parties using a notification system.

[0289] Examples of concrete examples and prompts

[0290] Examples:

[0291] For example, if a large-scale earthquake occurs, the server immediately launches multiple drones to fly over areas with severe damage. The drones collect video and image data, which the server inputs into a generative AI model in real time. The generative AI model then generates a three-dimensional map of the affected area, identifying collapsed buildings and disrupted roads. At the same time, users submit requests for relief supplies from evacuation centers, and this information is used to determine the optimal route for quickly delivering the necessary supplies. An emotion engine performs sentiment analysis, prioritizing requests with a high level of urgency from users. The server then distributes this information to relief workers, supporting rapid and appropriate relief efforts.

[0292] Example prompt sentence:

[0293] "In areas recently affected by disasters, please analyze the received image data and generate a 3D map of the damage. Also, please consider the relief supply requests sent from evacuation centers, determine the optimal delivery route, and prioritize requests with high priority."

[0294] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0295] Step 1:

[0296] Disaster detection

[0297] The server collects information from multiple data sources, such as seismometers, weather data APIs, and flood monitoring sensors, and monitors the occurrence of natural disasters in real time. The input is information obtained from each sensor and API, and based on this, it performs data calculations to compare it with a disaster occurrence threshold. The output is a signal that sets a "disaster occurrence detection" flag.

[0298] Specific operation:

[0299] 1. Get current weather information from the weather data API.

[0300] 2. Obtain real-time earthquake data from seismometers.

[0301] 3. Compare the acquired data with a threshold to determine whether a disaster has occurred.

[0302] 4. When a disaster is detected, the entire system begins operation.

[0303] Step 2:

[0304] Launching and flying the drone

[0305] When the server detects a disaster, it sends an instruction to launch multiple drones in a designated area. The input is a disaster detection signal and designated area information, and the output is a drone launch command. The drones follow an autonomous flight route and collect video and image data of the affected area.

[0306] Specific operation:

[0307] 1. Automatically launch multiple drones using the drone control API.

[0308] 2. Send the flight route and coordinate information of the affected area to the activated drone.

[0309] 3. The drone begins flying along the specified route.

[0310] Step 3:

[0311] Data collection and transmission

[0312] The drone (terminal) captures high-resolution video and image data using an onboard camera during flight and transmits this data to a server in real time. The input is the video and image data captured by the drone's camera, and the output is the raw data transmitted to the server.

[0313] Specific operation:

[0314] 1. Use the drone camera to capture video and images of the affected area.

[0315] 2. The acquired data is sent to the server in real time.

[0316] Step 4:

[0317] Data analysis and 3D map generation

[0318] The server stores the received video and image data in cloud storage and inputs it into a generative AI model. The generative AI model compares data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. The input is the data stored in cloud storage and data before and after the disaster, and the output is a 3D map of the affected area.

[0319] Specific operation:

[0320] 1. Save the received data in cloud storage.

[0321] 2. Input data into the generative AI model and analyze the damage situation.

[0322] 3. Generate a three-dimensional map of the affected area based on the analysis results.

[0323] Step 5:

[0324] Visualization of damage status and delivery of countermeasure guidance

[0325] The server visualizes and displays the generated 3D map on the terminal of the emergency response headquarters. The input is the generated 3D map, and the output is the map displayed on the terminal of the emergency response headquarters and countermeasure guidance. Countermeasure guidance is also automatically generated based on the analysis results of the generative AI model and distributed to relevant parties.

[0326] Specific operation:

[0327] 1. Use visualization tools to display 3D maps on the headquarters terminals.

[0328] 2. Create countermeasure guidance based on the analysis results of the generative AI model.

[0329] 3. Distribute the created guidance to each stakeholder.

[0330] Step 6:

[0331] Collecting information from the field and analyzing emotions

[0332] Users (refugee residents and relief workers) use a dedicated app or chatbot to request information about the local situation and necessary relief supplies. The server passes these messages to the emotion engine, which analyzes the emotions in the messages. The input is the message from the user, and the output is the analyzed emotion data.

[0333] Specific operation:

[0334] 1. Users submit information about the local situation and requests for relief supplies through a dedicated app or chatbot.

[0335] 2. The server receives the message and analyzes the emotion using the emotion engine.

[0336] Step 7:

[0337] Natural language analysis and extraction of important information

[0338] The server passes the emotion data recognized by the emotion engine to the generative AI model, which then performs natural language analysis of the message to extract important information. The input is the emotion data and the message, and the output is the extracted important information.

[0339] Specific operation:

[0340] 1. Analyze key messages based on sentiment data.

[0341] 2. Input emotion data into a generative AI model to extract key information.

[0342] Step 8:

[0343] Directing relief operations and determining routes for transporting relief supplies

[0344] Based on the analyzed information, the server updates the database with the status of relief supplies needed at each evacuation shelter. The generative AI model determines the optimal delivery route for relief supplies and generates rescue operation instructions. The input is the extracted important information and the current status of relief supplies, and the output is the confirmed delivery route and rescue operation instructions.

[0345] Specific operation:

[0346] 1. Based on the analysis results, the status of relief supplies is updated in the database.

[0347] 2. Determine the optimal delivery route using a generative AI model.

[0348] 3. Generate rescue operation instructions.

[0349] Step 9:

[0350] Distribution of rescue operation instructions

[0351] The server transmits rescue operation instructions and relief supply delivery route information to rescue workers in the field in real time. The input is the confirmed delivery route and rescue operation instructions, and the output is the information to be distributed to rescue workers in the field.

[0352] Specific operation:

[0353] 1. Send confirmed transport routes and rescue instructions to rescue workers on the scene.

[0354] 2. Deliver quickly with a real-time notification system.

[0355] (Application example 2)

[0356] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0357] When a disaster occurs, it is necessary to quickly and accurately grasp the damage situation and carry out efficient relief operations, but conventional systems have had problems with delays in information gathering and overlooking important information, making it difficult to respond quickly. Another issue was the inability to provide appropriate instructions for relief operations that take into account the user's emotions.

[0358] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for detecting the occurrence of a disaster, means for launching multiple unmanned aerial vehicles and collecting video and image data of the affected area, means for analyzing the collected video and image data and generating a three-dimensional map of the damage situation, means for visualizing the generated three-dimensional map and distributing countermeasure guidance, means for receiving information from residents of evacuation centers and relief workers and extracting important information through natural language analysis, means for analyzing the sentiment of users' messages and identifying high-priority requests, means for determining rescue operation instructions and delivery routes for relief supplies based on the extracted information, and means for transmitting rescue operation instructions to relief workers on the scene in real time. This enables a quick and accurate understanding of the damage situation and appropriate rescue operation instructions that take the user's sentiment into consideration.

[0359] "Means for detecting the occurrence of disasters" refers to sensors and data analysis systems that allow a server to monitor natural disasters such as earthquakes, heavy rain, and floods in real time and determine whether a disaster has occurred.

[0360] An "unmanned aerial vehicle" is a drone or other unmanned aircraft equipped with a camera and following an automated flight route.

[0361] The "means for collecting video and image data of the affected area" is a system that allows unmanned aerial vehicles to capture video and image data of the affected area and send it to a server.

[0362] The "means for generating a three-dimensional map" is a system that analyzes collected video and image data and visually displays the damage situation as a three-dimensional map.

[0363] The "means of delivering guidance on countermeasures" is a system that delivers information and guidelines for action to relevant organizations and personnel based on the generated three-dimensional map to support rapid decision-making.

[0364] The "means for extracting key information through natural language analysis" refers to a generative AI model and natural language processing system for analyzing messages received from evacuation center residents and relief workers and extracting key information from them.

[0365] The "means for analyzing emotions and identifying high-priority requests" is a system that analyzes the emotions in messages from users and determines the priority of requests according to their urgency and importance.

[0366] The "means for determining rescue operation instructions and transportation routes for relief supplies" is a system that determines optimal rescue operation instructions and transportation routes for relief supplies based on the extracted important information and emotion analysis results.

[0367] The "means for transmitting rescue operation instructions to rescue workers on the scene in real time" is a communication system for quickly transmitting the determined rescue operation instructions and information on the transportation route of relief supplies to rescue workers on the scene.

[0368] The disaster response system of the present invention includes various hardware and software components that enable rapid and efficient assessment of the damage situation and appropriate rescue operations when a disaster occurs. The main components of this system include a disaster detection means, an unmanned aerial vehicle (drone), a server, a generative AI, a user interface, and an emotion engine.

[0369] The server collects and analyzes information from weather data, earthquake sensors, flood monitoring sensors, etc. in order to monitor the occurrence of natural disasters such as earthquakes, heavy rain, and floods in real time. Based on this information, if it determines that a disaster has occurred, the entire system immediately begins operation. When the server detects a disaster, it sends instructions to launch multiple unmanned aerial vehicles (drones) in a specified area. The drones follow automatic flight routes and use cameras to collect video and image data of the affected area, which they then send to the server in real time.

[0370] The server stores the high-resolution video and image data sent from the drone in cloud storage (e.g., Amazon S3) and inputs it into a generation AI (e.g., OpenAI GPT model). The generation AI compares data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. Based on the analysis results, it generates a 3D map of the affected area, which is visualized and displayed on a terminal at the response headquarters. Furthermore, it automatically generates countermeasure guidance based on the analysis results and distributes it to all parties involved.

[0371] Users (refugee residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to a server. The server receives the messages sent by the users and analyzes the emotions in the messages using an emotion engine (e.g., IBM Watson (registered trademark)). The emotion engine analyzes the text data and determines the user's emotions. The server performs natural language analysis of the messages using generative AI along with the emotion data recognized by the emotion engine to extract important information. The emotion data is added to the analysis and serves as an indicator for identifying high-priority requests.

[0372] Based on the analyzed information, the server determines optimal rescue operation instructions and delivery routes for relief supplies. This is done using a generative AI model to automatically generate efficient routes and operation instructions. The determined rescue operation instructions and delivery route information for relief supplies are sent to rescue workers on the ground in real time, allowing rescue operations to be carried out quickly and efficiently.

[0373] As a specific example, if a large-scale earthquake occurs, the server immediately launches multiple drones to fly over areas with severe damage. The drones collect video and image data, which the server inputs into the generation AI in real time. The generation AI creates a three-dimensional map of the affected area and identifies collapsed buildings and severed roads. At the same time, users submit requests for relief supplies from evacuation centers, and based on this information, the optimal route for quickly delivering the necessary supplies is determined. Furthermore, an emotion engine performs emotion analysis, and requests that are deemed to be of high urgency by the user are prioritized.

[0374] For example, if a user sends a message such as "A major earthquake has occurred and I am in a shelter. I am short of water and food," emotion analysis and natural language analysis can quickly identify that emergency assistance is needed in that area. Based on this information, the server can then deploy efficient relief activities.

[0375] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0376] Step 1:

[0377] The server monitors various sensors and meteorological data to detect the occurrence of disasters. The input is data obtained from earthquake sensors and weather information APIs, and the server analyzes this data to determine the occurrence of a natural disaster. The output is an internal flag indicating that a disaster has occurred. Based on this flag, the operation to proceed to the next step is triggered.

[0378] Step 2:

[0379] When the server detects that a disaster has occurred, it sends an instruction to launch multiple unmanned aerial vehicles (drones) in a specified area. The input is a disaster detection flag and coordinate data of the affected area, and the output is a drone launch signal. Specifically, the server calls the drone's launch API and sets an automatic flight route.

[0380] Step 3:

[0381] The unmanned aerial vehicle (drone) follows an automated flight route and uses a camera to collect video and image data of the affected area. The input is the drone's flight route data and camera setting data, and the output is high-resolution video and image data. Specifically, the drone operates by continuously taking photos and videos while flying a predetermined route.

[0382] Step 4:

[0383] The drone transmits the collected video and image data to a server in real time. The input is the video and image data captured by the drone's camera, and the output is a data stream to the server. Specifically, the drone transmits the data to the server using wireless communication.

[0384] Step 5:

[0385] The server stores the data sent from the drone in cloud storage (e.g., Amazon S3) and then inputs it into the generation AI. The input is the video and image data received from the drone, and the output is an analysis request to the generation AI. Specifically, the server uploads the data to cloud storage and passes the reference link to the generation AI.

[0386] Step 6:

[0387] The generative AI (for example, the OpenAI GPT model) compares data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. The input is a cloud storage link and analysis instructions, and the output is 3D map data of the affected area. Specifically, the generative AI analyzes the data and visualizes the damage situation in 3D.

[0388] Step 7:

[0389] The server visualizes and displays the 3D map generated by the generation AI on the terminal at the emergency headquarters. The input is 3D map data, and the output is visual information displayed on the screen at the emergency headquarters. Specifically, the server renders the 3D map and displays it on the monitor at the emergency headquarters.

[0390] Step 8:

[0391] Users (shelter residents and relief workers) use a dedicated app or chatbot to send local information and requests for relief supplies to a server. The input is text information entered by the user into the app or chatbot, and the output is the request data received by the server. The specific operation is for the user to request the necessary relief supplies through the app.

[0392] Step 9:

[0393] The server receives messages sent by users and performs emotion analysis using an emotion engine (e.g., IBM Watson). The input is the user's text message, and the output is the analyzed emotion data. Specifically, the server sends messages to the emotion engine and receives emotion data.

[0394] Step 10:

[0395] The server uses the generation AI along with the emotion data recognized by the emotion engine to perform natural language analysis of the message and extract important information. The input is the user's message and emotion data, and the output is an extracted list of important information. Specifically, the generation AI performs natural language analysis and determines the priority of the request.

[0396] Step 11:

[0397] The server determines optimal rescue operation instructions and delivery routes for relief supplies based on the analyzed information. The input is an extracted list of important information and emotion data, and the output is rescue operation instructions and delivery route determination data. Specifically, the server calculates the optimal route and operation instructions with the help of generation AI.

[0398] Step 12:

[0399] The server transmits the determined rescue operation instructions and information on the delivery route of relief supplies to the rescue workers on the scene in real time. The input is the rescue operation instructions and delivery route determination data, and the output is a notification of instructions to the rescue workers. Specifically, the server transmits information to the scene via the communication system.

[0400] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0401] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0402] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0403] [Second embodiment]

[0404] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0405] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0406] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0407] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0408] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0409] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0410] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0411] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0412] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0413] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0414] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0415] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0416] The disaster response system of the present invention includes various hardware and software components that enable rapid and efficient assessment of the damage situation and appropriate rescue operations when a disaster occurs. The main components of this system include a disaster detection means, an unmanned aerial vehicle (drone), a server, a generation AI, and a user interface.

[0417] 1. Disaster detection methods

[0418] The server collects information from weather data, earthquake sensors, flood monitoring sensors, etc. in order to monitor the occurrence of natural disasters such as earthquakes, heavy rains, and floods in real time. If it determines that a disaster has occurred based on this information, the entire system will immediately begin operating.

[0419] 2. Launching and flying unmanned aerial vehicles (drones)

[0420] When the server detects a disaster, it sends instructions to launch multiple drones in a designated area, which follow automated flight routes and use their cameras to collect video and image data of the affected area.

[0421] 3. Data collection and transmission

[0422] The drone (terminal) transmits high-resolution video and image data acquired during flight to a server in real time, enabling a rapid assessment of the damage situation.

[0423] 4. Data Analysis and 3D Map Generation

[0424] The server stores the video and image data sent from the drone in cloud storage and inputs it into the generation AI. The generation AI compares the data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. Based on the results of this analysis, it generates a 3D map of the affected area.

[0425] 5. Visualizing the damage situation and providing guidance on countermeasures

[0426] The server visualizes and displays the 3D map generated by the AI ​​on the terminals at the emergency response headquarters. This allows for a visual understanding of the damage situation and enables quick decision-making. Furthermore, it automatically generates countermeasure guidance based on the analysis results and distributes it to all parties involved.

[0427] 6. Collecting and analyzing information from the field

[0428] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server. The server receives the information, and the generative AI performs natural language analysis to extract important information.

[0429] 7. Directing relief operations and determining routes for transporting relief supplies

[0430] Based on the extracted information, the server updates the database with the status of relief supplies needed at each evacuation shelter, determines instructions for relief operations and routes for delivering relief supplies, and transmits the instructions and route information to relief workers on the ground in real time.

[0431] 8. Specific Examples

[0432] For example, if a large-scale earthquake occurs, the server immediately launches multiple drones to fly over areas with severe damage. The drones collect video and image data, which the server inputs into the generation AI in real time. The generation AI then creates a three-dimensional map of the affected area, identifying collapsed buildings and disrupted roads. At the same time, users submit requests for relief supplies from evacuation centers, and this information is used to determine the optimal route for quickly delivering the necessary supplies. The server then distributes this information to relief workers, ensuring that appropriate relief efforts are carried out.

[0433] In this way, this system can collect information quickly and accurately when a disaster occurs, and support effective relief efforts.

[0434] The processing flow will be explained below.

[0435] Step 1:

[0436] The server collects and monitors weather data, earthquake sensors, and flood monitoring sensors in real time to detect the occurrence of disasters. When a disaster is detected, it immediately issues instructions to start operating the entire system.

[0437] Step 2:

[0438] When the server detects a disaster, it sends activation instructions to multiple unmanned aerial vehicles (drones) located within a pre-defined area.

[0439] Step 3:

[0440] Once the drone (terminal) receives a start command from the server, it begins flying within the designated area according to an automatic flight route. The drone uses a camera to collect high-resolution video and image data.

[0441] Step 4:

[0442] The drone (terminal) transmits video and image data collected during flight to a server in real time.

[0443] Step 5:

[0444] The server stores the video and image data received from the drone in cloud storage.

[0445] Step 6:

[0446] The server inputs the video and image data stored in cloud storage into the generation AI, which analyzes the damage situation.

[0447] Step 7:

[0448] The generating AI compares data before and after the disaster, automatically detecting building damage, road disruption, and the extent of flooding, and generates a three-dimensional map of the affected area.

[0449] Step 8:

[0450] The server visualizes the generated three-dimensional map and analysis results and displays them on a terminal at the emergency response headquarters.

[0451] Step 9:

[0452] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server.

[0453] Step 10:

[0454] The server receives messages sent by users and uses generative AI to perform natural language analysis and extract important information.

[0455] Step 11:

[0456] Based on the analyzed information, the server updates the database with the status of relief supplies at each evacuation shelter.

[0457] Step 12:

[0458] The server uses generative AI based on the information in the database to determine rescue operation instructions and the optimal route for transporting relief supplies.

[0459] Step 13:

[0460] The server transmits the determined rescue operation instructions and information on the route for transporting relief supplies to rescue workers on the scene in real time.

[0461] Step 14:

[0462] The relief workers (users) carry out their activities on site in accordance with the received relief operation instructions and the delivery route of relief supplies.

[0463] This system will enable rapid and accurate collection of information in the event of a disaster, enabling appropriate and effective relief efforts to be carried out.

[0464] Example 1

[0465] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0466] The problem is that when a disaster occurs, there is a lack of means to collect and analyze information to quickly and accurately grasp the damage situation and carry out effective relief activities. With conventional methods, it takes time to accurately grasp the damage situation on the ground, and the accuracy of the information is limited, making it difficult to formulate appropriate relief operation plans.

[0467] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0468] In this invention, the server includes means for detecting the occurrence of a disaster, means for launching multiple unmanned aerial vehicles and collecting video and image data of the affected area, means for analyzing the collected video and image data and generating a three-dimensional map of the damage situation, means for visualizing the generated three-dimensional map and distributing countermeasure instructions, means for receiving information from evacuation shelter residents and relief workers and extracting important information through natural language analysis, means for determining rescue operation instructions and delivery routes for relief supplies based on the extracted information, means for automatically analyzing the damage situation and providing countermeasure instructions using a generative AI model, means for immediately recognizing the occurrence of a disaster and emergency activation of the system, and means for saving data captured by the unmanned aerial vehicle's camera in real time to cloud storage. This enables rapid and accurate information collection and analysis, supporting effective rescue operations.

[0469] "Means for detecting the occurrence of disasters" refers to devices or systems that monitor the occurrence of natural disasters such as earthquakes, heavy rains, and floods in real time, collect and analyze data, and determine whether a disaster has occurred.

[0470] "Multiple unmanned aerial vehicles" means an aircraft system that includes drones that are flown by remote control or autonomous pilot and used to monitor a specific area.

[0471] "Video and image data of the affected areas" refers to video and photographic data taken by unmanned aerial vehicles to show the current state of the areas affected by the disaster.

[0472] The "means of collection" is a communication device or protocol for transferring video and image data captured by the unmanned aerial vehicle to a specific server or cloud storage.

[0473] "Means for generating a three-dimensional map" refers to software or algorithms that analyze collected video and image data and recreate the extent and details of the damage in three-dimensional space.

[0474] The "means for visualization and delivery of countermeasure guidance" is a system that displays the generated three-dimensional map on a terminal display and notifies relevant parties of rescue operation instructions and countermeasures.

[0475] "Means for extracting important information through natural language analysis" refers to algorithms or software that analyze text data sent by evacuation shelter residents and relief workers and automatically extract important information necessary for providing assistance.

[0476] The "means for determining rescue operation instructions and transportation routes for relief supplies" is a system that uses the extracted information to create a list of relief supplies needed at each evacuation shelter and makes a plan for transporting them via the optimal route.

[0477] "Means for automatically analyzing damage situations and providing guidance on countermeasures using generative AI models" refers to machine learning models or AI systems that analyze damage situations based on collected data and automatically propose optimal countermeasures.

[0478] "Means for immediately recognizing the occurrence of a disaster and for emergency activation of the system" refers to a combination of hardware and software that immediately activates the entire system and initiates necessary measures when a disaster is detected.

[0479] "Data captured by the camera of an unmanned aerial vehicle" refers to visual information collected through a camera mounted on an unmanned aerial vehicle, and is a means of storing this information in cloud storage in real time.

[0480] The present invention is directed to a system that aims to quickly and accurately grasp the damage situation when a disaster occurs and to support effective relief efforts. The present invention is configured by combining various hardware and software components.

[0481] Disaster detection

[0482] To detect the occurrence of a disaster, the server collects and analyzes real-time data from weather data, earthquake sensors, flood monitoring sensors, etc. This is done using, for example, the Japan Meteorological Agency's API or specific sensor devices. When the server detects data that exceeds a certain threshold, it recognizes the occurrence of a disaster and issues a command to immediately start up the operation of the entire system. Specifically, it uses software that immediately recognizes the occurrence of a disaster and immediately starts up the system.

[0483] Launching and flying unmanned aerial vehicles

[0484] The server launches multiple unmanned aerial vehicles (drones) via drone control software in areas where a disaster has been confirmed. These drones follow automated flight routes and use cameras to collect video and image data of the affected area. The drones are equipped with high-resolution camera modules and fly autonomously along pre-programmed flight paths. Flight control is performed using the DroneKit API, for example.

[0485] Data collection and transmission

[0486] The drone (terminal) transmits high-resolution video and image data acquired during flight to a server in real time over a 5G network, and the data is stored in a cloud storage service, specifically Amazon S3.

[0487] Data analysis and 3D map generation

[0488] The server inputs the video and image data stored in cloud storage into a generative AI model. This generative AI model was developed using TensorFlow and other tools, and analyzes the extent and details of the damage. Based on the analysis results, a three-dimensional map of the affected area is generated. This map is saved for further detailed analysis and is linked to the emergency response headquarters terminal.

[0489] Visualization of damage status and delivery of countermeasure guidance

[0490] The server then displays the generated 3D map on the device's display, using a visualization tool such as Tableau. The automated guidance generated by the AI ​​model is then distributed to relevant parties via the SendGrid API.

[0491] Collecting and analyzing information from the field

[0492] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server. This application is a mobile app using Firebase. The server receives this information and performs natural language analysis using a generative AI model, which automatically extracts important information.

[0493] Directing relief operations and determining routes for transporting relief supplies

[0494] The server updates a database with the status of relief supplies needed at each evacuation center based on the information extracted through natural language analysis. It then uses the Google Maps API to calculate optimal delivery routes and generate rescue operation instructions. This information is then sent in real time to rescue workers on the ground using Firebase Cloud Messaging.

[0495] Specific examples

[0496] For example, if a large-scale earthquake occurs, the server immediately recognizes the occurrence of a disaster based on data obtained from the Japan Meteorological Agency API and launches multiple drones using the DroneKit API. The drones store the acquired data in Amazon S3 via the 5G network, and this data is input into a generative AI model using TensorFlow. The analysis results are displayed in Tableau and distributed to relevant parties via the SendGrid API. Requests from evacuation centers are received through the Firebase app, and the information is analyzed using a natural language analysis model. The optimal route for delivering relief supplies is calculated using the Google Maps API and sent to relief workers via Firebase Cloud Messaging.

[0497] Prompt Sentence Examples

[0498] "Please tell me the detailed operation procedure of the system that quickly grasps the damage situation in an area where an earthquake has occurred and uses a generative AI model to plan rescue operations."

[0499] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0500] Step 1: Detecting the occurrence of a disaster

[0501] The server collects data in real time from sources such as weather data, earthquake sensors, and flood monitoring sensors. Specifically, it takes in data obtained from APIs and sensor devices every minute. Statistical analysis is performed based on this data, and if a certain threshold is exceeded, a disaster is recognized. Various disaster data is input, and a disaster occurrence signal is generated as output.

[0502] Step 2: Launch command for unmanned aerial vehicles

[0503] When the server confirms that a disaster has occurred, it sends instructions to launch multiple unmanned aerial vehicles (drones). This instruction is sent via drone control software (e.g., DroneKit API). The input is a disaster occurrence signal, and the output is a command to launch the drones. Specific operations include setting an automatic flight route based on GPS coordinates and activating cameras.

[0504] Step 3: Video and image data collection

[0505] The drone (terminal) collects high-resolution video and image data while flying along an automatic flight route. It uses a camera module (e.g., DJI Zenmuse X7) to capture images along the specified route. The input is the set flight route information, and the output is high-resolution video and image data.

[0506] Step 4: Sending data in real time

[0507] The drone transmits the collected video and image data to a server in real time. The communication method is a 5G network. The input is high-resolution video and image data, and the output is the data sent to the server. Specifically, the data is packetized and transmitted.

[0508] Step 5: Save your data to the cloud

[0509] The server stores the received video and image data in a cloud storage service (e.g., Amazon S3). The input is data sent in real time, and the output is data stored in the cloud storage. Specifically, the data is stream-processed and converted into files.

[0510] Step 6: Generate a 3D map

[0511] The server inputs the data stored in cloud storage into a generative AI model and analyzes the damage situation. Using the generative AI model (e.g., TensorFlow), it compares data before and after the disaster and generates a 3D map. The input is the stored video and image data, and the output is a 3D map. Specific operations include preprocessing the data, inputting it into the trained model, and outputting the analysis results.

[0512] Step 7: Visualize the damage situation

[0513] The server displays the generated 3D map on the emergency headquarters terminal, allowing the damage situation to be visually grasped. The input is the generated 3D map data, and the output is the map displayed on the terminal. Specifically, the 3D map data is rendered and displayed.

[0514] Step 8: Automatic generation and distribution of countermeasure guidance

[0515] The server uses a generative AI model to automatically generate countermeasure guidance based on the analysis results and distributes it to relevant parties. The input is the analysis results and a 3D map, and the output is the automatically generated countermeasure guidance and distributed guidance message. Specific operations involve the use of an email distribution system (e.g., SendGrid API).

[0516] Step 9: Gather information from the field

[0517] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for relief supplies to the server. The input is text, images, and audio data, and the output is the information sent to the server. Specific operations involve the use of a mobile app (e.g., Firebase app).

[0518] Step 10: Extracting important information through natural language analysis

[0519] The server uses a generative AI model to perform natural language analysis on the received local information and extract important information. The input is the transmitted text, images, and audio data, and the output is the extracted important information. The specific operation involves applying a natural language processing algorithm.

[0520] Step 11: Deciding on relief operation instructions and relief supply transport routes

[0521] The server updates the database based on the extracted information and determines rescue operation instructions and delivery routes for relief supplies. The input is important information and the current supply situation, and the output is optimized delivery routes and rescue instructions. Specifically, a route calculation algorithm (e.g., Google Maps API) is applied.

[0522] Step 12: Send instructions to relief workers

[0523] The server sends the determined instructions and transport route information to the rescue workers on the scene via push notification. The input is the rescue instructions and transport route, and the output is the information sent to the rescue workers' devices. The specific operation is to send a push notification (e.g., Firebase Cloud Messaging).

[0524] (Application example 1)

[0525] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0526] When a disaster occurs, it is important to quickly and accurately grasp the damage situation. To quickly and appropriately carry out relief efforts, it is necessary to collect and analyze information from various evacuation centers and relief activity sites. However, the current system does not efficiently coordinate the understanding of the damage situation and instructions for relief efforts, making it particularly difficult to optimize the transportation routes of supplies. Furthermore, logistics centers do not have the appropriate means to rapidly distribute relief supplies.

[0527] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0528] In this invention, the server includes means for detecting the occurrence of a disaster, means for launching multiple unmanned aerial vehicles to collect video and image data of the affected area, means for analyzing the collected video and image data and generating a 3D map of the damage situation, and means for displaying the generated 3D map on a terminal at the logistics center, and for quickly managing requests for relief supplies and optimizing delivery routes. This enables a detailed understanding of the damage situation in the event of a disaster and efficient distribution of relief supplies via the logistics center.

[0529] "Means for detecting the occurrence of disasters" refers to a system that monitors and notifies in real time the occurrence of natural disasters such as earthquakes, heavy rain, and floods.

[0530] An "unmanned aerial vehicle" is a device, such as a drone, that flies remotely or autonomously and collects video and image data of a specific area.

[0531] "Video and image data" refers to video and photographic data used to visually record the situation in the affected areas.

[0532] A "three-dimensional map" is a map that displays the state of the terrain and buildings before and after a disaster in three dimensions.

[0533] A "generative AI model" is an artificial intelligence system that analyzes collected data and generates new data and information.

[0534] The "means for delivering guidance on countermeasures" is a system that automatically notifies each relevant party of appropriate countermeasures and instructions based on the analyzed data.

[0535] "Means for receiving information from evacuation center residents and relief workers" refers to a system for receiving information on the local situation and requests for relief supplies from evacuation centers and local relief workers.

[0536] "Natural language analysis" is a technology for analyzing received information and extracting important information.

[0537] A "relief supply transport route" is a route for efficiently delivering necessary relief supplies to evacuation centers and disaster-stricken areas.

[0538] A "logistics center" is a facility for distributing and managing relief supplies.

[0539] A "terminal" is a computer system for displaying and manipulating information.

[0540] "Relief supply request management" is the process of receiving requests for relief supplies from evacuation centers and disaster-stricken areas and managing the necessary supplies.

[0541] "Route optimization" is the process of planning and executing the safe delivery of relief supplies via the shortest and most optimal route.

[0542] This invention provides a system for quickly and accurately assessing the damage situation when a disaster occurs, and for efficiently carrying out appropriate relief activities and transporting relief supplies. The system includes the following main hardware and software components:

[0543] 1. Means of detecting disasters

[0544] The server monitors disasters such as earthquakes, heavy rains, and floods in real time, collecting information from weather data, earthquake sensors, flood monitoring sensors, etc. If it determines that a disaster has occurred, it immediately starts operating the entire system.

[0545] 2. Starting and flying the drone

[0546] When the server detects a disaster, it sends instructions to launch multiple unmanned aerial vehicles in a designated area. The drones follow automated flight routes and use their onboard cameras to collect video and image data of the affected area.

[0547] 3. Data collection and transmission

[0548] The drones capture high-resolution video and image data during flight and transmit it to a server in real time, enabling a rapid assessment of the damage situation.

[0549] 4. Data Analysis and 3D Map Generation

[0550] The server stores the video and image data sent from the drone in cloud storage (AWS S3) and inputs it into a generative AI model (OpenAI API). The generative AI compares data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. Based on the results of this analysis, it generates a 3D map of the affected area.

[0551] 5. Visualizing the damage situation and providing guidance on countermeasures

[0552] The server visualizes and displays the 3D map generated by the AI ​​on terminals at the logistics center and the emergency response headquarters. This allows for a visual understanding of the damage situation, enabling quick decision-making. Furthermore, guidance on countermeasures based on the analysis results is distributed to all parties in real time.

[0553] 6. Collecting and analyzing information from the field

[0554] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server. The server receives the information, and the generative AI performs natural language analysis to extract important information.

[0555] 7. Directing relief operations and determining routes for transporting relief supplies

[0556] Based on the extracted information, the server updates the database with the status of relief supplies needed at each evacuation shelter, determines optimal instructions for relief efforts, and determines the optimal route for delivering relief supplies. The determined instructions and route information are sent in real time to relief workers on the ground.

[0557] Specific examples

[0558] For example, if a large-scale earthquake occurs, the server immediately launches multiple drones to fly over areas with severe damage. The drones collect video and image data, which the server inputs into the generation AI in real time. The generation AI generates a three-dimensional map of the affected area and identifies collapsed buildings and disrupted roads. At the same time, users submit requests for relief supplies from evacuation centers, and this information is used to determine the optimal route for quickly delivering the necessary supplies. The server then distributes this information to relief workers, ensuring that appropriate relief efforts are carried out.

[0559] Prompt Sentence Examples

[0560] I would like to use the "Disaster Logistics Manager" app to carry out appropriate relief efforts in the event of a large-scale earthquake. Collect data from drones, analyze the damage situation with AI, create a 3D map, manage requests for relief supplies, and generate optimal delivery routes.

[0561] This system will enable rapid and accurate information gathering in the event of a disaster and efficient distribution of relief supplies from logistics centers.

[0562] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0563] Step 1:

[0564] The server detects the occurrence of a disaster in real time. The main input data is weather data, information from earthquake sensors, and flood monitoring sensors. The server aggregates this data and determines whether a disaster has occurred. The output of this step is the determination result of "disaster occurrence." If a disaster occurs, the system begins operation.

[0565] Step 2:

[0566] When the server detects a disaster, it immediately sends instructions to launch multiple unmanned aerial vehicles (drones). These instructions include the flight area and automatic flight route. The drones receive the instructions and fly autonomously within the designated disaster area. The input for this step is instructions from the server, and the output is the launch and flight of the drones.

[0567] Step 3:

[0568] The drone uses its on-board camera to collect high-resolution video and image data of the affected area. This data is sent to the server in real time. The input is the aerial video and image data, and the output is the data sent to the server. In this step, the drone performs the specific operation of sending the captured data to the server.

[0569] Step 4:

[0570] The server stores the video and image data sent from the drone in cloud storage (AWS S3). Next, this data is input into a generative AI model (OpenAI API) to begin analyzing the data. During the analysis process, data before and after the disaster is compared to identify building damage, road disruption, the extent of flood damage, etc. The input for this step is the data stored in cloud storage, and the output is the analysis results.

[0571] Step 5:

[0572] The server generates a 3D map of the affected area based on the analysis results obtained from the generative AI model. The generated 3D map is then visualized and displayed by the server on terminals at the emergency response headquarters and logistics center. This 3D map allows the scale and location of the damage to be visually confirmed. The input to this step is the analysis results, and the output is a 3D map.

[0573] Step 6:

[0574] Users (refugee residents and relief workers) use a dedicated smartphone app or chatbot to send information about the local situation and requests for relief supplies to a server. The server then sends the received information to a generation AI, which performs natural language analysis to extract important information. The input for this step is information from the local area, and the output is the extracted important information.

[0575] Step 7:

[0576] The server updates the database with the required relief supplies based on the extracted information. At the same time, it determines appropriate instructions for relief operations and delivery routes for relief supplies. The determined instructions and route information are sent in real time to the logistics center and relief workers on the ground. The input of this step is the extracted important information, and the output is relief operation instructions and delivery route information.

[0577] This allows specific input data to be processed and calculated at each step to generate output, ultimately enabling fast and accurate relief operations and delivery of relief supplies.

[0578] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0579] The disaster response system of the present invention includes various hardware and software components that enable rapid and efficient assessment of the damage situation and appropriate rescue operations when a disaster occurs. The main components of this system include a disaster detection means, an unmanned aerial vehicle (drone), a server, a generative AI, a user interface, and an emotion engine.

[0580] 1. Disaster detection methods

[0581] The server collects information from weather data, earthquake sensors, flood monitoring sensors, etc. in order to monitor the occurrence of natural disasters such as earthquakes, heavy rains, and floods in real time. If it determines that a disaster has occurred based on this information, the entire system will immediately begin operating.

[0582] 2. Launching and flying unmanned aerial vehicles (drones)

[0583] When the server detects a disaster, it sends instructions to launch multiple drones in a designated area, which follow automated flight routes and use their cameras to collect video and image data of the affected area.

[0584] 3. Data collection and transmission

[0585] The drone (terminal) transmits high-resolution video and image data acquired during flight to a server in real time.

[0586] 4. Data Analysis and 3D Map Generation

[0587] The server stores the video and image data sent from the drone in cloud storage and inputs it into the generation AI. The generation AI compares the data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. Based on the results of this analysis, it generates a 3D map of the affected area.

[0588] 5. Visualizing the damage situation and providing guidance on countermeasures

[0589] The server visualizes and displays the 3D map generated by the AI ​​on the terminals at the emergency response headquarters. This allows for a visual understanding of the damage situation and enables quick decision-making. Furthermore, it automatically generates countermeasure guidance based on the analysis results and distributes it to all parties involved.

[0590] 6. Collecting information from the field and analyzing emotions

[0591] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server.

[0592] The server receives messages sent by users and analyzes the emotions in the messages using an emotion engine. The emotion engine analyzes the text data and determines what emotions the user is feeling.

[0593] 7. Natural Language Analysis and Extraction of Important Information

[0594] The server uses generative AI to perform natural language analysis of messages and extract important information, along with the emotional data recognized by the emotion engine. The emotional data is then added to the analysis to identify high-priority requests.

[0595] 8. Directing relief operations and determining routes for transporting relief supplies

[0596] Based on the analyzed information, the server updates a database with the status of relief supplies needed at each evacuation center, and uses generative AI to determine relief operation instructions and optimal delivery routes for relief supplies. It also takes into account the results of sentiment analysis, prioritizing requests with a high level of urgency.

[0597] 9. Distribution of rescue operation instructions

[0598] The server transmits the determined rescue operation instructions and information on the delivery route of relief supplies to rescue workers on the scene in real time, thereby enabling rescue operations to be carried out quickly and efficiently.

[0599] 10. Specific Examples

[0600] For example, if a large-scale earthquake occurs, the server immediately launches multiple drones to fly over areas with severe damage. The drones collect video and image data, which the server inputs into the generation AI in real time. The generation AI then creates a three-dimensional map of the affected area, identifying collapsed buildings and disrupted roads. At the same time, users submit requests for relief supplies from evacuation centers, and based on this information, the optimal route for quickly delivering the necessary supplies is determined. Furthermore, an emotion engine performs emotion analysis, and requests deemed to be of a high level of urgency are prioritized. The server then distributes this information to relief workers, ensuring that appropriate relief efforts are carried out.

[0601] In this way, this system can collect information quickly and accurately when a disaster occurs, and support effective relief efforts that also take into account the user's emotions.

[0602] The processing flow will be explained below.

[0603] Step 1:

[0604] The server collects and monitors weather data, earthquake sensors, and flood monitoring sensors in real time to detect the occurrence of disasters. When a disaster is detected, it immediately issues instructions to start operating the entire system.

[0605] Step 2:

[0606] When the server detects a disaster, it sends activation instructions to multiple unmanned aerial vehicles (drones) located within a pre-defined area.

[0607] Step 3:

[0608] Once the drone (terminal) receives a start command from the server, it begins flying within the designated area according to an automatic flight route. The drone uses a camera to collect high-resolution video and image data.

[0609] Step 4:

[0610] The drone (terminal) transmits video and image data collected during flight to a server in real time.

[0611] Step 5:

[0612] The server stores the video and image data received from the drone in cloud storage.

[0613] Step 6:

[0614] The server inputs the video and image data stored in cloud storage into the generation AI, which analyzes the damage situation.

[0615] Step 7:

[0616] The generating AI compares data before and after the disaster, automatically detecting building damage, road disruption, and the extent of flooding, and generates a three-dimensional map of the affected area.

[0617] Step 8:

[0618] The server visualizes the generated three-dimensional map and analysis results and displays them on a terminal at the emergency response headquarters.

[0619] Step 9:

[0620] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server.

[0621] Step 10:

[0622] The server receives messages sent by users.

[0623] Step 11:

[0624] The server inputs the received message into the emotion engine for emotion analysis. The emotion engine analyzes the text data and determines the emotion the user is feeling.

[0625] Step 12:

[0626] The server uses generative AI to perform natural language analysis of messages and extract important information, along with the emotional data recognized by the emotion engine. The emotional data is then added to the analysis to identify high-priority requests.

[0627] Step 13:

[0628] Based on the analyzed information, the server updates a database with the status of relief supplies needed at each evacuation center, and uses generative AI to determine relief operation instructions and optimal delivery routes for relief supplies. It also takes into account the results of sentiment analysis, prioritizing requests with a high level of urgency.

[0629] Step 14:

[0630] The server transmits the determined rescue operation instructions and information on the route for transporting relief supplies to rescue workers on the scene in real time.

[0631] Step 15:

[0632] The relief workers (users) carry out their activities on site in accordance with the received relief operation instructions and the delivery route of relief supplies.

[0633] This system will enable the rapid and accurate collection of information in the event of a disaster, and will support effective relief efforts that take into account the user's emotions.

[0634] Example 2

[0635] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0636] When a natural disaster occurs, the challenge is to quickly grasp the extent of damage at the scene and implement effective relief efforts. With conventional systems, it takes time to collect information after a disaster occurs, making it difficult to accurately grasp the extent of the damage. It is also difficult to respond appropriately, taking into account the urgency of the victims.

[0637] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0638] In this invention, the server includes means for detecting the occurrence of a disaster, means for launching multiple unmanned aerial vehicles and collecting video and image data of the affected area, means for analyzing the collected video and image data and generating a three-dimensional map of the damage situation, means for visualizing the generated three-dimensional map and distributing countermeasure guidance, means for receiving information from residents of evacuation centers and relief workers and extracting important information through natural language analysis including emotional data, and means for determining rescue operation instructions and delivery routes for relief supplies based on the extracted information. This enables rapid and accurate information collection when a disaster occurs, and enables effective rescue operations that take into account the emotions and urgency of the victims.

[0639] "Means for detecting the occurrence of disasters" refers to a system that collects information from data sources such as seismometers, meteorological data, and flood monitoring sensors, and detects the occurrence of natural disasters in real time.

[0640] An "unmanned aerial vehicle" is a device that flies automatically or remotely without a human on board and collects video and image data of the affected area.

[0641] "Means for collecting video and image data of the affected area" refers to a mechanism for collecting video and image data of the affected area using a camera mounted on an unmanned aerial vehicle.

[0642] "Means for analyzing collected video and image data and generating a three-dimensional map of the damage situation" refers to a system that uses image analysis technology to reconstruct the damage situation as a three-dimensional model based on the acquired data.

[0643] "Means for visualizing the generated 3D map and distributing countermeasure guidance" refers to a function that displays a 3D map on a terminal at the countermeasure headquarters, generates appropriate action guidelines according to the situation, and distributes them to relevant parties.

[0644] "Means for receiving information from evacuation center residents and relief workers and extracting important information through natural language analysis, including emotional data" refers to a system that receives messages sent from evacuation centers and on-site locations, analyzes them using natural language processing technology, including emotional data, and identifies information with a high level of urgency.

[0645] The "means for determining rescue operation instructions and transportation routes for relief supplies" is a system that, based on analyzed information, issues action instructions for efficiently providing necessary support and determines the optimal transportation route for relief supplies.

[0646] The disaster response system of the present invention is designed to quickly and efficiently assess the damage situation and carry out appropriate relief activities when a disaster occurs. The system includes various hardware and software components.

[0647] Hardware and Software Configuration

[0648] 1. Server:

[0649] The server collects information from weather data APIs, earthquake sensors, flood monitoring sensors, etc., and monitors disaster occurrences in real time. When it determines that a disaster has occurred, it starts operating the entire system.

[0650] 2. Unmanned Aerial Vehicles (Drones):

[0651] When a disaster occurs, the server sends instructions to launch multiple drones in a designated area. The drones follow autonomous flight routes and use their onboard cameras to collect high-resolution video and image data of the affected area.

[0652] 3. Data Collection and Transmission:

[0653] The drone (terminal) transmits the video and image data it acquires during flight to a server in real time using Wi-Fi or mobile networks.

[0654] 4. Data analysis and three-dimensional map generation:

[0655] The server stores the received data in cloud storage and inputs it into a generative AI model. The generative AI model compares data from before and after the disaster, analyzes building damage, road disruption, and the extent of flood damage, and generates a 3D map of the affected area. For example, it uses image recognition technology to identify damage and creates a 3D model based on this.

[0656] 5. Visualization of damage situation and guidance on countermeasures:

[0657] The server visualizes and displays the generated 3D map on the headquarters' terminal. This allows the headquarters to quickly grasp the damage situation and make decisions quickly. Furthermore, the generative AI model automatically generates countermeasure guidance based on the analysis results and distributes this to all parties involved. Specifically, the visualization tool is used to draw the 3D map, and the report generation function is used to create the guidance content.

[0658] 6. Collecting information from the field and analyzing sentiment:

[0659] Users (refugee residents and relief workers) request information about the local situation and necessary relief supplies through a dedicated app or chatbot. This request is sent to a server. The server passes the received message to an emotion engine, which analyzes the emotion in the message. For example, it can automatically determine the emotion of the message using a text analysis API.

[0660] 7. Natural Language Analysis and Key Information Extraction:

[0661] The server passes the emotion data recognized by the emotion engine to a generative AI model, which then performs natural language analysis of the message to extract important information. The emotion data is used as an indicator to identify high-priority requests. Specifically, the message is analyzed using a natural language processing API.

[0662] 8. Directing relief operations and determining routes for transporting relief supplies:

[0663] The server updates the database with the status of relief supplies needed at each evacuation center based on the analysis results. The generative AI model determines the optimal route for delivering relief supplies and generates instructions for relief operations. It also takes into account the results of sentiment analysis and prioritizes requests with high urgency. For example, it uses a route optimization algorithm to calculate efficient routes.

[0664] 9. Distribution of rescue operation instructions:

[0665] The server sends rescue operation instructions and delivery route information for relief supplies to rescue workers in real time, allowing rescue operations to proceed quickly and efficiently. Specifically, it sends instructions to relevant parties using a notification system.

[0666] Examples of concrete examples and prompts

[0667] Examples:

[0668] For example, if a large-scale earthquake occurs, the server immediately launches multiple drones to fly over areas with severe damage. The drones collect video and image data, which the server inputs into a generative AI model in real time. The generative AI model then generates a three-dimensional map of the affected area, identifying collapsed buildings and disrupted roads. At the same time, users submit requests for relief supplies from evacuation centers, and this information is used to determine the optimal route for quickly delivering the necessary supplies. An emotion engine performs sentiment analysis, prioritizing requests with a high level of urgency from users. The server then distributes this information to relief workers, supporting rapid and appropriate relief efforts.

[0669] Example prompt sentence:

[0670] "In areas recently affected by disasters, please analyze the received image data and generate a 3D map of the damage. Also, please consider the relief supply requests sent from evacuation centers, determine the optimal delivery route, and prioritize requests with high priority."

[0671] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0672] Step 1:

[0673] Disaster detection

[0674] The server collects information from multiple data sources, such as seismometers, weather data APIs, and flood monitoring sensors, and monitors the occurrence of natural disasters in real time. The input is information obtained from each sensor and API, and based on this, it performs data calculations to compare it with a disaster occurrence threshold. The output is a signal that sets a "disaster occurrence detection" flag.

[0675] Specific operation:

[0676] 1. Get current weather information from the weather data API.

[0677] 2. Obtain real-time earthquake data from seismometers.

[0678] 3. Compare the acquired data with a threshold to determine whether a disaster has occurred.

[0679] 4. When a disaster is detected, the entire system begins operation.

[0680] Step 2:

[0681] Launching and flying the drone

[0682] When the server detects a disaster, it sends an instruction to launch multiple drones in a designated area. The input is a disaster detection signal and designated area information, and the output is a drone launch command. The drones follow an autonomous flight route and collect video and image data of the affected area.

[0683] Specific operation:

[0684] 1. Automatically launch multiple drones using the drone control API.

[0685] 2. Send the flight route and coordinate information of the affected area to the activated drone.

[0686] 3. The drone begins flying along the specified route.

[0687] Step 3:

[0688] Data collection and transmission

[0689] The drone (terminal) captures high-resolution video and image data using an onboard camera during flight and transmits this data to a server in real time. The input is the video and image data captured by the drone's camera, and the output is the raw data transmitted to the server.

[0690] Specific operation:

[0691] 1. Use the drone camera to capture video and images of the affected area.

[0692] 2. The acquired data is sent to the server in real time.

[0693] Step 4:

[0694] Data analysis and 3D map generation

[0695] The server stores the received video and image data in cloud storage and inputs it into a generative AI model. The generative AI model compares data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. The input is the data stored in cloud storage and data before and after the disaster, and the output is a 3D map of the affected area.

[0696] Specific operation:

[0697] 1. Save the received data in cloud storage.

[0698] 2. Input data into the generative AI model and analyze the damage situation.

[0699] 3. Generate a three-dimensional map of the affected area based on the analysis results.

[0700] Step 5:

[0701] Visualization of damage status and delivery of countermeasure guidance

[0702] The server visualizes and displays the generated 3D map on the terminal of the emergency response headquarters. The input is the generated 3D map, and the output is the map displayed on the terminal of the emergency response headquarters and countermeasure guidance. Countermeasure guidance is also automatically generated based on the analysis results of the generative AI model and distributed to relevant parties.

[0703] Specific operation:

[0704] 1. Use visualization tools to display 3D maps on the headquarters terminals.

[0705] 2. Create countermeasure guidance based on the analysis results of the generative AI model.

[0706] 3. Distribute the created guidance to each stakeholder.

[0707] Step 6:

[0708] Collecting information from the field and analyzing emotions

[0709] Users (refugee residents and relief workers) use a dedicated app or chatbot to request information about the local situation and necessary relief supplies. The server passes these messages to the emotion engine, which analyzes the emotions in the messages. The input is the message from the user, and the output is the analyzed emotion data.

[0710] Specific operation:

[0711] 1. Users submit information about the local situation and requests for relief supplies through a dedicated app or chatbot.

[0712] 2. The server receives the message and analyzes the emotion using the emotion engine.

[0713] Step 7:

[0714] Natural language analysis and extraction of important information

[0715] The server passes the emotion data recognized by the emotion engine to the generative AI model, which then performs natural language analysis of the message to extract important information. The input is the emotion data and the message, and the output is the extracted important information.

[0716] Specific operation:

[0717] 1. Analyze key messages based on sentiment data.

[0718] 2. Input emotion data into a generative AI model to extract key information.

[0719] Step 8:

[0720] Directing relief operations and determining routes for transporting relief supplies

[0721] Based on the analyzed information, the server updates the database with the status of relief supplies needed at each evacuation shelter. The generative AI model determines the optimal delivery route for relief supplies and generates rescue operation instructions. The input is the extracted important information and the current status of relief supplies, and the output is the confirmed delivery route and rescue operation instructions.

[0722] Specific operation:

[0723] 1. Based on the analysis results, the status of relief supplies is updated in the database.

[0724] 2. Determine the optimal delivery route using a generative AI model.

[0725] 3. Generate rescue operation instructions.

[0726] Step 9:

[0727] Distribution of rescue operation instructions

[0728] The server transmits rescue operation instructions and relief supply delivery route information to rescue workers in the field in real time. The input is the confirmed delivery route and rescue operation instructions, and the output is the information to be distributed to rescue workers in the field.

[0729] Specific operation:

[0730] 1. Send confirmed transport routes and rescue instructions to rescue workers on the scene.

[0731] 2. Deliver quickly with a real-time notification system.

[0732] (Application example 2)

[0733] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0734] When a disaster occurs, it is necessary to quickly and accurately grasp the damage situation and carry out efficient relief operations, but conventional systems have had problems with delays in information gathering and overlooking important information, making it difficult to respond quickly. Another issue was the inability to provide appropriate instructions for relief operations that take into account the user's emotions.

[0735] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for detecting the occurrence of a disaster, means for launching multiple unmanned aerial vehicles and collecting video and image data of the affected area, means for analyzing the collected video and image data and generating a three-dimensional map of the damage situation, means for visualizing the generated three-dimensional map and distributing countermeasure guidance, means for receiving information from residents of evacuation centers and relief workers and extracting important information through natural language analysis, means for analyzing the sentiment of users' messages and identifying high-priority requests, means for determining rescue operation instructions and delivery routes for relief supplies based on the extracted information, and means for transmitting rescue operation instructions to relief workers on the scene in real time. This enables a quick and accurate understanding of the damage situation and appropriate rescue operation instructions that take the user's sentiment into consideration.

[0736] "Means for detecting the occurrence of disasters" refers to sensors and data analysis systems that allow a server to monitor natural disasters such as earthquakes, heavy rain, and floods in real time and determine whether a disaster has occurred.

[0737] An "unmanned aerial vehicle" is a drone or other unmanned aircraft equipped with a camera and following an automated flight route.

[0738] The "means for collecting video and image data of the affected area" is a system that allows unmanned aerial vehicles to capture video and image data of the affected area and send it to a server.

[0739] The "means for generating a three-dimensional map" is a system that analyzes collected video and image data and visually displays the damage situation as a three-dimensional map.

[0740] The "means of delivering guidance on countermeasures" is a system that delivers information and guidelines for action to relevant organizations and personnel based on the generated three-dimensional map to support rapid decision-making.

[0741] The "means for extracting key information through natural language analysis" refers to a generative AI model and natural language processing system for analyzing messages received from evacuation center residents and relief workers and extracting key information from them.

[0742] The "means for analyzing emotions and identifying high-priority requests" is a system that analyzes the emotions in messages from users and determines the priority of requests according to their urgency and importance.

[0743] The "means for determining rescue operation instructions and transportation routes for relief supplies" is a system that determines optimal rescue operation instructions and transportation routes for relief supplies based on the extracted important information and emotion analysis results.

[0744] The "means for transmitting rescue operation instructions to rescue workers on the scene in real time" is a communication system for quickly transmitting the determined rescue operation instructions and information on the transportation route of relief supplies to rescue workers on the scene.

[0745] The disaster response system of the present invention includes various hardware and software components that enable rapid and efficient assessment of the damage situation and appropriate rescue operations when a disaster occurs. The main components of this system include a disaster detection means, an unmanned aerial vehicle (drone), a server, a generative AI, a user interface, and an emotion engine.

[0746] The server collects and analyzes information from weather data, earthquake sensors, flood monitoring sensors, etc. in order to monitor the occurrence of natural disasters such as earthquakes, heavy rain, and floods in real time. Based on this information, if it determines that a disaster has occurred, the entire system immediately begins operation. When the server detects a disaster, it sends instructions to launch multiple unmanned aerial vehicles (drones) in a specified area. The drones follow automatic flight routes and use cameras to collect video and image data of the affected area, which they then send to the server in real time.

[0747] The server stores the high-resolution video and image data sent from the drone in cloud storage (e.g., Amazon S3) and inputs it into a generation AI (e.g., OpenAI GPT model). The generation AI compares data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. Based on the analysis results, it generates a 3D map of the affected area, which is visualized and displayed on a terminal at the response headquarters. Furthermore, it automatically generates countermeasure guidance based on the analysis results and distributes it to all parties involved.

[0748] Users (refugee residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to a server. The server receives the messages sent by the users and analyzes the emotions in the messages using an emotion engine (e.g., IBM Watson). The emotion engine analyzes the text data and determines the user's emotions. The server then uses generative AI to perform natural language analysis of the messages along with the emotion data recognized by the emotion engine, and extracts important information. The emotion data is added to the analysis and used as an indicator to identify high-priority requests.

[0749] Based on the analyzed information, the server determines optimal rescue operation instructions and delivery routes for relief supplies. This is done using a generative AI model to automatically generate efficient routes and operation instructions. The determined rescue operation instructions and delivery route information for relief supplies are sent to rescue workers on the ground in real time, allowing rescue operations to be carried out quickly and efficiently.

[0750] As a specific example, if a large-scale earthquake occurs, the server immediately launches multiple drones to fly over areas with severe damage. The drones collect video and image data, which the server inputs into the generation AI in real time. The generation AI creates a three-dimensional map of the affected area and identifies collapsed buildings and severed roads. At the same time, users submit requests for relief supplies from evacuation centers, and based on this information, the optimal route for quickly delivering the necessary supplies is determined. Furthermore, an emotion engine performs emotion analysis, and requests that are deemed to be of high urgency by the user are prioritized.

[0751] For example, if a user sends a message such as "A major earthquake has occurred and I am in a shelter. I am short of water and food," emotion analysis and natural language analysis can quickly identify that emergency assistance is needed in that area. Based on this information, the server can then deploy efficient relief activities.

[0752] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0753] Step 1:

[0754] The server monitors various sensors and meteorological data to detect the occurrence of disasters. The input is data obtained from earthquake sensors and weather information APIs, and the server analyzes this data to determine the occurrence of a natural disaster. The output is an internal flag indicating that a disaster has occurred. Based on this flag, the operation to proceed to the next step is triggered.

[0755] Step 2:

[0756] When the server detects that a disaster has occurred, it sends an instruction to launch multiple unmanned aerial vehicles (drones) in a specified area. The input is a disaster detection flag and coordinate data of the affected area, and the output is a drone launch signal. Specifically, the server calls the drone's launch API and sets an automatic flight route.

[0757] Step 3:

[0758] The unmanned aerial vehicle (drone) follows an automated flight route and uses a camera to collect video and image data of the affected area. The input is the drone's flight route data and camera setting data, and the output is high-resolution video and image data. Specifically, the drone operates by continuously taking photos and videos while flying a predetermined route.

[0759] Step 4:

[0760] The drone transmits the collected video and image data to a server in real time. The input is the video and image data captured by the drone's camera, and the output is a data stream to the server. Specifically, the drone transmits the data to the server using wireless communication.

[0761] Step 5:

[0762] The server stores the data sent from the drone in cloud storage (e.g., Amazon S3) and then inputs it into the generation AI. The input is the video and image data received from the drone, and the output is an analysis request to the generation AI. Specifically, the server uploads the data to cloud storage and passes the reference link to the generation AI.

[0763] Step 6:

[0764] The generative AI (for example, the OpenAI GPT model) compares data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. The input is a cloud storage link and analysis instructions, and the output is 3D map data of the affected area. Specifically, the generative AI analyzes the data and visualizes the damage situation in 3D.

[0765] Step 7:

[0766] The server visualizes and displays the 3D map generated by the generation AI on the terminal at the emergency headquarters. The input is 3D map data, and the output is visual information displayed on the screen at the emergency headquarters. Specifically, the server renders the 3D map and displays it on the monitor at the emergency headquarters.

[0767] Step 8:

[0768] Users (shelter residents and relief workers) use a dedicated app or chatbot to send local information and requests for relief supplies to a server. The input is text information entered by the user into the app or chatbot, and the output is the request data received by the server. The specific operation is for the user to request the necessary relief supplies through the app.

[0769] Step 9:

[0770] The server receives messages sent by users and performs emotion analysis using an emotion engine (e.g., IBM Watson). The input is the user's text message, and the output is the analyzed emotion data. Specifically, the server sends messages to the emotion engine and receives emotion data.

[0771] Step 10:

[0772] The server uses the generation AI along with the emotion data recognized by the emotion engine to perform natural language analysis of the message and extract important information. The input is the user's message and emotion data, and the output is an extracted list of important information. Specifically, the generation AI performs natural language analysis and determines the priority of the request.

[0773] Step 11:

[0774] The server determines optimal rescue operation instructions and delivery routes for relief supplies based on the analyzed information. The input is an extracted list of important information and emotion data, and the output is rescue operation instructions and delivery route determination data. Specifically, the server calculates the optimal route and operation instructions with the help of generation AI.

[0775] Step 12:

[0776] The server transmits the determined rescue operation instructions and information on the delivery route of relief supplies to the rescue workers on the scene in real time. The input is the rescue operation instructions and delivery route determination data, and the output is a notification of instructions to the rescue workers. Specifically, the server transmits information to the scene via the communication system.

[0777] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0778] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0779] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0780] [Third embodiment]

[0781] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0782] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0783] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0784] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0785] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0786] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0787] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0788] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0789] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0790] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0791] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0792] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0793] The disaster response system of the present invention includes various hardware and software components that enable rapid and efficient assessment of the damage situation and appropriate rescue operations when a disaster occurs. The main components of this system include a disaster detection means, an unmanned aerial vehicle (drone), a server, a generation AI, and a user interface.

[0794] 1. Disaster detection methods

[0795] The server collects information from weather data, earthquake sensors, flood monitoring sensors, etc. in order to monitor the occurrence of natural disasters such as earthquakes, heavy rains, and floods in real time. If it determines that a disaster has occurred based on this information, the entire system will immediately begin operating.

[0796] 2. Launching and flying unmanned aerial vehicles (drones)

[0797] When the server detects a disaster, it sends instructions to launch multiple drones in a designated area, which follow automated flight routes and use their cameras to collect video and image data of the affected area.

[0798] 3. Data collection and transmission

[0799] The drone (terminal) transmits high-resolution video and image data acquired during flight to a server in real time, enabling a rapid assessment of the damage situation.

[0800] 4. Data Analysis and 3D Map Generation

[0801] The server stores the video and image data sent from the drone in cloud storage and inputs it into the generation AI. The generation AI compares the data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. Based on the results of this analysis, it generates a 3D map of the affected area.

[0802] 5. Visualizing the damage situation and providing guidance on countermeasures

[0803] The server visualizes and displays the 3D map generated by the AI ​​on the terminals at the emergency response headquarters. This allows for a visual understanding of the damage situation and enables quick decision-making. Furthermore, it automatically generates countermeasure guidance based on the analysis results and distributes it to all parties involved.

[0804] 6. Collecting and analyzing information from the field

[0805] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server. The server receives the information, and the generative AI performs natural language analysis to extract important information.

[0806] 7. Directing relief operations and determining routes for transporting relief supplies

[0807] Based on the extracted information, the server updates the database with the status of relief supplies needed at each evacuation shelter, determines instructions for relief operations and routes for delivering relief supplies, and transmits the instructions and route information to relief workers on the ground in real time.

[0808] 8. Specific Examples

[0809] For example, if a large-scale earthquake occurs, the server immediately launches multiple drones to fly over areas with severe damage. The drones collect video and image data, which the server inputs into the generation AI in real time. The generation AI then creates a three-dimensional map of the affected area, identifying collapsed buildings and disrupted roads. At the same time, users submit requests for relief supplies from evacuation centers, and this information is used to determine the optimal route for quickly delivering the necessary supplies. The server then distributes this information to relief workers, ensuring that appropriate relief efforts are carried out.

[0810] In this way, this system can collect information quickly and accurately when a disaster occurs, and support effective relief efforts.

[0811] The processing flow will be explained below.

[0812] Step 1:

[0813] The server collects and monitors weather data, earthquake sensors, and flood monitoring sensors in real time to detect the occurrence of disasters. When a disaster is detected, it immediately issues instructions to start operating the entire system.

[0814] Step 2:

[0815] When the server detects a disaster, it sends activation instructions to multiple unmanned aerial vehicles (drones) located within a pre-defined area.

[0816] Step 3:

[0817] Once the drone (terminal) receives a start command from the server, it begins flying within the designated area according to an automatic flight route. The drone uses a camera to collect high-resolution video and image data.

[0818] Step 4:

[0819] The drone (terminal) transmits video and image data collected during flight to a server in real time.

[0820] Step 5:

[0821] The server stores the video and image data received from the drone in cloud storage.

[0822] Step 6:

[0823] The server inputs the video and image data stored in cloud storage into the generation AI, which analyzes the damage situation.

[0824] Step 7:

[0825] The generating AI compares data before and after the disaster, automatically detecting building damage, road disruption, and the extent of flooding, and generates a three-dimensional map of the affected area.

[0826] Step 8:

[0827] The server visualizes the generated three-dimensional map and analysis results and displays them on a terminal at the emergency response headquarters.

[0828] Step 9:

[0829] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server.

[0830] Step 10:

[0831] The server receives messages sent by users and uses generative AI to perform natural language analysis and extract important information.

[0832] Step 11:

[0833] Based on the analyzed information, the server updates the database with the status of relief supplies at each evacuation shelter.

[0834] Step 12:

[0835] The server uses generative AI based on the information in the database to determine rescue operation instructions and the optimal route for transporting relief supplies.

[0836] Step 13:

[0837] The server transmits the determined rescue operation instructions and information on the route for transporting relief supplies to rescue workers on the scene in real time.

[0838] Step 14:

[0839] The relief workers (users) carry out their activities on site in accordance with the received relief operation instructions and the delivery route of relief supplies.

[0840] This system will enable rapid and accurate collection of information in the event of a disaster, enabling appropriate and effective relief efforts to be carried out.

[0841] Example 1

[0842] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0843] The problem is that when a disaster occurs, there is a lack of means to collect and analyze information to quickly and accurately grasp the damage situation and carry out effective relief activities. With conventional methods, it takes time to accurately grasp the damage situation on the ground, and the accuracy of the information is limited, making it difficult to formulate appropriate relief operation plans.

[0844] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0845] In this invention, the server includes means for detecting the occurrence of a disaster, means for launching multiple unmanned aerial vehicles and collecting video and image data of the affected area, means for analyzing the collected video and image data and generating a three-dimensional map of the damage situation, means for visualizing the generated three-dimensional map and distributing countermeasure instructions, means for receiving information from evacuation shelter residents and relief workers and extracting important information through natural language analysis, means for determining rescue operation instructions and delivery routes for relief supplies based on the extracted information, means for automatically analyzing the damage situation and providing countermeasure instructions using a generative AI model, means for immediately recognizing the occurrence of a disaster and emergency activation of the system, and means for saving data captured by the unmanned aerial vehicle's camera in real time to cloud storage. This enables rapid and accurate information collection and analysis, supporting effective rescue operations.

[0846] "Means for detecting the occurrence of disasters" refers to devices or systems that monitor the occurrence of natural disasters such as earthquakes, heavy rains, and floods in real time, collect and analyze data, and determine whether a disaster has occurred.

[0847] "Multiple unmanned aerial vehicles" means an aircraft system that includes drones that are flown by remote control or autonomous pilot and used to monitor a specific area.

[0848] "Video and image data of the affected areas" refers to video and photographic data taken by unmanned aerial vehicles to show the current state of the areas affected by the disaster.

[0849] The "means of collection" is a communication device or protocol for transferring video and image data captured by the unmanned aerial vehicle to a specific server or cloud storage.

[0850] "Means for generating a three-dimensional map" refers to software or algorithms that analyze collected video and image data and recreate the extent and details of the damage in three-dimensional space.

[0851] The "means for visualization and delivery of countermeasure guidance" is a system that displays the generated three-dimensional map on a terminal display and notifies relevant parties of rescue operation instructions and countermeasures.

[0852] "Means for extracting important information through natural language analysis" refers to algorithms or software that analyze text data sent by evacuation shelter residents and relief workers and automatically extract important information necessary for providing assistance.

[0853] The "means for determining rescue operation instructions and transportation routes for relief supplies" is a system that uses the extracted information to create a list of relief supplies needed at each evacuation shelter and makes a plan for transporting them via the optimal route.

[0854] "Means for automatically analyzing damage situations and providing guidance on countermeasures using generative AI models" refers to machine learning models or AI systems that analyze damage situations based on collected data and automatically propose optimal countermeasures.

[0855] "Means for immediately recognizing the occurrence of a disaster and for emergency activation of the system" refers to a combination of hardware and software that immediately activates the entire system and initiates necessary measures when a disaster is detected.

[0856] "Data captured by the camera of an unmanned aerial vehicle" refers to visual information collected through a camera mounted on an unmanned aerial vehicle, and is a means of storing this information in cloud storage in real time.

[0857] The present invention is directed to a system that aims to quickly and accurately grasp the damage situation when a disaster occurs and to support effective relief efforts. The present invention is configured by combining various hardware and software components.

[0858] Disaster detection

[0859] To detect the occurrence of a disaster, the server collects and analyzes real-time data from weather data, earthquake sensors, flood monitoring sensors, etc. This is done using, for example, the Japan Meteorological Agency's API or specific sensor devices. When the server detects data that exceeds a certain threshold, it recognizes the occurrence of a disaster and issues a command to immediately start up the operation of the entire system. Specifically, it uses software that immediately recognizes the occurrence of a disaster and immediately starts up the system.

[0860] Launching and flying unmanned aerial vehicles

[0861] The server launches multiple unmanned aerial vehicles (drones) via drone control software in areas where a disaster has been confirmed. These drones follow automated flight routes and use cameras to collect video and image data of the affected area. The drones are equipped with high-resolution camera modules and fly autonomously along pre-programmed flight paths. Flight control is performed using the DroneKit API, for example.

[0862] Data collection and transmission

[0863] The drone (terminal) transmits high-resolution video and image data acquired during flight to a server in real time over a 5G network, and the data is stored in a cloud storage service, specifically Amazon S3.

[0864] Data analysis and 3D map generation

[0865] The server inputs the video and image data stored in cloud storage into a generative AI model. This generative AI model was developed using TensorFlow and other tools, and analyzes the extent and details of the damage. Based on the analysis results, a three-dimensional map of the affected area is generated. This map is saved for further detailed analysis and is linked to the emergency response headquarters terminal.

[0866] Visualization of damage status and delivery of countermeasure guidance

[0867] The server then displays the generated 3D map on the device's display, using a visualization tool such as Tableau. The automated guidance generated by the AI ​​model is then distributed to relevant parties via the SendGrid API.

[0868] Collecting and analyzing information from the field

[0869] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server. This application is a mobile app using Firebase. The server receives this information and performs natural language analysis using a generative AI model, which automatically extracts important information.

[0870] Directing relief operations and determining routes for transporting relief supplies

[0871] The server updates a database with the status of relief supplies needed at each evacuation center based on the information extracted through natural language analysis. It then uses the Google Maps API to calculate optimal delivery routes and generate rescue operation instructions. This information is then sent in real time to rescue workers on the ground using Firebase Cloud Messaging.

[0872] Specific examples

[0873] For example, if a large-scale earthquake occurs, the server immediately recognizes the occurrence of a disaster based on data obtained from the Japan Meteorological Agency API and launches multiple drones using the DroneKit API. The drones store the acquired data in Amazon S3 via the 5G network, and this data is input into a generative AI model using TensorFlow. The analysis results are displayed in Tableau and distributed to relevant parties via the SendGrid API. Requests from evacuation centers are received through the Firebase app, and the information is analyzed using a natural language analysis model. The optimal route for delivering relief supplies is calculated using the Google Maps API and sent to relief workers via Firebase Cloud Messaging.

[0874] Prompt Sentence Examples

[0875] "Please tell me the detailed operation procedure of the system that quickly grasps the damage situation in an area where an earthquake has occurred and uses a generative AI model to plan rescue operations."

[0876] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0877] Step 1: Detecting the occurrence of a disaster

[0878] The server collects data in real time from sources such as weather data, earthquake sensors, and flood monitoring sensors. Specifically, it takes in data obtained from APIs and sensor devices every minute. Statistical analysis is performed based on this data, and if a certain threshold is exceeded, a disaster is recognized. Various disaster data is input, and a disaster occurrence signal is generated as output.

[0879] Step 2: Launch command for unmanned aerial vehicles

[0880] When the server confirms that a disaster has occurred, it sends instructions to launch multiple unmanned aerial vehicles (drones). This instruction is sent via drone control software (e.g., DroneKit API). The input is a disaster occurrence signal, and the output is a command to launch the drones. Specific operations include setting an automatic flight route based on GPS coordinates and activating cameras.

[0881] Step 3: Video and image data collection

[0882] The drone (terminal) collects high-resolution video and image data while flying along an automatic flight route. It uses a camera module (e.g., DJI Zenmuse X7) to capture images along the specified route. The input is the set flight route information, and the output is high-resolution video and image data.

[0883] Step 4: Sending data in real time

[0884] The drone transmits the collected video and image data to a server in real time. The communication method is a 5G network. The input is high-resolution video and image data, and the output is the data sent to the server. Specifically, the data is packetized and transmitted.

[0885] Step 5: Save your data to the cloud

[0886] The server stores the received video and image data in a cloud storage service (e.g., Amazon S3). The input is data sent in real time, and the output is data stored in the cloud storage. Specifically, the data is stream-processed and converted into files.

[0887] Step 6: Generate a 3D map

[0888] The server inputs the data stored in cloud storage into a generative AI model and analyzes the damage situation. Using the generative AI model (e.g., TensorFlow), it compares data before and after the disaster and generates a 3D map. The input is the stored video and image data, and the output is a 3D map. Specific operations include preprocessing the data, inputting it into the trained model, and outputting the analysis results.

[0889] Step 7: Visualize the damage situation

[0890] The server displays the generated 3D map on the emergency headquarters terminal, allowing the damage situation to be visually grasped. The input is the generated 3D map data, and the output is the map displayed on the terminal. Specifically, the 3D map data is rendered and displayed.

[0891] Step 8: Automatic generation and distribution of countermeasure guidance

[0892] The server uses a generative AI model to automatically generate countermeasure guidance based on the analysis results and distributes it to relevant parties. The input is the analysis results and a 3D map, and the output is the automatically generated countermeasure guidance and distributed guidance message. Specific operations involve the use of an email distribution system (e.g., SendGrid API).

[0893] Step 9: Gather information from the field

[0894] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for relief supplies to the server. The input is text, images, and audio data, and the output is the information sent to the server. Specific operations involve the use of a mobile app (e.g., Firebase app).

[0895] Step 10: Extracting important information through natural language analysis

[0896] The server uses a generative AI model to perform natural language analysis on the received local information and extract important information. The input is the transmitted text, images, and audio data, and the output is the extracted important information. The specific operation involves applying a natural language processing algorithm.

[0897] Step 11: Deciding on relief operation instructions and relief supply transport routes

[0898] The server updates the database based on the extracted information and determines rescue operation instructions and delivery routes for relief supplies. The input is important information and the current supply situation, and the output is optimized delivery routes and rescue instructions. Specifically, a route calculation algorithm (e.g., Google Maps API) is applied.

[0899] Step 12: Send instructions to relief workers

[0900] The server sends the determined instructions and transport route information to the rescue workers on the scene via push notification. The input is the rescue instructions and transport route, and the output is the information sent to the rescue workers' devices. The specific operation is to send a push notification (e.g., Firebase Cloud Messaging).

[0901] (Application example 1)

[0902] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0903] When a disaster occurs, it is important to quickly and accurately grasp the damage situation. To quickly and appropriately carry out relief efforts, it is necessary to collect and analyze information from various evacuation centers and relief activity sites. However, the current system does not efficiently coordinate the understanding of the damage situation and instructions for relief efforts, making it particularly difficult to optimize the transportation routes of supplies. Furthermore, logistics centers do not have the appropriate means to rapidly distribute relief supplies.

[0904] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0905] In this invention, the server includes means for detecting the occurrence of a disaster, means for launching multiple unmanned aerial vehicles to collect video and image data of the affected area, means for analyzing the collected video and image data and generating a 3D map of the damage situation, and means for displaying the generated 3D map on a terminal at the logistics center, and for quickly managing requests for relief supplies and optimizing delivery routes. This enables a detailed understanding of the damage situation in the event of a disaster and efficient distribution of relief supplies via the logistics center.

[0906] "Means for detecting the occurrence of disasters" refers to a system that monitors and notifies in real time the occurrence of natural disasters such as earthquakes, heavy rain, and floods.

[0907] An "unmanned aerial vehicle" is a device, such as a drone, that flies remotely or autonomously and collects video and image data of a specific area.

[0908] "Video and image data" refers to video and photographic data used to visually record the situation in the affected areas.

[0909] A "three-dimensional map" is a map that displays the state of the terrain and buildings before and after a disaster in three dimensions.

[0910] A "generative AI model" is an artificial intelligence system that analyzes collected data and generates new data and information.

[0911] The "means for delivering guidance on countermeasures" is a system that automatically notifies each relevant party of appropriate countermeasures and instructions based on the analyzed data.

[0912] "Means for receiving information from evacuation center residents and relief workers" refers to a system for receiving information on the local situation and requests for relief supplies from evacuation centers and local relief workers.

[0913] "Natural language analysis" is a technology for analyzing received information and extracting important information.

[0914] A "relief supply transport route" is a route for efficiently delivering necessary relief supplies to evacuation centers and disaster-stricken areas.

[0915] A "logistics center" is a facility for distributing and managing relief supplies.

[0916] A "terminal" is a computer system for displaying and manipulating information.

[0917] "Relief supply request management" is the process of receiving requests for relief supplies from evacuation centers and disaster-stricken areas and managing the necessary supplies.

[0918] "Route optimization" is the process of planning and executing the safe delivery of relief supplies via the shortest and most optimal route.

[0919] This invention provides a system for quickly and accurately assessing the damage situation when a disaster occurs, and for efficiently carrying out appropriate relief activities and transporting relief supplies. The system includes the following main hardware and software components:

[0920] 1. Means of detecting disasters

[0921] The server monitors disasters such as earthquakes, heavy rains, and floods in real time, collecting information from weather data, earthquake sensors, flood monitoring sensors, etc. If it determines that a disaster has occurred, it immediately starts operating the entire system.

[0922] 2. Starting and flying the drone

[0923] When the server detects a disaster, it sends instructions to launch multiple unmanned aerial vehicles in a designated area. The drones follow automated flight routes and use their onboard cameras to collect video and image data of the affected area.

[0924] 3. Data collection and transmission

[0925] The drones capture high-resolution video and image data during flight and transmit it to a server in real time, enabling a rapid assessment of the damage situation.

[0926] 4. Data Analysis and 3D Map Generation

[0927] The server stores the video and image data sent from the drone in cloud storage (AWS S3) and inputs it into a generative AI model (OpenAI API). The generative AI compares data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. Based on the results of this analysis, it generates a 3D map of the affected area.

[0928] 5. Visualizing the damage situation and providing guidance on countermeasures

[0929] The server visualizes and displays the 3D map generated by the AI ​​on terminals at the logistics center and the emergency response headquarters. This allows for a visual understanding of the damage situation, enabling quick decision-making. Furthermore, guidance on countermeasures based on the analysis results is distributed to all parties in real time.

[0930] 6. Collecting and analyzing information from the field

[0931] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server. The server receives the information, and the generative AI performs natural language analysis to extract important information.

[0932] 7. Directing relief operations and determining routes for transporting relief supplies

[0933] Based on the extracted information, the server updates the database with the status of relief supplies needed at each evacuation shelter, determines optimal instructions for relief efforts, and determines the optimal route for delivering relief supplies. The determined instructions and route information are sent in real time to relief workers on the ground.

[0934] Specific examples

[0935] For example, if a large-scale earthquake occurs, the server immediately launches multiple drones to fly over areas with severe damage. The drones collect video and image data, which the server inputs into the generation AI in real time. The generation AI generates a three-dimensional map of the affected area and identifies collapsed buildings and disrupted roads. At the same time, users submit requests for relief supplies from evacuation centers, and this information is used to determine the optimal route for quickly delivering the necessary supplies. The server then distributes this information to relief workers, ensuring that appropriate relief efforts are carried out.

[0936] Prompt Sentence Examples

[0937] I would like to use the "Disaster Logistics Manager" app to carry out appropriate relief efforts in the event of a large-scale earthquake. Collect data from drones, analyze the damage situation with AI, create a 3D map, manage requests for relief supplies, and generate optimal delivery routes.

[0938] This system will enable rapid and accurate information gathering in the event of a disaster and efficient distribution of relief supplies from logistics centers.

[0939] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0940] Step 1:

[0941] The server detects the occurrence of a disaster in real time. The main input data is weather data, information from earthquake sensors, and flood monitoring sensors. The server aggregates this data and determines whether a disaster has occurred. The output of this step is the determination result of "disaster occurrence." If a disaster occurs, the system begins operation.

[0942] Step 2:

[0943] When the server detects a disaster, it immediately sends instructions to launch multiple unmanned aerial vehicles (drones). These instructions include the flight area and automatic flight route. The drones receive the instructions and fly autonomously within the designated disaster area. The input for this step is instructions from the server, and the output is the launch and flight of the drones.

[0944] Step 3:

[0945] The drone uses its on-board camera to collect high-resolution video and image data of the affected area. This data is sent to the server in real time. The input is the aerial video and image data, and the output is the data sent to the server. In this step, the drone performs the specific operation of sending the captured data to the server.

[0946] Step 4:

[0947] The server stores the video and image data sent from the drone in cloud storage (AWS S3). Next, this data is input into a generative AI model (OpenAI API) to begin analyzing the data. During the analysis process, data before and after the disaster is compared to identify building damage, road disruption, the extent of flood damage, etc. The input for this step is the data stored in cloud storage, and the output is the analysis results.

[0948] Step 5:

[0949] The server generates a 3D map of the affected area based on the analysis results obtained from the generative AI model. The generated 3D map is then visualized and displayed by the server on terminals at the emergency response headquarters and logistics center. This 3D map allows the scale and location of the damage to be visually confirmed. The input to this step is the analysis results, and the output is a 3D map.

[0950] Step 6:

[0951] Users (refugee residents and relief workers) use a dedicated smartphone app or chatbot to send information about the local situation and requests for relief supplies to a server. The server then sends the received information to a generation AI, which performs natural language analysis to extract important information. The input for this step is information from the local area, and the output is the extracted important information.

[0952] Step 7:

[0953] The server updates the database with the required relief supplies based on the extracted information. At the same time, it determines appropriate instructions for relief operations and delivery routes for relief supplies. The determined instructions and route information are sent in real time to the logistics center and relief workers on the ground. The input of this step is the extracted important information, and the output is relief operation instructions and delivery route information.

[0954] This allows specific input data to be processed and calculated at each step to generate output, ultimately enabling fast and accurate relief operations and delivery of relief supplies.

[0955] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0956] The disaster response system of the present invention includes various hardware and software components that enable rapid and efficient assessment of the damage situation and appropriate rescue operations when a disaster occurs. The main components of this system include a disaster detection means, an unmanned aerial vehicle (drone), a server, a generative AI, a user interface, and an emotion engine.

[0957] 1. Disaster detection methods

[0958] The server collects information from weather data, earthquake sensors, flood monitoring sensors, etc. in order to monitor the occurrence of natural disasters such as earthquakes, heavy rains, and floods in real time. If it determines that a disaster has occurred based on this information, the entire system will immediately begin operating.

[0959] 2. Launching and flying unmanned aerial vehicles (drones)

[0960] When the server detects a disaster, it sends instructions to launch multiple drones in a designated area, which follow automated flight routes and use their cameras to collect video and image data of the affected area.

[0961] 3. Data collection and transmission

[0962] The drone (terminal) transmits high-resolution video and image data acquired during flight to a server in real time.

[0963] 4. Data Analysis and 3D Map Generation

[0964] The server stores the video and image data sent from the drone in cloud storage and inputs it into the generation AI. The generation AI compares the data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. Based on the results of this analysis, it generates a 3D map of the affected area.

[0965] 5. Visualizing the damage situation and providing guidance on countermeasures

[0966] The server visualizes and displays the 3D map generated by the AI ​​on the terminals at the emergency response headquarters. This allows for a visual understanding of the damage situation and enables quick decision-making. Furthermore, it automatically generates countermeasure guidance based on the analysis results and distributes it to all parties involved.

[0967] 6. Collecting information from the field and analyzing emotions

[0968] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server.

[0969] The server receives messages sent by users and analyzes the emotions in the messages using an emotion engine. The emotion engine analyzes the text data and determines what emotions the user is feeling.

[0970] 7. Natural Language Analysis and Extraction of Important Information

[0971] The server uses generative AI to perform natural language analysis of messages and extract important information, along with the emotional data recognized by the emotion engine. The emotional data is then added to the analysis to identify high-priority requests.

[0972] 8. Directing relief operations and determining routes for transporting relief supplies

[0973] Based on the analyzed information, the server updates a database with the status of relief supplies needed at each evacuation center, and uses generative AI to determine relief operation instructions and optimal delivery routes for relief supplies. It also takes into account the results of sentiment analysis, prioritizing requests with a high level of urgency.

[0974] 9. Distribution of rescue operation instructions

[0975] The server transmits the determined rescue operation instructions and information on the delivery route of relief supplies to rescue workers on the scene in real time, thereby enabling rescue operations to be carried out quickly and efficiently.

[0976] 10. Specific Examples

[0977] For example, if a large-scale earthquake occurs, the server immediately launches multiple drones to fly over areas with severe damage. The drones collect video and image data, which the server inputs into the generation AI in real time. The generation AI then creates a three-dimensional map of the affected area, identifying collapsed buildings and disrupted roads. At the same time, users submit requests for relief supplies from evacuation centers, and based on this information, the optimal route for quickly delivering the necessary supplies is determined. Furthermore, an emotion engine performs emotion analysis, and requests deemed to be of a high level of urgency are prioritized. The server then distributes this information to relief workers, ensuring that appropriate relief efforts are carried out.

[0978] In this way, this system can collect information quickly and accurately when a disaster occurs, and support effective relief efforts that also take into account the user's emotions.

[0979] The processing flow will be explained below.

[0980] Step 1:

[0981] The server collects and monitors weather data, earthquake sensors, and flood monitoring sensors in real time to detect the occurrence of disasters. When a disaster is detected, it immediately issues instructions to start operating the entire system.

[0982] Step 2:

[0983] When the server detects a disaster, it sends activation instructions to multiple unmanned aerial vehicles (drones) located within a pre-defined area.

[0984] Step 3:

[0985] Once the drone (terminal) receives a start command from the server, it begins flying within the designated area according to an automatic flight route. The drone uses a camera to collect high-resolution video and image data.

[0986] Step 4:

[0987] The drone (terminal) transmits video and image data collected during flight to a server in real time.

[0988] Step 5:

[0989] The server stores the video and image data received from the drone in cloud storage.

[0990] Step 6:

[0991] The server inputs the video and image data stored in cloud storage into the generation AI, which analyzes the damage situation.

[0992] Step 7:

[0993] The generating AI compares data before and after the disaster, automatically detecting building damage, road disruption, and the extent of flooding, and generates a three-dimensional map of the affected area.

[0994] Step 8:

[0995] The server visualizes the generated three-dimensional map and analysis results and displays them on a terminal at the emergency response headquarters.

[0996] Step 9:

[0997] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server.

[0998] Step 10:

[0999] The server receives messages sent by users.

[1000] Step 11:

[1001] The server inputs the received message into the emotion engine for emotion analysis. The emotion engine analyzes the text data and determines the emotion the user is feeling.

[1002] Step 12:

[1003] The server uses generative AI to perform natural language analysis of messages and extract important information, along with the emotional data recognized by the emotion engine. The emotional data is then added to the analysis to identify high-priority requests.

[1004] Step 13:

[1005] Based on the analyzed information, the server updates a database with the status of relief supplies needed at each evacuation center, and uses generative AI to determine relief operation instructions and optimal delivery routes for relief supplies. It also takes into account the results of sentiment analysis, prioritizing requests with a high level of urgency.

[1006] Step 14:

[1007] The server transmits the determined rescue operation instructions and information on the route for transporting relief supplies to rescue workers on the scene in real time.

[1008] Step 15:

[1009] The relief workers (users) carry out their activities on site in accordance with the received relief operation instructions and the delivery route of relief supplies.

[1010] This system will enable the rapid and accurate collection of information in the event of a disaster, and will support effective relief efforts that take into account the user's emotions.

[1011] Example 2

[1012] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1013] When a natural disaster occurs, the challenge is to quickly grasp the extent of damage at the scene and implement effective relief efforts. With conventional systems, it takes time to collect information after a disaster occurs, making it difficult to accurately grasp the extent of the damage. It is also difficult to respond appropriately, taking into account the urgency of the victims.

[1014] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1015] In this invention, the server includes means for detecting the occurrence of a disaster, means for launching multiple unmanned aerial vehicles and collecting video and image data of the affected area, means for analyzing the collected video and image data and generating a three-dimensional map of the damage situation, means for visualizing the generated three-dimensional map and distributing countermeasure guidance, means for receiving information from residents of evacuation centers and relief workers and extracting important information through natural language analysis including emotional data, and means for determining rescue operation instructions and delivery routes for relief supplies based on the extracted information. This enables rapid and accurate information collection when a disaster occurs, and enables effective rescue operations that take into account the emotions and urgency of the victims.

[1016] "Means for detecting the occurrence of disasters" refers to a system that collects information from data sources such as seismometers, meteorological data, and flood monitoring sensors, and detects the occurrence of natural disasters in real time.

[1017] An "unmanned aerial vehicle" is a device that flies automatically or remotely without a human on board and collects video and image data of the affected area.

[1018] "Means for collecting video and image data of the affected area" refers to a mechanism for collecting video and image data of the affected area using a camera mounted on an unmanned aerial vehicle.

[1019] "Means for analyzing collected video and image data and generating a three-dimensional map of the damage situation" refers to a system that uses image analysis technology to reconstruct the damage situation as a three-dimensional model based on the acquired data.

[1020] "Means for visualizing the generated 3D map and distributing countermeasure guidance" refers to a function that displays a 3D map on a terminal at the countermeasure headquarters, generates appropriate action guidelines according to the situation, and distributes them to relevant parties.

[1021] "Means for receiving information from evacuation center residents and relief workers and extracting important information through natural language analysis, including emotional data" refers to a system that receives messages sent from evacuation centers and on-site locations, analyzes them using natural language processing technology, including emotional data, and identifies information with a high level of urgency.

[1022] The "means for determining rescue operation instructions and transportation routes for relief supplies" is a system that, based on analyzed information, issues action instructions for efficiently providing necessary support and determines the optimal transportation route for relief supplies.

[1023] The disaster response system of the present invention is designed to quickly and efficiently assess the damage situation and carry out appropriate relief activities when a disaster occurs. The system includes various hardware and software components.

[1024] Hardware and Software Configuration

[1025] 1. Server:

[1026] The server collects information from weather data APIs, earthquake sensors, flood monitoring sensors, etc., and monitors disaster occurrences in real time. When it determines that a disaster has occurred, it starts operating the entire system.

[1027] 2. Unmanned Aerial Vehicles (Drones):

[1028] When a disaster occurs, the server sends instructions to launch multiple drones in a designated area. The drones follow autonomous flight routes and use their onboard cameras to collect high-resolution video and image data of the affected area.

[1029] 3. Data Collection and Transmission:

[1030] The drone (terminal) transmits the video and image data it acquires during flight to a server in real time using Wi-Fi or mobile networks.

[1031] 4. Data analysis and three-dimensional map generation:

[1032] The server stores the received data in cloud storage and inputs it into a generative AI model. The generative AI model compares data from before and after the disaster, analyzes building damage, road disruption, and the extent of flood damage, and generates a 3D map of the affected area. For example, it uses image recognition technology to identify damage and creates a 3D model based on this.

[1033] 5. Visualization of damage situation and guidance on countermeasures:

[1034] The server visualizes and displays the generated 3D map on the headquarters' terminal. This allows the headquarters to quickly grasp the damage situation and make decisions quickly. Furthermore, the generative AI model automatically generates countermeasure guidance based on the analysis results and distributes this to all parties involved. Specifically, the visualization tool is used to draw the 3D map, and the report generation function is used to create the guidance content.

[1035] 6. Collecting information from the field and analyzing sentiment:

[1036] Users (refugee residents and relief workers) request information about the local situation and necessary relief supplies through a dedicated app or chatbot. This request is sent to a server. The server passes the received message to an emotion engine, which analyzes the emotion in the message. For example, it can automatically determine the emotion of the message using a text analysis API.

[1037] 7. Natural Language Analysis and Key Information Extraction:

[1038] The server passes the emotion data recognized by the emotion engine to a generative AI model, which then performs natural language analysis of the message to extract important information. The emotion data is used as an indicator to identify high-priority requests. Specifically, the message is analyzed using a natural language processing API.

[1039] 8. Directing relief operations and determining routes for transporting relief supplies:

[1040] The server updates the database with the status of relief supplies needed at each evacuation center based on the analysis results. The generative AI model determines the optimal route for delivering relief supplies and generates instructions for relief operations. It also takes into account the results of sentiment analysis and prioritizes requests with high urgency. For example, it uses a route optimization algorithm to calculate efficient routes.

[1041] 9. Distribution of rescue operation instructions:

[1042] The server sends rescue operation instructions and delivery route information for relief supplies to rescue workers in real time, allowing rescue operations to proceed quickly and efficiently. Specifically, it sends instructions to relevant parties using a notification system.

[1043] Examples of concrete examples and prompts

[1044] Examples:

[1045] For example, if a large-scale earthquake occurs, the server immediately launches multiple drones to fly over areas with severe damage. The drones collect video and image data, which the server inputs into a generative AI model in real time. The generative AI model then generates a three-dimensional map of the affected area, identifying collapsed buildings and disrupted roads. At the same time, users submit requests for relief supplies from evacuation centers, and this information is used to determine the optimal route for quickly delivering the necessary supplies. An emotion engine performs sentiment analysis, prioritizing requests with a high level of urgency from users. The server then distributes this information to relief workers, supporting rapid and appropriate relief efforts.

[1046] Example prompt sentence:

[1047] "In areas recently affected by disasters, please analyze the received image data and generate a 3D map of the damage. Also, please consider the relief supply requests sent from evacuation centers, determine the optimal delivery route, and prioritize requests with high priority."

[1048] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1049] Step 1:

[1050] Disaster detection

[1051] The server collects information from multiple data sources, such as seismometers, weather data APIs, and flood monitoring sensors, and monitors the occurrence of natural disasters in real time. The input is information obtained from each sensor and API, and based on this, it performs data calculations to compare it with a disaster occurrence threshold. The output is a signal that sets a "disaster occurrence detection" flag.

[1052] Specific operation:

[1053] 1. Get current weather information from the weather data API.

[1054] 2. Obtain real-time earthquake data from seismometers.

[1055] 3. Compare the acquired data with a threshold to determine whether a disaster has occurred.

[1056] 4. When a disaster is detected, the entire system begins operation.

[1057] Step 2:

[1058] Launching and flying the drone

[1059] When the server detects a disaster, it sends an instruction to launch multiple drones in a designated area. The input is a disaster detection signal and designated area information, and the output is a drone launch command. The drones follow an autonomous flight route and collect video and image data of the affected area.

[1060] Specific operation:

[1061] 1. Automatically launch multiple drones using the drone control API.

[1062] 2. Send the flight route and coordinate information of the affected area to the activated drone.

[1063] 3. The drone begins flying along the specified route.

[1064] Step 3:

[1065] Data collection and transmission

[1066] The drone (terminal) captures high-resolution video and image data using an onboard camera during flight and transmits this data to a server in real time. The input is the video and image data captured by the drone's camera, and the output is the raw data transmitted to the server.

[1067] Specific operation:

[1068] 1. Use the drone camera to capture video and images of the affected area.

[1069] 2. The acquired data is sent to the server in real time.

[1070] Step 4:

[1071] Data analysis and 3D map generation

[1072] The server stores the received video and image data in cloud storage and inputs it into a generative AI model. The generative AI model compares data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. The input is the data stored in cloud storage and data before and after the disaster, and the output is a 3D map of the affected area.

[1073] Specific operation:

[1074] 1. Save the received data in cloud storage.

[1075] 2. Input data into the generative AI model and analyze the damage situation.

[1076] 3. Generate a three-dimensional map of the affected area based on the analysis results.

[1077] Step 5:

[1078] Visualization of damage status and delivery of countermeasure guidance

[1079] The server visualizes and displays the generated 3D map on the terminal of the emergency response headquarters. The input is the generated 3D map, and the output is the map displayed on the terminal of the emergency response headquarters and countermeasure guidance. Countermeasure guidance is also automatically generated based on the analysis results of the generative AI model and distributed to relevant parties.

[1080] Specific operation:

[1081] 1. Use visualization tools to display 3D maps on the headquarters terminals.

[1082] 2. Create countermeasure guidance based on the analysis results of the generative AI model.

[1083] 3. Distribute the created guidance to each stakeholder.

[1084] Step 6:

[1085] Collecting information from the field and analyzing emotions

[1086] Users (refugee residents and relief workers) use a dedicated app or chatbot to request information about the local situation and necessary relief supplies. The server passes these messages to the emotion engine, which analyzes the emotions in the messages. The input is the message from the user, and the output is the analyzed emotion data.

[1087] Specific operation:

[1088] 1. Users submit information about the local situation and requests for relief supplies through a dedicated app or chatbot.

[1089] 2. The server receives the message and analyzes the emotion using the emotion engine.

[1090] Step 7:

[1091] Natural language analysis and extraction of important information

[1092] The server passes the emotion data recognized by the emotion engine to the generative AI model, which then performs natural language analysis of the message to extract important information. The input is the emotion data and the message, and the output is the extracted important information.

[1093] Specific operation:

[1094] 1. Analyze key messages based on sentiment data.

[1095] 2. Input emotion data into a generative AI model to extract key information.

[1096] Step 8:

[1097] Directing relief operations and determining routes for transporting relief supplies

[1098] Based on the analyzed information, the server updates the database with the status of relief supplies needed at each evacuation shelter. The generative AI model determines the optimal delivery route for relief supplies and generates rescue operation instructions. The input is the extracted important information and the current status of relief supplies, and the output is the confirmed delivery route and rescue operation instructions.

[1099] Specific operation:

[1100] 1. Based on the analysis results, the status of relief supplies is updated in the database.

[1101] 2. Determine the optimal delivery route using a generative AI model.

[1102] 3. Generate rescue operation instructions.

[1103] Step 9:

[1104] Distribution of rescue operation instructions

[1105] The server transmits rescue operation instructions and relief supply delivery route information to rescue workers in the field in real time. The input is the confirmed delivery route and rescue operation instructions, and the output is the information to be distributed to rescue workers in the field.

[1106] Specific operation:

[1107] 1. Send confirmed transport routes and rescue instructions to rescue workers on the scene.

[1108] 2. Deliver quickly with a real-time notification system.

[1109] (Application example 2)

[1110] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1111] When a disaster occurs, it is necessary to quickly and accurately grasp the damage situation and carry out efficient relief operations, but conventional systems have had problems with delays in information gathering and overlooking important information, making it difficult to respond quickly. Another issue was the inability to provide appropriate instructions for relief operations that take into account the user's emotions.

[1112] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for detecting the occurrence of a disaster, means for launching multiple unmanned aerial vehicles and collecting video and image data of the affected area, means for analyzing the collected video and image data and generating a three-dimensional map of the damage situation, means for visualizing the generated three-dimensional map and distributing countermeasure guidance, means for receiving information from residents of evacuation centers and relief workers and extracting important information through natural language analysis, means for analyzing the sentiment of users' messages and identifying high-priority requests, means for determining rescue operation instructions and delivery routes for relief supplies based on the extracted information, and means for transmitting rescue operation instructions to relief workers on the scene in real time. This enables a quick and accurate understanding of the damage situation and appropriate rescue operation instructions that take the user's sentiment into consideration.

[1113] "Means for detecting the occurrence of disasters" refers to sensors and data analysis systems that allow a server to monitor natural disasters such as earthquakes, heavy rain, and floods in real time and determine whether a disaster has occurred.

[1114] An "unmanned aerial vehicle" is a drone or other unmanned aircraft equipped with a camera and following an automated flight route.

[1115] The "means for collecting video and image data of the affected area" is a system that allows unmanned aerial vehicles to capture video and image data of the affected area and send it to a server.

[1116] The "means for generating a three-dimensional map" is a system that analyzes collected video and image data and visually displays the damage situation as a three-dimensional map.

[1117] The "means of delivering guidance on countermeasures" is a system that delivers information and guidelines for action to relevant organizations and personnel based on the generated three-dimensional map to support rapid decision-making.

[1118] The "means for extracting key information through natural language analysis" refers to a generative AI model and natural language processing system for analyzing messages received from evacuation center residents and relief workers and extracting key information from them.

[1119] The "means for analyzing emotions and identifying high-priority requests" is a system that analyzes the emotions in messages from users and determines the priority of requests according to their urgency and importance.

[1120] The "means for determining rescue operation instructions and transportation routes for relief supplies" is a system that determines optimal rescue operation instructions and transportation routes for relief supplies based on the extracted important information and emotion analysis results.

[1121] The "means for transmitting rescue operation instructions to rescue workers on the scene in real time" is a communication system for quickly transmitting the determined rescue operation instructions and information on the transportation route of relief supplies to rescue workers on the scene.

[1122] The disaster response system of the present invention includes various hardware and software components that enable rapid and efficient assessment of the damage situation and appropriate rescue operations when a disaster occurs. The main components of this system include a disaster detection means, an unmanned aerial vehicle (drone), a server, a generative AI, a user interface, and an emotion engine.

[1123] The server collects and analyzes information from weather data, earthquake sensors, flood monitoring sensors, etc. in order to monitor the occurrence of natural disasters such as earthquakes, heavy rain, and floods in real time. Based on this information, if it determines that a disaster has occurred, the entire system immediately begins operation. When the server detects a disaster, it sends instructions to launch multiple unmanned aerial vehicles (drones) in a specified area. The drones follow automatic flight routes and use cameras to collect video and image data of the affected area, which they then send to the server in real time.

[1124] The server stores the high-resolution video and image data sent from the drone in cloud storage (e.g., Amazon S3) and inputs it into a generation AI (e.g., OpenAI GPT model). The generation AI compares data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. Based on the analysis results, it generates a 3D map of the affected area, which is visualized and displayed on a terminal at the response headquarters. Furthermore, it automatically generates countermeasure guidance based on the analysis results and distributes it to all parties involved.

[1125] Users (refugee residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to a server. The server receives the messages sent by the users and analyzes the emotions in the messages using an emotion engine (e.g., IBM Watson). The emotion engine analyzes the text data and determines the user's emotions. The server then uses generative AI to perform natural language analysis of the messages along with the emotion data recognized by the emotion engine, and extracts important information. The emotion data is added to the analysis and used as an indicator to identify high-priority requests.

[1126] Based on the analyzed information, the server determines optimal rescue operation instructions and delivery routes for relief supplies. This is done using a generative AI model to automatically generate efficient routes and operation instructions. The determined rescue operation instructions and delivery route information for relief supplies are sent to rescue workers on the ground in real time, allowing rescue operations to be carried out quickly and efficiently.

[1127] As a specific example, if a large-scale earthquake occurs, the server immediately launches multiple drones to fly over areas with severe damage. The drones collect video and image data, which the server inputs into the generation AI in real time. The generation AI creates a three-dimensional map of the affected area and identifies collapsed buildings and severed roads. At the same time, users submit requests for relief supplies from evacuation centers, and based on this information, the optimal route for quickly delivering the necessary supplies is determined. Furthermore, an emotion engine performs emotion analysis, and requests that are deemed to be of high urgency by the user are prioritized.

[1128] For example, if a user sends a message such as "A major earthquake has occurred and I am in a shelter. I am short of water and food," emotion analysis and natural language analysis can quickly identify that emergency assistance is needed in that area. Based on this information, the server can then deploy efficient relief activities.

[1129] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1130] Step 1:

[1131] The server monitors various sensors and meteorological data to detect the occurrence of disasters. The input is data obtained from earthquake sensors and weather information APIs, and the server analyzes this data to determine the occurrence of a natural disaster. The output is an internal flag indicating that a disaster has occurred. Based on this flag, the operation to proceed to the next step is triggered.

[1132] Step 2:

[1133] When the server detects that a disaster has occurred, it sends an instruction to launch multiple unmanned aerial vehicles (drones) in a specified area. The input is a disaster detection flag and coordinate data of the affected area, and the output is a drone launch signal. Specifically, the server calls the drone's launch API and sets an automatic flight route.

[1134] Step 3:

[1135] The unmanned aerial vehicle (drone) follows an automated flight route and uses a camera to collect video and image data of the affected area. The input is the drone's flight route data and camera setting data, and the output is high-resolution video and image data. Specifically, the drone operates by continuously taking photos and videos while flying a predetermined route.

[1136] Step 4:

[1137] The drone transmits the collected video and image data to a server in real time. The input is the video and image data captured by the drone's camera, and the output is a data stream to the server. Specifically, the drone transmits the data to the server using wireless communication.

[1138] Step 5:

[1139] The server stores the data sent from the drone in cloud storage (e.g., Amazon S3) and then inputs it into the generation AI. The input is the video and image data received from the drone, and the output is an analysis request to the generation AI. Specifically, the server uploads the data to cloud storage and passes the reference link to the generation AI.

[1140] Step 6:

[1141] The generative AI (for example, the OpenAI GPT model) compares data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. The input is a cloud storage link and analysis instructions, and the output is 3D map data of the affected area. Specifically, the generative AI analyzes the data and visualizes the damage situation in 3D.

[1142] Step 7:

[1143] The server visualizes and displays the 3D map generated by the generation AI on the terminal at the emergency headquarters. The input is 3D map data, and the output is visual information displayed on the screen at the emergency headquarters. Specifically, the server renders the 3D map and displays it on the monitor at the emergency headquarters.

[1144] Step 8:

[1145] Users (shelter residents and relief workers) use a dedicated app or chatbot to send local information and requests for relief supplies to a server. The input is text information entered by the user into the app or chatbot, and the output is the request data received by the server. The specific operation is for the user to request the necessary relief supplies through the app.

[1146] Step 9:

[1147] The server receives messages sent by users and performs emotion analysis using an emotion engine (e.g., IBM Watson). The input is the user's text message, and the output is the analyzed emotion data. Specifically, the server sends messages to the emotion engine and receives emotion data.

[1148] Step 10:

[1149] The server uses the generation AI along with the emotion data recognized by the emotion engine to perform natural language analysis of the message and extract important information. The input is the user's message and emotion data, and the output is an extracted list of important information. Specifically, the generation AI performs natural language analysis and determines the priority of the request.

[1150] Step 11:

[1151] The server determines optimal rescue operation instructions and delivery routes for relief supplies based on the analyzed information. The input is an extracted list of important information and emotion data, and the output is rescue operation instructions and delivery route determination data. Specifically, the server calculates the optimal route and operation instructions with the help of generation AI.

[1152] Step 12:

[1153] The server transmits the determined rescue operation instructions and information on the delivery route of relief supplies to the rescue workers on the scene in real time. The input is the rescue operation instructions and delivery route determination data, and the output is a notification of instructions to the rescue workers. Specifically, the server transmits information to the scene via the communication system.

[1154] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1155] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1156] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1157] [Fourth embodiment]

[1158] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1159] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1160] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1161] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1162] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1163] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1164] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1165] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1166] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1167] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1168] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1169] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1170] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1171] The disaster response system of the present invention includes various hardware and software components that enable rapid and efficient assessment of the damage situation and appropriate rescue operations when a disaster occurs. The main components of this system include a disaster detection means, an unmanned aerial vehicle (drone), a server, a generation AI, and a user interface.

[1172] 1. Disaster detection methods

[1173] The server collects information from weather data, earthquake sensors, flood monitoring sensors, etc. in order to monitor the occurrence of natural disasters such as earthquakes, heavy rains, and floods in real time. If it determines that a disaster has occurred based on this information, the entire system will immediately begin operating.

[1174] 2. Launching and flying unmanned aerial vehicles (drones)

[1175] When the server detects a disaster, it sends instructions to launch multiple drones in a designated area, which follow automated flight routes and use their cameras to collect video and image data of the affected area.

[1176] 3. Data collection and transmission

[1177] The drone (terminal) transmits high-resolution video and image data acquired during flight to a server in real time, enabling a rapid assessment of the damage situation.

[1178] 4. Data Analysis and 3D Map Generation

[1179] The server stores the video and image data sent from the drone in cloud storage and inputs it into the generation AI. The generation AI compares the data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. Based on the results of this analysis, it generates a 3D map of the affected area.

[1180] 5. Visualizing the damage situation and providing guidance on countermeasures

[1181] The server visualizes and displays the 3D map generated by the AI ​​on the terminals at the emergency response headquarters. This allows for a visual understanding of the damage situation and enables quick decision-making. Furthermore, it automatically generates countermeasure guidance based on the analysis results and distributes it to all parties involved.

[1182] 6. Collecting and analyzing information from the field

[1183] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server. The server receives the information, and the generative AI performs natural language analysis to extract important information.

[1184] 7. Directing relief operations and determining routes for transporting relief supplies

[1185] Based on the extracted information, the server updates the database with the status of relief supplies needed at each evacuation shelter, determines instructions for relief operations and routes for delivering relief supplies, and transmits the instructions and route information to relief workers on the ground in real time.

[1186] 8. Specific Examples

[1187] For example, if a large-scale earthquake occurs, the server immediately launches multiple drones to fly over areas with severe damage. The drones collect video and image data, which the server inputs into the generation AI in real time. The generation AI then creates a three-dimensional map of the affected area, identifying collapsed buildings and disrupted roads. At the same time, users submit requests for relief supplies from evacuation centers, and this information is used to determine the optimal route for quickly delivering the necessary supplies. The server then distributes this information to relief workers, ensuring that appropriate relief efforts are carried out.

[1188] In this way, this system can collect information quickly and accurately when a disaster occurs, and support effective relief efforts.

[1189] The processing flow will be explained below.

[1190] Step 1:

[1191] The server collects and monitors weather data, earthquake sensors, and flood monitoring sensors in real time to detect the occurrence of disasters. When a disaster is detected, it immediately issues instructions to start operating the entire system.

[1192] Step 2:

[1193] When the server detects a disaster, it sends activation instructions to multiple unmanned aerial vehicles (drones) located within a pre-defined area.

[1194] Step 3:

[1195] Once the drone (terminal) receives a start command from the server, it begins flying within the designated area according to an automatic flight route. The drone uses a camera to collect high-resolution video and image data.

[1196] Step 4:

[1197] The drone (terminal) transmits video and image data collected during flight to a server in real time.

[1198] Step 5:

[1199] The server stores the video and image data received from the drone in cloud storage.

[1200] Step 6:

[1201] The server inputs the video and image data stored in cloud storage into the generation AI, which analyzes the damage situation.

[1202] Step 7:

[1203] The generating AI compares data before and after the disaster, automatically detecting building damage, road disruption, and the extent of flooding, and generates a three-dimensional map of the affected area.

[1204] Step 8:

[1205] The server visualizes the generated three-dimensional map and analysis results and displays them on a terminal at the emergency response headquarters.

[1206] Step 9:

[1207] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server.

[1208] Step 10:

[1209] The server receives messages sent by users and uses generative AI to perform natural language analysis and extract important information.

[1210] Step 11:

[1211] Based on the analyzed information, the server updates the database with the status of relief supplies at each evacuation shelter.

[1212] Step 12:

[1213] The server uses generative AI based on the information in the database to determine rescue operation instructions and the optimal route for transporting relief supplies.

[1214] Step 13:

[1215] The server transmits the determined rescue operation instructions and information on the route for transporting relief supplies to rescue workers on the scene in real time.

[1216] Step 14:

[1217] The relief workers (users) carry out their activities on site in accordance with the received relief operation instructions and the delivery route of relief supplies.

[1218] This system will enable rapid and accurate collection of information in the event of a disaster, enabling appropriate and effective relief efforts to be carried out.

[1219] Example 1

[1220] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1221] The problem is that when a disaster occurs, there is a lack of means to collect and analyze information to quickly and accurately grasp the damage situation and carry out effective relief activities. With conventional methods, it takes time to accurately grasp the damage situation on the ground, and the accuracy of the information is limited, making it difficult to formulate appropriate relief operation plans.

[1222] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1223] In this invention, the server includes means for detecting the occurrence of a disaster, means for launching multiple unmanned aerial vehicles and collecting video and image data of the affected area, means for analyzing the collected video and image data and generating a three-dimensional map of the damage situation, means for visualizing the generated three-dimensional map and distributing countermeasure instructions, means for receiving information from evacuation shelter residents and relief workers and extracting important information through natural language analysis, means for determining rescue operation instructions and delivery routes for relief supplies based on the extracted information, means for automatically analyzing the damage situation and providing countermeasure instructions using a generative AI model, means for immediately recognizing the occurrence of a disaster and emergency activation of the system, and means for saving data captured by the unmanned aerial vehicle's camera in real time to cloud storage. This enables rapid and accurate information collection and analysis, supporting effective rescue operations.

[1224] "Means for detecting the occurrence of disasters" refers to devices or systems that monitor the occurrence of natural disasters such as earthquakes, heavy rains, and floods in real time, collect and analyze data, and determine whether a disaster has occurred.

[1225] "Multiple unmanned aerial vehicles" means an aircraft system that includes drones that are flown by remote control or autonomous pilot and used to monitor a specific area.

[1226] "Video and image data of the affected areas" refers to video and photographic data taken by unmanned aerial vehicles to show the current state of the areas affected by the disaster.

[1227] The "means of collection" is a communication device or protocol for transferring video and image data captured by the unmanned aerial vehicle to a specific server or cloud storage.

[1228] "Means for generating a three-dimensional map" refers to software or algorithms that analyze collected video and image data and recreate the extent and details of the damage in three-dimensional space.

[1229] The "means for visualization and delivery of countermeasure guidance" is a system that displays the generated three-dimensional map on a terminal display and notifies relevant parties of rescue operation instructions and countermeasures.

[1230] "Means for extracting important information through natural language analysis" refers to algorithms or software that analyze text data sent by evacuation shelter residents and relief workers and automatically extract important information necessary for providing assistance.

[1231] The "means for determining rescue operation instructions and transportation routes for relief supplies" is a system that uses the extracted information to create a list of relief supplies needed at each evacuation shelter and makes a plan for transporting them via the optimal route.

[1232] "Means for automatically analyzing damage situations and providing guidance on countermeasures using generative AI models" refers to machine learning models or AI systems that analyze damage situations based on collected data and automatically propose optimal countermeasures.

[1233] "Means for immediately recognizing the occurrence of a disaster and for emergency activation of the system" refers to a combination of hardware and software that immediately activates the entire system and initiates necessary measures when a disaster is detected.

[1234] "Data captured by the camera of an unmanned aerial vehicle" refers to visual information collected through a camera mounted on an unmanned aerial vehicle, and is a means of storing this information in cloud storage in real time.

[1235] The present invention is directed to a system that aims to quickly and accurately grasp the damage situation when a disaster occurs and to support effective relief efforts. The present invention is configured by combining various hardware and software components.

[1236] Disaster detection

[1237] To detect the occurrence of a disaster, the server collects and analyzes real-time data from weather data, earthquake sensors, flood monitoring sensors, etc. This is done using, for example, the Japan Meteorological Agency's API or specific sensor devices. When the server detects data that exceeds a certain threshold, it recognizes the occurrence of a disaster and issues a command to immediately start up the operation of the entire system. Specifically, it uses software that immediately recognizes the occurrence of a disaster and immediately starts up the system.

[1238] Launching and flying unmanned aerial vehicles

[1239] The server launches multiple unmanned aerial vehicles (drones) via drone control software in areas where a disaster has been confirmed. These drones follow automated flight routes and use cameras to collect video and image data of the affected area. The drones are equipped with high-resolution camera modules and fly autonomously along pre-programmed flight paths. Flight control is performed using the DroneKit API, for example.

[1240] Data collection and transmission

[1241] The drone (terminal) transmits high-resolution video and image data acquired during flight to a server in real time over a 5G network, and the data is stored in a cloud storage service, specifically Amazon S3.

[1242] Data analysis and 3D map generation

[1243] The server inputs the video and image data stored in cloud storage into a generative AI model. This generative AI model was developed using TensorFlow and other tools, and analyzes the extent and details of the damage. Based on the analysis results, a three-dimensional map of the affected area is generated. This map is saved for further detailed analysis and is linked to the emergency response headquarters terminal.

[1244] Visualization of damage status and delivery of countermeasure guidance

[1245] The server then displays the generated 3D map on the device's display, using a visualization tool such as Tableau. The automated guidance generated by the AI ​​model is then distributed to relevant parties via the SendGrid API.

[1246] Collecting and analyzing information from the field

[1247] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server. This application is a mobile app using Firebase. The server receives this information and performs natural language analysis using a generative AI model, which automatically extracts important information.

[1248] Directing relief operations and determining routes for transporting relief supplies

[1249] The server updates a database with the status of relief supplies needed at each evacuation center based on the information extracted through natural language analysis. It then uses the Google Maps API to calculate optimal delivery routes and generate rescue operation instructions. This information is then sent in real time to rescue workers on the ground using Firebase Cloud Messaging.

[1250] Specific examples

[1251] For example, if a large-scale earthquake occurs, the server immediately recognizes the occurrence of a disaster based on data obtained from the Japan Meteorological Agency API and launches multiple drones using the DroneKit API. The drones store the acquired data in Amazon S3 via the 5G network, and this data is input into a generative AI model using TensorFlow. The analysis results are displayed in Tableau and distributed to relevant parties via the SendGrid API. Requests from evacuation centers are received through the Firebase app, and the information is analyzed using a natural language analysis model. The optimal route for delivering relief supplies is calculated using the Google Maps API and sent to relief workers via Firebase Cloud Messaging.

[1252] Prompt Sentence Examples

[1253] "Please tell me the detailed operation procedure of the system that quickly grasps the damage situation in an area where an earthquake has occurred and uses a generative AI model to plan rescue operations."

[1254] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1255] Step 1: Detecting the occurrence of a disaster

[1256] The server collects data in real time from sources such as weather data, earthquake sensors, and flood monitoring sensors. Specifically, it takes in data obtained from APIs and sensor devices every minute. Statistical analysis is performed based on this data, and if a certain threshold is exceeded, a disaster is recognized. Various disaster data is input, and a disaster occurrence signal is generated as output.

[1257] Step 2: Launch command for unmanned aerial vehicles

[1258] When the server confirms that a disaster has occurred, it sends instructions to launch multiple unmanned aerial vehicles (drones). This instruction is sent via drone control software (e.g., DroneKit API). The input is a disaster occurrence signal, and the output is a command to launch the drones. Specific operations include setting an automatic flight route based on GPS coordinates and activating cameras.

[1259] Step 3: Video and image data collection

[1260] The drone (terminal) collects high-resolution video and image data while flying along an automatic flight route. It uses a camera module (e.g., DJI Zenmuse X7) to capture images along the specified route. The input is the set flight route information, and the output is high-resolution video and image data.

[1261] Step 4: Sending data in real time

[1262] The drone transmits the collected video and image data to a server in real time. The communication method is a 5G network. The input is high-resolution video and image data, and the output is the data sent to the server. Specifically, the data is packetized and transmitted.

[1263] Step 5: Save your data to the cloud

[1264] The server stores the received video and image data in a cloud storage service (e.g., Amazon S3). The input is data sent in real time, and the output is data stored in the cloud storage. Specifically, the data is stream-processed and converted into files.

[1265] Step 6: Generate a 3D map

[1266] The server inputs the data stored in cloud storage into a generative AI model and analyzes the damage situation. Using the generative AI model (e.g., TensorFlow), it compares data before and after the disaster and generates a 3D map. The input is the stored video and image data, and the output is a 3D map. Specific operations include preprocessing the data, inputting it into the trained model, and outputting the analysis results.

[1267] Step 7: Visualize the damage situation

[1268] The server displays the generated 3D map on the emergency headquarters terminal, allowing the damage situation to be visually grasped. The input is the generated 3D map data, and the output is the map displayed on the terminal. Specifically, the 3D map data is rendered and displayed.

[1269] Step 8: Automatic generation and distribution of countermeasure guidance

[1270] The server uses a generative AI model to automatically generate countermeasure guidance based on the analysis results and distributes it to relevant parties. The input is the analysis results and a 3D map, and the output is the automatically generated countermeasure guidance and distributed guidance message. Specific operations involve the use of an email distribution system (e.g., SendGrid API).

[1271] Step 9: Gather information from the field

[1272] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for relief supplies to the server. The input is text, images, and audio data, and the output is the information sent to the server. Specific operations involve the use of a mobile app (e.g., Firebase app).

[1273] Step 10: Extracting important information through natural language analysis

[1274] The server uses a generative AI model to perform natural language analysis on the received local information and extract important information. The input is the transmitted text, images, and audio data, and the output is the extracted important information. The specific operation involves applying a natural language processing algorithm.

[1275] Step 11: Deciding on relief operation instructions and relief supply transport routes

[1276] The server updates the database based on the extracted information and determines rescue operation instructions and delivery routes for relief supplies. The input is important information and the current supply situation, and the output is optimized delivery routes and rescue instructions. Specifically, a route calculation algorithm (e.g., Google Maps API) is applied.

[1277] Step 12: Send instructions to relief workers

[1278] The server sends the determined instructions and transport route information to the rescue workers on the scene via push notification. The input is the rescue instructions and transport route, and the output is the information sent to the rescue workers' devices. The specific operation is to send a push notification (e.g., Firebase Cloud Messaging).

[1279] (Application example 1)

[1280] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1281] When a disaster occurs, it is important to quickly and accurately grasp the damage situation. To quickly and appropriately carry out relief efforts, it is necessary to collect and analyze information from various evacuation centers and relief activity sites. However, the current system does not efficiently coordinate the understanding of the damage situation and instructions for relief efforts, making it particularly difficult to optimize the transportation routes of supplies. Furthermore, logistics centers do not have the appropriate means to rapidly distribute relief supplies.

[1282] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1283] In this invention, the server includes means for detecting the occurrence of a disaster, means for launching multiple unmanned aerial vehicles to collect video and image data of the affected area, means for analyzing the collected video and image data and generating a 3D map of the damage situation, and means for displaying the generated 3D map on a terminal at the logistics center, and for quickly managing requests for relief supplies and optimizing delivery routes. This enables a detailed understanding of the damage situation in the event of a disaster and efficient distribution of relief supplies via the logistics center.

[1284] "Means for detecting the occurrence of disasters" refers to a system that monitors and notifies in real time the occurrence of natural disasters such as earthquakes, heavy rain, and floods.

[1285] An "unmanned aerial vehicle" is a device, such as a drone, that flies remotely or autonomously and collects video and image data of a specific area.

[1286] "Video and image data" refers to video and photographic data used to visually record the situation in the affected areas.

[1287] A "three-dimensional map" is a map that displays the state of the terrain and buildings before and after a disaster in three dimensions.

[1288] A "generative AI model" is an artificial intelligence system that analyzes collected data and generates new data and information.

[1289] The "means for delivering guidance on countermeasures" is a system that automatically notifies each relevant party of appropriate countermeasures and instructions based on the analyzed data.

[1290] "Means for receiving information from evacuation center residents and relief workers" refers to a system for receiving information on the local situation and requests for relief supplies from evacuation centers and local relief workers.

[1291] "Natural language analysis" is a technology for analyzing received information and extracting important information.

[1292] A "relief supply transport route" is a route for efficiently delivering necessary relief supplies to evacuation centers and disaster-stricken areas.

[1293] A "logistics center" is a facility for distributing and managing relief supplies.

[1294] A "terminal" is a computer system for displaying and manipulating information.

[1295] "Relief supply request management" is the process of receiving requests for relief supplies from evacuation centers and disaster-stricken areas and managing the necessary supplies.

[1296] "Route optimization" is the process of planning and executing the safe delivery of relief supplies via the shortest and most optimal route.

[1297] This invention provides a system for quickly and accurately assessing the damage situation when a disaster occurs, and for efficiently carrying out appropriate relief activities and transporting relief supplies. The system includes the following main hardware and software components:

[1298] 1. Means of detecting disasters

[1299] The server monitors disasters such as earthquakes, heavy rains, and floods in real time, collecting information from weather data, earthquake sensors, flood monitoring sensors, etc. If it determines that a disaster has occurred, it immediately starts operating the entire system.

[1300] 2. Starting and flying the drone

[1301] When the server detects a disaster, it sends instructions to launch multiple unmanned aerial vehicles in a designated area. The drones follow automated flight routes and use their onboard cameras to collect video and image data of the affected area.

[1302] 3. Data collection and transmission

[1303] The drones capture high-resolution video and image data during flight and transmit it to a server in real time, enabling a rapid assessment of the damage situation.

[1304] 4. Data Analysis and 3D Map Generation

[1305] The server stores the video and image data sent from the drone in cloud storage (AWS S3) and inputs it into a generative AI model (OpenAI API). The generative AI compares data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. Based on the results of this analysis, it generates a 3D map of the affected area.

[1306] 5. Visualizing the damage situation and providing guidance on countermeasures

[1307] The server visualizes and displays the 3D map generated by the AI ​​on terminals at the logistics center and the emergency response headquarters. This allows for a visual understanding of the damage situation, enabling quick decision-making. Furthermore, guidance on countermeasures based on the analysis results is distributed to all parties in real time.

[1308] 6. Collecting and analyzing information from the field

[1309] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server. The server receives the information, and the generative AI performs natural language analysis to extract important information.

[1310] 7. Directing relief operations and determining routes for transporting relief supplies

[1311] Based on the extracted information, the server updates the database with the status of relief supplies needed at each evacuation shelter, determines optimal instructions for relief efforts, and determines the optimal route for delivering relief supplies. The determined instructions and route information are sent in real time to relief workers on the ground.

[1312] Specific examples

[1313] For example, if a large-scale earthquake occurs, the server immediately launches multiple drones to fly over areas with severe damage. The drones collect video and image data, which the server inputs into the generation AI in real time. The generation AI generates a three-dimensional map of the affected area and identifies collapsed buildings and disrupted roads. At the same time, users submit requests for relief supplies from evacuation centers, and this information is used to determine the optimal route for quickly delivering the necessary supplies. The server then distributes this information to relief workers, ensuring that appropriate relief efforts are carried out.

[1314] Prompt Sentence Examples

[1315] I would like to use the "Disaster Logistics Manager" app to carry out appropriate relief efforts in the event of a large-scale earthquake. Collect data from drones, analyze the damage situation with AI, create a 3D map, manage requests for relief supplies, and generate optimal delivery routes.

[1316] This system will enable rapid and accurate information gathering in the event of a disaster and efficient distribution of relief supplies from logistics centers.

[1317] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1318] Step 1:

[1319] The server detects the occurrence of a disaster in real time. The main input data is weather data, information from earthquake sensors, and flood monitoring sensors. The server aggregates this data and determines whether a disaster has occurred. The output of this step is the determination result of "disaster occurrence." If a disaster occurs, the system begins operation.

[1320] Step 2:

[1321] When the server detects a disaster, it immediately sends instructions to launch multiple unmanned aerial vehicles (drones). These instructions include the flight area and automatic flight route. The drones receive the instructions and fly autonomously within the designated disaster area. The input for this step is instructions from the server, and the output is the launch and flight of the drones.

[1322] Step 3:

[1323] The drone uses its on-board camera to collect high-resolution video and image data of the affected area. This data is sent to the server in real time. The input is the aerial video and image data, and the output is the data sent to the server. In this step, the drone performs the specific operation of sending the captured data to the server.

[1324] Step 4:

[1325] The server stores the video and image data sent from the drone in cloud storage (AWS S3). Next, this data is input into a generative AI model (OpenAI API) to begin analyzing the data. During the analysis process, data before and after the disaster is compared to identify building damage, road disruption, the extent of flood damage, etc. The input for this step is the data stored in cloud storage, and the output is the analysis results.

[1326] Step 5:

[1327] The server generates a 3D map of the affected area based on the analysis results obtained from the generative AI model. The generated 3D map is then visualized and displayed by the server on terminals at the emergency response headquarters and logistics center. This 3D map allows the scale and location of the damage to be visually confirmed. The input to this step is the analysis results, and the output is a 3D map.

[1328] Step 6:

[1329] Users (refugee residents and relief workers) use a dedicated smartphone app or chatbot to send information about the local situation and requests for relief supplies to a server. The server then sends the received information to a generation AI, which performs natural language analysis to extract important information. The input for this step is information from the local area, and the output is the extracted important information.

[1330] Step 7:

[1331] The server updates the database with the required relief supplies based on the extracted information. At the same time, it determines appropriate instructions for relief operations and delivery routes for relief supplies. The determined instructions and route information are sent in real time to the logistics center and relief workers on the ground. The input of this step is the extracted important information, and the output is relief operation instructions and delivery route information.

[1332] This allows specific input data to be processed and calculated at each step to generate output, ultimately enabling fast and accurate relief operations and delivery of relief supplies.

[1333] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1334] The disaster response system of the present invention includes various hardware and software components that enable rapid and efficient assessment of the damage situation and appropriate rescue operations when a disaster occurs. The main components of this system include a disaster detection means, an unmanned aerial vehicle (drone), a server, a generative AI, a user interface, and an emotion engine.

[1335] 1. Disaster detection methods

[1336] The server collects information from weather data, earthquake sensors, flood monitoring sensors, etc. in order to monitor the occurrence of natural disasters such as earthquakes, heavy rains, and floods in real time. If it determines that a disaster has occurred based on this information, the entire system will immediately begin operating.

[1337] 2. Launching and flying unmanned aerial vehicles (drones)

[1338] When the server detects a disaster, it sends instructions to launch multiple drones in a designated area, which follow automated flight routes and use their cameras to collect video and image data of the affected area.

[1339] 3. Data collection and transmission

[1340] The drone (terminal) transmits high-resolution video and image data acquired during flight to a server in real time.

[1341] 4. Data Analysis and 3D Map Generation

[1342] The server stores the video and image data sent from the drone in cloud storage and inputs it into the generation AI. The generation AI compares the data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. Based on the results of this analysis, it generates a 3D map of the affected area.

[1343] 5. Visualizing the damage situation and providing guidance on countermeasures

[1344] The server visualizes and displays the 3D map generated by the AI ​​on the terminals at the emergency response headquarters. This allows for a visual understanding of the damage situation and enables quick decision-making. Furthermore, it automatically generates countermeasure guidance based on the analysis results and distributes it to all parties involved.

[1345] 6. Collecting information from the field and analyzing emotions

[1346] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server.

[1347] The server receives messages sent by users and analyzes the emotions in the messages using an emotion engine. The emotion engine analyzes the text data and determines what emotions the user is feeling.

[1348] 7. Natural Language Analysis and Extraction of Important Information

[1349] The server uses generative AI to perform natural language analysis of messages and extract important information, along with the emotional data recognized by the emotion engine. The emotional data is then added to the analysis to identify high-priority requests.

[1350] 8. Directing relief operations and determining routes for transporting relief supplies

[1351] Based on the analyzed information, the server updates a database with the status of relief supplies needed at each evacuation center, and uses generative AI to determine relief operation instructions and optimal delivery routes for relief supplies. It also takes into account the results of sentiment analysis, prioritizing requests with a high level of urgency.

[1352] 9. Distribution of rescue operation instructions

[1353] The server transmits the determined rescue operation instructions and information on the delivery route of relief supplies to rescue workers on the scene in real time, thereby enabling rescue operations to be carried out quickly and efficiently.

[1354] 10. Specific Examples

[1355] For example, if a large-scale earthquake occurs, the server immediately launches multiple drones to fly over areas with severe damage. The drones collect video and image data, which the server inputs into the generation AI in real time. The generation AI then creates a three-dimensional map of the affected area, identifying collapsed buildings and disrupted roads. At the same time, users submit requests for relief supplies from evacuation centers, and based on this information, the optimal route for quickly delivering the necessary supplies is determined. Furthermore, an emotion engine performs emotion analysis, and requests deemed to be of a high level of urgency are prioritized. The server then distributes this information to relief workers, ensuring that appropriate relief efforts are carried out.

[1356] In this way, this system can collect information quickly and accurately when a disaster occurs, and support effective relief efforts that also take into account the user's emotions.

[1357] The processing flow will be explained below.

[1358] Step 1:

[1359] The server collects and monitors weather data, earthquake sensors, and flood monitoring sensors in real time to detect the occurrence of disasters. When a disaster is detected, it immediately issues instructions to start operating the entire system.

[1360] Step 2:

[1361] When the server detects a disaster, it sends activation instructions to multiple unmanned aerial vehicles (drones) located within a pre-defined area.

[1362] Step 3:

[1363] Once the drone (terminal) receives a start command from the server, it begins flying within the designated area according to an automatic flight route. The drone uses a camera to collect high-resolution video and image data.

[1364] Step 4:

[1365] The drone (terminal) transmits video and image data collected during flight to a server in real time.

[1366] Step 5:

[1367] The server stores the video and image data received from the drone in cloud storage.

[1368] Step 6:

[1369] The server inputs the video and image data stored in cloud storage into the generation AI, which analyzes the damage situation.

[1370] Step 7:

[1371] The generating AI compares data before and after the disaster, automatically detecting building damage, road disruption, and the extent of flooding, and generates a three-dimensional map of the affected area.

[1372] Step 8:

[1373] The server visualizes the generated three-dimensional map and analysis results and displays them on a terminal at the emergency response headquarters.

[1374] Step 9:

[1375] Users (shelter residents and relief workers) use a dedicated app or chatbot to send information about the local situation and requests for needed relief supplies to the server.

[1376] Step 10:

[1377] The server receives messages sent by users.

[1378] Step 11:

[1379] The server inputs the received message into the emotion engine for emotion analysis. The emotion engine analyzes the text data and determines the emotion the user is feeling.

[1380] Step 12:

[1381] The server uses generative AI to perform natural language analysis of messages and extract important information, along with the emotional data recognized by the emotion engine. The emotional data is then added to the analysis to identify high-priority requests.

[1382] Step 13:

[1383] Based on the analyzed information, the server updates a database with the status of relief supplies needed at each evacuation center, and uses generative AI to determine relief operation instructions and optimal delivery routes for relief supplies. It also takes into account the results of sentiment analysis, prioritizing requests with a high level of urgency.

[1384] Step 14:

[1385] The server transmits the determined rescue operation instructions and information on the route for transporting relief supplies to rescue workers on the scene in real time.

[1386] Step 15:

[1387] The relief workers (users) carry out their activities on site in accordance with the received relief operation instructions and the delivery route of relief supplies.

[1388] This system will enable the rapid and accurate collection of information in the event of a disaster, and will support effective relief efforts that take into account the user's emotions.

[1389] Example 2

[1390] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1391] When a natural disaster occurs, the challenge is to quickly grasp the extent of damage at the scene and implement effective relief efforts. With conventional systems, it takes time to collect information after a disaster occurs, making it difficult to accurately grasp the extent of the damage. It is also difficult to respond appropriately, taking into account the urgency of the victims.

[1392] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1393] In this invention, the server includes means for detecting the occurrence of a disaster, means for launching multiple unmanned aerial vehicles and collecting video and image data of the affected area, means for analyzing the collected video and image data and generating a three-dimensional map of the damage situation, means for visualizing the generated three-dimensional map and distributing countermeasure guidance, means for receiving information from residents of evacuation centers and relief workers and extracting important information through natural language analysis including emotional data, and means for determining rescue operation instructions and delivery routes for relief supplies based on the extracted information. This enables rapid and accurate information collection when a disaster occurs, and enables effective rescue operations that take into account the emotions and urgency of the victims.

[1394] "Means for detecting the occurrence of disasters" refers to a system that collects information from data sources such as seismometers, meteorological data, and flood monitoring sensors, and detects the occurrence of natural disasters in real time.

[1395] An "unmanned aerial vehicle" is a device that flies automatically or remotely without a human on board and collects video and image data of the affected area.

[1396] "Means for collecting video and image data of the affected area" refers to a mechanism for collecting video and image data of the affected area using a camera mounted on an unmanned aerial vehicle.

[1397] "Means for analyzing collected video and image data and generating a three-dimensional map of the damage situation" refers to a system that uses image analysis technology to reconstruct the damage situation as a three-dimensional model based on the acquired data.

[1398] "Means for visualizing the generated 3D map and distributing countermeasure guidance" refers to a function that displays a 3D map on a terminal at the countermeasure headquarters, generates appropriate action guidelines according to the situation, and distributes them to relevant parties.

[1399] "Means for receiving information from evacuation center residents and relief workers and extracting important information through natural language analysis, including emotional data" refers to a system that receives messages sent from evacuation centers and on-site locations, analyzes them using natural language processing technology, including emotional data, and identifies information with a high level of urgency.

[1400] The "means for determining rescue operation instructions and transportation routes for relief supplies" is a system that, based on analyzed information, issues action instructions for efficiently providing necessary support and determines the optimal transportation route for relief supplies.

[1401] The disaster response system of the present invention is designed to quickly and efficiently assess the damage situation and carry out appropriate relief activities when a disaster occurs. The system includes various hardware and software components.

[1402] Hardware and Software Configuration

[1403] 1. Server:

[1404] The server collects information from weather data APIs, earthquake sensors, flood monitoring sensors, etc., and monitors disaster occurrences in real time. When it determines that a disaster has occurred, it starts operating the entire system.

[1405] 2. Unmanned Aerial Vehicles (Drones):

[1406] When a disaster occurs, the server sends instructions to launch multiple drones in a designated area. The drones follow autonomous flight routes and use their onboard cameras to collect high-resolution video and image data of the affected area.

[1407] 3. Data Collection and Transmission:

[1408] The drone (terminal) transmits the video and image data it acquires during flight to a server in real time using Wi-Fi or mobile networks.

[1409] 4. Data analysis and three-dimensional map generation:

[1410] The server stores the received data in cloud storage and inputs it into a generative AI model. The generative AI model compares data from before and after the disaster, analyzes building damage, road disruption, and the extent of flood damage, and generates a 3D map of the affected area. For example, it uses image recognition technology to identify damage and creates a 3D model based on this.

[1411] 5. Visualization of damage situation and guidance on countermeasures:

[1412] The server visualizes and displays the generated 3D map on the headquarters' terminal. This allows the headquarters to quickly grasp the damage situation and make decisions quickly. Furthermore, the generative AI model automatically generates countermeasure guidance based on the analysis results and distributes this to all parties involved. Specifically, the visualization tool is used to draw the 3D map, and the report generation function is used to create the guidance content.

[1413] 6. Collecting information from the field and analyzing sentiment:

[1414] Users (refugee residents and relief workers) request information about the local situation and necessary relief supplies through a dedicated app or chatbot. This request is sent to a server. The server passes the received message to an emotion engine, which analyzes the emotion in the message. For example, it can automatically determine the emotion of the message using a text analysis API.

[1415] 7. Natural Language Analysis and Key Information Extraction:

[1416] The server passes the emotion data recognized by the emotion engine to a generative AI model, which then performs natural language analysis of the message to extract important information. The emotion data is used as an indicator to identify high-priority requests. Specifically, the message is analyzed using a natural language processing API.

[1417] 8. Directing relief operations and determining routes for transporting relief supplies:

[1418] The server updates the database with the status of relief supplies needed at each evacuation center based on the analysis results. The generative AI model determines the optimal route for delivering relief supplies and generates instructions for relief operations. It also takes into account the results of sentiment analysis and prioritizes requests with high urgency. For example, it uses a route optimization algorithm to calculate efficient routes.

[1419] 9. Distribution of rescue operation instructions:

[1420] The server sends rescue operation instructions and delivery route information for relief supplies to rescue workers in real time, allowing rescue operations to proceed quickly and efficiently. Specifically, it sends instructions to relevant parties using a notification system.

[1421] Examples of concrete examples and prompts

[1422] Examples:

[1423] For example, if a large-scale earthquake occurs, the server immediately launches multiple drones to fly over areas with severe damage. The drones collect video and image data, which the server inputs into a generative AI model in real time. The generative AI model then generates a three-dimensional map of the affected area, identifying collapsed buildings and disrupted roads. At the same time, users submit requests for relief supplies from evacuation centers, and this information is used to determine the optimal route for quickly delivering the necessary supplies. An emotion engine performs sentiment analysis, prioritizing requests with a high level of urgency from users. The server then distributes this information to relief workers, supporting rapid and appropriate relief efforts.

[1424] Example prompt sentence:

[1425] "In areas recently affected by disasters, please analyze the received image data and generate a 3D map of the damage. Also, please consider the relief supply requests sent from evacuation centers, determine the optimal delivery route, and prioritize requests with high priority."

[1426] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1427] Step 1:

[1428] Disaster detection

[1429] The server collects information from multiple data sources, such as seismometers, weather data APIs, and flood monitoring sensors, and monitors the occurrence of natural disasters in real time. The input is information obtained from each sensor and API, and based on this, it performs data calculations to compare it with a disaster occurrence threshold. The output is a signal that sets a "disaster occurrence detection" flag.

[1430] Specific operation:

[1431] 1. Get current weather information from the weather data API.

[1432] 2. Obtain real-time earthquake data from seismometers.

[1433] 3. Compare the acquired data with a threshold to determine whether a disaster has occurred.

[1434] 4. When a disaster is detected, the entire system begins operation.

[1435] Step 2:

[1436] Launching and flying the drone

[1437] When the server detects a disaster, it sends an instruction to launch multiple drones in a designated area. The input is a disaster detection signal and designated area information, and the output is a drone launch command. The drones follow an autonomous flight route and collect video and image data of the affected area.

[1438] Specific operation:

[1439] 1. Automatically launch multiple drones using the drone control API.

[1440] 2. Send the flight route and coordinate information of the affected area to the activated drone.

[1441] 3. The drone begins flying along the specified route.

[1442] Step 3:

[1443] Data collection and transmission

[1444] The drone (terminal) captures high-resolution video and image data using an onboard camera during flight and transmits this data to a server in real time. The input is the video and image data captured by the drone's camera, and the output is the raw data transmitted to the server.

[1445] Specific operation:

[1446] 1. Use the drone camera to capture video and images of the affected area.

[1447] 2. The acquired data is sent to the server in real time.

[1448] Step 4:

[1449] Data analysis and 3D map generation

[1450] The server stores the received video and image data in cloud storage and inputs it into a generative AI model. The generative AI model compares data before and after the disaster and analyzes things like building damage, road disruption, and the extent of flood damage. The input is the data stored in cloud storage and data before and after the disaster, and the output is a 3D map of the affected area.

[1451] Specific operation:

[1452] 1. Save the received data in cloud storage.

[1453] 2. Input data into the generative AI model and analyze the damage situation.

[1454] 3. Generate a three-dimensional map of the affected area based on the analysis results.

[1455] Step 5:

[1456] Visualization of damage status and delivery of countermeasure guidance

[1457] The server visualizes and displays the generated 3D map on the terminal of the emergency response headquarters. The input is the generated 3D map, and the output is the map displayed on the terminal of the emergency response headquarters and countermeasure guidance. Countermeasure guidance is also automatically generated based on the analysis results of the generative AI model and distributed to relevant parties.

[1458] Specific operation:

[1459] 1. Use visualization tools to display 3D maps on the headquarters terminals.

[1460] 2. Create countermeasure guidance based on the analysis results of the generative AI model.

[1461] 3. Distribute the created guidance to each stakeholder.

[1462] Step 6:

[1463] Collecting information from the field and analyzing emotions

[1464] Users (refugee residents and relief workers) use a dedicated app or chatbot to request information about the local situation and necessary relief supplies. The server passes these messages to the emotion engine, which analyzes the emotions in the messages. The input is the message from the user, and the output is the analyzed emotion data.

[1465] Specific operation:

[1466] 1. Users submit information about the local situation and requests for relief supplies through a dedicated app or chatbot.

[1467] 2. The server receives the message and analyzes the emotion using the emotion engine.

[1468] Step 7:

[1469] Natural language analysis and extraction of important information

[1470] The server passes the emotion data recognized by the emotion engine to the generative AI model, which then performs natural language analysis of the message to extract important information. The input is the emotion data and the message, and the output is the extracted important information.

[1471] Specific operation:

[1472] 1. Analyze key messages based on sentiment data.

[1473] 2. Input emotion data into a generative AI model to extract key information.

[1474] Step 8:

[1475] Directing relief operations and determining routes for transporting relief supplies

[1476] Based on the analyzed information, the server updates the database with the status of relief supplies needed at each evacuation shelter. The generative AI model determines the optimal delivery route for relief supplies and generates rescue operation instructions. The input is the extracted important information and the current status of relief supplies, and the output is the confirmed delivery route and rescue operation instructions.

[1477] Specific operation:

[1478] 1. Based on the analysis results, the status of relief supplies is updated in the database.

[1479] 2. Determine the optimal delivery route using a generative AI model.

[1480] 3. Generate rescue operation instructions.

[1481] Step 9:

[1482] Distribution of rescue operation instructions

[1483] The server transmits rescue operation instructions and relief supply delivery route information to rescue workers in the field in real time. The input is the confirmed delivery route and rescue operation instructions, and the output is the information to be distributed to rescue workers in the field. 【...

Claims

1. A means for detecting the occurrence of a disaster; means for launching a plurality of unmanned aerial vehicles to collect video and image data of the affected area; A means for analyzing the collected video and image data and generating a three-dimensional map of the damage situation; A means for visualizing the generated three-dimensional map and delivering guidance on countermeasures; a means for receiving information from evacuation center residents and relief workers and extracting key information through natural language analysis; The system includes a means for determining rescue operation instructions and routes for delivering relief supplies based on the extracted information.

2. The system of claim 1 , wherein the unmanned aerial vehicle is equipped with a camera and flies according to an automated flight route.

3. The system of claim 1 , wherein the three-dimensional map is generated by comparing pre-disaster data and post-disaster data.

Images