system

Abstract

Provide a system. An operable flight device; an operable photographing means; An artificial intelligence processing means for analyzing the generated data in real time; a processing means for calculating an optimal rescue route based on the analysis results; means for carrying and transporting supplies on the flight vehicle; a means for deploying onboard equipment in the communications-blocked area to collect local situation data; means for analyzing the collected data and generating emergency response information; means for linking said emergency response information to external relief organizations; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] During disasters, it is difficult to quickly and accurately grasp the situation on the ground, and initial responses are delayed due to communication failures and land route disruptions. Furthermore, the transportation of supplies cannot be carried out efficiently, further delaying relief activities. These problems often hinder efforts to ensure the safety of disaster victims and to save lives. [Means for solving the problem]

[0005] In order to solve the above problems, we provide the following system. This system is equipped with an operable flying device and a photographing means, and includes an artificial intelligence processing means for analyzing and interpreting the obtained data in real time. The system also includes a processing means for calculating the optimal rescue route based on the results of the analysis, a means for loading supplies onto the flying device and transporting them, a means for deploying onboard equipment in areas where communications are cut off to collect data on the local situation, a means for analyzing the collected data and generating emergency response information, and a means for sharing the emergency response information with external rescue organizations. This invention enables rapid and accurate understanding of on-site information during disasters, enabling efficient transport of supplies and rapid initial response.

[0006] An "operable flying vehicle" is an air vehicle that operates autonomously or remotely to perform a designated mission while moving.

[0007] An "operable capture means" is a device such as a camera or microphone that is used to collect image, video, and audio data.

[0008] "Generated data" refers to information such as images, videos, and audio captured by an operable imaging means.

[0009] "Artificial intelligence processing means for real-time analysis" refers to a processing device or software that uses artificial intelligence to instantly analyze the generated data and grasp the situation on-site.

[0010] "Processing means for calculating optimal rescue route" means a computing device or algorithm for selecting the most suitable route for the rescue operation based on the analyzed data.

[0011] "Means for loading and transporting materials" means any device or mechanism for safely loading and transporting materials to a designated destination.

[0012] "Means for deploying on-board equipment in communication-blocked areas to gather on-site situation data" refers to devices such as satellite phones and digital cameras that are used to obtain on-site information in areas where there is no communication means.

[0013] "Means for analyzing collected data and generating emergency response information" refers to a device or algorithm that analyzes collected data to extract and process information that requires immediate action.

[0014] The "means for linking emergency response information to external rescue organizations" refers to a communication device or system for quickly transmitting the generated emergency response information to rescue organizations. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention is a system for realizing a rapid and accurate initial response in the event of a disaster, and includes an operable flying device (hereinafter referred to as a drone), an operable photographing means, an artificial intelligence processing means for analyzing and analysing generated data in real time, a processing means for calculating the optimal rescue route, a means for loading and transporting supplies, a means for placing onboard equipment in areas where communications are cut off and collecting data on the local situation, a means for analyzing the collected data and generating emergency response information, and a means for linking the emergency response information to external rescue organizations.

[0037] System programs and their processing

[0038] 1. Initial response

[0039] Server: Upon receiving notification of a disaster, the server dispatches a drone to the affected area and sends instructions to the drone, such as flight route, shooting points, and the type of data to be collected.

[0040] Drone (terminal): Arrives at the scene and uses a camera and microphone to collect still and moving images and audio data, which are then sent to a server in real time.

[0041] Server: Analyzes and analyzes the data received in real time to understand the situation on site.

[0042] 2. Analysis and rescue route calculation

[0043] Server: Analyzes collected data using generative AI to identify road closures, dangerous areas, and isolated regions. Then, calculates the optimal rescue route and provides it to the rescue team.

[0044] 3. Material transportation

[0045] Server: Identifies the supplies needed based on the situation in the affected area and creates a delivery plan, sending details such as delivery route, time, and quantity to the drone.

[0046] Drone (terminal): Loads designated supplies and transports them to the disaster area via the optimal route. Once there, the supplies are safely dropped off.

[0047] 4. Responses in areas where communications have been cut off

[0048] Server: In areas where communications have been cut off, the server issues instructions for the use of satellite phones and digital cameras. These devices are then loaded onto drones and sent to the scene.

[0049] Drone (terminal): Fly to designated area and hand over device to local responders, who collect data and transmit it back to the drone.

[0050] Server: Analyzes the returned data and understands the local situation.

[0051] 5. Coordination of emergency response information

[0052] Server: Based on data from the drone, collects information necessary for initial response within 72 hours and generates analysis results.

[0053] Server: Shares information requiring emergency response with nearby hospitals, local governments, and other related organizations, supporting rapid life-saving efforts.

[0054] Specific examples

[0055] For example, consider a case where an earthquake with a seismic intensity of 7 occurs and some areas of an urban area are isolated.

[0056] 1. The server receives notification of an earthquake and issues a command to dispatch a drone to the affected area. The drone is equipped with a camera and a generating AI processing means.

[0057] 2. The drone (terminal) arrives at the disaster area, activates the camera and recording device, collects data in real time, and transmits it to the server.

[0058] 3. The server analyzes the received data, identifies road closures and dangerous areas, calculates the optimal rescue route, and provides this information to the rescue team.

[0059] 4. The server issues instructions to drones capable of flight to load supplies such as medicines and food and deliver them to isolated areas. The drones (terminals) then safely deliver the supplies based on the optimal route.

[0060] 5. In areas where communications are cut off, the server will dispatch drones equipped with satellite phones and digital cameras to deliver the data to responders on the ground. The drones will then return the data collected by the responders to the server for analysis.

[0061] 6. Based on the collected information, the server extracts information that requires emergency response and immediately contacts hospitals and local governments. Medical and rescue teams then respond promptly.

[0062] In this way, by using the system of the present invention, on-site information during a disaster can be quickly and accurately grasped, enabling efficient transportation of supplies and rapid initial response.

[0063] The processing flow will be explained below.

[0064] Step 1:

[0065] The server receives notifications of disaster occurrences and obtains map data of the affected area, weather information, evacuation shelter information, etc. Based on this information, it determines the flight route, shooting points, and type of data to be collected, and sends instructions to the drone.

[0066] Step 2:

[0067] The drone (terminal) follows instructions from the server and flies to the disaster area. After arriving at the scene, it activates its camera and audio recording device to collect still images and audio data. The collected data is then sent to the server in real time.

[0068] Step 3:

[0069] The server receives real-time data sent from the drone and begins analysis using the AI ​​generation system. From the analysis results, it identifies areas where land routes are cut off, dangerous areas, and isolated areas.

[0070] Step 4:

[0071] The server calculates the optimal rescue route based on the analysis results, taking into account the geographical information and traffic conditions of the affected area, and provides the calculation results to the rescue team.

[0072] Step 5:

[0073] The server determines the type, quantity, and delivery destination of the required supplies, creates a delivery plan, and sends instructions to the drones from loading the supplies to delivery.

[0074] Step 6:

[0075] The drone (terminal) follows instructions from the server to load supplies, fly along the optimal route designated, and head to the destination. After arriving at the destination, it safely drops off the supplies.

[0076] Step 7:

[0077] The server sends instructions to drones for use in areas where communications are cut off, including satellite phone and digital camera instructions, including instructions to equip the drones with devices to gather information on the ground.

[0078] Step 8:

[0079] The drone (terminal) flies into a designated quarantined area and hands over its onboard equipment to local personnel, who use these devices to collect data and transmit it back to the drone.

[0080] Step 9:

[0081] The server receives the on-site data returned from the drone and begins analyzing it in real time, and based on the information obtained from the analysis, identifies situations requiring emergency response.

[0082] Step 10:

[0083] The server then sends the identified emergency response information to nearby medical institutions, local governments, and other relevant organizations in real time, enabling rapid life-saving efforts.

[0084] Step 11:

[0085] Based on the information provided by the server, users (local rescue teams) quickly head to the site and begin rescue operations. They also provide the latest on-site information as feedback to the server as necessary.

[0086] By implementing the above steps, the system of the present invention can quickly and efficiently realize initial responses in the event of a disaster, improving the survival rate of disaster victims and enabling the appropriate supply of supplies.

[0087] Example 1

[0088] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0089] In the event of a disaster, a fast and accurate initial response is required. However, it is extremely difficult to quickly grasp the situation in the affected area, calculate the optimal rescue route, and transport the necessary supplies. Furthermore, in areas where communications are cut off, it is difficult to gather information, which can delay emergency responses. To address these challenges, a system is needed that can collect and analyze on-site information in real time and make quick decisions.

[0090] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0091] In this invention, the server includes means for receiving disaster notifications, an operable flight device, an operable photographing device, an artificial intelligence processing means for analyzing generated data in real time, a processing means for calculating an optimal rescue route based on the results of the analysis, means for loading supplies onto the flight device and transporting them, means for deploying equipment in a communication-blocked area and collecting on-site situation data, means for analyzing the collected data and generating emergency response information, means for linking the emergency response information to external rescue organizations, means for analyzing initial response information using a generative AI model, and means for inputting information into the generative AI model using prompt sentences. This allows for quick and accurate understanding of on-site information during a disaster, enabling efficient delivery of supplies and a rapid initial response.

[0092] "Means for receiving disaster notifications" refers to devices or systems for receiving disaster information sent from government agencies, automatic sensors, the Japan Meteorological Agency, etc.

[0093] An "operable flying device" is an unmanned aerial vehicle (drone) that can fly remotely or autonomously and perform a specific task.

[0094] "Operable imaging means" refers to devices such as cameras and microphones that are mounted on the flying device and are used to collect moving and still images and audio data.

[0095] "Artificial intelligence processing means" refers to an artificial intelligence (AI) system that analyzes and parses the generated data in real time.

[0096] The "processing means for calculating optimal rescue routes" refers to calculation algorithms or software for optimizing rescue routes within the disaster area based on collected data.

[0097] "Means for carrying and transporting supplies" refers to a system for loading supplies such as medicines and food onto a flying device and transporting them safely to a designated destination.

[0098] "Means for deploying onboard equipment in areas where communications have been cut off and for collecting on-site situation data" refers to a method for deploying devices such as satellite phones and digital cameras to gather on-site information in areas where communications infrastructure has been destroyed.

[0099] The "means for generating emergency response information" is a system for extracting important disaster information based on collected data and outputting the analysis results.

[0100] The "means for sharing emergency response information with external relief organizations" is a communication system for transmitting the generated emergency response information to relief organizations, including nearby hospitals and local governments.

[0101] A "generative AI model" is a type of artificial intelligence, a machine learning model used to analyze data and support decision-making.

[0102] "Means of inputting information into a generative AI model using prompt sentences" refers to a method of inputting specific questions or commands (prompts) into a generative AI model to obtain appropriate analysis results or output.

[0103] The present invention provides a system for achieving rapid and accurate initial response in the event of a disaster. The system includes an operable flight device, an operable imaging means, an artificial intelligence (AI) processing means for real-time data analysis, a processing means for calculating an optimal rescue route, a means for loading and transporting supplies, a means for deploying an onboard device in a communication-blocked area to collect local situation data, a means for analyzing the collected data and generating emergency response information, and a means for linking the emergency response information to an external relief organization. The system further includes a means for analyzing the initial response information using a generative AI model and prompt sentences.

[0104] Specific hardware and software descriptions

[0105] Flying Devices (Drones):

[0106] The flying device is, for example, a DJI drone. This is an unmanned aerial vehicle that can fly remotely or autonomously and carry out designated tasks. The drone is equipped with a high-resolution camera (e.g., 4K camera) and a recording device. This flying device flies to the affected area in the event of a disaster and collects on-site information.

[0107] Photography Method:

[0108] The drone-mounted imaging device includes a camera and microphone for capturing high-resolution still and video images, as well as audio data, which is transmitted in real time to a server.

[0109] Artificial Intelligence Processing Means:

[0110] The server analyzes the data using a generative AI model (e.g., GPT-4 (registered trademark)). This AI model understands the situation at the disaster site based on the images and audio data received in real time.

[0111] Rescue route calculation method:

[0112] The server calculates the optimal rescue route based on the analysis results. This calculation uses real-time mapping software (e.g., Google® Maps API), the A algorithm, and the Dijkstra algorithm. The calculated route is provided to the rescue team.

[0113] Material transport:

[0114] The server identifies the necessary supplies (e.g., medicines, food, water) based on the situation in the disaster area and creates a transportation plan. The drones load the specified supplies and fly along the optimal route calculated by the server. Once they arrive at the site, they safely drop off the supplies.

[0115] Measures to be taken in areas affected by communication disruptions:

[0116] In areas where communications are cut off, the server dispatches drones equipped with satellite phones and digital cameras. The drones fly to designated areas and hand over these devices to local responders. The responders collect data and send it back to the drone, which the server then analyzes to understand the local situation.

[0117] Emergency response information generation and coordination:

[0118] The server generates information based on the collected data, requiring initial response within 72 hours. Generative AI models are used to analyze the priority and details of the information, and emergency response information is shared with nearby hospitals, local governments, and other relevant organizations. This ensures that life-saving efforts are carried out quickly and accurately.

[0119] Specific operation example

[0120] For example, consider a case where a magnitude 7 earthquake occurs and isolates part of a city. The server receives notification of the earthquake and dispatches a drone to the affected area. The drone is equipped with a camera and a generating AI processing means, and upon arriving at the site, it activates the camera and audio recording device to collect data in real time and send it to the server.

[0121] The server analyzes the received data, identifies road closures and dangerous areas, and calculates the optimal rescue route. This information is provided to the rescue team. It then issues instructions to drones capable of flying to load medicines, food, and other supplies and transport them to isolated areas. The drones then safely deliver the supplies based on the optimal route.

[0122] In areas where communications are cut off, the server will dispatch drones equipped with satellite phones and digital cameras, handing these devices over to responders on the scene, who will then collect data, which the drones will then send back to the server for analysis.

[0123] An example of a prompt is, "A magnitude 7 earthquake has occurred in an urban area, isolating some areas. Please collect the necessary data and calculate the optimal rescue route so that rescue teams can reach the affected areas in the shortest time." By inputting this prompt into a generative AI model, it is possible to obtain quick and accurate analysis results.

[0124] By using the system of the present invention, on-site information during a disaster can be quickly and accurately grasped, enabling efficient transportation of supplies and rapid initial response.

[0125] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0126] Step 1:

[0127] Receive notifications of disaster occurrences

[0128] Input: Disaster notifications from government agencies, weather agencies, or automated sensors

[0129] Processing: The server receives the disaster notification and records the information in a database. It analyzes detailed information such as the type, location, and scale of the disaster, and begins preparations for an initial response.

[0130] Output: Record of disaster information, instructions for preparation for initial response

[0131] Step 2:

[0132] Drone dispatch instructions

[0133] Input: Information on the occurrence of the disaster, geographical information of the affected area

[0134] Processing: The server calculates the most efficient flight route, shooting points, and type of data to collect based on the geographical information of the affected area. This calculation is done using real-time mapping software (e.g., Google Maps API).

[0135] Output: Instructions for the drone to fly, capture points, and collect data

[0136] Step 3:

[0137] Data collection

[0138] Input: Instructions sent from the server, local video stream, audio data

[0139] Processing: When the drone (terminal) receives the command, it begins flying toward the disaster area. After arriving at the scene, it activates the camera and microphone to collect video and audio data. The collected data is sent to the server in real time.

[0140] Output: Video and audio data sent to the server in real time

[0141] Step 4:

[0142] Data analysis

[0143] Input: Video and audio data transmitted from the drone

[0144] Processing: The server analyzes the received data. A generative AI model (e.g., GPT-4) is used for the analysis, and the situation in the disaster area is grasped through image and audio analysis. For example, the state of collapse of a building can be determined from the image.

[0145] Output: Situation report of disaster area, identification of dangerous areas

[0146] Step 5:

[0147] Rescue route calculation

[0148] Input: Situation report of disaster area, information of danger zone

[0149] Processing: The server calculates the optimal rescue route, avoiding road closures, dangerous areas, and isolated areas, using the A algorithm and Dijkstra algorithm.

[0150] Output: Optimal rescue route information for rescue teams

[0151] Step 6:

[0152] Material transportation plan

[0153] Input: Situation in the affected areas, list of necessary supplies

[0154] Processing: The server creates a transportation plan. It creates a plan to load the necessary supplies (e.g. medicines, food, water) onto the drone. It calculates details such as the delivery route, delivery time, and delivery volume, and sends instructions to the drone.

[0155] Output: Instructions for drones to deliver supplies

[0156] Step 7:

[0157] Material transportation

[0158] Input: Material transport instructions from the server

[0159] Processing: The drone (terminal) loads the designated supplies and begins flying based on the optimal route calculated by the server. Once it arrives at the disaster area, it safely drops off the supplies.

[0160] Output: Confirmation of delivery of supplies to disaster area

[0161] Step 8:

[0162] Response to communication blackout areas

[0163] Input: Information on communication-blocked areas, server instructions

[0164] Processing: The server dispatches a drone equipped with a satellite phone and digital camera. The drone (terminal) flies to the designated area and hands over the equipment to local responders. The local responders collect the data, which is then sent back by the drone.

[0165] Output: Data collection and return from field responders

[0166] Step 9:

[0167] Emergency response information generation and coordination

[0168] Input: Data from local responders, analysis data from the affected areas

[0169] Processing: The server generates the necessary emergency response information based on the collected data. It uses a generative AI model to analyze the priority and details of the information, and then shares the emergency response information with nearby hospitals, local governments, and other relevant organizations.

[0170] Output: Emergency response information, information on cooperation with each relief organization

[0171] This will enable quick and accurate understanding of on-site information during disasters, enabling efficient transportation of supplies and rapid initial response.

[0172] (Application example 1)

[0173] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0174] In today's world, there is a need for rapid and accurate responses to disasters and emergencies. However, delays in information gathering and emergency response can be problematic when it is difficult to grasp the situation or transport supplies, or in areas where communications are cut off. In the field of security services, real-time on-site monitoring and emergency response are also important, and efficient systems are needed.

[0175] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0176] In this invention, the server includes means for controlling an operable flying device to collect video and audio of the scene in real time, means for analyzing the collected data with a generative AI model to detect anomalies, means for calculating and proposing an optimal response route based on the detected anomalies, and means for controlling the operable flying device to transport necessary supplies to the scene, thereby enabling prompt and accurate situation assessment and response at the scene in the event of a disaster or security service.

[0177] An "operable flying device" is a device that has the ability to fly to a designated location, collect data such as video and audio, and transport supplies.

[0178] "Operable imaging means" means a device mounted on a flying device that has the capability to capture and record still images, video, and audio in real time.

[0179] "Artificial intelligence processing means" refers to computer systems or software that have the functionality to analyze and analyse collected data in real time.

[0180] "Processing means" refers to a system that has the functionality to calculate and generate optimal routes and action plans based on the results of analysis.

[0181] "Means for loading and transporting supplies" refers to devices or systems that have the function of loading the necessary supplies onto a flight vehicle and transporting them safely to a designated location.

[0182] "Means for deploying on-board equipment in areas where communications are cut off and collecting data on the local situation" refers to equipment that has the function of deploying equipment such as satellite phones and digital cameras in areas where communications are cut off using flying devices to grasp the local situation.

[0183] A "generative AI model" is a learning model used to analyze collected data and detect anomalies and specific patterns.

[0184] A "prompt sentence" is text data containing instructions or commands input to a generative AI model.

[0185] "Emergency response information" is information that requires a rapid response and is generated based on collected local data.

[0186] The "means for detecting anomalies" refers to an AI analysis system that analyzes collected video and audio data to detect abnormal situations or suspicious behavior.

[0187] The "means for proposing the optimal response route" is a system that calculates and proposes the most efficient route of action to security staff and rescue teams after detecting an abnormality.

[0188] In this invention, a comprehensive system is constructed to respond quickly and accurately to disasters and security services. The detailed configuration and operation of the system will be described below.

[0189] System Configuration

[0190] 1. Hardware Configuration

[0191] Flying Devices (Drones):

[0192] The flying device is equipped with a camera and microphone to collect video and audio in real time, and the drone is equipped with GPS for precise navigation to specific locations.

[0193] Camera and audio collection devices:

[0194] High-resolution cameras and sensitive microphones are used to capture detailed video and audio of the scene.

[0195] Communication equipment:

[0196] Using equipment such as satellite phones and digital cameras, it is possible to collect data and communicate even in areas where communications are cut off.

[0197] 2. Software Configuration

[0198] Artificial Intelligence Processing System:

[0199] Using a generative AI model, collected video and audio data is analyzed in real time to detect anomalies.

[0200] Route Calculation System:

[0201] It is equipped with an algorithm that calculates the optimal rescue or response route based on the analysis results.

[0202] Data Streaming System:

[0203] It provides a server that uses open source libraries (e.g., OpenCV) to stream video in real time.

[0204] Frontend application:

[0205] It provides an intuitive interface for security staff and users through smartphone apps and web applications.

[0206] Processing content

[0207] Data collection:

[0208] The drone moves to a designated location and uses a camera and microphone to collect video and audio, which is then sent to a server in real time.

[0209] Data Analysis:

[0210] The collected data is analyzed using a generative AI model on the server, and if abnormal behavior or dangerous situations are detected, an alert is sent immediately.

[0211] Route calculation and suggestions:

[0212] Based on the analysis results, the optimal response route is calculated, and this information is sent in real time to a smartphone app or web application, providing it to security staff and rescue teams.

[0213] Material transport:

[0214] If necessary, the drones will carry medicines and emergency supplies and deliver them based on the optimal route.

[0215] Specific examples

[0216] For example, if a large earthquake occurs and a certain area is isolated, the system operates as follows.

[0217] 1. Drones are quickly deployed to isolated areas and use high-resolution cameras and microphones to collect on-site information.

[0218] 2. The collected data is sent to a server in real time, where a generative AI model analyzes it to assess the extent of damage to buildings and the risk to human life.

[0219] 3. Based on the analysis results, the server calculates the optimal route for the rescue team and notifies them via a smartphone app.

[0220] 4. Drones will carry necessary medicines and food and deliver supplies to the site based on the optimal route.

[0221] 5. Users can check the situation on-site in real time through the application and respond quickly.

[0222] Prompt sentences to input to the generative AI model

[0223] Please analyze the following camera footage frames and predict an anomalous activity score:

[0224] Frame: [Camera image data]

[0225] This will enable quick and accurate situation assessment and response in the event of a disaster or security service.

[0226] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0227] Step 1:

[0228] Launching the drone and dispatching it to the site

[0229] When a user receives a notification of a disaster or emergency, they send a command to the server to dispatch a drone. The server calculates a flight route to the specified location and sends it to the drone. The drone then launches and moves to the scene along the instructed route.

[0230] Input: Disaster occurrence notification from user, coordinate information of designated location

[0231] Output: Drone flight route calculation results, drone arrival at the site

[0232] Step 2:

[0233] Data collection

[0234] The drone arrives at the scene and collects video and audio data in real time using its on-board camera and microphone, which is then sent to a server in real time.

[0235] Input: Collecting video and audio data on-site

[0236] Output: Real-time data transmission to the server

[0237] Step 3:

[0238] Data analysis

[0239] The server inputs the received video and audio data into a generative AI model for real-time analysis, which processes the data to detect abnormal behavior or dangerous situations and returns the results.

[0240] Input: Collected video and audio data

[0241] Output: Anomaly detection results (anomaly score)

[0242] Specific operation: The generative AI model analyzes video frames and outputs an abnormal behavior detection score.

[0243] Step 4:

[0244] Optimal route calculation

[0245] The server uses the analysis results to calculate the optimal route for rescue teams and security staff, and provides the calculated route information in real time to smartphone apps and web applications.

[0246] Input: Anomaly detection results, site geographic information

[0247] Output: Optimal route information

[0248] Specific operation: Combining GPS data and anomaly detection results to calculate the shortest and safest route and notify the application.

[0249] Step 5:

[0250] Material transportation

[0251] If necessary, the server sends instructions to the drone to load medicines or emergency supplies, and the drone then delivers the supplies to the site based on the optimal route.

[0252] Input: Required material information, optimal transport route information

[0253] Output: Supplies arrive at the site

[0254] Specific operation: Based on instructions from the server, the drone loads supplies and delivers them to the site.

[0255] Step 6:

[0256] Real-time monitoring and feedback

[0257] Users can check the on-site video and analysis results in real time via a smartphone app or web application and take necessary action. The server receives feedback from the user and adjusts the next action.

[0258] Input: User feedback, real-time analysis results

[0259] Output: Next action instructions

[0260] Specific operation: Based on the information provided by the user, the system responds on-site and sends feedback to the server to continuously optimize the response.

[0261] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0262] The present invention is a system for realizing a rapid and accurate initial response in the event of a disaster, and includes an operable flying device (hereinafter referred to as a drone), an operable photographing means, an artificial intelligence processing means for analyzing and analysing generated data in real time, a processing means for calculating the optimal rescue route, a means for loading and transporting supplies, a means for placing onboard equipment in areas where communications are cut off and collecting data on the local situation, a means for analyzing the collected data and generating emergency response information, and a means for linking the emergency response information to external rescue organizations, as well as an emotion engine that recognizes the user's emotions.

[0263] System programs and their processing

[0264] 1. Initial response

[0265] Server: Upon receiving notification of a disaster, it obtains map data of the affected area, weather information, evacuation shelter information, etc. Based on this information, it sends instructions to the drone, such as flight route, shooting points, and type of data to be collected.

[0266] Drone (terminal): Arrives at the scene and uses a camera and microphone to collect still and moving images and audio data, which are then sent to a server in real time.

[0267] Server: Analyzes and analyzes the data received in real time to understand the situation on site.

[0268] 2. Analysis and rescue route calculation

[0269] Server: Analyzes collected data using generative AI to identify road closures, dangerous areas, and isolated regions. Then, calculates the optimal rescue route and provides it to the rescue team.

[0270] 3. Material transportation

[0271] Server: Identifies the supplies needed based on the situation in the affected area and creates a delivery plan, sending details such as delivery route, time, and quantity to the drone.

[0272] Drone (terminal): Loads designated supplies and transports them to the disaster area via the optimal route. Once there, the supplies are safely dropped off.

[0273] 4. Responses in areas where communications have been cut off

[0274] Server: In areas where communications have been cut off, the server issues instructions for the use of satellite phones and digital cameras. These devices are then loaded onto drones and sent to the scene.

[0275] Drone (terminal): Fly to designated area and hand over device to local responders, who collect data and transmit it back to the drone.

[0276] Server: Analyzes the returned data and understands the local situation.

[0277] 5. Coordination of emergency response information

[0278] Server: Based on data from the drone, collects information necessary for an initial response within 72 hours and generates analysis results.

[0279] Server: Shares information requiring emergency response with nearby hospitals, local governments, and other related organizations, enabling rapid life-saving efforts.

[0280] Incorporating an emotion engine

[0281] 1. Collecting Emotional Data

[0282] Drone (terminal): Data collected in the disaster area includes audio and video footage of the victims. This data is sent to the emotion engine.

[0283] Server: The acquired audio and video data is passed to the emotion engine, which analyzes the emotional state of the victim.

[0284] 2. Sentiment analysis and prioritization of rescue operations

[0285] Emotion engine (on the server): Analyzes the emotional state of victims (e.g., fear, anxiety, relief) and reflects that information in planning rescue operations.

[0286] Server: Dynamically change the priority of rescue efforts based on the results of emotion analysis. For example, prioritize assistance to areas with many disaster victims who are feeling extreme fear or anxiety.

[0287] Specific examples

[0288] For example, consider a case where an earthquake with a seismic intensity of 7 occurs and some areas of an urban area are isolated.

[0289] 1. The server receives notification of a disaster and issues instructions to dispatch drones to the affected area. The drones are equipped with a camera, a generation AI processing means, and an emotion engine.

[0290] 2. The drone (terminal) arrives at the disaster area, activates its camera and recording device to collect data in real time, and transmits audio and video data to the emotion engine.

[0291] 3. The server analyzes the received data, identifies road closures and dangerous areas, calculates the optimal rescue route, and provides this information to the rescue team.

[0292] 4. The server issues instructions to drones capable of flight to load supplies such as medicines and food and deliver them to isolated areas. The drones (terminals) then safely deliver the supplies based on the optimal route.

[0293] 5. In areas where communications are cut off, the server will dispatch drones equipped with satellite phones and digital cameras to deliver the data to responders on the ground. The drones will then return the data collected by the responders to the server for analysis.

[0294] 6. Based on the collected information, the server extracts information that requires emergency response and immediately contacts hospitals and local governments. Medical and rescue teams then respond promptly.

[0295] 7. The emotion engine analyzes the voice and video data of victims to understand their emotional state, allowing the server to appropriately adjust the priority of rescue efforts and provide the necessary assistance quickly.

[0296] In this way, by using the system of the present invention, on-site information can be quickly and accurately grasped during a disaster, enabling efficient transportation of supplies and a rapid initial response. In addition, by incorporating an emotion engine, appropriate support can be provided taking into account the emotional state of the victims.

[0297] The processing flow will be explained below.

[0298] Step 1:

[0299] The server receives notifications of disaster occurrences and obtains map data of the affected area, weather information, evacuation shelter information, etc. Based on this information, a program is prepared to instruct the drone on flight routes, shooting points, and the type of data to be collected.

[0300] Step 2:

[0301] The server sends flight commands to the drone, instructing it to head towards the designated area.

[0302] Step 3:

[0303] The drone (terminal) flies to the disaster site according to instructions from the server, and upon arrival, activates its camera and recording device, collecting still and moving images and audio data, which it then transmits to the server in real time.

[0304] Step 4:

[0305] The server receives real-time data transmitted by the drone and begins analyzing it using generative AI, which identifies road disruptions, dangerous areas, and isolated regions.

[0306] Step 5:

[0307] Based on the analysis results, the server calculates the optimal rescue route to the disaster area, and provides the calculation results to the rescue team, enabling them to carry out rescue operations quickly.

[0308] Step 6:

[0309] The server determines the type and amount of supplies needed based on the situation in the disaster area, creates a supply delivery plan, including delivery routes, times, and quantities, and transmits this plan to the drone.

[0310] Step 7:

[0311] The drone (terminal) follows instructions from the server to load supplies, fly to the disaster area along the optimal route, and safely drop off the supplies upon arrival.

[0312] Step 8:

[0313] The server sends instructions to the drones to deploy satellite phones and digital cameras in areas where communications have been cut off, including specific operational procedures for gathering information on the situation on the ground.

[0314] Step 9:

[0315] The drone (terminal) flies into a designated blackout area and hands over a satellite phone and digital camera to local personnel, who use these devices to collect data and transmit it back to the drone.

[0316] Step 10:

[0317] The server receives the local situation data returned by the drone and analyzes it in real time using generative AI, extracting information necessary for emergency response.

[0318] Step 11:

[0319] The server then sends the identified emergency response information to nearby medical institutions, local governments, and related organizations in real time, enabling rapid life-saving efforts.

[0320] Step 12:

[0321] The drone (terminal) collects audio and video data from the victim and sends it to the emotion engine, which analyzes the audio and video to identify the victim's emotional state.

[0322] Step 13:

[0323] The emotion engine (in the server) analyzes the emotional state of the victims (e.g., fear, anxiety, relief) and incorporates this information into the rescue operation plan in the server.

[0324] Step 14:

[0325] The server executes a means for dynamically changing the priority of relief efforts based on the emotion information provided by the emotion engine, thereby giving priority to providing support to areas with a large number of disaster victims who are extremely emotionally unstable.

[0326] Step 15:

[0327] The users (local rescue teams) quickly carry out rescue operations on site based on real-time information and priority adjustments provided by the server. Feedback during operations is also provided to the server to help with future decisions.

[0328] Through these steps, the system of the present invention can quickly and efficiently realize initial responses in the event of a disaster. This system improves the survival rate of disaster victims and enables the appropriate supply of supplies. Furthermore, by incorporating an emotion engine, it becomes possible to provide appropriate support that takes into account the emotional state of disaster victims.

[0329] Example 2

[0330] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0331] While rapid and accurate initial response is crucial in the event of a disaster, conventional systems often delay the collection of information needed to properly grasp the situation on the ground, and rescue operations are often carried out based on inaccurate data. Furthermore, support does not take into account the emotional state of the victims, resulting in insufficient psychological care and making it difficult to properly prioritize rescue operations. To solve these issues, a system is needed that can collect and analyze on-site information in real time, calculate optimal rescue routes, and dynamically adjust the priority of rescue operations by recognizing the emotional state of the victims.

[0332] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an operable flying device, an operable photographing means, an artificial intelligence processing means for analyzing the generated data in real time, a processing means for calculating an optimal rescue route based on the analysis results, a means for loading supplies onto the flying device and transporting them, a means for deploying an onboard device in a communication-blocked area and collecting local situation data, a means for analyzing the collected data and generating emergency response information, a means for linking the emergency response information to external rescue organizations, an emotion analysis means for analyzing audio and video data of victims and recognizing their emotional states, and a means for dynamically adjusting the priority of rescue operations based on their emotional states. This enables rapid and accurate understanding of the situation at the scene of a disaster, enabling efficient rescue operations and delivery of supplies, and realizing appropriate support that takes into account the psychological care of victims.

[0333] An "operable flying device" is a device capable of airborne flight and reaching designated locations to collect data or deliver supplies.

[0334] "Operable imaging means" means a camera or other imaging device capable of capturing still or video images and attached to a flying device for data collection purposes.

[0335] "Artificial intelligence processing means for analyzing and analysing generated data in real time" refers to an artificial intelligence-based processing system used to instantly analyze and analyse collected data and grasp the situation on site.

[0336] "Processing means for calculating optimal rescue route" refers to a processing system for calculating the most efficient route for carrying out rescue operations based on the data analyzed in real time.

[0337] "Means for loading and transporting supplies onto a flying vehicle" means the mechanism or structure for loading relief supplies onto a flying vehicle and transporting them to a designated disaster area.

[0338] "Means for deploying on-board equipment in areas where communications are cut off and collecting on-site situation data" refers to methods and devices for deploying the necessary equipment and collecting on-site situation data so that data collection is possible even in areas where communications infrastructure has been destroyed.

[0339] "Means for analyzing collected data and generating emergency response information" refers to a processing system that analyzes collected data, identifies situations requiring emergency response, and generates response information.

[0340] The "means for sharing emergency response information with external relief organizations" refers to a communication means for quickly and accurately transmitting the generated emergency response information to external relief organizations.

[0341] An "emotion analysis means" is an analysis system that analyzes the audio and video data of victims to recognize their emotional state (e.g., fear, anxiety, relief).

[0342] "Means for dynamically adjusting the priority of rescue operations based on emotional state" refers to a processing system for changing the priority of rescue operations in real time based on the emotional state of the victim recognized by the emotion analysis means.

[0343] The present invention provides a system for achieving rapid and accurate initial response in the event of a disaster. The system includes an operable flight device, an operable imaging means, an artificial intelligence processing means for analyzing and interpreting generated data in real time, a processing means for calculating an optimal rescue route, a means for loading and transporting supplies, a means for deploying an onboard device in a communication-blocked area and collecting local situation data, a means for analyzing the collected data and generating emergency response information, a means for linking the emergency response information to an external rescue organization, an emotion analysis means for recognizing the emotions of disaster victims, and a means for dynamically adjusting the priority of rescue operations based on the emotional state.

[0344] When the server receives notification of a disaster, it obtains map data, weather information, and evacuation shelter information for the affected area. Based on this information, it determines the flight route, shooting points, type of data to be collected, and issues instructions to the drone. The drone (terminal) arrives at the scene and uses its camera and microphone to collect still and moving images and audio data, which it then sends to the server in real time. The server analyzes the data received in real time and uses a generative AI model to understand the situation. Machine learning libraries such as TENSORFLOW (registered trademark) and PyTorch are used for this analysis.

[0345] The server runs an algorithm (such as the A algorithm or Dijkstra algorithm) to calculate the optimal rescue route based on the analyzed data. This identifies areas where roads are cut off, dangerous areas, and isolated regions, and provides this information to rescue teams. At the same time, it identifies necessary supplies (food, medicine, etc.) and creates a delivery plan. It issues instructions to drones capable of flying to load the supplies and deliver them to the disaster area via the optimal route.

[0346] In areas where communications are cut off, the server dispatches drones equipped with satellite phones and digital cameras and hands them over to local responders. The data collected by the local responders is then sent back to the server by drone. The server then analyzes this data to accurately grasp the local situation.

[0347] Emergency response information is compiled based on data from drones, generating the information required for initial response within 72 hours. The server then shares this information with hospitals and local governments to support rapid life-saving efforts. Furthermore, audio and video data collected in the disaster area is sent to an emotion analysis system, which analyzes the emotional state of the victims (e.g., fear, anxiety, relief) and incorporates this information into rescue operation plans. Based on this information, the server dynamically adjusts the priority of rescue operations.

[0348] As a concrete example, consider a case where a magnitude 7 earthquake occurs and isolates part of an urban area. The server receives a disaster notification and immediately dispatches a drone to collect information on the local situation. The collected data is analyzed to determine the optimal rescue route and a plan for transporting supplies. At the same time, the emotional state of the victims is analyzed and optimal support activities are carried out, taking into consideration their mental health.

[0349] Below is an example of a prompt sentence:

[0350] "A magnitude 7 earthquake has occurred, isolating parts of urban areas. Dispatch drones to quickly collect on-site information from the affected areas, calculate rescue routes, and develop an optimal plan for delivering necessary supplies. Also, analyze the emotional state of the victims using audio and video data to appropriately prioritize rescue efforts."

[0351] The above is an embodiment of the present invention, and by using this system, on-site information can be quickly and accurately grasped during a disaster, enabling efficient rescue operations and the transportation of supplies. In addition, by using emotion analysis means, appropriate support can be provided taking into account the emotional state of the victims.

[0352] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0353] Step 1: Receive notifications and prepare

[0354] The server receives notifications of disaster occurrences, which are provided in real time by meteorological agencies and monitoring systems.

[0355] Input: Disaster notification (e.g. earthquake, flood)

[0356] The server obtains map data, weather information, and evacuation shelter information for the affected area, and integrates this information to determine flight routes, photography points, and the type of data to be collected.

[0357] Data processing: Map data, weather information, and evacuation shelter information are obtained using APIs, and this is integrated to generate drone instruction information.

[0358] Output: Flight route, shooting points, type of collected data

[0359] Specific operation: The server retrieves map data using the Google Maps API, weather information from the Japan Meteorological Agency API, and evacuation shelter information from government databases.

[0360] Step 2: Dispatch the drone and collect data

[0361] The server transmits the predetermined flight route and collection items to the drone's command center.

[0362] Input: Flight route, shooting points, type of collected data

[0363] The drone (terminal) follows a designated flight route, uses a camera and microphone to collect video and audio data, and transmits it to a server in real time.

[0364] Data calculation: Using GPS, the drone flies autonomously and collects data at designated points.

[0365] Output: Real-time transmission of collected video and audio data

[0366] Specific operation: The drone flies using its built-in GPS and communication module, takes pictures with its camera, records sounds with its microphone, and uploads the collected data to a server in real time.

[0367] Step 3: Data analysis and situation assessment

[0368] The server analyzes the received video and audio data using a generative AI model.

[0369] Input: Collected video and audio data

[0370] The server uses TensorFlow or PyTorch to analyze the image data and identify the situation at the scene (e.g., collapsed buildings, flooded areas), and converts the audio data into text using the Google Cloud Speech-to-Text API and passes it to the sentiment analysis tool.

[0371] Data processing: image data analysis, voice data text conversion

[0372] Output: Analysis results (collapsed buildings, flooded areas), audio data converted to text

[0373] Specific operation: The server analyzes image data using an object recognition algorithm, and audio data is converted into text and used for emotion analysis.

[0374] Step 4: Calculate the rescue route

[0375] The server calculates the optimal rescue route based on the analysis results.

[0376] Input: Analysis results (collapsed buildings, flooded areas)

[0377] The server calculates the optimal route using the A algorithm or Dijkstra algorithm and provides it to the rescue team.

[0378] Data calculation: Calculating the optimal rescue route

[0379] Output: Optimal rescue route

[0380] Specific operation: The server calculates the shortest and safest rescue route taking into account road conditions and obstacles, and notifies the rescue team.

[0381] Step 5: Prepare and execute the shipment

[0382] The server identifies the necessary supplies and issues instructions to load them onto the drone.

[0383] Input: Situation in the affected area, list of necessary supplies

[0384] The drone (terminal) will be loaded with designated supplies and delivered to the disaster area.

[0385] Data processing: material loading plans, transport route instructions

[0386] Output: Material transportation plan, transportation route

[0387] Specific operation: The drone loads the specified supplies and safely delivers them along the programmed route.

[0388] Step 6: Response to communication disruptions in affected areas

[0389] The server will dispatch drones equipped with satellite phones and digital cameras to areas where communications have been cut off.

[0390] Input: Information on communication-blocked areas, list of required equipment

[0391] The drone (terminal) drops off the equipment on-site and transmits the collected data back to the server.

[0392] Data processing: Equipment layout plan, return of collected data

[0393] Output: Local situation data

[0394] Specific operation: The drone hands over the equipment to the local responder and sends the collected data back to the server.

[0395] Step 7: Generate and coordinate emergency response information

[0396] The server generates emergency response information based on data from the drone.

[0397] Input: collected data, analysis results

[0398] The server aggregates the data and generates information necessary for initial response within 72 hours, which is then shared with hospitals and local governments.

[0399] Data processing: Emergency response information generation and notification

[0400] Output: Emergency response information

[0401] Specific operation: The server uses APIs and SMS gateways to directly communicate the generated emergency information to each agency.

[0402] Step 8: Sentiment analysis and prioritization

[0403] The server transmits audio and video data collected in the disaster area to the emotion analysis means.

[0404] Input: Text-encoded audio data, video data

[0405] The emotion analysis means recognizes the emotional state of the victim and sends that information back to the server.

[0406] Data Computing: Emotional State Analysis

[0407] Output: Emotion analysis results

[0408] The server dynamically adjusts the priority of rescue operations based on the results of emotion analysis.

[0409] Data processing: Dynamic adjustment of priorities

[0410] Output: Dynamically adjusted rescue operation priorities

[0411] How it works: The server uses a generative AI model to perform emotion analysis and re-prioritize based on the emotional state.

[0412] (Application example 2)

[0413] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0414] During disasters, prompt and accurate initial responses are required, but appropriate rescue efforts are often hindered due to the uncertainty of the situation on the ground. Furthermore, in areas where communications are cut off, it is difficult to gather information, making it impossible to properly prioritize relief efforts. Furthermore, prioritization of relief efforts does not take into account the emotional state of the victims, which is a factor in delaying appropriate assistance to victims.

[0415] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0416] In this invention, the server includes an operable flying device, an operable photographing means, an artificial intelligence processing means for analyzing generated data in real time, a processing means for calculating an optimal rescue route based on the results of the analysis, a means for loading supplies onto the flying device and transporting them, a means for deploying an onboard device in a communication-blocked area and collecting on-site situation data, a means for analyzing the collected data and generating emergency response information, a means for linking the emergency response information to external rescue organizations, a means for collecting user emotion data on an operating terminal and performing emotion analysis, and a means for adjusting the priority of rescue operations based on the results of the emotion analysis, thereby enabling quick and accurate understanding of on-site information in disaster-stricken areas, efficient delivery of supplies, and appropriate prioritization of rescue operations.

[0417] An "operable flying device" is a flying machine that can be remotely controlled or autonomously operated, such as a drone, and is used to collect information and transport supplies during disasters.

[0418] "Operable imaging means" means a camera or other imaging device used to capture still or video images of a scene and collect information in real time.

[0419] "Artificial intelligence processing means for analyzing and analysing generated data in real time" refers to artificial intelligence algorithms and systems that rapidly process collected data and extract and analyse necessary information.

[0420] The "processing means for calculating the optimal rescue route" refers to an algorithm or software for calculating the safest and most efficient rescue route in the event of a disaster.

[0421] "Means for loading and transporting supplies" refers to the equipment or mechanism for loading relief supplies and other items onto a flying device and transporting them to their destination.

[0422] "Means for deploying on-board equipment in areas where communications are disrupted and collecting local situation data" refers to methods and systems for deploying machines or devices in areas where communications infrastructure has been destroyed and collecting the necessary information.

[0423] "Means for generating emergency response information" refers to algorithms or systems that analyze collected data and generate information required for rapid response.

[0424] "Means for linking emergency response information to external relief agencies" refers to communication means or systems for quickly transmitting the generated emergency response information to relevant relief agencies or organizations.

[0425] "Means for collecting user emotional data on operating devices and performing emotional analysis" refers to technologies and systems for collecting audio and video data from devices used by disaster victims and analyzing their emotional states.

[0426] The "means for adjusting the priority of rescue operations based on the results of sentiment analysis" refers to an algorithm or system for dynamically changing the priority of rescue operations based on the results obtained from sentiment analysis.

[0427] In order to put this invention into practice, it is important to build a system that can collect information in real time during a disaster and provide a rapid and accurate initial response.

[0428] System Configuration

[0429] 1. Operable flying device

[0430] These devices, such as drones, are capable of remote control and autonomous operation, and are used to gather information and transport supplies at disaster sites.

[0431] 2. Operable photography means

[0432] The drone is equipped with a high-resolution camera that can take still images and videos of the disaster area and transmit them in real time.

[0433] 3. Artificial intelligence processing means for analyzing and analysing the generated data in real time

[0434] The server is equipped with a generative AI model that quickly analyzes and parses the data sent from the drone.

[0435] Example) AI model used: TensorFlow, PyTorch

[0436] 4. Processing means for calculating the optimal rescue route

[0437] Based on the analysis results, the server uses an algorithm to calculate the optimal rescue route and provides it to rescue agencies.

[0438] Example) Algorithms used: Dijkstra's Algorithm, A Search Algorithm

[0439] 5. Means for carrying and transporting supplies

[0440] The drones are equipped with a mechanism to load designated supplies and transport them quickly along a calculated, optimal route.

[0441] 6. Means of deploying onboard equipment in communication-blocked areas to gather on-site situation data.

[0442] The drones will deploy satellite-enabled equipment in areas where communications infrastructure has been destroyed and collect situational data.

[0443] 7. Means for generating emergency response information

[0444] The server analyzes the collected data and generates information needed for rapid response, including weather information, evacuation shelter information, and more.

[0445] 8. Means for communicating emergency response information to external relief agencies

[0446] The generated emergency response information is automatically sent to relevant relief agencies, enabling a rapid response.

[0447] 9. Means for collecting user emotion data and performing emotion analysis on the operating device

[0448] It collects audio and video data from devices such as drones and smartphones used by disaster victims, and analyzes the user's emotional state.

[0449] Example) Emotion analysis technology used: EmotionRecognizer

[0450] 10. A method for adjusting rescue operation priorities based on the results of sentiment analysis

[0451] Based on the results of emotion analysis, the server dynamically adjusts the priorities of rescue efforts, prioritizing support for areas with a high number of victims experiencing extreme fear and anxiety.

[0452] Specific examples of processing

[0453] For example, consider a case where a magnitude 7 earthquake occurs in an urban area, causing communication to be unavailable in some areas.

[0454] 1. The server receives information about the occurrence of a disaster and issues instructions to the drone to dispatch to the site.

[0455] 2. After arriving at the site, the drone uses a camera to collect data and transmits it to a server in real time.

[0456] 3. The server analyzes the received data using a generative AI model to identify road disruptions and dangerous areas.

[0457] 4. When the user reports their situation and emotional state using a smartphone app, the server uses that data to calculate the optimal rescue route.

[0458] Example prompt sentence:

[0459] "The user's current location is Tokyo, the surrounding area is in a state of great chaos, and they have entered the emotion message 'It's very scary; people around me are crying.' Please display the optimal evacuation route and their emotional state."

[0460] By building such a system, it will be possible to quickly and accurately grasp the situation in disaster-stricken areas, enabling efficient and appropriate relief efforts. Furthermore, prioritizing relief efforts based on emotion analysis will enable appropriate support that takes into account the emotional state of the victims.

[0461] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0462] Step 1:

[0463] The server receives information about the occurrence of a disaster. At the same time, it also acquires map data of the affected area, weather information, and evacuation shelter information. Based on this information, it sends instructions to the drone, such as flight route, shooting points, and the type of data to be collected.

[0464] Input: Disaster occurrence notification, map data, weather information, evacuation shelter information

[0465] Data processing and calculation: Data integration, calculation using flight route optimization algorithm

[0466] Output: Instruction data for the drone (flight route, shooting points, type of collected data)

[0467] Step 2:

[0468] The drone (terminal) arrives at the scene and uses a camera and microphone to collect still and moving images and audio data, which is then sent to a server in real time.

[0469] Input: Flight route, shooting points, type of collected data (date, time, location information, etc.)

[0470] Data processing and calculation: camera image and video capture, audio data recording

[0471] Output: Real-time data (video and audio files)

[0472] Step 3:

[0473] The server uses AI to analyze the received real-time data, and based on the video and audio data sent from the drone, it grasps the situation and identifies areas where land routes are blocked and dangerous areas.

[0474] Input: Real-time data (video and audio files)

[0475] Data processing and calculation: Image analysis, audio analysis, anomaly detection using generative AI models

[0476] Output: Disaster situation analysis results (land route cutoff points, dangerous areas)

[0477] Step 4:

[0478] The server calculates the optimal rescue route based on the disaster situation analysis, using an algorithm to find a route that avoids road closures and dangerous areas.

[0479] Input: Disaster situation analysis results

[0480] Data processing and calculation: Optimal route calculation using Dijkstra's Algorithm and A Search Algorithm

[0481] Output: Optimal rescue route

[0482] Step 5:

[0483] Users use a smartphone app to input their current location and emotional data, which is then sent to a server.

[0484] Input: current location information, emotional data (voice and text)

[0485] Data processing and calculation: GPS data acquisition, voice recognition, emotion analysis

[0486] Output: User position and emotion data

[0487] Step 6:

[0488] The server performs sentiment analysis based on the data received from the user, identifying the user's level of fear and anxiety and dynamically adjusting the priority of rescue operations.

[0489] Input: User location data, emotion data

[0490] Data processing and calculation: Emotion analysis using EmotionRecognizer

[0491] Output: Emotional state (e.g., fear, anxiety)

[0492] Step 7:

[0493] The system combines the emotion analysis results with the optimal rescue route and provides them to the rescue team to determine a specific rescue plan. It also issues instructions to dispatch drones loaded with supplies to deliver them to the disaster area as needed.

[0494] Input: Emotion analysis results, optimal rescue route

[0495] Data processing and calculation: Data integration, material distribution planning, optimal route selection

[0496] Output: Rescue instruction data (rescue plan, material transport instructions)

[0497] Step 8:

[0498] The drone (terminal) will load the designated supplies and transport them to the disaster area along a calculated optimal route. After arriving, the supplies will be safely dropped off.

[0499] Input: Material transport instructions, optimal route

[0500] Data processing and calculation: Loading of physical supplies, automatic flight route operation

[0501] Output: Material delivery completion notification

[0502] Example prompts

[0503] "The user's current location is Tokyo, the surrounding area is in a state of great chaos, and they have entered the emotion message 'It's very scary; people around me are crying.' Please display the optimal evacuation route and their emotional state."

[0504] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0505] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0506] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0507] [Second embodiment]

[0508] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0509] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0510] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0511] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0512] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0513] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0514] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0515] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0516] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0517] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0518] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0519] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0520] The present invention is a system for realizing a rapid and accurate initial response in the event of a disaster, and includes an operable flying device (hereinafter referred to as a drone), an operable photographing means, an artificial intelligence processing means for analyzing and analysing generated data in real time, a processing means for calculating the optimal rescue route, a means for loading and transporting supplies, a means for placing onboard equipment in areas where communications are cut off and collecting data on the local situation, a means for analyzing the collected data and generating emergency response information, and a means for linking the emergency response information to external rescue organizations.

[0521] System programs and their processing

[0522] 1. Initial response

[0523] Server: Upon receiving notification of a disaster, the server dispatches a drone to the affected area and sends instructions to the drone, such as flight route, shooting points, and the type of data to be collected.

[0524] Drone (terminal): Arrives at the scene and uses a camera and microphone to collect still and moving images and audio data, which are then sent to a server in real time.

[0525] Server: Analyzes and analyzes the data received in real time to understand the situation on site.

[0526] 2. Analysis and rescue route calculation

[0527] Server: Analyzes collected data using generative AI to identify road closures, dangerous areas, and isolated regions. Then, calculates the optimal rescue route and provides it to the rescue team.

[0528] 3. Material transportation

[0529] Server: Identifies the supplies needed based on the situation in the affected area and creates a delivery plan, sending details such as delivery route, time, and quantity to the drone.

[0530] Drone (terminal): Loads designated supplies and transports them to the disaster area via the optimal route. Once there, the supplies are safely dropped off.

[0531] 4. Responses in areas where communications have been cut off

[0532] Server: In areas where communications have been cut off, the server issues instructions for the use of satellite phones and digital cameras. These devices are then loaded onto drones and sent to the scene.

[0533] Drone (terminal): Fly to designated area and hand over device to local responders, who collect data and transmit it back to the drone.

[0534] Server: Analyzes the returned data and understands the local situation.

[0535] 5. Coordination of emergency response information

[0536] Server: Based on data from the drone, collects information necessary for initial response within 72 hours and generates analysis results.

[0537] Server: Shares information requiring emergency response with nearby hospitals, local governments, and other related organizations, supporting rapid life-saving efforts.

[0538] Specific examples

[0539] For example, consider a case where an earthquake with a seismic intensity of 7 occurs and some areas of an urban area are isolated.

[0540] 1. The server receives notification of an earthquake and issues a command to dispatch a drone to the affected area. The drone is equipped with a camera and a generating AI processing means.

[0541] 2. The drone (terminal) arrives at the disaster area, activates the camera and recording device, collects data in real time, and transmits it to the server.

[0542] 3. The server analyzes the received data, identifies road closures and dangerous areas, calculates the optimal rescue route, and provides this information to the rescue team.

[0543] 4. The server issues instructions to drones capable of flight to load supplies such as medicines and food and deliver them to isolated areas. The drones (terminals) then safely deliver the supplies based on the optimal route.

[0544] 5. In areas where communications are cut off, the server will dispatch drones equipped with satellite phones and digital cameras to deliver the data to responders on the ground. The drones will then return the data collected by the responders to the server for analysis.

[0545] 6. Based on the collected information, the server extracts information that requires emergency response and immediately contacts hospitals and local governments. Medical and rescue teams then respond promptly.

[0546] In this way, by using the system of the present invention, on-site information during a disaster can be quickly and accurately grasped, enabling efficient transportation of supplies and rapid initial response.

[0547] The processing flow will be explained below.

[0548] Step 1:

[0549] The server receives notifications of disaster occurrences and obtains map data of the affected area, weather information, evacuation shelter information, etc. Based on this information, it determines the flight route, shooting points, and type of data to be collected, and sends instructions to the drone.

[0550] Step 2:

[0551] The drone (terminal) follows instructions from the server and flies to the disaster area. After arriving at the scene, it activates its camera and audio recording device to collect still images and audio data. The collected data is then sent to the server in real time.

[0552] Step 3:

[0553] The server receives real-time data sent from the drone and begins analysis using the AI ​​generation system. From the analysis results, it identifies areas where land routes are cut off, dangerous areas, and isolated areas.

[0554] Step 4:

[0555] The server calculates the optimal rescue route based on the analysis results, taking into account the geographical information and traffic conditions of the affected area, and provides the calculation results to the rescue team.

[0556] Step 5:

[0557] The server determines the type, quantity, and delivery destination of the required supplies, creates a delivery plan, and sends instructions to the drones from loading the supplies to delivery.

[0558] Step 6:

[0559] The drone (terminal) follows instructions from the server to load supplies, fly along the optimal route designated, and head to the destination. After arriving at the destination, it safely drops off the supplies.

[0560] Step 7:

[0561] The server sends instructions to drones for use in areas where communications are cut off, including satellite phone and digital camera instructions, including instructions to equip the drones with devices to gather information on the ground.

[0562] Step 8:

[0563] The drone (terminal) flies into a designated quarantined area and hands over its onboard equipment to local personnel, who use these devices to collect data and transmit it back to the drone.

[0564] Step 9:

[0565] The server receives the on-site data returned from the drone and begins analyzing it in real time, and based on the information obtained from the analysis, identifies situations requiring emergency response.

[0566] Step 10:

[0567] The server then sends the identified emergency response information to nearby medical institutions, local governments, and other relevant organizations in real time, enabling rapid life-saving efforts.

[0568] Step 11:

[0569] Based on the information provided by the server, users (local rescue teams) quickly head to the site and begin rescue operations. They also provide the latest on-site information as feedback to the server as necessary.

[0570] By implementing the above steps, the system of the present invention can quickly and efficiently realize initial responses in the event of a disaster, improving the survival rate of disaster victims and enabling the appropriate supply of supplies.

[0571] Example 1

[0572] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0573] In the event of a disaster, a fast and accurate initial response is required. However, it is extremely difficult to quickly grasp the situation in the affected area, calculate the optimal rescue route, and transport the necessary supplies. Furthermore, in areas where communications are cut off, it is difficult to gather information, which can delay emergency responses. To address these challenges, a system is needed that can collect and analyze on-site information in real time and make quick decisions.

[0574] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0575] In this invention, the server includes means for receiving disaster notifications, an operable flight device, an operable photographing device, an artificial intelligence processing means for analyzing generated data in real time, a processing means for calculating an optimal rescue route based on the results of the analysis, means for loading supplies onto the flight device and transporting them, means for deploying equipment in a communication-blocked area and collecting on-site situation data, means for analyzing the collected data and generating emergency response information, means for linking the emergency response information to external rescue organizations, means for analyzing initial response information using a generative AI model, and means for inputting information into the generative AI model using prompt sentences. This allows for quick and accurate understanding of on-site information during a disaster, enabling efficient delivery of supplies and a rapid initial response.

[0576] "Means for receiving disaster notifications" refers to devices or systems for receiving disaster information sent from government agencies, automatic sensors, the Japan Meteorological Agency, etc.

[0577] An "operable flying device" is an unmanned aerial vehicle (drone) that can fly remotely or autonomously and perform a specific task.

[0578] "Operable imaging means" refers to devices such as cameras and microphones that are mounted on the flying device and are used to collect moving and still images and audio data.

[0579] "Artificial intelligence processing means" refers to an artificial intelligence (AI) system that analyzes and parses the generated data in real time.

[0580] The "processing means for calculating optimal rescue routes" refers to calculation algorithms or software for optimizing rescue routes within the disaster area based on collected data.

[0581] "Means for carrying and transporting supplies" refers to a system for loading supplies such as medicines and food onto a flying device and transporting them safely to a designated destination.

[0582] "Means for deploying onboard equipment in areas where communications have been cut off and for collecting on-site situation data" refers to a method for deploying devices such as satellite phones and digital cameras to gather on-site information in areas where communications infrastructure has been destroyed.

[0583] The "means for generating emergency response information" is a system for extracting important disaster information based on collected data and outputting the analysis results.

[0584] The "means for sharing emergency response information with external relief organizations" is a communication system for transmitting the generated emergency response information to relief organizations, including nearby hospitals and local governments.

[0585] A "generative AI model" is a type of artificial intelligence, a machine learning model used to analyze data and support decision-making.

[0586] "Means of inputting information into a generative AI model using prompt sentences" refers to a method of inputting specific questions or commands (prompts) into a generative AI model to obtain appropriate analysis results or output.

[0587] The present invention provides a system for achieving rapid and accurate initial response in the event of a disaster. The system includes an operable flight device, an operable imaging means, an artificial intelligence (AI) processing means for real-time data analysis, a processing means for calculating an optimal rescue route, a means for loading and transporting supplies, a means for deploying an onboard device in a communication-blocked area to collect local situation data, a means for analyzing the collected data and generating emergency response information, and a means for linking the emergency response information to an external relief organization. The system further includes a means for analyzing the initial response information using a generative AI model and prompt sentences.

[0588] Specific hardware and software descriptions

[0589] Flying Devices (Drones):

[0590] The flying device is, for example, a DJI drone. This is an unmanned aerial vehicle that can fly remotely or autonomously and carry out designated tasks. The drone is equipped with a high-resolution camera (e.g., 4K camera) and a recording device. This flying device flies to the affected area in the event of a disaster and collects on-site information.

[0591] Photography Method:

[0592] The drone-mounted imaging device includes a camera and microphone for capturing high-resolution still and video images, as well as audio data, which is transmitted in real time to a server.

[0593] Artificial Intelligence Processing Means:

[0594] The server analyzes the data using a generative AI model (e.g., GPT-4), which receives images and audio data in real time and uses them to understand the situation at the disaster site.

[0595] Rescue route calculation method:

[0596] The server calculates the optimal rescue route based on the analysis results. This calculation uses real-time mapping software (e.g., Google Maps API), the A algorithm, and the Dijkstra algorithm. The calculated route is provided to the rescue team.

[0597] Material transport:

[0598] The server identifies the necessary supplies (e.g., medicines, food, water) based on the situation in the disaster area and creates a transportation plan. The drones load the specified supplies and fly along the optimal route calculated by the server. Once they arrive at the site, they safely drop off the supplies.

[0599] Measures to be taken in areas affected by communication disruptions:

[0600] In areas where communications are cut off, the server dispatches drones equipped with satellite phones and digital cameras. The drones fly to designated areas and hand over these devices to local responders. The responders collect data and send it back to the drone, which the server then analyzes to understand the local situation.

[0601] Emergency response information generation and coordination:

[0602] The server generates information based on the collected data, requiring initial response within 72 hours. Generative AI models are used to analyze the priority and details of the information, and emergency response information is shared with nearby hospitals, local governments, and other relevant organizations. This ensures that life-saving efforts are carried out quickly and accurately.

[0603] Specific operation example

[0604] For example, consider a case where a magnitude 7 earthquake occurs and isolates part of a city. The server receives notification of the earthquake and dispatches a drone to the affected area. The drone is equipped with a camera and a generating AI processing means, and upon arriving at the site, it activates the camera and audio recording device to collect data in real time and send it to the server.

[0605] The server analyzes the received data, identifies road closures and dangerous areas, and calculates the optimal rescue route. This information is provided to the rescue team. It then issues instructions to drones capable of flying to load medicines, food, and other supplies and transport them to isolated areas. The drones then safely deliver the supplies based on the optimal route.

[0606] In areas where communications are cut off, the server will dispatch drones equipped with satellite phones and digital cameras, handing these devices over to responders on the scene, who will then collect data, which the drones will then send back to the server for analysis.

[0607] An example of a prompt is, "A magnitude 7 earthquake has occurred in an urban area, isolating some areas. Please collect the necessary data and calculate the optimal rescue route so that rescue teams can reach the affected areas in the shortest time." By inputting this prompt into a generative AI model, it is possible to obtain quick and accurate analysis results.

[0608] By using the system of the present invention, on-site information during a disaster can be quickly and accurately grasped, enabling efficient transportation of supplies and rapid initial response.

[0609] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0610] Step 1:

[0611] Receive notifications of disaster occurrences

[0612] Input: Disaster notifications from government agencies, weather agencies, or automated sensors

[0613] Processing: The server receives the disaster notification and records the information in a database. It analyzes detailed information such as the type, location, and scale of the disaster, and begins preparations for an initial response.

[0614] Output: Record of disaster information, instructions for preparation for initial response

[0615] Step 2:

[0616] Drone dispatch instructions

[0617] Input: Information on the occurrence of the disaster, geographical information of the affected area

[0618] Processing: The server calculates the most efficient flight route, shooting points, and type of data to collect based on the geographical information of the affected area. This calculation is done using real-time mapping software (e.g., Google Maps API).

[0619] Output: Instructions for the drone to fly, capture points, and collect data

[0620] Step 3:

[0621] Data collection

[0622] Input: Instructions sent from the server, local video stream, audio data

[0623] Processing: When the drone (terminal) receives the command, it begins flying toward the disaster area. After arriving at the scene, it activates the camera and microphone to collect video and audio data. The collected data is sent to the server in real time.

[0624] Output: Video and audio data sent to the server in real time

[0625] Step 4:

[0626] Data analysis

[0627] Input: Video and audio data transmitted from the drone

[0628] Processing: The server analyzes the received data. A generative AI model (e.g., GPT-4) is used for the analysis, and the situation in the disaster area is grasped through image and audio analysis. For example, the state of collapse of a building can be determined from the image.

[0629] Output: Situation report of disaster area, identification of dangerous areas

[0630] Step 5:

[0631] Rescue route calculation

[0632] Input: Situation report of disaster area, information of danger zone

[0633] Processing: The server calculates the optimal rescue route, avoiding road closures, dangerous areas, and isolated areas, using the A algorithm and Dijkstra algorithm.

[0634] Output: Optimal rescue route information for rescue teams

[0635] Step 6:

[0636] Material transportation plan

[0637] Input: Situation in the affected areas, list of necessary supplies

[0638] Processing: The server creates a transportation plan. It creates a plan to load the necessary supplies (e.g. medicines, food, water) onto the drone. It calculates details such as the delivery route, delivery time, and delivery volume, and sends instructions to the drone.

[0639] Output: Instructions for drones to deliver supplies

[0640] Step 7:

[0641] Material transportation

[0642] Input: Material transport instructions from the server

[0643] Processing: The drone (terminal) loads the designated supplies and begins flying based on the optimal route calculated by the server. Once it arrives at the disaster area, it safely drops off the supplies.

[0644] Output: Confirmation of delivery of supplies to disaster area

[0645] Step 8:

[0646] Response to communication blackout areas

[0647] Input: Information on communication-blocked areas, server instructions

[0648] Processing: The server dispatches a drone equipped with a satellite phone and digital camera. The drone (terminal) flies to the designated area and hands over the equipment to local responders. The local responders collect the data, which is then sent back by the drone.

[0649] Output: Data collection and return from field responders

[0650] Step 9:

[0651] Emergency response information generation and coordination

[0652] Input: Data from local responders, analysis data from the affected areas

[0653] Processing: The server generates the necessary emergency response information based on the collected data. It uses a generative AI model to analyze the priority and details of the information, and then shares the emergency response information with nearby hospitals, local governments, and other relevant organizations.

[0654] Output: Emergency response information, information on cooperation with each relief organization

[0655] This will enable quick and accurate understanding of on-site information during disasters, enabling efficient transportation of supplies and rapid initial response.

[0656] (Application example 1)

[0657] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0658] In today's world, there is a need for rapid and accurate responses to disasters and emergencies. However, delays in information gathering and emergency response can be problematic when it is difficult to grasp the situation or transport supplies, or in areas where communications are cut off. In the field of security services, real-time on-site monitoring and emergency response are also important, and efficient systems are needed.

[0659] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0660] In this invention, the server includes means for controlling an operable flying device to collect video and audio of the scene in real time, means for analyzing the collected data with a generative AI model to detect anomalies, means for calculating and proposing an optimal response route based on the detected anomalies, and means for controlling the operable flying device to transport necessary supplies to the scene, thereby enabling prompt and accurate situation assessment and response at the scene in the event of a disaster or security service.

[0661] An "operable flying device" is a device that has the ability to fly to a designated location, collect data such as video and audio, and transport supplies.

[0662] "Operable imaging means" means a device mounted on a flying device that has the capability to capture and record still images, video, and audio in real time.

[0663] "Artificial intelligence processing means" refers to computer systems or software that have the functionality to analyze and analyse collected data in real time.

[0664] "Processing means" refers to a system that has the functionality to calculate and generate optimal routes and action plans based on the results of analysis.

[0665] "Means for loading and transporting supplies" refers to devices or systems that have the function of loading the necessary supplies onto a flight vehicle and transporting them safely to a designated location.

[0666] "Means for deploying on-board equipment in areas where communications are cut off and collecting data on the local situation" refers to equipment that has the function of deploying equipment such as satellite phones and digital cameras in areas where communications are cut off using flying devices to grasp the local situation.

[0667] A "generative AI model" is a learning model used to analyze collected data and detect anomalies and specific patterns.

[0668] A "prompt sentence" is text data containing instructions or commands input to a generative AI model.

[0669] "Emergency response information" is information that requires a rapid response and is generated based on collected local data.

[0670] The "means for detecting anomalies" refers to an AI analysis system that analyzes collected video and audio data to detect abnormal situations or suspicious behavior.

[0671] The "means for proposing the optimal response route" is a system that calculates and proposes the most efficient route of action to security staff and rescue teams after detecting an abnormality.

[0672] In this invention, a comprehensive system is constructed to respond quickly and accurately to disasters and security services. The detailed configuration and operation of the system will be described below.

[0673] System Configuration

[0674] 1. Hardware Configuration

[0675] Flying Devices (Drones):

[0676] The flying device is equipped with a camera and microphone to collect video and audio in real time, and the drone is equipped with GPS for precise navigation to specific locations.

[0677] Camera and audio collection devices:

[0678] High-resolution cameras and sensitive microphones are used to capture detailed video and audio of the scene.

[0679] Communication equipment:

[0680] Using equipment such as satellite phones and digital cameras, it is possible to collect data and communicate even in areas where communications are cut off.

[0681] 2. Software Configuration

[0682] Artificial Intelligence Processing System:

[0683] Using a generative AI model, collected video and audio data is analyzed in real time to detect anomalies.

[0684] Route Calculation System:

[0685] It is equipped with an algorithm that calculates the optimal rescue or response route based on the analysis results.

[0686] Data Streaming System:

[0687] It provides a server that uses open source libraries (e.g., OpenCV) to stream video in real time.

[0688] Frontend application:

[0689] It provides an intuitive interface for security staff and users through smartphone apps and web applications.

[0690] Processing content

[0691] Data collection:

[0692] The drone moves to a designated location and uses a camera and microphone to collect video and audio, which is then sent to a server in real time.

[0693] Data Analysis:

[0694] The collected data is analyzed using a generative AI model on the server, and if abnormal behavior or dangerous situations are detected, an alert is sent immediately.

[0695] Route calculation and suggestions:

[0696] Based on the analysis results, the optimal response route is calculated, and this information is sent in real time to a smartphone app or web application, providing it to security staff and rescue teams.

[0697] Material transport:

[0698] If necessary, the drones will carry medicines and emergency supplies and deliver them based on the optimal route.

[0699] Specific examples

[0700] For example, if a large earthquake occurs and a certain area is isolated, the system operates as follows.

[0701] 1. Drones are quickly deployed to isolated areas and use high-resolution cameras and microphones to collect on-site information.

[0702] 2. The collected data is sent to a server in real time, where a generative AI model analyzes it to assess the extent of damage to buildings and the risk to human life.

[0703] 3. Based on the analysis results, the server calculates the optimal route for the rescue team and notifies them via a smartphone app.

[0704] 4. Drones will carry necessary medicines and food and deliver supplies to the site based on the optimal route.

[0705] 5. Users can check the situation on-site in real time through the application and respond quickly.

[0706] Prompt sentences to input to the generative AI model

[0707] Please analyze the following camera footage frames and predict an anomalous activity score:

[0708] Frame: [Camera image data]

[0709] This will enable quick and accurate situation assessment and response in the event of a disaster or security service.

[0710] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0711] Step 1:

[0712] Launching the drone and dispatching it to the site

[0713] When a user receives a notification of a disaster or emergency, they send a command to the server to dispatch a drone. The server calculates a flight route to the specified location and sends it to the drone. The drone then launches and moves to the scene along the instructed route.

[0714] Input: Disaster occurrence notification from user, coordinate information of designated location

[0715] Output: Drone flight route calculation results, drone arrival at the site

[0716] Step 2:

[0717] Data collection

[0718] The drone arrives at the scene and collects video and audio data in real time using its on-board camera and microphone, which is then sent to a server in real time.

[0719] Input: Collecting video and audio data on-site

[0720] Output: Real-time data transmission to the server

[0721] Step 3:

[0722] Data analysis

[0723] The server inputs the received video and audio data into a generative AI model for real-time analysis, which processes the data to detect abnormal behavior or dangerous situations and returns the results.

[0724] Input: Collected video and audio data

[0725] Output: Anomaly detection results (anomaly score)

[0726] Specific operation: The generative AI model analyzes video frames and outputs an abnormal behavior detection score.

[0727] Step 4:

[0728] Optimal route calculation

[0729] The server uses the analysis results to calculate the optimal route for rescue teams and security staff, and provides the calculated route information in real time to smartphone apps and web applications.

[0730] Input: Anomaly detection results, site geographic information

[0731] Output: Optimal route information

[0732] Specific operation: Combining GPS data and anomaly detection results to calculate the shortest and safest route and notify the application.

[0733] Step 5:

[0734] Material transportation

[0735] If necessary, the server sends instructions to the drone to load medicines or emergency supplies, and the drone then delivers the supplies to the site based on the optimal route.

[0736] Input: Required material information, optimal transport route information

[0737] Output: Supplies arrive at the site

[0738] Specific operation: Based on instructions from the server, the drone loads supplies and delivers them to the site.

[0739] Step 6:

[0740] Real-time monitoring and feedback

[0741] Users can check the on-site video and analysis results in real time via a smartphone app or web application and take necessary action. The server receives feedback from the user and adjusts the next action.

[0742] Input: User feedback, real-time analysis results

[0743] Output: Next action instructions

[0744] Specific operation: Based on the information provided by the user, the system responds on-site and sends feedback to the server to continuously optimize the response.

[0745] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0746] The present invention is a system for realizing a rapid and accurate initial response in the event of a disaster, and includes an operable flying device (hereinafter referred to as a drone), an operable photographing means, an artificial intelligence processing means for analyzing and analysing generated data in real time, a processing means for calculating the optimal rescue route, a means for loading and transporting supplies, a means for placing onboard equipment in areas where communications are cut off and collecting data on the local situation, a means for analyzing the collected data and generating emergency response information, and a means for linking the emergency response information to external rescue organizations, as well as an emotion engine that recognizes the user's emotions.

[0747] System programs and their processing

[0748] 1. Initial response

[0749] Server: Upon receiving notification of a disaster, it obtains map data of the affected area, weather information, evacuation shelter information, etc. Based on this information, it sends instructions to the drone, such as flight route, shooting points, and type of data to be collected.

[0750] Drone (terminal): Arrives at the scene and uses a camera and microphone to collect still and moving images and audio data, which are then sent to a server in real time.

[0751] Server: Analyzes and analyzes the data received in real time to understand the situation on site.

[0752] 2. Analysis and rescue route calculation

[0753] Server: Analyzes collected data using generative AI to identify road closures, dangerous areas, and isolated regions. Then, calculates the optimal rescue route and provides it to the rescue team.

[0754] 3. Material transportation

[0755] Server: Identifies the supplies needed based on the situation in the affected area and creates a delivery plan, sending details such as delivery route, time, and quantity to the drone.

[0756] Drone (terminal): Loads designated supplies and transports them to the disaster area via the optimal route. Once there, the supplies are safely dropped off.

[0757] 4. Responses in areas where communications have been cut off

[0758] Server: In areas where communications have been cut off, the server issues instructions for the use of satellite phones and digital cameras. These devices are then loaded onto drones and sent to the scene.

[0759] Drone (terminal): Fly to designated area and hand over device to local responders, who collect data and transmit it back to the drone.

[0760] Server: Analyzes the returned data and understands the local situation.

[0761] 5. Coordination of emergency response information

[0762] Server: Based on data from the drone, collects information necessary for an initial response within 72 hours and generates analysis results.

[0763] Server: Shares information requiring emergency response with nearby hospitals, local governments, and other related organizations, enabling rapid life-saving efforts.

[0764] Incorporating an emotion engine

[0765] 1. Collecting Emotional Data

[0766] Drone (terminal): Data collected in the disaster area includes audio and video footage of the victims. This data is sent to the emotion engine.

[0767] Server: The acquired audio and video data is passed to the emotion engine, which analyzes the emotional state of the victim.

[0768] 2. Sentiment analysis and prioritization of rescue operations

[0769] Emotion engine (on the server): Analyzes the emotional state of victims (e.g., fear, anxiety, relief) and reflects that information in planning rescue operations.

[0770] Server: Dynamically change the priority of rescue efforts based on the results of emotion analysis. For example, prioritize assistance to areas with many disaster victims who are feeling extreme fear or anxiety.

[0771] Specific examples

[0772] For example, consider a case where an earthquake with a seismic intensity of 7 occurs and some areas of an urban area are isolated.

[0773] 1. The server receives notification of a disaster and issues instructions to dispatch drones to the affected area. The drones are equipped with a camera, a generation AI processing means, and an emotion engine.

[0774] 2. The drone (terminal) arrives at the disaster area, activates its camera and recording device to collect data in real time, and transmits audio and video data to the emotion engine.

[0775] 3. The server analyzes the received data, identifies road closures and dangerous areas, calculates the optimal rescue route, and provides this information to the rescue team.

[0776] 4. The server issues instructions to drones capable of flight to load supplies such as medicines and food and deliver them to isolated areas. The drones (terminals) then safely deliver the supplies based on the optimal route.

[0777] 5. In areas where communications are cut off, the server will dispatch drones equipped with satellite phones and digital cameras to deliver the data to responders on the ground. The drones will then return the data collected by the responders to the server for analysis.

[0778] 6. Based on the collected information, the server extracts information that requires emergency response and immediately contacts hospitals and local governments. Medical and rescue teams then respond promptly.

[0779] 7. The emotion engine analyzes the voice and video data of victims to understand their emotional state, allowing the server to appropriately adjust the priority of rescue efforts and provide the necessary assistance quickly.

[0780] In this way, by using the system of the present invention, on-site information can be quickly and accurately grasped during a disaster, enabling efficient transportation of supplies and a rapid initial response. In addition, by incorporating an emotion engine, appropriate support can be provided taking into account the emotional state of the victims.

[0781] The processing flow will be explained below.

[0782] Step 1:

[0783] The server receives notifications of disaster occurrences and obtains map data of the affected area, weather information, evacuation shelter information, etc. Based on this information, a program is prepared to instruct the drone on flight routes, shooting points, and the type of data to be collected.

[0784] Step 2:

[0785] The server sends flight commands to the drone, instructing it to head towards the designated area.

[0786] Step 3:

[0787] The drone (terminal) flies to the disaster site according to instructions from the server, and upon arrival, activates its camera and recording device, collecting still and moving images and audio data, which it then transmits to the server in real time.

[0788] Step 4:

[0789] The server receives real-time data transmitted by the drone and begins analyzing it using generative AI, which identifies road disruptions, dangerous areas, and isolated regions.

[0790] Step 5:

[0791] Based on the analysis results, the server calculates the optimal rescue route to the disaster area, and provides the calculation results to the rescue team, enabling them to carry out rescue operations quickly.

[0792] Step 6:

[0793] The server determines the type and amount of supplies needed based on the situation in the disaster area, creates a supply delivery plan, including delivery routes, times, and quantities, and transmits this plan to the drone.

[0794] Step 7:

[0795] The drone (terminal) follows instructions from the server to load supplies, fly to the disaster area along the optimal route, and safely drop off the supplies upon arrival.

[0796] Step 8:

[0797] The server sends instructions to the drones to deploy satellite phones and digital cameras in areas where communications have been cut off, including specific operational procedures for gathering information on the situation on the ground.

[0798] Step 9:

[0799] The drone (terminal) flies into a designated blackout area and hands over a satellite phone and digital camera to local personnel, who use these devices to collect data and transmit it back to the drone.

[0800] Step 10:

[0801] The server receives the local situation data returned by the drone and analyzes it in real time using generative AI, extracting information necessary for emergency response.

[0802] Step 11:

[0803] The server then sends the identified emergency response information to nearby medical institutions, local governments, and related organizations in real time, enabling rapid life-saving efforts.

[0804] Step 12:

[0805] The drone (terminal) collects audio and video data from the victim and sends it to the emotion engine, which analyzes the audio and video to identify the victim's emotional state.

[0806] Step 13:

[0807] The emotion engine (in the server) analyzes the emotional state of the victims (e.g., fear, anxiety, relief) and incorporates this information into the rescue operation plan in the server.

[0808] Step 14:

[0809] The server executes a means for dynamically changing the priority of relief efforts based on the emotion information provided by the emotion engine, thereby giving priority to providing support to areas with a large number of disaster victims who are extremely emotionally unstable.

[0810] Step 15:

[0811] The users (local rescue teams) quickly carry out rescue operations on site based on real-time information and priority adjustments provided by the server. Feedback during operations is also provided to the server to help with future decisions.

[0812] Through these steps, the system of the present invention can quickly and efficiently realize initial responses in the event of a disaster. This system improves the survival rate of disaster victims and enables the appropriate supply of supplies. Furthermore, by incorporating an emotion engine, it becomes possible to provide appropriate support that takes into account the emotional state of disaster victims.

[0813] Example 2

[0814] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0815] While rapid and accurate initial response is crucial in the event of a disaster, conventional systems often delay the collection of information needed to properly grasp the situation on the ground, and rescue operations are often carried out based on inaccurate data. Furthermore, support does not take into account the emotional state of the victims, resulting in insufficient psychological care and making it difficult to properly prioritize rescue operations. To solve these issues, a system is needed that can collect and analyze on-site information in real time, calculate optimal rescue routes, and dynamically adjust the priority of rescue operations by recognizing the emotional state of the victims.

[0816] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an operable flying device, an operable photographing means, an artificial intelligence processing means for analyzing the generated data in real time, a processing means for calculating an optimal rescue route based on the analysis results, a means for loading supplies onto the flying device and transporting them, a means for deploying an onboard device in a communication-blocked area and collecting local situation data, a means for analyzing the collected data and generating emergency response information, a means for linking the emergency response information to external rescue organizations, an emotion analysis means for analyzing audio and video data of victims and recognizing their emotional states, and a means for dynamically adjusting the priority of rescue operations based on their emotional states. This enables rapid and accurate understanding of the situation at the scene of a disaster, enabling efficient rescue operations and delivery of supplies, and realizing appropriate support that takes into account the psychological care of victims.

[0817] An "operable flying device" is a device capable of airborne flight and reaching designated locations to collect data or deliver supplies.

[0818] "Operable imaging means" means a camera or other imaging device capable of capturing still or video images and attached to a flying device for data collection purposes.

[0819] "Artificial intelligence processing means for analyzing and analysing generated data in real time" refers to an artificial intelligence-based processing system used to instantly analyze and analyse collected data and grasp the situation on site.

[0820] "Processing means for calculating optimal rescue route" refers to a processing system for calculating the most efficient route for carrying out rescue operations based on the data analyzed in real time.

[0821] "Means for loading and transporting supplies onto a flying vehicle" means the mechanism or structure for loading relief supplies onto a flying vehicle and transporting them to a designated disaster area.

[0822] "Means for deploying on-board equipment in areas where communications are cut off and collecting on-site situation data" refers to methods and devices for deploying the necessary equipment and collecting on-site situation data so that data collection is possible even in areas where communications infrastructure has been destroyed.

[0823] "Means for analyzing collected data and generating emergency response information" refers to a processing system that analyzes collected data, identifies situations requiring emergency response, and generates response information.

[0824] The "means for sharing emergency response information with external relief organizations" refers to a communication means for quickly and accurately transmitting the generated emergency response information to external relief organizations.

[0825] An "emotion analysis means" is an analysis system that analyzes the audio and video data of victims to recognize their emotional state (e.g., fear, anxiety, relief).

[0826] "Means for dynamically adjusting the priority of rescue operations based on emotional state" refers to a processing system for changing the priority of rescue operations in real time based on the emotional state of the victim recognized by the emotion analysis means.

[0827] The present invention provides a system for achieving rapid and accurate initial response in the event of a disaster. The system includes an operable flight device, an operable imaging means, an artificial intelligence processing means for analyzing and interpreting generated data in real time, a processing means for calculating an optimal rescue route, a means for loading and transporting supplies, a means for deploying an onboard device in a communication-blocked area and collecting local situation data, a means for analyzing the collected data and generating emergency response information, a means for linking the emergency response information to an external rescue organization, an emotion analysis means for recognizing the emotions of disaster victims, and a means for dynamically adjusting the priority of rescue operations based on the emotional state.

[0828] When the server receives notification of a disaster, it obtains map data, weather information, and evacuation shelter information for the affected area. Based on this information, it determines the flight route, shooting points, type of data to collect, and issues instructions to the drone. The drone (terminal) arrives at the scene and uses its camera and microphone to collect video and still images and audio data, which it then sends to the server in real time. The server analyzes the data received in real time and uses a generative AI model to understand the situation. Machine learning libraries such as TensorFlow and PyTorch are used for this analysis.

[0829] The server runs an algorithm (such as the A algorithm or Dijkstra algorithm) to calculate the optimal rescue route based on the analyzed data. This identifies areas where roads are cut off, dangerous areas, and isolated regions, and provides this information to rescue teams. At the same time, it identifies necessary supplies (food, medicine, etc.) and creates a delivery plan. It issues instructions to drones capable of flying to load the supplies and deliver them to the disaster area via the optimal route.

[0830] In areas where communications are cut off, the server dispatches drones equipped with satellite phones and digital cameras and hands them over to local responders. The data collected by the local responders is then sent back to the server by drone. The server then analyzes this data to accurately grasp the local situation.

[0831] Emergency response information is compiled based on data from drones, generating the information required for initial response within 72 hours. The server then shares this information with hospitals and local governments to support rapid life-saving efforts. Furthermore, audio and video data collected in the disaster area is sent to an emotion analysis system, which analyzes the emotional state of the victims (e.g., fear, anxiety, relief) and incorporates this information into rescue operation plans. Based on this information, the server dynamically adjusts the priority of rescue operations.

[0832] As a concrete example, consider a case where a magnitude 7 earthquake occurs and isolates part of an urban area. The server receives a disaster notification and immediately dispatches a drone to collect information on the local situation. The collected data is analyzed to determine the optimal rescue route and a plan for transporting supplies. At the same time, the emotional state of the victims is analyzed and optimal support activities are carried out, taking into consideration their mental health.

[0833] Below is an example of a prompt sentence:

[0834] "A magnitude 7 earthquake has occurred, isolating parts of urban areas. Dispatch drones to quickly collect on-site information from the affected areas, calculate rescue routes, and develop an optimal plan for delivering necessary supplies. Also, analyze the emotional state of the victims using audio and video data to appropriately prioritize rescue efforts."

[0835] The above is an embodiment of the present invention, and by using this system, on-site information can be quickly and accurately grasped during a disaster, enabling efficient rescue operations and the transportation of supplies. In addition, by using emotion analysis means, appropriate support can be provided taking into account the emotional state of the victims.

[0836] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0837] Step 1: Receive notifications and prepare

[0838] The server receives notifications of disaster occurrences, which are provided in real time by meteorological agencies and monitoring systems.

[0839] Input: Disaster notification (e.g. earthquake, flood)

[0840] The server obtains map data, weather information, and evacuation shelter information for the affected area, and integrates this information to determine flight routes, photography points, and the type of data to be collected.

[0841] Data processing: Map data, weather information, and evacuation shelter information are obtained using APIs, and this is integrated to generate drone instruction information.

[0842] Output: Flight route, shooting points, type of collected data

[0843] Specific operation: The server retrieves map data using the Google Maps API, weather information from the Japan Meteorological Agency API, and evacuation shelter information from government databases.

[0844] Step 2: Dispatch the drone and collect data

[0845] The server transmits the predetermined flight route and collection items to the drone's command center.

[0846] Input: Flight route, shooting points, type of collected data

[0847] The drone (terminal) follows a designated flight route, uses a camera and microphone to collect video and audio data, and transmits it to a server in real time.

[0848] Data calculation: Using GPS, the drone flies autonomously and collects data at designated points.

[0849] Output: Real-time transmission of collected video and audio data

[0850] Specific operation: The drone flies using its built-in GPS and communication module, takes pictures with its camera, records sounds with its microphone, and uploads the collected data to a server in real time.

[0851] Step 3: Data analysis and situation assessment

[0852] The server analyzes the received video and audio data using a generative AI model.

[0853] Input: Collected video and audio data

[0854] The server uses TensorFlow or PyTorch to analyze the image data and identify the situation at the scene (e.g., collapsed buildings, flooded areas), and converts the audio data into text using the Google Cloud Speech-to-Text API and passes it to the sentiment analysis tool.

[0855] Data processing: image data analysis, voice data text conversion

[0856] Output: Analysis results (collapsed buildings, flooded areas), audio data converted to text

[0857] Specific operation: The server analyzes image data using an object recognition algorithm, and audio data is converted into text and used for emotion analysis.

[0858] Step 4: Calculate the rescue route

[0859] The server calculates the optimal rescue route based on the analysis results.

[0860] Input: Analysis results (collapsed buildings, flooded areas)

[0861] The server calculates the optimal route using the A algorithm or Dijkstra algorithm and provides it to the rescue team.

[0862] Data calculation: Calculating the optimal rescue route

[0863] Output: Optimal rescue route

[0864] Specific operation: The server calculates the shortest and safest rescue route taking into account road conditions and obstacles, and notifies the rescue team.

[0865] Step 5: Prepare and execute the shipment

[0866] The server identifies the necessary supplies and issues instructions to load them onto the drone.

[0867] Input: Situation in the affected area, list of necessary supplies

[0868] The drone (terminal) will be loaded with designated supplies and delivered to the disaster area.

[0869] Data processing: material loading plans, transport route instructions

[0870] Output: Material transportation plan, transportation route

[0871] Specific operation: The drone loads the specified supplies and safely delivers them along the programmed route.

[0872] Step 6: Response to communication disruptions in affected areas

[0873] The server will dispatch drones equipped with satellite phones and digital cameras to areas where communications have been cut off.

[0874] Input: Information on communication-blocked areas, list of required equipment

[0875] The drone (terminal) drops off the equipment on-site and transmits the collected data back to the server.

[0876] Data processing: Equipment layout plan, return of collected data

[0877] Output: Local situation data

[0878] Specific operation: The drone hands over the equipment to the local responder and sends the collected data back to the server.

[0879] Step 7: Generate and coordinate emergency response information

[0880] The server generates emergency response information based on data from the drone.

[0881] Input: collected data, analysis results

[0882] The server aggregates the data and generates information necessary for initial response within 72 hours, which is then shared with hospitals and local governments.

[0883] Data processing: Emergency response information generation and notification

[0884] Output: Emergency response information

[0885] Specific operation: The server uses APIs and SMS gateways to directly communicate the generated emergency information to each agency.

[0886] Step 8: Sentiment analysis and prioritization

[0887] The server transmits audio and video data collected in the disaster area to the emotion analysis means.

[0888] Input: Text-encoded audio data, video data

[0889] The emotion analysis means recognizes the emotional state of the victim and sends that information back to the server.

[0890] Data Computing: Emotional State Analysis

[0891] Output: Emotion analysis results

[0892] The server dynamically adjusts the priority of rescue operations based on the results of emotion analysis.

[0893] Data processing: Dynamic adjustment of priorities

[0894] Output: Dynamically adjusted rescue operation priorities

[0895] How it works: The server uses a generative AI model to perform emotion analysis and re-prioritize based on the emotional state.

[0896] (Application example 2)

[0897] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0898] During disasters, prompt and accurate initial responses are required, but appropriate rescue efforts are often hindered due to the uncertainty of the situation on the ground. Furthermore, in areas where communications are cut off, it is difficult to gather information, making it impossible to properly prioritize relief efforts. Furthermore, prioritization of relief efforts does not take into account the emotional state of the victims, which is a factor in delaying appropriate assistance to victims.

[0899] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0900] In this invention, the server includes an operable flying device, an operable photographing means, an artificial intelligence processing means for analyzing generated data in real time, a processing means for calculating an optimal rescue route based on the results of the analysis, a means for loading supplies onto the flying device and transporting them, a means for deploying an onboard device in a communication-blocked area and collecting on-site situation data, a means for analyzing the collected data and generating emergency response information, a means for linking the emergency response information to external rescue organizations, a means for collecting user emotion data on an operating terminal and performing emotion analysis, and a means for adjusting the priority of rescue operations based on the results of the emotion analysis, thereby enabling quick and accurate understanding of on-site information in disaster-stricken areas, efficient delivery of supplies, and appropriate prioritization of rescue operations.

[0901] An "operable flying device" is a flying machine that can be remotely controlled or autonomously operated, such as a drone, and is used to collect information and transport supplies during disasters.

[0902] "Operable imaging means" means a camera or other imaging device used to capture still or video images of a scene and collect information in real time.

[0903] "Artificial intelligence processing means for analyzing and analysing generated data in real time" refers to artificial intelligence algorithms and systems that rapidly process collected data and extract and analyse necessary information.

[0904] The "processing means for calculating the optimal rescue route" refers to an algorithm or software for calculating the safest and most efficient rescue route in the event of a disaster.

[0905] "Means for loading and transporting supplies" refers to the equipment or mechanism for loading relief supplies and other items onto a flying device and transporting them to their destination.

[0906] "Means for deploying on-board equipment in areas where communications are disrupted and collecting local situation data" refers to methods and systems for deploying machines or devices in areas where communications infrastructure has been destroyed and collecting the necessary information.

[0907] "Means for generating emergency response information" refers to algorithms or systems that analyze collected data and generate information required for rapid response.

[0908] "Means for linking emergency response information to external relief agencies" refers to communication means or systems for quickly transmitting the generated emergency response information to relevant relief agencies or organizations.

[0909] "Means for collecting user emotional data on operating devices and performing emotional analysis" refers to technologies and systems for collecting audio and video data from devices used by disaster victims and analyzing their emotional states.

[0910] The "means for adjusting the priority of rescue operations based on the results of sentiment analysis" refers to an algorithm or system for dynamically changing the priority of rescue operations based on the results obtained from sentiment analysis.

[0911] In order to put this invention into practice, it is important to build a system that can collect information in real time during a disaster and provide a rapid and accurate initial response.

[0912] System Configuration

[0913] 1. Operable flying device

[0914] These devices, such as drones, are capable of remote control and autonomous operation, and are used to gather information and transport supplies at disaster sites.

[0915] 2. Operable photography means

[0916] The drone is equipped with a high-resolution camera that can take still images and videos of the disaster area and transmit them in real time.

[0917] 3. Artificial intelligence processing means for analyzing and analysing the generated data in real time

[0918] The server is equipped with a generative AI model that quickly analyzes and parses the data sent from the drone.

[0919] Example) AI model used: TensorFlow, PyTorch

[0920] 4. Processing means for calculating the optimal rescue route

[0921] Based on the analysis results, the server uses an algorithm to calculate the optimal rescue route and provides it to rescue agencies.

[0922] Example) Algorithms used: Dijkstra's Algorithm, A Search Algorithm

[0923] 5. Means for carrying and transporting supplies

[0924] The drones are equipped with a mechanism to load designated supplies and transport them quickly along a calculated, optimal route.

[0925] 6. Means of deploying onboard equipment in communication-blocked areas to gather on-site situation data.

[0926] The drones will deploy satellite-enabled equipment in areas where communications infrastructure has been destroyed and collect situational data.

[0927] 7. Means for generating emergency response information

[0928] The server analyzes the collected data and generates information needed for rapid response, including weather information, evacuation shelter information, and more.

[0929] 8. Means for communicating emergency response information to external relief agencies

[0930] The generated emergency response information is automatically sent to relevant relief agencies, enabling a rapid response.

[0931] 9. Means for collecting user emotion data and performing emotion analysis on the operating device

[0932] It collects audio and video data from devices such as drones and smartphones used by disaster victims, and analyzes the user's emotional state.

[0933] Example) Emotion analysis technology used: EmotionRecognizer

[0934] 10. A method for adjusting rescue operation priorities based on the results of sentiment analysis

[0935] Based on the results of emotion analysis, the server dynamically adjusts the priorities of rescue efforts, prioritizing support for areas with a high number of victims experiencing extreme fear and anxiety.

[0936] Specific examples of processing

[0937] For example, consider a case where a magnitude 7 earthquake occurs in an urban area, causing communication to be unavailable in some areas.

[0938] 1. The server receives information about the occurrence of a disaster and issues instructions to the drone to dispatch to the site.

[0939] 2. After arriving at the site, the drone uses a camera to collect data and transmits it to a server in real time.

[0940] 3. The server analyzes the received data using a generative AI model to identify road disruptions and dangerous areas.

[0941] 4. When the user reports their situation and emotional state using a smartphone app, the server uses that data to calculate the optimal rescue route.

[0942] Example prompt sentence:

[0943] "The user's current location is Tokyo, the surrounding area is in a state of great chaos, and they have entered the emotion message 'It's very scary; people around me are crying.' Please display the optimal evacuation route and their emotional state."

[0944] By building such a system, it will be possible to quickly and accurately grasp the situation in disaster-stricken areas, enabling efficient and appropriate relief efforts. Furthermore, prioritizing relief efforts based on emotion analysis will enable appropriate support that takes into account the emotional state of the victims.

[0945] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0946] Step 1:

[0947] The server receives information about the occurrence of a disaster. At the same time, it also acquires map data of the affected area, weather information, and evacuation shelter information. Based on this information, it sends instructions to the drone, such as flight route, shooting points, and the type of data to be collected.

[0948] Input: Disaster occurrence notification, map data, weather information, evacuation shelter information

[0949] Data processing and calculation: Data integration, calculation using flight route optimization algorithm

[0950] Output: Instruction data for the drone (flight route, shooting points, type of collected data)

[0951] Step 2:

[0952] The drone (terminal) arrives at the scene and uses a camera and microphone to collect still and moving images and audio data, which is then sent to a server in real time.

[0953] Input: Flight route, shooting points, type of collected data (date, time, location information, etc.)

[0954] Data processing and calculation: camera image and video capture, audio data recording

[0955] Output: Real-time data (video and audio files)

[0956] Step 3:

[0957] The server uses AI to analyze the received real-time data, and based on the video and audio data sent from the drone, it grasps the situation and identifies areas where land routes are blocked and dangerous areas.

[0958] Input: Real-time data (video and audio files)

[0959] Data processing and calculation: Image analysis, audio analysis, anomaly detection using generative AI models

[0960] Output: Disaster situation analysis results (land route cutoff points, dangerous areas)

[0961] Step 4:

[0962] The server calculates the optimal rescue route based on the disaster situation analysis, using an algorithm to find a route that avoids road closures and dangerous areas.

[0963] Input: Disaster situation analysis results

[0964] Data processing and calculation: Optimal route calculation using Dijkstra's Algorithm and A Search Algorithm

[0965] Output: Optimal rescue route

[0966] Step 5:

[0967] Users use a smartphone app to input their current location and emotional data, which is then sent to a server.

[0968] Input: current location information, emotional data (voice and text)

[0969] Data processing and calculation: GPS data acquisition, voice recognition, emotion analysis

[0970] Output: User position and emotion data

[0971] Step 6:

[0972] The server performs sentiment analysis based on the data received from the user, identifying the user's level of fear and anxiety and dynamically adjusting the priority of rescue operations.

[0973] Input: User location data, emotion data

[0974] Data processing and calculation: Emotion analysis using EmotionRecognizer

[0975] Output: Emotional state (e.g., fear, anxiety)

[0976] Step 7:

[0977] The system combines the emotion analysis results with the optimal rescue route and provides them to the rescue team to determine a specific rescue plan. It also issues instructions to dispatch drones loaded with supplies to deliver them to the disaster area as needed.

[0978] Input: Emotion analysis results, optimal rescue route

[0979] Data processing and calculation: Data integration, material distribution planning, optimal route selection

[0980] Output: Rescue instruction data (rescue plan, material transport instructions)

[0981] Step 8:

[0982] The drone (terminal) will load the designated supplies and transport them to the disaster area along a calculated optimal route. After arriving, the supplies will be safely dropped off.

[0983] Input: Material transport instructions, optimal route

[0984] Data processing and calculation: Loading of physical supplies, automatic flight route operation

[0985] Output: Material delivery completion notification

[0986] Example prompts

[0987] "The user's current location is Tokyo, the surrounding area is in a state of great chaos, and they have entered the emotion message 'It's very scary; people around me are crying.' Please display the optimal evacuation route and their emotional state."

[0988] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0989] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0990] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0991] [Third embodiment]

[0992] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0993] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0994] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0995] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0996] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0997] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0998] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0999] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1000] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1001] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1002] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1003] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1004] The present invention is a system for realizing a rapid and accurate initial response in the event of a disaster, and includes an operable flying device (hereinafter referred to as a drone), an operable photographing means, an artificial intelligence processing means for analyzing and analysing generated data in real time, a processing means for calculating the optimal rescue route, a means for loading and transporting supplies, a means for placing onboard equipment in areas where communications are cut off and collecting data on the local situation, a means for analyzing the collected data and generating emergency response information, and a means for linking the emergency response information to external rescue organizations.

[1005] System programs and their processing

[1006] 1. Initial response

[1007] Server: Upon receiving notification of a disaster, the server dispatches a drone to the affected area and sends instructions to the drone, such as flight route, shooting points, and the type of data to be collected.

[1008] Drone (terminal): Arrives at the scene and uses a camera and microphone to collect still and moving images and audio data, which are then sent to a server in real time.

[1009] Server: Analyzes and analyzes the data received in real time to understand the situation on site.

[1010] 2. Analysis and rescue route calculation

[1011] Server: Analyzes collected data using generative AI to identify road closures, dangerous areas, and isolated regions. Then, calculates the optimal rescue route and provides it to the rescue team.

[1012] 3. Material transportation

[1013] Server: Identifies the supplies needed based on the situation in the affected area and creates a delivery plan, sending details such as delivery route, time, and quantity to the drone.

[1014] Drone (terminal): Loads designated supplies and transports them to the disaster area via the optimal route. Once there, the supplies are safely dropped off.

[1015] 4. Responses in areas where communications have been cut off

[1016] Server: In areas where communications have been cut off, the server issues instructions for the use of satellite phones and digital cameras. These devices are then loaded onto drones and sent to the scene.

[1017] Drone (terminal): Fly to designated area and hand over device to local responders, who collect data and transmit it back to the drone.

[1018] Server: Analyzes the returned data and understands the local situation.

[1019] 5. Coordination of emergency response information

[1020] Server: Based on data from the drone, collects information necessary for initial response within 72 hours and generates analysis results.

[1021] Server: Shares information requiring emergency response with nearby hospitals, local governments, and other related organizations, supporting rapid life-saving efforts.

[1022] Specific examples

[1023] For example, consider a case where an earthquake with a seismic intensity of 7 occurs and some areas of an urban area are isolated.

[1024] 1. The server receives notification of an earthquake and issues a command to dispatch a drone to the affected area. The drone is equipped with a camera and a generating AI processing means.

[1025] 2. The drone (terminal) arrives at the disaster area, activates the camera and recording device, collects data in real time, and transmits it to the server.

[1026] 3. The server analyzes the received data, identifies road closures and dangerous areas, calculates the optimal rescue route, and provides this information to the rescue team.

[1027] 4. The server issues instructions to drones capable of flight to load supplies such as medicines and food and deliver them to isolated areas. The drones (terminals) then safely deliver the supplies based on the optimal route.

[1028] 5. In areas where communications are cut off, the server will dispatch drones equipped with satellite phones and digital cameras to deliver the data to responders on the ground. The drones will then return the data collected by the responders to the server for analysis.

[1029] 6. Based on the collected information, the server extracts information that requires emergency response and immediately contacts hospitals and local governments. Medical and rescue teams then respond promptly.

[1030] In this way, by using the system of the present invention, on-site information during a disaster can be quickly and accurately grasped, enabling efficient transportation of supplies and rapid initial response.

[1031] The processing flow will be explained below.

[1032] Step 1:

[1033] The server receives notifications of disaster occurrences and obtains map data of the affected area, weather information, evacuation shelter information, etc. Based on this information, it determines the flight route, shooting points, and type of data to be collected, and sends instructions to the drone.

[1034] Step 2:

[1035] The drone (terminal) follows instructions from the server and flies to the disaster area. After arriving at the scene, it activates its camera and audio recording device to collect still images and audio data. The collected data is then sent to the server in real time.

[1036] Step 3:

[1037] The server receives real-time data sent from the drone and begins analysis using the AI ​​generation system. From the analysis results, it identifies areas where land routes are cut off, dangerous areas, and isolated areas.

[1038] Step 4:

[1039] The server calculates the optimal rescue route based on the analysis results, taking into account the geographical information and traffic conditions of the affected area, and provides the calculation results to the rescue team.

[1040] Step 5:

[1041] The server determines the type, quantity, and delivery destination of the required supplies, creates a delivery plan, and sends instructions to the drones from loading the supplies to delivery.

[1042] Step 6:

[1043] The drone (terminal) follows instructions from the server to load supplies, fly along the optimal route designated, and head to the destination. After arriving at the destination, it safely drops off the supplies.

[1044] Step 7:

[1045] The server sends instructions to drones for use in areas where communications are cut off, including satellite phone and digital camera instructions, including instructions to equip the drones with devices to gather information on the ground.

[1046] Step 8:

[1047] The drone (terminal) flies into a designated quarantined area and hands over its onboard equipment to local personnel, who use these devices to collect data and transmit it back to the drone.

[1048] Step 9:

[1049] The server receives the on-site data returned from the drone and begins analyzing it in real time, and based on the information obtained from the analysis, identifies situations requiring emergency response.

[1050] Step 10:

[1051] The server then sends the identified emergency response information to nearby medical institutions, local governments, and other relevant organizations in real time, enabling rapid life-saving efforts.

[1052] Step 11:

[1053] Based on the information provided by the server, users (local rescue teams) quickly head to the site and begin rescue operations. They also provide the latest on-site information as feedback to the server as necessary.

[1054] By implementing the above steps, the system of the present invention can quickly and efficiently realize initial responses in the event of a disaster, improving the survival rate of disaster victims and enabling the appropriate supply of supplies.

[1055] Example 1

[1056] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1057] In the event of a disaster, a fast and accurate initial response is required. However, it is extremely difficult to quickly grasp the situation in the affected area, calculate the optimal rescue route, and transport the necessary supplies. Furthermore, in areas where communications are cut off, it is difficult to gather information, which can delay emergency responses. To address these challenges, a system is needed that can collect and analyze on-site information in real time and make quick decisions.

[1058] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1059] In this invention, the server includes means for receiving disaster notifications, an operable flight device, an operable photographing device, an artificial intelligence processing means for analyzing generated data in real time, a processing means for calculating an optimal rescue route based on the results of the analysis, means for loading supplies onto the flight device and transporting them, means for deploying equipment in a communication-blocked area and collecting on-site situation data, means for analyzing the collected data and generating emergency response information, means for linking the emergency response information to external rescue organizations, means for analyzing initial response information using a generative AI model, and means for inputting information into the generative AI model using prompt sentences. This allows for quick and accurate understanding of on-site information during a disaster, enabling efficient delivery of supplies and a rapid initial response.

[1060] "Means for receiving disaster notifications" refers to devices or systems for receiving disaster information sent from government agencies, automatic sensors, the Japan Meteorological Agency, etc.

[1061] An "operable flying device" is an unmanned aerial vehicle (drone) that can fly remotely or autonomously and perform a specific task.

[1062] "Operable imaging means" refers to devices such as cameras and microphones that are mounted on the flying device and are used to collect moving and still images and audio data.

[1063] "Artificial intelligence processing means" refers to an artificial intelligence (AI) system that analyzes and parses the generated data in real time.

[1064] The "processing means for calculating optimal rescue routes" refers to calculation algorithms or software for optimizing rescue routes within the disaster area based on collected data.

[1065] "Means for carrying and transporting supplies" refers to a system for loading supplies such as medicines and food onto a flying device and transporting them safely to a designated destination.

[1066] "Means for deploying onboard equipment in areas where communications have been cut off and for collecting on-site situation data" refers to a method for deploying devices such as satellite phones and digital cameras to gather on-site information in areas where communications infrastructure has been destroyed.

[1067] The "means for generating emergency response information" is a system for extracting important disaster information based on collected data and outputting the analysis results.

[1068] The "means for sharing emergency response information with external relief organizations" is a communication system for transmitting the generated emergency response information to relief organizations, including nearby hospitals and local governments.

[1069] A "generative AI model" is a type of artificial intelligence, a machine learning model used to analyze data and support decision-making.

[1070] "Means of inputting information into a generative AI model using prompt sentences" refers to a method of inputting specific questions or commands (prompts) into a generative AI model to obtain appropriate analysis results or output.

[1071] The present invention provides a system for achieving rapid and accurate initial response in the event of a disaster. The system includes an operable flight device, an operable imaging means, an artificial intelligence (AI) processing means for real-time data analysis, a processing means for calculating an optimal rescue route, a means for loading and transporting supplies, a means for deploying an onboard device in a communication-blocked area to collect local situation data, a means for analyzing the collected data and generating emergency response information, and a means for linking the emergency response information to an external relief organization. The system further includes a means for analyzing the initial response information using a generative AI model and prompt sentences.

[1072] Specific hardware and software descriptions

[1073] Flying Devices (Drones):

[1074] The flying device is, for example, a DJI drone. This is an unmanned aerial vehicle that can fly remotely or autonomously and carry out designated tasks. The drone is equipped with a high-resolution camera (e.g., 4K camera) and a recording device. This flying device flies to the affected area in the event of a disaster and collects on-site information.

[1075] Photography Method:

[1076] The drone-mounted imaging device includes a camera and microphone for capturing high-resolution still and video images, as well as audio data, which is transmitted in real time to a server.

[1077] Artificial Intelligence Processing Means:

[1078] The server analyzes the data using a generative AI model (e.g., GPT-4), which receives images and audio data in real time and uses them to understand the situation at the disaster site.

[1079] Rescue route calculation method:

[1080] The server calculates the optimal rescue route based on the analysis results. This calculation uses real-time mapping software (e.g., Google Maps API), the A algorithm, and the Dijkstra algorithm. The calculated route is provided to the rescue team.

[1081] Material transport:

[1082] The server identifies the necessary supplies (e.g., medicines, food, water) based on the situation in the disaster area and creates a transportation plan. The drones load the specified supplies and fly along the optimal route calculated by the server. Once they arrive at the site, they safely drop off the supplies.

[1083] Measures to be taken in areas affected by communication disruptions:

[1084] In areas where communications are cut off, the server dispatches drones equipped with satellite phones and digital cameras. The drones fly to designated areas and hand over these devices to local responders. The responders collect data and send it back to the drone, which the server then analyzes to understand the local situation.

[1085] Emergency response information generation and coordination:

[1086] The server generates information based on the collected data, requiring initial response within 72 hours. Generative AI models are used to analyze the priority and details of the information, and emergency response information is shared with nearby hospitals, local governments, and other relevant organizations. This ensures that life-saving efforts are carried out quickly and accurately.

[1087] Specific operation example

[1088] For example, consider a case where a magnitude 7 earthquake occurs and isolates part of a city. The server receives notification of the earthquake and dispatches a drone to the affected area. The drone is equipped with a camera and a generating AI processing means, and upon arriving at the site, it activates the camera and audio recording device to collect data in real time and send it to the server.

[1089] The server analyzes the received data, identifies road closures and dangerous areas, and calculates the optimal rescue route. This information is provided to the rescue team. It then issues instructions to drones capable of flying to load medicines, food, and other supplies and transport them to isolated areas. The drones then safely deliver the supplies based on the optimal route.

[1090] In areas where communications are cut off, the server will dispatch drones equipped with satellite phones and digital cameras, handing these devices over to responders on the scene, who will then collect data, which the drones will then send back to the server for analysis.

[1091] An example of a prompt is, "A magnitude 7 earthquake has occurred in an urban area, isolating some areas. Please collect the necessary data and calculate the optimal rescue route so that rescue teams can reach the affected areas in the shortest time." By inputting this prompt into a generative AI model, it is possible to obtain quick and accurate analysis results.

[1092] By using the system of the present invention, on-site information during a disaster can be quickly and accurately grasped, enabling efficient transportation of supplies and rapid initial response.

[1093] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1094] Step 1:

[1095] Receive notifications of disaster occurrences

[1096] Input: Disaster notifications from government agencies, weather agencies, or automated sensors

[1097] Processing: The server receives the disaster notification and records the information in a database. It analyzes detailed information such as the type, location, and scale of the disaster, and begins preparations for an initial response.

[1098] Output: Record of disaster information, instructions for preparation for initial response

[1099] Step 2:

[1100] Drone dispatch instructions

[1101] Input: Information on the occurrence of the disaster, geographical information of the affected area

[1102] Processing: The server calculates the most efficient flight route, shooting points, and type of data to collect based on the geographical information of the affected area. This calculation is done using real-time mapping software (e.g., Google Maps API).

[1103] Output: Instructions for the drone to fly, capture points, and collect data

[1104] Step 3:

[1105] Data collection

[1106] Input: Instructions sent from the server, local video stream, audio data

[1107] Processing: When the drone (terminal) receives the command, it begins flying toward the disaster area. After arriving at the scene, it activates the camera and microphone to collect video and audio data. The collected data is sent to the server in real time.

[1108] Output: Video and audio data sent to the server in real time

[1109] Step 4:

[1110] Data analysis

[1111] Input: Video and audio data transmitted from the drone

[1112] Processing: The server analyzes the received data. A generative AI model (e.g., GPT-4) is used for the analysis, and the situation in the disaster area is grasped through image and audio analysis. For example, the state of collapse of a building can be determined from the image.

[1113] Output: Situation report of disaster area, identification of dangerous areas

[1114] Step 5:

[1115] Rescue route calculation

[1116] Input: Situation report of disaster area, information of danger zone

[1117] Processing: The server calculates the optimal rescue route, avoiding road closures, dangerous areas, and isolated areas, using the A algorithm and Dijkstra algorithm.

[1118] Output: Optimal rescue route information for rescue teams

[1119] Step 6:

[1120] Material transportation plan

[1121] Input: Situation in the affected areas, list of necessary supplies

[1122] Processing: The server creates a transportation plan. It creates a plan to load the necessary supplies (e.g. medicines, food, water) onto the drone. It calculates details such as the delivery route, delivery time, and delivery volume, and sends instructions to the drone.

[1123] Output: Instructions for drones to deliver supplies

[1124] Step 7:

[1125] Material transportation

[1126] Input: Material transport instructions from the server

[1127] Processing: The drone (terminal) loads the designated supplies and begins flying based on the optimal route calculated by the server. Once it arrives at the disaster area, it safely drops off the supplies.

[1128] Output: Confirmation of delivery of supplies to disaster area

[1129] Step 8:

[1130] Response to communication blackout areas

[1131] Input: Information on communication-blocked areas, server instructions

[1132] Processing: The server dispatches a drone equipped with a satellite phone and digital camera. The drone (terminal) flies to the designated area and hands over the equipment to local responders. The local responders collect the data, which is then sent back by the drone.

[1133] Output: Data collection and return from field responders

[1134] Step 9:

[1135] Emergency response information generation and coordination

[1136] Input: Data from local responders, analysis data from the affected areas

[1137] Processing: The server generates the necessary emergency response information based on the collected data. It uses a generative AI model to analyze the priority and details of the information, and then shares the emergency response information with nearby hospitals, local governments, and other relevant organizations.

[1138] Output: Emergency response information, information on cooperation with each relief organization

[1139] This will enable quick and accurate understanding of on-site information during disasters, enabling efficient transportation of supplies and rapid initial response.

[1140] (Application example 1)

[1141] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1142] In today's world, there is a need for rapid and accurate responses to disasters and emergencies. However, delays in information gathering and emergency response can be problematic when it is difficult to grasp the situation or transport supplies, or in areas where communications are cut off. In the field of security services, real-time on-site monitoring and emergency response are also important, and efficient systems are needed.

[1143] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1144] In this invention, the server includes means for controlling an operable flying device to collect video and audio of the scene in real time, means for analyzing the collected data with a generative AI model to detect anomalies, means for calculating and proposing an optimal response route based on the detected anomalies, and means for controlling the operable flying device to transport necessary supplies to the scene, thereby enabling prompt and accurate situation assessment and response at the scene in the event of a disaster or security service.

[1145] An "operable flying device" is a device that has the ability to fly to a designated location, collect data such as video and audio, and transport supplies.

[1146] "Operable imaging means" means a device mounted on a flying device that has the capability to capture and record still images, video, and audio in real time.

[1147] "Artificial intelligence processing means" refers to computer systems or software that have the functionality to analyze and analyse collected data in real time.

[1148] "Processing means" refers to a system that has the functionality to calculate and generate optimal routes and action plans based on the results of analysis.

[1149] "Means for loading and transporting supplies" refers to devices or systems that have the function of loading the necessary supplies onto a flight vehicle and transporting them safely to a designated location.

[1150] "Means for deploying on-board equipment in areas where communications are cut off and collecting data on the local situation" refers to equipment that has the function of deploying equipment such as satellite phones and digital cameras in areas where communications are cut off using flying devices to grasp the local situation.

[1151] A "generative AI model" is a learning model used to analyze collected data and detect anomalies and specific patterns.

[1152] A "prompt sentence" is text data containing instructions or commands input to a generative AI model.

[1153] "Emergency response information" is information that requires a rapid response and is generated based on collected local data.

[1154] The "means for detecting anomalies" refers to an AI analysis system that analyzes collected video and audio data to detect abnormal situations or suspicious behavior.

[1155] The "means for proposing the optimal response route" is a system that calculates and proposes the most efficient route of action to security staff and rescue teams after detecting an abnormality.

[1156] In this invention, a comprehensive system is constructed to respond quickly and accurately to disasters and security services. The detailed configuration and operation of the system will be described below.

[1157] System Configuration

[1158] 1. Hardware Configuration

[1159] Flying Devices (Drones):

[1160] The flying device is equipped with a camera and microphone to collect video and audio in real time, and the drone is equipped with GPS for precise navigation to specific locations.

[1161] Camera and audio collection devices:

[1162] High-resolution cameras and sensitive microphones are used to capture detailed video and audio of the scene.

[1163] Communication equipment:

[1164] Using equipment such as satellite phones and digital cameras, it is possible to collect data and communicate even in areas where communications are cut off.

[1165] 2. Software Configuration

[1166] Artificial Intelligence Processing System:

[1167] Using a generative AI model, collected video and audio data is analyzed in real time to detect anomalies.

[1168] Route Calculation System:

[1169] It is equipped with an algorithm that calculates the optimal rescue or response route based on the analysis results.

[1170] Data Streaming System:

[1171] It provides a server that uses open source libraries (e.g., OpenCV) to stream video in real time.

[1172] Frontend application:

[1173] It provides an intuitive interface for security staff and users through smartphone apps and web applications.

[1174] Processing content

[1175] Data collection:

[1176] The drone moves to a designated location and uses a camera and microphone to collect video and audio, which is then sent to a server in real time.

[1177] Data Analysis:

[1178] The collected data is analyzed using a generative AI model on the server, and if abnormal behavior or dangerous situations are detected, an alert is sent immediately.

[1179] Route calculation and suggestions:

[1180] Based on the analysis results, the optimal response route is calculated, and this information is sent in real time to a smartphone app or web application, providing it to security staff and rescue teams.

[1181] Material transport:

[1182] If necessary, the drones will carry medicines and emergency supplies and deliver them based on the optimal route.

[1183] Specific examples

[1184] For example, if a large earthquake occurs and a certain area is isolated, the system operates as follows.

[1185] 1. Drones are quickly deployed to isolated areas and use high-resolution cameras and microphones to collect on-site information.

[1186] 2. The collected data is sent to a server in real time, where a generative AI model analyzes it to assess the extent of damage to buildings and the risk to human life.

[1187] 3. Based on the analysis results, the server calculates the optimal route for the rescue team and notifies them via a smartphone app.

[1188] 4. Drones will carry necessary medicines and food and deliver supplies to the site based on the optimal route.

[1189] 5. Users can check the situation on-site in real time through the application and respond quickly.

[1190] Prompt sentences to input to the generative AI model

[1191] Please analyze the following camera footage frames and predict an anomalous activity score:

[1192] Frame: [Camera image data]

[1193] This will enable quick and accurate situation assessment and response in the event of a disaster or security service.

[1194] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1195] Step 1:

[1196] Launching the drone and dispatching it to the site

[1197] When a user receives a notification of a disaster or emergency, they send a command to the server to dispatch a drone. The server calculates a flight route to the specified location and sends it to the drone. The drone then launches and moves to the scene along the instructed route.

[1198] Input: Disaster occurrence notification from user, coordinate information of designated location

[1199] Output: Drone flight route calculation results, drone arrival at the site

[1200] Step 2:

[1201] Data collection

[1202] The drone arrives at the scene and collects video and audio data in real time using its on-board camera and microphone, which is then sent to a server in real time.

[1203] Input: Collecting video and audio data on-site

[1204] Output: Real-time data transmission to the server

[1205] Step 3:

[1206] Data analysis

[1207] The server inputs the received video and audio data into a generative AI model for real-time analysis, which processes the data to detect abnormal behavior or dangerous situations and returns the results.

[1208] Input: Collected video and audio data

[1209] Output: Anomaly detection results (anomaly score)

[1210] Specific operation: The generative AI model analyzes video frames and outputs an abnormal behavior detection score.

[1211] Step 4:

[1212] Optimal route calculation

[1213] The server uses the analysis results to calculate the optimal route for rescue teams and security staff, and provides the calculated route information in real time to smartphone apps and web applications.

[1214] Input: Anomaly detection results, site geographic information

[1215] Output: Optimal route information

[1216] Specific operation: Combining GPS data and anomaly detection results to calculate the shortest and safest route and notify the application.

[1217] Step 5:

[1218] Material transportation

[1219] If necessary, the server sends instructions to the drone to load medicines or emergency supplies, and the drone then delivers the supplies to the site based on the optimal route.

[1220] Input: Required material information, optimal transport route information

[1221] Output: Supplies arrive at the site

[1222] Specific operation: Based on instructions from the server, the drone loads supplies and delivers them to the site.

[1223] Step 6:

[1224] Real-time monitoring and feedback

[1225] Users can check the on-site video and analysis results in real time via a smartphone app or web application and take necessary action. The server receives feedback from the user and adjusts the next action.

[1226] Input: User feedback, real-time analysis results

[1227] Output: Next action instructions

[1228] Specific operation: Based on the information provided by the user, the system responds on-site and sends feedback to the server to continuously optimize the response.

[1229] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1230] The present invention is a system for realizing a rapid and accurate initial response in the event of a disaster, and includes an operable flying device (hereinafter referred to as a drone), an operable photographing means, an artificial intelligence processing means for analyzing and analysing generated data in real time, a processing means for calculating the optimal rescue route, a means for loading and transporting supplies, a means for placing onboard equipment in areas where communications are cut off and collecting data on the local situation, a means for analyzing the collected data and generating emergency response information, and a means for linking the emergency response information to external rescue organizations, as well as an emotion engine that recognizes the user's emotions.

[1231] System programs and their processing

[1232] 1. Initial response

[1233] Server: Upon receiving notification of a disaster, it obtains map data of the affected area, weather information, evacuation shelter information, etc. Based on this information, it sends instructions to the drone, such as flight route, shooting points, and type of data to be collected.

[1234] Drone (terminal): Arrives at the scene and uses a camera and microphone to collect still and moving images and audio data, which are then sent to a server in real time.

[1235] Server: Analyzes and analyzes the data received in real time to understand the situation on site.

[1236] 2. Analysis and rescue route calculation

[1237] Server: Analyzes collected data using generative AI to identify road closures, dangerous areas, and isolated regions. Then, calculates the optimal rescue route and provides it to the rescue team.

[1238] 3. Material transportation

[1239] Server: Identifies the supplies needed based on the situation in the affected area and creates a delivery plan, sending details such as delivery route, time, and quantity to the drone.

[1240] Drone (terminal): Loads designated supplies and transports them to the disaster area via the optimal route. Once there, the supplies are safely dropped off.

[1241] 4. Responses in areas where communications have been cut off

[1242] Server: In areas where communications have been cut off, the server issues instructions for the use of satellite phones and digital cameras. These devices are then loaded onto drones and sent to the scene.

[1243] Drone (terminal): Fly to designated area and hand over device to local responders, who collect data and transmit it back to the drone.

[1244] Server: Analyzes the returned data and understands the local situation.

[1245] 5. Coordination of emergency response information

[1246] Server: Based on data from the drone, collects information necessary for an initial response within 72 hours and generates analysis results.

[1247] Server: Shares information requiring emergency response with nearby hospitals, local governments, and other related organizations, enabling rapid life-saving efforts.

[1248] Incorporating an emotion engine

[1249] 1. Collecting Emotional Data

[1250] Drone (terminal): Data collected in the disaster area includes audio and video footage of the victims. This data is sent to the emotion engine.

[1251] Server: The acquired audio and video data is passed to the emotion engine, which analyzes the emotional state of the victim.

[1252] 2. Sentiment analysis and prioritization of rescue operations

[1253] Emotion engine (on the server): Analyzes the emotional state of victims (e.g., fear, anxiety, relief) and reflects that information in planning rescue operations.

[1254] Server: Dynamically change the priority of rescue efforts based on the results of emotion analysis. For example, prioritize assistance to areas with many disaster victims who are feeling extreme fear or anxiety.

[1255] Specific examples

[1256] For example, consider a case where an earthquake with a seismic intensity of 7 occurs and some areas of an urban area are isolated.

[1257] 1. The server receives notification of a disaster and issues instructions to dispatch drones to the affected area. The drones are equipped with a camera, a generation AI processing means, and an emotion engine.

[1258] 2. The drone (terminal) arrives at the disaster area, activates its camera and recording device to collect data in real time, and transmits audio and video data to the emotion engine.

[1259] 3. The server analyzes the received data, identifies road closures and dangerous areas, calculates the optimal rescue route, and provides this information to the rescue team.

[1260] 4. The server issues instructions to drones capable of flight to load supplies such as medicines and food and deliver them to isolated areas. The drones (terminals) then safely deliver the supplies based on the optimal route.

[1261] 5. In areas where communications are cut off, the server will dispatch drones equipped with satellite phones and digital cameras to deliver the data to responders on the ground. The drones will then return the data collected by the responders to the server for analysis.

[1262] 6. Based on the collected information, the server extracts information that requires emergency response and immediately contacts hospitals and local governments. Medical and rescue teams then respond promptly.

[1263] 7. The emotion engine analyzes the voice and video data of victims to understand their emotional state, allowing the server to appropriately adjust the priority of rescue efforts and provide the necessary assistance quickly.

[1264] In this way, by using the system of the present invention, on-site information can be quickly and accurately grasped during a disaster, enabling efficient transportation of supplies and a rapid initial response. In addition, by incorporating an emotion engine, appropriate support can be provided taking into account the emotional state of the victims.

[1265] The processing flow will be explained below.

[1266] Step 1:

[1267] The server receives notifications of disaster occurrences and obtains map data of the affected area, weather information, evacuation shelter information, etc. Based on this information, a program is prepared to instruct the drone on flight routes, shooting points, and the type of data to be collected.

[1268] Step 2:

[1269] The server sends flight commands to the drone, instructing it to head towards the designated area.

[1270] Step 3:

[1271] The drone (terminal) flies to the disaster site according to instructions from the server, and upon arrival, activates its camera and recording device, collecting still and moving images and audio data, which it then transmits to the server in real time.

[1272] Step 4:

[1273] The server receives real-time data transmitted by the drone and begins analyzing it using generative AI, which identifies road disruptions, dangerous areas, and isolated regions.

[1274] Step 5:

[1275] Based on the analysis results, the server calculates the optimal rescue route to the disaster area, and provides the calculation results to the rescue team, enabling them to carry out rescue operations quickly.

[1276] Step 6:

[1277] The server determines the type and amount of supplies needed based on the situation in the disaster area, creates a supply delivery plan, including delivery routes, times, and quantities, and transmits this plan to the drone.

[1278] Step 7:

[1279] The drone (terminal) follows instructions from the server to load supplies, fly to the disaster area along the optimal route, and safely drop off the supplies upon arrival.

[1280] Step 8:

[1281] The server sends instructions to the drones to deploy satellite phones and digital cameras in areas where communications have been cut off, including specific operational procedures for gathering information on the situation on the ground.

[1282] Step 9:

[1283] The drone (terminal) flies into a designated blackout area and hands over a satellite phone and digital camera to local personnel, who use these devices to collect data and transmit it back to the drone.

[1284] Step 10:

[1285] The server receives the local situation data returned by the drone and analyzes it in real time using generative AI, extracting information necessary for emergency response.

[1286] Step 11:

[1287] The server then sends the identified emergency response information to nearby medical institutions, local governments, and related organizations in real time, enabling rapid life-saving efforts.

[1288] Step 12:

[1289] The drone (terminal) collects audio and video data from the victim and sends it to the emotion engine, which analyzes the audio and video to identify the victim's emotional state.

[1290] Step 13:

[1291] The emotion engine (in the server) analyzes the emotional state of the victims (e.g., fear, anxiety, relief) and incorporates this information into the rescue operation plan in the server.

[1292] Step 14:

[1293] The server executes a means for dynamically changing the priority of relief efforts based on the emotion information provided by the emotion engine, thereby giving priority to providing support to areas with a large number of disaster victims who are extremely emotionally unstable.

[1294] Step 15:

[1295] The users (local rescue teams) quickly carry out rescue operations on site based on real-time information and priority adjustments provided by the server. Feedback during operations is also provided to the server to help with future decisions.

[1296] Through these steps, the system of the present invention can quickly and efficiently realize initial responses in the event of a disaster. This system improves the survival rate of disaster victims and enables the appropriate supply of supplies. Furthermore, by incorporating an emotion engine, it becomes possible to provide appropriate support that takes into account the emotional state of disaster victims.

[1297] Example 2

[1298] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1299] While rapid and accurate initial response is crucial in the event of a disaster, conventional systems often delay the collection of information needed to properly grasp the situation on the ground, and rescue operations are often carried out based on inaccurate data. Furthermore, support does not take into account the emotional state of the victims, resulting in insufficient psychological care and making it difficult to properly prioritize rescue operations. To solve these issues, a system is needed that can collect and analyze on-site information in real time, calculate optimal rescue routes, and dynamically adjust the priority of rescue operations by recognizing the emotional state of the victims.

[1300] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an operable flying device, an operable photographing means, an artificial intelligence processing means for analyzing the generated data in real time, a processing means for calculating an optimal rescue route based on the analysis results, a means for loading supplies onto the flying device and transporting them, a means for deploying an onboard device in a communication-blocked area and collecting local situation data, a means for analyzing the collected data and generating emergency response information, a means for linking the emergency response information to external rescue organizations, an emotion analysis means for analyzing audio and video data of victims and recognizing their emotional states, and a means for dynamically adjusting the priority of rescue operations based on their emotional states. This enables rapid and accurate understanding of the situation at the scene of a disaster, enabling efficient rescue operations and delivery of supplies, and realizing appropriate support that takes into account the psychological care of victims.

[1301] An "operable flying device" is a device capable of airborne flight and reaching designated locations to collect data or deliver supplies.

[1302] "Operable imaging means" means a camera or other imaging device capable of capturing still or video images and attached to a flying device for data collection purposes.

[1303] "Artificial intelligence processing means for analyzing and analysing generated data in real time" refers to an artificial intelligence-based processing system used to instantly analyze and analyse collected data and grasp the situation on site.

[1304] "Processing means for calculating optimal rescue route" refers to a processing system for calculating the most efficient route for carrying out rescue operations based on the data analyzed in real time.

[1305] "Means for loading and transporting supplies onto a flying vehicle" means the mechanism or structure for loading relief supplies onto a flying vehicle and transporting them to a designated disaster area.

[1306] "Means for deploying on-board equipment in areas where communications are cut off and collecting on-site situation data" refers to methods and devices for deploying the necessary equipment and collecting on-site situation data so that data collection is possible even in areas where communications infrastructure has been destroyed.

[1307] "Means for analyzing collected data and generating emergency response information" refers to a processing system that analyzes collected data, identifies situations requiring emergency response, and generates response information.

[1308] The "means for sharing emergency response information with external relief organizations" refers to a communication means for quickly and accurately transmitting the generated emergency response information to external relief organizations.

[1309] An "emotion analysis means" is an analysis system that analyzes the audio and video data of victims to recognize their emotional state (e.g., fear, anxiety, relief).

[1310] "Means for dynamically adjusting the priority of rescue operations based on emotional state" refers to a processing system for changing the priority of rescue operations in real time based on the emotional state of the victim recognized by the emotion analysis means.

[1311] The present invention provides a system for achieving rapid and accurate initial response in the event of a disaster. The system includes an operable flight device, an operable imaging means, an artificial intelligence processing means for analyzing and interpreting generated data in real time, a processing means for calculating an optimal rescue route, a means for loading and transporting supplies, a means for deploying an onboard device in a communication-blocked area and collecting local situation data, a means for analyzing the collected data and generating emergency response information, a means for linking the emergency response information to an external rescue organization, an emotion analysis means for recognizing the emotions of disaster victims, and a means for dynamically adjusting the priority of rescue operations based on the emotional state.

[1312] When the server receives notification of a disaster, it obtains map data, weather information, and evacuation shelter information for the affected area. Based on this information, it determines the flight route, shooting points, type of data to collect, and issues instructions to the drone. The drone (terminal) arrives at the scene and uses its camera and microphone to collect video and still images and audio data, which it then sends to the server in real time. The server analyzes the data received in real time and uses a generative AI model to understand the situation. Machine learning libraries such as TensorFlow and PyTorch are used for this analysis.

[1313] The server runs an algorithm (such as the A algorithm or Dijkstra algorithm) to calculate the optimal rescue route based on the analyzed data. This identifies areas where roads are cut off, dangerous areas, and isolated regions, and provides this information to rescue teams. At the same time, it identifies necessary supplies (food, medicine, etc.) and creates a delivery plan. It issues instructions to drones capable of flying to load the supplies and deliver them to the disaster area via the optimal route.

[1314] In areas where communications are cut off, the server dispatches drones equipped with satellite phones and digital cameras and hands them over to local responders. The data collected by the local responders is then sent back to the server by drone. The server then analyzes this data to accurately grasp the local situation.

[1315] Emergency response information is compiled based on data from drones, generating the information required for initial response within 72 hours. The server then shares this information with hospitals and local governments to support rapid life-saving efforts. Furthermore, audio and video data collected in the disaster area is sent to an emotion analysis system, which analyzes the emotional state of the victims (e.g., fear, anxiety, relief) and incorporates this information into rescue operation plans. Based on this information, the server dynamically adjusts the priority of rescue operations.

[1316] As a concrete example, consider a case where a magnitude 7 earthquake occurs and isolates part of an urban area. The server receives a disaster notification and immediately dispatches a drone to collect information on the local situation. The collected data is analyzed to determine the optimal rescue route and a plan for transporting supplies. At the same time, the emotional state of the victims is analyzed and optimal support activities are carried out, taking into consideration their mental health.

[1317] Below is an example of a prompt sentence:

[1318] "A magnitude 7 earthquake has occurred, isolating parts of urban areas. Dispatch drones to quickly collect on-site information from the affected areas, calculate rescue routes, and develop an optimal plan for delivering necessary supplies. Also, analyze the emotional state of the victims using audio and video data to appropriately prioritize rescue efforts."

[1319] The above is an embodiment of the present invention, and by using this system, on-site information can be quickly and accurately grasped during a disaster, enabling efficient rescue operations and the transportation of supplies. In addition, by using emotion analysis means, appropriate support can be provided taking into account the emotional state of the victims.

[1320] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1321] Step 1: Receive notifications and prepare

[1322] The server receives notifications of disaster occurrences, which are provided in real time by meteorological agencies and monitoring systems.

[1323] Input: Disaster notification (e.g. earthquake, flood)

[1324] The server obtains map data, weather information, and evacuation shelter information for the affected area, and integrates this information to determine flight routes, photography points, and the type of data to be collected.

[1325] Data processing: Map data, weather information, and evacuation shelter information are obtained using APIs, and this is integrated to generate drone instruction information.

[1326] Output: Flight route, shooting points, type of collected data

[1327] Specific operation: The server retrieves map data using the Google Maps API, weather information from the Japan Meteorological Agency API, and evacuation shelter information from government databases.

[1328] Step 2: Dispatch the drone and collect data

[1329] The server transmits the predetermined flight route and collection items to the drone's command center.

[1330] Input: Flight route, shooting points, type of collected data

[1331] The drone (terminal) follows a designated flight route, uses a camera and microphone to collect video and audio data, and transmits it to a server in real time.

[1332] Data calculation: Using GPS, the drone flies autonomously and collects data at designated points.

[1333] Output: Real-time transmission of collected video and audio data

[1334] Specific operation: The drone flies using its built-in GPS and communication module, takes pictures with its camera, records sounds with its microphone, and uploads the collected data to a server in real time.

[1335] Step 3: Data analysis and situation assessment

[1336] The server analyzes the received video and audio data using a generative AI model.

[1337] Input: Collected video and audio data

[1338] The server uses TensorFlow or PyTorch to analyze the image data and identify the situation at the scene (e.g., collapsed buildings, flooded areas), and converts the audio data into text using the Google Cloud Speech-to-Text API and passes it to the sentiment analysis tool.

[1339] Data processing: image data analysis, voice data text conversion

[1340] Output: Analysis results (collapsed buildings, flooded areas), audio data converted to text

[1341] Specific operation: The server analyzes image data using an object recognition algorithm, and audio data is converted into text and used for emotion analysis.

[1342] Step 4: Calculate the rescue route

[1343] The server calculates the optimal rescue route based on the analysis results.

[1344] Input: Analysis results (collapsed buildings, flooded areas)

[1345] The server calculates the optimal route using the A algorithm or Dijkstra algorithm and provides it to the rescue team.

[1346] Data calculation: Calculating the optimal rescue route

[1347] Output: Optimal rescue route

[1348] Specific operation: The server calculates the shortest and safest rescue route taking into account road conditions and obstacles, and notifies the rescue team.

[1349] Step 5: Prepare and execute the shipment

[1350] The server identifies the necessary supplies and issues instructions to load them onto the drone.

[1351] Input: Situation in the affected area, list of necessary supplies

[1352] The drone (terminal) will be loaded with designated supplies and delivered to the disaster area.

[1353] Data processing: material loading plans, transport route instructions

[1354] Output: Material transportation plan, transportation route

[1355] Specific operation: The drone loads the specified supplies and safely delivers them along the programmed route.

[1356] Step 6: Response to communication disruptions in affected areas

[1357] The server will dispatch drones equipped with satellite phones and digital cameras to areas where communications have been cut off.

[1358] Input: Information on communication-blocked areas, list of required equipment

[1359] The drone (terminal) drops off the equipment on-site and transmits the collected data back to the server.

[1360] Data processing: Equipment layout plan, return of collected data

[1361] Output: Local situation data

[1362] Specific operation: The drone hands over the equipment to the local responder and sends the collected data back to the server.

[1363] Step 7: Generate and coordinate emergency response information

[1364] The server generates emergency response information based on data from the drone.

[1365] Input: collected data, analysis results

[1366] The server aggregates the data and generates information necessary for initial response within 72 hours, which is then shared with hospitals and local governments.

[1367] Data processing: Emergency response information generation and notification

[1368] Output: Emergency response information

[1369] Specific operation: The server uses APIs and SMS gateways to directly communicate the generated emergency information to each agency.

[1370] Step 8: Sentiment analysis and prioritization

[1371] The server transmits audio and video data collected in the disaster area to the emotion analysis means.

[1372] Input: Text-encoded audio data, video data

[1373] The emotion analysis means recognizes the emotional state of the victim and sends that information back to the server.

[1374] Data Computing: Emotional State Analysis

[1375] Output: Emotion analysis results

[1376] The server dynamically adjusts the priority of rescue operations based on the results of emotion analysis.

[1377] Data processing: Dynamic adjustment of priorities

[1378] Output: Dynamically adjusted rescue operation priorities

[1379] How it works: The server uses a generative AI model to perform emotion analysis and re-prioritize based on the emotional state.

[1380] (Application example 2)

[1381] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1382] During disasters, prompt and accurate initial responses are required, but appropriate rescue efforts are often hindered due to the uncertainty of the situation on the ground. Furthermore, in areas where communications are cut off, it is difficult to gather information, making it impossible to properly prioritize relief efforts. Furthermore, prioritization of relief efforts does not take into account the emotional state of the victims, which is a factor in delaying appropriate assistance to victims.

[1383] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1384] In this invention, the server includes an operable flying device, an operable photographing means, an artificial intelligence processing means for analyzing generated data in real time, a processing means for calculating an optimal rescue route based on the results of the analysis, a means for loading supplies onto the flying device and transporting them, a means for deploying an onboard device in a communication-blocked area and collecting on-site situation data, a means for analyzing the collected data and generating emergency response information, a means for linking the emergency response information to external rescue organizations, a means for collecting user emotion data on an operating terminal and performing emotion analysis, and a means for adjusting the priority of rescue operations based on the results of the emotion analysis, thereby enabling quick and accurate understanding of on-site information in disaster-stricken areas, efficient delivery of supplies, and appropriate prioritization of rescue operations.

[1385] An "operable flying device" is a flying machine that can be remotely controlled or autonomously operated, such as a drone, and is used to collect information and transport supplies during disasters.

[1386] "Operable imaging means" means a camera or other imaging device used to capture still or video images of a scene and collect information in real time.

[1387] "Artificial intelligence processing means for analyzing and analysing generated data in real time" refers to artificial intelligence algorithms and systems that rapidly process collected data and extract and analyse necessary information.

[1388] The "processing means for calculating the optimal rescue route" refers to an algorithm or software for calculating the safest and most efficient rescue route in the event of a disaster.

[1389] "Means for loading and transporting supplies" refers to the equipment or mechanism for loading relief supplies and other items onto a flying device and transporting them to their destination.

[1390] "Means for deploying on-board equipment in areas where communications are disrupted and collecting local situation data" refers to methods and systems for deploying machines or devices in areas where communications infrastructure has been destroyed and collecting the necessary information.

[1391] "Means for generating emergency response information" refers to algorithms or systems that analyze collected data and generate information required for rapid response.

[1392] "Means for linking emergency response information to external relief agencies" refers to communication means or systems for quickly transmitting the generated emergency response information to relevant relief agencies or organizations.

[1393] "Means for collecting user emotional data on operating devices and performing emotional analysis" refers to technologies and systems for collecting audio and video data from devices used by disaster victims and analyzing their emotional states.

[1394] The "means for adjusting the priority of rescue operations based on the results of sentiment analysis" refers to an algorithm or system for dynamically changing the priority of rescue operations based on the results obtained from sentiment analysis.

[1395] In order to put this invention into practice, it is important to build a system that can collect information in real time during a disaster and provide a rapid and accurate initial response.

[1396] System Configuration

[1397] 1. Operable flying device

[1398] These devices, such as drones, are capable of remote control and autonomous operation, and are used to gather information and transport supplies at disaster sites.

[1399] 2. Operable photography means

[1400] The drone is equipped with a high-resolution camera that can take still images and videos of the disaster area and transmit them in real time.

[1401] 3. Artificial intelligence processing means for analyzing and analysing the generated data in real time

[1402] The server is equipped with a generative AI model that quickly analyzes and parses the data sent from the drone.

[1403] Example) AI model used: TensorFlow, PyTorch

[1404] 4. Processing means for calculating the optimal rescue route

[1405] Based on the analysis results, the server uses an algorithm to calculate the optimal rescue route and provides it to rescue agencies.

[1406] Example) Algorithms used: Dijkstra's Algorithm, A Search Algorithm

[1407] 5. Means for carrying and transporting supplies

[1408] The drones are equipped with a mechanism to load designated supplies and transport them quickly along a calculated, optimal route.

[1409] 6. Means of deploying onboard equipment in communication-blocked areas to gather on-site situation data.

[1410] The drones will deploy satellite-enabled equipment in areas where communications infrastructure has been destroyed and collect situational data.

[1411] 7. Means for generating emergency response information

[1412] The server analyzes the collected data and generates information needed for rapid response, including weather information, evacuation shelter information, and more.

[1413] 8. Means for communicating emergency response information to external relief agencies

[1414] The generated emergency response information is automatically sent to relevant relief agencies, enabling a rapid response.

[1415] 9. Means for collecting user emotion data and performing emotion analysis on the operating device

[1416] It collects audio and video data from devices such as drones and smartphones used by disaster victims, and analyzes the user's emotional state.

[1417] Example) Emotion analysis technology used: EmotionRecognizer

[1418] 10. A method for adjusting rescue operation priorities based on the results of sentiment analysis

[1419] Based on the results of emotion analysis, the server dynamically adjusts the priorities of rescue efforts, prioritizing support for areas with a high number of victims experiencing extreme fear and anxiety.

[1420] Specific examples of processing

[1421] For example, consider a case where a magnitude 7 earthquake occurs in an urban area, causing communication to be unavailable in some areas.

[1422] 1. The server receives information about the occurrence of a disaster and issues instructions to the drone to dispatch to the site.

[1423] 2. After arriving at the site, the drone uses a camera to collect data and transmits it to a server in real time.

[1424] 3. The server analyzes the received data using a generative AI model to identify road disruptions and dangerous areas.

[1425] 4. When the user reports their situation and emotional state using a smartphone app, the server uses that data to calculate the optimal rescue route.

[1426] Example prompt sentence:

[1427] "The user's current location is Tokyo, the surrounding area is in a state of great chaos, and they have entered the emotion message 'It's very scary; people around me are crying.' Please display the optimal evacuation route and their emotional state."

[1428] By building such a system, it will be possible to quickly and accurately grasp the situation in disaster-stricken areas, enabling efficient and appropriate relief efforts. Furthermore, prioritizing relief efforts based on emotion analysis will enable appropriate support that takes into account the emotional state of the victims.

[1429] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1430] Step 1:

[1431] The server receives information about the occurrence of a disaster. At the same time, it also acquires map data of the affected area, weather information, and evacuation shelter information. Based on this information, it sends instructions to the drone, such as flight route, shooting points, and the type of data to be collected.

[1432] Input: Disaster occurrence notification, map data, weather information, evacuation shelter information

[1433] Data processing and calculation: Data integration, calculation using flight route optimization algorithm

[1434] Output: Instruction data for the drone (flight route, shooting points, type of collected data)

[1435] Step 2:

[1436] The drone (terminal) arrives at the scene and uses a camera and microphone to collect still and moving images and audio data, which is then sent to a server in real time.

[1437] Input: Flight route, shooting points, type of collected data (date, time, location information, etc.)

[1438] Data processing and calculation: camera image and video capture, audio data recording

[1439] Output: Real-time data (video and audio files)

[1440] Step 3:

[1441] The server uses AI to analyze the received real-time data, and based on the video and audio data sent from the drone, it grasps the situation and identifies areas where land routes are blocked and dangerous areas.

[1442] Input: Real-time data (video and audio files)

[1443] Data processing and calculation: Image analysis, audio analysis, anomaly detection using generative AI models

[1444] Output: Disaster situation analysis results (land route cutoff points, dangerous areas)

[1445] Step 4:

[1446] The server calculates the optimal rescue route based on the disaster situation analysis, using an algorithm to find a route that avoids road closures and dangerous areas.

[1447] Input: Disaster situation analysis results

[1448] Data processing and calculation: Optimal route calculation using Dijkstra's Algorithm and A Search Algorithm

[1449] Output: Optimal rescue route

[1450] Step 5:

[1451] Users use a smartphone app to input their current location and emotional data, which is then sent to a server.

[1452] Input: current location information, emotional data (voice and text)

[1453] Data processing and calculation: GPS data acquisition, voice recognition, emotion analysis

[1454] Output: User position and emotion data

[1455] Step 6:

[1456] The server performs sentiment analysis based on the data received from the user, identifying the user's level of fear and anxiety and dynamically adjusting the priority of rescue operations.

[1457] Input: User location data, emotion data

[1458] Data processing and calculation: Emotion analysis using EmotionRecognizer

[1459] Output: Emotional state (e.g., fear, anxiety)

[1460] Step 7:

[1461] The system combines the emotion analysis results with the optimal rescue route and provides them to the rescue team to determine a specific rescue plan. It also issues instructions to dispatch drones loaded with supplies to deliver them to the disaster area as needed.

[1462] Input: Emotion analysis results, optimal rescue route

[1463] Data processing and calculation: Data integration, material distribution planning, optimal route selection

[1464] Output: Rescue instruction data (rescue plan, material transport instructions)

[1465] Step 8:

[1466] The drone (terminal) will load the designated supplies and transport them to the disaster area along a calculated optimal route. After arriving, the supplies will be safely dropped off.

[1467] Input: Material transport instructions, optimal route

[1468] Data processing and calculation: Loading of physical supplies, automatic flight route operation

[1469] Output: Material delivery completion notification

[1470] Example prompts

[1471] "The user's current location is Tokyo, the surrounding area is in a state of great chaos, and they have entered the emotion message 'It's very scary; people around me are crying.' Please display the optimal evacuation route and their emotional state."

[1472] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1473] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1474] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1475] [Fourth embodiment]

[1476] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1477] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1478] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1479] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1480] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1481] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1482] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1483] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1484] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1485] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1486] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1487] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1488] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1489] The present invention is a system for realizing a rapid and accurate initial response in the event of a disaster, and includes an operable flying device (hereinafter referred to as a drone), an operable photographing means, an artificial intelligence processing means for analyzing and analysing generated data in real time, a processing means for calculating the optimal rescue route, a means for loading and transporting supplies, a means for placing onboard equipment in areas where communications are cut off and collecting data on the local situation, a means for analyzing the collected data and generating emergency response information, and a means for linking the emergency response information to external rescue organizations.

[1490] System programs and their processing

[1491] 1. Initial response

[1492] Server: Upon receiving notification of a disaster, the server dispatches a drone to the affected area and sends instructions to the drone, such as flight route, shooting points, and the type of data to be collected.

[1493] Drone (terminal): Arrives at the scene and uses a camera and microphone to collect still and moving images and audio data, which are then sent to a server in real time.

[1494] Server: Analyzes and analyzes the data received in real time to understand the situation on site.

[1495] 2. Analysis and rescue route calculation

[1496] Server: Analyzes collected data using generative AI to identify road closures, dangerous areas, and isolated regions. Then, calculates the optimal rescue route and provides it to the rescue team.

[1497] 3. Material transportation

[1498] Server: Identifies the supplies needed based on the situation in the affected area and creates a delivery plan, sending details such as delivery route, time, and quantity to the drone.

[1499] Drone (terminal): Loads designated supplies and transports them to the disaster area via the optimal route. Once there, the supplies are safely dropped off.

[1500] 4. Responses in areas where communications have been cut off

[1501] Server: In areas where communications have been cut off, the server issues instructions for the use of satellite phones and digital cameras. These devices are then loaded onto drones and sent to the scene.

[1502] Drone (terminal): Fly to designated area and hand over device to local responders, who collect data and transmit it back to the drone.

[1503] Server: Analyzes the returned data and understands the local situation.

[1504] 5. Coordination of emergency response information

[1505] Server: Based on data from the drone, collects information necessary for initial response within 72 hours and generates analysis results.

[1506] Server: Shares information requiring emergency response with nearby hospitals, local governments, and other related organizations, supporting rapid life-saving efforts.

[1507] Specific examples

[1508] For example, consider a case where an earthquake with a seismic intensity of 7 occurs and some areas of an urban area are isolated.

[1509] 1. The server receives notification of an earthquake and issues a command to dispatch a drone to the affected area. The drone is equipped with a camera and a generating AI processing means.

[1510] 2. The drone (terminal) arrives at the disaster area, activates the camera and recording device, collects data in real time, and transmits it to the server.

[1511] 3. The server analyzes the received data, identifies road closures and dangerous areas, calculates the optimal rescue route, and provides this information to the rescue team.

[1512] 4. The server issues instructions to drones capable of flight to load supplies such as medicines and food and deliver them to isolated areas. The drones (terminals) then safely deliver the supplies based on the optimal route.

[1513] 5. In areas where communications are cut off, the server will dispatch drones equipped with satellite phones and digital cameras to deliver the data to responders on the ground. The drones will then return the data collected by the responders to the server for analysis.

[1514] 6. Based on the collected information, the server extracts information that requires emergency response and immediately contacts hospitals and local governments. Medical and rescue teams then respond promptly.

[1515] In this way, by using the system of the present invention, on-site information during a disaster can be quickly and accurately grasped, enabling efficient transportation of supplies and rapid initial response.

[1516] The processing flow will be explained below.

[1517] Step 1:

[1518] The server receives notifications of disaster occurrences and obtains map data of the affected area, weather information, evacuation shelter information, etc. Based on this information, it determines the flight route, shooting points, and type of data to be collected, and sends instructions to the drone.

[1519] Step 2:

[1520] The drone (terminal) follows instructions from the server and flies to the disaster area. After arriving at the scene, it activates its camera and audio recording device to collect still images and audio data. The collected data is then sent to the server in real time.

[1521] Step 3:

[1522] The server receives real-time data sent from the drone and begins analysis using the AI ​​generation system. From the analysis results, it identifies areas where land routes are cut off, dangerous areas, and isolated areas.

[1523] Step 4:

[1524] The server calculates the optimal rescue route based on the analysis results, taking into account the geographical information and traffic conditions of the affected area, and provides the calculation results to the rescue team.

[1525] Step 5:

[1526] The server determines the type, quantity, and delivery destination of the required supplies, creates a delivery plan, and sends instructions to the drones from loading the supplies to delivery.

[1527] Step 6:

[1528] The drone (terminal) follows instructions from the server to load supplies, fly along the optimal route designated, and head to the destination. After arriving at the destination, it safely drops off the supplies.

[1529] Step 7:

[1530] The server sends instructions to drones for use in areas where communications are cut off, including satellite phone and digital camera instructions, including instructions to equip the drones with devices to gather information on the ground.

[1531] Step 8:

[1532] The drone (terminal) flies into a designated quarantined area and hands over its onboard equipment to local personnel, who use these devices to collect data and transmit it back to the drone.

[1533] Step 9:

[1534] The server receives the on-site data returned from the drone and begins analyzing it in real time, and based on the information obtained from the analysis, identifies situations requiring emergency response.

[1535] Step 10:

[1536] The server then sends the identified emergency response information to nearby medical institutions, local governments, and other relevant organizations in real time, enabling rapid life-saving efforts.

[1537] Step 11:

[1538] Based on the information provided by the server, users (local rescue teams) quickly head to the site and begin rescue operations. They also provide the latest on-site information as feedback to the server as necessary.

[1539] By implementing the above steps, the system of the present invention can quickly and efficiently realize initial responses in the event of a disaster, improving the survival rate of disaster victims and enabling the appropriate supply of supplies.

[1540] Example 1

[1541] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1542] In the event of a disaster, a fast and accurate initial response is required. However, it is extremely difficult to quickly grasp the situation in the affected area, calculate the optimal rescue route, and transport the necessary supplies. Furthermore, in areas where communications are cut off, it is difficult to gather information, which can delay emergency responses. To address these challenges, a system is needed that can collect and analyze on-site information in real time and make quick decisions.

[1543] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1544] In this invention, the server includes means for receiving disaster notifications, an operable flight device, an operable photographing device, an artificial intelligence processing means for analyzing generated data in real time, a processing means for calculating an optimal rescue route based on the results of the analysis, means for loading supplies onto the flight device and transporting them, means for deploying equipment in a communication-blocked area and collecting on-site situation data, means for analyzing the collected data and generating emergency response information, means for linking the emergency response information to external rescue organizations, means for analyzing initial response information using a generative AI model, and means for inputting information into the generative AI model using prompt sentences. This allows for quick and accurate understanding of on-site information during a disaster, enabling efficient delivery of supplies and a rapid initial response.

[1545] "Means for receiving disaster notifications" refers to devices or systems for receiving disaster information sent from government agencies, automatic sensors, the Japan Meteorological Agency, etc.

[1546] An "operable flying device" is an unmanned aerial vehicle (drone) that can fly remotely or autonomously and perform a specific task.

[1547] "Operable imaging means" refers to devices such as cameras and microphones that are mounted on the flying device and are used to collect moving and still images and audio data.

[1548] "Artificial intelligence processing means" refers to an artificial intelligence (AI) system that analyzes and parses the generated data in real time.

[1549] The "processing means for calculating optimal rescue routes" refers to calculation algorithms or software for optimizing rescue routes within the disaster area based on collected data.

[1550] "Means for carrying and transporting supplies" refers to a system for loading supplies such as medicines and food onto a flying device and transporting them safely to a designated destination.

[1551] "Means for deploying onboard equipment in areas where communications have been cut off and for collecting on-site situation data" refers to a method for deploying devices such as satellite phones and digital cameras to gather on-site information in areas where communications infrastructure has been destroyed.

[1552] The "means for generating emergency response information" is a system for extracting important disaster information based on collected data and outputting the analysis results.

[1553] The "means for sharing emergency response information with external relief organizations" is a communication system for transmitting the generated emergency response information to relief organizations, including nearby hospitals and local governments.

[1554] A "generative AI model" is a type of artificial intelligence, a machine learning model used to analyze data and support decision-making.

[1555] "Means of inputting information into a generative AI model using prompt sentences" refers to a method of inputting specific questions or commands (prompts) into a generative AI model to obtain appropriate analysis results or output.

[1556] The present invention provides a system for achieving rapid and accurate initial response in the event of a disaster. The system includes an operable flight device, an operable imaging means, an artificial intelligence (AI) processing means for real-time data analysis, a processing means for calculating an optimal rescue route, a means for loading and transporting supplies, a means for deploying an onboard device in a communication-blocked area to collect local situation data, a means for analyzing the collected data and generating emergency response information, and a means for linking the emergency response information to an external relief organization. The system further includes a means for analyzing the initial response information using a generative AI model and prompt sentences.

[1557] Specific hardware and software descriptions

[1558] Flying Devices (Drones):

[1559] The flying device is, for example, a DJI drone. This is an unmanned aerial vehicle that can fly remotely or autonomously and carry out designated tasks. The drone is equipped with a high-resolution camera (e.g., 4K camera) and a recording device. This flying device flies to the affected area in the event of a disaster and collects on-site information.

[1560] Photography Method:

[1561] The drone-mounted imaging device includes a camera and microphone for capturing high-resolution still and video images, as well as audio data, which is transmitted in real time to a server.

[1562] Artificial Intelligence Processing Means:

[1563] The server analyzes the data using a generative AI model (e.g., GPT-4), which receives images and audio data in real time and uses them to understand the situation at the disaster site.

[1564] Rescue route calculation method:

[1565] The server calculates the optimal rescue route based on the analysis results. This calculation uses real-time mapping software (e.g., Google Maps API), the A algorithm, and the Dijkstra algorithm. The calculated route is provided to the rescue team.

[1566] Material transport:

[1567] The server identifies the necessary supplies (e.g., medicines, food, water) based on the situation in the disaster area and creates a transportation plan. The drones load the specified supplies and fly along the optimal route calculated by the server. Once they arrive at the site, they safely drop off the supplies.

[1568] Measures to be taken in areas affected by communication disruptions:

[1569] In areas where communications are cut off, the server dispatches drones equipped with satellite phones and digital cameras. The drones fly to designated areas and hand over these devices to local responders. The responders collect data and send it back to the drone, which the server then analyzes to understand the local situation.

[1570] Emergency response information generation and coordination:

[1571] The server generates information based on the collected data, requiring initial response within 72 hours. Generative AI models are used to analyze the priority and details of the information, and emergency response information is shared with nearby hospitals, local governments, and other relevant organizations. This ensures that life-saving efforts are carried out quickly and accurately.

[1572] Specific operation example

[1573] For example, consider a case where a magnitude 7 earthquake occurs and isolates part of a city. The server receives notification of the earthquake and dispatches a drone to the affected area. The drone is equipped with a camera and a generating AI processing means, and upon arriving at the site, it activates the camera and audio recording device to collect data in real time and send it to the server.

[1574] The server analyzes the received data, identifies road closures and dangerous areas, and calculates the optimal rescue route. This information is provided to the rescue team. It then issues instructions to drones capable of flying to load medicines, food, and other supplies and transport them to isolated areas. The drones then safely deliver the supplies based on the optimal route.

[1575] In areas where communications are cut off, the server will dispatch drones equipped with satellite phones and digital cameras, handing these devices over to responders on the scene, who will then collect data, which the drones will then send back to the server for analysis.

[1576] An example of a prompt is, "A magnitude 7 earthquake has occurred in an urban area, isolating some areas. Please collect the necessary data and calculate the optimal rescue route so that rescue teams can reach the affected areas in the shortest time." By inputting this prompt into a generative AI model, it is possible to obtain quick and accurate analysis results.

[1577] By using the system of the present invention, on-site information during a disaster can be quickly and accurately grasped, enabling efficient transportation of supplies and rapid initial response.

[1578] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1579] Step 1:

[1580] Receive notifications of disaster occurrences

[1581] Input: Disaster notifications from government agencies, weather agencies, or automated sensors

[1582] Processing: The server receives the disaster notification and records the information in a database. It analyzes detailed information such as the type, location, and scale of the disaster, and begins preparations for an initial response.

[1583] Output: Record of disaster information, instructions for preparation for initial response

[1584] Step 2:

[1585] Drone dispatch instructions

[1586] Input: Information on the occurrence of the disaster, geographical information of the affected area

[1587] Processing: The server calculates the most efficient flight route, shooting points, and type of data to collect based on the geographical information of the affected area. This calculation is done using real-time mapping software (e.g., Google Maps API).

[1588] Output: Instructions for the drone to fly, capture points, and collect data

[1589] Step 3:

[1590] Data collection

[1591] Input: Instructions sent from the server, local video stream, audio data

[1592] Processing: When the drone (terminal) receives the command, it begins flying toward the disaster area. After arriving at the scene, it activates the camera and microphone to collect video and audio data. The collected data is sent to the server in real time.

[1593] Output: Video and audio data sent to the server in real time

[1594] Step 4:

[1595] Data analysis

[1596] Input: Video and audio data transmitted from the drone

[1597] Processing: The server analyzes the received data. A generative AI model (e.g., GPT-4) is used for the analysis, and the situation in the disaster area is grasped through image and audio analysis. For example, the state of collapse of a building can be determined from the image.

[1598] Output: Situation report of disaster area, identification of dangerous areas

[1599] Step 5:

[1600] Rescue route calculation

[1601] Input: Situation report of disaster area, information of danger zone

[1602] Processing: The server calculates the optimal rescue route, avoiding road closures, dangerous areas, and isolated areas, using the A algorithm and Dijkstra algorithm.

[1603] Output: Optimal rescue route information for rescue teams

[1604] Step 6:

[1605] Material transportation plan

[1606] Input: Situation in the affected areas, list of necessary supplies

[1607] Processing: The server creates a transportation plan. It creates a plan to load the necessary supplies (e.g. medicines, food, water) onto the drone. It calculates details such as the delivery route, delivery time, and delivery volume, and sends instructions to the drone.

[1608] Output: Instructions for drones to deliver supplies

[1609] Step 7:

[1610] Material transportation

[1611] Input: Material transport instructions from the server

[1612] Processing: The drone (terminal) loads the designated supplies and begins flying based on the optimal route calculated by the server. Once it arrives at the disaster area, it safely drops off the supplies.

[1613] Output: Confirmation of delivery of supplies to disaster area

[1614] Step 8:

[1615] Response to communication blackout areas

[1616] Input: Information on communication-blocked areas, server instructions

[1617] Processing: The server dispatches a drone equipped with a satellite phone and digital camera. The drone (terminal) flies to the designated area and hands over the equipment to local responders. The local responders collect the data, which is then sent back by the drone.

[1618] Output: Data collection and return from field responders

[1619] Step 9:

[1620] Emergency response information generation and coordination

[1621] Input: Data from local responders, analysis data from the affected areas

[1622] Processing: The server generates the necessary emergency response information based on the collected data. It uses a generative AI model to analyze the priority and details of the information, and then shares the emergency response information with nearby hospitals, local governments, and other relevant organizations.

[1623] Output: Emergency response information, information on cooperation with each relief organization

[1624] This will enable quick and accurate understanding of on-site information during disasters, enabling efficient transportation of supplies and rapid initial response.

[1625] (Application example 1)

[1626] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1627] In today's world, there is a need for rapid and accurate responses to disasters and emergencies. However, delays in information gathering and emergency response can be problematic when it is difficult to grasp the situation or transport supplies, or in areas where communications are cut off. In the field of security services, real-time on-site monitoring and emergency response are also important, and efficient systems are needed.

[1628] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1629] In this invention, the server includes means for controlling an operable flying device to collect video and audio of the scene in real time, means for analyzing the collected data with a generative AI model to detect anomalies, means for calculating and proposing an optimal response route based on the detected anomalies, and means for controlling the operable flying device to transport necessary supplies to the scene, thereby enabling prompt and accurate situation assessment and response at the scene in the event of a disaster or security service.

[1630] An "operable flying device" is a device that has the ability to fly to a designated location, collect data such as video and audio, and transport supplies.

[1631] "Operable imaging means" means a device mounted on a flying device that has the capability to capture and record still images, video, and audio in real time.

[1632] "Artificial intelligence processing means" refers to computer systems or software that have the functionality to analyze and analyse collected data in real time.

[1633] "Processing means" refers to a system that has the functionality to calculate and generate optimal routes and action plans based on the results of analysis.

[1634] "Means for loading and transporting supplies" refers to devices or systems that have the function of loading the necessary supplies onto a flight vehicle and transporting them safely to a designated location.

[1635] "Means for deploying on-board equipment in areas where communications are cut off and collecting data on the local situation" refers to equipment that has the function of deploying equipment such as satellite phones and digital cameras in areas where communications are cut off using flying devices to grasp the local situation.

[1636] A "generative AI model" is a learning model used to analyze collected data and detect anomalies and specific patterns.

[1637] A "prompt sentence" is text data containing instructions or commands input to a generative AI model.

[1638] "Emergency response information" is information that requires a rapid response and is generated based on collected local data.

[1639] The "means for detecting anomalies" refers to an AI analysis system that analyzes collected video and audio data to detect abnormal situations or suspicious behavior.

[1640] The "means for proposing the optimal response route" is a system that calculates and proposes the most efficient route of action to security staff and rescue teams after detecting an abnormality.

[1641] In this invention, a comprehensive system is constructed to respond quickly and accurately to disasters and security services. The detailed configuration and operation of the system will be described below.

[1642] System Configuration

[1643] 1. Hardware Configuration

[1644] Flying Devices (Drones):

[1645] The flying device is equipped with a camera and microphone to collect video and audio in real time, and the drone is equipped with GPS for precise navigation to specific locations.

[1646] Camera and audio collection devices:

[1647] High-resolution cameras and sensitive microphones are used to capture detailed video and audio of the scene.

[1648] Communication equipment:

[1649] Using equipment such as satellite phones and digital cameras, it is possible to collect data and communicate even in areas where communications are cut off.

[1650] 2. Software Configuration

[1651] Artificial Intelligence Processing System:

[1652] Using a generative AI model, collected video and audio data is analyzed in real time to detect anomalies.

[1653] Route Calculation System:

[1654] It is equipped with an algorithm that calculates the optimal rescue or response route based on the analysis results.

[1655] Data Streaming System:

[1656] It provides a server that uses open source libraries (e.g., OpenCV) to stream video in real time.

[1657] Frontend application:

[1658] It provides an intuitive interface for security staff and users through smartphone apps and web applications.

[1659] Processing content

[1660] Data collection:

[1661] The drone moves to a designated location and uses a camera and microphone to collect video and audio, which is then sent to a server in real time.

[1662] Data Analysis:

[1663] The collected data is analyzed using a generative AI model on the server, and if abnormal behavior or dangerous situations are detected, an alert is sent immediately.

[1664] Route calculation and suggestions:

[1665] Based on the analysis results, the optimal response route is calculated, and this information is sent in real time to a smartphone app or web application, providing it to security staff and rescue teams.

[1666] Material transport:

[1667] If necessary, the drones will carry medicines and emergency supplies and deliver them based on the optimal route.

[1668] Specific examples

[1669] For example, if a large earthquake occurs and a certain area is isolated, the system operates as follows.

[1670] 1. Drones are quickly deployed to isolated areas and use high-resolution cameras and microphones to collect on-site information.

[1671] 2. The collected data is sent to a server in real time, where a generative AI model analyzes it to assess the extent of damage to buildings and the risk to human life.

[1672] 3. Based on the analysis results, the server calculates the optimal route for the rescue team and notifies them via a smartphone app.

[1673] 4. Drones will carry necessary medicines and food and deliver supplies to the site based on the optimal route.

[1674] 5. Users can check the situation on-site in real time through the application and respond quickly.

[1675] Prompt sentences to input to the generative AI model

[1676] Please analyze the following camera footage frames and predict an anomalous activity score:

[1677] Frame: [Camera image data]

[1678] This will enable quick and accurate situation assessment and response in the event of a disaster or security service.

[1679] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1680] Step 1:

[1681] Launching the drone and dispatching it to the site

[1682] When a user receives a notification of a disaster or emergency, they send a command to the server to dispatch a drone. The server calculates a flight route to the specified location and sends it to the drone. The drone then launches and moves to the scene along the instructed route.

[1683] Input: Disaster occurrence notification from user, coordinate information of designated location

[1684] Output: Drone flight route calculation results, drone arrival at the site

[1685] Step 2:

[1686] Data collection

[1687] The drone arrives at the scene and collects video and audio data in real time using its on-board camera and microphone, which is then sent to a server in real time.

[1688] Input: Collecting video and audio data on-site

[1689] Output: Real-time data transmission to the server

[1690] Step 3:

[1691] Data analysis

[1692] The server inputs the received video and audio data into a generative AI model for real-time analysis, which processes the data to detect abnormal behavior or dangerous situations and returns the results.

[1693] Input: Collected video and audio data

[1694] Output: Anomaly detection results (anomaly score)

[1695] Specific operation: The generative AI model analyzes video frames and outputs an abnormal behavior detection score.

[1696] Step 4:

[1697] Optimal route calculation

[1698] The server uses the analysis results to calculate the optimal route for rescue teams and security staff, and provides the calculated route information in real time to smartphone apps and web applications.

[1699] Input: Anomaly detection results, site geographic information

[1700] Output: Optimal route information

[1701] Specific operation: Combining GPS data and anomaly detection results to calculate the shortest and safest route and notify the application.

[1702] Step 5:

[1703] Material transportation

[1704] If necessary, the server sends instructions to the drone to load medicines or emergency supplies, and the drone then delivers the supplies to the site based on the optimal route.

[1705] Input: Required material information, optimal transport route information

[1706] Output: Supplies arrive at the site

[1707] Specific operation: Based on instructions from the server, the drone loads supplies and delivers them to the site.

[1708] Step 6:

[1709] Real-time monitoring and feedback

[1710] Users can check the on-site video and analysis results in real time via a smartphone app or web application and take necessary action. The server receives feedback from the user and adjusts the next action.

[1711] Input: User feedback, real-time analysis results

[1712] Output: Next action instructions

[1713] Specific operation: Based on the information provided by the user, the system responds on-site and sends feedback to the server to continuously optimize the response.

[1714] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1715] The present invention is a system for realizing a rapid and accurate initial response in the event of a disaster, and includes an operable flying device (hereinafter referred to as a drone), an operable photographing means, an artificial intelligence processing means for analyzing and analysing generated data in real time, a processing means for calculating the optimal rescue route, a means for loading and transporting supplies, a means for placing onboard equipment in areas where communications are cut off and collecting data on the local situation, a means for analyzing the collected data and generating emergency response information, and a means for linking the emergency response information to external rescue organizations, as well as an emotion engine that recognizes the user's emotions.

[1716] System programs and their processing

[1717] 1. Initial response

[1718] Server: Upon receiving notification of a disaster, it obtains map data of the affected area, weather information, evacuation shelter information, etc. Based on this information, it sends instructions to the drone, such as flight route, shooting points, and type of data to be collected.

[1719] Drone (terminal): Arrives at the scene and uses a camera and microphone to collect still and moving images and audio data, which are then sent to a server in real time.

[1720] Server: Analyzes and analyzes the data received in real time to understand the situation on site.

[1721] 2. Analysis and rescue route calculation

[1722] Server: Analyzes collected data using generative AI to identify road closures, dangerous areas, and isolated regions. Then, calculates the optimal rescue route and provides it to the rescue team.

[1723] 3. Material transportation

[1724] Server: Identifies the supplies needed based on the situation in the affected area and creates a delivery plan, sending details such as delivery route, time, and quantity to the drone.

[1725] Drone (terminal): Loads designated supplies and transports them to the disaster area via the optimal route. Once there, the supplies are safely dropped off.

[1726] 4. Responses in areas where communications have been cut off

[1727] Server: In areas where communications have been cut off, the server issues instructions for the use of satellite phones and digital cameras. These devices are then loaded onto drones and sent to the scene.

[1728] Drone (terminal): Fly to designated area and hand over device to local responders, who collect data and transmit it back to the drone.

[1729] Server: Analyzes the returned data and understands the local situation.

[1730] 5. Coordination of emergency response information

[1731] Server: Based on data from the drone, collects information necessary for an initial response within 72 hours and generates analysis results.

[1732] Server: Shares information requiring emergency response with nearby hospitals, local governments, and other related organizations, enabling rapid life-saving efforts.

[1733] Incorporating an emotion engine

[1734] 1. Collecting Emotional Data

[1735] Drone (terminal): Data collected in the disaster area includes audio and video footage of the victims. This data is sent to the emotion engine.

[1736] Server: The acquired audio and video data is passed to the emotion engine, which analyzes the emotional state of the victim.

[1737] 2. Sentiment analysis and prioritization of rescue operations

[1738] Emotion engine (on the server): Analyzes the emotional state of victims (e.g., fear, anxiety, relief) and reflects that information in planning rescue operations.

[1739] Server: Dynamically change the priority of rescue efforts based on the results of emotion analysis. For example, prioritize assistance to areas with many disaster victims who are feeling extreme fear or anxiety.

[1740] Specific examples

[1741] For example, consider a case where an earthquake with a seismic intensity of 7 occurs and some areas of an urban area are isolated.

[1742] 1. The server receives notification of a disaster and issues instructions to dispatch drones to the affected area. The drones are equipped with a camera, a generation AI processing means, and an emotion engine.

[1743] 2. The drone (terminal) arrives at the disaster area, activates its camera and recording device to collect data in real time, and transmits audio and video data to the emotion engine.

[1744] 3. The server analyzes the received data, identifies road closures and dangerous areas, calculates the optimal rescue route, and provides this information to the rescue team.

[1745] 4. The server issues instructions to drones capable of flight to load supplies such as medicines and food and deliver them to isolated areas. The drones (terminals) then safely deliver the supplies based on the optimal route.

[1746] 5. In areas where communications are cut off, the server will dispatch drones equipped with satellite phones and digital cameras to deliver the data to responders on the ground. The drones will then return the data collected by the responders to the server for analysis.

[1747] 6. Based on the collected information, the server extracts information that requires emergency response and immediately contacts hospitals and local governments. Medical and rescue teams then respond promptly.

[1748] 7. The emotion engine analyzes the voice and video data of victims to understand their emotional state, allowing the server to appropriately adjust the priority of rescue efforts and provide the necessary assistance quickly.

[1749] In this way, by using the system of the present invention, on-site information can be quickly and accurately grasped during a disaster, enabling efficient transportation of supplies and a rapid initial response. In addition, by incorporating an emotion engine, appropriate support can be provided taking into account the emotional state of the victims.

[1750] The processing flow will be explained below.

[1751] Step 1:

[1752] The server receives notifications of disaster occurrences and obtains map data of the affected area, weather information, evacuation shelter information, etc. Based on this information, a program is prepared to instruct the drone on flight routes, shooting points, and the type of data to be collected.

[1753] Step 2:

[1754] The server sends flight commands to the drone, instructing it to head towards the designated area.

[1755] Step 3:

[1756] The drone (terminal) flies to the disaster site according to instructions from the server, and upon arrival, activates its camera and recording device, collecting still and moving images and audio data, which it then transmits to the server in real time.

[1757] Step 4:

[1758] The server receives real-time data transmitted by the drone and begins analyzing it using generative AI, which identifies road disruptions, dangerous areas, and isolated regions.

[1759] Step 5:

[1760] Based on the analysis results, the server calculates the optimal rescue route to the disaster area, and provides the calculation results to the rescue team, enabling them to carry out rescue operations quickly.

[1761] Step 6:

[1762] The server determines the type and amount of supplies needed based on the situation in the disaster area, creates a supply delivery plan, including delivery routes, times, and quantities, and transmits this plan to the drone.

[1763] Step 7:

[1764] The drone (terminal) follows instructions from the server to load supplies, fly to the disaster area along the optimal route, and safely drop off the supplies upon arrival.

[1765] Step 8:

[1766] The server sends instructions to the drones to deploy satellite phones and digital cameras in areas where communications have been cut off, including specific operational procedures for gathering information on the situation on the ground.

[1767] Step 9:

[1768] The drone (terminal) flies into a designated blackout area and hands over a satellite phone and digital camera to local personnel, who use these devices to collect data and transmit it back to the drone.

[1769] Step 10:

[1770] The server receives the local situation data returned by the drone and analyzes it in real time using generative AI, extracting information necessary for emergency response.

[1771] Step 11:

[1772] The server then sends the identified emergency response information to nearby medical institutions, local governments, and related organizations in real time, enabling rapid life-saving efforts.

[1773] Step 12:

[1774] The drone (terminal) collects audio and video data from the victim and sends it to the emotion engine, which analyzes the audio and video to identify the victim's emotional state.

[1775] Step 13:

[1776] The emotion engine (in the server) analyzes the emotional state of the victims (e.g., fear, anxiety, relief) and incorporates this information into the rescue operation plan in the server.

[1777] Step 14:

[1778] The server executes a means for dynamically changing the priority of relief efforts based on the emotion information provided by the emotion engine, thereby giving priority to providing support to areas with a large number of disaster victims who are extremely emotionally unstable.

[1779] Step 15:

[1780] The users (local rescue teams) quickly carry out rescue operations on site based on real-time information and priority adjustments provided by the server. Feedback during operations is also provided to the server to help with future decisions.

[1781] Through these steps, the system of the present invention can quickly and efficiently realize initial responses in the event of a disaster. This system improves the survival rate of disaster victims and enables the appropriate supply of supplies. Furthermore, by incorporating an emotion engine, it becomes possible to provide appropriate support that takes into account the emotional state of disaster victims.

[1782] Example 2

[1783] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1784] While rapid and accurate initial response is crucial in the event of a disaster, conventional systems often delay the collection of information needed to properly grasp the situation on the ground, and rescue operations are often carried out based on inaccurate data. Furthermore, support does not take into account the emotional state of the victims, resulting in insufficient psychological care and making it difficult to properly prioritize rescue operations. To solve these issues, a system is needed that can collect and analyze on-site information in real time, calculate optimal rescue routes, and dynamically adjust the priority of rescue operations by recognizing the emotional state of the victims.

[1785] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an operable flying device, an operable photographing means, an artificial intelligence processing means for analyzing the generated data in real time, a processing means for calculating an optimal rescue route based on the analysis results, a means for loading supplies onto the flying device and transporting them, a means for deploying an onboard device in a communication-blocked area and collecting local situation data, a means for analyzing the collected data and generating emergency response information, a means for linking the emergency response information to external rescue organizations, an emotion analysis means for analyzing audio and video data of victims and recognizing their emotional states, and a means for dynamically adjusting the priority of rescue operations based on their emotional states. This enables rapid and accurate understanding of the situation at the scene of a disaster, enabling efficient rescue operations and delivery of supplies, and realizing appropriate support that takes into account the psychological care of victims.

[1786] An "operable flying device" is a device capable of airborne flight and reaching designated locations to collect data or deliver supplies.

[1787] "Operable imaging means" means a camera or other imaging device capable of capturing still or video images and attached to a flying device for data collection purposes.

[1788] "Artificial intelligence processing means for analyzing and analysing generated data in real time" refers to an artificial intelligence-based processing system used to instantly analyze and analyse collected data and grasp the situation on site.

[1789] "Processing means for calculating optimal rescue route" refers to a processing system for calculating the most efficient route for carrying out rescue operations based on the data analyzed in real time.

[1790] "Means for loading and transporting supplies onto a flying vehicle" means the mechanism or structure for loading relief supplies onto a flying vehicle and transporting them to a designated disaster area.

[1791] "Means for deploying on-board equipment in areas where communications are cut off and collecting on-site situation data" refers to methods and devices for deploying the necessary equipment and collecting on-site situation data so that data collection is possible even in areas where communications infrastructure has been destroyed.

[1792] "Means for analyzing collected data and generating emergency response information" refers to a processing system that analyzes collected data, identifies situations requiring emergency response, and generates response information.

[1793] The "means for sharing emergency response information with external relief organizations" refers to a communication means for quickly and accurately transmitting the generated emergency response information to external relief organizations.

[1794] An "emotion analysis means" is an analysis system that analyzes the audio and video data of victims to recognize their emotional state (e.g., fear, anxiety, relief).

[1795] "Means for dynamically adjusting the priority of rescue operations based on emotional state" refers to a processing system for changing the priority of rescue operations in real time based on the emotional state of the victim recognized by the emotion analysis means.

[1796] The present invention provides a system for achieving rapid and accurate initial response in the event of a disaster. The system includes an operable flight device, an operable imaging means, an artificial intelligence processing means for analyzing and interpreting generated data in real time, a processing means for calculating an optimal rescue route, a means for loading and transporting supplies, a means for deploying an onboard device in a communication-blocked area and collecting local situation data, a means for analyzing the collected data and generating emergency response information, a means for linking the emergency response information to an external rescue organization, an emotion analysis means for recognizing the emotions of disaster victims, and a means for dynamically adjusting the priority of rescue operations based on the emotional state.

[1797] When the server receives notification of a disaster, it obtains map data, weather information, and evacuation shelter information for the affected area. Based on this information, it determines the flight route, shooting points, type of data to collect, and issues instructions to the drone. The drone (terminal) arrives at the scene and uses its camera and microphone to collect video and still images and audio data, which it then sends to the server in real time. The server analyzes the data received in real time and uses a generative AI model to understand the situation. Machine learning libraries such as TensorFlow and PyTorch are used for this analysis.

[1798] The server runs an algorithm (such as the A algorithm or Dijkstra algorithm) to calculate the optimal rescue route based on the analyzed data. This identifies areas where roads are cut off, dangerous areas, and isolated regions, and provides this information to rescue teams. At the same time, it identifies necessary supplies (food, medicine, etc.) and creates a delivery plan. It issues instructions to drones capable of flying to load the supplies and deliver them to the disaster area via the optimal route.

[1799] In areas where communications are cut off, the server dispatches drones equipped with satellite phones and digital cameras and hands them over to local responders. The data collected by the local responders is then sent back to the server by drone. The server then analyzes this data to accurately grasp the local situation.

[1800] Emergency response information is compiled based on data from drones, generating the information required for initial response within 72 hours. The server then shares this information with hospitals and local governments to support rapid life-saving efforts. Furthermore, audio and video data collected in the disaster area is sent to an emotion analysis system, which analyzes the emotional state of the victims (e.g., fear, anxiety, relief) and incorporates this information into rescue operation plans. Based on this information, the server dynamically adjusts the priority of rescue operations.

[1801] As a concrete example, consider a case where a magnitude 7 earthquake occurs and isolates part of an urban area. The server receives a disaster notification and immediately dispatches a drone to collect information on the local situation. The collected data is analyzed to determine the optimal rescue route and a plan for transporting supplies. At the same time, the emotional state of the victims is analyzed and optimal support activities are carried out, taking into consideration their mental health.

[1802] Below is an example of a prompt sentence:

[1803] "A magnitude 7 earthquake has occurred, isolating parts of urban areas. Dispatch drones to quickly collect on-site information from the affected areas, calculate rescue routes, and develop an optimal plan for delivering necessary supplies. Also, analyze the emotional state of the victims using audio and video data to appropriately prioritize rescue efforts."

[1804] The above is an embodiment of the present invention, and by using this system, on-site information can be quickly and accurately grasped during a disaster, enabling efficient rescue operations and the transportation of supplies. In addition, by using emotion analysis means, appropriate support can be provided taking into account the emotional state of the victims.

[1805] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1806] Step 1: Receive notifications and prepare

[1807] The server receives notifications of disaster occurrences, which are provided in real time by meteorological agencies and monitoring systems.

[1808] Input: Disaster notification (e.g. earthquake, flood)

[1809] The server obtains map data, weather information, and evacuation shelter information for the affected area, and integrates this information to determine flight routes, photography points, and the type of data to be collected.

[1810] Data processing: Map data, weather information, and evacuation shelter information are obtained using APIs, and this is integrated to generate drone instruction information.

[1811] Output: Flight route, shooting points, type of collected data

[1812] Specific operation: The server retrieves map data using the Google Maps API, weather information from the Japan Meteorological Agency API, and evacuation shelter information from government databases.

[1813] Step 2: Dispatch the drone and collect data

[1814] The server transmits the predetermined flight route and collection items to the drone's command center.

[1815] Input: Flight route, shooting points, type of collected data

[1816] The drone (terminal) follows a designated flight route, uses a camera and microphone to collect video and audio data, and transmits it to a server in real time.

[1817] Data calculation: Using GPS, the drone flies autonomously and collects data at designated points.

[1818] Output: Real-time transmission of collected video and audio data

[1819] Specific operation: The drone flies using its built-in GPS and communication module, takes pictures with its camera, records sounds with its microphone, and uploads the collected data to a server in real time.

[1820] Step 3: Data analysis and situa...

Claims

1. an operable flight device; an operable photographing means; an artificial intelligence processing means for analyzing the generated data in real time; a processing means for calculating an optimal rescue route based on the analysis results; means for carrying and transporting supplies on the flight vehicle; a means for deploying onboard equipment in the communications-blocked area to collect local situation data; means for analyzing the collected data and generating emergency response information; means for linking said emergency response information to external relief organizations; A system including:

2. The system of claim 1 , wherein the operable flight device is configured to operate based on the analysis results.

3. 2. The system of claim 1, wherein the means for loading and transporting the material uses the generated data to select an optimal transport route.

Images