Information processing system

Abstract

The present invention provides an information processing system comprising: a means for detecting the occurrence of a disaster; the device is used for starting the plurality of unmanned aerial vehicles and collecting videos and image data of the affected area; the device is used for analyzing the collected video and image data and generating a disaster condition three-dimensional map; means for visualizing the generated three-dimensional map and distributing countermeasure guidance; the device is used for receiving information of residents and rescue workers in the refuge and extracting important information through natural language analysis; and a device for determining a rescue action instruction and a support material transportation route based on the extracted information.

Description

Technical Field

[0001] The technology disclosed herein relates to an information processing system. Background Technology

[0002] Japanese Patent Application Publication No. 2022-180282 discloses a method for controlling a role-based chatbot executed by at least one processor. The method includes the following steps: receiving a user's speech; adding the user's speech to a prompt word, the prompt word containing instruction statements associated with an explanation of the chatbot's role; encoding the prompt word; and inputting the encoded prompt word into a language model to generate a chatbot response to the user's speech.

[0003] Existing disaster response systems suffer from slow information collection and untimely data analysis during disasters, leading to delays in disaster loss assessment and relief decisions. Furthermore, the acquisition and integration of disaster information relies on manual operations, making it difficult to accurately and promptly grasp the disaster situation and evacuation needs. The allocation of support materials and the planning of transportation routes are also inefficient, impacting the overall effectiveness of relief efforts. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides an information processing system, comprising: a device for detecting disaster occurrence; a device for automatically launching multiple drones to collect video and image data of the disaster area; a device for analyzing the collected data and generating a three-dimensional disaster map; a device for visualizing the generated three-dimensional map and distributing countermeasure guidance; a device for receiving information from shelter residents and rescue personnel and extracting key information through natural language processing; and a device for determining rescue instructions and material transportation routes based on the extracted information. Through these means, rapid information collection, accurate analysis, real-time display, and intelligent rescue decision-making at disaster sites can be achieved, improving rescue efficiency and the scientific nature of material allocation.

[0005] "Disaster" refers to sudden destructive events such as earthquakes, floods, typhoons, and rainstorms caused by natural or human factors, which have a serious impact on human society, living environment, or property.

[0006] "Disaster detection devices" refers to equipment or systems that can acquire and analyze meteorological, seismic, and hydrological data in real time to determine whether a disaster event has occurred.

[0007] "Unmanned aerial vehicle" (UAV) refers to an aircraft that can fly automatically without human piloting and perform tasks such as data collection and monitoring.

[0008] "Video and image data" refers to image information of the disaster area collected by sensors such as cameras, including dynamic images and still photos, for subsequent analysis and processing.

[0009] A "3D map" is a digital map that combines various spatial data and image information to create a three-dimensional representation of the geographic spatial structure and disaster distribution.

[0010] "Visualization" refers to displaying the data obtained after analysis and processing in an intuitive form such as charts and maps, so that users can quickly understand and make judgments.

[0011] "Countermeasure guidance" refers to disaster response plans, rescue suggestions, and material allocation instructions generated based on the analysis results, which are used to guide relevant personnel to take corresponding measures.

[0012] "Refugee residents" refers to people who live in designated temporary shelters or await rescue after a disaster.

[0013] "Rescue personnel" refers to full-time or volunteer personnel who participate in disaster relief, rescue, and assistance in distributing supplies after a disaster occurs.

[0014] "Natural Language Interpretation" refers to the process of using artificial intelligence and other technologies to perform semantic understanding and element extraction on text and voice information input by users.

[0015] "Important information" refers to key data or content collected from user reports or other channels that is valuable for disaster relief decision-making and resource allocation.

[0016] "Rescue operation instructions" refer to specific operational instructions or action plans generated by the system to effectively carry out rescue, material distribution, and other tasks.

[0017] "Support material transportation routes" refer to the optimized transportation paths and related scheduling arrangements for materials from warehouses to disaster-stricken areas, based on the actual situation and needs of the disaster-stricken areas. Attached Figure Description

[0018] Figure 1 This is a conceptual diagram illustrating an example of the configuration of the data processing system according to the first embodiment.

[0019] Figure 2 This is a conceptual diagram illustrating an example of the main functions of the data processing apparatus and smart device according to the first embodiment.

[0020] Figure 3 This is a conceptual diagram illustrating an example of the configuration of the data processing system according to the second embodiment.

[0021] Figure 4 This is a conceptual diagram illustrating an example of the main functions of the data processing device and smart glasses according to the second embodiment.

[0022] Figure 5 This is a conceptual diagram illustrating an example of the configuration of the data processing system according to the third embodiment.

[0023] Figure 6 This is a conceptual diagram illustrating an example of the main functions of the data processing apparatus and head-mounted terminal according to the third embodiment.

[0024] Figure 7 This is a conceptual diagram illustrating an example of the configuration of the data processing system according to the fourth embodiment.

[0025] Figure 8 This is a conceptual diagram illustrating an example of the main functions of the data processing device and robot according to the fourth embodiment.

[0026] Figure 9 This represents an emotion map that maps multiple emotions.

[0027] Figure 10 This represents an emotion map that maps multiple emotions.

[0028] Figure 11 This is a sequence diagram illustrating the processing flow of the data processing system of the first embodiment.

[0029] Figure 12 This is a sequence diagram illustrating the processing flow of the data processing system in Application Example 1.

[0030] Figure 13 This is a sequence diagram illustrating the processing flow of the data processing system of the second embodiment.

[0031] Figure 14 This is a sequence diagram illustrating the processing flow of the data processing system in Application Example 2. Detailed Implementation

[0032] Hereinafter, an example of an implementation of the system according to the present disclosure will be described with reference to the accompanying drawings.

[0033] First, let me explain the terminology used in the following instructions.

[0034] In the following embodiments, the processor (hereinafter referred to as "processor") with reference numerals may be a single computing device or a combination of multiple computing devices. Furthermore, the processor may be a single computing device or a combination of multiple computing devices. Examples of computing devices include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), etc.

[0035] In the following embodiments, RAM (Random Access Memory), as indicated in the figures, is a memory that temporarily stores information and is used as working memory by the processor.

[0036] In the following embodiments, the memory, as indicated by the reference numerals, is one or more non-volatile storage devices that store various programs and parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), disks (e.g., hard disks), or magnetic tapes.

[0037] In the following embodiments, the communication I / F (Interface) with reference numerals is an interface that includes a communication processor and an antenna, etc. The communication I / F is responsible for communication between multiple computers. As an example of a communication specification applicable to the communication I / F, wireless communication specifications such as 5G (5th Generation Mobile Communication System), Wi-Fi (wireless fidelity) (registered trademark), or Bluetooth (registered trademark) can be listed.

[0038] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it can be only A, only B, or a combination of A and B. Furthermore, in this specification, when "and / or" connects to express more than three items, the same interpretation as "A and / or B" applies.

[0039] First Implementation Method

[0040] Figure 1 An example of the configuration of the data processing system 10 according to the first embodiment is shown.

[0041] like Figure 1 As shown, the data processing system 10 includes a data processing device 12 and an intelligent device 14. A server can be cited as an example of the data processing device 12.

[0042] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" as understood in this disclosure. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0043] The smart device 14 includes a computer 36, a receiving device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. In addition, the receiving device 38, output device 40, camera 42, and communication I / F 44 are also connected to the bus 52.

[0044] The receiving device 38 includes a touchscreen 38A and a microphone 38B, and receives user input. The touchscreen 38A receives user input via touch by detecting contact with an indicator (e.g., a pen or finger). The microphone 38B receives user input via sound by detecting the user's voice. The control unit 46A in the processor 46 sends data representing the user input received by the touchscreen 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data representing the user input.

[0045] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting data in a form perceptible to the user 20 (e.g., sound and / or text). The display 40A displays visual information such as text and images according to instructions from the processor 46. The speaker 40B outputs sound according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0046] Communication I / F44 is connected to network 54. Communication I / F44 and 26 are responsible for sending and receiving various information between processor 46 and processor 28 via network 54.

[0047] Figure 2 The diagram shows an example of the main functions of the data processing device 12 and the smart device 14.

[0048] like Figure 2 As shown, in the data processing apparatus 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the memory 32. The specific processing program 56 is an example of a "program" as understood in this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0049] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. In the emotion inference function (emotion-specific function) using the emotion-specific model 59, various inferences and predictions related to the user's emotions are performed, including inferences and predictions of the user's emotions, but this is not limited to this example. Furthermore, emotion inference and prediction may also include, for example, emotion analysis (parsing).

[0050] In the smart device 14, the processor 46 performs the acceptance output processing. The memory 50 stores the acceptance output program 60. The acceptance output program 60 is used in conjunction with the data processing system 10 and the specific processing program 56. The processor 46 reads the acceptance output program 60 from the memory 50 and executes the read acceptance output program 60 on the RAM 48. The acceptance output processing is implemented by the processor 46 acting as the control unit 46A according to the acceptance output program 60 executed on the RAM 48. Furthermore, the smart device 14 has the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and these models can also be used to perform the same processing as the specific processing unit 290. The acceptance output processing is implemented by the processor 46 acting as the control unit 46A according to the acceptance output program 60 executed on the RAM 48.

[0051] Alternatively, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. Furthermore, the data processing device 12 may be a server device or a user-held terminal device (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of the processing of the data processing system 10 of the first embodiment will be described.

[0052] Example 1

[0053] The flow of a specific process in Example 1 will be described. Each part of the system described below is implemented by the data processing device 12 and the smart device 14. Furthermore, the data processing device 12 is referred to as the "server," and the smart device 14 is referred to as the "terminal."

[0054] The specific processing performed by the specific processing unit 290 of the data processing apparatus 12 in Embodiment 1 is achieved by the following means.

[0055] In this invention, the server includes a device for detecting disaster occurrence, a device for remotely controlling multiple mobile devices and automatically collecting visual information of the disaster area via an information acquisition device, a device for generating three-dimensional geospatial information using various automated information parsing algorithms and generative artificial intelligence models, a device for automatically extracting and integrating information reported from the scene using natural language parsing algorithms, a device for automatically generating support activity instructions and optimal route planning schemes and uniformly sending them to the rescue terminal, and an intelligent device for recording the entire process of information in sequence and generating instruction statements using prompt statements. This enables rapid, comprehensive, and real-time understanding of the disaster situation, automatic generation and optimal allocation of rescue resources and transportation routes, improved timeliness and rationality of rescue decisions, and significantly enhanced intelligence and automation levels in disaster response and rescue operations.

[0056] "Device for detecting the occurrence of disasters" refers to a hardware and / or software system that can collect and analyze meteorological, earthquake, hydrological and other data related to natural disasters in real time, and determine whether a disaster has occurred.

[0057] "Mobile devices" refer to equipment that can move autonomously under driverless or remote control and is equipped with information acquisition tools to collect on-site data, such as drones and autonomous vehicles.

[0058] "Information acquisition device" refers to sensor equipment used to collect visual, audio or other types of raw information from disaster-stricken areas, including cameras, microphones, environmental sensors, etc.

[0059] "Visual information" refers to data such as static or dynamic images and videos obtained through cameras or other imaging devices that can reflect the situation on site.

[0060] "Information parsing algorithm" refers to the logical process of processing raw data, such as identification, extraction, and classification, including image recognition, signal processing, and feature extraction.

[0061] "Generative artificial intelligence models" refer to artificial intelligence systems based on technologies such as deep learning that can automatically generate and analyze data such as image reconstruction and text understanding.

[0062] "Three-dimensional geospatial information" refers to a set of digital information that is constructed through multi-dimensional data calculations and can reflect the spatial structure of a geographic region and the impact of disasters in a three-dimensional way.

[0063] "Information visualization processing device" refers to hardware or software tools used to display complex disaster analysis data to users in an intuitive way, such as 3D maps and dashboards.

[0064] "Command distribution processing device" refers to a communication and task distribution system that automatically pushes rescue strategies, tasks, or guidance information to multiple terminals based on disaster analysis results.

[0065] "Information sender" refers to a group or individual located at the disaster site who can report information such as the situation and needs at the site to the system through a terminal.

[0066] "General information communication terminal" refers to electronic devices that can be used to send and receive multimedia information, such as smartphones, tablets, and handheld wireless terminals.

[0067] "Natural language parsing algorithm" refers to an algorithm that segments, extracts, understands, and outputs structured results for natural language information such as text and speech.

[0068] "Key information" refers to the summary information from on-site feedback that, after processing, accurately reflects the core needs such as urgent disaster situations and resource shortages.

[0069] "Route planning algorithm" refers to the method and process used to calculate the optimal material transportation or action route based on parameters such as real-time maps, disaster impact, and resource status.

[0070] A "database" refers to a data management system used to store and manage various types of collected information and processing results in an orderly and structured manner.

[0071] "Prompt statements" refer to input text or commands designed to guide or improve the targeted and accurate output of generative artificial intelligence models.

[0072] "Input command statements" refer to programming statements or text commands designed for the operation of generative artificial intelligence models that can trigger specific automatic processing actions.

[0073] The embodiments of the present invention are described below.

[0074] This system aims to realize a disaster response and intelligent rescue dispatch system. The system mainly includes: a server, multiple terminals (such as unmanned vehicles, mobile information terminals, etc.), a database, and software components to support various data processing and artificial intelligence functions.

[0075] The server is equipped with devices to detect disasters. For example, the server can continuously collect and analyze relevant environmental data by installing hardware such as meteorological sensors, earthquake sensors, and flood monitoring sensors, or by connecting to external services such as meteorological agency APIs and earthquake monitoring APIs in real time through data interfaces. Through software (such as data acquisition and threshold judgment programs developed based on Python), the server can achieve rapid and automatic identification of natural disasters.

[0076] The server also includes remote control and management modules for multiple mobile devices (such as drones and automated guided vehicles). Utilizing common communication networks (such as 5G networks and wireless LANs), the server calls drone control software (such as a general drone API) to allocate and issue tasks. Terminal devices (such as drones) are equipped with high-definition cameras, sensors, and other information acquisition devices, enabling them to automatically take off and navigate autonomously under command control, collecting various types of data, including images and videos, along designated routes to the disaster-stricken area.

[0077] The server utilizes high-performance information processing devices (such as cloud servers equipped with GPUs) to perform multi-level information analysis on the collected visual information. Specifically, information analysis algorithms (such as image recognition and spatiotemporal data comparison algorithms) can be used to input the raw image data into generative artificial intelligence models, such as 3D reconstruction models developed based on TensorFlow, to compare changes in the area before and after the disaster, automatically generate 3D geospatial information, and make quantifiable judgments on the disaster area and damage status.

[0078] The server is equipped with an information visualization processing module, which can render 3D geospatial data using visualization tools (such as a general 3D geographic information display platform) to help decision-makers intuitively identify the disaster situation. The server can also automatically distribute rescue instructions and strategic suggestions obtained from artificial intelligence analysis to various rescue terminals or management terminals in the form of push notifications, emails, and SMS messages through an instruction distribution processing device, achieving real-time synchronization of information across multiple terminals.

[0079] Users (such as disaster victims and rescue workers) can directly report on the situation on-site and their needs for supplies to the server through general information and communication terminals (such as mobile-based smart applications, web forms, and remote intercom equipment). The server will input the collected various types of on-site information (such as text, voice, and images) into natural language processing algorithms (such as semantic understanding tools based on pre-trained AI models like BERT and GPT) to automatically extract key on-site information and urgent needs, and then classify and structure them for storage.

[0080] Based on the integration of all information, the server uses path planning algorithms (such as general map and navigation APIs) and combines real-time traffic information, disaster impact range, and other multi-dimensional data to automatically generate the optimal transportation routes and travel plans for relief supplies, and synchronizes them to each on-site terminal. All processed data, generated relief logs, instructions, and feedback information are uniformly managed by the database, achieving timeliness, traceability, and efficient collaboration.

[0081] In addition, to improve the accuracy of the analysis results and decisions automatically generated by the artificial intelligence model, the server is designed with a control process that automatically generates input command statements based on prompts, ensuring that the model output is highly targeted and accurate. Specific implementation examples:

[0083] For example, after an earthquake, the server acquires seismic source data in real time via the earthquake monitoring API and triggers automated drone patrol missions. The drones fly over the disaster area, collecting a large number of high-resolution images, which are then uploaded back to Amazon S3 storage via a 5G network. The server uses a generative artificial intelligence model to analyze image differences, generate 3D maps, and display them on a large screen in the command center using 3D visualization tools. Residents in shelters submit requests for "lack of drinking water" via a mobile app. The server automatically identifies this emergency information using a natural language processing AI model and uses path planning algorithms to plan the optimal route for sending relief supplies, sending the task to the nearest rescue team members via push notification.

[0084] Example of a prompt statement:

[0085] "Please explain in detail how to use generative artificial intelligence models to automatically analyze the disaster situation and generate rescue action suggestions based on images of the disaster area collected by drones."

[0086] "How can artificial intelligence be used to automatically plan material transportation routes and instruct on-site teams to execute them after a disaster?"

[0087] use Figure 11 The processing procedure is explained.

[0088] Step 1:

[0089] The server periodically acquires real-time data from sensors such as meteorological, earthquake, and flood sensors, or collects environmental data through external API interfaces. The input consists of various sensor data and information returned from third-party data interfaces. The server performs statistical analysis and threshold determination on this data. By comparing against set thresholds, it outputs a disaster warning signal and initiates subsequent automated processing. Specific actions include automatic data acquisition, parsing, and alarm event generation.

[0090] Step 2:

[0091] After detecting a disaster signal, the server assigns inspection tasks to multiple terminals (such as mobile devices like drones) via a remote control module, sending flight paths, data collection areas, and equipment operating modes. The input consists of disaster type and affected area parameters. The server generates path instructions and data collection configurations, which are then sent to the terminals. The output is the automatic task instructions received by the terminals. Specifically, this involves calling the drone control API, generating flight plans, and specifying shooting requirements.

[0092] Step 3:

[0093] The terminal (such as a drone) autonomously takes off and cruises along a designated route based on received instructions, simultaneously activating high-definition cameras and various environmental sensors to collect images, videos, and even environmental parameters of the target area. The input consists of mission instructions and the initial state of the equipment. The terminal completes autonomous navigation and multi-source information acquisition. The output is high-resolution images, videos, or other environmental data. Specific actions include flight engine self-checks, sensor activation, and real-time monitoring.

[0094] Step 4:

[0095] The terminal, via its built-in 5G communication module, uploads the collected raw data in real-time or periodically to a cloud database (such as an object storage service) designated by the server. The inputs are the collected data and the target cloud storage address. The terminal compresses and encrypts the data before uploading. The output is an upload confirmation signal and the URL of the data's cloud storage object. The specific actions involve data packaging, encryption, and network transmission.

[0096] Step 5:

[0097] The server monitors the arrival of new data in the cloud database and automatically triggers data preprocessing procedures, such as format conversion, defective image screening, and image denoising. The input is the newly uploaded raw data; the server performs data cleaning and standardization, and the output is a high-quality, usable dataset. Specific actions include image renaming, resolution adjustment, and data index label generation.

[0098] Step 6:

[0099] The server inputs the processed data into a generative artificial intelligence model, which uses a 3D reconstruction model trained with TensorFlow to analyze the disaster situation. The input is pre-processed, high-quality visual data. The server performs data calculations such as 3D map reconstruction, target area identification, and damage assessment, outputting 3D geospatial information, disaster analysis results, and a disaster distribution report. Specific actions include model inference, spatial data stitching, and structured output.

[0100] Step 7:

[0101] The server utilizes an information visualization and push module to display the generated 3D map and rescue suggestions through a data display platform, and synchronizes instructions and strategy content to relevant rescue terminals. Inputs include 3D geospatial data and strategy text. The server renders the data into intuitive charts and automatically distributes them to various user devices. Outputs include an intuitive map, rescue suggestions, and task assignments displayed on the terminals. Specific actions include 3D rendering, information push, and display refresh.

[0102] Step 8:

[0103] Users (including evacuees and rescue workers) can submit information such as the actual situation on-site and requests for supplies to the server at any time through mobile applications, web pages, or chatbots. Input consists of multimodal data such as text, images, or audio entered by the user, which the server receives, categorizes, and saves. Output is the on-site feedback information received by the server. Specific actions include filling out mobile forms, uploading images, and one-click submission.

[0104] Step 9:

[0105] The server automatically extracts key information from the received on-site information using natural language processing algorithms (such as BERT and GPT) to determine urgent needs and disaster hotspots. The input is collected multimodal on-site data; the server automatically performs semantic understanding, event extraction, and other processing, outputting structured demand items and priorities. Specific actions include text segmentation, intent recognition, and tag classification.

[0106] Step 10:

[0107] Based on all disaster situation and on-site demand data, the server uses a path planning algorithm to comprehensively calculate the optimal material transportation route and automatically generate rescue mission instructions. Inputs include disaster area information, material warehouse coordinates, and road information. Server output includes specific routes, distribution lists, and corresponding rescue missions. Specific actions include path modeling, route optimization, and task parameter combination.

[0108] Step 11:

[0109] The server uses a push notification system to synchronize newly generated rescue instructions, resource allocations, and optimal routes to the terminals of various rescue teams and management personnel in real time via notifications, SMS messages, or app push notifications. Inputs are task instructions and route data, and outputs are operational guidelines received by rescue personnel on their terminals. Specific actions include push message encapsulation, client message display, and feedback receipts.

[0110] Step 12:

[0111] During task execution, terminals and users can upload execution status, encountered problems, and newly discovered on-site conditions at any time. The server collects feedback data in real time and automatically adjusts subsequent tasks and assignments, forming a data-driven dynamic positive loop. The input is task execution feedback information; the server corrects the assignment scheme based on the feedback; and the output is updated instructions and real-time situational data. Specific actions include status uploading, feedback reception, and automatic task flow optimization.

[0112] Application Example 1

[0113] The process flow corresponding to the specific processing in Use Case 1 will be described below. The various parts of the system described below are implemented by the data processing device 12 and the intelligent device 14. Furthermore, the data processing device 12 is referred to as the "server" and the intelligent device 14 is referred to as the "terminal".

[0114] Existing disaster response systems suffer from technical challenges such as low efficiency, poor collaboration, slow response, and inability to fully integrate spatiotemporal data with emotional urgency analysis in areas such as real-time disaster information collection, accurate disaster situation analysis, intelligent optimization of support material transportation routes, and multi-terminal coordinated distribution. These challenges result in low disaster relief efficiency and make it difficult to meet the actual needs of disaster victims in a timely and efficient manner.

[0115] The specific processing performed by the specific processing unit 290 of the data processing apparatus 12 in Application Example 1 is achieved by the following means.

[0116] In this invention, the server includes: a module for acquiring observational data through an information processing device and automatically determining the occurrence of a disaster; a module for activating multiple remotely operated flight devices and acquiring images and information about the disaster area; a module for storing the acquired image and information in a storage device, then analyzing and comparing it with a data analysis device and a generative artificial intelligence model using prompts to generate a three-dimensional disaster map; a module for visualizing the three-dimensional map and analysis results, automatically generating disaster response guidance, and distributing it to various terminals; a module for receiving and analyzing information and emotional content uploaded by shelters or rescue personnel; and a module for optimally planning support routes based on the aforementioned multi-source information and issuing rescue instructions in real time. This allows for efficient and accurate assessment of the disaster situation during an event, and, combined with multi-source real-time data and automated artificial intelligence analysis, enhances the intelligence and responsiveness of rescue instructions and support logistics, significantly improving disaster response speed and the effectiveness of resource allocation.

[0117] "Information processing device" refers to an electronic device used to collect, process, and analyze various types of observation data to automatically determine the occurrence of disasters.

[0118] "Remotely operated flight devices" refer to flight platforms that can perform information collection tasks in designated areas through remote control or autonomous navigation, such as drones.

[0119] "Image information" refers to video data of disaster areas acquired through image acquisition equipment, including real-time video streams and recorded files.

[0120] "Image information" refers to visual data such as static pictures and photos of disaster areas acquired through image acquisition equipment.

[0121] "Storage device" refers to hardware equipment or cloud storage platform that can save and manage data such as image information and picture information.

[0122] "Data analysis device" refers to computing equipment used to automatically process, identify and compare collected data, including computing resources that support artificial intelligence model operations.

[0123] "Artificial intelligence model" refers to a machine learning or deep learning system that can perform data analysis, comparison, feature recognition, or 3D reconstruction based on input data and prompts.

[0124] "Prompt statements" refer to natural language text or structured commands input into an artificial intelligence model to guide it in performing specific analytical tasks.

[0125] "Three-dimensional map data" refers to a three-dimensional digital map that describes the spatial structure and attributes of a disaster area after analyzing and reconstructing multi-dimensional information of the area.

[0126] "Command management terminal" refers to the operating terminal equipment used to display, manage and distribute disaster maps and response instructions, including computers, tablets, etc.

[0127] "Disaster Response Guidance Information" refers to emergency response recommendations automatically generated based on disaster-affected area analysis, including action plans and resource allocation strategies.

[0128] "Input device" refers to a human-computer interaction device, such as a mobile terminal or computing terminal, that allows personnel in shelters or on-site responses to input situation reports and requests for information.

[0129] "Natural Language Processing Unit" refers to software or modules that can perform word segmentation, semantic understanding, sentiment extraction, and key information extraction on user-input text information.

[0130] "Emotional elements" refer to information about subjective expressions such as emotions and urgency obtained from user input, including descriptions of anxiety, tension, and urgency.

[0131] "Plan generation method" refers to an automated processing module or algorithm unit that calculates and optimizes material transportation routes and rescue instructions based on multi-source information.

[0132] "Transportation route" refers to the specific path of support resources from the distribution point to the demand point, including relevant information about the route planning results.

[0133] "Support instruction information" refers to specific action instructions generated based on the analysis results, which are used to guide relevant personnel in carrying out rescue and resource allocation work.

[0134] "Relevant terminals" refers to all electronic devices capable of receiving, displaying, or executing information and instructions issued by the system, including rescue personnel terminals and management center terminals.

[0135] This invention can be implemented in the following specific ways.

[0136] This system includes: an information processing server, multiple remote-controlled flight terminals (such as drones), an image acquisition device, a storage device (such as a cloud storage server), a data analysis device (such as a high-performance computer), artificial intelligence model components, an instruction management terminal (such as a workstation or mobile terminal), an input device (such as a user's mobile terminal, tablet, or computer), and software modules with natural language parsing and sentiment analysis capabilities.

[0137] In this embodiment of the invention, the server is responsible for several core functions. The server employs a high-performance processor and large-capacity storage, running a data acquisition and coordination program based on an operating system (such as Linux). The server integrates with meteorological information sources, earthquake and flood sensors through interfaces, receiving and processing observation data in real time. When disaster conditions are triggered, the server invokes a drone management system (such as a software platform like QGroundControl) to automatically issue takeoff and mission commands to multiple drone terminals.

[0138] The terminal (such as a drone) is equipped with a high-definition camera, communication module, and automatic navigation control unit. If commercially available drone hardware is used, after receiving instructions from the server, it automatically takes off along a pre-set route and collects high-definition video and photos within the designated disaster area. The image data collected by the terminal is transmitted back to the server in real time via 4G / 5G / wireless networks.

[0139] The server performs preliminary verification, classification, and deduplication of the received image and video data, and then stores it in the cloud (such as Amazon S3, Alibaba Cloud OSS, etc.). The server then invokes data analysis devices and generative artificial intelligence models (such as deep learning-based API platforms) to intelligently analyze and compare the collected pre- and post-disaster spatiotemporal data along with prompts, automatically generating 3D map data containing spatial damage markers. During this process, the server combines existing 3D reconstruction software (such as Open3D, Bentley ContextCapture, etc.) to complete the synthesis and output of the 3D model.

[0140] The server also features visualization and distribution capabilities, utilizing web visualization platforms (such as Three.js, CesiumJS, ArcGIS, etc.) to display 3D maps in real time on command and management terminals (such as management center computers or emergency site tablets). Based on the analysis results of artificial intelligence models, the server automatically generates disaster response guidance information and sends it to the terminals of relevant rescue personnel via instant messaging or dedicated software.

[0141] Users (such as shelter residents and rescue team members) can report from the scene via a dedicated app or web interface installed on their smartphones, tablets, or laptops. Users can report real-time conditions and specific material needs through manual input, voice-to-text conversion, and uploading photos. All information is uploaded to the server via encrypted communication.

[0142] After receiving user input, the server calls a natural language processing module (such as a model based on open-source NLP frameworks like BERT) to extract keywords and core requirements. The server also integrates sentiment analysis software (such as a machine learning-based sentiment recognition module) to automatically determine the emotional characteristics of the submitted information and assign higher response priority to needs with high urgency, high anxiety, or unease.

[0143] The server integrates 3D maps, real-time traffic conditions, user requests, and emergency supplies database information. It applies path planning algorithms (such as Dijkstra's algorithm, A* algorithm, and reinforcement learning models) to automatically calculate the optimal supply delivery route and scheduling plan for each rescue request. All rescue instructions and delivery routes are distributed to relevant terminals such as logistics centers and on-site rescue personnel via push notifications and dedicated software, ensuring that every request is met efficiently and accurately.

[0144] Specific examples: When an earthquake occurs, the server promptly analyzes the waveform data from seismic sensors and determines that a disaster has occurred. The server immediately activates drone terminals to collect video and image data of the damaged area. The server calls a generative artificial intelligence model to analyze the data and generate a 3D disaster map, marking situations such as collapsed buildings and blocked roads. Users report text and images such as "Insufficient water supply in the shelter, two people showing symptoms of dehydration" through a mobile app. After natural language understanding and sentiment analysis, the server determines this to be a high-priority request and automatically pushes the detection results and dispatch instructions to the nearest logistics and rescue personnel terminals through the command management terminal, prioritizing the delivery of necessary supplies and tracking the execution status.

[0145] Examples of prompts for generative artificial intelligence models are as follows:

[0146] "Based on the images and videos of the disaster area collected by drones, please generate a 3D disaster distribution map. Please compare the information before and after the disaster, automatically mark collapsed buildings and closed roads, and output the coordinates and damage level of the affected points."

[0147] "Please analyze the following text information reported by the disaster-stricken shelters, extract the most urgent material needs and user sentiment, use this information to adjust the rescue priority, and output the shelter number, material type, urgency level, and related sentiment."

[0148] Through the combination of the aforementioned hardware, software, and intelligent algorithms, the system can automatically, efficiently, intelligently, and safely complete the entire process from disaster occurrence to the issuance of rescue orders and the distribution of supplies, thus significantly improving the disaster emergency response capability.

[0149] use Figure 12 The processing procedure is explained.

[0150] Step 1:

[0151] The server collects data in real time from multiple information sources, including meteorological data, earthquake sensors, and flood monitoring equipment, and performs data aggregation and analysis. The input is raw observation data from various sensors. The server standardizes the data format and performs anomaly detection. If an earthquake magnitude or rainfall exceeding a threshold is detected, a disaster is determined. The output is a disaster event determination signal.

[0152] Step 2:

[0153] The server sends a start command, which remotely controls multiple terminals (drones) to automatically take off and assigns routes and tasks to disaster areas. The input consists of disaster assessment signals and preset drone mission parameters. The server then distributes the tasks to the terminals. Upon receiving the command, the terminals automatically perform electronic self-checks and take off. The output includes drone start confirmation and route execution status data.

[0154] Step 3:

[0155] The terminal (drone) flies along a designated route, periodically taking pictures and videos of the disaster area using its onboard camera, and transmitting them back to the server in real time via a wireless network. The input is the image and video streams captured by the drone's camera. The terminal performs local compression and preliminary deduplication on the collected data before uploading it. The output is a compressed and optimized data packet containing video and images.

[0156] Step 4:

[0157] The server receives and archives all data uploaded by terminals, storing it in the cloud. Input consists of image and video files uploaded by drones. The server automatically tags the data according to drone number and aerial capture time, checks image clarity, converts formats (e.g., to JPG and MP4), and archives the data by category. Output is a structured, multi-tagged archive of image and video data.

[0158] Step 5:

[0159] The server invokes a generative artificial intelligence model to analyze archived image and video data. Input consists of multi-timepoint images and videos of the disaster before and after, along with relevant warning messages. The server submits the data and warning messages to the AI ​​model for feature detection, such as building collapses and road blockages, and for pre- and post-disaster comparative analysis. Output includes 3D disaster spatial data, a damage level list, and a tagged analysis results file.

[0160] Step 6:

[0161] The server uses spatial data reconstruction software to visualize the coordinates of damaged buildings, obstructed roads, and other points output by AI into a 3D disaster distribution map, which is then displayed on the command terminal and management center. The input is AI-generated 3D spatial data, which the server loads onto the visualization platform and automatically adds highlighted markers. The output is a dynamic 3D map and a disaster distribution visualization layer.

[0162] Step 7:

[0163] Users (such as shelter managers and rescue workers) can manually or verbally input information about the actual situation in the disaster area and their requests for supplies, as well as upload photos of the scene, through an app or web terminal. Input includes user text descriptions, images, and geolocation information. Output is formatted on-site report data.

[0164] Step 8:

[0165] The server processes user input using natural language processing and sentiment analysis modules. The input consists of the user-reported text and images from step 7. The server performs word segmentation, entity recognition, and sentiment scoring on the text to extract the required material type, quantity, and urgency. The output is structured demand information and associated sentiment tags.

[0166] Step 9:

[0167] The server integrates all demand information with a 3D disaster map and real-time road network conditions, and applies route optimization algorithms to formulate a material distribution plan. Inputs include a structured demand table, 3D spatial distribution, road condition data, and material inventory information. The server runs route optimization algorithms (such as A* or genetic algorithms) to make decisions. Outputs include the optimal delivery route and a distribution execution list.

[0168] Step 10:

[0169] The server sends delivery instructions, routes, and disaster response suggestions to logistics and rescue terminals in real time via push service. Input consists of the pre-defined delivery route and emergency instructions. The server sends messages and monitors receipts; upon receiving the information, the terminal initiates actual rescue operations. Output includes confirmation of instruction receipt and feedback on the execution status.

[0170] Step 11:

[0171] Users (rescue personnel) move and deliver supplies according to received instructions. After completing the task, they provide real-time feedback on the completion status and on-site photos via the terminal. Input is the task confirmation command; users operate the terminal to record the status and re-upload on-site evidence. Output is the task completion record and actual rescue feedback data.

[0172] Alternatively, an emotion engine for inferring user emotions can be combined. That is, the specific processing unit 290 can also use the emotion-specific model 59 to infer user emotions and perform specific processing using user emotions.

[0173] Example 2

[0174] The flow of a specific process in Example 2 will be described. Each part of the system described below is implemented by the data processing device 12 and the smart device 14. The data processing device 12 will be referred to as the "server," and the smart device 14 as the "terminal."

[0175] During natural disasters, it is crucial to quickly and accurately assess the actual damage in affected areas and make dynamic decisions regarding rescue and resource allocation based on real-time changes. Traditional manual information collection methods suffer from problems such as delayed information transmission, inaccurate judgments, and slow emergency response, making it difficult to meet the requirements of rapid, scientific, and efficient rescue efforts. Furthermore, the lack of effective identification of the actual needs and emotional states of individuals on-site leads to irrational allocation of rescue resources, reducing rescue efficiency and post-disaster satisfaction. Therefore, there is an urgent need for a comprehensive emergency response system that can automatically detect disasters, intelligently collect and analyze disaster area data, and efficiently make rescue decisions and allocate resources based on artificial intelligence.

[0176] The specific processing performed by the specific processing unit 290 of the data processing apparatus 12 in Embodiment 2 is achieved by the following means.

[0177] In this invention, the server includes a device for automatically detecting disasters based on various time-series observation data collected by environmental information acquisition devices and external information acquisition devices; a device for automatically activating multiple remotely controlled mobile devices and collecting disaster area data through their image acquisition devices; a device for storing real-time acquired multidimensional data and comparing pre-disaster and post-disaster information using image analysis devices and generative artificial intelligence models to reconstruct three-dimensional spatial information; a device for visually displaying three-dimensional spatial information of the disaster area and generating countermeasure guidance; and a device for receiving unstructured information from on-site personnel, extracting key needs through natural language processing and emotion recognition, automatically calculating rescue operations and material delivery routes by combining path optimization and artificial intelligence, and pushing rescue instructions in real time through a notification device. This allows for rapid, automatic, and accurate acquisition and analysis of the actual situation in the disaster area after a disaster occurs, scientific allocation of limited resources, and intelligent adjustment of rescue priorities based on on-site needs and emotional urgency, achieving an efficient and intelligent disaster emergency rescue response.

[0178] "Environmental information acquisition device" refers to hardware equipment or system that can automatically collect various sensor information related to environmental conditions (such as meteorology, earthquakes, water levels, temperature, etc.).

[0179] "External information acquisition device" refers to equipment or program used to collect relevant auxiliary information (such as meteorological API data, pre-disaster historical records, etc.) from external systems, network platforms or remote databases.

[0180] "Time series observation data" refers to a series of quantitative or qualitative data values ​​collected and recorded continuously in chronological order from sensors or data sources.

[0181] "Remotely controlled mobile entities" refer to intelligent hardware devices that can move and perform tasks within a designated area through remote or automatic control, including drones and robots.

[0182] "Image acquisition device" refers to a hardware unit capable of acquiring video or still image data, including cameras, camcorders, etc.

[0183] "Information storage device" refers to a physical or virtual storage system used to store various collected data, structured information and processing results, such as disk arrays, databases, cloud storage, etc.

[0184] "Image analysis device" refers to a software platform or hardware unit that can automatically process acquired image data, such as feature analysis, target detection, and structure reconstruction.

[0185] "Generative artificial intelligence models" refer to artificial intelligence algorithm systems that are trained and can automatically generate output results such as text, 3D models, and solution suggestions based on input data, such as deep learning models.

[0186] "Three-dimensional spatial information" refers to a collection of digital information obtained by analyzing and reconstructing various data, which can express the structure and attributes of real space in three-dimensional coordinates.

[0187] "Visualization processing device" refers to hardware equipment and software systems that can intuitively display abstract data, analysis results, etc. in the form of graphics, images, three-dimensional models, etc.

[0188] "Information input terminal" refers to electronic devices that allow users or on-site staff to input text, voice, images, and other information, including smartphones, tablets, and terminals.

[0189] "Natural Language Processing Device" refers to a software platform or module capable of performing computational processing on natural language text, such as word segmentation, entity recognition, sentiment analysis, and information extraction.

[0190] "Emotion recognition processing" refers to the process and tools used to analyze the emotional tendencies and psychological states contained in text, audio, images, and other content.

[0191] "Structured data" refers to digital information that is organized in a predefined format and is easy to store and retrieve in a database.

[0192] "Path optimization algorithm" refers to the methods and programs used to automatically calculate the optimal resource allocation or driving route under multiple objectives and constraints.

[0193] "Notification device" refers to a hardware device or software system that can transmit generated instructions, prompts, alarm information to a designated recipient in real time via push notifications, pop-ups, audio, or other means.

[0194] This invention relates to a comprehensive emergency response system and its implementation method, which enables efficient decision-making and dissemination of information, intelligent data analysis, rescue instructions, and material dispatch in disaster-stricken areas during natural disasters. Specific implementation methods are as follows:

[0195] Servers can utilize high-performance information processing devices, such as general-purpose industrial-grade servers. Information storage can be based on cloud storage systems (such as general-purpose object storage services). Image analysis primarily employs open-source or commercial image processing libraries and 3D modeling platforms (such as OpenCV, Structure from Motion, etc.). Generative AI models can utilize mature deep learning platforms in the industry (such as models based on general-purpose convolutional neural networks, or existing open large-scale language model platforms). Natural language processing and emotion recognition can utilize mainstream AI-API platforms. Path optimization can be implemented using general-purpose optimization algorithm libraries (such as Google OR-Tools, etc.). The terminal (remotely controlled mobile device) preferably uses drones or ground robots that support the SDK, equipped with high-definition cameras, data transmission modules (Wi-Fi / 4G / 5G), and positioning devices (GPS). Users can provide on-site information and requirements through input terminals such as smartphones and tablets.

[0196] The server is responsible for periodically collecting real-time data on various types of weather, earthquakes, and water levels from environmental and external information acquisition devices. Through programming, it implements threshold monitoring to quickly detect disasters. Once the detection results exceed the warning threshold, the server immediately sends automatic start and flight path arrangement instructions to the terminal (such as a drone). After receiving the instructions, the terminal automatically navigates to the disaster area according to the set path, collects video and photos using its onboard image acquisition device, and transmits them back to the server in real time via the data communication module. After receiving the data, the server uses OpenCV to filter out invalid / low-quality data, selecting valid image materials for uploading to object storage.

[0197] The server inputs this data into generative artificial intelligence models (such as deep learning for image segmentation and 3D reconstruction), combines it with historical pre-disaster data to perform pre- and post-disaster comparative analysis, automatically generates 3D spatial information, and visualizes it through platforms such as Cesium.js. Based on the damage analysis, the server also uses natural language processing to parse and assess the sentiment of unstructured information such as user-input requests and descriptions, thereby determining the needs and urgency of various shelters and rescue workers, and extracting structured data. These structured needs and sentiment analysis results are then fed into optimization algorithms and generative artificial intelligence models to automatically plan rescue and material delivery routes. Instructions generated by the server are sent to frontline emergency terminals in real time via a push module.

[0198] Users submit their current location, material needs, and emotional requests to the server via smart terminals or integrated chatbots. The text and images entered by users automatically trigger natural language analysis and sentiment recognition in the background. An AI system automatically filters high-priority requests, ensuring that various rescue and support missions are tailored to local conditions and responded to at different levels. The terminals continuously report location and mission progress, and the server dynamically adjusts rescue instructions and delivery routes based on on-site feedback.

[0199] Specific example

[0200] For example, when an earthquake occurs in a region, the server detects a sudden change in seismic sensor data and immediately directs multiple drones to take off and conduct aerial photography in different areas, transmitting high-definition images and videos back to cloud storage in real time. The server uses generative artificial intelligence models to automatically analyze building collapses and road damage in various areas, reconstructing a 3D disaster map and visually displaying it on a large screen in the command center. Users can upload text messages such as "We need drinking water and medicine; we are very anxious" via the app. The server identifies the urgent needs, automatically prioritizes the shortest delivery routes for supplies, and pushes them to relevant rescue personnel in real time.

[0201] Example of a prompt statement:

[0202] "Please analyze the uploaded drone images of the disaster area, generate a 3D map, and mark the collapsed areas."

[0203] “Extract the required supplies and urgency level from the following shelter requests, and plan the fastest supply transportation route.”

[0204] "Please automatically generate a 3D spatial model of the damaged area based on real-time collected images and historical data."

[0205] "Analyze user messages, determine the urgency of emotions, and provide suggestions on the priority of resource allocation."

[0206] use Figure 13The processing procedure is explained.

[0207] Step 1:

[0208] The server periodically collects various time-series observation data from environmental information acquisition devices (such as meteorological sensors and seismic sensors) and external information acquisition devices (such as meteorological data APIs). The input consists of various types of real-time monitoring data. The server performs data processing operations such as formatting and threshold judgment on the collected data. The output is a judgment signal indicating whether a disaster has been detected and the corresponding area code.

[0209] The specific actions are as follows: the server automatically pulls real-time weather and earthquake data every minute, parses the data packets, and checks whether the main parameters exceed the preset threshold. If they do, the emergency process is activated.

[0210] Step 2:

[0211] The server automatically assigns flight missions to terminals (such as drones) based on the disaster-affected area code, generating control commands including target coordinates, flight path information, and shooting instructions. The input is the geographic information of the disaster-affected area. Data processing includes flight path planning and mission list compilation. The output is control commands with mission parameters.

[0212] The specific actions are as follows: the server calls the drone API to push detailed tasks to each drone controller, enabling multiple drones to take off automatically on demand.

[0213] Step 3:

[0214] The terminal (such as a drone) receives control commands and automatically flies over the disaster area according to a set route, using its onboard camera to collect real-time video and photos. The input is the drone's control parameters. During flight, the drone performs data collection and preliminary screening, and the output is the raw image and video data it has captured.

[0215] The specific actions are as follows: the terminal moves along a designated route, automatically captures images at regular intervals, and adjusts the shooting angle as needed to improve coverage.

[0216] Step 4:

[0217] The terminal uploads image and video data to the server in real time via a data communication module (such as 4G / 5G / Wi-Fi). The input is the raw multimedia data collected. The server calls the data receiving and verification module to perform data integrity checks and preliminary format conversion. The output is a usable, clear image and video file.

[0218] The specific actions are as follows: the terminal automatically connects to the server data interface, and the breakpoint resume technology ensures that large amounts of data are transmitted safely and error-free in batches.

[0219] Step 5:

[0220] The server uses image analysis software (such as OpenCV) to process the received image data, performing quality checks, redundancy removal, and keyframe extraction. The input is raw image and video data. The output is a filtered dataset of high-quality images.

[0221] The specific actions are as follows: the server automatically processes all image files in batches, deletes blurry or redundant data, and only retains the most effective data for storage.

[0222] Step 6:

[0223] The server inputs the filtered data into a generative artificial intelligence model, compares and analyzes pre-disaster and post-disaster data, and reconstructs the three-dimensional spatial information of the disaster-stricken area. The input consists of high-quality filtered image data and historical reference information. Data calculations such as image recognition, region classification, and 3D modeling are performed. The output is a 3D digital model containing disaster elements such as building collapses and road interruptions.

[0224] The specific action is as follows: the server automatically identifies the damaged target through an AI model and integrates its geographical coordinates and damage level into a three-dimensional digital map.

[0225] Step 7:

[0226] The server uses visualization processing devices (such as web-based 3D map platforms) to intuitively present 3D digital models and disaster elements to the command terminal. Input consists of 3D digital models and disaster analysis results. Data processing includes graphic rendering and map labeling. Output is a disaster map and analysis report that can be displayed interactively in real time.

[0227] The specific actions are as follows: the server automatically pushes the latest 3D map to the large-screen display terminal, and the disaster area is highlighted with different colors.

[0228] Step 8:

[0229] Users upload on-site descriptions and needs through information input terminals (such as smartphone apps), and the server receives unstructured information such as text, images, or voice. The input consists of the information recorded by the user. The server uses natural language processing and sentiment analysis to segment the information, extract key points, and assign sentiment scores. The output is structured information on needs and sentiment.

[0230] The specific actions are as follows: when a user submits a request such as "We urgently need drinking water", the server automatically identifies it as a high-priority request, structures it, and pushes it into the database.

[0231] Step 9:

[0232] The server inputs all structured demand and sentiment level data into a path optimization algorithm and a generative artificial intelligence model, automatically calculating the optimal material delivery routes and rescue dispatch plans. The input consists of real-time demand data from all shelters and rescue teams. The data is prioritized and optimized for allocation. The output is detailed rescue instructions and delivery routes.

[0233] The specific actions are as follows: the server automatically generates a detailed route and task allocation plan for "first delivering drinking water to point A, then proceeding to point B".

[0234] Step 10:

[0235] The server pushes the aforementioned scheduling results and rescue instructions to on-site terminals in real time via notification devices (such as app push notifications, SMS platforms, etc.). Input consists of rescue operation plans and route information. Data processing involves message packetization and push scheduling. Output consists of immediate action instructions received by rescue personnel and material transport terminals.

[0236] The specific actions are as follows: the server pushes instructions such as routes and delivery quantities to the designated rescue team's mobile phone and vehicle terminal, and waits for execution and subsequent feedback.

[0237] Application Example 2

[0238] The process flow corresponding to the specific processing in Use Case 2 will be described below. The various parts of the system described below are implemented by the data processing device 12 and the intelligent device 14. In addition, the data processing device 12 is referred to as the "server" and the intelligent device 14 is referred to as the "terminal".

[0239] Existing disaster emergency information collection and processing systems suffer from problems such as information collection delays, omissions of important information, inability to efficiently integrate multi-source data, and difficulty in providing optimal support instructions based on users' emotional states. These issues result in untimely and inaccurate post-disaster relief responses, reducing rescue efficiency and effectiveness.

[0240] The specific processing performed by the specific processing unit 290 of the data processing apparatus 12 in Application Example 2 is achieved by the following means.

[0241] In this invention, the server includes a device for detecting disaster occurrence, a device for launching multiple aircraft and collecting image information of the disaster area, a device for analyzing the collected image information and generating three-dimensional information about the disaster situation, a device for visualizing and distributing response instructions, a language processing and sentiment analysis device for receiving and processing information from users of shelters and rescue personnel, a device for deciding on support activity instructions and supply routes based on importance and priority, and a device for using a generative artificial intelligence model to perform each processing step and generate prompt statements. This enables a rapid and accurate understanding of the disaster site situation and the actual needs of users, and optimizes rescue instructions and resource allocation through efficient automation, thereby improving the comprehensiveness and intelligence of disaster emergency response.

[0242] "Disaster detection" refers to the process or function of monitoring and determining in real time whether a natural disaster (such as an earthquake, flood, or rainstorm) has occurred through sensors, data interfaces, or information acquisition modules.

[0243] "Aircraft" refers to unmanned aerial vehicles that can fly autonomously or remotely, carry image acquisition equipment, and perform aviation missions, including drones.

[0244] "Image information" refers to still images, video data, and related metadata of the disaster-stricken area collected by equipment such as cameras and video cameras.

[0245] "Three-dimensional information" refers to three-dimensional maps, three-dimensional models, or three-dimensional reconstruction data generated based on image information, which can express the disaster situation in a three-dimensional spatial form.

[0246] "Visualization" refers to the process of presenting data or analysis results in an intuitive way on the user's terminal, such as charts, maps, and models, so that users can clearly understand the current situation of the disaster.

[0247] "Response instructions" refers to the instructions, such as action plans, operation routes, and material delivery paths, automatically generated by the system for organizing disaster relief and resource allocation.

[0248] "User information" refers to on-site descriptions, requests for help, or feedback information submitted by people in shelters or participants in rescue activities through the client.

[0249] "Language information processing" refers to the technical process of automating natural language text processing, such as word segmentation, semantic understanding, and key information extraction.

[0250] "Emotional information analysis" refers to the process of analyzing textual information to identify emotional states such as urgency, anxiety, and fear.

[0251] "Priority" refers to the standards for ranking and classifying various rescue requests or operations based on factors such as the severity of the disaster and user sentiment.

[0252] "Support Activity Instructions" refer to rescue suggestions, operational tasks, or division of labor automatically generated by the system based on important information and priorities.

[0253] "Supply route" refers to the optimal transportation path automatically generated for relief supplies, equipment and other resources to reach designated shelters from the point of origin.

[0254] Generative artificial intelligence refers to artificial intelligence technology that can automatically generate text, images, analysis reports, and other content based on input data and prompts.

[0255] "Prompt statements" refer to natural language or formatted statements that have clear instructive meanings and are generated to guide artificial intelligence in correctly processing and analyzing input data.

[0256] "Data storage" refers to the process of systematically storing various types of raw data, analysis results, model files, etc., on data carriers (such as local storage and cloud platforms).

[0257] "Distributed processing" refers to a technical approach that enables data analysis, task execution, and other functions in a system to be completed collaboratively and in parallel across multiple devices or nodes in order to improve efficiency and reliability.

[0258] To facilitate others' understanding and implementation of this invention, the following detailed description of the embodiments of this invention is provided in conjunction with specific hardware, software, data processing methods, and examples.

[0259] This invention relates to a disaster emergency response system, in which a server, terminals, and users each perform different functions. The server is primarily responsible for data collection, processing, judgment, scheduling, and information distribution. Terminals (including unmanned aerial vehicles, command terminals, and on-site mobile terminals) assist in data collection and information reception. Users, as shelter personnel or rescue workers, proactively report disaster situations and needs through applications and other means.

[0260] In this system, the servers utilize a general-purpose computing platform, which can be a multi-core server cluster based on x86 or ARM architecture, and the operating system can be Linux. The servers integrate a data parsing and management system (such as a Python-based processing framework), an artificial intelligence platform (deploying generative artificial intelligence models, such as software based on large language models and multimodal AI models), database servers (such as MySQL and PostgreSQL), and a distributed storage system (such as object storage and distributed file systems).

[0261] Front-end terminals include, but are not limited to, unmanned aerial vehicles (UAVs) equipped with image acquisition equipment and wireless communication modules (such as multi-rotor UAVs), desktop terminals (PCs, displays) in command centers, and smartphones or tablets used by personnel in on-site rescue or shelters. UAVs are equipped with high-resolution cameras, enabling them to autonomously acquire images along predetermined flight paths and transmit the data back to the server in real time via mobile cellular networks or dedicated wireless networks.

[0262] The system's software component mainly includes the following components:

[0263] 1. Data Acquisition Module: Communicates with various types of sensors (such as earthquake sensors, meteorological APIs, and image camera equipment) to achieve real-time monitoring and information collection of disaster signals.

[0264] 2. Data Upload and Storage Module: Utilize cloud storage services (such as Object Storage S3 compatible interface) to save images or videos collected by the drone according to scene tags.

[0265] 3. Artificial Intelligence Analysis Module: Employing generative artificial intelligence models (such as large language models and image multimodal analysis models), the module first automatically analyzes the disaster situation in images and generates a 3D disaster area visualization model by inputting corresponding prompts. Simultaneously, it performs intent and sentiment understanding on user text requests.

[0266] 4. Visualization Module: Renders 3D models and key information, and displays them on command, management, and rescue terminals through a web interface or client application.

[0267] 5. Communication and Dispatch Module: Based on message middleware and API services, it pushes high-priority rescue instructions and supply route information to the mobile terminals of rescue personnel in real time.

[0268] The specific data processing and information flow methods are as follows:

[0269] After receiving images and videos uploaded by the drone terminal, the server first calls the AI ​​model to input formatted prompts to perform damage analysis and 3D reconstruction of the disaster area. For example, the server can call the following prompts:

[0270] "Based on the following drone photo URL, please automatically detect the building damage level, major road break points, and generate structured disaster coordinates and a brief description. Image URL: https: / / example-bucket.s3.amazonaws.com / disaster202406 / uav01_0930.jpg"

[0271] When a user reports specific help requests through a terminal application, the server generates the following prompts based on the input text to perform intent and emotion recognition:

[0272] Input: User submitted the message "Supplies are almost gone, and we're out of water. We're very anxious."

[0273] Task: Identify the type of supplies, their urgency priority, and corresponding emotional tone, and output the result as structured text.

[0274] The server combines the disaster analysis, demand extraction, and sentiment recognition results returned by the AI ​​model with the distribution of rescue resources and map services (such as third-party navigation APIs) to automatically generate the optimal support activity instructions and supply routes for each request in real time. These instructions are then distributed to the terminal devices of on-site rescue personnel via push notifications, enabling intelligent and efficient disaster emergency response decision-making and actions.

[0275] This invention combines efficient data acquisition, distributed processing, and generative artificial intelligence models to achieve automatic processing and decision support for multi-source information, significantly improving the response speed and scientific decision-making level of disaster emergencies. It is applicable to the emergency management of various public safety incidents.

[0276] use Figure 14 The processing procedure is explained.

[0277] Step 1:

[0278] The server monitors data streams from earthquake sensors, meteorological APIs, and flood sensors. The input is raw natural disaster data collected in real-time by the sensors and APIs. The server parses the collected data and performs threshold checks, outputting a status indicator indicating whether a disaster has been detected, as well as basic information about the affected area. Specific actions include periodically calling the data interfaces of various sensors, analyzing the information, and immediately issuing a disaster detection signal once the values ​​exceed set thresholds.

[0279] Step 2:

[0280] Based on disaster detection results, the server sends start commands and parameters to unmanned aerial vehicles (UAVs) (terminals) in the designated area. The inputs are disaster status indicators and geographic coordinate data, and the output is the issued UAV start and mission configuration instructions. Specifically, the server pushes the UAV takeoff command via API, sets the flight path, and collects parameters.

[0281] Step 3:

[0282] After receiving mission instructions from the server, the terminal (drone) automatically takes off and flies along a preset route, simultaneously collecting image and video data of the disaster area using its onboard high-definition camera. The input is the flight path and camera parameters configured by the server, and the output is the collected image and video data. The terminal periodically collects data during flight and stores it locally.

[0283] Step 4:

[0284] The terminal (drone) transmits the collected image and video data back to the server in real time via a wireless network. The input consists of locally stored image and video files, and the output is a data stream uploaded to the server. Specifically, the terminal automatically establishes a 4G / 5G or dedicated wireless link and continuously sends new data packets to a designated port on the server.

[0285] Step 5:

[0286] After receiving data uploaded by the drone, the server performs preliminary format validation and metadata tagging before uploading the data to the cloud repository. The input is the image / video files received from the drone's port, and the output is a secure link and metadata in cloud storage. The server calls the object storage API to upload data in batches and generate URLs for subsequent processing.

[0287] Step 6:

[0288] The server analyzes disaster-stricken areas based on image, video, and metadata data, combined with a generative artificial intelligence model, generating a 3D disaster model and analysis report. Inputs include cloud-stored image links and analysis requirements; outputs include a 3D disaster map, a list of damaged point coordinates, and brief text descriptions. In this step, the server constructs targeted prompts and submits the image links and on-site information to the AI ​​model to obtain the processing results.

[0289] Step 7:

[0290] The server visualizes the generated 3D disaster information and distributes it to the command center and management terminals (terminals) via the network. The input is 3D disaster data and visualization configuration; the output is an interactive 3D map displayed on the command center terminal. The server uses a web rendering engine or client-side push method to achieve dynamic presentation of disaster data.

[0291] Step 8:

[0292] Users (shelter personnel, rescue participants) input and submit on-site descriptions, supply requests, and other information through a dedicated application or chatbot. Input consists of text or voice messages from the user's client, and output is the demand data received by the server. Users actively input their actual needs on the client and send the information directly to the server after clicking send.

[0293] Step 9:

[0294] The server processes user-uploaded information, first using a sentiment analysis engine to detect emotions in the user's text, and then combining this with a generative artificial intelligence model for natural language understanding and key information extraction. The input is the user-uploaded message text, and the output is sentiment tags, demand terms, and priority determination. The server organizes the message content with sentiment analysis and inputs it into the artificial intelligence model through prompts to determine the request category and urgency.

[0295] Step 10:

[0296] The server integrates disaster maps, relief resource distribution, and all user needs, and invokes generative artificial intelligence models and route optimization tools to automatically generate optimal relief action instructions and supply routes. Inputs include 3D disaster information, demand and sentiment data, and resource distribution; outputs include relief instruction text and supply route plans. The server assembles comprehensive information and automatically generates action suggestions for frontline rescue personnel.

[0297] Step 11:

[0298] The server synchronously distributes the final rescue operation instructions and supply routes to the on-site rescue terminals (terminals) via push notifications, SMS, or other communication methods. The input is the generated instructions and routes, and the output is the task notification and navigation information displayed on the terminal. The server calls the push service API, and the terminal automatically pops up a reminder, supporting clicking to view real-time navigation.

[0299] The specific processing unit 290 sends the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires sound representing user input regarding the result of the specific processing. The control unit 46A sends the sound data representing user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the sound data.

[0300] Data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of data generation models 58 include ChatGPT (registered trademark) (accessible via the internet (URL: https: / / openai.com / blog / chatgpt)). Data generation model 58 is obtained through deep learning on a neural network. Input to data generation model 58 are prompt words containing instructions, and inference data such as sound data representing sound, text data representing text, and image data representing images (e.g., still image data or animation data). Data generation model 58 infers from the input inference data based on the instructions represented by the prompt words and outputs the inference result in one or more data forms, such as sound data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or induction. The specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. The data generation model 58 can also be a model finely tuned to output inference results from prompts that do not contain instructions. In this case, the data generation model 58 can output inference results based on prompts that do not contain instructions. The data processing apparatus 12, etc., includes various data generation models 58, including AI other than the generation AI. AI other than the generation AI can be, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and can perform various processes, but is not limited to this example. Furthermore, the AI ​​can also be an AI agent. Furthermore, when the processing of the above-mentioned parts is performed by AI, the processing can be partially or entirely performed by AI, but is not limited to this example. Furthermore, the processing performed by the AI ​​including the generation AI can be replaced by processing in the rule base, and the processing in the rule base can also be replaced by processing performed by the AI ​​including the generation AI.

[0301] Furthermore, the processing of the aforementioned data processing system 10 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it can also be performed by both the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or external devices, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or external devices.

[0302] For example, the collection unit is implemented by the control unit 46A of the smart device 14 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit uses the camera 42 or communication I / F 44 of the smart device 14 to acquire step data, which is then processed by the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12, which analyzes the data from the collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12, which uses a generation AI to generate a menu. For example, the serving unit is implemented by the output device 40 of the smart device 14 or the specific processing unit 290 of the data processing device 12, which provides the generated menu to the user. The correspondence between each unit and the device or control unit is not limited to the above examples and various changes can be made.

[0303] In the above embodiments, examples of specific processing by the data processing device 12 are given, but the technology disclosed herein is not limited to this, and specific processing may also be performed by the smart device 14.

[0304] Second Implementation Method

[0305] Figure 3 An example of the configuration of the data processing system 210 according to the second embodiment is shown.

[0306] like Figure 3 As shown, the data processing system 210 includes a data processing device 12 and smart glasses 214. A server can be cited as an example of the data processing device 12.

[0307] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" as understood in this disclosure. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0308] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, and communication I / F 44 are also connected to the bus 52.

[0309] Microphone 238 receives instructions from user 20 by receiving sounds emitted by user 20. Microphone 238 captures sounds emitted by user 20 and converts the captured sounds into sound data, which is then output to processor 46. Speaker 240 outputs sound according to instructions from processor 46.

[0310] Camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and imaging elements such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, to capture images of the user 20's surroundings (e.g., the field of view defined by an angle equivalent to the field of vision of an average healthy person).

[0311] Communication I / F44 is connected to network 54. Communication I / F44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F44 and 26 is performed in a secure state.

[0312] Figure 4 This illustrates an example of the main functions of the data processing device 12 and the smart glasses 214. For example... Figure 4 As shown, in the data processing device 12, specific processing is performed by the processor 28. The specific processing program 56 is stored in the memory 32.

[0313] The specific processing program 56 is an example of a "program" involved in the technology of this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0314] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. In the emotion inference function (emotion-specific function) using the emotion-specific model 59, various inferences and predictions related to the user's emotions are performed, including inferences and predictions of the user's emotions, but this is not limited to this example. Furthermore, emotion inference and prediction may also include, for example, emotion analysis (parsing).

[0315] In the smart glasses 214, the processor 46 performs reception and output processing. The memory 50 stores the reception and output program 60. The processor 46 reads the reception and output program 60 from the memory 50 and executes the read reception and output program 60 on the RAM 48. The reception and output processing is implemented by the processor 46 operating as a control unit 46A according to the reception and output program 60 executed on the RAM 48. Furthermore, the smart glasses 214 has the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and these models can also be used to perform the same processing as the specific processing unit 290.

[0316] Next, the specific processing of the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is implemented by the data processing device 12 and the smart glasses 214. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0317] Example 1

[0318] The process is the same as that of the specific process described in Embodiment 1 in the first embodiment above, so the description is omitted.

[0319] Application Example 1

[0320] The process is the same as that in the specific processing described in Application Example 1 of the first embodiment above, so the description is omitted.

[0321] Example 2

[0322] The process is the same as that of the specific process in Embodiment 2 described in the first embodiment above, so the description is omitted.

[0323] Application Example 2

[0324] The process is the same as that in the specific processing described in Application Example 2 of the first embodiment above, so the description is omitted.

[0325] The specific processing unit 290 sends the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A outputs the result of the specific processing to the speaker 240. The microphone 238 acquires sound input representing the user's input regarding the result of the specific processing. The control unit 46A sends the sound data representing the user's input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the sound data.

[0326] Data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of data generation models 58 include ChatGPT (registered trademark) (accessible via the internet (URL: https: / / openai.com / blog / chatgpt)). Data generation model 58 is obtained through deep learning on a neural network. Input to data generation model 58 are prompt words containing instructions, and inference data such as sound data representing sound, text data representing text, and image data representing images (e.g., still image data or animation data). Data generation model 58 infers from the input inference data based on the instructions represented by the prompt words and outputs the inference result in one or more data forms, such as sound data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or induction. The specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. The data generation model 58 can also be a model finely tuned to output inference results from prompts that do not contain instructions. In this case, the data generation model 58 can output inference results based on prompts that do not contain instructions. The data processing apparatus 12, etc., includes various data generation models 58, including AI other than the generation AI. AI other than the generation AI can be, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and can perform various processes, but is not limited to this example. Furthermore, the AI ​​can also be an AI agent. Furthermore, when the processing of the above-mentioned parts is performed by AI, the processing can be partially or entirely performed by AI, but is not limited to this example. Furthermore, the processing performed by the AI ​​including the generation AI can be replaced by processing in the rule base, and the processing in the rule base can also be replaced by processing performed by the AI ​​including the generation AI.

[0327] Furthermore, the processing of the aforementioned data processing system 10 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it can also be performed by both the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or external devices, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or external devices.

[0328] For example, the collection unit is implemented by the control unit 46A of the smart glasses 214 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit uses the camera 42 or communication I / F 44 of the smart glasses 214 to acquire step data, which is then processed by the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12, which analyzes the data from the collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12, which uses a generation AI to generate a menu. For example, the serving unit is implemented by the speaker 240 of the smart glasses 214 or the specific processing unit 290 of the data processing device 12, which provides the generated menu to the user. The correspondence between each unit and the device or control unit is not limited to the above examples and various changes can be made.

[0329] In the above embodiments, examples of specific processing by the data processing device 12 are given, but the technology disclosed herein is not limited to this, and specific processing may also be performed by the smart glasses 214.

[0330] Third Implementation Method

[0331] Figure 5 An example of the configuration of the data processing system 310 according to the third embodiment is shown.

[0332] like Figure 5 As shown, the data processing system 310 includes a data processing device 12 and a head-mounted terminal 314. A server can be cited as an example of the data processing device 12.

[0333] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" as understood in this disclosure. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0334] The head-mounted terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, display 343, and communication I / F 44 are also connected to the bus 52.

[0335] Microphone 238 receives instructions from user 20 by receiving sounds emitted by user 20. Microphone 238 captures sounds emitted by user 20 and converts the captured sounds into sound data, which is then output to processor 46. Speaker 240 outputs sound according to instructions from processor 46.

[0336] Camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and imaging elements such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, to capture images of the user 20's surroundings (e.g., the field of view defined by an angle equivalent to the field of vision of an average healthy person).

[0337] Communication I / F44 is connected to network 54. Communication I / F44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F44 and 26 is performed in a secure state.

[0338] Figure 6 This illustrates an example of the main functions of the data processing device 12 and the head-mounted terminal 314. For example... Figure 6 As shown, in the data processing device 12, specific processing is performed by the processor 28. The specific processing program 56 is stored in the memory 32.

[0339] The specific processing program 56 is an example of a "program" involved in the technology of this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0340] The memory 32 stores the data generation model 58 and the emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290.

[0341] In the head-mounted terminal 314, the processor 46 performs the acceptance / output processing. The memory 50 stores the acceptance / output program 60. The processor 46 reads the acceptance / output program 60 from the memory 50 and executes the read acceptance / output program 60 on the RAM 48. The acceptance / output processing is implemented by the processor 46 operating as a control unit 46A according to the acceptance / output program 60 executed on the RAM 48.

[0342] Next, the specific processing of the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is implemented by the data processing device 12 and the head-mounted terminal 314. In the following description, the data processing device 12 will be referred to as the "server" and the head-mounted terminal 314 will be referred to as the "terminal".

[0343] Example 1

[0344] The process is the same as that of the specific process described in Embodiment 1 in the first embodiment above, so the description is omitted.

[0345] Application Example 1

[0346] The process is the same as that in the specific processing described in Application Example 1 of the first embodiment above, so the description is omitted.

[0347] Example 2

[0348] The process is the same as that of the specific process in Embodiment 2 described in the first embodiment above, so the description is omitted.

[0349] Application Example 2

[0350] The process is the same as that in the specific processing described in Application Example 2 of the first embodiment above, so the description is omitted.

[0351] The specific processing unit 290 sends the result of the specific processing to the head-mounted terminal 314. In the head-mounted terminal 314, the control unit 46A outputs the result of the specific processing to the speaker 240 and the display 343. The microphone 238 acquires sound input representing the result of the specific processing. The control unit 46A sends the sound data representing the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the sound data.

[0352] Data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of data generation models 58 include ChatGPT (registered trademark) (accessible via the internet (URL: https: / / openai.com / blog / chatgpt)). Data generation model 58 is obtained through deep learning on a neural network. Input to data generation model 58 includes prompt words containing instructions, and inference data such as sound data representing sound, text data representing text, and image data representing images (e.g., still image data or animation data). Data generation model 58 infers the input inference data based on the instructions represented by the prompt words and outputs the inference result in one or more data forms such as sound data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or induction. The specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. The data generation model 58 can also be a model finely tuned to output inference results from prompts that do not contain instructions. In this case, the data generation model 58 can output inference results based on prompts that do not contain instructions. The data processing apparatus 12, etc., includes various data generation models 58, including AI other than the generation AI. AI other than the generation AI can be, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and can perform various processes, but is not limited to this example. Furthermore, the AI ​​can also be an AI agent. Furthermore, when the processing of the above-mentioned parts is performed by AI, the processing can be partially or entirely performed by AI, but is not limited to this example. Furthermore, the processing performed by the AI ​​including the generation AI can be replaced by processing in the rule base, and the processing in the rule base can also be replaced by processing performed by the AI ​​including the generation AI.

[0353] Furthermore, the processing of the aforementioned data processing system 10 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the head-mounted terminal 314, but it can also be performed by both the specific processing unit 290 of the data processing device 12 and the control unit 46A of the head-mounted terminal 314. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the head-mounted terminal 314 or external devices, and the head-mounted terminal 314 acquires or collects information required for processing from the data processing device 12 or external devices.

[0354] For example, the collection unit is implemented by the control unit 46A of the head-mounted terminal 314 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit uses the camera 42 or communication I / F 44 of the head-mounted terminal 314 to acquire step data, which is then processed by the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12, which analyzes the data from the collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12, which uses a generation AI to generate a menu. For example, the serving unit is implemented by the speaker 240 and display 343 of the head-mounted terminal 314 or the specific processing unit 290 of the data processing device 12, which provides the generated menu to the user. The correspondence between each unit and the device or control unit is not limited to the above examples and various changes can be made.

[0355] In the above embodiments, examples of specific processing by the data processing device 12 are given, but the technology disclosed herein is not limited to this, and specific processing may also be performed by the head-mounted terminal 314.

[0356] Fourth Implementation Method

[0357] Figure 7 An example of the configuration of the data processing system 410 according to the fourth embodiment is shown.

[0358] like Figure 7 As shown, the data processing system 410 includes a data processing device 12 and a robot 414. A server can be cited as an example of the data processing device 12.

[0359] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" as understood in this disclosure. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0360] Robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, controlled object 443, and communication I / F 44 are also connected to the bus 52.

[0361] Microphone 238 receives instructions from user 20 by receiving sounds emitted by user 20. Microphone 238 captures sounds emitted by user 20 and converts the captured sounds into sound data, which is then output to processor 46. Speaker 240 outputs sound according to instructions from processor 46.

[0362] Camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and imaging elements such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, to photograph the area around robot 414 (e.g., the field of view defined by a perspective equivalent to the field of vision of an average healthy person).

[0363] Communication I / F44 is connected to network 54. Communication I / F44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F44 and 26 is performed in a secure state.

[0364] The controlled object 443 includes a display device, LEDs (light-emitting diodes) for the eyes, and motors for driving the arms, hands, and feet. The posture or movement of the robot 414 is controlled by controlling the motors in the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. In addition, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0365] Figure 8 This illustrates an example of the main functions of the data processing device 12 and the robot 414. For example... Figure 8 As shown, in the data processing device 12, specific processing is performed by the processor 28. The specific processing program 56 is stored in the memory 32.

[0366] The specific processing program 56 is an example of a "program" involved in the technology of this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0367] The memory 32 stores the data generation model 58 and the emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290.

[0368] In robot 414, the processor 46 performs the acceptance and output processing. The memory 50 stores the acceptance and output program 60. The processor 46 reads the acceptance and output program 60 from the memory 50 and executes the read acceptance and output program 60 on RAM 48. The acceptance and output processing is implemented by the processor 46 acting as the control unit 46A according to the acceptance and output program 60 executed on RAM 48.

[0369] Next, the specific processing of the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is implemented by the data processing device 12 and the robot 414. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal".

[0370] Example 1

[0371] The process is the same as that of the specific process described in Embodiment 1 in the first embodiment above, so the description is omitted.

[0372] Application Example 1

[0373] The process is the same as that in the specific processing described in Application Example 1 of the first embodiment above, so the description is omitted.

[0374] Example 2

[0375] The process is the same as that of the specific process in Embodiment 2 described in the first embodiment above, so the description is omitted.

[0376] Application Example 2

[0377] The process is the same as that in the specific processing described in Application Example 2 of the first embodiment above, so the description is omitted.

[0378] The specific processing unit 290 sends the result of the specific processing to the robot 414. In the robot 414, the control unit 46A outputs the result of the specific processing to the speaker 240 and the controlled object 443. The microphone 238 acquires sound input from the user representing the result of the specific processing. The control unit 46A sends the sound data representing the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the sound data.

[0379] Data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of data generation models 58 include ChatGPT (registered trademark) (accessible via the internet (URL: https: / / openai.com / blog / chatgpt)). Data generation model 58 is obtained through deep learning on a neural network. Input to data generation model 58 are prompt words containing instructions, and inference data such as sound data representing sound, text data representing text, and image data representing images (e.g., still image data or animation data). Data generation model 58 infers from the input inference data based on the instructions represented by the prompt words and outputs the inference result in one or more data forms, such as sound data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or induction. The specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. The data generation model 58 can also be a model finely tuned to output inference results from prompts that do not contain instructions. In this case, the data generation model 58 can output inference results based on prompts that do not contain instructions. The data processing apparatus 12, etc., includes various data generation models 58, including AI other than the generation AI. AI other than the generation AI can be, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and can perform various processes, but is not limited to this example. Furthermore, the AI ​​can also be an AI agent. Furthermore, when the processing of the above-mentioned parts is performed by AI, the processing can be partially or entirely performed by AI, but is not limited to this example. Furthermore, the processing performed by the AI ​​including the generation AI can be replaced by processing in the rule base, and the processing in the rule base can also be replaced by processing performed by the AI ​​including the generation AI.

[0380] Furthermore, the processing of the aforementioned data processing system 10 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it can also be performed by both the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or external devices, and the robot 414 acquires or collects information required for processing from the data processing device 12 or external devices.

[0381] For example, the collection unit is implemented by the control unit 46A of the robot 414 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit uses the camera 42 or communication I / F 44 of the robot 414 to acquire step data, which is then processed by the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12, which analyzes the data from the collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12, which uses a generation AI to generate a menu. For example, the serving unit is implemented by the speaker 240 of the robot 414 and the control object 443 or the specific processing unit 290 of the data processing device 12, which provides the generated menu to the user. The correspondence between each unit and the device or control unit is not limited to the above examples and various changes can be made.

[0382] In the above embodiments, examples of specific processing by the data processing device 12 are given, but the technology disclosed herein is not limited to this, and specific processing may also be performed by the robot 414.

[0383] Furthermore, the emotion-specific model 59, acting as an emotion engine, can determine a user's emotion based on a specific mapping. Specifically, the emotion-specific model 59 can determine a user's emotion based on an emotion graph that serves as a specific mapping (see [reference]). Figure 9 The system determines the user's emotions. Furthermore, the emotion-specific model 59 can similarly determine the robot's emotions, and the specific processing unit 290 performs specific processing based on the robot's emotions.

[0384] Figure 9 This is a diagram representing an emotion map 400 that maps multiple emotions. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotion is. On the outer side of the concentric circles, emotions representing states or behaviors arising from mood are arranged. Emotions are concepts that include feelings and mental states. Emotions generated by reactions occurring in the brain are arranged roughly to the left of the concentric circles. Emotions derived from situational judgments are arranged roughly to the right of the concentric circles. Emotions generated by reactions occurring in the brain and derived from situational judgments are arranged roughly above and below the concentric circles. Furthermore, "pleasant" emotions are arranged above the concentric circles, and "unpleasant" emotions are arranged below them. Thus, in the emotion map 400, multiple emotions are mapped based on the structure that generates emotions, and emotions that are likely to occur simultaneously are mapped close to each other.

[0385] These emotions are distributed at the three o'clock position of the emotion map 400, typically fluctuating between peace and anxiety. In the right half of the emotion map 400, situational awareness dominates over internal sensation, thus resulting in an impression of calm.

[0386] The inner side of the emotion map 400 represents the inner state, while the outer side represents behavior. Therefore, the further outward you are from the emotion map 400, the more visible the emotion becomes (manifested in behavior).

[0387] Here, human emotions are based on various balances such as posture and blood sugar levels. When these balances deviate from an ideal state, it indicates an unpleasant state; when they approach the ideal state, it indicates a pleasant state. Emotions in robots, cars, motorcycles, etc., can also be created in the following way: based on various balances such as posture and remaining battery power, when these balances deviate from an ideal state, it indicates an unpleasant state; when they approach the ideal state, it indicates a pleasant state. Emotion maps can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a Brain Physiological Signal Analysis System for Voice Emotion Recognition and Emotion, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). In the left half of the emotion map, emotions belonging to the sensory-dominated region, called "response," are arranged. Furthermore, in the right half of the emotion map, emotions belonging to the situational cognition-dominated region, called "situation," are arranged.

[0388] In the emotion map, two types of emotions that promote learning are defined. One is a negative emotion on the situational side, in the middle or peripheral region of "repentance" or "reflection." This occurs when the robot experiences negative emotions such as "I don't want to experience this feeling again" or "I don't want to be blamed again." The other is a positive emotion on the response side, near the "desire" region. This occurs when there are positive feelings such as "wanting more" or "wanting to know more."

[0389] The emotion-specific model 59 inputs user input into a pre-trained neural network to obtain emotion values ​​representing each emotion shown in the emotion map 400, thereby determining the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network... Figure 10 As shown in the sentiment graph 900, it was trained in a way that sentiments that are configured close to each other have similar values. Figure 10 The text shows examples of emotions such as "peace of mind", "stability", and "reassurance" that have similar emotion values.

[0390] The above description focuses on the functions of the data processing device 12, but the system of this disclosure is not necessarily installed on a server. The system of this disclosure can also be installed as a general information processing system. This disclosure can also be installed, for example, as a software program running on a personal computer, an application running on a smartphone, etc. The method of this disclosure can also be provided to users in the form of SaaS (Software as a Service).

[0391] In the above embodiments, an example of a specific process being performed by a single computer 22 is given. However, the technology disclosed herein is not limited to this, and the specific process can also be distributed among multiple computers, including computer 22. For example, the data generation model 58 can be located on an external device of the data processing apparatus 12, where data is generated based on the input data.

[0392] In the above embodiments, examples of storing a specific processing program 56 in the memory 32 have been described, but the technology disclosed herein is not limited thereto. For example, the specific processing program 56 may also be stored in a portable computer-readable non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed into the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0393] Alternatively, a specific processing program 56 may be pre-stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 according to the requirements of the data processing device 12.

[0394] In addition, it is not necessary to store all the specific processing program 56 in the storage device such as the server connected to the data processing device 12 via the network 54 or in the memory 32; a portion of the specific processing program 56 may be stored in advance.

[0395] As hardware resources for performing specific processes, various processors, as shown below, can be used. For example, a CPU can be listed as a processor, which functions as a general-purpose processor that performs specific processes by executing software, i.e., a program. Furthermore, processors can be listed as special-purpose circuits such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application-Specific Integrated Circuits), which are processors with circuitry specifically designed to perform specific processes. Each processor has built-in or connected memory, and each processor executes specific processes using that memory.

[0396] The hardware resources for performing a specific process can consist of one of these various processors, or a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resources for performing a specific process can be a single processor.

[0397] As an example of a single processor, there are two approaches: First, a processor is composed of a combination of one or more CPUs and software, which functions as a hardware resource to perform a specific process; second, as represented by a SoC (System-on-a-chip), a processor is used to implement the functionality of the entire system, which includes multiple hardware resources for performing a specific process, using a single IC (Integrated Circuit) chip. In this way, the specific process is implemented by using one or more of the aforementioned processors as hardware resources.

[0398] Furthermore, the hardware architecture of these various processors, more specifically, can utilize circuits that combine semiconductor elements and other circuit components. Moreover, the specific process described above is just one example. Therefore, without departing from the main point, unnecessary steps can certainly be deleted, new steps added, or the processing order changed.

[0399] The descriptions and illustrations above are detailed explanations of a portion of the technology disclosed herein, and are merely one example of the technology disclosed herein. For example, the above descriptions of the structure, function, effect, and results are just one example of the structure, function, effect, and results of a portion of the technology disclosed herein. Therefore, without departing from the spirit of the technology disclosed herein, unnecessary parts may be deleted, new elements added, or replacements may be made to the descriptions and illustrations above. Furthermore, to avoid confusion and facilitate understanding of a portion of the technology disclosed herein, explanations of common technical knowledge that do not require special explanation under the premise of being able to implement the technology disclosed herein have been omitted from the descriptions and illustrations above.

[0400] All documents, patent applications and technical specifications set forth in this specification are incorporated herein by reference to the same extent that each document, patent application and technical specification is specifically and individually described therein and referenced by reference.

[0401] In addition, the following notes are provided in response to the above explanation.

[0402] Example 1

[0403] (Note 1)

[0404] An information processing system includes: a device for detecting disaster occurrence; a device for remotely controlling multiple mobile devices and collecting visual information of the disaster area through an information acquisition device; a device for processing the collected visual information using various information parsing algorithms and generating three-dimensional geospatial information through a generative artificial intelligence model; a device for displaying the generated three-dimensional geospatial information on a countermeasure maker's terminal through an information visualization processing device and automatically sending support activity guidance information to multiple terminals through an instruction distribution processing device; a device for collecting information such as on-site conditions and material needs from on-site information senders through a general information communication terminal; a device for processing the collected information using a natural language parsing algorithm and extracting key information; a device for automatically generating support activity instruction information and mobile path planning information based on the extracted key information and three-dimensional geospatial information; a device for determining the optimal delivery path using a path planning algorithm and automatically sending the path information and support activity instructions to on-site terminals; a device for recording all collected information into a database in chronological order and performing unified automatic processing of the entire process from disaster occurrence to support completion; and a device for generating and applying input instruction statements to a generative artificial intelligence model based on prompt statements.

[0405] (Note 2)

[0406] According to the information processing system described in Appendix 1, the mobile device is equipped with an information acquisition device and acts autonomously based on flight path data received by the information processing device.

[0407] (Note 3)

[0408] According to the information processing system described in Appendix 1, the three-dimensional geospatial information is generated by comparing temporal visual information before and after a disaster.

[0409] Application Example 1

[0410] (Note 1)

[0411] An information processing system includes: means for acquiring observation data and automatically determining the occurrence of a disaster through an information processing device; means for activating multiple remotely operated flight devices to acquire image and graphic information of the disaster area; means for storing the acquired image and graphic information in a storage device, and then, through a data analysis device, inputting this data along with prompts into an artificial intelligence model for information analysis and comparison to generate three-dimensional map data containing three-dimensional disaster information; means for visualizing the generated three-dimensional map data and analysis results, displaying them on an instruction management terminal, and automatically generating disaster response guidance information and distributing it to relevant terminals; means for receiving status reports and support material information from input devices of shelters or on-site response personnel, and extracting important matters and emotional elements through a natural language processing unit; means for generating plans based on the extracted information, assessed urgency, and three-dimensional map data, road condition data, etc., to optimize the transportation routes of support resources for multiple locations; and means for instantly distributing the optimized transportation routes and support instruction information to relevant terminals.

[0412] (Note 2)

[0413] According to the information processing system described in Appendix 1, the remote-operated flight device is equipped with an image acquisition device and autonomously collects information according to a set flight path via a communication device and a navigation control device.

[0414] (Note 3)

[0415] According to the information processing system described in Appendix 1, the three-dimensional map data is generated by inputting multi-time-point spatial information before and after the disaster, along with prompt statements, into an artificial intelligence model, and then performing before-and-after comparative analysis.

[0416] Example 2

[0417] (Note 1)

[0418] An information processing system includes: a device for detecting the occurrence of a disaster based on various time-series observation data collected from environmental information acquisition devices and external information acquisition devices; a device for automatically activating multiple remotely controlled mobile vehicles and collecting and processing information about the disaster-stricken area through onboard image acquisition devices; a device for storing the acquired various spatiotemporal information in an information storage device and comparing and analyzing the information before and after the disaster using an image analysis device and a generative artificial intelligence model to reconstruct three-dimensional spatial information; a device for displaying the reconstructed three-dimensional spatial information on a visualization processing device and distributing countermeasure information derived from the generative artificial intelligence model to relevant terminals; a device for receiving unstructured information provided by on-site residents or staff through an information input terminal and inputting it into a natural language processing device, and extracting and structuring important information by combining emotion recognition processing; a device for automatically calculating rescue operation instructions and resource distribution routes by using the extracted structured data and emotion recognition results as input, combined with a generative artificial intelligence model and path optimization algorithm; and a device for distributing rescue instructions and resource distribution route information to on-site staff in real time using a notification device.

[0419] (Note 2)

[0420] According to the information processing system described in Appendix 1, the remotely controlled mobile body is equipped with an image acquisition device and automatically moves in the disaster area and collects information based on a predefined autonomous navigation path or a dynamically set path.

[0421] (Note 3)

[0422] According to the information processing system described in Appendix 1, the three-dimensional spatial information is automatically generated by comparing various types of information before and after the disaster and using an image analysis device and a generative artificial intelligence model.

[0423] Application Example 2

[0424] (Note 1)

[0425] An information processing system includes: a device for detecting the occurrence of a disaster; a device for launching multiple aircraft and collecting image information of the disaster area; a device for analyzing the collected image information and generating three-dimensional information of the disaster situation; a device for visualizing the generated three-dimensional information and issuing response instructions; a device for receiving on-site information from users of shelters and rescue personnel, and extracting important information through language information processing; a device for performing sentiment analysis on the received language information to determine information priority; a device for determining support activity instructions and supply routes based on the extracted important information and priority; a device for instantly distributing the determined support activity instructions and supply route information to rescue personnel; a device for using generative artificial intelligence to process data analysis, image recognition, language processing, sentiment analysis, and support activity instruction generation, and generating corresponding prompt statements as input for each of the above processes; and a device for transmitting the generated information among the devices through data storage and distributed processing.

[0426] (Note 2)

[0427] According to the information processing system described in Appendix 1, the aircraft is equipped with image acquisition equipment and flies along an autonomous flight path to acquire and transmit the image information.

[0428] (Note 3)

[0429] According to the information processing system described in Appendix 1, the three-dimensional information is automatically generated by comparing information before and after the disaster.

Claims

1. An information processing system, characterized in that, include: Devices used to detect the occurrence of disasters; A device for launching multiple drones and collecting video and image data of the disaster-stricken area; A device for analyzing acquired video and image data and generating a three-dimensional map of the disaster situation; A device for visualizing the generated 3D map and distributing countermeasures guidance; A device for receiving information from shelter residents and rescue workers and extracting important information through natural language processing; A device for determining rescue operation instructions and support material transportation routes based on the extracted information.

2. The information processing system according to claim 1, characterized in that, The drone is equipped with a camera and flies along an automatic flight path.

3. The information processing system according to claim 1, characterized in that, The 3D map is generated by comparing data before and after the disaster.

Images