System

Abstract

A system is provided.SOLUTION: A system including, as information collection means, means for collecting data in real time from a plurality of information sources, means for pre-processing the collected data, means for integrating the pre-processed data, means for analyzing the integrated data by generative artificial intelligence, means for automatically setting a priority of rescue based on an analysis result, and means for sharing the set priority and information with each institution.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Rescue operations during large-scale disasters are often delayed due to fragmented information and a lack of real-time information. This has frequently resulted in many casualties in past disasters. Therefore, there is a need for a comprehensive information collection and analysis system that can quickly and accurately grasp the situation and carry out efficient rescue operations. [Means for solving the problem]

[0005] To solve this problem, the present invention provides the following means: a system including an information collection means for collecting data in real time from multiple information sources, a means for preprocessing the collected data, a means for integrating the preprocessed data, a means for analyzing the integrated data using generative artificial intelligence, a means for automatically setting rescue priorities based on the analysis results, and a means for sharing the set priorities and information with various organizations. The system also includes a navigation means for processing image data, text data, and voice data, and for indicating the optimal route using GPS.

[0006] 1. "Information collection means" means a component that has the function of collecting data from multiple sources in real time.

[0007] 2. "Preprocessing" is the process of removing noise from collected data and converting it into a form suitable for analysis and integration.

[0008] 3. "Information integration means" is a component that has the function of unifying and associating pre-processed data.

[0009] 4. "Generative AI" refers to artificial intelligence technology that uses algorithms and models to analyze diverse collected data and derive appropriate results.

[0010] 5. "Analysis means" means a component that has the function of analyzing the integrated data using generative artificial intelligence and deriving results.

[0011] 6. "Rescue priority setting means" is a component having the function of automatically determining the priority of rescue operations based on the analysis results.

[0012] 7. "Information sharing means" refers to a component that has the function of providing set priorities and analytical results to each institution in real time.

[0013] 8. "Image data" is a data format that refers to visual information such as satellite images and on-site photographs.

[0014] 9. "Text data" is a data format that refers to character data such as social media posts and emergency call text information.

[0015] 10. "Audio Data" is a data format that refers to acoustic information such as emergency calls and audio recordings from the scene.

[0016] 11. "Navigation means" means a component that has the function of indicating the optimal route for rescue operations using GPS data. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The system of the present invention is designed to carry out rescue operations quickly and efficiently in the event of a disaster. This system is composed of a server, terminals, and users.

[0039] The server collects data in real time from multiple sources, including using social media APIs to retrieve posts based on specific hashtags or keywords, sending requests to download the latest image data from satellite data providers, and connecting to emergency call systems to receive voice data.

[0040] The server then preprocesses the collected data. For SMS data, text mining tools are used to remove noise and extract post metadata (such as location and time). For satellite image data, image processing tools are used to standardize resolution and perform filtering and segmentation. For emergency call data, speech-to-text conversion is performed to extract key keywords and phrases.

[0041] The server then integrates the pre-processed data into a centralized platform that uses a database with complex query capabilities to store social media posts, satellite imagery, and emergency call data in a single repository, resulting in a connected, integrated dataset.

[0042] The server then analyzes the combined data using generative AI models. For example, the server uses computer vision models to identify the extent of damage from satellite imagery and quantify building damage. It also uses natural language processing algorithms to extract key damage information from text data and map the frequency and location of damage reports.

[0043] Based on the analysis results, the server automatically assigns rescue priorities. Using a risk assessment algorithm, the server calculates a score to assess the urgency and importance of each data point, taking into account various factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls. Rescue priorities are then assigned based on this score.

[0044] Finally, the server provides the set priorities and analysis results to each institution in real time. A REST API is provided so that each institution can access the data in real time. Each institution can check the information through the provided data dashboard.

[0045] The device receives rescue priority and physical location information sent from the server. The device accesses the server's API via the internet using a mobile device or PC to retrieve the latest information. The data is also stored in a local cache.

[0046] The device uses a map display library to visually display the received data on a map, color-coding areas according to priority: for example, red for high urgency, yellow for medium urgency, and green for low urgency.

[0047] Additionally, the device provides a GPS-based navigation tool to help users find the optimal rescue route. The navigation application calculates the shortest route in real time and provides audio and visual navigation instructions.

[0048] Users can check the data displayed on the device and act according to the instructions. For example, a rescue team leader can use the tablet to check the current rescue priority and immediately begin action.

[0049] After completing an operation in the field, users input new information into the device and provide feedback to the server, which continuously updates the information and allows other rescue teams to keep up to date with the latest situation.

[0050] As a concrete example, if a large earthquake occurs in a certain area, the server collects damage reports from social media posts and analyzes satellite images to identify the extent of the damage. It then converts voice data from emergency calls into text, extracts important information, and centralizes it. Generative AI analyzes this information, identifies areas with severe damage, sets rescue priorities, and notifies each rescue team. Rescue teams can act quickly based on this information, potentially saving many lives.

[0051] The processing flow will be explained below.

[0052] Step 1: Gather information

[0053] The server collects social media data, satellite image data, and emergency call data.

[0054] The server uses the Twitter API and other social media APIs to retrieve posts based on specific hashtags or keywords in real time.

[0055] The server periodically sends an API request to retrieve the latest image data from the satellite data provider.

[0056] The server connects to the emergency call system and receives emergency call voice data in real time.

[0057] Step 2: Data Preprocessing

[0058] The server preprocesses the collected data.

[0059] For social media data, the server uses text mining tools to remove noise, identify language, and extract metadata (e.g., posting time and location information).

[0060] The server uses image processing tools to standardize the resolution of the satellite image data and perform filtering and segmentation processing.

[0061] For emergency call data, the server converts the speech to text and uses automatic speech recognition (ASR) tools to extract key keywords and phrases.

[0062] Step 3: Information integration

[0063] The server consolidates the pre-processed data.

[0064] The server uses complex queries to store social media posts, satellite images, and emergency call data in a centralized database, creating a unified dataset that correlates these data.

[0065] Step 4: Generative AI analysis

[0066] The server analyzes the combined data using the generated AI model.

[0067] The server uses computer vision models to identify the extent of damage from satellite imagery and quantify the damage to specific objects (e.g., buildings).

[0068] The server uses natural language processing (NLP) algorithms to extract important damage information from the text data and map it to the frequency and location of damage reports.

[0069] Step 5: Setting rescue priorities

[0070] The server automatically sets rescue priorities based on the analysis results.

[0071] The server uses a risk assessment algorithm to rate the urgency and importance of each data point, calculating a score based on various factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls, and then assigning a priority to each.

[0072] Step 6: Information sharing

[0073] The server shares the set priorities and analysis results with each institution.

[0074] The server provides a REST API, allowing organizations to access the information they need in real time. Organizations can connect to the data dashboard through the REST API to view the latest information.

[0075] Step 7: Receiving Data

[0076] The terminal receives the rescue priority and location information transmitted from the server.

[0077] Mobile devices and PCs access the server's API via the Internet to periodically retrieve the latest data, which is then stored in a local cache.

[0078] Step 8: Visualization

[0079] The data received by the device is displayed on a map.

[0080] The device uses a mapping library (e.g., Google Maps API or Leaflet) to display areas color-coded based on priority: high urgency areas are displayed in red, medium urgency areas in yellow, and low urgency areas in green.

[0081] Step 9: Navigation Instructions

[0082] The device will instruct the user on the optimal rescue route.

[0083] The device uses GPS data to connect to a map application (e.g., Mapbox API) to provide real-time navigation, providing audio and visual navigation instructions to help users reach their destination efficiently.

[0084] Step 10: Verify the information

[0085] The user checks the information displayed on the device and acts according to the instructions.

[0086] Users can check the latest rescue priority and route information through the application on their device and begin taking action immediately.

[0087] Step 11: Provide feedback

[0088] The user inputs new local information into the terminal and provides feedback to the server.

[0089] Users input new information obtained on-site (e.g., new damage or progress of rescue efforts) into their devices and send it as data to the server. The server updates the database based on this feedback information and shares the latest information with other rescue teams.

[0090] Example 1

[0091] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0092] Rescue operations during disasters must be carried out quickly and efficiently, but it is difficult to properly collect, analyze, and integrate data from multiple sources. Furthermore, it is not easy to grasp the damage situation in real time and provide the optimal rescue route. With conventional systems, data collection and integration can take time, making it difficult to effectively prioritize rescue operations. This can reduce the efficiency of rescue operations and put many lives at risk. The purpose of this invention is to solve these problems and enable fast and efficient rescue operations during disasters.

[0093] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0094] In this invention, the server includes information collection means, including means for collecting data in real time from multiple sources, means for preprocessing the collected data, means for integrating the preprocessed data, means for analyzing the integrated data using a generated AI model, means for automatically setting rescue priorities based on the analysis results, means for providing the set priorities and information to each organization, means for visually displaying the rescue priorities received by the terminal on a color-coded map, and means for providing optimal rescue routes using GPS. This allows for the rapid and effective collection, preprocessing, and integration of data from multiple sources and analysis using the generated AI model, enabling accurate rescue prioritization and real-time information provision. Furthermore, the efficiency of rescue operations is significantly improved by visualizing the damage situation and providing optimal route guidance through the terminal.

[0095] "Information collection means" refers to a means for collecting data in real time from multiple information sources.

[0096] "Preprocessing means" refers to a means of converting collected data into a form that is easier to analyze by performing processes such as noise removal and metadata extraction on the data.

[0097] The "integration means" is a means for integrating pre-processed data into a centralized platform and storing it in a database that allows for complex queries, etc.

[0098] A "generative AI model" is an artificial intelligence model used to analyze collected and pre-processed data, including, for example, computer vision models and natural language processing algorithms.

[0099] "Data analysis means" refers to a means of analyzing the integrated data using a generative AI model to identify the extent of damage and extract important information.

[0100] The "priority setting means" is a means for automatically determining the priority of rescue operations based on the analysis results and using a risk assessment algorithm.

[0101] "Information provision means" refers to the means for providing the set priorities and related information to each relevant organization in real time. Specifically, this includes REST APIs and data dashboards.

[0102] The "color-coded visual display means" is a means of displaying rescue priority and damage information received by the terminal on a map in different colors. By indicating the urgency level by color for each area, it makes it easier to visually check the information.

[0103] "Navigation Means" means a means that utilizes GPS to provide an optimal rescue route, calculates the shortest route in real time, and includes audio and visual instructions.

[0104] MODE FOR CARRYING OUT THE INVENTION

[0105] The system of the present invention is designed to carry out rescue operations quickly and efficiently in the event of a disaster. This system is composed of a server, terminals, and users.

[0106] First, the server collects data in real time from multiple sources as an information gathering means. Specifically, the server uses the API of social media to retrieve posts based on specific hashtags or keywords. It also sends requests to download the latest image data from satellite data providers and connects to emergency notification systems to receive voice data.

[0107] The server then processes the collected data with pre-processing tools. For social media data, text mining tools are used to remove noise and extract post metadata (such as location and time). For satellite image data, image processing tools are used to standardize resolution and perform filtering and segmentation. For emergency call data, speech recognition tools are used to convert speech to text and extract key keywords and phrases.

[0108] The server integrates the preprocessed data into a centralized platform using an integration method. Specifically, it stores the data in a database using a database management system (e.g., MySQL, PostgreSQL) that allows for complex queries. Social media posts, satellite images, and emergency call data are stored in a single repository, creating a linked, integrated dataset.

[0109] The combined data is then analyzed using generative AI models and data analytics, such as computer vision models to identify the extent of damage from satellite imagery and quantify building damage, and natural language processing algorithms (NLP) to extract key damage information from text data and map the frequency and location of damage reports.

[0110] Based on the analysis results, the server automatically assigns rescue priorities using a prioritization method, which uses a risk assessment algorithm to calculate a score that takes into account various factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls to assess the urgency and importance of each data point.

[0111] Finally, the server provides the priorities and analysis results set by the information provision means to each institution in real time. A REST API is provided so that each institution can access the data in real time. Each institution can check the information through the provided data dashboard.

[0112] The terminal receives rescue priority and physical location information sent from the server. The terminal accesses the server's API via the Internet using a mobile device or PC to obtain the latest information. The received data is also stored in a local cache.

[0113] The device uses a color-coded visual display method to visually display the received data on a map. Using a map display library (e.g., Leaflet, Google Maps API), areas are color-coded according to the level of urgency. For example, areas with high urgency are displayed in red, areas with medium urgency in yellow, and areas with low urgency in green.

[0114] Additionally, the device provides GPS-based navigation to help users find the optimal rescue route. Navigation applications (e.g., Google Maps, Waze) calculate the shortest route in real time and provide audio and visual navigation instructions.

[0115] Users check the data displayed on the device and act according to the instructions. For example, a rescue team leader uses the tablet to check the current rescue priority and immediately begin action.

[0116] After completing their activities in the field, users input feedback data into their devices and send it to the server, which continuously updates the information and allows other rescue teams to keep up to date with the latest situation.

[0117] As a specific example of how it works, let's consider the occurrence of a large-scale earthquake. The server collects damage reports from social media posts using hashtags such as "earthquake" and "rescue," and analyzes satellite images to identify the extent of the damage. It also converts voice data from emergency calls into text, extracting and integrating important information. A generative AI model analyzes this data, identifies areas with severe damage, sets rescue priorities, and notifies each rescue team. This is expected to enable rescue teams to act quickly and efficiently, saving many lives.

[0118] Prompt Sentence Examples

[0119] "Collect posts containing specific hashtags from social media data, filter out noise, and extract location information. For example, posts containing the tags earthquake or rescue are targeted."

[0120] "We need to acquire satellite image data, standardize the resolution, and then perform a filtering process. The goal is to identify damaged buildings."

[0121] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0122] Step 1: Data collection

[0123] The server collects data in real time from multiple sources. As input, it uses data from social media APIs, satellite data providers, and emergency notification systems. Specifically, it calls social media APIs to retrieve posts based on specific hashtags or keywords (e.g., "earthquake" or "rescue"). It also sends requests to satellite data providers to download the latest image data. It also receives audio data from emergency notification systems. The output is the collected raw data.

[0124] Step 2: Data Preprocessing

[0125] The server preprocesses the collected data. It uses the raw data collected in step 1 as input. For social media data, it uses text mining tools to remove noise and extract metadata (such as location and time). For satellite image data, it uses image processing tools to standardize the resolution and perform filtering and segmentation processes. For emergency call data, it uses speech recognition tools to convert speech to text and extract important keywords and phrases. The output is the preprocessed data with noise removed and important information extracted.

[0126] Step 3: Data Integration

[0127] The server integrates the preprocessed data. It uses the preprocessed data from step 2 as input. For data integration, it uses a database management system (e.g., MySQL, PostgreSQL) and stores the data in a centralized platform. It generates a linked integrated dataset by integrating social media posts, satellite images, and emergency call data into a single repository. The output is the integrated dataset.

[0128] Step 4: Data analysis

[0129] The server analyzes the merged data using a generative AI model. It uses the dataset merged in step 3 as input. It uses a computer vision model to identify the extent of damage from satellite imagery and quantify the state of building damage. It also uses natural language processing algorithms to extract key damage information from text data and map the frequency and location of damage reports. The output is the analysis results.

[0130] Step 5: Rescue Priority Setting

[0131] The server automatically sets rescue priorities using a risk assessment algorithm. It uses the analysis results obtained in step 4 as input. It evaluates the extent of damage, population density of the affected area, number of emergency calls, etc., and calculates the urgency and importance of each area to set priorities. The output is a prioritized rescue plan.

[0132] Step 6: Delivering results

[0133] The server provides the set priorities and analysis results to each agency in real time. It uses the rescue plan set in step 5 as input. It provides a REST API so that each agency can access the data in real time. Each agency can check the information through a data dashboard. The output is the data provided to each agency.

[0134] Step 7: Displaying and navigating data on your device

[0135] The terminal receives the rescue priority and physical location information sent from the server. It uses the data provided in step 6 as input. It accesses the server's API using a mobile device or PC to obtain the latest information and stores it in a local cache. It uses a map display library to display the received data on a map and color-codes it according to the urgency level. It also provides a navigation tool using GPS to calculate and guide the optimal rescue route. The output is a visual display and navigation information provided to the user.

[0136] Step 8: User feedback and data updates

[0137] The user checks the data displayed on the device, and after completing their activities in the field, they input new information into the device and send it to the server. The input uses new damage information and rescue progress information collected by the user in the field. Based on the feedback, the server updates the database so that other rescue teams can also keep up to date with the latest situation. The output is an updated database and a revised rescue plan.

[0138] The above is the processing flow and specific operation of the program for this system.

[0139] (Application example 1)

[0140] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0141] Rescue operations during disasters must be carried out quickly and efficiently, but integrating data from multiple sources and formulating effective rescue plans is difficult. Local rescue teams also lack the means to visually view real-time updates, which can hinder the selection of optimal rescue routes and rapid decision-making.

[0142] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0143] In this invention, the server includes an information collection means for collecting data in real time from multiple sources, a means for preprocessing the collected data, a means for integrating the preprocessed data, a means for analyzing the integrated data using generative artificial intelligence, a means for automatically setting rescue priorities based on the analysis results, a means for sharing the set priorities and information with each organization, and a means for presenting data and navigating rescue operations using a mobile device or visual display device. This enables efficient and appropriate rescue operations based on real-time information collection and integration. Furthermore, rescue teams can check the latest information in real time through the visual display device, making it easier to select the optimal rescue route.

[0144] "Information collection means" refers to a means for collecting data in real time from multiple information sources.

[0145] "Preprocessing means" refers to means for converting collected data into a format that is easy to analyze.

[0146] The "integration means" is a means for centralizing and integrating the pre-processed data.

[0147] "Generative AI" is AI that generates information that humans can naturally understand through machine learning and data analysis.

[0148] "Analysis means" refers to the means for analyzing the integrated data and extracting the necessary knowledge.

[0149] The "priority setting means" is a means for automatically setting rescue priorities based on the analysis results.

[0150] "Information sharing means" are means for providing set priorities and information to each agency.

[0151] A "mobile device" is a portable information terminal that can be used while on the move.

[0152] A "visual display device" is a device for visually displaying information.

[0153] This invention relates to a system for supporting rapid and efficient rescue operations in the event of a disaster. This system collects data from multiple sources in real time, preprocesses it, integrates it, and analyzes it using a generative AI model, and based on the results, sets rescue priorities and shares them with various organizations to ensure effective rescue operations.

[0154] First, the server collects data in real time from social media, satellite data providers, emergency notification systems, etc. It uses the social media API to obtain posts based on specific hashtags or keywords, sends a request to download the latest image data from the satellite data provider, and receives audio data from the emergency notification system.

[0155] The server then preprocesses the collected data. It uses text mining tools to remove noise from social media data and extract metadata. It uses image processing tools to standardize the resolution of satellite image data and perform filtering and segmentation processes. For emergency call data, it converts speech to text and extracts key keywords and phrases.

[0156] The pre-processed data is then consolidated using an integration method, which stores social media posts, satellite images, and emergency call data in a single repository to generate an integrated dataset.

[0157] The server then analyzes the combined data using generative AI models, using computer vision models to quantify the extent of damage and building damage from satellite imagery, and natural language processing algorithms to extract key damage information from text data and map the frequency and location of damage reports.

[0158] Based on the analysis, the server automatically prioritizes rescue efforts, using a risk assessment algorithm to assess the urgency and importance of each data point and calculate a score that takes into account factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls.

[0159] The set priorities and information are provided to each agency through information sharing channels, which allow agencies to access data in real time using REST APIs and view the information through a provided data dashboard.

[0160] Furthermore, mobile devices and visual display devices are used to visually present the latest information, providing rescue teams with the ability to navigate rescue routes and priorities in real time. The terminal receives rescue priority and location information sent from the server and uses a map display library to color-code the information according to the level of urgency. GPS is also used to provide navigation means for indicating the optimal route, and navigation instructions are given via voice and visuals.

[0161] For example, when an earthquake occurs, the server collects damage reports from social media posts and analyzes satellite images to identify the extent of the damage. It converts voice data from emergency notification systems into text, extracts and centralizes important information, and then the generative AI analyzes this information, identifies areas with severe damage, sets rescue priorities, and notifies each rescue team.

[0162] Example prompt sentence:

[0163] "Please analyze the satellite image data to identify the extent of the damage and quantify the extent of the damage to the buildings."

[0164] "Collect reports of damage from social media posts and prioritize them according to urgency."

[0165] "Convert the voice data from the emergency call system into text and extract important keywords."

[0166] This allows each agency to grasp the situation in real time and carry out rescue operations quickly and efficiently.

[0167] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0168] Step 1:

[0169] The server uses information collection tools to collect data in real time from multiple sources. Here, it uses SNS APIs to obtain posts based on specific hashtags or keywords, sends requests to download the latest image data from satellite data providers, and receives audio data from emergency notification systems. The input data are SNS posts, satellite image data, and audio data, and the output data are these raw data.

[0170] Step 2:

[0171] The server preprocesses the collected data. Specifically, it uses text mining tools to remove noise from the social media data and extract metadata (such as location and time). It also uses image processing tools to standardize the resolution of the satellite image data and perform filtering and segmentation processes. Furthermore, for audio data, it converts the audio into text and extracts important keywords and phrases. The input data is raw data, and the output data is preprocessed data.

[0172] Step 3:

[0173] The server integrates the preprocessed data. The integration method stores the social media posts, satellite images, and emergency call data in a single database to generate an integrated dataset. The input data is the preprocessed individual data, and the output data is the unified integrated dataset.

[0174] Step 4:

[0175] The server analyzes the integrated data using a generative AI model. It uses computer vision models to identify the extent of damage from satellite imagery and quantify the extent of building damage. It also uses natural language processing algorithms to extract key damage information from text data and map the frequency and location of damage reports. The input data is the integrated dataset, and the output data is the analysis results.

[0176] Step 5:

[0177] The server automatically sets rescue priorities based on the analysis results. Using a risk assessment algorithm, it calculates a score to assess the urgency and importance of each data point, taking into account various factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls. The input data are the analysis results, and the output data are rescue priorities.

[0178] Step 6:

[0179] The server shares the set priorities and information with each agency. A REST API is provided as a means of information sharing, allowing each agency to access data in real time. Information is confirmed through the provided data dashboard. The input data is the rescue priority, and the output data is the information accessed by each agency.

[0180] Step 7:

[0181] The terminal receives rescue priority and physical location information sent from the server. Using a map display library, it colors areas according to the level of emergency, for example, red for high emergency, yellow for medium emergency, and green for low emergency. The input data is the information sent from the server, and the output data is the displayed map.

[0182] Step 8:

[0183] The device provides a GPS-based navigation method to help users find the optimal rescue route. The navigation application calculates the shortest route in real time and provides audio and visual navigation instructions. The input data is location information, and the output data is navigation instructions.

[0184] Step 9:

[0185] Users check the data displayed on the terminal and act according to the instructions. The rescue team leader uses the tablet to check the current rescue priorities and immediately begin action. Furthermore, after activities on site, users input new information into the terminal and provide feedback to the server. This continuously updates the information, allowing other rescue teams to understand the latest situation. The input data is new damage information, and the output data is the updated data.

[0186] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0187] The system of this invention is designed to enable rapid and efficient rescue operations in the event of a disaster, and aims to recognize and support the user's emotional state by incorporating an emotion engine. This system is composed of a server, a terminal, and a user.

[0188] First, the server collects data in real time from multiple sources: it uses social media APIs to retrieve posts based on specific hashtags or keywords, it sends API requests to download the latest image data from satellite data providers, and it connects to emergency notification systems to receive voice data.

[0189] The server then preprocesses the collected data. For social media data, it uses text mining tools to remove noise, identify languages, and extract metadata. For satellite image data, it uses image processing tools to standardize resolution and perform filtering and segmentation. For emergency call data, it converts speech to text and uses automatic speech recognition (ASR) tools to extract key keywords and phrases.

[0190] The server then integrates the pre-processed data into a centralized platform that uses a database for complex queries to store social media posts, satellite imagery, and emergency call data in a single repository, creating a unified, interrelated dataset.

[0191] The server analyzes the combined data using generative AI models, such as computer vision models to identify the extent of damage from satellite imagery and quantify building damage, and natural language processing (NLP) algorithms to extract key damage information from text data and map the frequency and location of damage reports.

[0192] Based on the analysis results, the server automatically sets rescue priorities. It uses a risk assessment algorithm to evaluate the urgency and importance of each data point. It calculates a score based on various factors, such as the scale of the damage, the population density of the affected area, and the number of emergency calls, and sets priorities.

[0193] In addition, the system incorporates an emotion engine, which recognizes the user's emotional state based on voice and text data. For example, it analyzes emergency call audio to assess the speaker's level of tension and fear. It also extracts emotional expressions from text data from social media and emergency calls to understand the user's stress level and panic state.

[0194] Using this information, the server can make a more detailed assessment of the situation and re-establish priorities. In particular, if there is a user with a high level of urgency and emotional instability, it can set the rescue of that user as the top priority.

[0195] The server shares the set priorities and analysis results with each institution. A REST API is provided so that each institution can access the information they need in real time. Each institution can check the information through a data dashboard.

[0196] The terminal receives rescue priority and physical location information sent from the server. The terminal accesses the server's API via the Internet using a mobile device or PC to obtain the latest data. The received data is also stored in a local cache.

[0197] The device visually displays the received data on a map, using a map display library to color-code areas according to priority: red for high urgency, yellow for medium urgency, and green for low urgency.

[0198] Additionally, the device provides GPS-based navigation to help users find the optimal rescue route. The navigation application calculates the shortest route in real time and provides audio and visual navigation instructions.

[0199] Users can check the data displayed on the device and act according to the instructions. For example, a rescue team leader can use the tablet to check the latest rescue priorities and immediately begin action.

[0200] After completing an operation in the field, users input new information into the device and provide feedback to the server, which continuously updates the information and allows other rescue teams to keep up to date with the latest situation.

[0201] As a concrete example, if a large earthquake occurs in a certain area, the server collects damage reports from social media posts and analyzes satellite images to identify the extent of the damage. It then converts voice data from emergency calls into text, extracts important information, and centralizes it. Generative AI analyzes this information, identifies areas with severe damage, sets rescue priorities, and notifies each rescue team. Rescue teams can act quickly based on this information, potentially saving many lives. Furthermore, an emotion engine can analyze the emotions in emergency call voices, identify the speaker's state of panic, and prioritize rescue areas.

[0202] The processing flow will be explained below.

[0203] Step 1: Gather information

[0204] The server collects social media data, satellite image data, and emergency call data.

[0205] The server uses the Twitter API and other social media APIs to retrieve posts based on specific hashtags or keywords in real time.

[0206] The server periodically sends an API request to retrieve the latest image data from the satellite data provider.

[0207] The server connects to the emergency call system and receives emergency call voice data in real time.

[0208] Step 2: Data Preprocessing

[0209] The server preprocesses the collected data.

[0210] For social media data, the server uses text mining tools to remove noise, identify language, and extract metadata (e.g., posting time and location information).

[0211] The server uses image processing tools to standardize the resolution of the satellite image data and perform filtering and segmentation processing.

[0212] For emergency call data, the server converts the speech to text and uses automatic speech recognition (ASR) tools to extract key keywords and phrases.

[0213] Step 3: Information integration

[0214] The server consolidates the pre-processed data.

[0215] The server uses complex queries to store social media posts, satellite images, and emergency call data in a centralized database, creating a unified dataset that correlates these data.

[0216] Step 4: Generative AI analysis

[0217] The server analyzes the combined data using the generated AI model.

[0218] The server uses computer vision models to identify the extent of damage from satellite imagery and quantify the damage to specific objects (e.g., buildings).

[0219] The server uses natural language processing (NLP) algorithms to extract important damage information from the text data and map it to the frequency and location of damage reports.

[0220] Step 5: Setting rescue priorities

[0221] The server automatically sets rescue priorities based on the analysis results.

[0222] The server uses a risk assessment algorithm to rate the urgency and importance of each data point, calculating a score based on various factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls, and then assigning a priority to each.

[0223] Step 6: Sentiment Engine Analysis

[0224] The server uses an emotion engine to analyze the user's emotional state from the voice data and text data.

[0225] The server analyzes the voice data of the emergency call and evaluates the speaker's level of tension and fear.

[0226] The server extracts emotional expressions from text data from social media and emergency calls to understand the user's stress level and panic state.

[0227] Step 7: Emotionally based reprioritization

[0228] The server re-establishes rescue priorities based on the analysis results of the emotion engine.

[0229] If the server finds an emotionally unstable user, it updates the rescue priority to give that area top priority.

[0230] Step 8: Information sharing

[0231] The server shares the set priorities and analysis results with each institution.

[0232] The server provides a REST API, allowing organizations to access the information they need in real time. Organizations can connect to the data dashboard through the REST API to view the latest information.

[0233] Step 9: Receiving Data

[0234] The terminal receives the rescue priority and location information transmitted from the server.

[0235] Mobile devices and PCs access the server's API via the Internet to periodically retrieve the latest data, which is then stored in a local cache.

[0236] Step 10: Visualization

[0237] The data received by the device is displayed on a map.

[0238] The device uses a mapping library (e.g., Google Maps API or Leaflet) to display areas color-coded based on priority: high urgency areas are displayed in red, medium urgency areas in yellow, and low urgency areas in green.

[0239] Step 11: Navigation Instructions

[0240] The device will instruct the user on the optimal rescue route.

[0241] The device uses GPS data to connect to a map application (e.g., Mapbox API) to provide real-time navigation, providing audio and visual navigation instructions to help users reach their destination efficiently.

[0242] Step 12: Verify the information

[0243] The user checks the information displayed on the device and acts according to the instructions.

[0244] Users can check the latest rescue priority and route information through the application on their device and begin taking action immediately.

[0245] Step 13: Provide feedback

[0246] The user inputs new local information into the terminal and provides feedback to the server.

[0247] Users input new information obtained on-site (e.g., new damage or progress of rescue efforts) into their devices and send it as data to the server. The server updates the database based on this feedback information and shares the latest information with other rescue teams.

[0248] Example 2

[0249] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0250] In order to realize rapid and efficient rescue operations during disasters, it is important to collect and analyze data from multiple sources in real time. However, data collected from many sources comes in different formats and has different qualities, making it difficult to integrate them. Furthermore, it is also challenging to appropriately prioritize rescue operations based on the collected data and to take into account the emotional state of users.

[0251] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for collecting data in real time from multiple information sources as information collection means; means for preprocessing the collected data using technologies such as text mining, image processing, and automatic speech recognition; means for integrating the preprocessed data; means for integrating the preprocessed data using a centralized platform; means for analyzing the integrated data using a generative artificial intelligence model to identify the extent of damage and the state of destruction caused by the disaster; means for automatically setting rescue priorities using a risk assessment algorithm based on the analysis results; means for recognizing the user's emotional state from voice data and text data using an emotion engine and resetting priorities; and means for sharing the set priorities and information with various organizations. This makes it possible to collect data in real time from multiple information sources, integrate and analyze it quickly and efficiently, and realize appropriate rescue operations according to the situation.

[0252] "Information collection means" refers to a means for collecting data in real time from multiple information sources.

[0253] "Text mining" is a technology that removes noise from text data and extracts specific information.

[0254] "Image processing" is a technology that standardizes the resolution of image data and performs filtering and segmentation processing.

[0255] "Automatic speech recognition" is a technology that converts voice data into text data.

[0256] A "centralized platform" is a platform for integrating and managing pre-processed data in various formats.

[0257] A "generative artificial intelligence model" is a machine learning model used to perform analysis based on collected data.

[0258] The "risk assessment algorithm" is an algorithm that calculates the urgency and importance of data and sets rescue priorities.

[0259] "Emotion engine" is a technology that analyzes a user's emotional state from voice data and text data.

[0260] "Rescue priorities" are the order of importance and urgency of rescue operations established based on collected and analyzed data.

[0261] "Data sharing tools" are tools for communicating established priorities and related information to each agency.

[0262] "Navigation means" refers to a means that uses GPS to indicate the optimal route and guide the user.

[0263] The system of this invention is designed to carry out rapid and efficient rescue operations in the event of a disaster. The system consists of three entities: a server, a terminal, and a user, each of which has a specific function.

[0264] First, the server collects data in real time from multiple sources: it uses social media APIs to retrieve posts based on specific hashtags or keywords, it makes API requests to download the latest image data from satellite data providers, and it connects to emergency notification systems to receive voice data.

[0265] The server then preprocesses the collected data. For social media data, text mining tools are used to remove noise, identify languages, and extract metadata. For satellite image data, image processing tools (e.g., OpenCV) are used to standardize resolution and perform filtering and segmentation. For emergency call data, speech-to-text conversion is performed and key keywords and phrases are extracted using automatic speech recognition (ASR) tools (e.g., Google Cloud Speech-to-Text).

[0266] The pre-processed data is then integrated into a centralized platform that utilizes a database (e.g., an SQL database) for complex queries, storing social media posts, satellite imagery, and emergency call data in a single repository to generate a unified dataset.

[0267] The server analyzes the combined data using generative AI models (e.g., TensorFlow, BERT), for example, computer vision models to identify the extent of damage from satellite imagery and quantify the extent of building damage, and natural language processing (NLP) algorithms to extract key damage information from text data and map the frequency and location of damage reports.

[0268] Based on the analysis results, the server uses a risk assessment algorithm to automatically prioritize rescue efforts. For example, it calculates scores based on various factors, such as the scale of the damage, the population density of the affected area, and the number of emergency calls, and then sets priorities. It also incorporates an emotion engine into the system to recognize the user's emotional state based on voice and text data. This allows it to re-set priorities, and if there is a user with a particularly high level of urgency and emotional instability, it can assign top priority to their rescue.

[0269] The server shares the set priorities and analysis results with each institution via a REST API, and each institution can view this information in real time through a data dashboard (e.g., a web application).

[0270] The terminal receives rescue priority and physical location information sent from the server. The terminal accesses the server's API via the Internet using a mobile device or PC to obtain the latest data. The received data is also stored in a local cache. The terminal uses a map display library (e.g., Leaflet.js) to visually display the data on a map. Areas are color-coded according to the level of urgency, with red for medium-level areas, yellow for low-level areas, and green for low-level areas.

[0271] Additionally, the device provides GPS-based navigation to help users find the optimal rescue route. The navigation application (e.g., Google Maps API) calculates the shortest route in real time and provides audio and visual navigation instructions.

[0272] Users check the data displayed on the device and act according to the instructions. For example, a rescue team leader can use the tablet to check the latest rescue priorities and immediately begin action. After completing their activities on the ground, the user enters new information into the device and provides feedback to the server. This updates the information, allowing other rescue teams to understand the latest situation.

[0273] As a concrete example, if a large earthquake occurs in a certain area, the server collects damage reports from social media posts and analyzes satellite images to identify the extent of the damage. It then converts voice data from emergency calls into text, extracts important information, and centralizes it. Generative AI analyzes this information, identifies areas with severe damage, sets rescue priorities, and notifies each rescue team. Rescue teams can act quickly based on this information, potentially saving many lives. Furthermore, an emotion engine can analyze the emotions in emergency call voices, identify the speaker's state of panic, and prioritize rescue areas.

[0274] Example prompt sentence:

[0275] "An earthquake occurs in a certain area. Explain the steps to collect disaster information, identify the extent of the damage, and set rescue priorities. Also, explain how you use an emotion engine to perform sentiment analysis of emergency call voices and re-prioritize them."

[0276] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0277] System program processing flow

[0278] Step 1: Collect data

[0279] 1. Input: Social media posts, satellite images, emergency call audio data

[0280] 2. Specific server operations:

[0281] The server uses the social media API to retrieve posts based on specific hashtags or keywords.

[0282] Example: GET https: / / api.sns.service / tweets?query=%23Earthquake Damage

[0283] The server sends an API request to download the latest image data from the satellite data provider.

[0284] Example: GET https: / / api.satellite.service / images

[0285] A server receives voice data from an emergency notification system.

[0286] Example: GET https: / / api.emergency.service / calls

[0287] 3. Output: Collected raw data

[0288] Step 2: Preprocessing the data

[0289] 1. Input: Collected raw data (SNS data, satellite images, emergency call audio data)

[0290] 2. Specific server operations:

[0291] Text mining tools are used on social media data to remove noise, identify language, and extract metadata.

[0292] Example: text = clean_noise(raw_text)

[0293] Image processing tools are used on satellite image data to unify resolution, perform filtering and segmentation.

[0294] Example: processed_image = cv2.resize(raw_image, new_size)

[0295] Automatic speech recognition (ASR) tools are used on emergency call voice data to convert speech to text and extract key keywords and phrases.

[0296] Example: text_data = asr_tool.recognize(audio_data)

[0297] 3. Output: Preprocessed data (clean text data, processed image data, converted audio data)

[0298] Step 3: Integrate the data

[0299] 1. Input: Preprocessed data (clean text data, processed image data, converted audio data)

[0300] 2. Specific server operations:

[0301] The pre-processed data is integrated into a centralized platform, where it is stored in a database and linked to the same repository.

[0302] Example: db.insert('disaster_data', processed_data)

[0303] 3. Output: Integrated dataset

[0304] Step 4: Analyze the data

[0305] 1. Input: Integrated dataset

[0306] 2. Specific server operations:

[0307] Analyze data using generative artificial intelligence models (e.g., TensorFlow, BERT).

[0308] The extent of the damage is identified from satellite images, and the extent of damage to the building is quantified using a computer vision model.

[0309] Example: damage_score = vision_model.predict(image)

[0310] Important damage information is extracted from text data using natural language processing (NLP) algorithms, and the frequency and location of damage reports are mapped.

[0311] Example: key_info = nlp_model.extract(text)

[0312] 3. Output: Damage extent, damage status assessment, frequency and location of damage information

[0313] Step 5: Setting rescue priorities

[0314] 1. Input: Analysis results (damage extent, damage status assessment, frequency and location of damage information)

[0315] 2. Specific server operations:

[0316] Using risk assessment algorithms, the system assesses the urgency and importance of each data point and automatically prioritizes rescue efforts.

[0317] Example: priority_score = risk_algorithm.calculate(data_points)

[0318] 3. Output: List of rescue priorities

[0319] Step 6: Use the Emotion Engine

[0320] 1. Input: Audio data, text data

[0321] 2. Specific server operations:

[0322] An emotion engine is used to analyze the user's emotional state from the voice and text data.

[0323] Example: emotion_score = emotion_engine.analyze(audio_text_data)

[0324] 3. Output: User's emotion evaluation score

[0325] Step 7: Reprioritize

[0326] 1. Input: List of rescue priorities, user's emotional evaluation score

[0327] 2. Specific server operations:

[0328] Reassess and update rescue priority list based on emotional state.

[0329] Example: updated_priority = update_priority_based_on_emotion(priority_list, emotion_scores)

[0330] 3. Output: Updated rescue priority list

[0331] Step 8: Share and view your data

[0332] 1. Input: Updated rescue priority list

[0333] 2. Specific server operations:

[0334] The set priorities and related information are shared with each institution via a REST API.

[0335] Example: POST https: / / api.rescue.service / priorities

[0336] 3. Specific device behavior:

[0337] The device accesses the server's API to receive the latest rescue priority and location information.

[0338] Example: GET https: / / api.rescue.service / latest

[0339] The received data is visually displayed on a map using a map display library (e.g., Leaflet.js). Areas are color-coded in red, yellow, or green according to the level of urgency.

[0340] Example: L.marker(coordinates).addTo(map)

[0341] It uses GPS to perform navigation functions that provide users with the best rescue route.

[0342] Example: directionsService.route(request, callback)

[0343] 4. Output: Display rescue priority and optimal rescue route

[0344] The user checks the data displayed on the device and takes action according to the instructions. After completing their activities in the field, they input new information into the device and provide feedback to the server. This continuously updates the information, allowing other rescue teams to keep up to date with the latest situation.

[0345] (Application example 2)

[0346] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0347] In the event of a disaster, it is extremely important to carry out rapid and efficient rescue operations. Damage assessment is particularly difficult in closed environments such as factories, and the presence of workers with unstable emotional states can reduce the efficiency of rescue operations. The present invention aims to achieve more effective rescue operations by using an emotion engine to recognize the emotional state of victims and integrating it with damage information to set rescue priorities.

[0348] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes, as information collection means, means for collecting data in real time from multiple information sources, means for preprocessing the collected data, means for integrating the preprocessed data, means for analyzing the integrated data using generative artificial intelligence, an emotion engine that recognizes the user's emotional state, means for automatically setting rescue priorities based on the analysis results and the user's emotional state, and means for sharing the set priorities and information with each organization. This makes it possible to set rescue priorities taking damage information and emotional state into consideration, thereby realizing efficient and prompt rescue operations.

[0349] "Information collection means" refers to a means for collecting data in real time from multiple information sources.

[0350] "Preprocessing" refers to processing the collected data such as noise removal, filtering, and standardization.

[0351] "Integration" means centralizing pre-processed data and putting it into a shareable format.

[0352] "Generative AI" is AI that uses technologies such as machine learning and deep learning to extract useful information and patterns from large amounts of data.

[0353] An "emotion engine" is an engine that recognizes and evaluates a user's emotional state from voice data, text data, and the like.

[0354] "Rescue priority" is a criterion for determining the order in which rescue operations should be carried out based on the analysis results and the user's emotional state.

[0355] "Agency" means any public or private entity involved in disaster relief or damage assessment.

[0356] "Image data" is digital data that includes visual information captured by a camera or satellite.

[0357] "Text data" refers to digital data that includes text information such as social media posts and emergency calls.

[0358] "Audio data" is digital data that includes sound information, such as emergency calls and the voices of disaster victims.

[0359] "User emotional state data" is data that expresses the emotions (for example, stress, panic, fear, etc.) shown by the user as numerical values ​​or evaluations.

[0360] "Navigation means" refers to a means of providing the optimal route using GPS and guiding the person to the rescue location.

[0361] "Visual navigation instruction means" refers to a means of providing visual navigation information through maps, real-time video, etc.

[0362] The present invention provides a system for providing effective rescue operations in the event of a disaster, incorporating information collection means, preprocessing means, data integration means, data analysis means using generative artificial intelligence, and an emotion engine. Specific embodiments of the system are described below.

[0363] 1. Information gathering methods

[0364] The server collects data from multiple sources. The hardware used includes mobile devices (smartphones), smart glasses, and factory robots. The software uses APIs from social media platforms, satellite data providers, and emergency call systems. Through these, social media posts, satellite image data, and emergency call audio data are acquired in real time.

[0365] 2. Data preprocessing methods

[0366] The server performs preprocessing on the collected data. For example, for social media data, a text mining tool is used to remove noise and extract metadata. For satellite image data, an image processing tool (e.g., OpenCV) is used to standardize the resolution and perform filtering. For emergency call data, a voice recognition tool (e.g., Google Speech-to-Text API) is used to convert speech to text.

[0367] 3. Data Integration Methods

[0368] The server integrates the pre-processed data using a database capable of handling complex queries. Data from different sources (social media posts, satellite images, emergency call data) is stored in a single repository and managed as an integrated dataset.

[0369] 4. Data Analysis Methods

[0370] Generative AI models (e.g., TensorFlow, PyTorch) are used to analyze the integrated data, computer vision models are used to identify the extent of damage from satellite imagery and quantify the extent of building damage, and natural language processing (NLP) algorithms are used to extract key information from text data from social media posts and emergency calls, mapping the frequency and location of damage reports.

[0371] 5. Emotion Engine

[0372] The emotion engine recognizes the user's emotional state from voice and text data. For example, it analyzes emergency call audio to assess the speaker's level of tension and fear. It also extracts emotional expressions from text data from social media and emergency calls to understand the user's stress level and panic state.

[0373] 6. Rescue Priority Setting

[0374] The server automatically prioritizes rescue efforts based on the analysis results and the emotion engine's assessment. It uses a risk assessment algorithm to determine the urgency and importance of each data point and calculates a score based on factors such as the scale of damage, population density, and number of emergency calls.

[0375] 7. Information sharing methods

[0376] The server shares the set priorities and analysis results with each institution, allowing them to access the necessary information in real time via a REST API and providing the information through a data dashboard.

[0377] 8. Device Navigation Methods

[0378] The device includes a navigation tool that uses GPS to provide optimal rescue route guidance. The navigation application calculates the shortest route in real time and provides audio and visual navigation instructions.

[0379] Specific examples

[0380] For example, if an earthquake occurs at Factory A, the server collects data from social media posts and emergency calls and uses an emotion engine to evaluate the speaker's emotional state. The server then identifies the extent of the damage and sends rescue instructions to robots, prioritizing areas with the highest level of emergency. Managers can grasp the situation in real time through smart glasses or smartphones and take appropriate command.

[0381] Prompt Sentence Examples

[0382] Please prioritize the damage based on the following social media posts and emergency call data. Also, use sentiment analysis to identify cases with the highest urgency.

[0383] ---

[0384] Social Media Posts:

[0385] Earthquake. Building collapsed. Help needed.

[0386] Disaster: The road is blocked, please come and help immediately.

[0387] Emergency Call Data:

[0388] Please help, my house has collapsed and I can't get out.

[0389] The water is rising rapidly and it is very dangerous.

[0390] ---

[0391] output:

[0392] High-urgency location: A place where the house has collapsed and you cannot get out (sentiment analysis: high stress level)

[0393] Priority rescue instruction: dispatch robots to begin immediate rescue

[0394] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0395] Step 1:

[0396] The server collects data from multiple sources: it uses social media APIs to retrieve posts based on specific hashtags or keywords in real time, downloads the latest image data through the satellite data provider's API, and connects to the emergency call system to receive voice data. The inputs for this step are social media posts, satellite image data, and emergency call voice data, and the output is the collected raw data.

[0397] Step 2:

[0398] The server preprocesses the collected data. For social media data, text mining tools are used to remove noise and extract metadata. For satellite image data, image processing tools (e.g., OpenCV) are used to standardize the resolution and perform filtering and segmentation. For emergency call data, speech recognition tools (e.g., Google Speech-to-Text API) are used to convert speech to text and extract important keywords and phrases. The input of this step is the collected raw data, and the output is the preprocessed data.

[0399] Step 3:

[0400] The server consolidates the preprocessed data. It integrates it into a database system and centralizes data from different sources. This involves storing social media posts, satellite images, and emergency call data in a single repository to generate a consolidated dataset. The input of this step is the preprocessed data, and the output is the consolidated dataset.

[0401] Step 4:

[0402] The server analyzes the integrated dataset with a generative AI model. It uses computer vision models to identify the extent of damage from satellite imagery and quantify the extent of building damage. It uses NLP algorithms to extract key damage information from text data and map the frequency and location of damage reports. The input for this step is the integrated dataset, and the output is a damage assessment and damage location mapping results.

[0403] Step 5:

[0404] The server uses an emotion engine to recognize the user's emotional state from voice and text data. It analyzes emergency call voice to evaluate the speaker's level of tension and fear. It also extracts emotional expressions from text data from social media and emergency calls to understand the user's stress level and panic state. The input for this step is voice and text data, and the output is data on the user's emotional state.

[0405] Step 6:

[0406] The server automatically sets rescue priorities based on the analysis results and emotional state data. It uses a risk assessment algorithm to evaluate the urgency and importance of each data point. It calculates a score based on factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls, and sets rescue priorities. The inputs to this step are the analysis results and emotional state data, and the output is a rescue priority list.

[0407] Step 7:

[0408] The server provides information through a REST API to share the set priorities and analysis results with each agency. Each agency can view the information in real time using a data dashboard. The input of this step is the rescue priority list and analysis results, and the output is the rescue instructions sent to each agency.

[0409] Step 8:

[0410] The device receives rescue priority and physical location information sent from the server. The received data is visually displayed on a map and provides the optimal rescue route using GPS. The navigation application calculates the shortest route in real time and provides audio and visual navigation instructions. The input of this step is rescue instruction data, and the output is instructions for the optimal rescue route.

[0411] Specific examples

[0412] For example, if an earthquake occurs at Factory A, the server collects data from social media posts and emergency calls and uses an emotion engine to evaluate the speaker's emotional state. The server then identifies the extent of the damage and sends rescue instructions to robots, prioritizing areas with the highest level of emergency. Managers can grasp the situation in real time through smart glasses or smartphones and take appropriate command.

[0413] Prompt Sentence Examples

[0414] Please prioritize the damage based on the following social media posts and emergency call data. Also, use sentiment analysis to identify cases with the highest urgency.

[0415] ---

[0416] Social Media Posts:

[0417] Earthquake. Building collapsed. Help needed.

[0418] Disaster: The road is blocked, please come and help immediately.

[0419] Emergency Call Data:

[0420] Please help, my house has collapsed and I can't get out.

[0421] The water is rising rapidly and it is very dangerous.

[0422] ---

[0423] output:

[0424] High-urgency location: A place where the house has collapsed and you cannot get out (sentiment analysis: high stress level)

[0425] Priority rescue instruction: dispatch robots to begin immediate rescue

[0426] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0427] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0428] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0429] [Second embodiment]

[0430] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0431] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0432] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0433] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0434] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0435] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0436] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0437] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0438] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0439] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0440] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0441] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0442] The system of the present invention is designed to carry out rescue operations quickly and efficiently in the event of a disaster. This system is composed of a server, terminals, and users.

[0443] The server collects data in real time from multiple sources, including using social media APIs to retrieve posts based on specific hashtags or keywords, sending requests to download the latest image data from satellite data providers, and connecting to emergency call systems to receive voice data.

[0444] The server then preprocesses the collected data. For SMS data, text mining tools are used to remove noise and extract post metadata (such as location and time). For satellite image data, image processing tools are used to standardize resolution and perform filtering and segmentation. For emergency call data, speech-to-text conversion is performed to extract key keywords and phrases.

[0445] The server then integrates the pre-processed data into a centralized platform that uses a database with complex query capabilities to store social media posts, satellite imagery, and emergency call data in a single repository, resulting in a connected, integrated dataset.

[0446] The server then analyzes the combined data using generative AI models. For example, the server uses computer vision models to identify the extent of damage from satellite imagery and quantify building damage. It also uses natural language processing algorithms to extract key damage information from text data and map the frequency and location of damage reports.

[0447] Based on the analysis results, the server automatically assigns rescue priorities. Using a risk assessment algorithm, the server calculates a score to assess the urgency and importance of each data point, taking into account various factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls. Rescue priorities are then assigned based on this score.

[0448] Finally, the server provides the set priorities and analysis results to each institution in real time. A REST API is provided so that each institution can access the data in real time. Each institution can check the information through the provided data dashboard.

[0449] The device receives rescue priority and physical location information sent from the server. The device accesses the server's API via the internet using a mobile device or PC to retrieve the latest information. The data is also stored in a local cache.

[0450] The device uses a map display library to visually display the received data on a map, color-coding areas according to priority: for example, red for high urgency, yellow for medium urgency, and green for low urgency.

[0451] Additionally, the device provides a GPS-based navigation tool to help users find the optimal rescue route. The navigation application calculates the shortest route in real time and provides audio and visual navigation instructions.

[0452] Users can check the data displayed on the device and act according to the instructions. For example, a rescue team leader can use the tablet to check the current rescue priority and immediately begin action.

[0453] After completing an operation in the field, users input new information into the device and provide feedback to the server, which continuously updates the information and allows other rescue teams to keep up to date with the latest situation.

[0454] As a concrete example, if a large earthquake occurs in a certain area, the server collects damage reports from social media posts and analyzes satellite images to identify the extent of the damage. It then converts voice data from emergency calls into text, extracts important information, and centralizes it. Generative AI analyzes this information, identifies areas with severe damage, sets rescue priorities, and notifies each rescue team. Rescue teams can act quickly based on this information, potentially saving many lives.

[0455] The processing flow will be explained below.

[0456] Step 1: Gather information

[0457] The server collects social media data, satellite image data, and emergency call data.

[0458] The server uses the Twitter API and other social media APIs to retrieve posts based on specific hashtags or keywords in real time.

[0459] The server periodically sends an API request to retrieve the latest image data from the satellite data provider.

[0460] The server connects to the emergency call system and receives emergency call voice data in real time.

[0461] Step 2: Data Preprocessing

[0462] The server preprocesses the collected data.

[0463] For social media data, the server uses text mining tools to remove noise, identify language, and extract metadata (e.g., posting time and location information).

[0464] The server uses image processing tools to standardize the resolution of the satellite image data and perform filtering and segmentation processing.

[0465] For emergency call data, the server converts the speech to text and uses automatic speech recognition (ASR) tools to extract key keywords and phrases.

[0466] Step 3: Information integration

[0467] The server consolidates the pre-processed data.

[0468] The server uses complex queries to store social media posts, satellite images, and emergency call data in a centralized database, creating a unified dataset that correlates these data.

[0469] Step 4: Generative AI analysis

[0470] The server analyzes the combined data using the generated AI model.

[0471] The server uses computer vision models to identify the extent of damage from satellite imagery and quantify the damage to specific objects (e.g., buildings).

[0472] The server uses natural language processing (NLP) algorithms to extract important damage information from the text data and map it to the frequency and location of damage reports.

[0473] Step 5: Setting rescue priorities

[0474] The server automatically sets rescue priorities based on the analysis results.

[0475] The server uses a risk assessment algorithm to rate the urgency and importance of each data point, calculating a score based on various factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls, and then assigning a priority to each.

[0476] Step 6: Information sharing

[0477] The server shares the set priorities and analysis results with each institution.

[0478] The server provides a REST API, allowing organizations to access the information they need in real time. Organizations can connect to the data dashboard through the REST API to view the latest information.

[0479] Step 7: Receiving Data

[0480] The terminal receives the rescue priority and location information transmitted from the server.

[0481] Mobile devices and PCs access the server's API via the Internet to periodically retrieve the latest data, which is then stored in a local cache.

[0482] Step 8: Visualization

[0483] The data received by the device is displayed on a map.

[0484] The device uses a mapping library (e.g., Google Maps API or Leaflet) to display areas color-coded based on priority: high urgency areas are displayed in red, medium urgency areas in yellow, and low urgency areas in green.

[0485] Step 9: Navigation Instructions

[0486] The device will instruct the user on the optimal rescue route.

[0487] The device uses GPS data to connect to a map application (e.g., Mapbox API) to provide real-time navigation, providing audio and visual navigation instructions to help users reach their destination efficiently.

[0488] Step 10: Verify the information

[0489] The user checks the information displayed on the device and acts according to the instructions.

[0490] Users can check the latest rescue priority and route information through the application on their device and begin taking action immediately.

[0491] Step 11: Provide feedback

[0492] The user inputs new local information into the terminal and provides feedback to the server.

[0493] Users input new information obtained on-site (e.g., new damage or progress of rescue efforts) into their devices and send it as data to the server. The server updates the database based on this feedback information and shares the latest information with other rescue teams.

[0494] Example 1

[0495] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0496] Rescue operations during disasters must be carried out quickly and efficiently, but it is difficult to properly collect, analyze, and integrate data from multiple sources. Furthermore, it is not easy to grasp the damage situation in real time and provide the optimal rescue route. With conventional systems, data collection and integration can take time, making it difficult to effectively prioritize rescue operations. This can reduce the efficiency of rescue operations and put many lives at risk. The purpose of this invention is to solve these problems and enable fast and efficient rescue operations during disasters.

[0497] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0498] In this invention, the server includes information collection means, including means for collecting data in real time from multiple sources, means for preprocessing the collected data, means for integrating the preprocessed data, means for analyzing the integrated data using a generated AI model, means for automatically setting rescue priorities based on the analysis results, means for providing the set priorities and information to each organization, means for visually displaying the rescue priorities received by the terminal on a color-coded map, and means for providing optimal rescue routes using GPS. This allows for the rapid and effective collection, preprocessing, and integration of data from multiple sources and analysis using the generated AI model, enabling accurate rescue prioritization and real-time information provision. Furthermore, the efficiency of rescue operations is significantly improved by visualizing the damage situation and providing optimal route guidance through the terminal.

[0499] "Information collection means" refers to a means for collecting data in real time from multiple information sources.

[0500] "Preprocessing means" refers to a means of converting collected data into a form that is easier to analyze by performing processes such as noise removal and metadata extraction on the data.

[0501] The "integration means" is a means for integrating pre-processed data into a centralized platform and storing it in a database that allows for complex queries, etc.

[0502] A "generative AI model" is an artificial intelligence model used to analyze collected and pre-processed data, including, for example, computer vision models and natural language processing algorithms.

[0503] "Data analysis means" refers to a means of analyzing the integrated data using a generative AI model to identify the extent of damage and extract important information.

[0504] The "priority setting means" is a means for automatically determining the priority of rescue operations based on the analysis results and using a risk assessment algorithm.

[0505] "Information provision means" refers to the means for providing the set priorities and related information to each relevant organization in real time. Specifically, this includes REST APIs and data dashboards.

[0506] The "color-coded visual display means" is a means of displaying rescue priority and damage information received by the terminal on a map in different colors. By indicating the urgency level by color for each area, it makes it easier to visually check the information.

[0507] "Navigation Means" means a means that utilizes GPS to provide an optimal rescue route, calculates the shortest route in real time, and includes audio and visual instructions.

[0508] MODE FOR CARRYING OUT THE INVENTION

[0509] The system of the present invention is designed to carry out rescue operations quickly and efficiently in the event of a disaster. This system is composed of a server, terminals, and users.

[0510] First, the server collects data in real time from multiple sources as an information gathering means. Specifically, the server uses the API of social media to retrieve posts based on specific hashtags or keywords. It also sends requests to download the latest image data from satellite data providers and connects to emergency notification systems to receive voice data.

[0511] The server then processes the collected data with pre-processing tools. For social media data, text mining tools are used to remove noise and extract post metadata (such as location and time). For satellite image data, image processing tools are used to standardize resolution and perform filtering and segmentation. For emergency call data, speech recognition tools are used to convert speech to text and extract key keywords and phrases.

[0512] The server integrates the preprocessed data into a centralized platform using an integration method. Specifically, it stores the data in a database using a database management system (e.g., MySQL, PostgreSQL) that allows for complex queries. Social media posts, satellite images, and emergency call data are stored in a single repository, creating a linked, integrated dataset.

[0513] The combined data is then analyzed using generative AI models and data analytics, such as computer vision models to identify the extent of damage from satellite imagery and quantify building damage, and natural language processing algorithms (NLP) to extract key damage information from text data and map the frequency and location of damage reports.

[0514] Based on the analysis results, the server automatically assigns rescue priorities using a prioritization method, which uses a risk assessment algorithm to calculate a score that takes into account various factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls to assess the urgency and importance of each data point.

[0515] Finally, the server provides the priorities and analysis results set by the information provision means to each institution in real time. A REST API is provided so that each institution can access the data in real time. Each institution can check the information through the provided data dashboard.

[0516] The terminal receives rescue priority and physical location information sent from the server. The terminal accesses the server's API via the Internet using a mobile device or PC to obtain the latest information. The received data is also stored in a local cache.

[0517] The device uses a color-coded visual display method to visually display the received data on a map. Using a map display library (e.g., Leaflet, Google Maps API), areas are color-coded according to the level of urgency. For example, areas with high urgency are displayed in red, areas with medium urgency in yellow, and areas with low urgency in green.

[0518] Additionally, the device provides GPS-based navigation to help users find the optimal rescue route. Navigation applications (e.g., Google Maps, Waze) calculate the shortest route in real time and provide audio and visual navigation instructions.

[0519] Users check the data displayed on the device and act according to the instructions. For example, a rescue team leader uses the tablet to check the current rescue priority and immediately begin action.

[0520] After completing their activities in the field, users input feedback data into their devices and send it to the server, which continuously updates the information and allows other rescue teams to keep up to date with the latest situation.

[0521] As a specific example of how it works, let's consider the occurrence of a large-scale earthquake. The server collects damage reports from social media posts using hashtags such as "earthquake" and "rescue," and analyzes satellite images to identify the extent of the damage. It also converts voice data from emergency calls into text, extracting and integrating important information. A generative AI model analyzes this data, identifies areas with severe damage, sets rescue priorities, and notifies each rescue team. This is expected to enable rescue teams to act quickly and efficiently, saving many lives.

[0522] Prompt Sentence Examples

[0523] "Collect posts containing specific hashtags from social media data, filter out noise, and extract location information. For example, posts containing the tags earthquake or rescue are targeted."

[0524] "We need to acquire satellite image data, standardize the resolution, and then perform a filtering process. The goal is to identify damaged buildings."

[0525] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0526] Step 1: Data collection

[0527] The server collects data in real time from multiple sources. As input, it uses data from social media APIs, satellite data providers, and emergency notification systems. Specifically, it calls social media APIs to retrieve posts based on specific hashtags or keywords (e.g., "earthquake" or "rescue"). It also sends requests to satellite data providers to download the latest image data. It also receives audio data from emergency notification systems. The output is the collected raw data.

[0528] Step 2: Data Preprocessing

[0529] The server preprocesses the collected data. It uses the raw data collected in step 1 as input. For social media data, it uses text mining tools to remove noise and extract metadata (such as location and time). For satellite image data, it uses image processing tools to standardize the resolution and perform filtering and segmentation processes. For emergency call data, it uses speech recognition tools to convert speech to text and extract important keywords and phrases. The output is the preprocessed data with noise removed and important information extracted.

[0530] Step 3: Data Integration

[0531] The server integrates the preprocessed data. It uses the preprocessed data from step 2 as input. For data integration, it uses a database management system (e.g., MySQL, PostgreSQL) and stores the data in a centralized platform. It generates a linked integrated dataset by integrating social media posts, satellite images, and emergency call data into a single repository. The output is the integrated dataset.

[0532] Step 4: Data analysis

[0533] The server analyzes the merged data using a generative AI model. It uses the dataset merged in step 3 as input. It uses a computer vision model to identify the extent of damage from satellite imagery and quantify the state of building damage. It also uses natural language processing algorithms to extract key damage information from text data and map the frequency and location of damage reports. The output is the analysis results.

[0534] Step 5: Rescue Priority Setting

[0535] The server automatically sets rescue priorities using a risk assessment algorithm. It uses the analysis results obtained in step 4 as input. It evaluates the extent of damage, population density of the affected area, number of emergency calls, etc., and calculates the urgency and importance of each area to set priorities. The output is a prioritized rescue plan.

[0536] Step 6: Delivering results

[0537] The server provides the set priorities and analysis results to each agency in real time. It uses the rescue plan set in step 5 as input. It provides a REST API so that each agency can access the data in real time. Each agency can check the information through a data dashboard. The output is the data provided to each agency.

[0538] Step 7: Displaying and navigating data on your device

[0539] The terminal receives the rescue priority and physical location information sent from the server. It uses the data provided in step 6 as input. It accesses the server's API using a mobile device or PC to obtain the latest information and stores it in a local cache. It uses a map display library to display the received data on a map and color-codes it according to the urgency level. It also provides a navigation tool using GPS to calculate and guide the optimal rescue route. The output is a visual display and navigation information provided to the user.

[0540] Step 8: User feedback and data updates

[0541] The user checks the data displayed on the device, and after completing their activities in the field, they input new information into the device and send it to the server. The input uses new damage information and rescue progress information collected by the user in the field. Based on the feedback, the server updates the database so that other rescue teams can also keep up to date with the latest situation. The output is an updated database and a revised rescue plan.

[0542] The above is the processing flow and specific operation of the program for this system.

[0543] (Application example 1)

[0544] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0545] Rescue operations during disasters must be carried out quickly and efficiently, but integrating data from multiple sources and formulating effective rescue plans is difficult. Local rescue teams also lack the means to visually view real-time updates, which can hinder the selection of optimal rescue routes and rapid decision-making.

[0546] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0547] In this invention, the server includes an information collection means for collecting data in real time from multiple sources, a means for preprocessing the collected data, a means for integrating the preprocessed data, a means for analyzing the integrated data using generative artificial intelligence, a means for automatically setting rescue priorities based on the analysis results, a means for sharing the set priorities and information with each organization, and a means for presenting data and navigating rescue operations using a mobile device or visual display device. This enables efficient and appropriate rescue operations based on real-time information collection and integration. Furthermore, rescue teams can check the latest information in real time through the visual display device, making it easier to select the optimal rescue route.

[0548] "Information collection means" refers to a means for collecting data in real time from multiple information sources.

[0549] "Preprocessing means" refers to means for converting collected data into a format that is easy to analyze.

[0550] The "integration means" is a means for centralizing and integrating the pre-processed data.

[0551] "Generative AI" is AI that generates information that humans can naturally understand through machine learning and data analysis.

[0552] "Analysis means" refers to the means for analyzing the integrated data and extracting the necessary knowledge.

[0553] The "priority setting means" is a means for automatically setting rescue priorities based on the analysis results.

[0554] "Information sharing means" are means for providing set priorities and information to each agency.

[0555] A "mobile device" is a portable information terminal that can be used while on the move.

[0556] A "visual display device" is a device for visually displaying information.

[0557] This invention relates to a system for supporting rapid and efficient rescue operations in the event of a disaster. This system collects data from multiple sources in real time, preprocesses it, integrates it, and analyzes it using a generative AI model, and based on the results, sets rescue priorities and shares them with various organizations to ensure effective rescue operations.

[0558] First, the server collects data in real time from social media, satellite data providers, emergency notification systems, etc. It uses the social media API to obtain posts based on specific hashtags or keywords, sends a request to download the latest image data from the satellite data provider, and receives audio data from the emergency notification system.

[0559] The server then preprocesses the collected data. It uses text mining tools to remove noise from social media data and extract metadata. It uses image processing tools to standardize the resolution of satellite image data and perform filtering and segmentation processes. For emergency call data, it converts speech to text and extracts key keywords and phrases.

[0560] The pre-processed data is then consolidated using an integration method, which stores social media posts, satellite images, and emergency call data in a single repository to generate an integrated dataset.

[0561] The server then analyzes the combined data using generative AI models, using computer vision models to quantify the extent of damage and building damage from satellite imagery, and natural language processing algorithms to extract key damage information from text data and map the frequency and location of damage reports.

[0562] Based on the analysis, the server automatically prioritizes rescue efforts, using a risk assessment algorithm to assess the urgency and importance of each data point and calculate a score that takes into account factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls.

[0563] The set priorities and information are provided to each agency through information sharing channels, which allow agencies to access data in real time using REST APIs and view the information through a provided data dashboard.

[0564] Furthermore, mobile devices and visual display devices are used to visually present the latest information, providing rescue teams with the ability to navigate rescue routes and priorities in real time. The terminal receives rescue priority and location information sent from the server and uses a map display library to color-code the information according to the level of urgency. GPS is also used to provide navigation means for indicating the optimal route, and navigation instructions are given via voice and visuals.

[0565] For example, when an earthquake occurs, the server collects damage reports from social media posts and analyzes satellite images to identify the extent of the damage. It converts voice data from emergency notification systems into text, extracts and centralizes important information, and then the generative AI analyzes this information, identifies areas with severe damage, sets rescue priorities, and notifies each rescue team.

[0566] Example prompt sentence:

[0567] "Please analyze the satellite image data to identify the extent of the damage and quantify the extent of the damage to the buildings."

[0568] "Collect reports of damage from social media posts and prioritize them according to urgency."

[0569] "Convert the voice data from the emergency call system into text and extract important keywords."

[0570] This allows each agency to grasp the situation in real time and carry out rescue operations quickly and efficiently.

[0571] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0572] Step 1:

[0573] The server uses information collection tools to collect data in real time from multiple sources. Here, it uses SNS APIs to obtain posts based on specific hashtags or keywords, sends requests to download the latest image data from satellite data providers, and receives audio data from emergency notification systems. The input data are SNS posts, satellite image data, and audio data, and the output data are these raw data.

[0574] Step 2:

[0575] The server preprocesses the collected data. Specifically, it uses text mining tools to remove noise from the social media data and extract metadata (such as location and time). It also uses image processing tools to standardize the resolution of the satellite image data and perform filtering and segmentation processes. Furthermore, for audio data, it converts the audio into text and extracts important keywords and phrases. The input data is raw data, and the output data is preprocessed data.

[0576] Step 3:

[0577] The server integrates the preprocessed data. The integration method stores the social media posts, satellite images, and emergency call data in a single database to generate an integrated dataset. The input data is the preprocessed individual data, and the output data is the unified integrated dataset.

[0578] Step 4:

[0579] The server analyzes the integrated data using a generative AI model. It uses computer vision models to identify the extent of damage from satellite imagery and quantify the extent of building damage. It also uses natural language processing algorithms to extract key damage information from text data and map the frequency and location of damage reports. The input data is the integrated dataset, and the output data is the analysis results.

[0580] Step 5:

[0581] The server automatically sets rescue priorities based on the analysis results. Using a risk assessment algorithm, it calculates a score to assess the urgency and importance of each data point, taking into account various factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls. The input data are the analysis results, and the output data are rescue priorities.

[0582] Step 6:

[0583] The server shares the set priorities and information with each agency. A REST API is provided as a means of information sharing, allowing each agency to access data in real time. Information is confirmed through the provided data dashboard. The input data is the rescue priority, and the output data is the information accessed by each agency.

[0584] Step 7:

[0585] The terminal receives rescue priority and physical location information sent from the server. Using a map display library, it colors areas according to the level of emergency, for example, red for high emergency, yellow for medium emergency, and green for low emergency. The input data is the information sent from the server, and the output data is the displayed map.

[0586] Step 8:

[0587] The device provides a GPS-based navigation method to help users find the optimal rescue route. The navigation application calculates the shortest route in real time and provides audio and visual navigation instructions. The input data is location information, and the output data is navigation instructions.

[0588] Step 9:

[0589] Users check the data displayed on the terminal and act according to the instructions. The rescue team leader uses the tablet to check the current rescue priorities and immediately begin action. Furthermore, after activities on site, users input new information into the terminal and provide feedback to the server. This continuously updates the information, allowing other rescue teams to understand the latest situation. The input data is new damage information, and the output data is the updated data.

[0590] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0591] The system of this invention is designed to enable rapid and efficient rescue operations in the event of a disaster, and aims to recognize and support the user's emotional state by incorporating an emotion engine. This system is composed of a server, a terminal, and a user.

[0592] First, the server collects data in real time from multiple sources: it uses social media APIs to retrieve posts based on specific hashtags or keywords, it sends API requests to download the latest image data from satellite data providers, and it connects to emergency notification systems to receive voice data.

[0593] The server then preprocesses the collected data. For social media data, it uses text mining tools to remove noise, identify languages, and extract metadata. For satellite image data, it uses image processing tools to standardize resolution and perform filtering and segmentation. For emergency call data, it converts speech to text and uses automatic speech recognition (ASR) tools to extract key keywords and phrases.

[0594] The server then integrates the pre-processed data into a centralized platform that uses a database for complex queries to store social media posts, satellite imagery, and emergency call data in a single repository, creating a unified, interrelated dataset.

[0595] The server analyzes the combined data using generative AI models, such as computer vision models to identify the extent of damage from satellite imagery and quantify building damage, and natural language processing (NLP) algorithms to extract key damage information from text data and map the frequency and location of damage reports.

[0596] Based on the analysis results, the server automatically sets rescue priorities. It uses a risk assessment algorithm to evaluate the urgency and importance of each data point. It calculates a score based on various factors, such as the scale of the damage, the population density of the affected area, and the number of emergency calls, and sets priorities.

[0597] In addition, the system incorporates an emotion engine, which recognizes the user's emotional state based on voice and text data. For example, it analyzes emergency call audio to assess the speaker's level of tension and fear. It also extracts emotional expressions from text data from social media and emergency calls to understand the user's stress level and panic state.

[0598] Using this information, the server can make a more detailed assessment of the situation and re-establish priorities. In particular, if there is a user with a high level of urgency and emotional instability, it can set the rescue of that user as the top priority.

[0599] The server shares the set priorities and analysis results with each institution. A REST API is provided so that each institution can access the information they need in real time. Each institution can check the information through a data dashboard.

[0600] The terminal receives rescue priority and physical location information sent from the server. The terminal accesses the server's API via the Internet using a mobile device or PC to obtain the latest data. The received data is also stored in a local cache.

[0601] The device visually displays the received data on a map, using a map display library to color-code areas according to priority: red for high urgency, yellow for medium urgency, and green for low urgency.

[0602] Additionally, the device provides GPS-based navigation to help users find the optimal rescue route. The navigation application calculates the shortest route in real time and provides audio and visual navigation instructions.

[0603] Users can check the data displayed on the device and act according to the instructions. For example, a rescue team leader can use the tablet to check the latest rescue priorities and immediately begin action.

[0604] After completing an operation in the field, users input new information into the device and provide feedback to the server, which continuously updates the information and allows other rescue teams to keep up to date with the latest situation.

[0605] As a concrete example, if a large earthquake occurs in a certain area, the server collects damage reports from social media posts and analyzes satellite images to identify the extent of the damage. It then converts voice data from emergency calls into text, extracts important information, and centralizes it. Generative AI analyzes this information, identifies areas with severe damage, sets rescue priorities, and notifies each rescue team. Rescue teams can act quickly based on this information, potentially saving many lives. Furthermore, an emotion engine can analyze the emotions in emergency call voices, identify the speaker's state of panic, and prioritize rescue areas.

[0606] The processing flow will be explained below.

[0607] Step 1: Gather information

[0608] The server collects social media data, satellite image data, and emergency call data.

[0609] The server uses the Twitter API and other social media APIs to retrieve posts based on specific hashtags or keywords in real time.

[0610] The server periodically sends an API request to retrieve the latest image data from the satellite data provider.

[0611] The server connects to the emergency call system and receives emergency call voice data in real time.

[0612] Step 2: Data Preprocessing

[0613] The server preprocesses the collected data.

[0614] For social media data, the server uses text mining tools to remove noise, identify language, and extract metadata (e.g., posting time and location information).

[0615] The server uses image processing tools to standardize the resolution of the satellite image data and perform filtering and segmentation processing.

[0616] For emergency call data, the server converts the speech to text and uses automatic speech recognition (ASR) tools to extract key keywords and phrases.

[0617] Step 3: Information integration

[0618] The server consolidates the pre-processed data.

[0619] The server uses complex queries to store social media posts, satellite images, and emergency call data in a centralized database, creating a unified dataset that correlates these data.

[0620] Step 4: Generative AI analysis

[0621] The server analyzes the combined data using the generated AI model.

[0622] The server uses computer vision models to identify the extent of damage from satellite imagery and quantify the damage to specific objects (e.g., buildings).

[0623] The server uses natural language processing (NLP) algorithms to extract important damage information from the text data and map it to the frequency and location of damage reports.

[0624] Step 5: Setting rescue priorities

[0625] The server automatically sets rescue priorities based on the analysis results.

[0626] The server uses a risk assessment algorithm to rate the urgency and importance of each data point, calculating a score based on various factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls, and then assigning a priority to each.

[0627] Step 6: Sentiment Engine Analysis

[0628] The server uses an emotion engine to analyze the user's emotional state from the voice data and text data.

[0629] The server analyzes the voice data of the emergency call and evaluates the speaker's level of tension and fear.

[0630] The server extracts emotional expressions from text data from social media and emergency calls to understand the user's stress level and panic state.

[0631] Step 7: Emotionally based reprioritization

[0632] The server re-establishes rescue priorities based on the analysis results of the emotion engine.

[0633] If the server finds an emotionally unstable user, it updates the rescue priority to give that area top priority.

[0634] Step 8: Information sharing

[0635] The server shares the set priorities and analysis results with each institution.

[0636] The server provides a REST API, allowing organizations to access the information they need in real time. Organizations can connect to the data dashboard through the REST API to view the latest information.

[0637] Step 9: Receiving Data

[0638] The terminal receives the rescue priority and location information transmitted from the server.

[0639] Mobile devices and PCs access the server's API via the Internet to periodically retrieve the latest data, which is then stored in a local cache.

[0640] Step 10: Visualization

[0641] The data received by the device is displayed on a map.

[0642] The device uses a mapping library (e.g., Google Maps API or Leaflet) to display areas color-coded based on priority: high urgency areas are displayed in red, medium urgency areas in yellow, and low urgency areas in green.

[0643] Step 11: Navigation Instructions

[0644] The device will instruct the user on the optimal rescue route.

[0645] The device uses GPS data to connect to a map application (e.g., Mapbox API) to provide real-time navigation, providing audio and visual navigation instructions to help users reach their destination efficiently.

[0646] Step 12: Verify the information

[0647] The user checks the information displayed on the device and acts according to the instructions.

[0648] Users can check the latest rescue priority and route information through the application on their device and begin taking action immediately.

[0649] Step 13: Provide feedback

[0650] The user inputs new local information into the terminal and provides feedback to the server.

[0651] Users input new information obtained on-site (e.g., new damage or progress of rescue efforts) into their devices and send it as data to the server. The server updates the database based on this feedback information and shares the latest information with other rescue teams.

[0652] Example 2

[0653] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0654] In order to realize rapid and efficient rescue operations during disasters, it is important to collect and analyze data from multiple sources in real time. However, data collected from many sources comes in different formats and has different qualities, making it difficult to integrate them. Furthermore, it is also challenging to appropriately prioritize rescue operations based on the collected data and to take into account the emotional state of users.

[0655] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for collecting data in real time from multiple information sources as information collection means; means for preprocessing the collected data using technologies such as text mining, image processing, and automatic speech recognition; means for integrating the preprocessed data; means for integrating the preprocessed data using a centralized platform; means for analyzing the integrated data using a generative artificial intelligence model to identify the extent of damage and the state of destruction caused by the disaster; means for automatically setting rescue priorities using a risk assessment algorithm based on the analysis results; means for recognizing the user's emotional state from voice data and text data using an emotion engine and resetting priorities; and means for sharing the set priorities and information with various organizations. This makes it possible to collect data in real time from multiple information sources, integrate and analyze it quickly and efficiently, and realize appropriate rescue operations according to the situation.

[0656] "Information collection means" refers to a means for collecting data in real time from multiple information sources.

[0657] "Text mining" is a technology that removes noise from text data and extracts specific information.

[0658] "Image processing" is a technology that standardizes the resolution of image data and performs filtering and segmentation processing.

[0659] "Automatic speech recognition" is a technology that converts voice data into text data.

[0660] A "centralized platform" is a platform for integrating and managing pre-processed data in various formats.

[0661] A "generative artificial intelligence model" is a machine learning model used to perform analysis based on collected data.

[0662] The "risk assessment algorithm" is an algorithm that calculates the urgency and importance of data and sets rescue priorities.

[0663] "Emotion engine" is a technology that analyzes a user's emotional state from voice data and text data.

[0664] "Rescue priorities" are the order of importance and urgency of rescue operations established based on collected and analyzed data.

[0665] "Data sharing tools" are tools for communicating established priorities and related information to each agency.

[0666] "Navigation means" refers to a means that uses GPS to indicate the optimal route and guide the user.

[0667] The system of this invention is designed to carry out rapid and efficient rescue operations in the event of a disaster. The system consists of three entities: a server, a terminal, and a user, each of which has a specific function.

[0668] First, the server collects data in real time from multiple sources: it uses social media APIs to retrieve posts based on specific hashtags or keywords, it makes API requests to download the latest image data from satellite data providers, and it connects to emergency notification systems to receive voice data.

[0669] The server then preprocesses the collected data. For social media data, text mining tools are used to remove noise, identify languages, and extract metadata. For satellite image data, image processing tools (e.g., OpenCV) are used to standardize resolution and perform filtering and segmentation. For emergency call data, speech-to-text conversion is performed and key keywords and phrases are extracted using automatic speech recognition (ASR) tools (e.g., Google Cloud Speech-to-Text).

[0670] The pre-processed data is then integrated into a centralized platform that utilizes a database (e.g., an SQL database) for complex queries, storing social media posts, satellite imagery, and emergency call data in a single repository to generate a unified dataset.

[0671] The server analyzes the combined data using generative AI models (e.g., TensorFlow, BERT), for example, computer vision models to identify the extent of damage from satellite imagery and quantify the extent of building damage, and natural language processing (NLP) algorithms to extract key damage information from text data and map the frequency and location of damage reports.

[0672] Based on the analysis results, the server uses a risk assessment algorithm to automatically prioritize rescue efforts. For example, it calculates scores based on various factors, such as the scale of the damage, the population density of the affected area, and the number of emergency calls, and then sets priorities. It also incorporates an emotion engine into the system to recognize the user's emotional state based on voice and text data. This allows it to re-set priorities, and if there is a user with a particularly high level of urgency and emotional instability, it can assign top priority to their rescue.

[0673] The server shares the set priorities and analysis results with each institution via a REST API, and each institution can view this information in real time through a data dashboard (e.g., a web application).

[0674] The terminal receives rescue priority and physical location information sent from the server. The terminal accesses the server's API via the Internet using a mobile device or PC to obtain the latest data. The received data is also stored in a local cache. The terminal uses a map display library (e.g., Leaflet.js) to visually display the data on a map. Areas are color-coded according to the level of urgency, with red for medium-level areas, yellow for low-level areas, and green for low-level areas.

[0675] Additionally, the device provides GPS-based navigation to help users find the optimal rescue route. The navigation application (e.g., Google Maps API) calculates the shortest route in real time and provides audio and visual navigation instructions.

[0676] Users check the data displayed on the device and act according to the instructions. For example, a rescue team leader can use the tablet to check the latest rescue priorities and immediately begin action. After completing their activities on the ground, the user enters new information into the device and provides feedback to the server. This updates the information, allowing other rescue teams to understand the latest situation.

[0677] As a concrete example, if a large earthquake occurs in a certain area, the server collects damage reports from social media posts and analyzes satellite images to identify the extent of the damage. It then converts voice data from emergency calls into text, extracts important information, and centralizes it. Generative AI analyzes this information, identifies areas with severe damage, sets rescue priorities, and notifies each rescue team. Rescue teams can act quickly based on this information, potentially saving many lives. Furthermore, an emotion engine can analyze the emotions in emergency call voices, identify the speaker's state of panic, and prioritize rescue areas.

[0678] Example prompt sentence:

[0679] "An earthquake occurs in a certain area. Explain the steps to collect disaster information, identify the extent of the damage, and set rescue priorities. Also, explain how you use an emotion engine to perform sentiment analysis of emergency call voices and re-prioritize them."

[0680] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0681] System program processing flow

[0682] Step 1: Collect data

[0683] 1. Input: Social media posts, satellite images, emergency call audio data

[0684] 2. Specific server operations:

[0685] The server uses the social media API to retrieve posts based on specific hashtags or keywords.

[0686] Example: GET https: / / api.sns.service / tweets?query=%23Earthquake Damage

[0687] The server sends an API request to download the latest image data from the satellite data provider.

[0688] Example: GET https: / / api.satellite.service / images

[0689] A server receives voice data from an emergency notification system.

[0690] Example: GET https: / / api.emergency.service / calls

[0691] 3. Output: Collected raw data

[0692] Step 2: Preprocessing the data

[0693] 1. Input: Collected raw data (SNS data, satellite images, emergency call audio data)

[0694] 2. Specific server operations:

[0695] Text mining tools are used on social media data to remove noise, identify language, and extract metadata.

[0696] Example: text = clean_noise(raw_text)

[0697] Image processing tools are used on satellite image data to unify resolution, perform filtering and segmentation.

[0698] Example: processed_image = cv2.resize(raw_image, new_size)

[0699] Automatic speech recognition (ASR) tools are used on emergency call voice data to convert speech to text and extract key keywords and phrases.

[0700] Example: text_data = asr_tool.recognize(audio_data)

[0701] 3. Output: Preprocessed data (clean text data, processed image data, converted audio data)

[0702] Step 3: Integrate the data

[0703] 1. Input: Preprocessed data (clean text data, processed image data, converted audio data)

[0704] 2. Specific server operations:

[0705] The pre-processed data is integrated into a centralized platform, where it is stored in a database and linked to the same repository.

[0706] Example: db.insert('disaster_data', processed_data)

[0707] 3. Output: Integrated dataset

[0708] Step 4: Analyze the data

[0709] 1. Input: Integrated dataset

[0710] 2. Specific server operations:

[0711] Analyze data using generative artificial intelligence models (e.g., TensorFlow, BERT).

[0712] The extent of the damage is identified from satellite images, and the extent of damage to the building is quantified using a computer vision model.

[0713] Example: damage_score = vision_model.predict(image)

[0714] Important damage information is extracted from text data using natural language processing (NLP) algorithms, and the frequency and location of damage reports are mapped.

[0715] Example: key_info = nlp_model.extract(text)

[0716] 3. Output: Damage extent, damage status assessment, frequency and location of damage information

[0717] Step 5: Setting rescue priorities

[0718] 1. Input: Analysis results (damage extent, damage status assessment, frequency and location of damage information)

[0719] 2. Specific server operations:

[0720] Using risk assessment algorithms, the system assesses the urgency and importance of each data point and automatically prioritizes rescue efforts.

[0721] Example: priority_score = risk_algorithm.calculate(data_points)

[0722] 3. Output: List of rescue priorities

[0723] Step 6: Use the Emotion Engine

[0724] 1. Input: Audio data, text data

[0725] 2. Specific server operations:

[0726] An emotion engine is used to analyze the user's emotional state from the voice and text data.

[0727] Example: emotion_score = emotion_engine.analyze(audio_text_data)

[0728] 3. Output: User's emotion evaluation score

[0729] Step 7: Reprioritize

[0730] 1. Input: List of rescue priorities, user's emotional evaluation score

[0731] 2. Specific server operations:

[0732] Reassess and update rescue priority list based on emotional state.

[0733] Example: updated_priority = update_priority_based_on_emotion(priority_list, emotion_scores)

[0734] 3. Output: Updated rescue priority list

[0735] Step 8: Share and view your data

[0736] 1. Input: Updated rescue priority list

[0737] 2. Specific server operations:

[0738] The set priorities and related information are shared with each institution via a REST API.

[0739] Example: POST https: / / api.rescue.service / priorities

[0740] 3. Specific device behavior:

[0741] The device accesses the server's API to receive the latest rescue priority and location information.

[0742] Example: GET https: / / api.rescue.service / latest

[0743] The received data is visually displayed on a map using a map display library (e.g., Leaflet.js). Areas are color-coded in red, yellow, or green according to the level of urgency.

[0744] Example: L.marker(coordinates).addTo(map)

[0745] It uses GPS to perform navigation functions that provide users with the best rescue route.

[0746] Example: directionsService.route(request, callback)

[0747] 4. Output: Display rescue priority and optimal rescue route

[0748] The user checks the data displayed on the device and takes action according to the instructions. After completing their activities in the field, they input new information into the device and provide feedback to the server. This continuously updates the information, allowing other rescue teams to keep up to date with the latest situation.

[0749] (Application example 2)

[0750] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0751] In the event of a disaster, it is extremely important to carry out rapid and efficient rescue operations. Damage assessment is particularly difficult in closed environments such as factories, and the presence of workers with unstable emotional states can reduce the efficiency of rescue operations. The present invention aims to achieve more effective rescue operations by using an emotion engine to recognize the emotional state of victims and integrating it with damage information to set rescue priorities.

[0752] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes, as information collection means, means for collecting data in real time from multiple information sources, means for preprocessing the collected data, means for integrating the preprocessed data, means for analyzing the integrated data using generative artificial intelligence, an emotion engine that recognizes the user's emotional state, means for automatically setting rescue priorities based on the analysis results and the user's emotional state, and means for sharing the set priorities and information with each organization. This makes it possible to set rescue priorities taking damage information and emotional state into consideration, thereby realizing efficient and prompt rescue operations.

[0753] "Information collection means" refers to a means for collecting data in real time from multiple information sources.

[0754] "Preprocessing" refers to processing the collected data such as noise removal, filtering, and standardization.

[0755] "Integration" means centralizing pre-processed data and putting it into a shareable format.

[0756] "Generative AI" is AI that uses technologies such as machine learning and deep learning to extract useful information and patterns from large amounts of data.

[0757] An "emotion engine" is an engine that recognizes and evaluates a user's emotional state from voice data, text data, and the like.

[0758] "Rescue priority" is a criterion for determining the order in which rescue operations should be carried out based on the analysis results and the user's emotional state.

[0759] "Agency" means any public or private entity involved in disaster relief or damage assessment.

[0760] "Image data" is digital data that includes visual information captured by a camera or satellite.

[0761] "Text data" refers to digital data that includes text information such as social media posts and emergency calls.

[0762] "Audio data" is digital data that includes sound information, such as emergency calls and the voices of disaster victims.

[0763] "User emotional state data" is data that expresses the emotions (for example, stress, panic, fear, etc.) shown by the user as numerical values ​​or evaluations.

[0764] "Navigation means" refers to a means of providing the optimal route using GPS and guiding the person to the rescue location.

[0765] "Visual navigation instruction means" refers to a means of providing visual navigation information through maps, real-time video, etc.

[0766] The present invention provides a system for providing effective rescue operations in the event of a disaster, incorporating information collection means, preprocessing means, data integration means, data analysis means using generative artificial intelligence, and an emotion engine. Specific embodiments of the system are described below.

[0767] 1. Information gathering methods

[0768] The server collects data from multiple sources. The hardware used includes mobile devices (smartphones), smart glasses, and factory robots. The software uses APIs from social media platforms, satellite data providers, and emergency call systems. Through these, social media posts, satellite image data, and emergency call audio data are acquired in real time.

[0769] 2. Data preprocessing methods

[0770] The server performs preprocessing on the collected data. For example, for social media data, a text mining tool is used to remove noise and extract metadata. For satellite image data, an image processing tool (e.g., OpenCV) is used to standardize the resolution and perform filtering. For emergency call data, a voice recognition tool (e.g., Google Speech-to-Text API) is used to convert speech to text.

[0771] 3. Data Integration Methods

[0772] The server integrates the pre-processed data using a database capable of handling complex queries. Data from different sources (social media posts, satellite images, emergency call data) is stored in a single repository and managed as an integrated dataset.

[0773] 4. Data Analysis Methods

[0774] Generative AI models (e.g., TensorFlow, PyTorch) are used to analyze the integrated data, computer vision models are used to identify the extent of damage from satellite imagery and quantify the extent of building damage, and natural language processing (NLP) algorithms are used to extract key information from text data from social media posts and emergency calls, mapping the frequency and location of damage reports.

[0775] 5. Emotion Engine

[0776] The emotion engine recognizes the user's emotional state from voice and text data. For example, it analyzes emergency call audio to assess the speaker's level of tension and fear. It also extracts emotional expressions from text data from social media and emergency calls to understand the user's stress level and panic state.

[0777] 6. Rescue Priority Setting

[0778] The server automatically prioritizes rescue efforts based on the analysis results and the emotion engine's assessment. It uses a risk assessment algorithm to determine the urgency and importance of each data point and calculates a score based on factors such as the scale of damage, population density, and number of emergency calls.

[0779] 7. Information sharing methods

[0780] The server shares the set priorities and analysis results with each institution, allowing them to access the necessary information in real time via a REST API and providing the information through a data dashboard.

[0781] 8. Device Navigation Methods

[0782] The device includes a navigation tool that uses GPS to provide optimal rescue route guidance. The navigation application calculates the shortest route in real time and provides audio and visual navigation instructions.

[0783] Specific examples

[0784] For example, if an earthquake occurs at Factory A, the server collects data from social media posts and emergency calls and uses an emotion engine to evaluate the speaker's emotional state. The server then identifies the extent of the damage and sends rescue instructions to robots, prioritizing areas with the highest level of emergency. Managers can grasp the situation in real time through smart glasses or smartphones and take appropriate command.

[0785] Prompt Sentence Examples

[0786] Please prioritize the damage based on the following social media posts and emergency call data. Also, use sentiment analysis to identify cases with the highest urgency.

[0787] ---

[0788] Social Media Posts:

[0789] Earthquake. Building collapsed. Help needed.

[0790] Disaster: The road is blocked, please come and help immediately.

[0791] Emergency Call Data:

[0792] Please help, my house has collapsed and I can't get out.

[0793] The water is rising rapidly and it is very dangerous.

[0794] ---

[0795] output:

[0796] High-urgency location: A place where the house has collapsed and you cannot get out (sentiment analysis: high stress level)

[0797] Priority rescue instruction: dispatch robots to begin immediate rescue

[0798] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0799] Step 1:

[0800] The server collects data from multiple sources: it uses social media APIs to retrieve posts based on specific hashtags or keywords in real time, downloads the latest image data through the satellite data provider's API, and connects to the emergency call system to receive voice data. The inputs for this step are social media posts, satellite image data, and emergency call voice data, and the output is the collected raw data.

[0801] Step 2:

[0802] The server preprocesses the collected data. For social media data, text mining tools are used to remove noise and extract metadata. For satellite image data, image processing tools (e.g., OpenCV) are used to standardize the resolution and perform filtering and segmentation. For emergency call data, speech recognition tools (e.g., Google Speech-to-Text API) are used to convert speech to text and extract important keywords and phrases. The input of this step is the collected raw data, and the output is the preprocessed data.

[0803] Step 3:

[0804] The server consolidates the preprocessed data. It integrates it into a database system and centralizes data from different sources. This involves storing social media posts, satellite images, and emergency call data in a single repository to generate a consolidated dataset. The input of this step is the preprocessed data, and the output is the consolidated dataset.

[0805] Step 4:

[0806] The server analyzes the integrated dataset with a generative AI model. It uses computer vision models to identify the extent of damage from satellite imagery and quantify the extent of building damage. It uses NLP algorithms to extract key damage information from text data and map the frequency and location of damage reports. The input for this step is the integrated dataset, and the output is a damage assessment and damage location mapping results.

[0807] Step 5:

[0808] The server uses an emotion engine to recognize the user's emotional state from voice and text data. It analyzes emergency call voice to evaluate the speaker's level of tension and fear. It also extracts emotional expressions from text data from social media and emergency calls to understand the user's stress level and panic state. The input for this step is voice and text data, and the output is data on the user's emotional state.

[0809] Step 6:

[0810] The server automatically sets rescue priorities based on the analysis results and emotional state data. It uses a risk assessment algorithm to evaluate the urgency and importance of each data point. It calculates a score based on factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls, and sets rescue priorities. The inputs to this step are the analysis results and emotional state data, and the output is a rescue priority list.

[0811] Step 7:

[0812] The server provides information through a REST API to share the set priorities and analysis results with each agency. Each agency can view the information in real time using a data dashboard. The input of this step is the rescue priority list and analysis results, and the output is the rescue instructions sent to each agency.

[0813] Step 8:

[0814] The device receives rescue priority and physical location information sent from the server. The received data is visually displayed on a map and provides the optimal rescue route using GPS. The navigation application calculates the shortest route in real time and provides audio and visual navigation instructions. The input of this step is rescue instruction data, and the output is instructions for the optimal rescue route.

[0815] Specific examples

[0816] For example, if an earthquake occurs at Factory A, the server collects data from social media posts and emergency calls and uses an emotion engine to evaluate the speaker's emotional state. The server then identifies the extent of the damage and sends rescue instructions to robots, prioritizing areas with the highest level of emergency. Managers can grasp the situation in real time through smart glasses or smartphones and take appropriate command.

[0817] Prompt Sentence Examples

[0818] Please prioritize the damage based on the following social media posts and emergency call data. Also, use sentiment analysis to identify cases with the highest urgency.

[0819] ---

[0820] Social Media Posts:

[0821] Earthquake. Building collapsed. Help needed.

[0822] Disaster: The road is blocked, please come and help immediately.

[0823] Emergency Call Data:

[0824] Please help, my house has collapsed and I can't get out.

[0825] The water is rising rapidly and it is very dangerous.

[0826] ---

[0827] output:

[0828] High-urgency location: A place where the house has collapsed and you cannot get out (sentiment analysis: high stress level)

[0829] Priority rescue instruction: dispatch robots to begin immediate rescue

[0830] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0831] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0832] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0833] [Third embodiment]

[0834] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0835] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0836] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0837] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0838] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0839] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0840] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0841] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0842] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0843] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0844] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0845] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0846] The system of the present invention is designed to carry out rescue operations quickly and efficiently in the event of a disaster. This system is composed of a server, terminals, and users.

[0847] The server collects data in real time from multiple sources, including using social media APIs to retrieve posts based on specific hashtags or keywords, sending requests to download the latest image data from satellite data providers, and connecting to emergency call systems to receive voice data.

[0848] The server then preprocesses the collected data. For SMS data, text mining tools are used to remove noise and extract post metadata (such as location and time). For satellite image data, image processing tools are used to standardize resolution and perform filtering and segmentation. For emergency call data, speech-to-text conversion is performed to extract key keywords and phrases.

[0849] The server then integrates the pre-processed data into a centralized platform that uses a database with complex query capabilities to store social media posts, satellite imagery, and emergency call data in a single repository, resulting in a connected, integrated dataset.

[0850] The server then analyzes the combined data using generative AI models. For example, the server uses computer vision models to identify the extent of damage from satellite imagery and quantify building damage. It also uses natural language processing algorithms to extract key damage information from text data and map the frequency and location of damage reports.

[0851] Based on the analysis results, the server automatically assigns rescue priorities. Using a risk assessment algorithm, the server calculates a score to assess the urgency and importance of each data point, taking into account various factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls. Rescue priorities are then assigned based on this score.

[0852] Finally, the server provides the set priorities and analysis results to each institution in real time. A REST API is provided so that each institution can access the data in real time. Each institution can check the information through the provided data dashboard.

[0853] The device receives rescue priority and physical location information sent from the server. The device accesses the server's API via the internet using a mobile device or PC to retrieve the latest information. The data is also stored in a local cache.

[0854] The device uses a map display library to visually display the received data on a map, color-coding areas according to priority: for example, red for high urgency, yellow for medium urgency, and green for low urgency.

[0855] Additionally, the device provides a GPS-based navigation tool to help users find the optimal rescue route. The navigation application calculates the shortest route in real time and provides audio and visual navigation instructions.

[0856] Users can check the data displayed on the device and act according to the instructions. For example, a rescue team leader can use the tablet to check the current rescue priority and immediately begin action.

[0857] After completing an operation in the field, users input new information into the device and provide feedback to the server, which continuously updates the information and allows other rescue teams to keep up to date with the latest situation.

[0858] As a concrete example, if a large earthquake occurs in a certain area, the server collects damage reports from social media posts and analyzes satellite images to identify the extent of the damage. It then converts voice data from emergency calls into text, extracts important information, and centralizes it. Generative AI analyzes this information, identifies areas with severe damage, sets rescue priorities, and notifies each rescue team. Rescue teams can act quickly based on this information, potentially saving many lives.

[0859] The processing flow will be explained below.

[0860] Step 1: Gather information

[0861] The server collects social media data, satellite image data, and emergency call data.

[0862] The server uses the Twitter API and other social media APIs to retrieve posts based on specific hashtags or keywords in real time.

[0863] The server periodically sends an API request to retrieve the latest image data from the satellite data provider.

[0864] The server connects to the emergency call system and receives emergency call voice data in real time.

[0865] Step 2: Data Preprocessing

[0866] The server preprocesses the collected data.

[0867] For social media data, the server uses text mining tools to remove noise, identify language, and extract metadata (e.g., posting time and location information).

[0868] The server uses image processing tools to standardize the resolution of the satellite image data and perform filtering and segmentation processing.

[0869] For emergency call data, the server converts the speech to text and uses automatic speech recognition (ASR) tools to extract key keywords and phrases.

[0870] Step 3: Information integration

[0871] The server consolidates the pre-processed data.

[0872] The server uses complex queries to store social media posts, satellite images, and emergency call data in a centralized database, creating a unified dataset that correlates these data.

[0873] Step 4: Generative AI analysis

[0874] The server analyzes the combined data using the generated AI model.

[0875] The server uses computer vision models to identify the extent of damage from satellite imagery and quantify the damage to specific objects (e.g., buildings).

[0876] The server uses natural language processing (NLP) algorithms to extract important damage information from the text data and map it to the frequency and location of damage reports.

[0877] Step 5: Setting rescue priorities

[0878] The server automatically sets rescue priorities based on the analysis results.

[0879] The server uses a risk assessment algorithm to rate the urgency and importance of each data point, calculating a score based on various factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls, and then assigning a priority to each.

[0880] Step 6: Information sharing

[0881] The server shares the set priorities and analysis results with each institution.

[0882] The server provides a REST API, allowing organizations to access the information they need in real time. Organizations can connect to the data dashboard through the REST API to view the latest information.

[0883] Step 7: Receiving Data

[0884] The terminal receives the rescue priority and location information transmitted from the server.

[0885] Mobile devices and PCs access the server's API via the Internet to periodically retrieve the latest data, which is then stored in a local cache.

[0886] Step 8: Visualization

[0887] The data received by the device is displayed on a map.

[0888] The device uses a mapping library (e.g., Google Maps API or Leaflet) to display areas color-coded based on priority: high urgency areas are displayed in red, medium urgency areas in yellow, and low urgency areas in green.

[0889] Step 9: Navigation Instructions

[0890] The device will instruct the user on the optimal rescue route.

[0891] The device uses GPS data to connect to a map application (e.g., Mapbox API) to provide real-time navigation, providing audio and visual navigation instructions to help users reach their destination efficiently.

[0892] Step 10: Verify the information

[0893] The user checks the information displayed on the device and acts according to the instructions.

[0894] Users can check the latest rescue priority and route information through the application on their device and begin taking action immediately.

[0895] Step 11: Provide feedback

[0896] The user inputs new local information into the terminal and provides feedback to the server.

[0897] Users input new information obtained on-site (e.g., new damage or progress of rescue efforts) into their devices and send it as data to the server. The server updates the database based on this feedback information and shares the latest information with other rescue teams.

[0898] Example 1

[0899] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0900] Rescue operations during disasters must be carried out quickly and efficiently, but it is difficult to properly collect, analyze, and integrate data from multiple sources. Furthermore, it is not easy to grasp the damage situation in real time and provide the optimal rescue route. With conventional systems, data collection and integration can take time, making it difficult to effectively prioritize rescue operations. This can reduce the efficiency of rescue operations and put many lives at risk. The purpose of this invention is to solve these problems and enable fast and efficient rescue operations during disasters.

[0901] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0902] In this invention, the server includes information collection means, including means for collecting data in real time from multiple sources, means for preprocessing the collected data, means for integrating the preprocessed data, means for analyzing the integrated data using a generated AI model, means for automatically setting rescue priorities based on the analysis results, means for providing the set priorities and information to each organization, means for visually displaying the rescue priorities received by the terminal on a color-coded map, and means for providing optimal rescue routes using GPS. This allows for the rapid and effective collection, preprocessing, and integration of data from multiple sources and analysis using the generated AI model, enabling accurate rescue prioritization and real-time information provision. Furthermore, the efficiency of rescue operations is significantly improved by visualizing the damage situation and providing optimal route guidance through the terminal.

[0903] "Information collection means" refers to a means for collecting data in real time from multiple information sources.

[0904] "Preprocessing means" refers to a means of converting collected data into a form that is easier to analyze by performing processes such as noise removal and metadata extraction on the data.

[0905] The "integration means" is a means for integrating pre-processed data into a centralized platform and storing it in a database that allows for complex queries, etc.

[0906] A "generative AI model" is an artificial intelligence model used to analyze collected and pre-processed data, including, for example, computer vision models and natural language processing algorithms.

[0907] "Data analysis means" refers to a means of analyzing the integrated data using a generative AI model to identify the extent of damage and extract important information.

[0908] The "priority setting means" is a means for automatically determining the priority of rescue operations based on the analysis results and using a risk assessment algorithm.

[0909] "Information provision means" refers to the means for providing the set priorities and related information to each relevant organization in real time. Specifically, this includes REST APIs and data dashboards.

[0910] The "color-coded visual display means" is a means of displaying rescue priority and damage information received by the terminal on a map in different colors. By indicating the urgency level by color for each area, it makes it easier to visually check the information.

[0911] "Navigation Means" means a means that utilizes GPS to provide an optimal rescue route, calculates the shortest route in real time, and includes audio and visual instructions.

[0912] MODE FOR CARRYING OUT THE INVENTION

[0913] The system of the present invention is designed to carry out rescue operations quickly and efficiently in the event of a disaster. This system is composed of a server, terminals, and users.

[0914] First, the server collects data in real time from multiple sources as an information gathering means. Specifically, the server uses the API of social media to retrieve posts based on specific hashtags or keywords. It also sends requests to download the latest image data from satellite data providers and connects to emergency notification systems to receive voice data.

[0915] The server then processes the collected data with pre-processing tools. For social media data, text mining tools are used to remove noise and extract post metadata (such as location and time). For satellite image data, image processing tools are used to standardize resolution and perform filtering and segmentation. For emergency call data, speech recognition tools are used to convert speech to text and extract key keywords and phrases.

[0916] The server integrates the preprocessed data into a centralized platform using an integration method. Specifically, it stores the data in a database using a database management system (e.g., MySQL, PostgreSQL) that allows for complex queries. Social media posts, satellite images, and emergency call data are stored in a single repository, creating a linked, integrated dataset.

[0917] The combined data is then analyzed using generative AI models and data analytics, such as computer vision models to identify the extent of damage from satellite imagery and quantify building damage, and natural language processing algorithms (NLP) to extract key damage information from text data and map the frequency and location of damage reports.

[0918] Based on the analysis results, the server automatically assigns rescue priorities using a prioritization method, which uses a risk assessment algorithm to calculate a score that takes into account various factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls to assess the urgency and importance of each data point.

[0919] Finally, the server provides the priorities and analysis results set by the information provision means to each institution in real time. A REST API is provided so that each institution can access the data in real time. Each institution can check the information through the provided data dashboard.

[0920] The terminal receives rescue priority and physical location information sent from the server. The terminal accesses the server's API via the Internet using a mobile device or PC to obtain the latest information. The received data is also stored in a local cache.

[0921] The device uses a color-coded visual display method to visually display the received data on a map. Using a map display library (e.g., Leaflet, Google Maps API), areas are color-coded according to the level of urgency. For example, areas with high urgency are displayed in red, areas with medium urgency in yellow, and areas with low urgency in green.

[0922] Additionally, the device provides GPS-based navigation to help users find the optimal rescue route. Navigation applications (e.g., Google Maps, Waze) calculate the shortest route in real time and provide audio and visual navigation instructions.

[0923] Users check the data displayed on the device and act according to the instructions. For example, a rescue team leader uses the tablet to check the current rescue priority and immediately begin action.

[0924] After completing their activities in the field, users input feedback data into their devices and send it to the server, which continuously updates the information and allows other rescue teams to keep up to date with the latest situation.

[0925] As a specific example of how it works, let's consider the occurrence of a large-scale earthquake. The server collects damage reports from social media posts using hashtags such as "earthquake" and "rescue," and analyzes satellite images to identify the extent of the damage. It also converts voice data from emergency calls into text, extracting and integrating important information. A generative AI model analyzes this data, identifies areas with severe damage, sets rescue priorities, and notifies each rescue team. This is expected to enable rescue teams to act quickly and efficiently, saving many lives.

[0926] Prompt Sentence Examples

[0927] "Collect posts containing specific hashtags from social media data, filter out noise, and extract location information. For example, posts containing the tags earthquake or rescue are targeted."

[0928] "We need to acquire satellite image data, standardize the resolution, and then perform a filtering process. The goal is to identify damaged buildings."

[0929] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0930] Step 1: Data collection

[0931] The server collects data in real time from multiple sources. As input, it uses data from social media APIs, satellite data providers, and emergency notification systems. Specifically, it calls social media APIs to retrieve posts based on specific hashtags or keywords (e.g., "earthquake" or "rescue"). It also sends requests to satellite data providers to download the latest image data. It also receives audio data from emergency notification systems. The output is the collected raw data.

[0932] Step 2: Data Preprocessing

[0933] The server preprocesses the collected data. It uses the raw data collected in step 1 as input. For social media data, it uses text mining tools to remove noise and extract metadata (such as location and time). For satellite image data, it uses image processing tools to standardize the resolution and perform filtering and segmentation processes. For emergency call data, it uses speech recognition tools to convert speech to text and extract important keywords and phrases. The output is the preprocessed data with noise removed and important information extracted.

[0934] Step 3: Data Integration

[0935] The server integrates the preprocessed data. It uses the preprocessed data from step 2 as input. For data integration, it uses a database management system (e.g., MySQL, PostgreSQL) and stores the data in a centralized platform. It generates a linked integrated dataset by integrating social media posts, satellite images, and emergency call data into a single repository. The output is the integrated dataset.

[0936] Step 4: Data analysis

[0937] The server analyzes the merged data using a generative AI model. It uses the dataset merged in step 3 as input. It uses a computer vision model to identify the extent of damage from satellite imagery and quantify the state of building damage. It also uses natural language processing algorithms to extract key damage information from text data and map the frequency and location of damage reports. The output is the analysis results.

[0938] Step 5: Rescue Priority Setting

[0939] The server automatically sets rescue priorities using a risk assessment algorithm. It uses the analysis results obtained in step 4 as input. It evaluates the extent of damage, population density of the affected area, number of emergency calls, etc., and calculates the urgency and importance of each area to set priorities. The output is a prioritized rescue plan.

[0940] Step 6: Delivering results

[0941] The server provides the set priorities and analysis results to each agency in real time. It uses the rescue plan set in step 5 as input. It provides a REST API so that each agency can access the data in real time. Each agency can check the information through a data dashboard. The output is the data provided to each agency.

[0942] Step 7: Displaying and navigating data on your device

[0943] The terminal receives the rescue priority and physical location information sent from the server. It uses the data provided in step 6 as input. It accesses the server's API using a mobile device or PC to obtain the latest information and stores it in a local cache. It uses a map display library to display the received data on a map and color-codes it according to the urgency level. It also provides a navigation tool using GPS to calculate and guide the optimal rescue route. The output is a visual display and navigation information provided to the user.

[0944] Step 8: User feedback and data updates

[0945] The user checks the data displayed on the device, and after completing their activities in the field, they input new information into the device and send it to the server. The input uses new damage information and rescue progress information collected by the user in the field. Based on the feedback, the server updates the database so that other rescue teams can also keep up to date with the latest situation. The output is an updated database and a revised rescue plan.

[0946] The above is the processing flow and specific operation of the program for this system.

[0947] (Application example 1)

[0948] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0949] Rescue operations during disasters must be carried out quickly and efficiently, but integrating data from multiple sources and formulating effective rescue plans is difficult. Local rescue teams also lack the means to visually view real-time updates, which can hinder the selection of optimal rescue routes and rapid decision-making.

[0950] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0951] In this invention, the server includes an information collection means for collecting data in real time from multiple sources, a means for preprocessing the collected data, a means for integrating the preprocessed data, a means for analyzing the integrated data using generative artificial intelligence, a means for automatically setting rescue priorities based on the analysis results, a means for sharing the set priorities and information with each organization, and a means for presenting data and navigating rescue operations using a mobile device or visual display device. This enables efficient and appropriate rescue operations based on real-time information collection and integration. Furthermore, rescue teams can check the latest information in real time through the visual display device, making it easier to select the optimal rescue route.

[0952] "Information collection means" refers to a means for collecting data in real time from multiple information sources.

[0953] "Preprocessing means" refers to means for converting collected data into a format that is easy to analyze.

[0954] The "integration means" is a means for centralizing and integrating the pre-processed data.

[0955] "Generative AI" is AI that generates information that humans can naturally understand through machine learning and data analysis.

[0956] "Analysis means" refers to the means for analyzing the integrated data and extracting the necessary knowledge.

[0957] The "priority setting means" is a means for automatically setting rescue priorities based on the analysis results.

[0958] "Information sharing means" are means for providing set priorities and information to each agency.

[0959] A "mobile device" is a portable information terminal that can be used while on the move.

[0960] A "visual display device" is a device for visually displaying information.

[0961] This invention relates to a system for supporting rapid and efficient rescue operations in the event of a disaster. This system collects data from multiple sources in real time, preprocesses it, integrates it, and analyzes it using a generative AI model, and based on the results, sets rescue priorities and shares them with various organizations to ensure effective rescue operations.

[0962] First, the server collects data in real time from social media, satellite data providers, emergency notification systems, etc. It uses the social media API to obtain posts based on specific hashtags or keywords, sends a request to download the latest image data from the satellite data provider, and receives audio data from the emergency notification system.

[0963] The server then preprocesses the collected data. It uses text mining tools to remove noise from social media data and extract metadata. It uses image processing tools to standardize the resolution of satellite image data and perform filtering and segmentation processes. For emergency call data, it converts speech to text and extracts key keywords and phrases.

[0964] The pre-processed data is then consolidated using an integration method, which stores social media posts, satellite images, and emergency call data in a single repository to generate an integrated dataset.

[0965] The server then analyzes the combined data using generative AI models, using computer vision models to quantify the extent of damage and building damage from satellite imagery, and natural language processing algorithms to extract key damage information from text data and map the frequency and location of damage reports.

[0966] Based on the analysis, the server automatically prioritizes rescue efforts, using a risk assessment algorithm to assess the urgency and importance of each data point and calculate a score that takes into account factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls.

[0967] The set priorities and information are provided to each agency through information sharing channels, which allow agencies to access data in real time using REST APIs and view the information through a provided data dashboard.

[0968] Furthermore, mobile devices and visual display devices are used to visually present the latest information, providing rescue teams with the ability to navigate rescue routes and priorities in real time. The terminal receives rescue priority and location information sent from the server and uses a map display library to color-code the information according to the level of urgency. GPS is also used to provide navigation means for indicating the optimal route, and navigation instructions are given via voice and visuals.

[0969] For example, when an earthquake occurs, the server collects damage reports from social media posts and analyzes satellite images to identify the extent of the damage. It converts voice data from emergency notification systems into text, extracts and centralizes important information, and then the generative AI analyzes this information, identifies areas with severe damage, sets rescue priorities, and notifies each rescue team.

[0970] Example prompt sentence:

[0971] "Please analyze the satellite image data to identify the extent of the damage and quantify the extent of the damage to the buildings."

[0972] "Collect reports of damage from social media posts and prioritize them according to urgency."

[0973] "Convert the voice data from the emergency call system into text and extract important keywords."

[0974] This allows each agency to grasp the situation in real time and carry out rescue operations quickly and efficiently.

[0975] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0976] Step 1:

[0977] The server uses information collection tools to collect data in real time from multiple sources. Here, it uses SNS APIs to obtain posts based on specific hashtags or keywords, sends requests to download the latest image data from satellite data providers, and receives audio data from emergency notification systems. The input data are SNS posts, satellite image data, and audio data, and the output data are these raw data.

[0978] Step 2:

[0979] The server preprocesses the collected data. Specifically, it uses text mining tools to remove noise from the social media data and extract metadata (such as location and time). It also uses image processing tools to standardize the resolution of the satellite image data and perform filtering and segmentation processes. Furthermore, for audio data, it converts the audio into text and extracts important keywords and phrases. The input data is raw data, and the output data is preprocessed data.

[0980] Step 3:

[0981] The server integrates the preprocessed data. The integration method stores the social media posts, satellite images, and emergency call data in a single database to generate an integrated dataset. The input data is the preprocessed individual data, and the output data is the unified integrated dataset.

[0982] Step 4:

[0983] The server analyzes the integrated data using a generative AI model. It uses computer vision models to identify the extent of damage from satellite imagery and quantify the extent of building damage. It also uses natural language processing algorithms to extract key damage information from text data and map the frequency and location of damage reports. The input data is the integrated dataset, and the output data is the analysis results.

[0984] Step 5:

[0985] The server automatically sets rescue priorities based on the analysis results. Using a risk assessment algorithm, it calculates a score to assess the urgency and importance of each data point, taking into account various factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls. The input data are the analysis results, and the output data are rescue priorities.

[0986] Step 6:

[0987] The server shares the set priorities and information with each agency. A REST API is provided as a means of information sharing, allowing each agency to access data in real time. Information is confirmed through the provided data dashboard. The input data is the rescue priority, and the output data is the information accessed by each agency.

[0988] Step 7:

[0989] The terminal receives rescue priority and physical location information sent from the server. Using a map display library, it colors areas according to the level of emergency, for example, red for high emergency, yellow for medium emergency, and green for low emergency. The input data is the information sent from the server, and the output data is the displayed map.

[0990] Step 8:

[0991] The device provides a GPS-based navigation method to help users find the optimal rescue route. The navigation application calculates the shortest route in real time and provides audio and visual navigation instructions. The input data is location information, and the output data is navigation instructions.

[0992] Step 9:

[0993] Users check the data displayed on the terminal and act according to the instructions. The rescue team leader uses the tablet to check the current rescue priorities and immediately begin action. Furthermore, after activities on site, users input new information into the terminal and provide feedback to the server. This continuously updates the information, allowing other rescue teams to understand the latest situation. The input data is new damage information, and the output data is the updated data.

[0994] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0995] The system of this invention is designed to enable rapid and efficient rescue operations in the event of a disaster, and aims to recognize and support the user's emotional state by incorporating an emotion engine. This system is composed of a server, a terminal, and a user.

[0996] First, the server collects data in real time from multiple sources: it uses social media APIs to retrieve posts based on specific hashtags or keywords, it sends API requests to download the latest image data from satellite data providers, and it connects to emergency notification systems to receive voice data.

[0997] The server then preprocesses the collected data. For social media data, it uses text mining tools to remove noise, identify languages, and extract metadata. For satellite image data, it uses image processing tools to standardize resolution and perform filtering and segmentation. For emergency call data, it converts speech to text and uses automatic speech recognition (ASR) tools to extract key keywords and phrases.

[0998] The server then integrates the pre-processed data into a centralized platform that uses a database for complex queries to store social media posts, satellite imagery, and emergency call data in a single repository, creating a unified, interrelated dataset.

[0999] The server analyzes the combined data using generative AI models, such as computer vision models to identify the extent of damage from satellite imagery and quantify building damage, and natural language processing (NLP) algorithms to extract key damage information from text data and map the frequency and location of damage reports.

[1000] Based on the analysis results, the server automatically sets rescue priorities. It uses a risk assessment algorithm to evaluate the urgency and importance of each data point. It calculates a score based on various factors, such as the scale of the damage, the population density of the affected area, and the number of emergency calls, and sets priorities.

[1001] In addition, the system incorporates an emotion engine, which recognizes the user's emotional state based on voice and text data. For example, it analyzes emergency call audio to assess the speaker's level of tension and fear. It also extracts emotional expressions from text data from social media and emergency calls to understand the user's stress level and panic state.

[1002] Using this information, the server can make a more detailed assessment of the situation and re-establish priorities. In particular, if there is a user with a high level of urgency and emotional instability, it can set the rescue of that user as the top priority.

[1003] The server shares the set priorities and analysis results with each institution. A REST API is provided so that each institution can access the information they need in real time. Each institution can check the information through a data dashboard.

[1004] The terminal receives rescue priority and physical location information sent from the server. The terminal accesses the server's API via the Internet using a mobile device or PC to obtain the latest data. The received data is also stored in a local cache.

[1005] The device visually displays the received data on a map, using a map display library to color-code areas according to priority: red for high urgency, yellow for medium urgency, and green for low urgency.

[1006] Additionally, the device provides GPS-based navigation to help users find the optimal rescue route. The navigation application calculates the shortest route in real time and provides audio and visual navigation instructions.

[1007] Users can check the data displayed on the device and act according to the instructions. For example, a rescue team leader can use the tablet to check the latest rescue priorities and immediately begin action.

[1008] After completing an operation in the field, users input new information into the device and provide feedback to the server, which continuously updates the information and allows other rescue teams to keep up to date with the latest situation.

[1009] As a concrete example, if a large earthquake occurs in a certain area, the server collects damage reports from social media posts and analyzes satellite images to identify the extent of the damage. It then converts voice data from emergency calls into text, extracts important information, and centralizes it. Generative AI analyzes this information, identifies areas with severe damage, sets rescue priorities, and notifies each rescue team. Rescue teams can act quickly based on this information, potentially saving many lives. Furthermore, an emotion engine can analyze the emotions in emergency call voices, identify the speaker's state of panic, and prioritize rescue areas.

[1010] The processing flow will be explained below.

[1011] Step 1: Gather information

[1012] The server collects social media data, satellite image data, and emergency call data.

[1013] The server uses the Twitter API and other social media APIs to retrieve posts based on specific hashtags or keywords in real time.

[1014] The server periodically sends an API request to retrieve the latest image data from the satellite data provider.

[1015] The server connects to the emergency call system and receives emergency call voice data in real time.

[1016] Step 2: Data Preprocessing

[1017] The server preprocesses the collected data.

[1018] For social media data, the server uses text mining tools to remove noise, identify language, and extract metadata (e.g., posting time and location information).

[1019] The server uses image processing tools to standardize the resolution of the satellite image data and perform filtering and segmentation processing.

[1020] For emergency call data, the server converts the speech to text and uses automatic speech recognition (ASR) tools to extract key keywords and phrases.

[1021] Step 3: Information integration

[1022] The server consolidates the pre-processed data.

[1023] The server uses complex queries to store social media posts, satellite images, and emergency call data in a centralized database, creating a unified dataset that correlates these data.

[1024] Step 4: Generative AI analysis

[1025] The server analyzes the combined data using the generated AI model.

[1026] The server uses computer vision models to identify the extent of damage from satellite imagery and quantify the damage to specific objects (e.g., buildings).

[1027] The server uses natural language processing (NLP) algorithms to extract important damage information from the text data and map it to the frequency and location of damage reports.

[1028] Step 5: Setting rescue priorities

[1029] The server automatically sets rescue priorities based on the analysis results.

[1030] The server uses a risk assessment algorithm to rate the urgency and importance of each data point, calculating a score based on various factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls, and then assigning a priority to each.

[1031] Step 6: Sentiment Engine Analysis

[1032] The server uses an emotion engine to analyze the user's emotional state from the voice data and text data.

[1033] The server analyzes the voice data of the emergency call and evaluates the speaker's level of tension and fear.

[1034] The server extracts emotional expressions from text data from social media and emergency calls to understand the user's stress level and panic state.

[1035] Step 7: Emotionally based reprioritization

[1036] The server re-establishes rescue priorities based on the analysis results of the emotion engine.

[1037] If the server finds an emotionally unstable user, it updates the rescue priority to give that area top priority.

[1038] Step 8: Information sharing

[1039] The server shares the set priorities and analysis results with each institution.

[1040] The server provides a REST API, allowing organizations to access the information they need in real time. Organizations can connect to the data dashboard through the REST API to view the latest information.

[1041] Step 9: Receiving Data

[1042] The terminal receives the rescue priority and location information transmitted from the server.

[1043] Mobile devices and PCs access the server's API via the Internet to periodically retrieve the latest data, which is then stored in a local cache.

[1044] Step 10: Visualization

[1045] The data received by the device is displayed on a map.

[1046] The device uses a mapping library (e.g., Google Maps API or Leaflet) to display areas color-coded based on priority: high urgency areas are displayed in red, medium urgency areas in yellow, and low urgency areas in green.

[1047] Step 11: Navigation Instructions

[1048] The device will instruct the user on the optimal rescue route.

[1049] The device uses GPS data to connect to a map application (e.g., Mapbox API) to provide real-time navigation, providing audio and visual navigation instructions to help users reach their destination efficiently.

[1050] Step 12: Verify the information

[1051] The user checks the information displayed on the device and acts according to the instructions.

[1052] Users can check the latest rescue priority and route information through the application on their device and begin taking action immediately.

[1053] Step 13: Provide feedback

[1054] The user inputs new local information into the terminal and provides feedback to the server.

[1055] Users input new information obtained on-site (e.g., new damage or progress of rescue efforts) into their devices and send it as data to the server. The server updates the database based on this feedback information and shares the latest information with other rescue teams.

[1056] Example 2

[1057] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1058] In order to realize rapid and efficient rescue operations during disasters, it is important to collect and analyze data from multiple sources in real time. However, data collected from many sources comes in different formats and has different qualities, making it difficult to integrate them. Furthermore, it is also challenging to appropriately prioritize rescue operations based on the collected data and to take into account the emotional state of users.

[1059] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for collecting data in real time from multiple information sources as information collection means; means for preprocessing the collected data using technologies such as text mining, image processing, and automatic speech recognition; means for integrating the preprocessed data; means for integrating the preprocessed data using a centralized platform; means for analyzing the integrated data using a generative artificial intelligence model to identify the extent of damage and the state of destruction caused by the disaster; means for automatically setting rescue priorities using a risk assessment algorithm based on the analysis results; means for recognizing the user's emotional state from voice data and text data using an emotion engine and resetting priorities; and means for sharing the set priorities and information with various organizations. This makes it possible to collect data in real time from multiple information sources, integrate and analyze it quickly and efficiently, and realize appropriate rescue operations according to the situation.

[1060] "Information collection means" refers to a means for collecting data in real time from multiple information sources.

[1061] "Text mining" is a technology that removes noise from text data and extracts specific information.

[1062] "Image processing" is a technology that standardizes the resolution of image data and performs filtering and segmentation processing.

[1063] "Automatic speech recognition" is a technology that converts voice data into text data.

[1064] A "centralized platform" is a platform for integrating and managing pre-processed data in various formats.

[1065] A "generative artificial intelligence model" is a machine learning model used to perform analysis based on collected data.

[1066] The "risk assessment algorithm" is an algorithm that calculates the urgency and importance of data and sets rescue priorities.

[1067] "Emotion engine" is a technology that analyzes a user's emotional state from voice data and text data.

[1068] "Rescue priorities" are the order of importance and urgency of rescue operations established based on collected and analyzed data.

[1069] "Data sharing tools" are tools for communicating established priorities and related information to each agency.

[1070] "Navigation means" refers to a means that uses GPS to indicate the optimal route and guide the user.

[1071] The system of this invention is designed to carry out rapid and efficient rescue operations in the event of a disaster. The system consists of three entities: a server, a terminal, and a user, each of which has a specific function.

[1072] First, the server collects data in real time from multiple sources: it uses social media APIs to retrieve posts based on specific hashtags or keywords, it makes API requests to download the latest image data from satellite data providers, and it connects to emergency notification systems to receive voice data.

[1073] The server then preprocesses the collected data. For social media data, text mining tools are used to remove noise, identify languages, and extract metadata. For satellite image data, image processing tools (e.g., OpenCV) are used to standardize resolution and perform filtering and segmentation. For emergency call data, speech-to-text conversion is performed and key keywords and phrases are extracted using automatic speech recognition (ASR) tools (e.g., Google Cloud Speech-to-Text).

[1074] The pre-processed data is then integrated into a centralized platform that utilizes a database (e.g., an SQL database) for complex queries, storing social media posts, satellite imagery, and emergency call data in a single repository to generate a unified dataset.

[1075] The server analyzes the combined data using generative AI models (e.g., TensorFlow, BERT), for example, computer vision models to identify the extent of damage from satellite imagery and quantify the extent of building damage, and natural language processing (NLP) algorithms to extract key damage information from text data and map the frequency and location of damage reports.

[1076] Based on the analysis results, the server uses a risk assessment algorithm to automatically prioritize rescue efforts. For example, it calculates scores based on various factors, such as the scale of the damage, the population density of the affected area, and the number of emergency calls, and then sets priorities. It also incorporates an emotion engine into the system to recognize the user's emotional state based on voice and text data. This allows it to re-set priorities, and if there is a user with a particularly high level of urgency and emotional instability, it can assign top priority to their rescue.

[1077] The server shares the set priorities and analysis results with each institution via a REST API, and each institution can view this information in real time through a data dashboard (e.g., a web application).

[1078] The terminal receives rescue priority and physical location information sent from the server. The terminal accesses the server's API via the Internet using a mobile device or PC to obtain the latest data. The received data is also stored in a local cache. The terminal uses a map display library (e.g., Leaflet.js) to visually display the data on a map. Areas are color-coded according to the level of urgency, with red for medium-level areas, yellow for low-level areas, and green for low-level areas.

[1079] Additionally, the device provides GPS-based navigation to help users find the optimal rescue route. The navigation application (e.g., Google Maps API) calculates the shortest route in real time and provides audio and visual navigation instructions.

[1080] Users check the data displayed on the device and act according to the instructions. For example, a rescue team leader can use the tablet to check the latest rescue priorities and immediately begin action. After completing their activities on the ground, the user enters new information into the device and provides feedback to the server. This updates the information, allowing other rescue teams to understand the latest situation.

[1081] As a concrete example, if a large earthquake occurs in a certain area, the server collects damage reports from social media posts and analyzes satellite images to identify the extent of the damage. It then converts voice data from emergency calls into text, extracts important information, and centralizes it. Generative AI analyzes this information, identifies areas with severe damage, sets rescue priorities, and notifies each rescue team. Rescue teams can act quickly based on this information, potentially saving many lives. Furthermore, an emotion engine can analyze the emotions in emergency call voices, identify the speaker's state of panic, and prioritize rescue areas.

[1082] Example prompt sentence:

[1083] "An earthquake occurs in a certain area. Explain the steps to collect disaster information, identify the extent of the damage, and set rescue priorities. Also, explain how you use an emotion engine to perform sentiment analysis of emergency call voices and re-prioritize them."

[1084] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1085] System program processing flow

[1086] Step 1: Collect data

[1087] 1. Input: Social media posts, satellite images, emergency call audio data

[1088] 2. Specific server operations:

[1089] The server uses the social media API to retrieve posts based on specific hashtags or keywords.

[1090] Example: GET https: / / api.sns.service / tweets?query=%23Earthquake Damage

[1091] The server sends an API request to download the latest image data from the satellite data provider.

[1092] Example: GET https: / / api.satellite.service / images

[1093] A server receives voice data from an emergency notification system.

[1094] Example: GET https: / / api.emergency.service / calls

[1095] 3. Output: Collected raw data

[1096] Step 2: Preprocessing the data

[1097] 1. Input: Collected raw data (SNS data, satellite images, emergency call audio data)

[1098] 2. Specific server operations:

[1099] Text mining tools are used on social media data to remove noise, identify language, and extract metadata.

[1100] Example: text = clean_noise(raw_text)

[1101] Image processing tools are used on satellite image data to unify resolution, perform filtering and segmentation.

[1102] Example: processed_image = cv2.resize(raw_image, new_size)

[1103] Automatic speech recognition (ASR) tools are used on emergency call voice data to convert speech to text and extract key keywords and phrases.

[1104] Example: text_data = asr_tool.recognize(audio_data)

[1105] 3. Output: Preprocessed data (clean text data, processed image data, converted audio data)

[1106] Step 3: Integrate the data

[1107] 1. Input: Preprocessed data (clean text data, processed image data, converted audio data)

[1108] 2. Specific server operations:

[1109] The pre-processed data is integrated into a centralized platform, where it is stored in a database and linked to the same repository.

[1110] Example: db.insert('disaster_data', processed_data)

[1111] 3. Output: Integrated dataset

[1112] Step 4: Analyze the data

[1113] 1. Input: Integrated dataset

[1114] 2. Specific server operations:

[1115] Analyze data using generative artificial intelligence models (e.g., TensorFlow, BERT).

[1116] The extent of the damage is identified from satellite images, and the extent of damage to the building is quantified using a computer vision model.

[1117] Example: damage_score = vision_model.predict(image)

[1118] Important damage information is extracted from text data using natural language processing (NLP) algorithms, and the frequency and location of damage reports are mapped.

[1119] Example: key_info = nlp_model.extract(text)

[1120] 3. Output: Damage extent, damage status assessment, frequency and location of damage information

[1121] Step 5: Setting rescue priorities

[1122] 1. Input: Analysis results (damage extent, damage status assessment, frequency and location of damage information)

[1123] 2. Specific server operations:

[1124] Using risk assessment algorithms, the system assesses the urgency and importance of each data point and automatically prioritizes rescue efforts.

[1125] Example: priority_score = risk_algorithm.calculate(data_points)

[1126] 3. Output: List of rescue priorities

[1127] Step 6: Use the Emotion Engine

[1128] 1. Input: Audio data, text data

[1129] 2. Specific server operations:

[1130] An emotion engine is used to analyze the user's emotional state from the voice and text data.

[1131] Example: emotion_score = emotion_engine.analyze(audio_text_data)

[1132] 3. Output: User's emotion evaluation score

[1133] Step 7: Reprioritize

[1134] 1. Input: List of rescue priorities, user's emotional evaluation score

[1135] 2. Specific server operations:

[1136] Reassess and update rescue priority list based on emotional state.

[1137] Example: updated_priority = update_priority_based_on_emotion(priority_list, emotion_scores)

[1138] 3. Output: Updated rescue priority list

[1139] Step 8: Share and view your data

[1140] 1. Input: Updated rescue priority list

[1141] 2. Specific server operations:

[1142] The set priorities and related information are shared with each institution via a REST API.

[1143] Example: POST https: / / api.rescue.service / priorities

[1144] 3. Specific device behavior:

[1145] The device accesses the server's API to receive the latest rescue priority and location information.

[1146] Example: GET https: / / api.rescue.service / latest

[1147] The received data is visually displayed on a map using a map display library (e.g., Leaflet.js). Areas are color-coded in red, yellow, or green according to the level of urgency.

[1148] Example: L.marker(coordinates).addTo(map)

[1149] It uses GPS to perform navigation functions that provide users with the best rescue route.

[1150] Example: directionsService.route(request, callback)

[1151] 4. Output: Display rescue priority and optimal rescue route

[1152] The user checks the data displayed on the device and takes action according to the instructions. After completing their activities in the field, they input new information into the device and provide feedback to the server. This continuously updates the information, allowing other rescue teams to keep up to date with the latest situation.

[1153] (Application example 2)

[1154] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1155] In the event of a disaster, it is extremely important to carry out rapid and efficient rescue operations. Damage assessment is particularly difficult in closed environments such as factories, and the presence of workers with unstable emotional states can reduce the efficiency of rescue operations. The present invention aims to achieve more effective rescue operations by using an emotion engine to recognize the emotional state of victims and integrating it with damage information to set rescue priorities.

[1156] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes, as information collection means, means for collecting data in real time from multiple information sources, means for preprocessing the collected data, means for integrating the preprocessed data, means for analyzing the integrated data using generative artificial intelligence, an emotion engine that recognizes the user's emotional state, means for automatically setting rescue priorities based on the analysis results and the user's emotional state, and means for sharing the set priorities and information with each organization. This makes it possible to set rescue priorities taking damage information and emotional state into consideration, thereby realizing efficient and prompt rescue operations.

[1157] "Information collection means" refers to a means for collecting data in real time from multiple information sources.

[1158] "Preprocessing" refers to processing the collected data such as noise removal, filtering, and standardization.

[1159] "Integration" means centralizing pre-processed data and putting it into a shareable format.

[1160] "Generative AI" is AI that uses technologies such as machine learning and deep learning to extract useful information and patterns from large amounts of data.

[1161] An "emotion engine" is an engine that recognizes and evaluates a user's emotional state from voice data, text data, and the like.

[1162] "Rescue priority" is a criterion for determining the order in which rescue operations should be carried out based on the analysis results and the user's emotional state.

[1163] "Agency" means any public or private entity involved in disaster relief or damage assessment.

[1164] "Image data" is digital data that includes visual information captured by a camera or satellite.

[1165] "Text data" refers to digital data that includes text information such as social media posts and emergency calls.

[1166] "Audio data" is digital data that includes sound information, such as emergency calls and the voices of disaster victims.

[1167] "User emotional state data" is data that expresses the emotions (for example, stress, panic, fear, etc.) shown by the user as numerical values ​​or evaluations.

[1168] "Navigation means" refers to a means of providing the optimal route using GPS and guiding the person to the rescue location.

[1169] "Visual navigation instruction means" refers to a means of providing visual navigation information through maps, real-time video, etc.

[1170] The present invention provides a system for providing effective rescue operations in the event of a disaster, incorporating information collection means, preprocessing means, data integration means, data analysis means using generative artificial intelligence, and an emotion engine. Specific embodiments of the system are described below.

[1171] 1. Information gathering methods

[1172] The server collects data from multiple sources. The hardware used includes mobile devices (smartphones), smart glasses, and factory robots. The software uses APIs from social media platforms, satellite data providers, and emergency call systems. Through these, social media posts, satellite image data, and emergency call audio data are acquired in real time.

[1173] 2. Data preprocessing methods

[1174] The server performs preprocessing on the collected data. For example, for social media data, a text mining tool is used to remove noise and extract metadata. For satellite image data, an image processing tool (e.g., OpenCV) is used to standardize the resolution and perform filtering. For emergency call data, a voice recognition tool (e.g., Google Speech-to-Text API) is used to convert speech to text.

[1175] 3. Data Integration Methods

[1176] The server integrates the pre-processed data using a database capable of handling complex queries. Data from different sources (social media posts, satellite images, emergency call data) is stored in a single repository and managed as an integrated dataset.

[1177] 4. Data Analysis Methods

[1178] Generative AI models (e.g., TensorFlow, PyTorch) are used to analyze the integrated data, computer vision models are used to identify the extent of damage from satellite imagery and quantify the extent of building damage, and natural language processing (NLP) algorithms are used to extract key information from text data from social media posts and emergency calls, mapping the frequency and location of damage reports.

[1179] 5. Emotion Engine

[1180] The emotion engine recognizes the user's emotional state from voice and text data. For example, it analyzes emergency call audio to assess the speaker's level of tension and fear. It also extracts emotional expressions from text data from social media and emergency calls to understand the user's stress level and panic state.

[1181] 6. Rescue Priority Setting

[1182] The server automatically prioritizes rescue efforts based on the analysis results and the emotion engine's assessment. It uses a risk assessment algorithm to determine the urgency and importance of each data point and calculates a score based on factors such as the scale of damage, population density, and number of emergency calls.

[1183] 7. Information sharing methods

[1184] The server shares the set priorities and analysis results with each institution, allowing them to access the necessary information in real time via a REST API and providing the information through a data dashboard.

[1185] 8. Device Navigation Methods

[1186] The device includes a navigation tool that uses GPS to provide optimal rescue route guidance. The navigation application calculates the shortest route in real time and provides audio and visual navigation instructions.

[1187] Specific examples

[1188] For example, if an earthquake occurs at Factory A, the server collects data from social media posts and emergency calls and uses an emotion engine to evaluate the speaker's emotional state. The server then identifies the extent of the damage and sends rescue instructions to robots, prioritizing areas with the highest level of emergency. Managers can grasp the situation in real time through smart glasses or smartphones and take appropriate command.

[1189] Prompt Sentence Examples

[1190] Please prioritize the damage based on the following social media posts and emergency call data. Also, use sentiment analysis to identify cases with the highest urgency.

[1191] ---

[1192] Social Media Posts:

[1193] Earthquake. Building collapsed. Help needed.

[1194] Disaster: The road is blocked, please come and help immediately.

[1195] Emergency Call Data:

[1196] Please help, my house has collapsed and I can't get out.

[1197] The water is rising rapidly and it is very dangerous.

[1198] ---

[1199] output:

[1200] High-urgency location: A place where the house has collapsed and you cannot get out (sentiment analysis: high stress level)

[1201] Priority rescue instruction: dispatch robots to begin immediate rescue

[1202] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1203] Step 1:

[1204] The server collects data from multiple sources: it uses social media APIs to retrieve posts based on specific hashtags or keywords in real time, downloads the latest image data through the satellite data provider's API, and connects to the emergency call system to receive voice data. The inputs for this step are social media posts, satellite image data, and emergency call voice data, and the output is the collected raw data.

[1205] Step 2:

[1206] The server preprocesses the collected data. For social media data, text mining tools are used to remove noise and extract metadata. For satellite image data, image processing tools (e.g., OpenCV) are used to standardize the resolution and perform filtering and segmentation. For emergency call data, speech recognition tools (e.g., Google Speech-to-Text API) are used to convert speech to text and extract important keywords and phrases. The input of this step is the collected raw data, and the output is the preprocessed data.

[1207] Step 3:

[1208] The server consolidates the preprocessed data. It integrates it into a database system and centralizes data from different sources. This involves storing social media posts, satellite images, and emergency call data in a single repository to generate a consolidated dataset. The input of this step is the preprocessed data, and the output is the consolidated dataset.

[1209] Step 4:

[1210] The server analyzes the integrated dataset with a generative AI model. It uses computer vision models to identify the extent of damage from satellite imagery and quantify the extent of building damage. It uses NLP algorithms to extract key damage information from text data and map the frequency and location of damage reports. The input for this step is the integrated dataset, and the output is a damage assessment and damage location mapping results.

[1211] Step 5:

[1212] The server uses an emotion engine to recognize the user's emotional state from voice and text data. It analyzes emergency call voice to evaluate the speaker's level of tension and fear. It also extracts emotional expressions from text data from social media and emergency calls to understand the user's stress level and panic state. The input for this step is voice and text data, and the output is data on the user's emotional state.

[1213] Step 6:

[1214] The server automatically sets rescue priorities based on the analysis results and emotional state data. It uses a risk assessment algorithm to evaluate the urgency and importance of each data point. It calculates a score based on factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls, and sets rescue priorities. The inputs to this step are the analysis results and emotional state data, and the output is a rescue priority list.

[1215] Step 7:

[1216] The server provides information through a REST API to share the set priorities and analysis results with each agency. Each agency can view the information in real time using a data dashboard. The input of this step is the rescue priority list and analysis results, and the output is the rescue instructions sent to each agency.

[1217] Step 8:

[1218] The device receives rescue priority and physical location information sent from the server. The received data is visually displayed on a map and provides the optimal rescue route using GPS. The navigation application calculates the shortest route in real time and provides audio and visual navigation instructions. The input of this step is rescue instruction data, and the output is instructions for the optimal rescue route.

[1219] Specific examples

[1220] For example, if an earthquake occurs at Factory A, the server collects data from social media posts and emergency calls and uses an emotion engine to evaluate the speaker's emotional state. The server then identifies the extent of the damage and sends rescue instructions to robots, prioritizing areas with the highest level of emergency. Managers can grasp the situation in real time through smart glasses or smartphones and take appropriate command.

[1221] Prompt Sentence Examples

[1222] Please prioritize the damage based on the following social media posts and emergency call data. Also, use sentiment analysis to identify cases with the highest urgency.

[1223] ---

[1224] Social Media Posts:

[1225] Earthquake. Building collapsed. Help needed.

[1226] Disaster: The road is blocked, please come and help immediately.

[1227] Emergency Call Data:

[1228] Please help, my house has collapsed and I can't get out.

[1229] The water is rising rapidly and it is very dangerous.

[1230] ---

[1231] output:

[1232] High-urgency location: A place where the house has collapsed and you cannot get out (sentiment analysis: high stress level)

[1233] Priority rescue instruction: dispatch robots to begin immediate rescue

[1234] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1235] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1236] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1237] [Fourth embodiment]

[1238] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1239] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1240] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1241] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1242] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1243] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1244] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1245] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1246] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1247] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1248] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1249] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1250] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1251] The system of the present invention is designed to carry out rescue operations quickly and efficiently in the event of a disaster. This system is composed of a server, terminals, and users.

[1252] The server collects data in real time from multiple sources, including using social media APIs to retrieve posts based on specific hashtags or keywords, sending requests to download the latest image data from satellite data providers, and connecting to emergency call systems to receive voice data.

[1253] The server then preprocesses the collected data. For SMS data, text mining tools are used to remove noise and extract post metadata (such as location and time). For satellite image data, image processing tools are used to standardize resolution and perform filtering and segmentation. For emergency call data, speech-to-text conversion is performed to extract key keywords and phrases.

[1254] The server then integrates the pre-processed data into a centralized platform that uses a database with complex query capabilities to store social media posts, satellite imagery, and emergency call data in a single repository, resulting in a connected, integrated dataset.

[1255] The server then analyzes the combined data using generative AI models. For example, the server uses computer vision models to identify the extent of damage from satellite imagery and quantify building damage. It also uses natural language processing algorithms to extract key damage information from text data and map the frequency and location of damage reports.

[1256] Based on the analysis results, the server automatically assigns rescue priorities. Using a risk assessment algorithm, the server calculates a score to assess the urgency and importance of each data point, taking into account various factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls. Rescue priorities are then assigned based on this score.

[1257] Finally, the server provides the set priorities and analysis results to each institution in real time. A REST API is provided so that each institution can access the data in real time. Each institution can check the information through the provided data dashboard.

[1258] The device receives rescue priority and physical location information sent from the server. The device accesses the server's API via the internet using a mobile device or PC to retrieve the latest information. The data is also stored in a local cache.

[1259] The device uses a map display library to visually display the received data on a map, color-coding areas according to priority: for example, red for high urgency, yellow for medium urgency, and green for low urgency.

[1260] Additionally, the device provides a GPS-based navigation tool to help users find the optimal rescue route. The navigation application calculates the shortest route in real time and provides audio and visual navigation instructions.

[1261] Users can check the data displayed on the device and act according to the instructions. For example, a rescue team leader can use the tablet to check the current rescue priority and immediately begin action.

[1262] After completing an operation in the field, users input new information into the device and provide feedback to the server, which continuously updates the information and allows other rescue teams to keep up to date with the latest situation.

[1263] As a concrete example, if a large earthquake occurs in a certain area, the server collects damage reports from social media posts and analyzes satellite images to identify the extent of the damage. It then converts voice data from emergency calls into text, extracts important information, and centralizes it. Generative AI analyzes this information, identifies areas with severe damage, sets rescue priorities, and notifies each rescue team. Rescue teams can act quickly based on this information, potentially saving many lives.

[1264] The processing flow will be explained below.

[1265] Step 1: Gather information

[1266] The server collects social media data, satellite image data, and emergency call data.

[1267] The server uses the Twitter API and other social media APIs to retrieve posts based on specific hashtags or keywords in real time.

[1268] The server periodically sends an API request to retrieve the latest image data from the satellite data provider.

[1269] The server connects to the emergency call system and receives emergency call voice data in real time.

[1270] Step 2: Data Preprocessing

[1271] The server preprocesses the collected data.

[1272] For social media data, the server uses text mining tools to remove noise, identify language, and extract metadata (e.g., posting time and location information).

[1273] The server uses image processing tools to standardize the resolution of the satellite image data and perform filtering and segmentation processing.

[1274] For emergency call data, the server converts the speech to text and uses automatic speech recognition (ASR) tools to extract key keywords and phrases.

[1275] Step 3: Information integration

[1276] The server consolidates the pre-processed data.

[1277] The server uses complex queries to store social media posts, satellite images, and emergency call data in a centralized database, creating a unified dataset that correlates these data.

[1278] Step 4: Generative AI analysis

[1279] The server analyzes the combined data using the generated AI model.

[1280] The server uses computer vision models to identify the extent of damage from satellite imagery and quantify the damage to specific objects (e.g., buildings).

[1281] The server uses natural language processing (NLP) algorithms to extract important damage information from the text data and map it to the frequency and location of damage reports.

[1282] Step 5: Setting rescue priorities

[1283] The server automatically sets rescue priorities based on the analysis results.

[1284] The server uses a risk assessment algorithm to rate the urgency and importance of each data point, calculating a score based on various factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls, and then assigning a priority to each.

[1285] Step 6: Information sharing

[1286] The server shares the set priorities and analysis results with each institution.

[1287] The server provides a REST API, allowing organizations to access the information they need in real time. Organizations can connect to the data dashboard through the REST API to view the latest information.

[1288] Step 7: Receiving Data

[1289] The terminal receives the rescue priority and location information transmitted from the server.

[1290] Mobile devices and PCs access the server's API via the Internet to periodically retrieve the latest data, which is then stored in a local cache.

[1291] Step 8: Visualization

[1292] The data received by the device is displayed on a map.

[1293] The device uses a mapping library (e.g., Google Maps API or Leaflet) to display areas color-coded based on priority: high urgency areas are displayed in red, medium urgency areas in yellow, and low urgency areas in green.

[1294] Step 9: Navigation Instructions

[1295] The device will instruct the user on the optimal rescue route.

[1296] The device uses GPS data to connect to a map application (e.g., Mapbox API) to provide real-time navigation, providing audio and visual navigation instructions to help users reach their destination efficiently.

[1297] Step 10: Verify the information

[1298] The user checks the information displayed on the device and acts according to the instructions.

[1299] Users can check the latest rescue priority and route information through the application on their device and begin taking action immediately.

[1300] Step 11: Provide feedback

[1301] The user inputs new local information into the terminal and provides feedback to the server.

[1302] Users input new information obtained on-site (e.g., new damage or progress of rescue efforts) into their devices and send it as data to the server. The server updates the database based on this feedback information and shares the latest information with other rescue teams.

[1303] Example 1

[1304] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1305] Rescue operations during disasters must be carried out quickly and efficiently, but it is difficult to properly collect, analyze, and integrate data from multiple sources. Furthermore, it is not easy to grasp the damage situation in real time and provide the optimal rescue route. With conventional systems, data collection and integration can take time, making it difficult to effectively prioritize rescue operations. This can reduce the efficiency of rescue operations and put many lives at risk. The purpose of this invention is to solve these problems and enable fast and efficient rescue operations during disasters.

[1306] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1307] In this invention, the server includes information collection means, including means for collecting data in real time from multiple sources, means for preprocessing the collected data, means for integrating the preprocessed data, means for analyzing the integrated data using a generated AI model, means for automatically setting rescue priorities based on the analysis results, means for providing the set priorities and information to each organization, means for visually displaying the rescue priorities received by the terminal on a color-coded map, and means for providing optimal rescue routes using GPS. This allows for the rapid and effective collection, preprocessing, and integration of data from multiple sources and analysis using the generated AI model, enabling accurate rescue prioritization and real-time information provision. Furthermore, the efficiency of rescue operations is significantly improved by visualizing the damage situation and providing optimal route guidance through the terminal.

[1308] "Information collection means" refers to a means for collecting data in real time from multiple information sources.

[1309] "Preprocessing means" refers to a means of converting collected data into a form that is easier to analyze by performing processes such as noise removal and metadata extraction on the data.

[1310] The "integration means" is a means for integrating pre-processed data into a centralized platform and storing it in a database that allows for complex queries, etc.

[1311] A "generative AI model" is an artificial intelligence model used to analyze collected and pre-processed data, including, for example, computer vision models and natural language processing algorithms.

[1312] "Data analysis means" refers to a means of analyzing the integrated data using a generative AI model to identify the extent of damage and extract important information.

[1313] The "priority setting means" is a means for automatically determining the priority of rescue operations based on the analysis results and using a risk assessment algorithm.

[1314] "Information provision means" refers to the means for providing the set priorities and related information to each relevant organization in real time. Specifically, this includes REST APIs and data dashboards.

[1315] The "color-coded visual display means" is a means of displaying rescue priority and damage information received by the terminal on a map in different colors. By indicating the urgency level by color for each area, it makes it easier to visually check the information.

[1316] "Navigation Means" means a means that utilizes GPS to provide an optimal rescue route, calculates the shortest route in real time, and includes audio and visual instructions.

[1317] MODE FOR CARRYING OUT THE INVENTION

[1318] The system of the present invention is designed to carry out rescue operations quickly and efficiently in the event of a disaster. This system is composed of a server, terminals, and users.

[1319] First, the server collects data in real time from multiple sources as an information gathering means. Specifically, the server uses the API of social media to retrieve posts based on specific hashtags or keywords. It also sends requests to download the latest image data from satellite data providers and connects to emergency notification systems to receive voice data.

[1320] The server then processes the collected data with pre-processing tools. For social media data, text mining tools are used to remove noise and extract post metadata (such as location and time). For satellite image data, image processing tools are used to standardize resolution and perform filtering and segmentation. For emergency call data, speech recognition tools are used to convert speech to text and extract key keywords and phrases.

[1321] The server integrates the preprocessed data into a centralized platform using an integration method. Specifically, it stores the data in a database using a database management system (e.g., MySQL, PostgreSQL) that allows for complex queries. Social media posts, satellite images, and emergency call data are stored in a single repository, creating a linked, integrated dataset.

[1322] The combined data is then analyzed using generative AI models and data analytics, such as computer vision models to identify the extent of damage from satellite imagery and quantify building damage, and natural language processing algorithms (NLP) to extract key damage information from text data and map the frequency and location of damage reports.

[1323] Based on the analysis results, the server automatically assigns rescue priorities using a prioritization method, which uses a risk assessment algorithm to calculate a score that takes into account various factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls to assess the urgency and importance of each data point.

[1324] Finally, the server provides the priorities and analysis results set by the information provision means to each institution in real time. A REST API is provided so that each institution can access the data in real time. Each institution can check the information through the provided data dashboard.

[1325] The terminal receives rescue priority and physical location information sent from the server. The terminal accesses the server's API via the Internet using a mobile device or PC to obtain the latest information. The received data is also stored in a local cache.

[1326] The device uses a color-coded visual display method to visually display the received data on a map. Using a map display library (e.g., Leaflet, Google Maps API), areas are color-coded according to the level of urgency. For example, areas with high urgency are displayed in red, areas with medium urgency in yellow, and areas with low urgency in green.

[1327] Additionally, the device provides GPS-based navigation to help users find the optimal rescue route. Navigation applications (e.g., Google Maps, Waze) calculate the shortest route in real time and provide audio and visual navigation instructions.

[1328] Users check the data displayed on the device and act according to the instructions. For example, a rescue team leader uses the tablet to check the current rescue priority and immediately begin action.

[1329] After completing their activities in the field, users input feedback data into their devices and send it to the server, which continuously updates the information and allows other rescue teams to keep up to date with the latest situation.

[1330] As a specific example of how it works, let's consider the occurrence of a large-scale earthquake. The server collects damage reports from social media posts using hashtags such as "earthquake" and "rescue," and analyzes satellite images to identify the extent of the damage. It also converts voice data from emergency calls into text, extracting and integrating important information. A generative AI model analyzes this data, identifies areas with severe damage, sets rescue priorities, and notifies each rescue team. This is expected to enable rescue teams to act quickly and efficiently, saving many lives.

[1331] Prompt Sentence Examples

[1332] "Collect posts containing specific hashtags from social media data, filter out noise, and extract location information. For example, posts containing the tags earthquake or rescue are targeted."

[1333] "We need to acquire satellite image data, standardize the resolution, and then perform a filtering process. The goal is to identify damaged buildings."

[1334] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1335] Step 1: Data collection

[1336] The server collects data in real time from multiple sources. As input, it uses data from social media APIs, satellite data providers, and emergency notification systems. Specifically, it calls social media APIs to retrieve posts based on specific hashtags or keywords (e.g., "earthquake" or "rescue"). It also sends requests to satellite data providers to download the latest image data. It also receives audio data from emergency notification systems. The output is the collected raw data.

[1337] Step 2: Data Preprocessing

[1338] The server preprocesses the collected data. It uses the raw data collected in step 1 as input. For social media data, it uses text mining tools to remove noise and extract metadata (such as location and time). For satellite image data, it uses image processing tools to standardize the resolution and perform filtering and segmentation processes. For emergency call data, it uses speech recognition tools to convert speech to text and extract important keywords and phrases. The output is the preprocessed data with noise removed and important information extracted.

[1339] Step 3: Data Integration

[1340] The server integrates the preprocessed data. It uses the preprocessed data from step 2 as input. For data integration, it uses a database management system (e.g., MySQL, PostgreSQL) and stores the data in a centralized platform. It generates a linked integrated dataset by integrating social media posts, satellite images, and emergency call data into a single repository. The output is the integrated dataset.

[1341] Step 4: Data analysis

[1342] The server analyzes the merged data using a generative AI model. It uses the dataset merged in step 3 as input. It uses a computer vision model to identify the extent of damage from satellite imagery and quantify the state of building damage. It also uses natural language processing algorithms to extract key damage information from text data and map the frequency and location of damage reports. The output is the analysis results.

[1343] Step 5: Rescue Priority Setting

[1344] The server automatically sets rescue priorities using a risk assessment algorithm. It uses the analysis results obtained in step 4 as input. It evaluates the extent of damage, population density of the affected area, number of emergency calls, etc., and calculates the urgency and importance of each area to set priorities. The output is a prioritized rescue plan.

[1345] Step 6: Delivering results

[1346] The server provides the set priorities and analysis results to each agency in real time. It uses the rescue plan set in step 5 as input. It provides a REST API so that each agency can access the data in real time. Each agency can check the information through a data dashboard. The output is the data provided to each agency.

[1347] Step 7: Displaying and navigating data on your device

[1348] The terminal receives the rescue priority and physical location information sent from the server. It uses the data provided in step 6 as input. It accesses the server's API using a mobile device or PC to obtain the latest information and stores it in a local cache. It uses a map display library to display the received data on a map and color-codes it according to the urgency level. It also provides a navigation tool using GPS to calculate and guide the optimal rescue route. The output is a visual display and navigation information provided to the user.

[1349] Step 8: User feedback and data updates

[1350] The user checks the data displayed on the device, and after completing their activities in the field, they input new information into the device and send it to the server. The input uses new damage information and rescue progress information collected by the user in the field. Based on the feedback, the server updates the database so that other rescue teams can also keep up to date with the latest situation. The output is an updated database and a revised rescue plan.

[1351] The above is the processing flow and specific operation of the program for this system.

[1352] (Application example 1)

[1353] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1354] Rescue operations during disasters must be carried out quickly and efficiently, but integrating data from multiple sources and formulating effective rescue plans is difficult. Local rescue teams also lack the means to visually view real-time updates, which can hinder the selection of optimal rescue routes and rapid decision-making.

[1355] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1356] In this invention, the server includes an information collection means for collecting data in real time from multiple sources, a means for preprocessing the collected data, a means for integrating the preprocessed data, a means for analyzing the integrated data using generative artificial intelligence, a means for automatically setting rescue priorities based on the analysis results, a means for sharing the set priorities and information with each organization, and a means for presenting data and navigating rescue operations using a mobile device or visual display device. This enables efficient and appropriate rescue operations based on real-time information collection and integration. Furthermore, rescue teams can check the latest information in real time through the visual display device, making it easier to select the optimal rescue route.

[1357] "Information collection means" refers to a means for collecting data in real time from multiple information sources.

[1358] "Preprocessing means" refers to means for converting collected data into a format that is easy to analyze.

[1359] The "integration means" is a means for centralizing and integrating the pre-processed data.

[1360] "Generative AI" is AI that generates information that humans can naturally understand through machine learning and data analysis.

[1361] "Analysis means" refers to the means for analyzing the integrated data and extracting the necessary knowledge.

[1362] The "priority setting means" is a means for automatically setting rescue priorities based on the analysis results.

[1363] "Information sharing means" are means for providing set priorities and information to each agency.

[1364] A "mobile device" is a portable information terminal that can be used while on the move.

[1365] A "visual display device" is a device for visually displaying information.

[1366] This invention relates to a system for supporting rapid and efficient rescue operations in the event of a disaster. This system collects data from multiple sources in real time, preprocesses it, integrates it, and analyzes it using a generative AI model, and based on the results, sets rescue priorities and shares them with various organizations to ensure effective rescue operations.

[1367] First, the server collects data in real time from social media, satellite data providers, emergency notification systems, etc. It uses the social media API to obtain posts based on specific hashtags or keywords, sends a request to download the latest image data from the satellite data provider, and receives audio data from the emergency notification system.

[1368] The server then preprocesses the collected data. It uses text mining tools to remove noise from social media data and extract metadata. It uses image processing tools to standardize the resolution of satellite image data and perform filtering and segmentation processes. For emergency call data, it converts speech to text and extracts key keywords and phrases.

[1369] The pre-processed data is then consolidated using an integration method, which stores social media posts, satellite images, and emergency call data in a single repository to generate an integrated dataset.

[1370] The server then analyzes the combined data using generative AI models, using computer vision models to quantify the extent of damage and building damage from satellite imagery, and natural language processing algorithms to extract key damage information from text data and map the frequency and location of damage reports.

[1371] Based on the analysis, the server automatically prioritizes rescue efforts, using a risk assessment algorithm to assess the urgency and importance of each data point and calculate a score that takes into account factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls.

[1372] The set priorities and information are provided to each agency through information sharing channels, which allow agencies to access data in real time using REST APIs and view the information through a provided data dashboard.

[1373] Furthermore, mobile devices and visual display devices are used to visually present the latest information, providing rescue teams with the ability to navigate rescue routes and priorities in real time. The terminal receives rescue priority and location information sent from the server and uses a map display library to color-code the information according to the level of urgency. GPS is also used to provide navigation means for indicating the optimal route, and navigation instructions are given via voice and visuals.

[1374] For example, when an earthquake occurs, the server collects damage reports from social media posts and analyzes satellite images to identify the extent of the damage. It converts voice data from emergency notification systems into text, extracts and centralizes important information, and then the generative AI analyzes this information, identifies areas with severe damage, sets rescue priorities, and notifies each rescue team.

[1375] Example prompt sentence:

[1376] "Please analyze the satellite image data to identify the extent of the damage and quantify the extent of the damage to the buildings."

[1377] "Collect reports of damage from social media posts and prioritize them according to urgency."

[1378] "Convert the voice data from the emergency call system into text and extract important keywords."

[1379] This allows each agency to grasp the situation in real time and carry out rescue operations quickly and efficiently.

[1380] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1381] Step 1:

[1382] The server uses information collection tools to collect data in real time from multiple sources. Here, it uses SNS APIs to obtain posts based on specific hashtags or keywords, sends requests to download the latest image data from satellite data providers, and receives audio data from emergency notification systems. The input data are SNS posts, satellite image data, and audio data, and the output data are these raw data.

[1383] Step 2:

[1384] The server preprocesses the collected data. Specifically, it uses text mining tools to remove noise from the social media data and extract metadata (such as location and time). It also uses image processing tools to standardize the resolution of the satellite image data and perform filtering and segmentation processes. Furthermore, for audio data, it converts the audio into text and extracts important keywords and phrases. The input data is raw data, and the output data is preprocessed data.

[1385] Step 3:

[1386] The server integrates the preprocessed data. The integration method stores the social media posts, satellite images, and emergency call data in a single database to generate an integrated dataset. The input data is the preprocessed individual data, and the output data is the unified integrated dataset.

[1387] Step 4:

[1388] The server analyzes the integrated data using a generative AI model. It uses computer vision models to identify the extent of damage from satellite imagery and quantify the extent of building damage. It also uses natural language processing algorithms to extract key damage information from text data and map the frequency and location of damage reports. The input data is the integrated dataset, and the output data is the analysis results.

[1389] Step 5:

[1390] The server automatically sets rescue priorities based on the analysis results. Using a risk assessment algorithm, it calculates a score to assess the urgency and importance of each data point, taking into account various factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls. The input data are the analysis results, and the output data are rescue priorities.

[1391] Step 6:

[1392] The server shares the set priorities and information with each agency. A REST API is provided as a means of information sharing, allowing each agency to access data in real time. Information is confirmed through the provided data dashboard. The input data is the rescue priority, and the output data is the information accessed by each agency.

[1393] Step 7:

[1394] The terminal receives rescue priority and physical location information sent from the server. Using a map display library, it colors areas according to the level of emergency, for example, red for high emergency, yellow for medium emergency, and green for low emergency. The input data is the information sent from the server, and the output data is the displayed map.

[1395] Step 8:

[1396] The device provides a GPS-based navigation method to help users find the optimal rescue route. The navigation application calculates the shortest route in real time and provides audio and visual navigation instructions. The input data is location information, and the output data is navigation instructions.

[1397] Step 9:

[1398] Users check the data displayed on the terminal and act according to the instructions. The rescue team leader uses the tablet to check the current rescue priorities and immediately begin action. Furthermore, after activities on site, users input new information into the terminal and provide feedback to the server. This continuously updates the information, allowing other rescue teams to understand the latest situation. The input data is new damage information, and the output data is the updated data.

[1399] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1400] The system of this invention is designed to enable rapid and efficient rescue operations in the event of a disaster, and aims to recognize and support the user's emotional state by incorporating an emotion engine. This system is composed of a server, a terminal, and a user.

[1401] First, the server collects data in real time from multiple sources: it uses social media APIs to retrieve posts based on specific hashtags or keywords, it sends API requests to download the latest image data from satellite data providers, and it connects to emergency notification systems to receive voice data.

[1402] The server then preprocesses the collected data. For social media data, it uses text mining tools to remove noise, identify languages, and extract metadata. For satellite image data, it uses image processing tools to standardize resolution and perform filtering and segmentation. For emergency call data, it converts speech to text and uses automatic speech recognition (ASR) tools to extract key keywords and phrases.

[1403] The server then integrates the pre-processed data into a centralized platform that uses a database for complex queries to store social media posts, satellite imagery, and emergency call data in a single repository, creating a unified, interrelated dataset.

[1404] The server analyzes the combined data using generative AI models, such as computer vision models to identify the extent of damage from satellite imagery and quantify building damage, and natural language processing (NLP) algorithms to extract key damage information from text data and map the frequency and location of damage reports.

[1405] Based on the analysis results, the server automatically sets rescue priorities. It uses a risk assessment algorithm to evaluate the urgency and importance of each data point. It calculates a score based on various factors, such as the scale of the damage, the population density of the affected area, and the number of emergency calls, and sets priorities.

[1406] In addition, the system incorporates an emotion engine, which recognizes the user's emotional state based on voice and text data. For example, it analyzes emergency call audio to assess the speaker's level of tension and fear. It also extracts emotional expressions from text data from social media and emergency calls to understand the user's stress level and panic state.

[1407] Using this information, the server can make a more detailed assessment of the situation and re-establish priorities. In particular, if there is a user with a high level of urgency and emotional instability, it can set the rescue of that user as the top priority.

[1408] The server shares the set priorities and analysis results with each institution. A REST API is provided so that each institution can access the information they need in real time. Each institution can check the information through a data dashboard.

[1409] The terminal receives rescue priority and physical location information sent from the server. The terminal accesses the server's API via the Internet using a mobile device or PC to obtain the latest data. The received data is also stored in a local cache.

[1410] The device visually displays the received data on a map, using a map display library to color-code areas according to priority: red for high urgency, yellow for medium urgency, and green for low urgency.

[1411] Additionally, the device provides GPS-based navigation to help users find the optimal rescue route. The navigation application calculates the shortest route in real time and provides audio and visual navigation instructions.

[1412] Users can check the data displayed on the device and act according to the instructions. For example, a rescue team leader can use the tablet to check the latest rescue priorities and immediately begin action.

[1413] After completing an operation in the field, users input new information into the device and provide feedback to the server, which continuously updates the information and allows other rescue teams to keep up to date with the latest situation.

[1414] As a concrete example, if a large earthquake occurs in a certain area, the server collects damage reports from social media posts and analyzes satellite images to identify the extent of the damage. It then converts voice data from emergency calls into text, extracts important information, and centralizes it. Generative AI analyzes this information, identifies areas with severe damage, sets rescue priorities, and notifies each rescue team. Rescue teams can act quickly based on this information, potentially saving many lives. Furthermore, an emotion engine can analyze the emotions in emergency call voices, identify the speaker's state of panic, and prioritize rescue areas.

[1415] The processing flow will be explained below.

[1416] Step 1: Gather information

[1417] The server collects social media data, satellite image data, and emergency call data.

[1418] The server uses the Twitter API and other social media APIs to retrieve posts based on specific hashtags or keywords in real time.

[1419] The server periodically sends an API request to retrieve the latest image data from the satellite data provider.

[1420] The server connects to the emergency call system and receives emergency call voice data in real time.

[1421] Step 2: Data Preprocessing

[1422] The server preprocesses the collected data.

[1423] For social media data, the server uses text mining tools to remove noise, identify language, and extract metadata (e.g., posting time and location information).

[1424] The server uses image processing tools to standardize the resolution of the satellite image data and perform filtering and segmentation processing.

[1425] For emergency call data, the server converts the speech to text and uses automatic speech recognition (ASR) tools to extract key keywords and phrases.

[1426] Step 3: Information integration

[1427] The server consolidates the pre-processed data.

[1428] The server uses complex queries to store social media posts, satellite images, and emergency call data in a centralized database, creating a unified dataset that correlates these data.

[1429] Step 4: Generative AI analysis

[1430] The server analyzes the combined data using the generated AI model.

[1431] The server uses computer vision models to identify the extent of damage from satellite imagery and quantify the damage to specific objects (e.g., buildings).

[1432] The server uses natural language processing (NLP) algorithms to extract important damage information from the text data and map it to the frequency and location of damage reports.

[1433] Step 5: Setting rescue priorities

[1434] The server automatically sets rescue priorities based on the analysis results.

[1435] The server uses a risk assessment algorithm to rate the urgency and importance of each data point, calculating a score based on various factors such as the scale of the damage, the population density of the affected area, and the number of emergency calls, and then assigning a priority to each.

[1436] Step 6: Sentiment Engine Analysis

[1437] The server uses an emotion engine to analyze the user's emotional state from the voice data and text data.

[1438] The server analyzes the voice data of the emergency call and evaluates the speaker's level of tension and fear.

[1439] The server extracts emotional expressions from text data from social media and emergency calls to understand the user's stress level and panic state.

[1440] Step 7: Emotionally based reprioritization

[1441] The server re-establishes rescue priorities based on the analysis results of the emotion engine.

[1442] If the server finds an emotionally unstable user, it updates the rescue priority to give that area top priority.

[1443] Step 8: Information sharing

[1444] The server shares the set priorities and analysis results with each institution.

[1445] The server provides a REST API, allowing organizations to access the information they need in real time. Organizations can connect to the data dashboard through the REST API to view the latest information.

[1446] Step 9: Receiving Data

[1447] The terminal receives the rescue priority and location information transmitted from the server.

[1448] Mobile devices and PCs access the server's API via the Internet to periodically retrieve the latest data, which is then stored in a local cache.

[1449] Step 10: Visualization

[1450] The data received by the device is displayed on a map.

[1451] The device uses a mapping library (e.g., Google Maps API or Leaflet) to display areas color-coded based on priority: high urgency areas are displayed in red, medium urgency areas in yellow, and low urgency areas in green.

[1452] Step 11: Navigation Instructions

[1453] The device will instruct the user on the optimal rescue route.

[1454] The device uses GPS data to connect to a map application (e.g., Mapbox API) to provide real-time navigation, providing audio and visual navigation instructions to help users reach their destination efficiently.

[1455] Step 12: Verify the information

[1456] The user checks the information displayed on the device and acts according to the instructions.

[1457] Users can check the latest rescue priority and route information through the application on their device and begin taking action immediately.

[1458] Step 13: Provide feedback

[1459] The user inputs new local information into the terminal and provides feedback to the server.

[1460] Users input new information obtained on-site (e.g., new damage or progress of rescue efforts) into their devices and send it as data to the server. The server updates the database based on this feedback information and shares the latest information with other rescue teams.

[1461] Example 2

[1462] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1463] In order to realize rapid and efficient rescue operations during disasters, it is important to collect and analyze data from multiple sources in real time. However, data collected from many sources comes in different formats and has different qualities, making it difficult to integrate them. Furthermore, it is also challenging to appropriately prioritize rescue operations based on the collected data and to take into account the emotional state of users.

[1464] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for collecting data in real time from multiple information sources as information collection means; means for preprocessing the collected data using technologies such as text mining, image processing, and automatic speech recognition; means for integrating the preprocessed data; means for integrating the preprocessed data using a centralized platform; means for analyzing the integrated data using a generative artificial intelligence model to identify the extent of damage and the state of destruction caused by the disaster; means for automatically setting rescue priorities using a risk assessment algorithm based on the analysis results; means for recognizing the user's emotional state from voice data and text data using an emotion engine and resetting priorities; and means for sharing the set priorities and information with various organizations. This makes it possible to collect data in real time from multiple information sources, integrate and analyze it quickly and efficiently, and realize appropriate rescue operations according to the situation.

[1465] "Information collection means" refers to a means for collecting data in real time from multiple information sources.

[1466] "Text mining" is a technology that removes noise from text data and extracts specific information.

[1467] "Image processing" is a technology that standardizes the resolution of image data and performs filtering and segmentation processing.

[1468] "Automatic speech recognition" is a technology that converts voice data into text data.

[1469] A "centralized platform" is a platform for integrating and managing pre-processed data in various formats.

[1470] A "generative artificial intelligence model" is a machine learning model used to perform analysis based on collected data.

[1471] The "risk assessment algorithm" is an algorithm that calculates the urgency and importance of data and sets rescue priorities.

[1472] "Emotion engine" is a technology that analyzes a user's emotional state from voice data and text data.

[1473] "Rescue priorities" are the order of importance and urgency of rescue operations established based on collected and analyzed data.

[1474] "Data sharing tools" are tools for communicating established priorities and related information to each agency.

[1475] "Navigation means" refers to a means that uses GPS to indicate the optimal route and guide the user.

[1476] The system of this invention is designed to carry out rapid and efficient rescue operations in the event of a disaster. The system consists of three entities: a server, a terminal, and a user, each of which has a specific function.

[1477] First, the server collects data in real time from multiple sources: it uses social media APIs to retrieve posts based on specific hashtags or keywords, it makes API requests to download the latest image data from satellite data providers, and it connects to emergency notification systems to receive voice data.

[1478] The server then preprocesses the collected data. For social media data, text mining tools are used to remove noise, identify languages, and extract metadata. For satellite image data, image processing tools (e.g., OpenCV) are used to standardize resolution and perform filtering and segmentation. For emergency call data, speech-to-text conversion is performed and key keywords and phrases are extracted using automatic speech recognition (ASR) tools (e.g., Google Cloud Speech-to-Text).

[1479] The pre-processed data is then integrated into a centralized platform that utilizes a database (e.g., an SQL database) for complex queries, storing social media posts, satellite imagery, and emergency call data in a single repository to generate a unified dataset.

[1480] The server analyzes the combined data using generative AI models (e.g., TensorFlow, BERT), for example, computer vision models to identify the extent of damage from satellite imagery and quantify the extent of building damage, and natural language processing (NLP) algorithms to extract key damage information from text data and map the frequency and location of damage reports.

[1481] Based on the analysis results, the server uses a risk assessment algorithm to automatically prioritize rescue efforts. For example, it calculates scores based on various factors, such as the scale of the damage, the population density of the affected area, and the number of emergency calls, and then sets priorities. It also incorporates an emotion engine into the system to recognize the user's emotional state based on voice and text data. This allows it to re-set priorities, and if there is a user with a particularly high level of urgency and emotional instability, it can assign top priority to their rescue.

[1482] The server shares the set priorities and analysis results with each institution via a REST API, and each institution can view this information in real time through a data dashboard (e.g., a web application).

[1483] The terminal receives rescue priority and physical location information sent from the server. The terminal accesses the server's API via the Internet using a mobile device or PC to obtain the latest data. The received data is also stored in a local cache. The terminal uses a map display library (e.g., Leaflet.js) to visually display the data on a map. Areas are color-coded according to the level of urgency, with red for medium-level areas, yellow for low-level areas, and green for low-level areas.

[1484] Additionally, the device provides GPS-based navigation to help users find the optimal rescue route. The navigation application (e.g., Google Maps API) calculates the shortest route in real time and provides audio and visual navigation instructions.

[1485] Users check the data displayed on the device and act according to the instructions. For example, a rescue team leader can use the tablet to check the latest rescue priorities and immediately begin action. After completing their activities on the ground, the user enters new information into the device and provides feedback to the server. This updates the information, allowing other rescue teams to understand the latest situation.

[1486] As a concrete example, if a large earthquake occurs in a certain area, the server collects damage reports from social media posts and analyzes satellite images to identify the extent of the damage. It then converts voice data from emergency calls into text, extracts important information, and centralizes it. Generative AI analyzes this information, identifies areas with severe damage, sets rescue priorities, and notifies each rescue team. Rescue teams can act quickly based on this information, potentially saving many lives. Furthermore, an emotion engine can analyze the emotions in emergency call voices, identify the speaker's state of panic, and prioritize rescue areas.

[1487] Example prompt sentence:

[1488] "An earthquake occurs in a certain area. Explain the steps to collect disaster information, identify the extent of the damage, and set rescue priorities. Also, explain how you use an emotion engine to perform sentiment analysis of emergency call voices and re-prioritize them."

[1489] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1490] System program processing flow

[1491] Step 1: Collect data

[1492] 1. Input: Social media posts, satellite images, emergency call audio data

[1493] 2. Specific server operations:

[1494] The server uses the social media API to retrieve posts based on specific hashtags or keywords.

[1495] Example: GET https: / / api.sns.service / tweets?query=%23Earthquake Damage

[1496] The server sends an API request to download the latest image data from the satellite data provider.

[1497] Example: GET https: / / api.satellite.service / images

[1498] A server receives voice data from an emergency notification system.

[1499] Example: GET https: / / api.emergency.service / calls

[1500] 3. Output: Collected raw data

[1501] Step 2: Preprocessing the data

[1502] 1. Input: Collected raw data (SNS data, satellite images, emergency call audio data)

[1503] 2. Specific server operations:

[1504] Text mining tools are used on social media data to remove noise, identify language, and extract metadata.

[1505] Example: text = clean_noise(raw_text)

[1506] Image processing tools are used on satellite image data to unify resolution, perform filtering and segmentation.

[1507] Example: processed_image = cv2.resize(raw_image, new_size)

[1508] Automatic speech recognition (ASR) tools are used on emergency call voice data to convert speech to text and extract key keywords and phrases.

[1509] Example: text_data = asr_tool.recognize(audio_data)

[1510] 3. Output: Preprocessed data (clean text data, processed image data, converted audio data)

[1511] Step 3: Integrate the data

[1512] 1. Input: Preprocessed data (clean text data, processed image data, converted audio data)

[1513] 2. Specific server operations:

[1514] The pre-processed data is integrated into a centralized platform, where it is stored in a database and linked to the same repository.

[1515] Example: db.insert('disaster_data', processed_data)

[1516] 3. Output: Integrated dataset

[1517] Step 4: Analyze the data

[1518] 1. Input: Integrated dataset

[1519] 2. Specific server operations:

[1520] Analyze data using generative artificial intelligence models (e.g., TensorFlow, BERT).

[1521] The extent of the damage is identified from satellite images, and the extent of damage to the building is quantified using a computer vision model.

[1522] Example: damage_score = vision_model.predict(image)

[1523] Important damage information is extracted from text data using natural language processing (NLP) algorithms, and the frequency and location of damage reports are mapped.

[1524] Example: key_info = nlp_model.extract(text)

[1525] 3. Output: Damage extent, damage status assessment, frequency and location of damage information

[1526] Step 5: Setting rescue priorities

[1527] 1. Input: Analysis results (damage extent, damage status assessment, frequency and location of damage information)

[1528] 2. Specific server operations:

[1529] Using risk assessment algorithms, the system assesses the urgency and importance of each data point and automatically prioritizes rescue efforts.

[1530] Example: priority_score = risk_algorithm.calculate(data_points)

[1531] 3. Output: List of rescue priorities

[1532] Step 6: Use the Emotion Engine

[1533] 1. Input: Audio data, text data

[1534] 2. Specific server operations:

[1535] An emotion engine is used to analyze the user's emotional state from the voice and text data.

[1536] Example: emotion_score = emotion_engine.analyze(audio_text_data)

[1537] 3. Output: User's emotion evaluation score

[1538] Step 7: Reprioritize

[1539] 1. Input: List of rescue priorities, user's emotional evaluation score

[1540] 2. Specific server operations:

[1541] Reassess and update rescue priority list based on emotional state.

[1542] Example: updated_priority = update_priority_based_on_emotion(priority_list, emotion_scores)

[1543] 3. Output: Updated rescue priority list

[1544] Step 8: Share and view your data

[1545] 1. Input: Updated rescue priority list

[1546] 2. Specific server operations:

[1547] The set priorities and related information are shared with each institution via a REST API.

[1548] Example: POST https: / / api.rescue.service / priorities

[1549] 3. Specific device behavior:

[1550] The device accesses the server's API to receive the latest rescue priority and location information.

[1551] Example: GET https: / / api.rescue.service / latest

[1552] The received data is visually displayed on a map using a map display library (e.g., Leaflet.js). Areas are color-coded in red, yellow, or green according to the level of urgency.

[1553] Example: L.marker(coordinates).addTo(map)

[1554] It uses GPS to perform navigation functions that provide users with the best rescue route.

[1555] Example: directionsService.route(request, callback)

[1556] 4. Output: Display rescue priority and optimal rescue route

[1557] The user checks the data displayed on the device and takes action according to the instructions. After completing their activities in the field, they input new information into the device and provide feedback to the server. This continuously updates the information, allowing other rescue teams to keep up to date with the latest situation.

[1558] (Application example 2)

[1559] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1560] In the event of a disaster, it is extremely important to carry out rapid and efficient rescue operations. Damage assessment is particularly difficult in closed environments such as factories, and the presence of workers with unstable emotional states can reduce the efficiency of rescue operations. The present invention aims to achieve more effective rescue operations by using an emotion engine to recognize the emotional state of victims and integrating it with damage information to set rescue priorities.

[1561] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes, as information collection means, means for collecting data in real time from multiple information sources, means for preprocessing the collected data, means for integrating the preprocessed data, means for analyzing the integrated data using generative artificial intelligence, an emotion engine that recognizes the user's emotional state, means for automatically setting rescue priorities based on the analysis results and the user's emotional state, and means for sharing the set priorities and information with each organiza...

Claims

1. As a means of collecting information, there are means to collect data in real time from multiple sources, and means for pre-processing the collected data; a means for integrating the preprocessed data; means for analyzing the synthesized data by a generative artificial intelligence; means for automatically prioritizing rescue efforts based on the analysis results; and A means of sharing established priorities and information with each agency; A system including:

2. The system of claim 1 , including image data, text data, and audio data.

3. 10. The system of claim 1, further comprising a navigation means for indicating an optimal route using a GPS.

Images