System

Abstract

A system is provided.SOLUTION: A system comprising: means for real-time data collection from a plurality of information sources; means for pre-processing the collected information; means for integrating and analyzing the pre-processed information using a generative AI model; means for creating a disaster-affected map based on the analysis results; and means for visualizing and providing the disaster-affected map to a user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When a natural disaster occurs, it is extremely important to quickly grasp the situation in the affected area and carry out efficient rescue operations. However, with conventional methods, collecting and analyzing information requires a great deal of time and effort, making it difficult to make quick decisions. This can delay the rescue of lives and minimizing damage, potentially leading to the expansion of extensive damage. To solve this problem, a new method is needed that integrates and analyzes real-time data and instantly visualizes the damage situation. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for collecting data in real time from multiple sources, a means for preprocessing the collected data, a means for integrating the preprocessed data and analyzing it using a generative AI model, a means for creating a damage map based on the analysis results, and a means for visualizing and providing the damage map to users. This enables instantaneous understanding of the situation in the disaster-stricken area and supports efficient rescue operations. Furthermore, by collecting and preprocessing satellite images and drone footage as visual data, more accurate damage maps can be generated. Furthermore, by extracting disaster-related information from social media data streams as text data and analyzing it in real time, it is possible to effectively grasp the ever-changing disaster situation.

[0006] "Source" is a general term for devices, services, platforms, etc. used to provide various data.

[0007] "Real-time" is the concept of providing information without any time lag by instantly processing and displaying ongoing events.

[0008] "Data collection" is the process of gathering information for a specific purpose, obtaining data from a variety of sources.

[0009] "Preprocessing" is the process of applying initial processing to collected raw data to make it easier to analyze.

[0010] "Integration" is the process of bringing together multiple disparate data into one unified format.

[0011] A "generative AI model" is an AI system that uses machine learning algorithms to analyze data and generate new information and predictions.

[0012] A "disaster map" is a tool that visually shows the damage caused by natural disasters and other disasters on a map.

[0013] "Visualization" is the process of representing data or information in a visual form, such as a graph or map.

[0014] A "user" is an individual or organization that uses the system to obtain information.

[0015] "Satellite imagery" refers to image data of the Earth's surface taken from an artificial satellite.

[0016] "Drone footage" is video data captured by an unmanned aerial vehicle (drone).

[0017] "SNS data stream" refers to the flow of posted data that flows in real time from social networking services (SNS).

[0018] "Text analysis" is a technique for analyzing text data and extracting meaningful information from it. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] This invention is a system that collects data in real time from multiple sources, analyzes the damage situation using a generative AI model, and automatically creates a damage map, which will be an important tool for rapid disaster relief efforts.

[0041] The server collects data from various sources, integrates it, and preprocesses it. Specifically, it acquires real-time weather information, geographic information, satellite images, drone footage, social media data streams, demographic statistics, automobile and smartphone movement information, and information on the operation of lifeline facilities. It then preprocesses the collected data and prepares it for analysis using a generative AI model.

[0042] For example, the server periodically downloads satellite images from a satellite imagery service and uses an image processing module to enhance resolution and remove noise.The server also collects posts from social media data streams in real time and uses natural language processing algorithms to extract important information related to the disaster.

[0043] The server integrates the preprocessed data and inputs it into a generative AI model. The generative AI model analyzes the input data and generates detailed information such as the flooding status of the affected area, the damage status of buildings, and evacuation routes. For example, areas at high risk of flooding are displayed in red, and evacuation routes are displayed in green.

[0044] Once the damage map is generated, the server uploads the information to a web interface for visual display. Users and devices can access the web portal and check the damage situation in real time. In addition, the server quickly notifies government and local government officials and disaster relief teams of the damage map and related information to support appropriate rescue operations.

[0045] As a specific example, the server obtains the latest precipitation data from an API provided by the Japan Meteorological Agency and downloads satellite images at 8:00, 10:00, and 12:00 from a satellite imagery service. These images are then processed by an image processing module to remove noise and improve resolution before being input into the generative AI model. Posts tagged with "disaster" are extracted from the social media data stream, and these are also analyzed by the generative AI model. The generative AI model then creates a disaster area map based on information such as flood risk, damaged buildings, and evacuation routes.

[0046] Users can view this disaster map through a web portal and quickly grasp the situation in the affected areas, which will enable efficient rescue operations and evacuation instructions, saving lives and minimizing damage.

[0047] The processing flow will be explained below.

[0048] Step 1:

[0049] The server connects to multiple information sources, such as meteorological information services, geographic information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems, making it ready to collect the necessary data in real time.

[0050] Step 2:

[0051] The server retrieves the latest weather data from a weather information service, the latest satellite images from a satellite image service, real-time video streams from the drone operation team, and traffic data from a traffic information system, and stores them in temporary storage.

[0052] Step 3:

[0053] The server sends the latest satellite image data to the image processing module, which enhances the resolution of the satellite images and removes noise. The processed images are stored for further analysis.

[0054] Step 4:

[0055] The server collects posts from social media data streams in real time and uses natural language processing algorithms to extract information related to the disaster. For example, it filters posts tagged with "disaster" and analyzes them using a text analysis module to extract location information and details of the disaster situation.

[0056] Step 5:

[0057] The server captures the real-time stream of drone footage frame by frame, tags the parts it deems important, and pre-processes them, for example analyzing the condition of a building using object recognition algorithms to identify damage to the building.

[0058] Step 6:

[0059] The server consolidates all collected data and converts it into one unified format, where all data is linked, including weather data, visual data, traffic data, social media data, etc.

[0060] Step 7:

[0061] The server inputs the integrated data into a generative AI model, which analyzes the flood risk, building damage risk, traffic congestion status, and other factors in the affected area to calculate the risk level for each area.

[0062] Step 8:

[0063] The server then creates a disaster map based on the analysis results obtained from the generative AI model. For example, areas at high risk of flooding are displayed in red, while areas that are not flooded are displayed in green. Information on evacuation routes and first-aid stations is also added to each area.

[0064] Step 9:

[0065] The server uploads the created disaster map to a web interface for access by users and devices. Users can view the disaster map in real time through the web portal and check the information needed to take appropriate action.

[0066] Step 10:

[0067] The server will quickly send disaster maps and related information to government and local government officials and disaster relief teams, enabling rescue efforts to be carried out quickly and effectively, saving lives and minimizing damage.

[0068] Example 1

[0069] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0070] Conventional disaster response systems have difficulty collecting data from multiple sources, analyzing it quickly, and creating detailed maps of affected areas. They also lack the ability to update data in real time or preprocess data in a variety of formats. This hinders the speed and efficiency of disaster response, creating challenges in saving lives and minimizing damage.

[0071] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0072] In this invention, the server includes a means for collecting data from multiple sources in real time, a means for preprocessing the collected data, and a means for integrating the preprocessed data and analyzing it using a generative artificial intelligence model. This enables noise removal and extraction of important information from the collected visual and text data. Flood risk and evacuation routes are then identified based on the preprocessed data, and a disaster situation map is created based on the results and uploaded to a web interface. This allows users to grasp the latest damage situation in real time, enabling rapid and efficient disaster response. Furthermore, information can be quickly provided to government and local government officials and disaster relief teams, enabling appropriate responses.

[0073] "Multiple sources" refers to multiple data providers providing different types of data, such as meteorological data providers, satellite imagery providers, aircraft footage providers, digital social media platforms, and utility information providers.

[0074] "Real-time data collection means" refers to technologies or methods for obtaining data immediately from each source, including obtaining data via an API or by monitoring data streams in real time.

[0075] "Preprocessing means" refers to a series of processing techniques used to convert collected data into a format suitable for generative AI models, including processes such as noise removal, resolution enhancement, and key information extraction.

[0076] A "generative artificial intelligence model" is a model that uses a set of algorithms and data analysis techniques to analyze collected data and generate useful information. This includes deep learning models and natural language processing models.

[0077] "Means for analysis" refers to the techniques and methods for analyzing preprocessed data using a generative artificial intelligence model to identify the damage situation and related information.

[0078] "Disaster Situation Map" refers to a map that visually shows the situation in a disaster-stricken area based on the analysis results of a generative AI model, including information on flood risk, building damage, evacuation routes, etc.

[0079] "Visualization means" refers to technologies for displaying analysis results in a format that is easy for users to understand, including technologies for displaying maps on a graphical user interface or web browser.

[0080] "Noise reduction" refers to the process of removing unnecessary information and errors from collected data, improving the quality of the data and increasing the accuracy of analysis results.

[0081] "Important information extraction" refers to the process of extracting only the useful information needed for analysis from collected data. This includes extracting keywords from text data and extracting features from image data.

[0082] "Flood risk" refers to areas that are more likely to experience natural disasters such as floods. A generative AI model identifies these and displays them on the map.

[0083] "Evacuation routes" refer to routes for safe evacuation in the event of a disaster. These are also analyzed by the generative AI model and displayed on the map.

[0084] "Means for uploading to a web interface" refers to the technology used to publish disaster maps and related information on an internet platform and make them accessible to users.

[0085] This invention is a system that collects data in real time from multiple sources, analyzes the damage situation using a generative artificial intelligence model, and automatically creates a disaster situation map, which will be an important tool for carrying out disaster relief activities quickly and efficiently.

[0086] The server collects data in real time from various sources, including weather information, geographic information, satellite imagery, aircraft footage, digital social media data streams, demographic statistics, automobile and smartphone movement information, and utility equipment operation information. This data is collected from different data providers via APIs and data streams.

[0087] The server then preprocesses the collected data. For image data, it uses image processing modules (e.g., OpenCV) to improve resolution and remove noise. For text data, it uses natural language processing algorithms (e.g., BERT from the Transformers library) to extract important information. This improves the quality of the data and increases the accuracy of the subsequent analysis process.

[0088] The preprocessed data is integrated by a server and fed into a generative artificial intelligence model. This model uses deep learning techniques to analyze the collected visual and text data to identify flood risk, building damage, evacuation routes, and other information. For example, using satellite imagery and meteorological data as input, the generative AI model identifies areas at high risk of flooding. It also uses text data from aerial footage and digital social media data streams to identify building damage.

[0089] The generated analysis results are compiled into a disaster situation map by the server, which is visually displayed using a visualization tool (e.g., D3.js) and provided to users. The server then uploads the disaster situation map to a web interface (e.g., a React app) so that users can access it in real time.

[0090] Furthermore, the server will quickly send disaster situation maps and related information to government and local government officials and disaster relief teams, allowing relevant organizations to grasp the latest damage situation and take appropriate action.

[0091] For example, the server obtains the latest precipitation data from a weather data provider and periodically retrieves satellite images from a satellite image provider. These images are denoised and enhanced in resolution using OpenCV before being input into a generative AI model. It also collects posts tagged with "disaster" from digital social media data streams and extracts key information using the BERT model. Based on the collected data, the generative AI model analyzes information such as flood risk, damaged buildings, and evacuation routes to create a disaster situation map.

[0092] An example of a prompt is:

[0093] Identify areas at high risk of flooding based on the latest satellite imagery and weather data, and provide evacuation routes based on information extracted from the text of digital social media posts.

[0094] In this way, the system uses data collected in real time to enable fast and efficient disaster response.

[0095] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0096] Step 1: Data collection

[0097] The server collects data in real time from various sources. Specifically, it uses APIs to obtain the latest precipitation, temperature, wind speed, etc. from weather information services, and periodically downloads satellite images from satellite image services. It also collects real-time video feeds from aircraft, posts tagged with "disaster" from digital social media platforms, and the operational status of lifeline facilities. The inputs are API calls and data streams from various sources. The output is raw data.

[0098] Step 2: Preprocessing the data

[0099] The server preprocesses the collected data. For image data, OpenCV is used to increase the resolution and remove noise. For example, the resolution of satellite images is doubled and noise is removed. For text data, a natural language processing algorithm (e.g., BERT from the Transformers library) is used to extract important information related to the disaster. For example, keywords such as "flood" and "evacuation" are extracted from posts tagged with "disaster." The input is the collected raw data, and the output is the preprocessed data.

[0100] Step 3: Integrate the data

[0101] The server integrates the preprocessed data. It integrates data from different sources based on time and geographic information to create a consistent dataset. The input is the preprocessed data, and the output is the integrated dataset. For example, it performs processing such as matching the posting time of social media data with the shooting time of satellite images.

[0102] Step 4: Analysis by generative AI model

[0103] The server inputs the integrated data into a generative AI model to analyze the damage situation. Based on the input data, areas at high risk of flooding, evacuation routes, and the extent of building damage are identified. For example, areas at high risk of flooding are shown in red, and evacuation routes in green. The input is the integrated data set, and the output is the analysis results. An analytical algorithm using deep learning is applied to the generative AI model.

[0104] Step 5: Generate a damage map

[0105] The server generates a disaster map based on the analysis results of the generative AI model. The analysis results are visualized using a visualization tool (e.g., D3.js) and displayed in a format that is easy for users to understand. The analysis results are input, and a visualized disaster map is obtained as output. For example, a map can be created that color-codes flood risk and evacuation routes.

[0106] Step 6: Upload to the web interface

[0107] The server uploads the generated disaster map to a web interface (e.g., a React app) so that users can access it in real time. As input, we have a visualized disaster map, and as output, we have a web page that users can view. For example, the server publishes the latest disaster map on a website.

[0108] Step 7: Sending notifications

[0109] The server quickly notifies government and local government officials and disaster relief teams of the analysis results and damage maps. The inputs are the visualized damage maps and analysis results, and the output is notifications sent via email, SMS, or a dedicated app. For example, the server could send the latest damage map to relevant organizations via email.

[0110] (Application example 1)

[0111] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0112] Conventional disaster area information collection systems have difficulty in collecting and analyzing information in real time, making it difficult to carry out prompt rescue operations in disaster-stricken areas and provide safe evacuation routes. Furthermore, conventional navigation systems are unable to provide routes that take into account the conditions in disaster-stricken areas, making it difficult to guarantee the safe movement of autonomous vehicles in disaster-stricken areas.

[0113] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0114] In this invention, the server includes means for collecting data in real time from multiple information sources, means for preprocessing the collected data, means for integrating the preprocessed data and analyzing it using a generative AI model, means for creating a damage map based on the analysis results, means for visualizing the damage map and providing it to users, and means for linking the information on the damage map to a navigation system installed in an autonomous vehicle. This makes it possible to grasp the real-time situation in the disaster area and provide safe and efficient routes.

[0115] "Multiple sources" refers to multiple types of information sources, including weather information, geographic information, satellite images, aerial footage, social networking service data streams, demographic statistics, automobile and smartphone movement information, and information on the operation of lifeline facilities.

[0116] "Collection means" refers to a means for acquiring data from multiple information sources in real time and collecting it on a server.

[0117] "Preprocessing means" refers to means for performing processes such as noise removal, resolution enhancement, and text filtering in order to convert collected data into a format or state suitable for analysis.

[0118] The "integration means" is a means for collecting preprocessed data from multiple information sources and treating them as a single analysis target.

[0119] A "generative AI model" is an artificial intelligence model that analyzes input data and generates disaster information such as flooding status, building damage status, and evacuation routes.

[0120] The "disaster map generation means" is a means of creating a disaster map that visually displays flooding conditions, building damage conditions, evacuation routes, etc. based on the analysis results obtained by the generative AI model.

[0121] The "visualization means" refers to a means including a web interface or application for displaying the generated disaster map in a format that can be viewed by a user.

[0122] "User provision means" refers to means including a web portal and notification system that allow users and relevant organizations to access the visualized disaster map.

[0123] An "autonomous vehicle" is a vehicle that moves on its own using sensors and AI without the need for human operation.

[0124] The "navigation system" is a system that is installed in self-driving vehicles and provides safe and efficient travel routes in real time based on disaster maps.

[0125] This invention is a system that collects data in real time from multiple sources, analyzes the damage situation using a generative AI model, and automatically creates a damage map. This system is an important tool for rapid disaster relief efforts, and in particular, when linked to the navigation systems installed in autonomous vehicles, it enables safe movement in disaster-stricken areas.

[0126] The server first collects real-time data from multiple sources, including meteorological information, geographical information, satellite images, aerial footage, social networking service data streams, demographic statistics, automobile and smartphone movement information, and utility facility operation information. After this data is collected on the server, it undergoes pre-processing.

[0127] As preprocessing means, the server performs image processing such as noise removal and resolution enhancement, and text data filtering. Specifically, for satellite images obtained from a satellite imagery service, resolution enhancement and noise removal are performed, and for posts from the social networking service Data Stream, natural language processing algorithms are used to extract important information related to the disaster.

[0128] The preprocessed data is integrated and input into a generative AI model, which uses the input data to generate detailed damage information, such as flooding status, building damage status, and evacuation routes. The results of this analysis are visualized as a damage map.

[0129] The visualized disaster map is provided to users. Specifically, the server publishes the disaster map through a web interface, allowing users to check the damage situation in real time. The information from this disaster map is also linked to the navigation system installed in the autonomous vehicle. The navigation system then provides safe and efficient travel routes in real time based on the disaster map.

[0130] As a practical example, by inputting the following prompts into a generative AI model, a highly accurate disaster map can be created:

[0131] Prompt: Based on the real-time information below, what is the best evacuation route?

[0132] Precipitation: 20mm / h

[0133] Wind speed: 15km / h

[0134] Latest SNS information: Disasters, Floods, Evacuation

[0135] This will enable navigation that always reflects the latest disaster situation based on data updated in real time, ensuring safe travel.

[0136] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0137] Step 1:

[0138] The server collects data in real time from multiple sources, including meteorological information, geographical information, satellite images, aerial footage, social networking service data streams, demographic statistics, automobile and smartphone movement information, and information on the operation of lifeline facilities.

[0139] Input: Real-time data from multiple sources

[0140] Output: Raw data collected

[0141] Specific behavior:

[0142] The server retrieves weather data using WeatherAPI, periodically downloads images from a satellite imagery service, and collects posts tagged with "disaster" from social networking service data streams.

[0143] Step 2:

[0144] The server preprocesses the collected data, performing image processing such as noise removal and resolution improvement, and filtering of text data.

[0145] Input: Raw data collected

[0146] Output: Preprocessed data

[0147] Specific behavior:

[0148] That is, the pre-processing means performs noise removal and resolution enhancement on satellite images, and uses natural language processing algorithms to extract important disaster-related information from posts from social networking service data streams.

[0149] Step 3:

[0150] The server integrates the preprocessed data and inputs it into the generative AI model as a single analysis target. Based on this data, the generative AI model generates detailed disaster information such as flooding status, building damage status, and evacuation routes.

[0151] Input: Preprocessed data

[0152] Output: Disaster information data

[0153] Specific behavior:

[0154] The integration method combines image data and text data into a single dataset and inputs it into a generative AI model, which analyzes the dataset and generates important disaster information.

[0155] Step 4:

[0156] The server creates a disaster map based on the generated damage information, visually displaying the flooding situation, building damage, evacuation routes, and other information.

[0157] Input: Disaster information data

[0158] Output: Damage map

[0159] Specific behavior:

[0160] The disaster map generation method visually represents the disaster information obtained by the generative AI model and displays flooded areas and evacuation routes on a map. For example, areas with a high flood risk are shown in red, and safe evacuation routes are shown in green.

[0161] Step 5:

[0162] The server publishes the visualized disaster map through a web interface to provide users with real-time information on the disaster situation.

[0163] Input: Disaster map

[0164] Output: A damage map on a user-accessible web interface

[0165] Specific behavior:

[0166] The provision means uploads the generated disaster map to a web interface so that users can access it through a browser. It also supports a disaster information notification function as needed.

[0167] Step 6:

[0168] The device then links the disaster map information to the autonomous vehicle's navigation system, which then provides safe and efficient travel routes in real time based on the disaster map.

[0169] Input: Disaster map on web interface

[0170] Output: Navigation route for autonomous vehicles

[0171] Specific behavior:

[0172] The navigation system receives disaster map data and calculates a safe route based on the analysis results. The autonomous vehicle then navigates safely through the disaster area, enabling rescue operations to be carried out quickly and efficiently.

[0173] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0174] This invention combines a system that collects data in real time from multiple sources, analyzes the damage situation using a generative AI model, and automatically creates a damage map, with an emotion engine that recognizes the user's emotions. This system is an important tool for swift disaster relief activities, and can also provide psychological support to users.

[0175] The server connects to multiple information sources, such as weather information services, geographic information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems, to collect the necessary data in real time. The collected data is integrated and preprocessed. For example, weather data, satellite imagery, drone footage, and social media post data are temporarily stored and then subjected to appropriate preprocessing before various analyses.

[0176] Specifically, the server periodically downloads the latest satellite images from a satellite imagery service, and uses an image processing module to improve resolution and remove noise. Social media posts are collected in real time from the data stream, and natural language processing algorithms may be used to extract information related to the disaster.

[0177] The server integrates the collected data and inputs it into a generative AI model. The generative AI model analyzes the current situation in the affected areas and creates a damage map. It also uses an emotion engine to analyze the emotional information in the post text extracted from the social media data stream. For example, from a post tagged with "disaster," the emotion engine analyzes the poster's emotions and identifies emotional states such as fear, relief, and tension.

[0178] The emotional information analyzed by the emotion engine is reflected on a disaster map. For example, areas with heightened fear and anxiety can be color-coded, and this information can be used to provide appropriate psychological support to government and local government officials.

[0179] The generated disaster map is uploaded to a web interface by the server and can be accessed by users and their devices. Users can check the damage situation and emotional information in real time through the web portal. This not only provides users with information to take appropriate actions, but also helps them understand where psychological support is needed.

[0180] For example, the server retrieves the latest precipitation data from an API provided by the Japan Meteorological Agency and downloads satellite imagery from a satellite imagery service at 8:00, 10:00, and 12:00. The processed satellite imagery is analyzed by a generative AI model to identify areas at high risk of flooding and evacuation routes. At the same time, posts extracted from social media data streams are analyzed by an emotion engine, and areas with a large number of posts expressing fear or anxiety are highlighted in red.

[0181] Users can view this disaster map on a web portal and quickly grasp the situation and emotional information of the affected areas, which will enable efficient rescue operations and evacuation instructions, not only saving lives and minimizing damage, but also providing psychological support to victims.

[0182] The processing flow will be explained below.

[0183] Step 1:

[0184] The server connects to multiple information sources, such as weather information services, geographic information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems, thereby preparing to collect the necessary data in real time.

[0185] Step 2:

[0186] The server retrieves current weather data from a weather information service, the latest satellite images from a satellite imagery service, real-time video streams from the drone operation team, and traffic data from the traffic information system, and stores them in temporary storage.

[0187] Step 3:

[0188] The server sends the acquired satellite image data to the image processing module, which processes it to improve resolution and remove noise, and stores the processed image data for analysis.

[0189] Step 4:

[0190] The server collects posts from social media data streams in real time and uses natural language processing algorithms to extract disaster-related information. For example, it filters posts tagged with "disaster" and analyzes location information and emotional states using a text analysis module.

[0191] Step 5:

[0192] The server uses an emotion engine to analyze users' emotional information from the filtered SNS posts. For example, it identifies emotional states such as fear, anxiety, and relief, and adds this information to the text data.

[0193] Step 6:

[0194] The server captures the real-time stream of drone footage frame by frame, tags the parts it deems important, and pre-processes them, for example analyzing the condition of a building using object recognition algorithms to identify damage to the building.

[0195] Step 7:

[0196] The server integrates pre-processed weather data, visual data, traffic data, social media data, etc., and converts them into a unified format, which prepares the input data for the generative AI model.

[0197] Step 8:

[0198] The server inputs the integrated data into a generative AI model for analysis, which analyzes the risk of flooding in the affected area, the risk of building damage, traffic congestion, etc., and calculates the risk level for each.

[0199] Step 9:

[0200] Based on the analysis results obtained from the generative AI model, the server creates a disaster map that shows color-coded areas at high risk of flooding, areas that are not flooded, evacuation routes, and areas at high emotional risk that require psychological support.

[0201] Step 10:

[0202] The server uploads the created disaster map to a web interface, making it accessible to users and devices. Users can check the damage situation and emotional information in real time through the web portal and obtain information to take appropriate actions.

[0203] Step 11:

[0204] The server will quickly send disaster maps and related information to government and local government officials and disaster relief teams, enabling them to carry out relief efforts quickly and effectively, and also to provide psychological support to victims.

[0205] Example 2

[0206] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0207] Conventional disaster response systems have difficulty collecting and analyzing data from multiple sources in real time to grasp the damage situation. It is also extremely difficult to accurately visualize the current situation in the disaster area and to reflect the emotional state of the victims. Therefore, while efficient and rapid disaster response is required, current systems are unable to meet this demand.

[0208] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting data in real time from multiple information sources, means for preprocessing the collected data, means for integrating the preprocessed data and analyzing it using a generative AI model, means for analyzing emotional information in text data using an emotion engine, and means for reflecting the analyzed emotional information in a damage map. This makes it possible not only to accurately grasp and visualize the damage situation, but also to create a damage map that reflects the emotional state of the victims.

[0209] "Multiple sources" are data sources that provide different types of information, including weather information services, geographic information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems.

[0210] "Real-time data collection means" is a combination of APIs, sensors, and data streams that obtain up-to-date data from various sources via the internet.

[0211] "Means for data preprocessing" refers to techniques that perform processes such as data cleansing, noise removal, resolution improvement, and format conversion in order to prepare collected data in a form suitable for analysis.

[0212] Data synthesis is the process of bringing together data collected from different sources over time into a single integrated data set.

[0213] "Means of analysis using generative AI models" refer to algorithms and models that utilize machine learning and artificial intelligence technology to analyze the damage situation based on integrated data.

[0214] The "emotion engine" is a system that uses natural language processing technology to automatically analyze the poster's emotions and psychological state from text data.

[0215] "Means for analyzing emotional information from text data" refers to the process of using an emotion engine to extract the emotions of posters from text data such as social media posts and quantifying them.

[0216] "Means for reflecting analyzed emotional information on a disaster map" refers to a technique that uses color coding and icons to visually show analyzed emotional information on a disaster map.

[0217] The "means for visualizing a disaster map and providing it to users" is a system that generates an intuitively easy-to-understand map based on the analyzed disaster situation and emotional information, and provides this to users through a web interface or application.

[0218] The present invention is a system that collects data in real time from multiple information sources, preprocesses it, and analyzes it using a generative AI model. It also uses an emotion engine to analyze emotional information from text data and reflects the results in a disaster map, providing users with information about the damage situation and emotions. An embodiment of this system is described in detail below.

[0219] Data collection

[0220] The server collects data in real time from multiple sources, including meteorological information services, geographical information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems, making it possible to grasp the damage situation from multiple perspectives.

[0221] For example, the server can use the API of the Japan Meteorological Agency to obtain the latest precipitation data, or it can periodically download satellite images from a satellite imagery service.

[0222] Data Preprocessing

[0223] The server pre-processes the collected data, which includes data cleansing, noise removal, resolution enhancement, format conversion, etc. Specifically, it uses a specific image processing module to enhance the resolution and remove noise from the satellite image data.

[0224] In addition, natural language processing algorithms are used to extract information related to the disaster from social media data.

[0225] Data integration

[0226] The server then integrates the pre-processed data, collating data from different sources into a single integrated dataset along a timeline, thereby building a database that can provide a comprehensive understanding of the disaster situation.

[0227] Analysis of the damage situation

[0228] The server inputs the integrated data into a generative AI model, which analyzes the damage situation in real time. The generative AI model uses machine learning and artificial intelligence technology to analyze the current situation in the disaster area based on various data.

[0229] Emotional information analysis

[0230] The server uses an emotion engine to analyze the emotional information of posts extracted from SNS data. The emotional information obtained through this analysis is used to visualize the psychological state of people in the disaster-stricken areas.

[0231] Creating a disaster map

[0232] The server integrates the damage situation and emotional information to create a damage map. The damage map is visually easy to understand and is designed to allow users to quickly grasp the current situation in the affected areas. Areas at high risk of flooding are displayed in blue, while areas with rising feelings of fear and anxiety are displayed in red.

[0233] Upload to the web interface

[0234] The server uploads the generated damage map to a web interface, allowing users and devices to check the damage situation and emotional information in real time.

[0235] User Use

[0236] Users can view the disaster map through a web portal and check the damage situation and sentiment information in real time, which provides information for taking appropriate actions.

[0237] Examples of concrete examples and prompts

[0238] Examples:

[0239] 1. The server obtains the latest precipitation data using the Japan Meteorological Agency's API.

[0240] 2. The server downloads satellite images at 8:00, 10:00, and 12:00 from the satellite imagery service.

[0241] 3. The server uses the generative AI model to analyze the processed satellite imagery and identify areas at high risk of flooding.

[0242] 4. The server uses an emotion engine to analyze posts extracted from the social media data stream and displays areas with a large number of posts expressing fear or anxiety in red.

[0243] 5. Users can view the damage map on the web portal and obtain information to take appropriate actions.

[0244] Example prompt sentence:

[0245] "Identify areas at high risk of flooding based on the latest rainfall data and satellite imagery."

[0246] "Please extract emotional information from social media data and reflect it in the disaster map."

[0247] "View detailed sentiment information for areas highlighted in red"

[0248] This will provide a concrete example of how to put the invention into practice, making it possible to speed up disaster relief efforts, provide accurate information, and provide psychological support to disaster victims.

[0249] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0250] Step 1:

[0251] The server collects data in real time from multiple sources, such as meteorological information services, geographic information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems. The input is the data retrieved by sending requests from these sources, and the output is the temporarily stored, unprocessed data.

[0252] Step 2:

[0253] The server preprocesses the collected data. Preprocessing includes data cleansing, noise removal, resolution enhancement, and format conversion. For example, it performs resolution enhancement on satellite image data and performs natural language processing on social media data to extract relevant information. The input is unprocessed data, and the output is processed data.

[0254] Step 3:

[0255] The server integrates the preprocessed data. It combines data collected from different sources into a single integrated dataset along a time axis. For example, it integrates weather data, satellite image data, and social media data into a single dataset. The input is preprocessed data, and the output is the integrated dataset.

[0256] Step 4:

[0257] The server inputs the integrated data into a generative AI model, which analyzes the damage situation in real time. The generative AI model uses machine learning and artificial intelligence technology to analyze the current situation in the disaster-stricken area based on various data. The input is the integrated dataset, and the output is the analysis result of the damage situation. For example, it can identify areas with a high risk of flooding.

[0258] Step 5:

[0259] The server uses an emotion engine to analyze the emotional information of posts extracted from SNS data. The emotion engine uses natural language processing technology to analyze emotional information from text data. The input is the text data extracted from SNS, and the output is the analysis result of the emotional information. For example, emotions indicating fear or anxiety are extracted.

[0260] Step 6:

[0261] The server integrates the damage situation and emotional information to create a damage map. The damage map visually reflects areas at high risk of flooding and emotional information. The input is the analysis results of the damage situation and emotional information, and the output is a damage map. For example, areas at high risk of flooding are displayed in blue, and heightened emotions in red.

[0262] Step 7:

[0263] The server uploads the generated disaster map to a web interface, allowing users and devices to check the disaster situation and emotional information in real time. The input is the disaster map, and the output is the disaster map displayed on the web portal.

[0264] Step 8:

[0265] Users can view the disaster map through the web portal and check the damage situation and emotional information, which can provide information to take appropriate actions. For example, users can check the disaster map and find a safe evacuation route.

[0266] (Application example 2)

[0267] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0268] Current navigation systems for autonomous vehicles lack the functionality to support a rapid and safe response in disaster-stricken areas. The lack of a means to analyze and provide information on the disaster situation and emotions in real time makes it difficult to select safe routes or provide psychological support in emergencies. Furthermore, there is insufficient integration of information sources that can accurately grasp the situation in disaster-stricken areas, and the development of a system that encourages appropriate action is required.

[0269] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting data from multiple information sources in real time and linking with the vehicle's navigation system, means for preprocessing the collected data and calculating safe routes and emotional information, and means for integrating the preprocessed data, analyzing it using a generative AI model, and calculating the situation in the disaster area and an appropriate route to take. This makes it possible to accurately grasp the situation in the disaster area and provide the optimal route in real time. The user can take safe and efficient evacuation actions, and psychological support can also be provided to the disaster victims.

[0270] "Multiple information sources" refers to data sources such as weather information, geographic information, satellite images, drone footage, social network data, traffic information, and information on lifeline facilities.

[0271] "Real-time" refers to the acquisition and processing of data almost simultaneously, and the provision of that information to users without delay.

[0272] "Preprocessing" refers to a process for converting acquired raw data into an analyzable format, and includes processes such as removing noise from the data and improving resolution.

[0273] A "generative AI model" refers to an artificial intelligence model that analyzes collected data and generates new information based on the damage situation and emotional data.

[0274] A "disaster map" is a map that visually displays the current situation and emotional state of the disaster-stricken area based on analyzed data.

[0275] An "emotion engine" refers to an algorithm that analyzes emotional information from text data such as social network data.

[0276] "Automobile navigation system" refers to a system installed in an autonomous vehicle that provides guidance on the vehicle's current location and route to a destination.

[0277] A "safe route" refers to a route that takes into account the disaster situation and emotional information and minimizes damage and danger in the disaster area.

[0278] "Visualization" refers to displaying analysis results and damage maps in a format that is easy to see and understand.

[0279] "User" refers to the people who receive and use information within an autonomous vehicle.

[0280] In this invention, the server collects data from multiple sources in real time, preprocesses the data, and calculates safe routes and emotional information. It also uses a generative AI model to integrate the preprocessed data and calculate the situation in the disaster area and appropriate routes. Next, it creates a disaster map based on the analysis results and updates it in real time. Finally, it visualizes the disaster map and safe routes and provides them to users.

[0281] A specific example is given below.

[0282] Source data collection and preprocessing

[0283] The server connects to multiple sources, including meteorological information, geographical information, satellite images, drone footage, social network data, traffic information, and information on utility facilities, to collect the necessary data in real time. For example, it periodically downloads the latest satellite images from satellite imagery services, and uses an image processing module to improve resolution and remove noise. It also collects posts from social network data streams in real time and uses natural language processing algorithms to extract information related to the disaster.

[0284] Data integration and analysis

[0285] The collected data is integrated and preprocessed on a server. The preprocessed data is then input into a generative AI model to analyze the current situation in the disaster-stricken areas and automatically create a disaster map. An emotion engine is also used to analyze the emotional information in the post text extracted from social network data. For example, from a post tagged with "disaster," the emotion engine analyzes the poster's emotions and identifies emotional states such as fear, relief, and tension.

[0286] Creation and updating of disaster maps

[0287] The generated disaster map is updated in real time by the server and provided to users via a web interface. Users can view the disaster map in real time and quickly grasp the situation and emotional information in the affected area, which enables efficient evacuation instructions and psychological support.

[0288] Examples and prompts

[0289] For example, the server retrieves the latest precipitation data from a weather service's API and downloads satellite imagery from a satellite imagery service at 8:00, 10:00, and 12:00. The processed satellite imagery is analyzed by a generative AI model to identify areas at high risk of flooding and evacuation routes. At the same time, posts extracted from social network data streams are analyzed by an emotion engine, and areas with a high number of posts expressing fear or anxiety are highlighted in red.

[0290] An example of a prompt sentence to input to the generative AI model is as follows:

[0291] Social network posts from near the affected area (disaster):

[0292] 1. I'm terrified. I need help.

[0293] 2. The area is relatively safe.

[0294] Please analyze the data and suggest the best safe route and support points.

[0295] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0296] Step 1:

[0297] The server collects data in real time from multiple sources, such as weather information services, geographic information services, satellite imagery services, drone operation teams, social network data streams, traffic information systems, and lifeline facility management systems. The input is data obtained from various APIs, and the output is the collected raw data.

[0298] Step 2:

[0299] The server preprocesses the collected data, which includes improving the resolution of satellite images, removing noise, and extracting relevant information from social network data. The input for preprocessing is the raw data collected in step 1, and the output is the preprocessed data converted into an analyzable format.

[0300] Step 3:

[0301] The server integrates the preprocessed data. In this step, meteorological data, geographic data, satellite image data, drone footage, social network data, etc. are converted into a unified format to create an integrated dataset. The input is the preprocessed data, and the output is the integrated data.

[0302] Step 4:

[0303] The server inputs the integrated data into a generative AI model to analyze the situation in the disaster-stricken areas. The generative AI model analyzes the damage information and emotional information to create a damage map. In this process, it also uses an emotional engine to analyze the emotional information of posters from social network data. The input is the integrated data, and the output is a damage map and emotional information.

[0304] Step 5:

[0305] The server creates a disaster map based on the analysis results and updates it in real time. Specifically, it visually displays flood risk, evacuation routes, emotional information, etc. based on the output of the generative AI model. The input is the analysis results from the generative AI model, and the output is a disaster map.

[0306] Step 6:

[0307] The server visualizes the disaster map and the safe route and provides it to the user. Specifically, the disaster map is displayed on a web interface or a car navigation system so that the user can check it in real time. The input is the disaster map, and the output is the visualized disaster map.

[0308] Step 7:

[0309] Users can use the visualized disaster map to take appropriate actions, such as choosing a safe evacuation route and paying attention to areas where psychological support is needed. The input is the visualized disaster map, and the output is the user's actions.

[0310] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0311] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0312] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0313] [Second embodiment]

[0314] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0315] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0316] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0317] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0318] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0319] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0320] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0321] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0322] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0323] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0324] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0325] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0326] This invention is a system that collects data in real time from multiple sources, analyzes the damage situation using a generative AI model, and automatically creates a damage map, which will be an important tool for rapid disaster relief efforts.

[0327] The server collects data from various sources, integrates it, and preprocesses it. Specifically, it acquires real-time weather information, geographic information, satellite images, drone footage, social media data streams, demographic statistics, automobile and smartphone movement information, and information on the operation of lifeline facilities. It then preprocesses the collected data and prepares it for analysis using a generative AI model.

[0328] For example, the server periodically downloads satellite images from a satellite imagery service and uses an image processing module to enhance resolution and remove noise.The server also collects posts from social media data streams in real time and uses natural language processing algorithms to extract important information related to the disaster.

[0329] The server integrates the preprocessed data and inputs it into a generative AI model. The generative AI model analyzes the input data and generates detailed information such as the flooding status of the affected area, the damage status of buildings, and evacuation routes. For example, areas at high risk of flooding are displayed in red, and evacuation routes are displayed in green.

[0330] Once the damage map is generated, the server uploads the information to a web interface for visual display. Users and devices can access the web portal and check the damage situation in real time. In addition, the server quickly notifies government and local government officials and disaster relief teams of the damage map and related information to support appropriate rescue operations.

[0331] As a specific example, the server obtains the latest precipitation data from an API provided by the Japan Meteorological Agency and downloads satellite images at 8:00, 10:00, and 12:00 from a satellite imagery service. These images are then processed by an image processing module to remove noise and improve resolution before being input into the generative AI model. Posts tagged with "disaster" are extracted from the social media data stream, and these are also analyzed by the generative AI model. The generative AI model then creates a disaster area map based on information such as flood risk, damaged buildings, and evacuation routes.

[0332] Users can view this disaster map through a web portal and quickly grasp the situation in the affected areas, which will enable efficient rescue operations and evacuation instructions, saving lives and minimizing damage.

[0333] The processing flow will be explained below.

[0334] Step 1:

[0335] The server connects to multiple information sources, such as meteorological information services, geographic information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems, making it ready to collect the necessary data in real time.

[0336] Step 2:

[0337] The server retrieves the latest weather data from a weather information service, the latest satellite images from a satellite image service, real-time video streams from the drone operation team, and traffic data from a traffic information system, and stores them in temporary storage.

[0338] Step 3:

[0339] The server sends the latest satellite image data to the image processing module, which enhances the resolution of the satellite images and removes noise. The processed images are stored for further analysis.

[0340] Step 4:

[0341] The server collects posts from social media data streams in real time and uses natural language processing algorithms to extract information related to the disaster. For example, it filters posts tagged with "disaster" and analyzes them using a text analysis module to extract location information and details of the disaster situation.

[0342] Step 5:

[0343] The server captures the real-time stream of drone footage frame by frame, tags the parts it deems important, and pre-processes them, for example analyzing the condition of a building using object recognition algorithms to identify damage to the building.

[0344] Step 6:

[0345] The server consolidates all collected data and converts it into one unified format, where all data is linked, including weather data, visual data, traffic data, social media data, etc.

[0346] Step 7:

[0347] The server inputs the integrated data into a generative AI model, which analyzes the flood risk, building damage risk, traffic congestion status, and other factors in the affected area to calculate the risk level for each area.

[0348] Step 8:

[0349] The server then creates a disaster map based on the analysis results obtained from the generative AI model. For example, areas at high risk of flooding are displayed in red, while areas that are not flooded are displayed in green. Information on evacuation routes and first-aid stations is also added to each area.

[0350] Step 9:

[0351] The server uploads the created disaster map to a web interface for access by users and devices. Users can view the disaster map in real time through the web portal and check the information needed to take appropriate action.

[0352] Step 10:

[0353] The server will quickly send disaster maps and related information to government and local government officials and disaster relief teams, enabling rescue efforts to be carried out quickly and effectively, saving lives and minimizing damage.

[0354] Example 1

[0355] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0356] Conventional disaster response systems have difficulty collecting data from multiple sources, analyzing it quickly, and creating detailed maps of affected areas. They also lack the ability to update data in real time or preprocess data in a variety of formats. This hinders the speed and efficiency of disaster response, creating challenges in saving lives and minimizing damage.

[0357] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0358] In this invention, the server includes a means for collecting data from multiple sources in real time, a means for preprocessing the collected data, and a means for integrating the preprocessed data and analyzing it using a generative artificial intelligence model. This enables noise removal and extraction of important information from the collected visual and text data. Flood risk and evacuation routes are then identified based on the preprocessed data, and a disaster situation map is created based on the results and uploaded to a web interface. This allows users to grasp the latest damage situation in real time, enabling rapid and efficient disaster response. Furthermore, information can be quickly provided to government and local government officials and disaster relief teams, enabling appropriate responses.

[0359] "Multiple sources" refers to multiple data providers providing different types of data, such as meteorological data providers, satellite imagery providers, aircraft footage providers, digital social media platforms, and utility information providers.

[0360] "Real-time data collection means" refers to technologies or methods for obtaining data immediately from each source, including obtaining data via an API or by monitoring data streams in real time.

[0361] "Preprocessing means" refers to a series of processing techniques used to convert collected data into a format suitable for generative AI models, including processes such as noise removal, resolution enhancement, and key information extraction.

[0362] A "generative artificial intelligence model" is a model that uses a set of algorithms and data analysis techniques to analyze collected data and generate useful information. This includes deep learning models and natural language processing models.

[0363] "Means for analysis" refers to the techniques and methods for analyzing preprocessed data using a generative artificial intelligence model to identify the damage situation and related information.

[0364] "Disaster Situation Map" refers to a map that visually shows the situation in a disaster-stricken area based on the analysis results of a generative AI model, including information on flood risk, building damage, evacuation routes, etc.

[0365] "Visualization means" refers to technologies for displaying analysis results in a format that is easy for users to understand, including technologies for displaying maps on a graphical user interface or web browser.

[0366] "Noise reduction" refers to the process of removing unnecessary information and errors from collected data, improving the quality of the data and increasing the accuracy of analysis results.

[0367] "Important information extraction" refers to the process of extracting only the useful information needed for analysis from collected data. This includes extracting keywords from text data and extracting features from image data.

[0368] "Flood risk" refers to areas that are more likely to experience natural disasters such as floods. A generative AI model identifies these and displays them on the map.

[0369] "Evacuation routes" refer to routes for safe evacuation in the event of a disaster. These are also analyzed by the generative AI model and displayed on the map.

[0370] "Means for uploading to a web interface" refers to the technology used to publish disaster maps and related information on an internet platform and make them accessible to users.

[0371] This invention is a system that collects data in real time from multiple sources, analyzes the damage situation using a generative artificial intelligence model, and automatically creates a disaster situation map, which will be an important tool for carrying out disaster relief activities quickly and efficiently.

[0372] The server collects data in real time from various sources, including weather information, geographic information, satellite imagery, aircraft footage, digital social media data streams, demographic statistics, automobile and smartphone movement information, and utility equipment operation information. This data is collected from different data providers via APIs and data streams.

[0373] The server then preprocesses the collected data. For image data, it uses image processing modules (e.g., OpenCV) to improve resolution and remove noise. For text data, it uses natural language processing algorithms (e.g., BERT from the Transformers library) to extract important information. This improves the quality of the data and increases the accuracy of the subsequent analysis process.

[0374] The preprocessed data is integrated by a server and fed into a generative artificial intelligence model. This model uses deep learning techniques to analyze the collected visual and text data to identify flood risk, building damage, evacuation routes, and other information. For example, using satellite imagery and meteorological data as input, the generative AI model identifies areas at high risk of flooding. It also uses text data from aerial footage and digital social media data streams to identify building damage.

[0375] The generated analysis results are compiled into a disaster situation map by the server, which is visually displayed using a visualization tool (e.g., D3.js) and provided to users. The server then uploads the disaster situation map to a web interface (e.g., a React app) so that users can access it in real time.

[0376] Furthermore, the server will quickly send disaster situation maps and related information to government and local government officials and disaster relief teams, allowing relevant organizations to grasp the latest damage situation and take appropriate action.

[0377] For example, the server obtains the latest precipitation data from a weather data provider and periodically retrieves satellite images from a satellite image provider. These images are denoised and enhanced in resolution using OpenCV before being input into a generative AI model. It also collects posts tagged with "disaster" from digital social media data streams and extracts key information using the BERT model. Based on the collected data, the generative AI model analyzes information such as flood risk, damaged buildings, and evacuation routes to create a disaster situation map.

[0378] An example of a prompt is:

[0379] Identify areas at high risk of flooding based on the latest satellite imagery and weather data, and provide evacuation routes based on information extracted from the text of digital social media posts.

[0380] In this way, the system uses data collected in real time to enable fast and efficient disaster response.

[0381] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0382] Step 1: Data collection

[0383] The server collects data in real time from various sources. Specifically, it uses APIs to obtain the latest precipitation, temperature, wind speed, etc. from weather information services, and periodically downloads satellite images from satellite image services. It also collects real-time video feeds from aircraft, posts tagged with "disaster" from digital social media platforms, and the operational status of lifeline facilities. The inputs are API calls and data streams from various sources. The output is raw data.

[0384] Step 2: Preprocessing the data

[0385] The server preprocesses the collected data. For image data, OpenCV is used to increase the resolution and remove noise. For example, the resolution of satellite images is doubled and noise is removed. For text data, a natural language processing algorithm (e.g., BERT from the Transformers library) is used to extract important information related to the disaster. For example, keywords such as "flood" and "evacuation" are extracted from posts tagged with "disaster." The input is the collected raw data, and the output is the preprocessed data.

[0386] Step 3: Integrate the data

[0387] The server integrates the preprocessed data. It integrates data from different sources based on time and geographic information to create a consistent dataset. The input is the preprocessed data, and the output is the integrated dataset. For example, it performs processing such as matching the posting time of social media data with the shooting time of satellite images.

[0388] Step 4: Analysis by generative AI model

[0389] The server inputs the integrated data into a generative AI model to analyze the damage situation. Based on the input data, areas at high risk of flooding, evacuation routes, and the extent of building damage are identified. For example, areas at high risk of flooding are shown in red, and evacuation routes in green. The input is the integrated data set, and the output is the analysis results. An analytical algorithm using deep learning is applied to the generative AI model.

[0390] Step 5: Generate a damage map

[0391] The server generates a disaster map based on the analysis results of the generative AI model. The analysis results are visualized using a visualization tool (e.g., D3.js) and displayed in a format that is easy for users to understand. The analysis results are input, and a visualized disaster map is obtained as output. For example, a map can be created that color-codes flood risk and evacuation routes.

[0392] Step 6: Upload to the web interface

[0393] The server uploads the generated disaster map to a web interface (e.g., a React app) so that users can access it in real time. As input, we have a visualized disaster map, and as output, we have a web page that users can view. For example, the server publishes the latest disaster map on a website.

[0394] Step 7: Sending notifications

[0395] The server quickly notifies government and local government officials and disaster relief teams of the analysis results and damage maps. The inputs are the visualized damage maps and analysis results, and the output is notifications sent via email, SMS, or a dedicated app. For example, the server could send the latest damage map to relevant organizations via email.

[0396] (Application example 1)

[0397] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0398] Conventional disaster area information collection systems have difficulty in collecting and analyzing information in real time, making it difficult to carry out prompt rescue operations in disaster-stricken areas and provide safe evacuation routes. Furthermore, conventional navigation systems are unable to provide routes that take into account the conditions in disaster-stricken areas, making it difficult to guarantee the safe movement of autonomous vehicles in disaster-stricken areas.

[0399] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0400] In this invention, the server includes means for collecting data in real time from multiple information sources, means for preprocessing the collected data, means for integrating the preprocessed data and analyzing it using a generative AI model, means for creating a damage map based on the analysis results, means for visualizing the damage map and providing it to users, and means for linking the information on the damage map to a navigation system installed in an autonomous vehicle. This makes it possible to grasp the real-time situation in the disaster area and provide safe and efficient routes.

[0401] "Multiple sources" refers to multiple types of information sources, including weather information, geographic information, satellite images, aerial footage, social networking service data streams, demographic statistics, automobile and smartphone movement information, and information on the operation of lifeline facilities.

[0402] "Collection means" refers to a means for acquiring data from multiple information sources in real time and collecting it on a server.

[0403] "Preprocessing means" refers to means for performing processes such as noise removal, resolution enhancement, and text filtering in order to convert collected data into a format or state suitable for analysis.

[0404] The "integration means" is a means for collecting preprocessed data from multiple information sources and treating them as a single analysis target.

[0405] A "generative AI model" is an artificial intelligence model that analyzes input data and generates disaster information such as flooding status, building damage status, and evacuation routes.

[0406] The "disaster map generation means" is a means of creating a disaster map that visually displays flooding conditions, building damage conditions, evacuation routes, etc. based on the analysis results obtained by the generative AI model.

[0407] The "visualization means" refers to a means including a web interface or application for displaying the generated disaster map in a format that can be viewed by a user.

[0408] "User provision means" refers to means including a web portal and notification system that allow users and relevant organizations to access the visualized disaster map.

[0409] An "autonomous vehicle" is a vehicle that moves on its own using sensors and AI without the need for human operation.

[0410] The "navigation system" is a system that is installed in self-driving vehicles and provides safe and efficient travel routes in real time based on disaster maps.

[0411] This invention is a system that collects data in real time from multiple sources, analyzes the damage situation using a generative AI model, and automatically creates a damage map. This system is an important tool for rapid disaster relief efforts, and in particular, when linked to the navigation systems installed in autonomous vehicles, it enables safe movement in disaster-stricken areas.

[0412] The server first collects real-time data from multiple sources, including meteorological information, geographical information, satellite images, aerial footage, social networking service data streams, demographic statistics, automobile and smartphone movement information, and utility facility operation information. After this data is collected on the server, it undergoes pre-processing.

[0413] As preprocessing means, the server performs image processing such as noise removal and resolution enhancement, and text data filtering. Specifically, for satellite images obtained from a satellite imagery service, resolution enhancement and noise removal are performed, and for posts from the social networking service Data Stream, natural language processing algorithms are used to extract important information related to the disaster.

[0414] The preprocessed data is integrated and input into a generative AI model, which uses the input data to generate detailed damage information, such as flooding status, building damage status, and evacuation routes. The results of this analysis are visualized as a damage map.

[0415] The visualized disaster map is provided to users. Specifically, the server publishes the disaster map through a web interface, allowing users to check the damage situation in real time. The information from this disaster map is also linked to the navigation system installed in the autonomous vehicle. The navigation system then provides safe and efficient travel routes in real time based on the disaster map.

[0416] As a practical example, by inputting the following prompts into a generative AI model, a highly accurate disaster map can be created:

[0417] Prompt: Based on the real-time information below, what is the best evacuation route?

[0418] Precipitation: 20mm / h

[0419] Wind speed: 15km / h

[0420] Latest SNS information: Disasters, Floods, Evacuation

[0421] This will enable navigation that always reflects the latest disaster situation based on data updated in real time, ensuring safe travel.

[0422] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0423] Step 1:

[0424] The server collects data in real time from multiple sources, including meteorological information, geographical information, satellite images, aerial footage, social networking service data streams, demographic statistics, automobile and smartphone movement information, and information on the operation of lifeline facilities.

[0425] Input: Real-time data from multiple sources

[0426] Output: Raw data collected

[0427] Specific behavior:

[0428] The server retrieves weather data using WeatherAPI, periodically downloads images from a satellite imagery service, and collects posts tagged with "disaster" from social networking service data streams.

[0429] Step 2:

[0430] The server preprocesses the collected data, performing image processing such as noise removal and resolution improvement, and filtering of text data.

[0431] Input: Raw data collected

[0432] Output: Preprocessed data

[0433] Specific behavior:

[0434] That is, the pre-processing means performs noise removal and resolution enhancement on satellite images, and uses natural language processing algorithms to extract important disaster-related information from posts from social networking service data streams.

[0435] Step 3:

[0436] The server integrates the preprocessed data and inputs it into the generative AI model as a single analysis target. Based on this data, the generative AI model generates detailed disaster information such as flooding status, building damage status, and evacuation routes.

[0437] Input: Preprocessed data

[0438] Output: Disaster information data

[0439] Specific behavior:

[0440] The integration method combines image data and text data into a single dataset and inputs it into a generative AI model, which analyzes the dataset and generates important disaster information.

[0441] Step 4:

[0442] The server creates a disaster map based on the generated damage information, visually displaying the flooding situation, building damage, evacuation routes, and other information.

[0443] Input: Disaster information data

[0444] Output: Damage map

[0445] Specific behavior:

[0446] The disaster map generation method visually represents the disaster information obtained by the generative AI model and displays flooded areas and evacuation routes on a map. For example, areas with a high flood risk are shown in red, and safe evacuation routes are shown in green.

[0447] Step 5:

[0448] The server publishes the visualized disaster map through a web interface to provide users with real-time information on the disaster situation.

[0449] Input: Disaster map

[0450] Output: A damage map on a user-accessible web interface

[0451] Specific behavior:

[0452] The provision means uploads the generated disaster map to a web interface so that users can access it through a browser. It also supports a disaster information notification function as needed.

[0453] Step 6:

[0454] The device then links the disaster map information to the autonomous vehicle's navigation system, which then provides safe and efficient travel routes in real time based on the disaster map.

[0455] Input: Disaster map on web interface

[0456] Output: Navigation route for autonomous vehicles

[0457] Specific behavior:

[0458] The navigation system receives disaster map data and calculates a safe route based on the analysis results. The autonomous vehicle then navigates safely through the disaster area, enabling rescue operations to be carried out quickly and efficiently.

[0459] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0460] This invention combines a system that collects data in real time from multiple sources, analyzes the damage situation using a generative AI model, and automatically creates a damage map, with an emotion engine that recognizes the user's emotions. This system is an important tool for swift disaster relief activities, and can also provide psychological support to users.

[0461] The server connects to multiple information sources, such as weather information services, geographic information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems, to collect the necessary data in real time. The collected data is integrated and preprocessed. For example, weather data, satellite imagery, drone footage, and social media post data are temporarily stored and then subjected to appropriate preprocessing before various analyses.

[0462] Specifically, the server periodically downloads the latest satellite images from a satellite imagery service, and uses an image processing module to improve resolution and remove noise. Social media posts are collected in real time from the data stream, and natural language processing algorithms may be used to extract information related to the disaster.

[0463] The server integrates the collected data and inputs it into a generative AI model. The generative AI model analyzes the current situation in the affected areas and creates a damage map. It also uses an emotion engine to analyze the emotional information in the post text extracted from the social media data stream. For example, from a post tagged with "disaster," the emotion engine analyzes the poster's emotions and identifies emotional states such as fear, relief, and tension.

[0464] The emotional information analyzed by the emotion engine is reflected on a disaster map. For example, areas with heightened fear and anxiety can be color-coded, and this information can be used to provide appropriate psychological support to government and local government officials.

[0465] The generated disaster map is uploaded to a web interface by the server and can be accessed by users and their devices. Users can check the damage situation and emotional information in real time through the web portal. This not only provides users with information to take appropriate actions, but also helps them understand where psychological support is needed.

[0466] For example, the server retrieves the latest precipitation data from an API provided by the Japan Meteorological Agency and downloads satellite imagery from a satellite imagery service at 8:00, 10:00, and 12:00. The processed satellite imagery is analyzed by a generative AI model to identify areas at high risk of flooding and evacuation routes. At the same time, posts extracted from social media data streams are analyzed by an emotion engine, and areas with a large number of posts expressing fear or anxiety are highlighted in red.

[0467] Users can view this disaster map on a web portal and quickly grasp the situation and emotional information of the affected areas, which will enable efficient rescue operations and evacuation instructions, not only saving lives and minimizing damage, but also providing psychological support to victims.

[0468] The processing flow will be explained below.

[0469] Step 1:

[0470] The server connects to multiple information sources, such as weather information services, geographic information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems, thereby preparing to collect the necessary data in real time.

[0471] Step 2:

[0472] The server retrieves current weather data from a weather information service, the latest satellite images from a satellite imagery service, real-time video streams from the drone operation team, and traffic data from the traffic information system, and stores them in temporary storage.

[0473] Step 3:

[0474] The server sends the acquired satellite image data to the image processing module, which processes it to improve resolution and remove noise, and stores the processed image data for analysis.

[0475] Step 4:

[0476] The server collects posts from social media data streams in real time and uses natural language processing algorithms to extract disaster-related information. For example, it filters posts tagged with "disaster" and analyzes location information and emotional states using a text analysis module.

[0477] Step 5:

[0478] The server uses an emotion engine to analyze users' emotional information from the filtered SNS posts. For example, it identifies emotional states such as fear, anxiety, and relief, and adds this information to the text data.

[0479] Step 6:

[0480] The server captures the real-time stream of drone footage frame by frame, tags the parts it deems important, and pre-processes them, for example analyzing the condition of a building using object recognition algorithms to identify damage to the building.

[0481] Step 7:

[0482] The server integrates pre-processed weather data, visual data, traffic data, social media data, etc., and converts them into a unified format, which prepares the input data for the generative AI model.

[0483] Step 8:

[0484] The server inputs the integrated data into a generative AI model for analysis, which analyzes the risk of flooding in the affected area, the risk of building damage, traffic congestion, etc., and calculates the risk level for each.

[0485] Step 9:

[0486] Based on the analysis results obtained from the generative AI model, the server creates a disaster map that shows color-coded areas at high risk of flooding, areas that are not flooded, evacuation routes, and areas at high emotional risk that require psychological support.

[0487] Step 10:

[0488] The server uploads the created disaster map to a web interface, making it accessible to users and devices. Users can check the damage situation and emotional information in real time through the web portal and obtain information to take appropriate actions.

[0489] Step 11:

[0490] The server will quickly send disaster maps and related information to government and local government officials and disaster relief teams, enabling them to carry out relief efforts quickly and effectively, and also to provide psychological support to victims.

[0491] Example 2

[0492] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0493] Conventional disaster response systems have difficulty collecting and analyzing data from multiple sources in real time to grasp the damage situation. It is also extremely difficult to accurately visualize the current situation in the disaster area and to reflect the emotional state of the victims. Therefore, while efficient and rapid disaster response is required, current systems are unable to meet this demand.

[0494] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting data in real time from multiple information sources, means for preprocessing the collected data, means for integrating the preprocessed data and analyzing it using a generative AI model, means for analyzing emotional information in text data using an emotion engine, and means for reflecting the analyzed emotional information in a damage map. This makes it possible not only to accurately grasp and visualize the damage situation, but also to create a damage map that reflects the emotional state of the victims.

[0495] "Multiple sources" are data sources that provide different types of information, including weather information services, geographic information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems.

[0496] "Real-time data collection means" is a combination of APIs, sensors, and data streams that obtain up-to-date data from various sources via the internet.

[0497] "Means for data preprocessing" refers to techniques that perform processes such as data cleansing, noise removal, resolution improvement, and format conversion in order to prepare collected data in a form suitable for analysis.

[0498] Data synthesis is the process of bringing together data collected from different sources over time into a single integrated data set.

[0499] "Means of analysis using generative AI models" refer to algorithms and models that utilize machine learning and artificial intelligence technology to analyze the damage situation based on integrated data.

[0500] The "emotion engine" is a system that uses natural language processing technology to automatically analyze the poster's emotions and psychological state from text data.

[0501] "Means for analyzing emotional information from text data" refers to the process of using an emotion engine to extract the emotions of posters from text data such as social media posts and quantifying them.

[0502] "Means for reflecting analyzed emotional information on a disaster map" refers to a technique that uses color coding and icons to visually show analyzed emotional information on a disaster map.

[0503] The "means for visualizing a disaster map and providing it to users" is a system that generates an intuitively easy-to-understand map based on the analyzed disaster situation and emotional information, and provides this to users through a web interface or application.

[0504] The present invention is a system that collects data in real time from multiple information sources, preprocesses it, and analyzes it using a generative AI model. It also uses an emotion engine to analyze emotional information from text data and reflects the results in a disaster map, providing users with information about the damage situation and emotions. The following describes in detail an embodiment of this system.

[0505] Data collection

[0506] The server collects data in real time from multiple sources, including meteorological information services, geographical information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems, making it possible to grasp the damage situation from multiple perspectives.

[0507] For example, the server can use the API of the Japan Meteorological Agency to obtain the latest precipitation data, or it can periodically download satellite images from a satellite imagery service.

[0508] Data Preprocessing

[0509] The server pre-processes the collected data, which includes data cleansing, noise removal, resolution enhancement, format conversion, etc. Specifically, it uses a specific image processing module to enhance the resolution and remove noise from the satellite image data.

[0510] In addition, natural language processing algorithms are used to extract information related to the disaster from social media data.

[0511] Data integration

[0512] The server then integrates the pre-processed data, collating data from different sources into a single integrated dataset along a timeline, thereby building a database that can provide a comprehensive understanding of the disaster situation.

[0513] Analysis of the damage situation

[0514] The server inputs the integrated data into a generative AI model, which analyzes the damage situation in real time. The generative AI model uses machine learning and artificial intelligence technology to analyze the current situation in the disaster area based on various data.

[0515] Emotional information analysis

[0516] The server uses an emotion engine to analyze the emotional information of posts extracted from SNS data. The emotional information obtained through this analysis is used to visualize the psychological state of people in the disaster-stricken areas.

[0517] Creating a disaster map

[0518] The server integrates the damage situation and emotional information to create a damage map. The damage map is visually easy to understand and is designed to allow users to quickly grasp the current situation in the affected areas. Areas at high risk of flooding are displayed in blue, while areas with rising feelings of fear and anxiety are displayed in red.

[0519] Upload to the web interface

[0520] The server uploads the generated damage map to a web interface, allowing users and devices to check the damage situation and emotional information in real time.

[0521] User Use

[0522] Users can view the disaster map through a web portal and check the damage situation and sentiment information in real time, which provides information for taking appropriate actions.

[0523] Examples of concrete examples and prompts

[0524] Examples:

[0525] 1. The server obtains the latest precipitation data using the Japan Meteorological Agency's API.

[0526] 2. The server downloads satellite images at 8:00, 10:00, and 12:00 from the satellite imagery service.

[0527] 3. The server uses the generative AI model to analyze the processed satellite imagery and identify areas at high risk of flooding.

[0528] 4. The server uses an emotion engine to analyze posts extracted from the social media data stream and displays areas with a large number of posts expressing fear or anxiety in red.

[0529] 5. Users can view the damage map on the web portal and obtain information to take appropriate actions.

[0530] Example prompt sentence:

[0531] "Identify areas at high risk of flooding based on the latest rainfall data and satellite imagery."

[0532] "Please extract emotional information from social media data and reflect it in the disaster map."

[0533] "View detailed sentiment information for areas highlighted in red"

[0534] This will provide a concrete example of how to put the invention into practice, making it possible to speed up disaster relief efforts, provide accurate information, and provide psychological support to disaster victims.

[0535] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0536] Step 1:

[0537] The server collects data in real time from multiple sources, such as meteorological information services, geographic information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems. The input is the data retrieved by sending requests from these sources, and the output is the temporarily stored, unprocessed data.

[0538] Step 2:

[0539] The server preprocesses the collected data. Preprocessing includes data cleansing, noise removal, resolution enhancement, and format conversion. For example, it performs resolution enhancement on satellite image data and performs natural language processing on social media data to extract relevant information. The input is unprocessed data, and the output is processed data.

[0540] Step 3:

[0541] The server integrates the preprocessed data. It combines data collected from different sources into a single integrated dataset along a time axis. For example, it integrates weather data, satellite image data, and social media data into a single dataset. The input is preprocessed data, and the output is the integrated dataset.

[0542] Step 4:

[0543] The server inputs the integrated data into a generative AI model, which analyzes the damage situation in real time. The generative AI model uses machine learning and artificial intelligence technology to analyze the current situation in the disaster-stricken area based on various data. The input is the integrated dataset, and the output is the analysis result of the damage situation. For example, it can identify areas with a high risk of flooding.

[0544] Step 5:

[0545] The server uses an emotion engine to analyze the emotional information of posts extracted from SNS data. The emotion engine uses natural language processing technology to analyze emotional information from text data. The input is the text data extracted from SNS, and the output is the analysis result of the emotional information. For example, emotions indicating fear or anxiety are extracted.

[0546] Step 6:

[0547] The server integrates the damage situation and emotional information to create a damage map. The damage map visually reflects areas at high risk of flooding and emotional information. The input is the analysis results of the damage situation and emotional information, and the output is a damage map. For example, areas at high risk of flooding are displayed in blue, and heightened emotions in red.

[0548] Step 7:

[0549] The server uploads the generated disaster map to a web interface, allowing users and devices to check the disaster situation and emotional information in real time. The input is the disaster map, and the output is the disaster map displayed on the web portal.

[0550] Step 8:

[0551] Users can view the disaster map through the web portal and check the damage situation and emotional information, which can provide information to take appropriate actions. For example, users can check the disaster map and find a safe evacuation route.

[0552] (Application example 2)

[0553] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0554] Current navigation systems for autonomous vehicles lack the functionality to support a rapid and safe response in disaster-stricken areas. The lack of a means to analyze and provide information on the disaster situation and emotions in real time makes it difficult to select safe routes or provide psychological support in emergencies. Furthermore, there is insufficient integration of information sources that can accurately grasp the situation in disaster-stricken areas, and the development of a system that encourages appropriate action is required.

[0555] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting data from multiple information sources in real time and linking with the vehicle's navigation system, means for preprocessing the collected data and calculating safe routes and emotional information, and means for integrating the preprocessed data, analyzing it using a generative AI model, and calculating the situation in the disaster area and an appropriate route to take. This makes it possible to accurately grasp the situation in the disaster area and provide the optimal route in real time. The user can take safe and efficient evacuation actions, and psychological support can also be provided to the disaster victims.

[0556] "Multiple information sources" refers to data sources such as weather information, geographic information, satellite images, drone footage, social network data, traffic information, and information on lifeline facilities.

[0557] "Real-time" refers to the acquisition and processing of data almost simultaneously, and the provision of that information to users without delay.

[0558] "Preprocessing" refers to a process for converting acquired raw data into an analyzable format, and includes processes such as removing noise from the data and improving resolution.

[0559] A "generative AI model" refers to an artificial intelligence model that analyzes collected data and generates new information based on the damage situation and emotional data.

[0560] A "disaster map" is a map that visually displays the current situation and emotional state of the disaster-stricken area based on analyzed data.

[0561] An "emotion engine" refers to an algorithm that analyzes emotional information from text data such as social network data.

[0562] "Automobile navigation system" refers to a system installed in an autonomous vehicle that provides guidance on the vehicle's current location and route to a destination.

[0563] A "safe route" refers to a route that takes into account the disaster situation and emotional information and minimizes damage and danger in the disaster area.

[0564] "Visualization" refers to displaying analysis results and damage maps in a format that is easy to see and understand.

[0565] "User" refers to the people who receive and use information within an autonomous vehicle.

[0566] In this invention, the server collects data from multiple sources in real time, preprocesses the data, and calculates safe routes and emotional information. It also uses a generative AI model to integrate the preprocessed data and calculate the situation in the disaster area and appropriate routes. Next, it creates a disaster map based on the analysis results and updates it in real time. Finally, it visualizes the disaster map and safe routes and provides them to users.

[0567] A specific example is given below.

[0568] Source data collection and preprocessing

[0569] The server connects to multiple sources, including meteorological information, geographical information, satellite images, drone footage, social network data, traffic information, and information on utility facilities, to collect the necessary data in real time. For example, it periodically downloads the latest satellite images from satellite imagery services, and uses an image processing module to improve resolution and remove noise. It also collects posts from social network data streams in real time and uses natural language processing algorithms to extract information related to the disaster.

[0570] Data integration and analysis

[0571] The collected data is integrated and preprocessed on a server. The preprocessed data is then input into a generative AI model to analyze the current situation in the disaster-stricken areas and automatically create a disaster map. An emotion engine is also used to analyze the emotional information in the post text extracted from social network data. For example, from a post tagged with "disaster," the emotion engine analyzes the poster's emotions and identifies emotional states such as fear, relief, and tension.

[0572] Creation and updating of disaster maps

[0573] The generated disaster map is updated in real time by the server and provided to users via a web interface. Users can view the disaster map in real time and quickly grasp the situation and emotional information in the affected area, which enables efficient evacuation instructions and psychological support.

[0574] Examples and prompts

[0575] For example, the server retrieves the latest precipitation data from a weather service's API and downloads satellite imagery from a satellite imagery service at 8:00, 10:00, and 12:00. The processed satellite imagery is analyzed by a generative AI model to identify areas at high risk of flooding and evacuation routes. At the same time, posts extracted from social network data streams are analyzed by an emotion engine, and areas with a high number of posts expressing fear or anxiety are highlighted in red.

[0576] An example of a prompt sentence to input to the generative AI model is as follows:

[0577] Social network posts from near the affected area (disaster):

[0578] 1. I'm terrified. I need help.

[0579] 2. The area is relatively safe.

[0580] Please analyze the data and suggest the best safe route and support points.

[0581] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0582] Step 1:

[0583] The server collects data in real time from multiple sources, such as weather information services, geographic information services, satellite imagery services, drone operation teams, social network data streams, traffic information systems, and lifeline facility management systems. The input is data obtained from various APIs, and the output is the collected raw data.

[0584] Step 2:

[0585] The server preprocesses the collected data, which includes improving the resolution of satellite images, removing noise, and extracting relevant information from social network data. The input for preprocessing is the raw data collected in step 1, and the output is the preprocessed data converted into an analyzable format.

[0586] Step 3:

[0587] The server integrates the preprocessed data. In this step, meteorological data, geographic data, satellite image data, drone footage, social network data, etc. are converted into a unified format to create an integrated dataset. The input is the preprocessed data, and the output is the integrated data.

[0588] Step 4:

[0589] The server inputs the integrated data into a generative AI model to analyze the situation in the disaster-stricken areas. The generative AI model analyzes the damage information and emotional information to create a damage map. In this process, it also uses an emotional engine to analyze the emotional information of posters from social network data. The input is the integrated data, and the output is a damage map and emotional information.

[0590] Step 5:

[0591] The server creates a disaster map based on the analysis results and updates it in real time. Specifically, it visually displays flood risk, evacuation routes, emotional information, etc. based on the output of the generative AI model. The input is the analysis results from the generative AI model, and the output is a disaster map.

[0592] Step 6:

[0593] The server visualizes the disaster map and the safe route and provides it to the user. Specifically, the disaster map is displayed on a web interface or a car navigation system so that the user can check it in real time. The input is the disaster map, and the output is the visualized disaster map.

[0594] Step 7:

[0595] Users can use the visualized disaster map to take appropriate actions, such as choosing a safe evacuation route and paying attention to areas where psychological support is needed. The input is the visualized disaster map, and the output is the user's actions.

[0596] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0597] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0598] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0599] [Third embodiment]

[0600] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0601] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0602] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0603] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0604] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0605] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0606] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0607] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0608] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0609] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0610] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0611] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0612] This invention is a system that collects data in real time from multiple sources, analyzes the damage situation using a generative AI model, and automatically creates a damage map, which will be an important tool for rapid disaster relief efforts.

[0613] The server collects data from various sources, integrates it, and preprocesses it. Specifically, it acquires real-time weather information, geographic information, satellite images, drone footage, social media data streams, demographic statistics, automobile and smartphone movement information, and information on the operation of lifeline facilities. It then preprocesses the collected data and prepares it for analysis using a generative AI model.

[0614] For example, the server periodically downloads satellite images from a satellite imagery service and uses an image processing module to enhance resolution and remove noise.The server also collects posts from social media data streams in real time and uses natural language processing algorithms to extract important information related to the disaster.

[0615] The server integrates the preprocessed data and inputs it into a generative AI model. The generative AI model analyzes the input data and generates detailed information such as the flooding status of the affected area, the damage status of buildings, and evacuation routes. For example, areas at high risk of flooding are displayed in red, and evacuation routes are displayed in green.

[0616] Once the damage map is generated, the server uploads the information to a web interface for visual display. Users and devices can access the web portal and check the damage situation in real time. In addition, the server quickly notifies government and local government officials and disaster relief teams of the damage map and related information to support appropriate rescue operations.

[0617] As a specific example, the server obtains the latest precipitation data from an API provided by the Japan Meteorological Agency and downloads satellite images at 8:00, 10:00, and 12:00 from a satellite imagery service. These images are then processed by an image processing module to remove noise and improve resolution before being input into the generative AI model. Posts tagged with "disaster" are extracted from the social media data stream, and these are also analyzed by the generative AI model. The generative AI model then creates a disaster area map based on information such as flood risk, damaged buildings, and evacuation routes.

[0618] Users can view this disaster map through a web portal and quickly grasp the situation in the affected areas, which will enable efficient rescue operations and evacuation instructions, saving lives and minimizing damage.

[0619] The processing flow will be explained below.

[0620] Step 1:

[0621] The server connects to multiple information sources, such as meteorological information services, geographic information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems, making it ready to collect the necessary data in real time.

[0622] Step 2:

[0623] The server retrieves the latest weather data from a weather information service, the latest satellite images from a satellite image service, real-time video streams from the drone operation team, and traffic data from a traffic information system, and stores them in temporary storage.

[0624] Step 3:

[0625] The server sends the latest satellite image data to the image processing module, which enhances the resolution of the satellite images and removes noise. The processed images are stored for further analysis.

[0626] Step 4:

[0627] The server collects posts from social media data streams in real time and uses natural language processing algorithms to extract information related to the disaster. For example, it filters posts tagged with "disaster" and analyzes them using a text analysis module to extract location information and details of the disaster situation.

[0628] Step 5:

[0629] The server captures the real-time stream of drone footage frame by frame, tags the parts it deems important, and pre-processes them, for example analyzing the condition of a building using object recognition algorithms to identify damage to the building.

[0630] Step 6:

[0631] The server consolidates all collected data and converts it into one unified format, where all data is linked, including weather data, visual data, traffic data, social media data, etc.

[0632] Step 7:

[0633] The server inputs the integrated data into a generative AI model, which analyzes the flood risk, building damage risk, traffic congestion status, and other factors in the affected area to calculate the risk level for each area.

[0634] Step 8:

[0635] The server then creates a disaster map based on the analysis results obtained from the generative AI model. For example, areas at high risk of flooding are displayed in red, while areas that are not flooded are displayed in green. Information on evacuation routes and first-aid stations is also added to each area.

[0636] Step 9:

[0637] The server uploads the created disaster map to a web interface for access by users and devices. Users can view the disaster map in real time through the web portal and check the information needed to take appropriate action.

[0638] Step 10:

[0639] The server will quickly send disaster maps and related information to government and local government officials and disaster relief teams, enabling rescue efforts to be carried out quickly and effectively, saving lives and minimizing damage.

[0640] Example 1

[0641] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0642] Conventional disaster response systems have difficulty collecting data from multiple sources, analyzing it quickly, and creating detailed maps of affected areas. They also lack the ability to update data in real time or preprocess data in a variety of formats. This hinders the speed and efficiency of disaster response, creating challenges in saving lives and minimizing damage.

[0643] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0644] In this invention, the server includes a means for collecting data from multiple sources in real time, a means for preprocessing the collected data, and a means for integrating the preprocessed data and analyzing it using a generative artificial intelligence model. This enables noise removal and extraction of important information from the collected visual and text data. Flood risk and evacuation routes are then identified based on the preprocessed data, and a disaster situation map is created based on the results and uploaded to a web interface. This allows users to grasp the latest damage situation in real time, enabling rapid and efficient disaster response. Furthermore, information can be quickly provided to government and local government officials and disaster relief teams, enabling appropriate responses.

[0645] "Multiple sources" refers to multiple data providers providing different types of data, such as meteorological data providers, satellite imagery providers, aircraft footage providers, digital social media platforms, and utility information providers.

[0646] "Real-time data collection means" refers to technologies or methods for obtaining data immediately from each source, including obtaining data via an API or by monitoring data streams in real time.

[0647] "Preprocessing means" refers to a series of processing techniques used to convert collected data into a format suitable for generative AI models, including processes such as noise removal, resolution enhancement, and key information extraction.

[0648] A "generative artificial intelligence model" is a model that uses a set of algorithms and data analysis techniques to analyze collected data and generate useful information. This includes deep learning models and natural language processing models.

[0649] "Means for analysis" refers to the techniques and methods for analyzing preprocessed data using a generative artificial intelligence model to identify the damage situation and related information.

[0650] "Disaster Situation Map" refers to a map that visually shows the situation in a disaster-stricken area based on the analysis results of a generative AI model, including information on flood risk, building damage, evacuation routes, etc.

[0651] "Visualization means" refers to technologies for displaying analysis results in a format that is easy for users to understand, including technologies for displaying maps on a graphical user interface or web browser.

[0652] "Noise reduction" refers to the process of removing unnecessary information and errors from collected data, improving the quality of the data and increasing the accuracy of analysis results.

[0653] "Important information extraction" refers to the process of extracting only the useful information needed for analysis from collected data. This includes extracting keywords from text data and extracting features from image data.

[0654] "Flood risk" refers to areas that are more likely to experience natural disasters such as floods. A generative AI model identifies these and displays them on the map.

[0655] "Evacuation routes" refer to routes for safe evacuation in the event of a disaster. These are also analyzed by the generative AI model and displayed on the map.

[0656] "Means for uploading to a web interface" refers to the technology used to publish disaster maps and related information on an internet platform and make them accessible to users.

[0657] This invention is a system that collects data in real time from multiple sources, analyzes the damage situation using a generative artificial intelligence model, and automatically creates a disaster situation map, which will be an important tool for carrying out disaster relief activities quickly and efficiently.

[0658] The server collects data in real time from various sources, including weather information, geographic information, satellite imagery, aircraft footage, digital social media data streams, demographic statistics, automobile and smartphone movement information, and utility equipment operation information. This data is collected from different data providers via APIs and data streams.

[0659] The server then preprocesses the collected data. For image data, it uses image processing modules (e.g., OpenCV) to improve resolution and remove noise. For text data, it uses natural language processing algorithms (e.g., BERT from the Transformers library) to extract important information. This improves the quality of the data and increases the accuracy of the subsequent analysis process.

[0660] The preprocessed data is integrated by a server and fed into a generative artificial intelligence model. This model uses deep learning techniques to analyze the collected visual and text data to identify flood risk, building damage, evacuation routes, and other information. For example, using satellite imagery and meteorological data as input, the generative AI model identifies areas at high risk of flooding. It also uses text data from aerial footage and digital social media data streams to identify building damage.

[0661] The generated analysis results are compiled into a disaster situation map by the server, which is visually displayed using a visualization tool (e.g., D3.js) and provided to users. The server then uploads the disaster situation map to a web interface (e.g., a React app) so that users can access it in real time.

[0662] Furthermore, the server will quickly send disaster situation maps and related information to government and local government officials and disaster relief teams, allowing relevant organizations to grasp the latest damage situation and take appropriate action.

[0663] For example, the server obtains the latest precipitation data from a weather data provider and periodically retrieves satellite images from a satellite image provider. These images are denoised and enhanced in resolution using OpenCV before being input into a generative AI model. It also collects posts tagged with "disaster" from digital social media data streams and extracts key information using the BERT model. Based on the collected data, the generative AI model analyzes information such as flood risk, damaged buildings, and evacuation routes to create a disaster situation map.

[0664] An example of a prompt is:

[0665] Identify areas at high risk of flooding based on the latest satellite imagery and weather data, and provide evacuation routes based on information extracted from the text of digital social media posts.

[0666] In this way, the system uses data collected in real time to enable fast and efficient disaster response.

[0667] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0668] Step 1: Data collection

[0669] The server collects data in real time from various sources. Specifically, it uses APIs to obtain the latest precipitation, temperature, wind speed, etc. from weather information services, and periodically downloads satellite images from satellite image services. It also collects real-time video feeds from aircraft, posts tagged with "disaster" from digital social media platforms, and the operational status of lifeline facilities. The inputs are API calls and data streams from various sources. The output is raw data.

[0670] Step 2: Preprocessing the data

[0671] The server preprocesses the collected data. For image data, OpenCV is used to increase the resolution and remove noise. For example, the resolution of satellite images is doubled and noise is removed. For text data, a natural language processing algorithm (e.g., BERT from the Transformers library) is used to extract important information related to the disaster. For example, keywords such as "flood" and "evacuation" are extracted from posts tagged with "disaster." The input is the collected raw data, and the output is the preprocessed data.

[0672] Step 3: Integrate the data

[0673] The server integrates the preprocessed data. It integrates data from different sources based on time and geographic information to create a consistent dataset. The input is the preprocessed data, and the output is the integrated dataset. For example, it performs processing such as matching the posting time of social media data with the shooting time of satellite images.

[0674] Step 4: Analysis by generative AI model

[0675] The server inputs the integrated data into a generative AI model to analyze the damage situation. Based on the input data, areas at high risk of flooding, evacuation routes, and the extent of building damage are identified. For example, areas at high risk of flooding are shown in red, and evacuation routes in green. The input is the integrated data set, and the output is the analysis results. An analytical algorithm using deep learning is applied to the generative AI model.

[0676] Step 5: Generate a damage map

[0677] The server generates a disaster map based on the analysis results of the generative AI model. The analysis results are visualized using a visualization tool (e.g., D3.js) and displayed in a format that is easy for users to understand. The analysis results are input, and a visualized disaster map is obtained as output. For example, a map can be created that color-codes flood risk and evacuation routes.

[0678] Step 6: Upload to the web interface

[0679] The server uploads the generated disaster map to a web interface (e.g., a React app) so that users can access it in real time. As input, we have a visualized disaster map, and as output, we have a web page that users can view. For example, the server publishes the latest disaster map on a website.

[0680] Step 7: Sending notifications

[0681] The server quickly notifies government and local government officials and disaster relief teams of the analysis results and damage maps. The inputs are the visualized damage maps and analysis results, and the output is notifications sent via email, SMS, or a dedicated app. For example, the server could send the latest damage map to relevant organizations via email.

[0682] (Application example 1)

[0683] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0684] Conventional disaster area information collection systems have difficulty in collecting and analyzing information in real time, making it difficult to carry out prompt rescue operations in disaster-stricken areas and provide safe evacuation routes. Furthermore, conventional navigation systems are unable to provide routes that take into account the conditions in disaster-stricken areas, making it difficult to guarantee the safe movement of autonomous vehicles in disaster-stricken areas.

[0685] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0686] In this invention, the server includes means for collecting data in real time from multiple information sources, means for preprocessing the collected data, means for integrating the preprocessed data and analyzing it using a generative AI model, means for creating a damage map based on the analysis results, means for visualizing the damage map and providing it to users, and means for linking the information on the damage map to a navigation system installed in an autonomous vehicle. This makes it possible to grasp the real-time situation in the disaster area and provide safe and efficient routes.

[0687] "Multiple sources" refers to multiple types of information sources, including weather information, geographic information, satellite images, aerial footage, social networking service data streams, demographic statistics, automobile and smartphone movement information, and information on the operation of lifeline facilities.

[0688] "Collection means" refers to a means for acquiring data from multiple information sources in real time and collecting it on a server.

[0689] "Preprocessing means" refers to means for performing processes such as noise removal, resolution enhancement, and text filtering in order to convert collected data into a format or state suitable for analysis.

[0690] The "integration means" is a means for collecting preprocessed data from multiple information sources and treating them as a single analysis target.

[0691] A "generative AI model" is an artificial intelligence model that analyzes input data and generates disaster information such as flooding status, building damage status, and evacuation routes.

[0692] The "disaster map generation means" is a means of creating a disaster map that visually displays flooding conditions, building damage conditions, evacuation routes, etc. based on the analysis results obtained by the generative AI model.

[0693] The "visualization means" refers to a means including a web interface or application for displaying the generated disaster map in a format that can be viewed by a user.

[0694] "User provision means" refers to means including a web portal and notification system that allow users and relevant organizations to access the visualized disaster map.

[0695] An "autonomous vehicle" is a vehicle that moves on its own using sensors and AI without the need for human operation.

[0696] The "navigation system" is a system that is installed in self-driving vehicles and provides safe and efficient travel routes in real time based on disaster maps.

[0697] This invention is a system that collects data in real time from multiple sources, analyzes the damage situation using a generative AI model, and automatically creates a damage map. This system is an important tool for rapid disaster relief efforts, and in particular, when linked to the navigation systems installed in autonomous vehicles, it enables safe movement in disaster-stricken areas.

[0698] The server first collects real-time data from multiple sources, including meteorological information, geographical information, satellite images, aerial footage, social networking service data streams, demographic statistics, automobile and smartphone movement information, and utility facility operation information. After this data is collected on the server, it undergoes pre-processing.

[0699] As preprocessing means, the server performs image processing such as noise removal and resolution enhancement, and text data filtering. Specifically, for satellite images obtained from a satellite imagery service, resolution enhancement and noise removal are performed, and for posts from the social networking service Data Stream, natural language processing algorithms are used to extract important information related to the disaster.

[0700] The preprocessed data is integrated and input into a generative AI model, which uses the input data to generate detailed damage information, such as flooding status, building damage status, and evacuation routes. The results of this analysis are visualized as a damage map.

[0701] The visualized disaster map is provided to users. Specifically, the server publishes the disaster map through a web interface, allowing users to check the damage situation in real time. The information from this disaster map is also linked to the navigation system installed in the autonomous vehicle. The navigation system then provides safe and efficient travel routes in real time based on the disaster map.

[0702] As a practical example, by inputting the following prompts into a generative AI model, a highly accurate disaster map can be created:

[0703] Prompt: Based on the real-time information below, what is the best evacuation route?

[0704] Precipitation: 20mm / h

[0705] Wind speed: 15km / h

[0706] Latest SNS information: Disasters, Floods, Evacuation

[0707] This will enable navigation that always reflects the latest disaster situation based on data updated in real time, ensuring safe travel.

[0708] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0709] Step 1:

[0710] The server collects data in real time from multiple sources, including meteorological information, geographical information, satellite images, aerial footage, social networking service data streams, demographic statistics, automobile and smartphone movement information, and information on the operation of lifeline facilities.

[0711] Input: Real-time data from multiple sources

[0712] Output: Raw data collected

[0713] Specific behavior:

[0714] The server retrieves weather data using WeatherAPI, periodically downloads images from a satellite imagery service, and collects posts tagged with "disaster" from social networking service data streams.

[0715] Step 2:

[0716] The server preprocesses the collected data, performing image processing such as noise removal and resolution improvement, and filtering of text data.

[0717] Input: Raw data collected

[0718] Output: Preprocessed data

[0719] Specific behavior:

[0720] That is, the pre-processing means performs noise removal and resolution enhancement on satellite images, and uses natural language processing algorithms to extract important disaster-related information from posts from social networking service data streams.

[0721] Step 3:

[0722] The server integrates the preprocessed data and inputs it into the generative AI model as a single analysis target. Based on this data, the generative AI model generates detailed disaster information such as flooding status, building damage status, and evacuation routes.

[0723] Input: Preprocessed data

[0724] Output: Disaster information data

[0725] Specific behavior:

[0726] The integration method combines image data and text data into a single dataset and inputs it into a generative AI model, which analyzes the dataset and generates important disaster information.

[0727] Step 4:

[0728] The server creates a disaster map based on the generated damage information, visually displaying the flooding situation, building damage, evacuation routes, and other information.

[0729] Input: Disaster information data

[0730] Output: Damage map

[0731] Specific behavior:

[0732] The disaster map generation method visually represents the disaster information obtained by the generative AI model and displays flooded areas and evacuation routes on a map. For example, areas with a high flood risk are shown in red, and safe evacuation routes are shown in green.

[0733] Step 5:

[0734] The server publishes the visualized disaster map through a web interface to provide users with real-time information on the disaster situation.

[0735] Input: Disaster map

[0736] Output: A damage map on a user-accessible web interface

[0737] Specific behavior:

[0738] The provision means uploads the generated disaster map to a web interface so that users can access it through a browser. It also supports a disaster information notification function as needed.

[0739] Step 6:

[0740] The device then links the disaster map information to the autonomous vehicle's navigation system, which then provides safe and efficient travel routes in real time based on the disaster map.

[0741] Input: Disaster map on web interface

[0742] Output: Navigation route for autonomous vehicles

[0743] Specific behavior:

[0744] The navigation system receives disaster map data and calculates a safe route based on the analysis results. The autonomous vehicle then navigates safely through the disaster area, enabling rescue operations to be carried out quickly and efficiently.

[0745] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0746] This invention combines a system that collects data in real time from multiple sources, analyzes the damage situation using a generative AI model, and automatically creates a damage map, with an emotion engine that recognizes the user's emotions. This system is an important tool for swift disaster relief activities, and can also provide psychological support to users.

[0747] The server connects to multiple information sources, such as weather information services, geographic information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems, to collect the necessary data in real time. The collected data is integrated and preprocessed. For example, weather data, satellite imagery, drone footage, and social media post data are temporarily stored and then subjected to appropriate preprocessing before various analyses.

[0748] Specifically, the server periodically downloads the latest satellite images from a satellite imagery service, and uses an image processing module to improve resolution and remove noise. Social media posts are collected in real time from the data stream, and natural language processing algorithms may be used to extract information related to the disaster.

[0749] The server integrates the collected data and inputs it into a generative AI model. The generative AI model analyzes the current situation in the affected areas and creates a damage map. It also uses an emotion engine to analyze the emotional information in the post text extracted from the social media data stream. For example, from a post tagged with "disaster," the emotion engine analyzes the poster's emotions and identifies emotional states such as fear, relief, and tension.

[0750] The emotional information analyzed by the emotion engine is reflected on a disaster map. For example, areas with heightened fear and anxiety can be color-coded, and this information can be used to provide appropriate psychological support to government and local government officials.

[0751] The generated disaster map is uploaded to a web interface by the server and can be accessed by users and their devices. Users can check the damage situation and emotional information in real time through the web portal. This not only provides users with information to take appropriate actions, but also helps them understand where psychological support is needed.

[0752] For example, the server retrieves the latest precipitation data from an API provided by the Japan Meteorological Agency and downloads satellite imagery from a satellite imagery service at 8:00, 10:00, and 12:00. The processed satellite imagery is analyzed by a generative AI model to identify areas at high risk of flooding and evacuation routes. At the same time, posts extracted from social media data streams are analyzed by an emotion engine, and areas with a large number of posts expressing fear or anxiety are highlighted in red.

[0753] Users can view this disaster map on a web portal and quickly grasp the situation and emotional information of the affected areas, which will enable efficient rescue operations and evacuation instructions, not only saving lives and minimizing damage, but also providing psychological support to victims.

[0754] The processing flow will be explained below.

[0755] Step 1:

[0756] The server connects to multiple information sources, such as weather information services, geographic information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems, thereby preparing to collect the necessary data in real time.

[0757] Step 2:

[0758] The server retrieves current weather data from a weather information service, the latest satellite images from a satellite imagery service, real-time video streams from the drone operation team, and traffic data from the traffic information system, and stores them in temporary storage.

[0759] Step 3:

[0760] The server sends the acquired satellite image data to the image processing module, which processes it to improve resolution and remove noise, and stores the processed image data for analysis.

[0761] Step 4:

[0762] The server collects posts from social media data streams in real time and uses natural language processing algorithms to extract disaster-related information. For example, it filters posts tagged with "disaster" and analyzes location information and emotional states using a text analysis module.

[0763] Step 5:

[0764] The server uses an emotion engine to analyze users' emotional information from the filtered SNS posts. For example, it identifies emotional states such as fear, anxiety, and relief, and adds this information to the text data.

[0765] Step 6:

[0766] The server captures the real-time stream of drone footage frame by frame, tags the parts it deems important, and pre-processes them, for example analyzing the condition of a building using object recognition algorithms to identify damage to the building.

[0767] Step 7:

[0768] The server integrates pre-processed weather data, visual data, traffic data, social media data, etc., and converts them into a unified format, which prepares the input data for the generative AI model.

[0769] Step 8:

[0770] The server inputs the integrated data into a generative AI model for analysis, which analyzes the risk of flooding in the affected area, the risk of building damage, traffic congestion, etc., and calculates the risk level for each.

[0771] Step 9:

[0772] Based on the analysis results obtained from the generative AI model, the server creates a disaster map that shows color-coded areas at high risk of flooding, areas that are not flooded, evacuation routes, and areas at high emotional risk that require psychological support.

[0773] Step 10:

[0774] The server uploads the created disaster map to a web interface, making it accessible to users and devices. Users can check the damage situation and emotional information in real time through the web portal and obtain information to take appropriate actions.

[0775] Step 11:

[0776] The server will quickly send disaster maps and related information to government and local government officials and disaster relief teams, enabling them to carry out relief efforts quickly and effectively, and also to provide psychological support to victims.

[0777] Example 2

[0778] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0779] Conventional disaster response systems have difficulty collecting and analyzing data from multiple sources in real time to grasp the damage situation. It is also extremely difficult to accurately visualize the current situation in the disaster area and to reflect the emotional state of the victims. Therefore, while efficient and rapid disaster response is required, current systems are unable to meet this demand.

[0780] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting data in real time from multiple information sources, means for preprocessing the collected data, means for integrating the preprocessed data and analyzing it using a generative AI model, means for analyzing emotional information in text data using an emotion engine, and means for reflecting the analyzed emotional information in a damage map. This makes it possible not only to accurately grasp and visualize the damage situation, but also to create a damage map that reflects the emotional state of the victims.

[0781] "Multiple sources" are data sources that provide different types of information, including weather information services, geographic information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems.

[0782] "Real-time data collection means" is a combination of APIs, sensors, and data streams that obtain up-to-date data from various sources via the internet.

[0783] "Means for data preprocessing" refers to techniques that perform processes such as data cleansing, noise removal, resolution improvement, and format conversion in order to prepare collected data in a form suitable for analysis.

[0784] Data synthesis is the process of bringing together data collected from different sources over time into a single integrated data set.

[0785] "Means of analysis using generative AI models" refer to algorithms and models that utilize machine learning and artificial intelligence technology to analyze the damage situation based on integrated data.

[0786] The "emotion engine" is a system that uses natural language processing technology to automatically analyze the poster's emotions and psychological state from text data.

[0787] "Means for analyzing emotional information from text data" refers to the process of using an emotion engine to extract the emotions of posters from text data such as social media posts and quantifying them.

[0788] "Means for reflecting analyzed emotional information on a disaster map" refers to a technique that uses color coding and icons to visually show analyzed emotional information on a disaster map.

[0789] The "means for visualizing a disaster map and providing it to users" is a system that generates an intuitively easy-to-understand map based on the analyzed disaster situation and emotional information, and provides this to users through a web interface or application.

[0790] The present invention is a system that collects data in real time from multiple information sources, preprocesses it, and analyzes it using a generative AI model. It also uses an emotion engine to analyze emotional information from text data and reflects the results in a disaster map, providing users with information about the damage situation and emotions. The following describes in detail an embodiment of this system.

[0791] Data collection

[0792] The server collects data in real time from multiple sources, including meteorological information services, geographical information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems, making it possible to grasp the damage situation from multiple perspectives.

[0793] For example, the server can use the API of the Japan Meteorological Agency to obtain the latest precipitation data, or it can periodically download satellite images from a satellite imagery service.

[0794] Data Preprocessing

[0795] The server pre-processes the collected data, which includes data cleansing, noise removal, resolution enhancement, format conversion, etc. Specifically, it uses a specific image processing module to enhance the resolution and remove noise from the satellite image data.

[0796] In addition, natural language processing algorithms are used to extract information related to the disaster from social media data.

[0797] Data integration

[0798] The server then integrates the pre-processed data, collating data from different sources into a single integrated dataset along a timeline, thereby building a database that can provide a comprehensive understanding of the disaster situation.

[0799] Analysis of the damage situation

[0800] The server inputs the integrated data into a generative AI model, which analyzes the damage situation in real time. The generative AI model uses machine learning and artificial intelligence technology to analyze the current situation in the disaster area based on various data.

[0801] Emotional information analysis

[0802] The server uses an emotion engine to analyze the emotional information of posts extracted from SNS data. The emotional information obtained through this analysis is used to visualize the psychological state of people in the disaster-stricken areas.

[0803] Creating a disaster map

[0804] The server integrates the damage situation and emotional information to create a damage map. The damage map is visually easy to understand and is designed to allow users to quickly grasp the current situation in the affected areas. Areas at high risk of flooding are displayed in blue, while areas with rising feelings of fear and anxiety are displayed in red.

[0805] Upload to the web interface

[0806] The server uploads the generated damage map to a web interface, allowing users and devices to check the damage situation and emotional information in real time.

[0807] User Use

[0808] Users can view the disaster map through a web portal and check the damage situation and sentiment information in real time, which provides information for taking appropriate actions.

[0809] Examples of concrete examples and prompts

[0810] Examples:

[0811] 1. The server obtains the latest precipitation data using the Japan Meteorological Agency's API.

[0812] 2. The server downloads satellite images at 8:00, 10:00, and 12:00 from the satellite imagery service.

[0813] 3. The server uses the generative AI model to analyze the processed satellite imagery and identify areas at high risk of flooding.

[0814] 4. The server uses an emotion engine to analyze posts extracted from the social media data stream and displays areas with a large number of posts expressing fear or anxiety in red.

[0815] 5. Users can view the damage map on the web portal and obtain information to take appropriate actions.

[0816] Example prompt sentence:

[0817] "Identify areas at high risk of flooding based on the latest rainfall data and satellite imagery."

[0818] "Please extract emotional information from social media data and reflect it in the disaster map."

[0819] "View detailed sentiment information for areas highlighted in red"

[0820] This will provide a concrete example of how to put the invention into practice, making it possible to speed up disaster relief efforts, provide accurate information, and provide psychological support to disaster victims.

[0821] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0822] Step 1:

[0823] The server collects data in real time from multiple sources, such as meteorological information services, geographic information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems. The input is the data retrieved by sending requests from these sources, and the output is the temporarily stored, unprocessed data.

[0824] Step 2:

[0825] The server preprocesses the collected data. Preprocessing includes data cleansing, noise removal, resolution enhancement, and format conversion. For example, it performs resolution enhancement on satellite image data and performs natural language processing on social media data to extract relevant information. The input is unprocessed data, and the output is processed data.

[0826] Step 3:

[0827] The server integrates the preprocessed data. It combines data collected from different sources into a single integrated dataset along a time axis. For example, it integrates weather data, satellite image data, and social media data into a single dataset. The input is preprocessed data, and the output is the integrated dataset.

[0828] Step 4:

[0829] The server inputs the integrated data into a generative AI model, which analyzes the damage situation in real time. The generative AI model uses machine learning and artificial intelligence technology to analyze the current situation in the disaster-stricken area based on various data. The input is the integrated dataset, and the output is the analysis result of the damage situation. For example, it can identify areas with a high risk of flooding.

[0830] Step 5:

[0831] The server uses an emotion engine to analyze the emotional information of posts extracted from SNS data. The emotion engine uses natural language processing technology to analyze emotional information from text data. The input is the text data extracted from SNS, and the output is the analysis result of the emotional information. For example, emotions indicating fear or anxiety are extracted.

[0832] Step 6:

[0833] The server integrates the damage situation and emotional information to create a damage map. The damage map visually reflects areas at high risk of flooding and emotional information. The input is the analysis results of the damage situation and emotional information, and the output is a damage map. For example, areas at high risk of flooding are displayed in blue, and heightened emotions in red.

[0834] Step 7:

[0835] The server uploads the generated disaster map to a web interface, allowing users and devices to check the disaster situation and emotional information in real time. The input is the disaster map, and the output is the disaster map displayed on the web portal.

[0836] Step 8:

[0837] Users can view the disaster map through the web portal and check the damage situation and emotional information, which can provide information to take appropriate actions. For example, users can check the disaster map and find a safe evacuation route.

[0838] (Application example 2)

[0839] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0840] Current navigation systems for autonomous vehicles lack the functionality to support a rapid and safe response in disaster-stricken areas. The lack of a means to analyze and provide information on the disaster situation and emotions in real time makes it difficult to select safe routes or provide psychological support in emergencies. Furthermore, there is insufficient integration of information sources that can accurately grasp the situation in disaster-stricken areas, and the development of a system that encourages appropriate action is required.

[0841] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting data from multiple information sources in real time and linking with the vehicle's navigation system, means for preprocessing the collected data and calculating safe routes and emotional information, and means for integrating the preprocessed data, analyzing it using a generative AI model, and calculating the situation in the disaster area and an appropriate route to take. This makes it possible to accurately grasp the situation in the disaster area and provide the optimal route in real time. The user can take safe and efficient evacuation actions, and psychological support can also be provided to the disaster victims.

[0842] "Multiple information sources" refers to data sources such as weather information, geographic information, satellite images, drone footage, social network data, traffic information, and information on lifeline facilities.

[0843] "Real-time" refers to the acquisition and processing of data almost simultaneously, and the provision of that information to users without delay.

[0844] "Preprocessing" refers to a process for converting acquired raw data into an analyzable format, and includes processes such as removing noise from the data and improving resolution.

[0845] A "generative AI model" refers to an artificial intelligence model that analyzes collected data and generates new information based on the damage situation and emotional data.

[0846] A "disaster map" is a map that visually displays the current situation and emotional state of the disaster-stricken area based on analyzed data.

[0847] An "emotion engine" refers to an algorithm that analyzes emotional information from text data such as social network data.

[0848] "Automobile navigation system" refers to a system installed in an autonomous vehicle that provides guidance on the vehicle's current location and route to a destination.

[0849] A "safe route" refers to a route that takes into account the disaster situation and emotional information and minimizes damage and danger in the disaster area.

[0850] "Visualization" refers to displaying analysis results and damage maps in a format that is easy to see and understand.

[0851] "User" refers to the people who receive and use information within an autonomous vehicle.

[0852] In this invention, the server collects data from multiple sources in real time, preprocesses the data, and calculates safe routes and emotional information. It also uses a generative AI model to integrate the preprocessed data and calculate the situation in the disaster area and appropriate routes. Next, it creates a disaster map based on the analysis results and updates it in real time. Finally, it visualizes the disaster map and safe routes and provides them to users.

[0853] A specific example is given below.

[0854] Source data collection and preprocessing

[0855] The server connects to multiple sources, including meteorological information, geographical information, satellite images, drone footage, social network data, traffic information, and information on utility facilities, to collect the necessary data in real time. For example, it periodically downloads the latest satellite images from satellite imagery services, and uses an image processing module to improve resolution and remove noise. It also collects posts from social network data streams in real time and uses natural language processing algorithms to extract information related to the disaster.

[0856] Data integration and analysis

[0857] The collected data is integrated and preprocessed on a server. The preprocessed data is then input into a generative AI model to analyze the current situation in the disaster-stricken areas and automatically create a disaster map. An emotion engine is also used to analyze the emotional information in the post text extracted from social network data. For example, from a post tagged with "disaster," the emotion engine analyzes the poster's emotions and identifies emotional states such as fear, relief, and tension.

[0858] Creation and updating of disaster maps

[0859] The generated disaster map is updated in real time by the server and provided to users via a web interface. Users can view the disaster map in real time and quickly grasp the situation and emotional information in the affected area, which enables efficient evacuation instructions and psychological support.

[0860] Examples and prompts

[0861] For example, the server retrieves the latest precipitation data from a weather service's API and downloads satellite imagery from a satellite imagery service at 8:00, 10:00, and 12:00. The processed satellite imagery is analyzed by a generative AI model to identify areas at high risk of flooding and evacuation routes. At the same time, posts extracted from social network data streams are analyzed by an emotion engine, and areas with a high number of posts expressing fear or anxiety are highlighted in red.

[0862] An example of a prompt sentence to input to the generative AI model is as follows:

[0863] Social network posts from near the affected area (disaster):

[0864] 1. I'm terrified. I need help.

[0865] 2. The area is relatively safe.

[0866] Please analyze the data and suggest the best safe route and support points.

[0867] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0868] Step 1:

[0869] The server collects data in real time from multiple sources, such as weather information services, geographic information services, satellite imagery services, drone operation teams, social network data streams, traffic information systems, and lifeline facility management systems. The input is data obtained from various APIs, and the output is the collected raw data.

[0870] Step 2:

[0871] The server preprocesses the collected data, which includes improving the resolution of satellite images, removing noise, and extracting relevant information from social network data. The input for preprocessing is the raw data collected in step 1, and the output is the preprocessed data converted into an analyzable format.

[0872] Step 3:

[0873] The server integrates the preprocessed data. In this step, meteorological data, geographic data, satellite image data, drone footage, social network data, etc. are converted into a unified format to create an integrated dataset. The input is the preprocessed data, and the output is the integrated data.

[0874] Step 4:

[0875] The server inputs the integrated data into a generative AI model to analyze the situation in the disaster-stricken areas. The generative AI model analyzes the damage information and emotional information to create a damage map. In this process, it also uses an emotional engine to analyze the emotional information of posters from social network data. The input is the integrated data, and the output is a damage map and emotional information.

[0876] Step 5:

[0877] The server creates a disaster map based on the analysis results and updates it in real time. Specifically, it visually displays flood risk, evacuation routes, emotional information, etc. based on the output of the generative AI model. The input is the analysis results from the generative AI model, and the output is a disaster map.

[0878] Step 6:

[0879] The server visualizes the disaster map and the safe route and provides it to the user. Specifically, the disaster map is displayed on a web interface or a car navigation system so that the user can check it in real time. The input is the disaster map, and the output is the visualized disaster map.

[0880] Step 7:

[0881] Users can use the visualized disaster map to take appropriate actions, such as choosing a safe evacuation route and paying attention to areas where psychological support is needed. The input is the visualized disaster map, and the output is the user's actions.

[0882] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0883] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0884] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[0885] [Fourth embodiment]

[0886] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0887] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0888] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0889] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0890] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0891] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0892] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0893] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0894] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0895] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0896] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0897] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0898] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0899] This invention is a system that collects data in real time from multiple sources, analyzes the damage situation using a generative AI model, and automatically creates a damage map, which will be an important tool for rapid disaster relief efforts.

[0900] The server collects data from various sources, integrates it, and preprocesses it. Specifically, it acquires real-time weather information, geographic information, satellite images, drone footage, social media data streams, demographic statistics, automobile and smartphone movement information, and information on the operation of lifeline facilities. It then preprocesses the collected data and prepares it for analysis using a generative AI model.

[0901] For example, the server periodically downloads satellite images from a satellite imagery service and uses an image processing module to enhance resolution and remove noise.The server also collects posts from social media data streams in real time and uses natural language processing algorithms to extract important information related to the disaster.

[0902] The server integrates the preprocessed data and inputs it into a generative AI model. The generative AI model analyzes the input data and generates detailed information such as the flooding status of the affected area, the damage status of buildings, and evacuation routes. For example, areas at high risk of flooding are displayed in red, and evacuation routes are displayed in green.

[0903] Once the damage map is generated, the server uploads the information to a web interface for visual display. Users and devices can access the web portal and check the damage situation in real time. In addition, the server quickly notifies government and local government officials and disaster relief teams of the damage map and related information to support appropriate rescue operations.

[0904] As a specific example, the server obtains the latest precipitation data from an API provided by the Japan Meteorological Agency and downloads satellite images at 8:00, 10:00, and 12:00 from a satellite imagery service. These images are then processed by an image processing module to remove noise and improve resolution before being input into the generative AI model. Posts tagged with "disaster" are extracted from the social media data stream, and these are also analyzed by the generative AI model. The generative AI model then creates a disaster area map based on information such as flood risk, damaged buildings, and evacuation routes.

[0905] Users can view this disaster map through a web portal and quickly grasp the situation in the affected areas, which will enable efficient rescue operations and evacuation instructions, saving lives and minimizing damage.

[0906] The processing flow will be explained below.

[0907] Step 1:

[0908] The server connects to multiple information sources, such as meteorological information services, geographic information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems, making it ready to collect the necessary data in real time.

[0909] Step 2:

[0910] The server retrieves the latest weather data from a weather information service, the latest satellite images from a satellite image service, real-time video streams from the drone operation team, and traffic data from a traffic information system, and stores them in temporary storage.

[0911] Step 3:

[0912] The server sends the latest satellite image data to the image processing module, which enhances the resolution of the satellite images and removes noise. The processed images are stored for further analysis.

[0913] Step 4:

[0914] The server collects posts from social media data streams in real time and uses natural language processing algorithms to extract information related to the disaster. For example, it filters posts tagged with "disaster" and analyzes them using a text analysis module to extract location information and details of the disaster situation.

[0915] Step 5:

[0916] The server captures the real-time stream of drone footage frame by frame, tags the parts it deems important, and pre-processes them, for example analyzing the condition of a building using object recognition algorithms to identify damage to the building.

[0917] Step 6:

[0918] The server consolidates all collected data and converts it into one unified format, where all data is linked, including weather data, visual data, traffic data, social media data, etc.

[0919] Step 7:

[0920] The server inputs the integrated data into a generative AI model, which analyzes the flood risk, building damage risk, traffic congestion status, and other factors in the affected area to calculate the risk level for each area.

[0921] Step 8:

[0922] The server then creates a disaster map based on the analysis results obtained from the generative AI model. For example, areas at high risk of flooding are displayed in red, while areas that are not flooded are displayed in green. Information on evacuation routes and first-aid stations is also added to each area.

[0923] Step 9:

[0924] The server uploads the created disaster map to a web interface for access by users and devices. Users can view the disaster map in real time through the web portal and check the information needed to take appropriate action.

[0925] Step 10:

[0926] The server will quickly send disaster maps and related information to government and local government officials and disaster relief teams, enabling rescue efforts to be carried out quickly and effectively, saving lives and minimizing damage.

[0927] Example 1

[0928] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0929] Conventional disaster response systems have difficulty collecting data from multiple sources, analyzing it quickly, and creating detailed maps of affected areas. They also lack the ability to update data in real time or preprocess data in a variety of formats. This hinders the speed and efficiency of disaster response, creating challenges in saving lives and minimizing damage.

[0930] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0931] In this invention, the server includes a means for collecting data from multiple sources in real time, a means for preprocessing the collected data, and a means for integrating the preprocessed data and analyzing it using a generative artificial intelligence model. This enables noise removal and extraction of important information from the collected visual and text data. Flood risk and evacuation routes are then identified based on the preprocessed data, and a disaster situation map is created based on the results and uploaded to a web interface. This allows users to grasp the latest damage situation in real time, enabling rapid and efficient disaster response. Furthermore, information can be quickly provided to government and local government officials and disaster relief teams, enabling appropriate responses.

[0932] "Multiple sources" refers to multiple data providers providing different types of data, such as meteorological data providers, satellite imagery providers, aircraft footage providers, digital social media platforms, and utility information providers.

[0933] "Real-time data collection means" refers to technologies or methods for obtaining data immediately from each source, including obtaining data via an API or by monitoring data streams in real time.

[0934] "Preprocessing means" refers to a series of processing techniques used to convert collected data into a format suitable for generative AI models, including processes such as noise removal, resolution enhancement, and key information extraction.

[0935] A "generative artificial intelligence model" is a model that uses a set of algorithms and data analysis techniques to analyze collected data and generate useful information. This includes deep learning models and natural language processing models.

[0936] "Means for analysis" refers to the techniques and methods for analyzing preprocessed data using a generative artificial intelligence model to identify the damage situation and related information.

[0937] "Disaster Situation Map" refers to a map that visually shows the situation in a disaster-stricken area based on the analysis results of a generative AI model, including information on flood risk, building damage, evacuation routes, etc.

[0938] "Visualization means" refers to technologies for displaying analysis results in a format that is easy for users to understand, including technologies for displaying maps on a graphical user interface or web browser.

[0939] "Noise reduction" refers to the process of removing unnecessary information and errors from collected data, improving the quality of the data and increasing the accuracy of analysis results.

[0940] "Important information extraction" refers to the process of extracting only the useful information needed for analysis from collected data. This includes extracting keywords from text data and extracting features from image data.

[0941] "Flood risk" refers to areas that are more likely to experience natural disasters such as floods. A generative AI model identifies these and displays them on the map.

[0942] "Evacuation routes" refer to routes for safe evacuation in the event of a disaster. These are also analyzed by the generative AI model and displayed on the map.

[0943] "Means for uploading to a web interface" refers to the technology used to publish disaster maps and related information on an internet platform and make them accessible to users.

[0944] This invention is a system that collects data in real time from multiple sources, analyzes the damage situation using a generative artificial intelligence model, and automatically creates a disaster situation map, which will be an important tool for carrying out disaster relief activities quickly and efficiently.

[0945] The server collects data in real time from various sources, including weather information, geographic information, satellite imagery, aircraft footage, digital social media data streams, demographic statistics, automobile and smartphone movement information, and utility equipment operation information. This data is collected from different data providers via APIs and data streams.

[0946] The server then preprocesses the collected data. For image data, it uses image processing modules (e.g., OpenCV) to improve resolution and remove noise. For text data, it uses natural language processing algorithms (e.g., BERT from the Transformers library) to extract important information. This improves the quality of the data and increases the accuracy of the subsequent analysis process.

[0947] The preprocessed data is integrated by a server and fed into a generative artificial intelligence model. This model uses deep learning techniques to analyze the collected visual and text data to identify flood risk, building damage, evacuation routes, and other information. For example, using satellite imagery and meteorological data as input, the generative AI model identifies areas at high risk of flooding. It also uses text data from aerial footage and digital social media data streams to identify building damage.

[0948] The generated analysis results are compiled into a disaster situation map by the server, which is visually displayed using a visualization tool (e.g., D3.js) and provided to users. The server then uploads the disaster situation map to a web interface (e.g., a React app) so that users can access it in real time.

[0949] Furthermore, the server will quickly send disaster situation maps and related information to government and local government officials and disaster relief teams, allowing relevant organizations to grasp the latest damage situation and take appropriate action.

[0950] For example, the server obtains the latest precipitation data from a weather data provider and periodically retrieves satellite images from a satellite image provider. These images are denoised and enhanced in resolution using OpenCV before being input into a generative AI model. It also collects posts tagged with "disaster" from digital social media data streams and extracts key information using the BERT model. Based on the collected data, the generative AI model analyzes information such as flood risk, damaged buildings, and evacuation routes to create a disaster situation map.

[0951] An example of a prompt is:

[0952] Identify areas at high risk of flooding based on the latest satellite imagery and weather data, and provide evacuation routes based on information extracted from the text of digital social media posts.

[0953] In this way, the system uses data collected in real time to enable fast and efficient disaster response.

[0954] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0955] Step 1: Data collection

[0956] The server collects data in real time from various sources. Specifically, it uses APIs to obtain the latest precipitation, temperature, wind speed, etc. from weather information services, and periodically downloads satellite images from satellite image services. It also collects real-time video feeds from aircraft, posts tagged with "disaster" from digital social media platforms, and the operational status of lifeline facilities. The inputs are API calls and data streams from various sources. The output is raw data.

[0957] Step 2: Preprocessing the data

[0958] The server preprocesses the collected data. For image data, OpenCV is used to increase the resolution and remove noise. For example, the resolution of satellite images is doubled and noise is removed. For text data, a natural language processing algorithm (e.g., BERT from the Transformers library) is used to extract important information related to the disaster. For example, keywords such as "flood" and "evacuation" are extracted from posts tagged with "disaster." The input is the collected raw data, and the output is the preprocessed data.

[0959] Step 3: Integrate the data

[0960] The server integrates the preprocessed data. It integrates data from different sources based on time and geographic information to create a consistent dataset. The input is the preprocessed data, and the output is the integrated dataset. For example, it performs processing such as matching the posting time of social media data with the shooting time of satellite images.

[0961] Step 4: Analysis by generative AI model

[0962] The server inputs the integrated data into a generative AI model to analyze the damage situation. Based on the input data, areas at high risk of flooding, evacuation routes, and the extent of building damage are identified. For example, areas at high risk of flooding are shown in red, and evacuation routes in green. The input is the integrated data set, and the output is the analysis results. An analytical algorithm using deep learning is applied to the generative AI model.

[0963] Step 5: Generate a damage map

[0964] The server generates a disaster map based on the analysis results of the generative AI model. The analysis results are visualized using a visualization tool (e.g., D3.js) and displayed in a format that is easy for users to understand. The analysis results are input, and a visualized disaster map is obtained as output. For example, a map can be created that color-codes flood risk and evacuation routes.

[0965] Step 6: Upload to the web interface

[0966] The server uploads the generated disaster map to a web interface (e.g., a React app) so that users can access it in real time. As input, we have a visualized disaster map, and as output, we have a web page that users can view. For example, the server publishes the latest disaster map on a website.

[0967] Step 7: Sending notifications

[0968] The server quickly notifies government and local government officials and disaster relief teams of the analysis results and damage maps. The inputs are the visualized damage maps and analysis results, and the output is notifications sent via email, SMS, or a dedicated app. For example, the server could send the latest damage map to relevant organizations via email.

[0969] (Application example 1)

[0970] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0971] Conventional disaster area information collection systems have difficulty in collecting and analyzing information in real time, making it difficult to carry out prompt rescue operations in disaster-stricken areas and provide safe evacuation routes. Furthermore, conventional navigation systems are unable to provide routes that take into account the conditions in disaster-stricken areas, making it difficult to guarantee the safe movement of autonomous vehicles in disaster-stricken areas.

[0972] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0973] In this invention, the server includes means for collecting data in real time from multiple information sources, means for preprocessing the collected data, means for integrating the preprocessed data and analyzing it using a generative AI model, means for creating a damage map based on the analysis results, means for visualizing the damage map and providing it to users, and means for linking the information on the damage map to a navigation system installed in an autonomous vehicle. This makes it possible to grasp the real-time situation in the disaster area and provide safe and efficient routes.

[0974] "Multiple sources" refers to multiple types of information sources, including weather information, geographic information, satellite images, aerial footage, social networking service data streams, demographic statistics, automobile and smartphone movement information, and information on the operation of lifeline facilities.

[0975] "Collection means" refers to a means for acquiring data from multiple information sources in real time and collecting it on a server.

[0976] "Preprocessing means" refers to means for performing processes such as noise removal, resolution enhancement, and text filtering in order to convert collected data into a format or state suitable for analysis.

[0977] The "integration means" is a means for collecting preprocessed data from multiple information sources and treating them as a single analysis target.

[0978] A "generative AI model" is an artificial intelligence model that analyzes input data and generates disaster information such as flooding status, building damage status, and evacuation routes.

[0979] The "disaster map generation means" is a means of creating a disaster map that visually displays flooding conditions, building damage conditions, evacuation routes, etc. based on the analysis results obtained by the generative AI model.

[0980] The "visualization means" refers to a means including a web interface or application for displaying the generated disaster map in a format that can be viewed by a user.

[0981] "User provision means" refers to means including a web portal and notification system that allow users and relevant organizations to access the visualized disaster map.

[0982] An "autonomous vehicle" is a vehicle that moves on its own using sensors and AI without the need for human operation.

[0983] The "navigation system" is a system that is installed in self-driving vehicles and provides safe and efficient travel routes in real time based on disaster maps.

[0984] This invention is a system that collects data in real time from multiple sources, analyzes the damage situation using a generative AI model, and automatically creates a damage map. This system is an important tool for rapid disaster relief efforts, and in particular, when linked to the navigation systems installed in autonomous vehicles, it enables safe movement in disaster-stricken areas.

[0985] The server first collects real-time data from multiple sources, including meteorological information, geographical information, satellite images, aerial footage, social networking service data streams, demographic statistics, automobile and smartphone movement information, and utility facility operation information. After this data is collected on the server, it undergoes pre-processing.

[0986] As preprocessing means, the server performs image processing such as noise removal and resolution enhancement, and text data filtering. Specifically, for satellite images obtained from a satellite imagery service, resolution enhancement and noise removal are performed, and for posts from the social networking service Data Stream, natural language processing algorithms are used to extract important information related to the disaster.

[0987] The preprocessed data is integrated and input into a generative AI model, which uses the input data to generate detailed damage information, such as flooding status, building damage status, and evacuation routes. The results of this analysis are visualized as a damage map.

[0988] The visualized disaster map is provided to users. Specifically, the server publishes the disaster map through a web interface, allowing users to check the damage situation in real time. The information from this disaster map is also linked to the navigation system installed in the autonomous vehicle. The navigation system then provides safe and efficient travel routes in real time based on the disaster map.

[0989] As a practical example, by inputting the following prompts into a generative AI model, a highly accurate disaster map can be created:

[0990] Prompt: Based on the real-time information below, what is the best evacuation route?

[0991] Precipitation: 20mm / h

[0992] Wind speed: 15km / h

[0993] Latest SNS information: Disasters, Floods, Evacuation

[0994] This will enable navigation that always reflects the latest disaster situation based on data updated in real time, ensuring safe travel.

[0995] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0996] Step 1:

[0997] The server collects data in real time from multiple sources, including meteorological information, geographical information, satellite images, aerial footage, social networking service data streams, demographic statistics, automobile and smartphone movement information, and information on the operation of lifeline facilities.

[0998] Input: Real-time data from multiple sources

[0999] Output: Raw data collected

[1000] Specific behavior:

[1001] The server retrieves weather data using WeatherAPI, periodically downloads images from a satellite imagery service, and collects posts tagged with "disaster" from social networking service data streams.

[1002] Step 2:

[1003] The server preprocesses the collected data, performing image processing such as noise removal and resolution improvement, and filtering of text data.

[1004] Input: Raw data collected

[1005] Output: Preprocessed data

[1006] Specific behavior:

[1007] That is, the pre-processing means performs noise removal and resolution enhancement on satellite images, and uses natural language processing algorithms to extract important disaster-related information from posts from social networking service data streams.

[1008] Step 3:

[1009] The server integrates the preprocessed data and inputs it into the generative AI model as a single analysis target. Based on this data, the generative AI model generates detailed disaster information such as flooding status, building damage status, and evacuation routes.

[1010] Input: Preprocessed data

[1011] Output: Disaster information data

[1012] Specific behavior:

[1013] The integration method combines image data and text data into a single dataset and inputs it into a generative AI model, which analyzes the dataset and generates important disaster information.

[1014] Step 4:

[1015] The server creates a disaster map based on the generated damage information, visually displaying the flooding situation, building damage, evacuation routes, and other information.

[1016] Input: Disaster information data

[1017] Output: Damage map

[1018] Specific behavior:

[1019] The disaster map generation method visually represents the disaster information obtained by the generative AI model and displays flooded areas and evacuation routes on a map. For example, areas with a high flood risk are shown in red, and safe evacuation routes are shown in green.

[1020] Step 5:

[1021] The server publishes the visualized disaster map through a web interface to provide users with real-time information on the disaster situation.

[1022] Input: Disaster map

[1023] Output: A damage map on a user-accessible web interface

[1024] Specific behavior:

[1025] The provision means uploads the generated disaster map to a web interface so that users can access it through a browser. It also supports a disaster information notification function as needed.

[1026] Step 6:

[1027] The device then links the disaster map information to the autonomous vehicle's navigation system, which then provides safe and efficient travel routes in real time based on the disaster map.

[1028] Input: Disaster map on web interface

[1029] Output: Navigation route for autonomous vehicles

[1030] Specific behavior:

[1031] The navigation system receives disaster map data and calculates a safe route based on the analysis results. The autonomous vehicle then navigates safely through the disaster area, enabling rescue operations to be carried out quickly and efficiently.

[1032] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1033] This invention combines a system that collects data in real time from multiple sources, analyzes the damage situation using a generative AI model, and automatically creates a damage map, with an emotion engine that recognizes the user's emotions. This system is an important tool for swift disaster relief activities, and can also provide psychological support to users.

[1034] The server connects to multiple information sources, such as weather information services, geographic information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems, to collect the necessary data in real time. The collected data is integrated and preprocessed. For example, weather data, satellite imagery, drone footage, and social media post data are temporarily stored and then subjected to appropriate preprocessing before various analyses.

[1035] Specifically, the server periodically downloads the latest satellite images from a satellite imagery service, and uses an image processing module to improve resolution and remove noise. Social media posts are collected in real time from the data stream, and natural language processing algorithms may be used to extract information related to the disaster.

[1036] The server integrates the collected data and inputs it into a generative AI model. The generative AI model analyzes the current situation in the affected areas and creates a damage map. It also uses an emotion engine to analyze the emotional information in the post text extracted from the social media data stream. For example, from a post tagged with "disaster," the emotion engine analyzes the poster's emotions and identifies emotional states such as fear, relief, and tension.

[1037] The emotional information analyzed by the emotion engine is reflected on a disaster map. For example, areas with heightened fear and anxiety can be color-coded, and this information can be used to provide appropriate psychological support to government and local government officials.

[1038] The generated disaster map is uploaded to a web interface by the server and can be accessed by users and their devices. Users can check the damage situation and emotional information in real time through the web portal. This not only provides users with information to take appropriate actions, but also helps them understand where psychological support is needed.

[1039] For example, the server retrieves the latest precipitation data from an API provided by the Japan Meteorological Agency and downloads satellite imagery from a satellite imagery service at 8:00, 10:00, and 12:00. The processed satellite imagery is analyzed by a generative AI model to identify areas at high risk of flooding and evacuation routes. At the same time, posts extracted from social media data streams are analyzed by an emotion engine, and areas with a large number of posts expressing fear or anxiety are highlighted in red.

[1040] Users can view this disaster map on a web portal and quickly grasp the situation and emotional information of the affected areas, which will enable efficient rescue operations and evacuation instructions, not only saving lives and minimizing damage, but also providing psychological support to victims.

[1041] The processing flow will be explained below.

[1042] Step 1:

[1043] The server connects to multiple information sources, such as weather information services, geographic information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems, thereby preparing to collect the necessary data in real time.

[1044] Step 2:

[1045] The server retrieves current weather data from a weather information service, the latest satellite images from a satellite imagery service, real-time video streams from the drone operation team, and traffic data from the traffic information system, and stores them in temporary storage.

[1046] Step 3:

[1047] The server sends the acquired satellite image data to the image processing module, which processes it to improve resolution and remove noise, and stores the processed image data for analysis.

[1048] Step 4:

[1049] The server collects posts from social media data streams in real time and uses natural language processing algorithms to extract disaster-related information. For example, it filters posts tagged with "disaster" and analyzes location information and emotional states using a text analysis module.

[1050] Step 5:

[1051] The server uses an emotion engine to analyze users' emotional information from the filtered SNS posts. For example, it identifies emotional states such as fear, anxiety, and relief, and adds this information to the text data.

[1052] Step 6:

[1053] The server captures the real-time stream of drone footage frame by frame, tags the parts it deems important, and pre-processes them, for example analyzing the condition of a building using object recognition algorithms to identify damage to the building.

[1054] Step 7:

[1055] The server integrates pre-processed weather data, visual data, traffic data, social media data, etc., and converts them into a unified format, which prepares the input data for the generative AI model.

[1056] Step 8:

[1057] The server inputs the integrated data into a generative AI model for analysis, which analyzes the risk of flooding in the affected area, the risk of building damage, traffic congestion, etc., and calculates the risk level for each.

[1058] Step 9:

[1059] Based on the analysis results obtained from the generative AI model, the server creates a disaster map that shows color-coded areas at high risk of flooding, areas that are not flooded, evacuation routes, and areas at high emotional risk that require psychological support.

[1060] Step 10:

[1061] The server uploads the created disaster map to a web interface, making it accessible to users and devices. Users can check the damage situation and emotional information in real time through the web portal and obtain information to take appropriate actions.

[1062] Step 11:

[1063] The server will quickly send disaster maps and related information to government and local government officials and disaster relief teams, enabling them to carry out relief efforts quickly and effectively, and also to provide psychological support to victims.

[1064] Example 2

[1065] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1066] Conventional disaster response systems have difficulty collecting and analyzing data from multiple sources in real time to grasp the damage situation. It is also extremely difficult to accurately visualize the current situation in the disaster area and to reflect the emotional state of the victims. Therefore, while efficient and rapid disaster response is required, current systems are unable to meet this demand.

[1067] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting data in real time from multiple information sources, means for preprocessing the collected data, means for integrating the preprocessed data and analyzing it using a generative AI model, means for analyzing emotional information in text data using an emotion engine, and means for reflecting the analyzed emotional information in a damage map. This makes it possible not only to accurately grasp and visualize the damage situation, but also to create a damage map that reflects the emotional state of the victims.

[1068] "Multiple sources" are data sources that provide different types of information, including weather information services, geographic information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems.

[1069] "Real-time data collection means" is a combination of APIs, sensors, and data streams that obtain up-to-date data from various sources via the internet.

[1070] "Means for data preprocessing" refers to techniques that perform processes such as data cleansing, noise removal, resolution improvement, and format conversion in order to prepare collected data in a form suitable for analysis.

[1071] Data synthesis is the process of bringing together data collected from different sources over time into a single integrated data set.

[1072] "Means of analysis using generative AI models" refer to algorithms and models that utilize machine learning and artificial intelligence technology to analyze the damage situation based on integrated data.

[1073] The "emotion engine" is a system that uses natural language processing technology to automatically analyze the poster's emotions and psychological state from text data.

[1074] "Means for analyzing emotional information from text data" refers to the process of using an emotion engine to extract the emotions of posters from text data such as social media posts and quantifying them.

[1075] "Means for reflecting analyzed emotional information on a disaster map" refers to a technique that uses color coding and icons to visually show analyzed emotional information on a disaster map.

[1076] The "means for visualizing a disaster map and providing it to users" is a system that generates an intuitively easy-to-understand map based on the analyzed disaster situation and emotional information, and provides this to users through a web interface or application.

[1077] The present invention is a system that collects data in real time from multiple information sources, preprocesses it, and analyzes it using a generative AI model. It also uses an emotion engine to analyze emotional information from text data and reflects the results in a disaster map, providing users with information about the damage situation and emotions. The following describes in detail an embodiment of this system.

[1078] Data collection

[1079] The server collects data in real time from multiple sources, including meteorological information services, geographical information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems, making it possible to grasp the damage situation from multiple perspectives.

[1080] For example, the server can use the API of the Japan Meteorological Agency to obtain the latest precipitation data, or it can periodically download satellite images from a satellite imagery service.

[1081] Data Preprocessing

[1082] The server pre-processes the collected data, which includes data cleansing, noise removal, resolution enhancement, format conversion, etc. Specifically, it uses a specific image processing module to enhance the resolution and remove noise from the satellite image data.

[1083] In addition, natural language processing algorithms are used to extract information related to the disaster from social media data.

[1084] Data integration

[1085] The server then integrates the pre-processed data, collating data from different sources into a single integrated dataset along a timeline, thereby building a database that can provide a comprehensive understanding of the disaster situation.

[1086] Analysis of the damage situation

[1087] The server inputs the integrated data into a generative AI model, which analyzes the damage situation in real time. The generative AI model uses machine learning and artificial intelligence technology to analyze the current situation in the disaster area based on various data.

[1088] Emotional information analysis

[1089] The server uses an emotion engine to analyze the emotional information of posts extracted from SNS data. The emotional information obtained through this analysis is used to visualize the psychological state of people in the disaster-stricken areas.

[1090] Creating a disaster map

[1091] The server integrates the damage situation and emotional information to create a damage map. The damage map is visually easy to understand and is designed to allow people to quickly grasp the current situation in the affected areas. Areas at high risk of flooding are displayed in blue, and areas where feelings of fear and anxiety are rising are displayed in red.

[1092] Upload to the web interface

[1093] The server uploads the generated disaster map to a web interface, allowing users and devices to check the damage situation and emotional information in real time.

[1094] User Use

[1095] Users can view the disaster map through a web portal and check the damage situation and sentiment information in real time, which provides information for taking appropriate actions.

[1096] Examples of concrete examples and prompts

[1097] Examples:

[1098] 1. The server obtains the latest precipitation data using the Japan Meteorological Agency's API.

[1099] 2. The server downloads satellite images at 8:00, 10:00, and 12:00 from the satellite imagery service.

[1100] 3. The server uses the generative AI model to analyze the processed satellite imagery and identify areas at high risk of flooding.

[1101] 4. The server uses an emotion engine to analyze posts extracted from the social media data stream and displays areas with a large number of posts expressing fear or anxiety in red.

[1102] 5. Users can view the damage map on the web portal and obtain information to take appropriate actions.

[1103] Example prompt sentence:

[1104] "Identify areas at high risk of flooding based on the latest rainfall data and satellite imagery."

[1105] "Please extract emotional information from social media data and reflect it in the disaster map."

[1106] "View detailed sentiment information for areas highlighted in red"

[1107] This will provide a concrete example of how to put the invention into practice, making it possible to speed up disaster relief efforts, provide accurate information, and provide psychological support to disaster victims.

[1108] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1109] Step 1:

[1110] The server collects data in real time from multiple sources, such as meteorological information services, geographic information services, satellite imagery services, drone operation teams, social media data streams, traffic information systems, and lifeline facility management systems. The input is the data retrieved by sending requests from these sources, and the output is the temporarily stored, unprocessed data.

[1111] Step 2:

[1112] The server preprocesses the collected data. Preprocessing includes data cleansing, noise removal, resolution enhancement, and format conversion. For example, it performs resolution enhancement on satellite image data and performs natural language processing on social media data to extract relevant information. The input is unprocessed data, and the output is processed data.

[1113] Step 3:

[1114] The server integrates the preprocessed data. It combines data collected from different sources into a single integrated dataset along a time axis. For example, it integrates weather data, satellite image data, and social media data into a single dataset. The input is preprocessed data, and the output is the integrated dataset.

[1115] Step 4:

[1116] The server inputs the integrated data into a generative AI model, which analyzes the damage situation in real time. The generative AI model uses machine learning and artificial intelligence technology to analyze the current situation in the disaster-stricken area based on various data. The input is the integrated dataset, and the output is the analysis result of the damage situation. For example, it can identify areas with a high risk of flooding.

[1117] Step 5:

[1118] The server uses an emotion engine to analyze the emotional information of posts extracted from SNS data. The emotion engine uses natural language processing technology to analyze emotional information from text data. The input is the text data extracted from SNS, and the output is the analysis result of the emotional information. For example, emotions indicating fear or anxiety are extracted.

[1119] Step 6:

[1120] The server integrates the damage situation and emotional information to create a damage map. The damage map visually reflects areas at high risk of flooding and emotional information. The input is the analysis results of the damage situation and emotional information, and the output is a damage map. For example, areas at high risk of flooding are displayed in blue, and heightened emotions in red.

[1121] Step 7:

[1122] The server uploads the generated disaster map to a web interface, allowing users and devices to check the disaster situation and emotional information in real time. The input is the disaster map, and the output is the disaster map displayed on the web portal.

[1123] Step 8:

[1124] Users can view the disaster map through the web portal and check the damage situation and emotional information, which can provide information to take appropriate actions. For example, users can check the disaster map and find a safe evacuation route.

[1125] (Application example 2)

[1126] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1127] Current navigation systems for autonomous vehicles lack the functionality to support a rapid and safe response in disaster-stricken areas. The lack of a means to analyze and provide information on the disaster situation and emotions in real time makes it difficult to select safe routes or provide psychological support in emergencies. Furthermore, there is insufficient integration of information sources that can accurately grasp the situation in disaster-stricken areas, and the development of a system that encourages appropriate action is required.

[1128] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting data from multiple information sources in real time and linking with the vehicle's navigation system, means for preprocessing the collected data and calculating safe routes and emotional information, and means for integrating the preprocessed data, analyzing it using a generative AI model, and calculating the situation in the disaster area and an appropriate route to take. This makes it possible to accurately grasp the situation in the disaster area and provide the optimal route in real time. The user can take safe and efficient evacuation actions, and psychological support can also be provided to the disaster victims.

[1129] "Multiple information sources" refers to data sources such as weather information, geographic information, satellite images, drone footage, social network data, traffic information, and information on lifeline facilities.

[1130] "Real-time" refers to the acquisition and processing of data almost simultaneously, and the provision of that information to users without delay.

[1131] "Preprocessing" refers to a process for converting acquired raw data into an analyzable format, and includes processes such as removing noise from the data and improving resolution.

[1132] A "generative AI model" refers to an artificial intelligence model that analyzes collected data and generates new information based on the damage situation and emotional data.

[1133] A "disaster map" is a map that visually displays the current situation and emotional state of the disaster-stricken area based on analyzed data.

[1134] An "emotion engine" refers to an algorithm that analyzes emotional information from text data such as social network data.

[1135] "Automobile navigation system" refers to a system installed in an autonomous vehicle that provides guidance on the vehicle's current location and route to a destination.

[1136] A "safe route" refers to a route that takes into account the disaster situation and emotional information and minimizes damage and danger in the disaster area.

[1137] "Visualization" refers to displaying analysis results and damage maps in a format that is easy to see and understand.

[1138] "User" refers to the people who receive and use information within an autonomous vehicle.

[1139] In this invention, the server collects data from multiple sources in real time, preprocesses the data, and calculates safe routes and emotional information. It also uses a generative AI model to integrate the preprocessed data and calculate the situation in the disaster area and appropriate routes. Next, it creates a disaster map based on the analysis results and updates it in real time. Finally, it visualizes the disaster map and safe routes and provides them to users.

[1140] A specific example is given below.

[1141] Source data collection and preprocessing

[1142] The server connects to multiple sources, including meteorological information, geographical information, satellite images, drone footage, social network data, traffic information, and information on utility facilities, to collect the necessary data in real time. For example, it periodically downloads the latest satellite images from satellite imagery services, and uses an image processing module to improve resolution and remove noise. It also collects posts from social network data streams in real time and uses natural language processing algorithms to extract information related to the disaster.

[1143] Data integration and analysis

[1144] The collected data is integrated and preprocessed on a server. The preprocessed data is then input into a generative AI model to analyze the current situation in the disaster-stricken areas and automatically create a disaster map. An emotion engine is also used to analyze the emotional information in the post text extracted from social network data. For example, from a post tagged with "disaster," the emotion engine analyzes the poster's emotions and identifies emotional states such as fear, relief, and tension.

[1145] Creation and updating of disaster maps

[1146] The generated disaster map is updated in real time by the server and provided to users via a web interface. Users can view the disaster map in real time and quickly grasp the situation and emotional information in the affected area, which enables efficient evacuation instructions and psychological support.

[1147] Examples and prompts

[1148] For example, the server retrieves the latest precipitation data from a weather service's API and downloads satellite imagery from a satellite imagery service at 8:00, 10:00, and 12:00. The processed satellite imagery is analyzed by a generative AI model to identify areas at high risk of flooding and evacuation routes. At the same time, posts extracted from social network data streams are analyzed by an emotion engine, and areas with a high number of posts expressing fear or anxiety are highlighted in red.

[1149] An example of a prompt sentence to input to the generative AI model is as follows:

[1150] Social network posts from near the affected area (disaster):

[1151] 1. I'm terrified. I need help.

[1152] 2. The area is relatively safe.

[1153] Please analyze the data and suggest the best safe route and support points.

[1154] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1155] Step 1:

[1156] The server collects data in real time from multiple sources, such as weather information services, geographic information services, satellite imagery services, drone operation teams, social network data streams, traffic information systems, and lifeline facility management systems. The input is data obtained from various APIs, and the output is the collected raw data.

[1157] Step 2:

[1158] The server preprocesses the collected data, which includes improving the resolution of satellite images, removing noise, and extracting relevant information from social network data. The input for preprocessing is the raw data collected in step 1, and the output is the preprocessed data converted into an analyzable format.

[1159] Step 3:

[1160] The server integrates the preprocessed data. In this step, meteorological data, geographic data, satellite image data, drone footage, social network data, etc. are converted into a unified format to create an integrated dataset. The input is the preprocessed data, and the output is the integrated data.

[1161] Step 4:

[1162] The server inputs the integrated data into a generative AI model to analyze the situation in the disaster-stricken areas. The generative AI model analyzes the damage information and emotional information to create a damage map. In this process, it also uses an emotional engine to analyze the emotional information of posters from social network data. The input is the integrated data, and the output is a damage map and emotional information.

[1163] Step 5:

[1164] The server creates a disaster map based on the analysis results and updates it in real time. Specifically, it visually displays flood risk, evacuation routes, emotional information, etc. based on the output of the generative AI model. The input is the analysis results from the generative AI model, and the output is a disaster map.

[1165] Step 6:

[1166] The server visualizes the disaster map and the safe route and provides it to the user. Specifically, the disaster map is displayed on a web interface or a car navigation system so that the user can check it in real time. The input is the disaster map, and the output is the visualized disaster map.

[1167] Step 7:

[1168] Users can use the visualized disaster map to take appropriate actions, such as choosing a safe evacuation route and paying attention to areas where psychological support is needed. The input is the visualized disaster map, and the output is the user's actions.

[1169] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1170] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1171] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1172] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1173] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1174] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1175] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1176] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1177] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1178] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1179] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1180] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1181] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1182] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1183] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1184] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1185] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1186] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1187] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1188] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1189] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1190] The following is further disclosed regarding the above embodiment.

[1191] (Claim 1)

[1192] a means of collecting data in real time from multiple sources;

[1193] a means for pre-processing the collected data;

[1194] A means of integrating the pre-processed data and analyzing it using a generative AI model;

[1195] A means of creating a damage map based on the analysis results, and

[1196] A means for visualizing the disaster map and providing it to users;

[1197] A system including:

[1198] (Claim 2)

[1199] 10. The system of claim 1, wherein the visual data includes satellite imagery and drone footage that are collected and preprocessed.

[1200] (Claim 3)

[1201] 2. The system according to claim 1, wherein disaster-related information is extracted from a social networking service data stream as text data.

[1202] "Example 1"

[1203] (Claim 1)

[1204] a means of collecting data in real time from multiple sources;

[1205] a means for pre-processing the collected data;

[1206] means for integrating the pre-processed data and analyzing it using a generative artificial intelligence model;

[1207] A means of creating a disaster situation map based on the analysis results;

[1208] A means for visualizing and providing a disaster situation map to a user;

[1209] A means for removing noise and extracting important information from the collected visual data and text data;

[1210] a means for identifying flood risks and evacuation routes based on the pre-processed data;

[1211] means for uploading the disaster situation map to a web interface;

[1212] A system including:

[1213] (Claim 2)

[1214] 10. The system of claim 1, wherein the visual data includes satellite imagery and aerial footage that are collected and preprocessed.

[1215] (Claim 3)

[1216] 10. The system of claim 1, wherein the system extracts disaster-related information from digital social media data streams as text data.

[1217] "Application Example 1"

[1218] (Claim 1)

[1219] a means of collecting data in real time from multiple sources;

[1220] a means for pre-processing the collected data;

[1221] A means of integrating the pre-processed data and analyzing it using a generative AI model;

[1222] A means of creating a damage map based on the analysis results, and

[1223] A means for visualizing the disaster map and providing it to users;

[1224] A means of linking disaster map information to the navigation system installed in autonomous vehicles;

[1225] A system including:

[1226] (Claim 2)

[1227] 10. The system of claim 1, wherein the visual data includes satellite imagery and airborne video footage that are collected and preprocessed.

[1228] (Claim 3)

[1229] 2. The system of claim 1, wherein disaster-related information is extracted from a social networking service data stream as text data.

[1230] "Example 2: Combining Emotion Engines"

[1231] (Claim 1)

[1232] a means of collecting data in real time from multiple sources;

[1233] a means for pre-processing the collected data;

[1234] A means of integrating the pre-processed data and analyzing it using a generative AI model;

[1235] A means of creating a damage map based on the analysis results, and

[1236] means for analyzing emotional information in the text data using an emotion engine;

[1237] A means of reflecting the analyzed emotional information on the disaster map;

[1238] A means for visualizing the disaster map and providing it to users;

[1239] A system including:

[1240] (Claim 2)

[1241] 10. The system of claim 1, wherein the visual data includes satellite imagery and drone footage that are collected and preprocessed.

[1242] (Claim 3)

[1243] The system according to claim 1, wherein disaster-related information is extracted from the SNS data stream as text data, and the emotional information of the posts is analyzed using an emotion engine.

[1244] "Application example 2 when combining emotion engines"

[1245] New Claims

[1246] (Claim 1)

[1247] A means of collecting data in real time from multiple sources and linking it with the car's navigation system;

[1248] A means for preprocessing the collected data and calculating safe routes and emotional information;

[1249] A means to integrate the pre-processed data, analyze it using a generative AI model, and calculate the situation in the affected area and the appropriate route of action;

[1250] A method to create a damage map based on the analysis results and update it in real time,

[1251] A means for visualizing and providing disaster maps and safe routes to users;

[1252] A system including:

[1253] (Claim 2)

[1254] 10. The system of claim 1, wherein the visual data includes satellite imagery and drone footage that are collected and preprocessed.

[1255] (Claim 3)

[1256] 2. The system according to claim 1, wherein disaster-related information is extracted as text data from a social network data stream and analyzed using an emotion engine. [Explanation of symbols]

[1257] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means of collecting data in real time from multiple sources; a means for pre-processing the collected data; A means of integrating the pre-processed data and analyzing it using a generative AI model; A means of creating a damage map based on the analysis results, and A means for visualizing the disaster map and providing it to users; A system including:

2. The system of claim 1 , wherein the visual data includes satellite imagery and drone footage that are collected and preprocessed.

3. The system according to claim 1, wherein disaster-related information is extracted as text data from an SNS data stream.

Images