system

Abstract

We provide the system. [Solution] Means for collecting sensor data, Means for transmitting the aforementioned sensor data to a server, Means for analyzing the aforementioned sensor data, A means of acquiring and integrating additional data from other institutions, A means for generating an emergency notification based on the aforementioned analysis results and integrated information, A means for sending the aforementioned emergency notification to the terminal, Means by which the user takes action in response to an emergency notification displayed on the aforementioned terminal. A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, a large number of damages have occurred due to natural disasters such as earthquakes and floods. Conventional disaster response systems are insufficient in real-time information collection and emergency notifications, making it difficult for disaster victims to take appropriate evacuation actions. Also, effective management has not been carried out in the provision of support resources, making it difficult to provide prompt support to disaster victims. Against this background, there is a need to collect and analyze data in real-time during a disaster and provide prompt and accurate emergency notifications and support.

Means for Solving the Problems

[0005] The present invention solves these problems with a system that includes means for collecting sensor data, means for transmitting the sensor data to a server, means for analyzing the sensor data, means for acquiring and integrating additional data from other organizations, means for generating emergency notifications based on the analysis results and integrated information, means for transmitting the emergency notifications to a terminal, and means for the user to take action in response to the emergency notification displayed on the terminal.

[0006] By collecting sensor data and transmitting it to a server, the disaster situation can be understood in real time. The server then analyzes the received data to identify the scale and location of the disaster, and by acquiring and integrating additional data from other organizations, more accurate information can be provided. Furthermore, emergency notifications are generated based on the analysis results and integrated information and sent to terminals, enabling users to take quick and appropriate evacuation actions.

[0007] Furthermore, by including means for obtaining the user's current location and providing the optimal evacuation route according to emergency notifications, the system enables disaster victims to evacuate via the safest route. In addition, by including means for managing the location information of evacuation centers and relief supplies and providing users with the most suitable support resources, the system enables rapid assistance to disaster victims. In this way, the present invention provides a system that enables rapid and accurate information provision and allocation of support resources during natural disasters.

[0008] "Sensor data" refers to information collected from devices used to detect changes in the physical environment.

[0009] A "server" is a computer system used to manage and process the transmission and reception of data over a network.

[0010] "Analysis" is the process of examining collected data in detail and extracting useful information.

[0011] "Integration" is the process of combining information obtained from multiple different data sources into a single, unified set of information.

[0012] An "emergency notification" is a warning message that is sent immediately to the user when a disaster or danger is imminent.

[0013] A "device" is a device that the user directly operates (e.g., a smartphone or tablet).

[0014] A "user" is an individual or group that uses this system.

[0015] "Support resources" is a general term for resources (e.g., shelters, supplies) provided to support victims during a disaster.

[0016] An "evacuation route" is a recommended route for safe evacuation in the event of a disaster. [Brief explanation of the drawing]

[0017] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9]Shows an emotion map to which a plurality of emotions are mapped. [Figure 10] Shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Modes for Carrying Out the Invention

[0018] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a processor with a reference number (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0021] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0022] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0023] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0025] [First Embodiment]

[0026] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0027] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0030] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0033] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0037] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0038] This invention is a system that collects and analyzes information in real time during natural disasters and provides users with rapid emergency notifications. This system consists of three main components: a terminal, a server, and a user.

[0039] System Overview

[0040] 1. Data Collection

[0041] Device: Each user's device (e.g., smartphone) is equipped with a GPS module for acquiring location information, as well as seismometers, accelerometers, water level sensors, and other sensors. These sensors periodically collect data and transmit it to a server via the network.

[0042] 2. Data transmission

[0043] Terminal: Collected sensor data is periodically batch-processed and sent to the server via a secure communication protocol (e.g., HTTPS).

[0044] 3. Data reception and storage

[0045] Server: The server receives data sent from terminals and stores it in the database. During this process, data validation and filtering are also performed to remove duplicate and invalid data.

[0046] 4. Data Analysis

[0047] Server: The server analyzes the received data to determine the scale and location of the disaster. For example, it analyzes earthquake vibration data to determine the epicenter and seismic intensity. This analysis is performed using algorithms (e.g., FFT analysis).

[0048] 5. Information Integration

[0049] Server: The server also acquires data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with its internal data. This generates more accurate and detailed disaster information.

[0050] 6. Emergency notification generation

[0051] Server: The server generates emergency notifications for each user based on the analysis results and integrated information. These emergency notifications include evacuation orders and recommended evacuation routes.

[0052] 7. Send emergency notification

[0053] Server: The generated emergency notification is sent to each user's device via a push notification service (e.g., Firebase Cloud Messaging).

[0054] 8. User behavior

[0055] User: The user receives an emergency notification from their device and begins evacuation. They check the information provided by the device (e.g., evacuation orders, evacuation routes) and move to a safe location.

[0056] Specific examples

[0057] For example, consider the case of an earthquake. When the device's accelerometer detects a strong tremor, it immediately sends this data to the server. The server analyzes the data from multiple devices to calculate the epicenter and seismic intensity. The server then obtains additional data from the Japan Meteorological Agency to further determine the extent of the disaster's impact. Finally, the server generates an emergency notification and sends a push notification to the user's device. The user checks this notification and evacuates to a safe place according to the recommended evacuation route.

[0058] Furthermore, if a flood warning is issued, the terminal collects water level sensor data and sends it to the server. The server analyzes this data and predicts the likelihood of flooding. It integrates additional data (e.g., water level data from the river management authority) to determine the extent of the flood's impact. The generated emergency notification is sent to users in the affected area as a notification including evacuation instructions. Users see this notification and take action to evacuate quickly and safely.

[0059] In this way, the present invention provides users with rapid and accurate information and supports appropriate responses during natural disasters.

[0060] The following describes the processing flow.

[0061] Step 1:

[0062] Terminals: Each terminal periodically collects sensor data (e.g., GPS location information, seismometer data). The smartphone's GPS module acquires location information, and the accelerometer collects ambient vibration data. This data is collected at regular time intervals and processed in batches.

[0063] Step 2:

[0064] Terminal: Collects sensor data and sends it to the server using a secure communication protocol (e.g., HTTPS). The data is packetized in a format such as JSON and transmitted over the network.

[0065] Step 3:

[0066] Server: The server receives data sent from the terminal. Data received via the API endpoint is immediately stored in the database. Data is validated and filtered upon receipt to remove duplicate and invalid data.

[0067] Step 4:

[0068] Server: Analyzes received data to determine the scale and location of the disaster. For example, if earthquake vibration data is transmitted, the server uses algorithms such as FFT analysis to calculate the epicenter and seismic intensity.

[0069] Step 5:

[0070] Server: Acquires additional data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with internal data. This external data is retrieved using a REST API and stored in the server's database. This generates more detailed and accurate disaster information.

[0071] Step 6:

[0072] Server: Based on analysis results and integrated information, generates emergency notifications to send to users. These emergency notifications include evacuation orders and recommended evacuation routes. The generated notifications are formatted using templates.

[0073] Step 7:

[0074] Server: Generates emergency notifications and sends them to each user's device using a push notification service (e.g., Firebase Cloud Messaging). The notification range is determined based on the target region and the user's current location.

[0075] Step 8:

[0076] User: The user receives an emergency notification from their device and checks it immediately. They begin evacuation according to the notification's contents. The smartphone app displays evacuation shelters and recommended evacuation routes to help the user evacuate safely.

[0077] The above outlines the specific flow of the system's program processing. This process enables a swift and appropriate response in the event of a disaster.

[0078] (Example 1)

[0079] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0080] Conventional disaster notification systems lacked real-time capabilities in collecting and analyzing sensor data, making it difficult to provide users with timely emergency notifications. Furthermore, the integration of data from multiple sources was insufficient, preventing improvements in the accuracy and detail of disaster information. As a result, users were unable to take appropriate evacuation actions quickly, increasing the risk of disaster damage.

[0081] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0082] In this invention, the server includes means for verifying, filtering, and storing sensor data; means for analyzing the sensor data using FFT analysis; and means for acquiring and integrating additional data from other organizations. This enables the generation of highly accurate and detailed disaster information and the rapid provision of emergency notifications to users through real-time analysis of sensor data and integration of data from multiple organizations.

[0083] "Sensor data" refers to data collected by various sensors installed in the device (e.g., GPS module, seismometer, accelerometer, water level sensor).

[0084] A "server" is a central computer system that receives, stores, and analyzes sensor data, and integrates additional data from other organizations.

[0085] "Verification and filtering" is the process of checking data sent to the server from the perspectives of accuracy, duplication, and fraud, and saving only the appropriate data.

[0086] "FFT analysis" refers to the Fast Fourier Transform algorithm used by servers to analyze sensor data, and is particularly used for vibration analysis of earthquake data.

[0087] "Additional data from other organizations" refers to supplementary data obtained from external organizations such as the Japan Meteorological Agency and river management bureaus, which are integrated with sensor data.

[0088] An "emergency notification" is a notification message, including warnings and evacuation orders, that is generated based on analysis results and integrated information when a disaster occurs.

[0089] A "communication network" is the infrastructure used to send and receive digital data, such as the internet or dedicated lines.

[0090] A "terminal" refers to a device, such as a smartphone or tablet, that collects sensor data and receives emergency notifications.

[0091] "Means of taking action" refers to the user checking emergency notifications from their device and then acting according to the designated evacuation instructions and evacuation routes.

[0092] This invention is a system that collects and analyzes information in real time during natural disasters and provides users with rapid emergency notifications. This system consists of three main components: a terminal, a server, and a user. Details of each are described below.

[0093] 1. Data Collection

[0094] Device: The user's device (e.g., smartphone) collects data using various sensors such as GPS modules, seismometers, accelerometers, and water level sensors. These sensors operate continuously and measure data at regular intervals. For example, a smartphone's accelerometer captures tremor information at a frequency of 5 times per second.

[0095] 2. Data transmission

[0096] Terminal: The collected sensor data is periodically batch-processed and securely sent to the server via the HTTPS protocol. For example, the terminal sends the data it collects to the server via HTTPS every 5 minutes.

[0097] 3. Data reception and storage

[0098] Server: The server receives data sent from the terminal and stores it in a database using Python and MySQL®. During this process, it validates and filters the data to remove invalid and duplicate data.

[0099] 4. Data Analysis

[0100] Server: The server analyzes the received data to determine the scale and location of the disaster. In particular, in the case of earthquakes, it uses FFT (Fast Fourier Transform) analysis to calculate the epicenter and seismic intensity. The server identifies the epicenter and seismic intensity by performing FFT analysis on the vibration data it receives.

[0101] 5. Information Integration

[0102] Server: The server also acquires data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with its internal data. This generates more accurate and detailed information about disasters. It obtains the latest weather data from the Japan Meteorological Agency's API and integrates it with its own data.

[0103] 6. Emergency notification generation

[0104] Server: The server generates emergency notifications based on analysis results and integrated information. These notifications include evacuation orders and recommended evacuation routes. The server creates emergency notifications containing evacuation orders and routes based on the specified format.

[0105] 7. Send emergency notification

[0106] Server: Generated emergency notifications are pushed to each user's device using Firebase Cloud Messaging. The server calls Firebase Cloud Messaging to deliver the generated emergency notifications to all users' devices.

[0107] 8. User behavior

[0108] User: The user checks the emergency notification received on their device and begins evacuation. The notification includes detailed information on evacuation routes and locations, and the user evacuates to a safe place according to these instructions. The user checks the notification on their smartphone and heads to the nearest evacuation center according to the displayed evacuation instructions and evacuation route.

[0109] Specific example

[0110] For example, in the event of an earthquake, if the device's accelerometer detects strong shaking, it immediately sends this data to the server. The server analyzes the data from multiple devices to calculate the epicenter and seismic intensity. The server then obtains additional data from the Japan Meteorological Agency to further determine the extent of the disaster's impact. Finally, the server generates an emergency notification and sends a push notification to the user's device. The user checks this notification and evacuates to a safe place following the recommended evacuation route.

[0111] Example of a prompt

[0112] "An earthquake has occurred. Please evacuate to a safe place immediately. The nearest evacuation center is point A. Taking route B will ensure your safety."

[0113] Thus, the present invention provides users with rapid and accurate information and supports appropriate responses during natural disasters.

[0114] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0115] Step 1:

[0116] Data collection

[0117] Device: The user's device (smartphone) collects data in real time using its built-in GPS module, seismometer, accelerometer, and water level sensor.

[0118] Input: Measurement data from the sensor.

[0119] Data processing: Batch processing of data within the terminal (e.g., grouping data at regular intervals).

[0120] Output: Batch-processed sensor data.

[0121] Specific operation: The smartphone's accelerometer captures and temporarily stores information about shaking at a frequency of 5 times per second.

[0122] Step 2:

[0123] Data transmission

[0124] Terminal: Collected sensor data is periodically sent to the server via the HTTPS protocol.

[0125] Input: Batch-processed sensor data.

[0126] Data processing: Data encryption and secure transmission.

[0127] Output: Encrypted sensor data is sent to the server.

[0128] Specific operation: Every 5 minutes, the device sends the collected data to the server via HTTPS.

[0129] Step 3:

[0130] Data reception and storage

[0131] Server: Receives data sent from terminals, verifies and filters the data to ensure its accuracy, and then stores it in a database.

[0132] Input: Transmitted sensor data.

[0133] Data processing: Verification (e.g., checking data accuracy) and filtering (e.g., removing duplicate data).

[0134] Output: Clean sensor data is saved to the database.

[0135] Specific operation: The server uses Python and MySQL to validate the data and then save it to the database.

[0136] Step 4:

[0137] Data Analysis

[0138] Server: Analyzes stored data to determine the scale and location of disasters. In particular, in the case of earthquakes, it uses FFT (Fast Fourier Transform) analysis to calculate the epicenter and seismic intensity.

[0139] Input: Stored sensor data.

[0140] Data processing: Perform FFT analysis.

[0141] Output: Analysis results such as epicenter and seismic intensity.

[0142] Specific operation: The server applies FFT analysis to the vibration data it receives to identify the epicenter.

[0143] Step 5:

[0144] Information integration

[0145] Server: Acquires additional data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with internal data.

[0146] Input: Analysis results and additional data from external organizations.

[0147] Data processing: Data integration and analysis.

[0148] Output: Even more accurate disaster information.

[0149] Specific operation: Obtain the latest earthquake information from the Japan Meteorological Agency's API and integrate it with our own data.

[0150] Step 6:

[0151] Emergency notification generation

[0152] Server: Based on analysis results and integrated information, it generates optimized emergency notifications for each user (including evacuation orders and recommended evacuation routes).

[0153] Input: Analysis results and integrated information.

[0154] Data processing: Creating emergency notifications.

[0155] Output: Emergency notification message.

[0156] Specific operation: The server creates an emergency notification, including evacuation instructions and evacuation routes, based on the specified format.

[0157] Step 7:

[0158] Emergency notification sent

[0159] Server: The generated emergency notification is pushed to each user's device using Firebase Cloud Messaging.

[0160] Input: Generated emergency notification message.

[0161] Data processing: Sending notification messages.

[0162] Output: Emergency notification sent to the user's device.

[0163] Specific operation: The server calls Firebase Cloud Messaging and delivers the generated emergency notification to all users' devices.

[0164] Step 8:

[0165] User behavior

[0166] User: The user checks the emergency notification received on their device and begins evacuation according to the instructions in the notification.

[0167] Input: Emergency notification displayed on the device.

[0168] Output: User evacuation actions.

[0169] Specific actions: The user checks the notification on their smartphone and heads to the nearest evacuation shelter according to the displayed evacuation instructions and evacuation route.

[0170] (Application Example 1)

[0171] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0172] While systems exist that provide real-time information to help users evacuate quickly and safely during natural disasters, these systems generally assume that users will move manually. Therefore, especially in urban areas with a large number of autonomous vehicles, there is a lack of efficient and immediate means of evacuation. To overcome this drawback, a method is needed that automatically controls autonomous vehicles during natural disasters to select appropriate evacuation routes and guide them to safe locations.

[0173] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0174] In this invention, the server includes means for collecting sensor data, means for transmitting sensor data to the server, means for analyzing sensor data, means for acquiring and integrating additional data from other organizations, means for generating emergency notifications based on the analysis results and integrated information, means for transmitting emergency notifications to a terminal, means for the user to take action in response to the emergency notification displayed on the terminal, means for controlling the operation of the autonomous mobile vehicle and calculating a route for safe evacuation, and means for the autonomous mobile vehicle to move according to the calculated evacuation route. This enables the autonomous mobile vehicle to perform appropriate evacuation actions quickly and safely.

[0175] "Sensor data" refers to data collected from a wide variety of sensors (e.g., GPS, seismometers, accelerometers, water level sensors).

[0176] A "server" is a computer system that receives and analyzes sensor data via a network, and processes information by integrating additional data from other organizations.

[0177] "Means of sending to the server" refers to the function of securely and efficiently transferring collected sensor data to the server.

[0178] "Means of analysis" refers to methods of evaluating the scale, location, and impact of a disaster using collected sensor data.

[0179] "Methods for acquiring and integrating additional data from other organizations" refers to methods of collecting data from organizations such as the Japan Meteorological Agency and river management bureaus, and combining it with the data analyzed on the server.

[0180] "Means for generating emergency notifications" refers to a method of creating emergency information (e.g., evacuation orders, recommended evacuation routes) that should be conveyed to users, based on analysis results and integrated information.

[0181] "Means of sending emergency notifications to devices" refers to a function that transmits generated emergency notifications to user devices such as smartphones and tablets.

[0182] "Means of action for the user" refers to a function that facilitates actions to evacuate to a safe place based on emergency notifications displayed on the device.

[0183] An "autonomous mobile vehicle" is a vehicle that utilizes artificial intelligence and sensor technology to operate on its own judgment without external instructions.

[0184] "Means of controlling operation" refers to functions that manage the movement of autonomous mobile vehicles and ensure safe operation.

[0185] The "means for calculating routes" refer to a function that calculates the optimal evacuation route in the event of a disaster and instructs autonomous mobile vehicles accordingly.

[0186] "Means of transportation" refers to the ability of an autonomous vehicle to travel to its destination according to a calculated route.

[0187] This invention is a system that enables autonomous mobile vehicles to evacuate quickly and safely in the event of a natural disaster. Specific embodiments for realizing this system are described below.

[0188] 1. Data Collection

[0189] The terminal is equipped with various sensors, including a GPS module, seismometer, accelerometer, and water level sensor. These sensors periodically collect data and transmit it to a server via the network. For example, if an autonomous vehicle detects an earthquake, its accelerometer will acquire vibration data.

[0190] 2. Data transmission

[0191] The terminal periodically processes the collected sensor data in batches and sends it to the server via a secure communication protocol (such as HTTPS). It is recommended to use a high-speed and fault-tolerant communication method for this process.

[0192] 3. Data reception and storage

[0193] The server receives sensor data transmitted from terminals and stores it in a database. During this process, data verification and filtering are performed to remove duplicate and invalid data. For example, earthquake data transmitted simultaneously from multiple vehicles can be integrated.

[0194] 4. Data Analysis

[0195] The server analyzes the received data to determine the scale and location of the disaster. In the case of an earthquake, it uses algorithms such as FFT analysis to calculate the epicenter and seismic intensity. It also acquires data from the Japan Meteorological Agency and river management bureaus and integrates it with internal data to generate more accurate disaster information.

[0196] 5. Generate and send emergency notifications

[0197] The server generates an emergency notification based on the analysis results and integrated information. This notification includes evacuation instructions and recommended evacuation routes. The generated emergency notification is sent to each terminal via a push notification service. Based on the emergency notification received by the autonomous mobile vehicle, the vehicle's operation management system recalculates the route and selects the optimal evacuation route.

[0198] 6. Operation control of autonomous mobile vehicles

[0199] The autonomous mobile vehicle will automatically suspend its operation and move to a safe location in accordance with emergency notifications and recommended evacuation routes received from the server. During this process, in-vehicle sensors such as LiDAR and cameras will detect the surrounding environment to ensure safety during evacuation.

[0200] Specific example

[0201] For example, if an autonomous mobile vehicle detects an earthquake in central Tokyo, it would calculate the safest route to evacuate from that location and control the vehicle to move to a safe location. In this process, the system would need to monitor and avoid the conditions inside the vehicle and surrounding obstacles in real time.

[0202] Example of a prompt

[0203] "We are developing an application to enable the safe evacuation of autonomous mobile vehicles using a system that collects sensor data in real time during natural disasters and provides rapid emergency notifications. Data is collected from seismometers, accelerometers, water level sensors, etc., and transmitted to a server using a secure protocol. The server analyzes the data, generates an emergency notification, and sends it to the autonomous mobile vehicle. Based on the notification, the vehicle selects a safe evacuation route and moves to a safe location."

[0204] This embodiment enables autonomous mobile vehicles to evacuate quickly and safely in the event of a natural disaster, thereby protecting the lives and property of users.

[0205] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0206] Step 1: Data Collection

[0207] The device collects data from a GPS module, seismometer, accelerometer, and water level sensor. This allows it to obtain sensor data such as location information, seismic intensity, and surrounding water level information. The input consists of various sensor data, which is acquired in real time by the device. The output is the collected sensor data.

[0208] Step 2: Data transmission

[0209] The terminal periodically batches the collected sensor data and sends it to the server via a secure communication protocol (e.g., HTTPS). The input is the collected sensor data, which is formatted and encrypted for secure transmission to the server. The output is the transmitted sensor data arriving at the server.

[0210] Step 3: Data reception and storage

[0211] The server receives sensor data transmitted from the terminal and stores it in a database. During this process, the data is validated and filtered to remove duplicate and invalid data. The input is the transmitted sensor data, and the output is a clean and consistent dataset stored in the database.

[0212] Step 4: Data Analysis

[0213] The server analyzes sensor data stored in the database. Specifically, it analyzes earthquake vibration data using algorithms such as FFT analysis to determine the scale and location of the disaster. It also acquires additional data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with the internal data. The input is the stored sensor data and additional data, and the output is the analysis results that identify the scale and location of the disaster.

[0214] Step 5: Generate emergency notification

[0215] The server generates emergency notifications based on the analysis results and integrated information. These notifications include evacuation orders and recommended evacuation routes. The input is the data analysis results and integrated information, and the output is the generated emergency notification.

[0216] Step 6: Send emergency notification

[0217] The server sends the generated emergency notification to the device. This allows the user to receive important information in real time. The input is the generated emergency notification, and the output is displayed on the device as a push notification.

[0218] Step 7: Operation control of autonomous mobile vehicles

[0219] The terminal relays received emergency notifications to the autonomous mobile vehicle's operation management system, thereby controlling autonomous driving. The input is the received emergency notification, and the output is a recalculation of the vehicle's route, selecting a safe evacuation path.

[0220] Step 8: Evacuation Action

[0221] The user takes action based on the emergency notification displayed on their device. The autonomous mobile vehicle moves along a calculated route and evacuates to a safe location. The input is the selected evacuation route, and the output is the user's action of evacuating to a safe location.

[0222] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0223] This invention combines a system that collects and analyzes information in real time during natural disasters and provides users with rapid emergency notifications with an emotion engine that recognizes the user's emotions. This system consists of four main components: a terminal, a server, an emotion engine, and a user.

[0224] System Overview

[0225] 1. Data Collection

[0226] Device: Each user's device (e.g., smartphone) is equipped with a GPS module for acquiring location information, as well as seismometers, accelerometers, water level sensors, and other sensors. These sensors periodically collect data and transmit it to a server via the network.

[0227] 2. Data transmission

[0228] Terminal: Collected sensor data is periodically batch-processed and sent to the server via a secure communication protocol (e.g., HTTPS). The data is packetized in a format such as JSON and transmitted over the network.

[0229] 3. Data reception and storage

[0230] Server: The server receives data sent from terminals and stores it in the database. Data is validated and filtered upon receipt, removing duplicate and invalid data.

[0231] 4. Data Analysis

[0232] Server: The server analyzes the received data to determine the scale and location of the disaster. For example, if earthquake vibration data is transmitted, the server uses algorithms such as FFT analysis to calculate the epicenter and seismic intensity.

[0233] 5. Information Integration

[0234] Server: The server also acquires data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with internal data. This external data is retrieved using a REST API and stored in the server's database. This generates more detailed and accurate disaster information.

[0235] 6. Analysis using an emotion engine

[0236] Server: The emotion engine built into the server analyzes the user's voice and text messages to identify the user's emotional state. For example, it analyzes in real time what the user says to the device and the messages they type to detect stress or panic.

[0237] 7. Emergency notification generation

[0238] Server: Based on the analysis results of the emotion engine, the server generates emergency notifications considering the user's emotional state. Emergency notifications include evacuation orders, recommended evacuation routes, and psychological support information. The content and wording of the notifications are adjusted as needed.

[0239] 8. Send emergency notification

[0240] Server: Generates emergency notifications and sends them to each user's device using a push notification service (e.g., Firebase Cloud Messaging). The notification range is determined based on the target region and the user's current location.

[0241] 9. User behavior

[0242] User: The user receives an emergency notification from their device and checks it immediately. They begin evacuation actions according to the notification. The smartphone app displays evacuation shelters and recommended evacuation routes to help the user evacuate safely. Furthermore, the user can act calmly by referring to psychological support information provided by the emotion engine.

[0243] Specific examples

[0244] For example, consider the case of an earthquake. When the device's accelerometer detects a strong tremor, it immediately sends this data to the server. The server analyzes the data from multiple devices to calculate the epicenter and seismic intensity. The server then obtains additional data from the Japan Meteorological Agency to further determine the extent of the disaster's impact. When the emotion engine detects the user's stress level from their voice or text, the server generates an emergency notification that takes this into account and sends it to the user's device. The user checks this notification and evacuates to a safe place following the recommended evacuation route. At the same time, psychological support messages provided by the emotion engine help the user remain calm.

[0245] Furthermore, if a flood warning is issued, the device collects water level sensor data and sends it to the server. The server analyzes this data and predicts the likelihood of flooding. It integrates additional data (e.g., water level data from the river management authority) to determine the extent of the flood's impact. The emotion engine analyzes the user's emotional state and, if necessary, includes psychological support information to help them stay calm in the emergency notification. The user receives this notification and evacuates safely.

[0246] In this way, the present invention not only provides users with rapid and accurate information and supports appropriate responses during natural disasters, but also, by combining it with an emotion engine, takes into account the user's psychological state and provides more effective support.

[0247] The following describes the processing flow.

[0248] Step 1:

[0249] Terminals: Each terminal periodically collects sensor data (e.g., GPS location information, seismometer data). The smartphone's GPS module acquires location information, and the accelerometer collects ambient vibration data. This data is collected at regular time intervals and processed in batches.

[0250] Step 2:

[0251] Terminal: Collects sensor data and sends it to the server using a secure communication protocol (e.g., HTTPS). The data is packetized in a format such as JSON and transmitted over the network.

[0252] Step 3:

[0253] Server: The server receives data sent from the terminal. Data received via the API endpoint is immediately stored in the database. Data is validated and filtered upon receipt, removing duplicate and invalid data.

[0254] Step 4:

[0255] Server: Analyzes received data to determine the scale and location of the disaster. For example, if earthquake vibration data is transmitted, the server uses algorithms such as FFT analysis to calculate the epicenter and seismic intensity.

[0256] Step 5:

[0257] Server: Acquires additional data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with internal data. This external data is retrieved using a REST API and stored in the server's database. This generates more detailed and accurate disaster information.

[0258] Step 6:

[0259] Server: The emotion engine built into the server analyzes the user's voice and text messages to identify the user's emotional state. For example, it analyzes in real time what the user says to the device and the messages they type to detect stress or panic.

[0260] Step 7:

[0261] Server: Based on the analysis results of the emotion engine, the server generates emergency notifications considering the user's emotional state. Emergency notifications include evacuation orders, recommended evacuation routes, and psychological support information. The content and wording of the notifications are adjusted as needed.

[0262] Step 8:

[0263] Server: Generates emergency notifications and sends them to each user's device using a push notification service (e.g., Firebase Cloud Messaging). The notification range is determined based on the target region and the user's current location.

[0264] Step 9:

[0265] User: The user receives an emergency notification from their device and checks it immediately. They begin evacuation according to the notification's contents. The smartphone app displays evacuation shelters and recommended evacuation routes to help the user evacuate safely. They also act calmly, referring to psychological support information provided by the emotion engine.

[0266] The above outlines the specific program processing flow of the system that incorporates the emotion engine. This process enables a rapid and appropriate response in the event of a disaster, while simultaneously providing psychological support to the user.

[0267] (Example 2)

[0268] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0269] When natural disasters occur, there is a need for rapid and accurate information provision and appropriate responses that take into account the psychological state of users. Conventional systems do not adequately collect and analyze real-time data, making it difficult to provide support that takes into account the emotional state of users. In addition, emergency notifications are uniform, making it difficult to respond to the individual circumstances of users.

[0270] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting sensor data, means for analyzing the sensor data, means for acquiring and integrating additional data from other organizations, means for collecting user voice and text data and identifying their emotional state, means for generating an emergency notification based on the analysis results, integrated information, and the user's emotional state, and means for sending the emergency notification as a push notification to the terminal. This enables the rapid and accurate provision of disaster information and individualized responses that take into account the user's psychological state.

[0271] "Sensor data" refers to data collected from various sensors installed in the user's device (e.g., GPS module, seismometer, accelerometer, water level sensor).

[0272] A "server" is a remote computing device that receives, analyzes, and stores sensor data transmitted from terminals, acquires additional data from other organizations, and generates and transmits emergency notifications.

[0273] "Analysis" is the process of applying algorithms to received sensor data to extract meaningful information.

[0274] "Integration" refers to the process of combining data obtained from other organizations with internal data to create more detailed and accurate disaster information.

[0275] An "emotion engine" is software that analyzes a user's voice and text data to identify the user's emotional state.

[0276] An "emergency notification" is a message generated during a disaster that provides users with evacuation instructions and psychological support information.

[0277] "Push notifications" are a real-time notification format that a server sends directly to a user's device.

[0278] A "terminal" refers to a personal information terminal (e.g., a smartphone) used by a user, which is a device that collects sensor data and displays emergency notifications.

[0279] A "user" is a person who uses this system to receive emergency notifications and take action.

[0280]

[0281] This invention is a system that collects and analyzes information in real time during natural disasters and provides users with rapid emergency notifications. This system consists of four main components: a terminal, a server, an emotion engine, and a user.

[0282] First, the device is the user's smartphone, equipped with a GPS module, seismometer, accelerometer, water level sensor, and other sensors. These sensors periodically collect current location information, earthquake vibration data, and water level data. For example, when an earthquake occurs, the device's accelerometer detects the shaking and records the data.

[0283] Next, the collected sensor data is batch-processed at regular intervals and sent to the server using the HTTPS protocol. The data is sent in JSON format. Specifically, for example, seismometer data is converted into a JSON structure such as "{ 'timestamp': '2023-10-01T12:00:00Z', 'sensor': 'accelerometer', 'value': 5.6}" and sent as an HTTPS request. The server waits for reception at a specific endpoint and receives the data when it receives an HTTPS request from the terminal.

[0284] The server stores the received sensor data in a database and performs data validation and filtering. For example, it checks the timestamp and sensor type included in the received data to eliminate invalid data. The filtered data is then applied for analysis. Using FFT analysis and other algorithms, for example, it identifies the epicenter and seismic intensity from earthquake vibration data.

[0285] Furthermore, the server obtains additional data from other institutions (e.g., the Meteorological Agency, the River Management Bureau) and integrates it with the internal data. At this time, the REST API is used to obtain external data, and detailed and accurate disaster information after integration is generated. For example, an earthquake early warning is obtained from the Meteorological Agency and integrated with the data from the terminal to determine the affected area.

[0286] An emotion engine is also incorporated into the server to analyze the voice and text data input by the user into the terminal. This engine can identify the user's emotional state and detect stress and panic states. For example, when the user inputs "I'm afraid of earthquakes", the emotion engine detects the stress state from this message.

[0287] Based on the analysis results and the results of the emotion engine, the server generates an emergency notification. The emergency notification also includes evacuation instructions and psychological support information considering the user's emotional state, and the content is customized as needed. For example, in addition to the basic notification "Please evacuate", a psychological support message such as "Please act calmly" is also sent.

[0288] Finally, the server sends the generated emergency notification to the user's terminal using a push notification service such as Firebase Cloud Messaging. The notification is performed in real time, and the transmission range is set based on the target area and the user's current location. The user receives this notification and takes appropriate actions according to the displayed evacuation route and recommended actions.

[0289] Thereby, this system provides users with rapid and accurate disaster information and psychological support, and supports appropriate responses in an emergency. Considering the operation example of the system, examples of prompt texts using the generative AI model include "There is an earthquake. Please evacuate immediately. Let's act calmly."

[0290] The flow of the specific process in Example 2 will be described using FIG. 13.

[0291] Step 1: Data Collection

[0292] The device collects sensor data from the user's smartphone, including GPS modules, seismometers, accelerometers, and water level sensors. For example, when a seismometer detects shaking, it collects the data and stores it in a buffer. The input is data acquired from the sensor modules, and the output is raw data stored in the device's internal buffer.

[0293] Step 2: Convert Data Format

[0294] The terminal processes the collected sensor data in batches at regular time intervals and converts it to JSON format. For example, it converts seismometer data into a JSON structure. The input is raw data stored in the terminal's buffer, and the output is data formatted in JSON format.

[0295] Step 3: Data transmission

[0296] The terminal sends data converted to JSON format to the server using the HTTPS protocol. For example, the converted JSON data is sent to the server's endpoint via an HTTPS request. The input is data formatted in JSON format, and the output is the data sent to the server.

[0297] Step 4: Received

[0298] The server receives data sent from the terminal. For example, it receives HTTPS requests at a specific endpoint on the server. The input is the data sent from the terminal, and the output is the JSON data received by the server.

[0299] Step 5: Data validation and filtering

[0300] The server verifies the received data and removes illegal and duplicate data. For example, it checks the data timestamp and sensor type to exclude illegal and duplicate data. The input is the JSON data received by the server, and the output is the filtered and accurate data.

[0301] Step 6: Data storage

[0302] The server stores the filtered data in the database. For example, it adds the verified data to the MySQL database. The input is the filtered and accurate data, and the output is the data stored in the database.

[0303] Step 7: Data analysis

[0304] The server analyzes the received and stored data to identify the scale and location of the disaster. For example, it uses FFT analysis to identify the earthquake epicenter and intensity. The input is the sensor data stored in the database, and the output is the analysis result.

[0305] Step 8: Acquisition of external data

[0306] The server obtains additional data from external institutions (e.g., Meteorological Agency) via the REST API. For example, it uses the API of the Meteorological Agency to obtain the latest earthquake observation data. The input is the endpoint information of the external API, and the output is the obtained external data.

[0307] Step 9: Data integration

[0308] The server integrates the obtained external data and internal data to generate detailed and accurate disaster information. The input is the analysis result and external data, and the output is the integrated disaster information.

[0309] Step 10: Collection of sentiment data

[0310] The server collects voice and text data that the user inputs into the terminal. For example, the user inputs "I'm scared of earthquakes" into the terminal. The input is voice or text data from the user, and the output is collected emotion data.

[0311] Step 11: Analyzing emotional data

[0312] The server uses an emotion engine to analyze voice and text data to identify the user's emotional state. For example, it can detect a stressed state from the input text "scared." The input is the collected emotion data, and the output is the emotional state identified through the analysis.

[0313] Step 12: Generate an emergency notification

[0314] The server generates emergency notifications based on analysis results and emotional states. For example, it creates notification messages that include evacuation orders and psychological support information. The input is the analysis results and emotional states, and the output is the generated emergency notification.

[0315] Step 13: Preparing for push notifications

[0316] The server prepares push notifications using services such as Firebase Cloud Messaging. For example, it uses the Firebase API to create a list of target devices. The input is the generated emergency notification, and the output is the prepared push notification.

[0317] Step 14: Sending Push Notifications

[0318] The server sends the generated emergency notification to the user's device. For example, it might prioritize sending notifications to users in a specific region. The input is a prepared push notification, and the output is the emergency notification sent to the user's device.

[0319] Step 15: Check the notification

[0320] The user receives an emergency notification from their device and checks it immediately. For example, they might check a notification that says, "An earthquake has occurred. Please evacuate." The input is the emergency notification sent to the device, and the output is the user's action upon receiving the notification.

[0321] Step 16: Evacuation Action

[0322] The user initiates evacuation actions according to the notification. For example, they might move to a safe location using the notified evacuation route. The input is the notification content, and the output is the user's evacuation actions.

[0323] Step 17: Supporting calm behavior

[0324] The user acts calmly based on the psychological support information provided by the emotion engine. For example, they reduce stress by reading a message that says, "Please act calmly." The input is the support information from the emotion engine, and the output is the user acting calmly.

[0325] (Application Example 2)

[0326] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0327] In recent years, the frequency and scale of natural disasters have increased, creating a demand for rapid and accurate information provision and support. However, conventional systems struggle to efficiently integrate and analyze sensor data and data from external organizations, and furthermore, they are unable to provide emergency responses that take into account the emotional state of users. As a result, users are unable to take appropriate actions during disasters, leading to increased psychological burden. In addition, even in autonomous vehicles, providing real-time disaster information and appropriate responses based on emotional analysis are necessary to ensure passenger safety.

[0328] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting sensor data, means for transmitting the sensor data to the server, means for analyzing the sensor data, means for acquiring and integrating additional data from other organizations, means for generating emergency notifications based on the analysis results and integrated information, means for transmitting the emergency notifications to the terminal, means for analyzing the emotional state from the user's voice and facial expressions, and means for adjusting the content and expression of the emergency notification considering the emotional state. This enables not only the real-time and accurate provision of disaster information, but also appropriate notifications and support that respond to the user's emotions. Furthermore, by applying this to autonomous vehicles, passenger safety and a sense of psychological security can be ensured.

[0329] "Sensor data" refers to physical or environmental data collected by sensors, including location information, acceleration, vibration, water level, sound, and images.

[0330] "Means of sending to a server" refers to means that have the function of transferring collected sensor data to a server via the internet or other communication networks.

[0331] "Methods for data analysis" refer to methods of analyzing sensor data received on a server and using algorithms and computational techniques to determine the scale and location of a disaster.

[0332] "Means for acquiring and integrating additional data" refers to methods for acquiring data provided by external organizations and combining it with existing sensor data to generate more detailed information.

[0333] "Means for generating emergency notifications" refers to means that have the function of creating notification messages, including warnings and instructions for users, based on analyzed data and integrated information.

[0334] "Means of sending to a terminal" refers to means that have the function of sending the generated emergency notification to the user's communication device (e.g., smartphone, tablet, PC, etc.).

[0335] "Methods for analyzing emotional states from voice and facial expressions" refers to methods that have the function of detecting and analyzing a user's emotional state based on the user's voice and facial expression data.

[0336] "Means for adjusting the content and wording of emergency notifications in consideration of emotional state" refers to means that have the function of adjusting emergency notification messages to be more appropriate and effective based on the analyzed emotional state of the user.

[0337] "Means of taking action" refers to the means by which a user takes appropriate action or response measures based on an emergency notification displayed on their device.

[0338] "Means for obtaining current location" refers to means that have the function of obtaining the user's current physical location using location information technology such as GPS.

[0339] "Means of providing optimal evacuation routes" refers to a means that has the function of suggesting and directing the safest and most efficient evacuation route based on the user's current location and disaster information.

[0340] "Means for managing the location information of support resources" refers to means that have the function of centrally managing the location information of evacuation centers and relief supplies, and providing that information as needed.

[0341] "Means of providing support resources" refers to means of delivering information on evacuation sites and relief supplies to users and providing the maximum possible support.

[0342] Modes for carrying out the invention

[0343] The system according to the present invention collects and analyzes information in real time when a natural disaster occurs and provides users with a rapid emergency notification. This system consists of the following components.

[0344] 1. Terminal components and data collection

[0345] The device is equipped with various sensors (GPS, altimeter, accelerometer, microphone, camera, etc.) and collects data from these sensors in real time. For example, recent location information, earthquake vibration data, audio data, and video data are collected.

[0346] 2. Data transmission

[0347] The collected sensor data is securely transmitted to the server using the HTTPS communication protocol. The data is packetized in JSON format and transmitted periodically through batch processing.

[0348] 3. Data Analysis and Information Integration

[0349] The server analyzes received sensor data in real time. For example, it analyzes earthquake vibration data using FFT analysis to identify the epicenter and seismic intensity. Based on these results, it confirms the location and scale of the disaster. At the same time, it acquires and integrates additional data from other organizations (e.g., external data from the Japan Meteorological Agency and river management bureaus) using REST APIs to generate more detailed and accurate disaster information.

[0350] 4. Emotion analysis

[0351] The emotion engine embedded in the server analyzes audio and video data transmitted from the terminal to identify the user's emotional state in real time. Specifically, it uses audio and facial expression analysis algorithms to detect stress levels and panic states. Machine learning libraries such as Python, TENSORFLOW®, and Keras are used for emotion analysis.

[0352] 5. Generation and transmission of emergency notifications

[0353] The server generates emergency notifications based on analytical data, including the results of sentiment analysis. These emergency notifications include evacuation routes, psychological support information, and specific action instructions. The notifications are sent to the user's device using Firebase Cloud Messaging (FCM).

[0354] 6. User behavior and evacuation support

[0355] Users take action according to emergency notifications displayed on their devices. The device displays their current location and navigates them to the optimal evacuation route. It also provides location information for shelters and support resources to help users evacuate quickly and safely.

[0356] Specific examples

[0357] For example, in the event of an earthquake, the accelerometer in the device detects strong shaking and sends that data to a server. The server analyzes the data received from multiple devices to identify the epicenter and seismic intensity. Simultaneously, an emotion engine analyzes the user's voice and facial expressions to detect their stress level. Based on this, the server generates an emergency notification, which includes evacuation routes and psychological support information, and sends it to the user. The user can then receive this notification and take safe evacuation actions.

[0358] Examples of input prompts for a generative AI model

[0359] Write an algorithm to collect sensor data and send it to a server when an autonomous vehicle detects a strong tremor. Also, write code to generate emergency notifications based on the passenger's stress level and adjust the autonomous driving mode accordingly.

[0360] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0361] Step 1:

[0362] Data collection

[0363] The device uses sensors such as a GPS module, accelerometer, camera, and microphone to collect sensor data including current location information, vibration data, video data, and audio data. This data is collected in real time and temporarily stored in internal memory for batch processing. The main input is sensor data, and the output is batch-processed data.

[0364] Step 2:

[0365] Data transmission

[0366] The terminal transmits collected sensor data to the server using the HTTPS communication protocol. The data is packetized in JSON format and securely transferred to the server. The input is batch-processed sensor data, and the output is the data sent to the server.

[0367] Step 3:

[0368] Data reception and storage

[0369] The server receives sensor data transmitted from the terminal. Before the received data is stored in the database, it undergoes validation and filtering. Duplicate and invalid data are removed at this stage. The input is the received sensor data, and the output is the validated data stored in the database.

[0370] Step 4:

[0371] Data Analysis

[0372] The server analyzes sensor data stored in the database to determine the scale and location of a disaster. For example, it uses FFT analysis to calculate the earthquake's epicenter and intensity. The input is validated sensor data, and the output is the analysis results regarding the scale and location of the disaster.

[0373] Step 5:

[0374] Information integration

[0375] The server acquires additional data in real time from external sources. It uses a REST API to obtain weather information, river management information, and other data, and integrates it with internal sensor data. The input consists of additional data acquired from external sources and internal analysis results, and the output is integrated, detailed disaster information.

[0376] Step 6:

[0377] Emotion analysis

[0378] The emotion engine embedded in the server analyzes voice and facial expression data transmitted from the terminal. It uses machine learning models to identify the user's stress level and emotional state. The input is voice and facial expression data, and the output is the analyzed emotional state.

[0379] Step 7:

[0380] Emergency notification generation

[0381] The server generates emergency notifications based on analyzed disaster information and the user's emotional state. These notifications include evacuation orders, recommended evacuation routes, and psychological support information. The input is integrated disaster information and emotional state, while the output is the generated emergency notification.

[0382] Step 8:

[0383] Emergency notification sent

[0384] The server sends the generated emergency notification to each user's device using Firebase Cloud Messaging (FCM). The input is the emergency notification, and the output is the notification sent to the device.

[0385] Step 9:

[0386] User behavior

[0387] The user receives an emergency notification displayed on the device and takes evacuation action according to the instructions. The device displays the user's current location and the optimal evacuation route, guiding them along the way. The input is the received emergency notification, and the output is the user's evacuation action. Psychological support information is also provided to help the user act calmly.

[0388] In this way, the system can provide users with rapid and accurate information and psychological support through multi-layered data collection and analysis.

[0389] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0390] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0391] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0392] [Second Embodiment]

[0393] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0394] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0395] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0396] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0397] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0398] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0399] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0400] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0401] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0402] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0403] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0404] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0405] This invention is a system that collects and analyzes information in real time during natural disasters and provides users with rapid emergency notifications. This system consists of three main components: a terminal, a server, and a user.

[0406] System Overview

[0407] 1. Data Collection

[0408] Device: Each user's device (e.g., smartphone) is equipped with a GPS module for acquiring location information, as well as seismometers, accelerometers, water level sensors, and other sensors. These sensors periodically collect data and transmit it to a server via the network.

[0409] 2. Data transmission

[0410] Terminal: Collected sensor data is periodically batch-processed and sent to the server via a secure communication protocol (e.g., HTTPS).

[0411] 3. Data reception and storage

[0412] Server: The server receives data sent from terminals and stores it in the database. During this process, data validation and filtering are also performed to remove duplicate and invalid data.

[0413] 4. Data Analysis

[0414] Server: The server analyzes the received data to determine the scale and location of the disaster. For example, it analyzes earthquake vibration data to determine the epicenter and seismic intensity. This analysis is performed using algorithms (e.g., FFT analysis).

[0415] 5. Information Integration

[0416] Server: The server also acquires data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with its internal data. This generates more accurate and detailed disaster information.

[0417] 6. Emergency notification generation

[0418] Server: The server generates emergency notifications for each user based on the analysis results and integrated information. These emergency notifications include evacuation orders and recommended evacuation routes.

[0419] 7. Send emergency notification

[0420] Server: The generated emergency notification is sent to each user's device via a push notification service (e.g., Firebase Cloud Messaging).

[0421] 8. User behavior

[0422] User: The user receives an emergency notification from their device and begins evacuation. They check the information provided by the device (e.g., evacuation orders, evacuation routes) and move to a safe location.

[0423] Specific examples

[0424] For example, consider the case of an earthquake. When the device's accelerometer detects a strong tremor, it immediately sends this data to the server. The server analyzes the data from multiple devices to calculate the epicenter and seismic intensity. The server then obtains additional data from the Japan Meteorological Agency to further determine the extent of the disaster's impact. Finally, the server generates an emergency notification and sends a push notification to the user's device. The user checks this notification and evacuates to a safe place according to the recommended evacuation route.

[0425] Furthermore, if a flood warning is issued, the terminal collects water level sensor data and sends it to the server. The server analyzes this data and predicts the likelihood of flooding. It integrates additional data (e.g., water level data from the river management authority) to determine the extent of the flood's impact. The generated emergency notification is sent to users in the affected area as a notification including evacuation instructions. Users see this notification and take action to evacuate quickly and safely.

[0426] In this way, the present invention provides users with rapid and accurate information and supports appropriate responses during natural disasters.

[0427] The following describes the processing flow.

[0428] Step 1:

[0429] Terminals: Each terminal periodically collects sensor data (e.g., GPS location information, seismometer data). The smartphone's GPS module acquires location information, and the accelerometer collects ambient vibration data. This data is collected at regular time intervals and processed in batches.

[0430] Step 2:

[0431] Terminal: Collects sensor data and sends it to the server using a secure communication protocol (e.g., HTTPS). The data is packetized in a format such as JSON and transmitted over the network.

[0432] Step 3:

[0433] Server: The server receives data sent from the terminal. Data received via the API endpoint is immediately stored in the database. Data is validated and filtered upon receipt to remove duplicate and invalid data.

[0434] Step 4:

[0435] Server: Analyzes received data to determine the scale and location of the disaster. For example, if earthquake vibration data is transmitted, the server uses algorithms such as FFT analysis to calculate the epicenter and seismic intensity.

[0436] Step 5:

[0437] Server: Acquires additional data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with internal data. This external data is retrieved using a REST API and stored in the server's database. This generates more detailed and accurate disaster information.

[0438] Step 6:

[0439] Server: Based on analysis results and integrated information, generates emergency notifications to send to users. These emergency notifications include evacuation orders and recommended evacuation routes. The generated notifications are formatted using templates.

[0440] Step 7:

[0441] Server: Generates emergency notifications and sends them to each user's device using a push notification service (e.g., Firebase Cloud Messaging). The notification range is determined based on the target region and the user's current location.

[0442] Step 8:

[0443] User: The user receives an emergency notification from their device and checks it immediately. They begin evacuation according to the notification's contents. The smartphone app displays evacuation shelters and recommended evacuation routes to help the user evacuate safely.

[0444] The above outlines the specific flow of the system's program processing. This process enables a swift and appropriate response in the event of a disaster.

[0445] (Example 1)

[0446] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0447] Conventional disaster notification systems lacked real-time capabilities in collecting and analyzing sensor data, making it difficult to provide users with timely emergency notifications. Furthermore, the integration of data from multiple sources was insufficient, preventing improvements in the accuracy and detail of disaster information. As a result, users were unable to take appropriate evacuation actions quickly, increasing the risk of disaster damage.

[0448] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0449] In this invention, the server includes means for verifying, filtering, and storing sensor data; means for analyzing the sensor data using FFT analysis; and means for acquiring and integrating additional data from other organizations. This enables the generation of highly accurate and detailed disaster information and the rapid provision of emergency notifications to users through real-time analysis of sensor data and integration of data from multiple organizations.

[0450] "Sensor data" refers to data collected by various sensors installed in the device (e.g., GPS module, seismometer, accelerometer, water level sensor).

[0451] A "server" is a central computer system that receives, stores, and analyzes sensor data, and integrates additional data from other organizations.

[0452] "Verification and filtering" is the process of checking data sent to the server from the perspectives of accuracy, duplication, and fraud, and saving only the appropriate data.

[0453] "FFT analysis" refers to the Fast Fourier Transform algorithm used by servers to analyze sensor data, and is particularly used for vibration analysis of earthquake data.

[0454] "Additional data from other organizations" refers to supplementary data obtained from external organizations such as the Japan Meteorological Agency and river management bureaus, which are integrated with sensor data.

[0455] An "emergency notification" is a notification message, including warnings and evacuation orders, that is generated based on analysis results and integrated information when a disaster occurs.

[0456] A "communication network" is the infrastructure used to send and receive digital data, such as the internet or dedicated lines.

[0457] A "terminal" refers to a device, such as a smartphone or tablet, that collects sensor data and receives emergency notifications.

[0458] "Means of taking action" refers to the user checking emergency notifications from their device and then acting according to the designated evacuation instructions and evacuation routes.

[0459] This invention is a system that collects and analyzes information in real time during natural disasters and provides users with rapid emergency notifications. This system consists of three main components: a terminal, a server, and a user. Details of each are described below.

[0460] 1. Data Collection

[0461] Device: The user's device (e.g., smartphone) collects data using various sensors such as GPS modules, seismometers, accelerometers, and water level sensors. These sensors operate continuously and measure data at regular intervals. For example, a smartphone's accelerometer captures tremor information at a frequency of 5 times per second.

[0462] 2. Data transmission

[0463] Terminal: The collected sensor data is periodically batch-processed and securely sent to the server via the HTTPS protocol. For example, the terminal sends the data it collects to the server via HTTPS every 5 minutes.

[0464] 3. Data reception and storage

[0465] Server: The server receives data sent from the terminal and stores it in a database using Python and MySQL. During this process, it validates and filters the data to remove invalid or duplicate data.

[0466] 4. Data Analysis

[0467] Server: The server analyzes the received data to determine the scale and location of the disaster. In particular, in the case of earthquakes, it uses FFT (Fast Fourier Transform) analysis to calculate the epicenter and seismic intensity. The server identifies the epicenter and seismic intensity by performing FFT analysis on the vibration data it receives.

[0468] 5. Information Integration

[0469] Server: The server also acquires data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with its internal data. This generates more accurate and detailed information about disasters. It obtains the latest weather data from the Japan Meteorological Agency's API and integrates it with its own data.

[0470] 6. Emergency notification generation

[0471] Server: The server generates emergency notifications based on analysis results and integrated information. These notifications include evacuation orders and recommended evacuation routes. The server creates emergency notifications containing evacuation orders and routes based on the specified format.

[0472] 7. Send emergency notification

[0473] Server: Generated emergency notifications are pushed to each user's device using Firebase Cloud Messaging. The server calls Firebase Cloud Messaging to deliver the generated emergency notifications to all users' devices.

[0474] 8. User behavior

[0475] User: The user checks the emergency notification received on their device and begins evacuation. The notification includes detailed information on evacuation routes and locations, and the user evacuates to a safe place according to these instructions. The user checks the notification on their smartphone and heads to the nearest evacuation center according to the displayed evacuation instructions and evacuation route.

[0476] Specific example

[0477] For example, in the event of an earthquake, if the device's accelerometer detects strong shaking, it immediately sends this data to the server. The server analyzes the data from multiple devices to calculate the epicenter and seismic intensity. The server then obtains additional data from the Japan Meteorological Agency to further determine the extent of the disaster's impact. Finally, the server generates an emergency notification and sends a push notification to the user's device. The user checks this notification and evacuates to a safe place following the recommended evacuation route.

[0478] Example of a prompt

[0479] "An earthquake has occurred. Please evacuate to a safe place immediately. The nearest evacuation center is point A. Taking route B will ensure your safety."

[0480] Thus, the present invention provides users with rapid and accurate information and supports appropriate responses during natural disasters.

[0481] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0482] Step 1:

[0483] Data collection

[0484] Device: The user's device (smartphone) collects data in real time using its built-in GPS module, seismometer, accelerometer, and water level sensor.

[0485] Input: Measurement data from the sensor.

[0486] Data processing: Batch processing of data within the terminal (e.g., grouping data at regular intervals).

[0487] Output: Batch-processed sensor data.

[0488] Specific operation: The smartphone's accelerometer captures and temporarily stores information about shaking at a frequency of 5 times per second.

[0489] Step 2:

[0490] Data transmission

[0491] Terminal: Collected sensor data is periodically sent to the server via the HTTPS protocol.

[0492] Input: Batch-processed sensor data.

[0493] Data processing: Data encryption and secure transmission.

[0494] Output: Encrypted sensor data is sent to the server.

[0495] Specific operation: Every 5 minutes, the device sends the collected data to the server via HTTPS.

[0496] Step 3:

[0497] Data reception and storage

[0498] Server: Receives data sent from terminals, verifies and filters the data to ensure its accuracy, and then stores it in a database.

[0499] Input: Transmitted sensor data.

[0500] Data processing: Verification (e.g., checking data accuracy) and filtering (e.g., removing duplicate data).

[0501] Output: Clean sensor data is saved to the database.

[0502] Specific operation: The server uses Python and MySQL to validate the data and then save it to the database.

[0503] Step 4:

[0504] Data Analysis

[0505] Server: Analyzes stored data to determine the scale and location of disasters. In particular, in the case of earthquakes, it uses FFT (Fast Fourier Transform) analysis to calculate the epicenter and seismic intensity.

[0506] Input: Stored sensor data.

[0507] Data processing: Perform FFT analysis.

[0508] Output: Analysis results such as epicenter and seismic intensity.

[0509] Specific operation: The server applies FFT analysis to the vibration data it receives to identify the epicenter.

[0510] Step 5:

[0511] Information integration

[0512] Server: Acquires additional data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with internal data.

[0513] Input: Analysis results and additional data from external organizations.

[0514] Data processing: Data integration and analysis.

[0515] Output: Even more accurate disaster information.

[0516] Specific operation: Obtain the latest earthquake information from the Japan Meteorological Agency's API and integrate it with our own data.

[0517] Step 6:

[0518] Emergency notification generation

[0519] Server: Based on analysis results and integrated information, it generates optimized emergency notifications for each user (including evacuation orders and recommended evacuation routes).

[0520] Input: Analysis results and integrated information.

[0521] Data processing: Creating emergency notifications.

[0522] Output: Emergency notification message.

[0523] Specific operation: The server creates an emergency notification, including evacuation instructions and evacuation routes, based on the specified format.

[0524] Step 7:

[0525] Emergency notification sent

[0526] Server: The generated emergency notification is pushed to each user's device using Firebase Cloud Messaging.

[0527] Input: Generated emergency notification message.

[0528] Data processing: Sending notification messages.

[0529] Output: Emergency notification sent to the user's device.

[0530] Specific operation: The server calls Firebase Cloud Messaging and delivers the generated emergency notification to all users' devices.

[0531] Step 8:

[0532] User behavior

[0533] User: The user checks the emergency notification received on their device and begins evacuation according to the instructions in the notification.

[0534] Input: Emergency notification displayed on the device.

[0535] Output: User evacuation actions.

[0536] Specific actions: The user checks the notification on their smartphone and heads to the nearest evacuation shelter according to the displayed evacuation instructions and evacuation route.

[0537] (Application Example 1)

[0538] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0539] While systems exist that provide real-time information to help users evacuate quickly and safely during natural disasters, these systems generally assume that users will move manually. Therefore, especially in urban areas with a large number of autonomous vehicles, there is a lack of efficient and immediate means of evacuation. To overcome this drawback, a method is needed that automatically controls autonomous vehicles during natural disasters to select appropriate evacuation routes and guide them to safe locations.

[0540] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0541] In this invention, the server includes means for collecting sensor data, means for transmitting sensor data to the server, means for analyzing sensor data, means for acquiring and integrating additional data from other organizations, means for generating emergency notifications based on the analysis results and integrated information, means for transmitting emergency notifications to a terminal, means for the user to take action in response to the emergency notification displayed on the terminal, means for controlling the operation of the autonomous mobile vehicle and calculating a route for safe evacuation, and means for the autonomous mobile vehicle to move according to the calculated evacuation route. This enables the autonomous mobile vehicle to perform appropriate evacuation actions quickly and safely.

[0542] "Sensor data" refers to data collected from a wide variety of sensors (e.g., GPS, seismometers, accelerometers, water level sensors).

[0543] A "server" is a computer system that receives and analyzes sensor data via a network, and processes information by integrating additional data from other organizations.

[0544] "Means of sending to the server" refers to the function of securely and efficiently transferring collected sensor data to the server.

[0545] "Means of analysis" refers to methods of evaluating the scale, location, and impact of a disaster using collected sensor data.

[0546] "Methods for acquiring and integrating additional data from other organizations" refers to methods of collecting data from organizations such as the Japan Meteorological Agency and river management bureaus, and combining it with the data analyzed on the server.

[0547] "Means for generating emergency notifications" refers to a method of creating emergency information (e.g., evacuation orders, recommended evacuation routes) that should be conveyed to users, based on analysis results and integrated information.

[0548] "Means of sending emergency notifications to devices" refers to a function that transmits generated emergency notifications to user devices such as smartphones and tablets.

[0549] "Means of action for the user" refers to a function that facilitates actions to evacuate to a safe place based on emergency notifications displayed on the device.

[0550] An "autonomous mobile vehicle" is a vehicle that utilizes artificial intelligence and sensor technology to operate on its own judgment without external instructions.

[0551] "Means of controlling operation" refers to functions that manage the movement of autonomous mobile vehicles and ensure safe operation.

[0552] The "means for calculating routes" refer to a function that calculates the optimal evacuation route in the event of a disaster and instructs autonomous mobile vehicles accordingly.

[0553] "Means of transportation" refers to the ability of an autonomous vehicle to travel to its destination according to a calculated route.

[0554] This invention is a system that enables autonomous mobile vehicles to evacuate quickly and safely in the event of a natural disaster. Specific embodiments for realizing this system are described below.

[0555] 1. Data Collection

[0556] The terminal is equipped with various sensors, including a GPS module, seismometer, accelerometer, and water level sensor. These sensors periodically collect data and transmit it to a server via the network. For example, if an autonomous vehicle detects an earthquake, its accelerometer will acquire vibration data.

[0557] 2. Data transmission

[0558] The terminal periodically processes the collected sensor data in batches and sends it to the server via a secure communication protocol (such as HTTPS). It is recommended to use a high-speed and fault-tolerant communication method for this process.

[0559] 3. Data reception and storage

[0560] The server receives sensor data transmitted from terminals and stores it in a database. During this process, data verification and filtering are performed to remove duplicate and invalid data. For example, earthquake data transmitted simultaneously from multiple vehicles can be integrated.

[0561] 4. Data Analysis

[0562] The server analyzes the received data to determine the scale and location of the disaster. In the case of an earthquake, it uses algorithms such as FFT analysis to calculate the epicenter and seismic intensity. It also acquires data from the Japan Meteorological Agency and river management bureaus and integrates it with internal data to generate more accurate disaster information.

[0563] 5. Generate and send emergency notifications

[0564] The server generates an emergency notification based on the analysis results and integrated information. This notification includes evacuation instructions and recommended evacuation routes. The generated emergency notification is sent to each terminal via a push notification service. Based on the emergency notification received by the autonomous mobile vehicle, the vehicle's operation management system recalculates the route and selects the optimal evacuation route.

[0565] 6. Operation control of autonomous mobile vehicles

[0566] The autonomous mobile vehicle will automatically suspend its operation and move to a safe location in accordance with emergency notifications and recommended evacuation routes received from the server. During this process, in-vehicle sensors such as LiDAR and cameras will detect the surrounding environment to ensure safety during evacuation.

[0567] Specific example

[0568] For example, if an autonomous mobile vehicle detects an earthquake in central Tokyo, it would calculate the safest route to evacuate from that location and control the vehicle to move to a safe location. In this process, the system would need to monitor and avoid the conditions inside the vehicle and surrounding obstacles in real time.

[0569] Example of a prompt

[0570] "We are developing an application to enable the safe evacuation of autonomous mobile vehicles using a system that collects sensor data in real time during natural disasters and provides rapid emergency notifications. Data is collected from seismometers, accelerometers, water level sensors, etc., and transmitted to a server using a secure protocol. The server analyzes the data, generates an emergency notification, and sends it to the autonomous mobile vehicle. Based on the notification, the vehicle selects a safe evacuation route and moves to a safe location."

[0571] This embodiment enables autonomous mobile vehicles to evacuate quickly and safely in the event of a natural disaster, thereby protecting the lives and property of users.

[0572] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0573] Step 1: Data Collection

[0574] The device collects data from a GPS module, seismometer, accelerometer, and water level sensor. This allows it to obtain sensor data such as location information, seismic intensity, and surrounding water level information. The input consists of various sensor data, which is acquired in real time by the device. The output is the collected sensor data.

[0575] Step 2: Data transmission

[0576] The terminal periodically batches the collected sensor data and sends it to the server via a secure communication protocol (e.g., HTTPS). The input is the collected sensor data, which is formatted and encrypted for secure transmission to the server. The output is the transmitted sensor data arriving at the server.

[0577] Step 3: Data reception and storage

[0578] The server receives sensor data transmitted from the terminal and stores it in a database. During this process, the data is validated and filtered to remove duplicate and invalid data. The input is the transmitted sensor data, and the output is a clean and consistent dataset stored in the database.

[0579] Step 4: Data Analysis

[0580] The server analyzes sensor data stored in the database. Specifically, it analyzes earthquake vibration data using algorithms such as FFT analysis to determine the scale and location of the disaster. It also acquires additional data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with the internal data. The input is the stored sensor data and additional data, and the output is the analysis results that identify the scale and location of the disaster.

[0581] Step 5: Generate emergency notification

[0582] The server generates emergency notifications based on the analysis results and integrated information. These notifications include evacuation orders and recommended evacuation routes. The input is the data analysis results and integrated information, and the output is the generated emergency notification.

[0583] Step 6: Send emergency notification

[0584] The server sends the generated emergency notification to the device. This allows the user to receive important information in real time. The input is the generated emergency notification, and the output is displayed on the device as a push notification.

[0585] Step 7: Operation control of autonomous mobile vehicles

[0586] The terminal relays received emergency notifications to the autonomous mobile vehicle's operation management system, thereby controlling autonomous driving. The input is the received emergency notification, and the output is a recalculation of the vehicle's route, selecting a safe evacuation path.

[0587] Step 8: Evacuation Action

[0588] The user takes action based on the emergency notification displayed on their device. The autonomous mobile vehicle moves along a calculated route and evacuates to a safe location. The input is the selected evacuation route, and the output is the user's action of evacuating to a safe location.

[0589] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0590] This invention combines a system that collects and analyzes information in real time during natural disasters and provides users with rapid emergency notifications with an emotion engine that recognizes the user's emotions. This system consists of four main components: a terminal, a server, an emotion engine, and a user.

[0591] System Overview

[0592] 1. Data Collection

[0593] Device: Each user's device (e.g., smartphone) is equipped with a GPS module for acquiring location information, as well as seismometers, accelerometers, water level sensors, and other sensors. These sensors periodically collect data and transmit it to a server via the network.

[0594] 2. Data transmission

[0595] Terminal: Collected sensor data is periodically batch-processed and sent to the server via a secure communication protocol (e.g., HTTPS). The data is packetized in a format such as JSON and transmitted over the network.

[0596] 3. Data reception and storage

[0597] Server: The server receives data sent from terminals and stores it in the database. Data is validated and filtered upon receipt, removing duplicate and invalid data.

[0598] 4. Data Analysis

[0599] Server: The server analyzes the received data to determine the scale and location of the disaster. For example, if earthquake vibration data is transmitted, the server uses algorithms such as FFT analysis to calculate the epicenter and seismic intensity.

[0600] 5. Information Integration

[0601] Server: The server also acquires data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with internal data. This external data is retrieved using a REST API and stored in the server's database. This generates more detailed and accurate disaster information.

[0602] 6. Analysis using an emotion engine

[0603] Server: The emotion engine built into the server analyzes the user's voice and text messages to identify the user's emotional state. For example, it analyzes in real time what the user says to the device and the messages they type to detect stress or panic.

[0604] 7. Emergency notification generation

[0605] Server: Based on the analysis results of the emotion engine, the server generates emergency notifications considering the user's emotional state. Emergency notifications include evacuation orders, recommended evacuation routes, and psychological support information. The content and wording of the notifications are adjusted as needed.

[0606] 8. Send emergency notification

[0607] Server: Generates emergency notifications and sends them to each user's device using a push notification service (e.g., Firebase Cloud Messaging). The notification range is determined based on the target region and the user's current location.

[0608] 9. User behavior

[0609] User: The user receives an emergency notification from their device and checks it immediately. They begin evacuation actions according to the notification. The smartphone app displays evacuation shelters and recommended evacuation routes to help the user evacuate safely. Furthermore, the user can act calmly by referring to psychological support information provided by the emotion engine.

[0610] Specific examples

[0611] For example, consider the case of an earthquake. When the device's accelerometer detects a strong tremor, it immediately sends this data to the server. The server analyzes the data from multiple devices to calculate the epicenter and seismic intensity. The server then obtains additional data from the Japan Meteorological Agency to further determine the extent of the disaster's impact. When the emotion engine detects the user's stress level from their voice or text, the server generates an emergency notification that takes this into account and sends it to the user's device. The user checks this notification and evacuates to a safe place following the recommended evacuation route. At the same time, psychological support messages provided by the emotion engine help the user remain calm.

[0612] Furthermore, if a flood warning is issued, the device collects water level sensor data and sends it to the server. The server analyzes this data and predicts the likelihood of flooding. It integrates additional data (e.g., water level data from the river management authority) to determine the extent of the flood's impact. The emotion engine analyzes the user's emotional state and, if necessary, includes psychological support information to help them stay calm in the emergency notification. The user receives this notification and evacuates safely.

[0613] In this way, the present invention not only provides users with rapid and accurate information and supports appropriate responses during natural disasters, but also, by combining it with an emotion engine, takes into account the user's psychological state and provides more effective support.

[0614] The following describes the processing flow.

[0615] Step 1:

[0616] Terminals: Each terminal periodically collects sensor data (e.g., GPS location information, seismometer data). The smartphone's GPS module acquires location information, and the accelerometer collects ambient vibration data. This data is collected at regular time intervals and processed in batches.

[0617] Step 2:

[0618] Terminal: Collects sensor data and sends it to the server using a secure communication protocol (e.g., HTTPS). The data is packetized in a format such as JSON and transmitted over the network.

[0619] Step 3:

[0620] Server: The server receives data sent from the terminal. Data received via the API endpoint is immediately stored in the database. Data is validated and filtered upon receipt, removing duplicate and invalid data.

[0621] Step 4:

[0622] Server: Analyzes received data to determine the scale and location of the disaster. For example, if earthquake vibration data is transmitted, the server uses algorithms such as FFT analysis to calculate the epicenter and seismic intensity.

[0623] Step 5:

[0624] Server: Acquires additional data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with internal data. This external data is retrieved using a REST API and stored in the server's database. This generates more detailed and accurate disaster information.

[0625] Step 6:

[0626] Server: The emotion engine built into the server analyzes the user's voice and text messages to identify the user's emotional state. For example, it analyzes in real time what the user says to the device and the messages they type to detect stress or panic.

[0627] Step 7:

[0628] Server: Based on the analysis results of the emotion engine, the server generates emergency notifications considering the user's emotional state. Emergency notifications include evacuation orders, recommended evacuation routes, and psychological support information. The content and wording of the notifications are adjusted as needed.

[0629] Step 8:

[0630] Server: Generates emergency notifications and sends them to each user's device using a push notification service (e.g., Firebase Cloud Messaging). The notification range is determined based on the target region and the user's current location.

[0631] Step 9:

[0632] User: The user receives an emergency notification from their device and checks it immediately. They begin evacuation according to the notification's contents. The smartphone app displays evacuation shelters and recommended evacuation routes to help the user evacuate safely. They also act calmly, referring to psychological support information provided by the emotion engine.

[0633] The above outlines the specific program processing flow of the system that incorporates the emotion engine. This process enables a rapid and appropriate response in the event of a disaster, while simultaneously providing psychological support to the user.

[0634] (Example 2)

[0635] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0636] When natural disasters occur, there is a need for rapid and accurate information provision and appropriate responses that take into account the psychological state of users. Conventional systems do not adequately collect and analyze real-time data, making it difficult to provide support that takes into account the emotional state of users. In addition, emergency notifications are uniform, making it difficult to respond to the individual circumstances of users.

[0637] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting sensor data, means for analyzing the sensor data, means for acquiring and integrating additional data from other organizations, means for collecting user voice and text data and identifying their emotional state, means for generating an emergency notification based on the analysis results, integrated information, and the user's emotional state, and means for sending the emergency notification as a push notification to the terminal. This enables the rapid and accurate provision of disaster information and individualized responses that take into account the user's psychological state.

[0638] "Sensor data" refers to data collected from various sensors installed in the user's device (e.g., GPS module, seismometer, accelerometer, water level sensor).

[0639] A "server" is a remote computing device that receives, analyzes, and stores sensor data transmitted from terminals, acquires additional data from other organizations, and generates and transmits emergency notifications.

[0640] "Analysis" is the process of applying algorithms to received sensor data to extract meaningful information.

[0641] "Integration" refers to the process of combining data obtained from other organizations with internal data to create more detailed and accurate disaster information.

[0642] An "emotion engine" is software that analyzes a user's voice and text data to identify the user's emotional state.

[0643] An "emergency notification" is a message generated during a disaster that provides users with evacuation instructions and psychological support information.

[0644] "Push notifications" are a real-time notification format that a server sends directly to a user's device.

[0645] A "terminal" refers to a personal information terminal (e.g., a smartphone) used by a user, which is a device that collects sensor data and displays emergency notifications.

[0646] A "user" is a person who uses this system to receive emergency notifications and take action.

[0647]

[0648] This invention is a system that collects and analyzes information in real time during natural disasters and provides users with rapid emergency notifications. This system consists of four main components: a terminal, a server, an emotion engine, and a user.

[0649] First, the device is the user's smartphone, equipped with a GPS module, seismometer, accelerometer, water level sensor, and other sensors. These sensors periodically collect current location information, earthquake vibration data, and water level data. For example, when an earthquake occurs, the device's accelerometer detects the shaking and records the data.

[0650] Next, the collected sensor data is batch-processed at regular intervals and sent to the server using the HTTPS protocol. The data is sent in JSON format. Specifically, for example, seismometer data is converted into a JSON structure such as "{ 'timestamp': '2023-10-01T12:00:00Z', 'sensor': 'accelerometer', 'value': 5.6}" and sent as an HTTPS request. The server waits for reception at a specific endpoint and receives the data when it receives an HTTPS request from the terminal.

[0651] The server stores the received sensor data in a database and performs data validation and filtering. For example, it checks the timestamp and sensor type included in the received data to eliminate invalid data. The filtered data is then applied for analysis. Using FFT analysis and other algorithms, for example, it identifies the epicenter and seismic intensity from earthquake vibration data.

[0652] Furthermore, the server acquires additional data from other organizations (e.g., the Japan Meteorological Agency, river management bureaus) and integrates it with internal data. At this time, it uses a REST API to retrieve external data and generates detailed and accurate disaster information after integration. For example, it acquires earthquake reports from the Japan Meteorological Agency and integrates them with data from terminals to determine the affected area.

[0653] An emotion engine is also integrated into the server, analyzing the voice and text data that users input into their devices. This engine can identify the user's emotional state and detect stress or panic. For example, if a user inputs "I'm scared of earthquakes," the emotion engine will detect a stressed state from that message.

[0654] The server generates emergency notifications based on analysis results and the results of the emotion engine. These emergency notifications include evacuation instructions and psychological support information that takes the user's emotional state into consideration, and the content is customized as needed. For example, in addition to a basic notification such as "Please evacuate," it also sends psychological support messages such as "Please stay calm."

[0655] Finally, the server sends the generated emergency notification to the user's device using a push notification service such as Firebase Cloud Messaging. The notification is sent in real time, and the delivery range is set based on the target area and the user's current location. The user receives this notification and takes appropriate action according to the displayed evacuation route and recommended actions.

[0656] This system provides users with rapid and accurate disaster information and psychological support, assisting them in taking appropriate action during emergencies. An example of a prompt message generated using the AI ​​model is: "There is an earthquake. Please evacuate immediately. Please remain calm."

[0657] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0658] Step 1: Data Collection

[0659] The device collects sensor data from the user's smartphone, including GPS modules, seismometers, accelerometers, and water level sensors. For example, when a seismometer detects shaking, it collects the data and stores it in a buffer. The input is data acquired from the sensor modules, and the output is raw data stored in the device's internal buffer.

[0660] Step 2: Convert Data Format

[0661] The terminal processes the collected sensor data in batches at regular time intervals and converts it to JSON format. For example, it converts seismometer data into a JSON structure. The input is raw data stored in the terminal's buffer, and the output is data formatted in JSON format.

[0662] Step 3: Data transmission

[0663] The terminal sends data converted to JSON format to the server using the HTTPS protocol. For example, the converted JSON data is sent to the server's endpoint via an HTTPS request. The input is data formatted in JSON format, and the output is the data sent to the server.

[0664] Step 4: Received

[0665] The server receives data sent from the terminal. For example, it receives HTTPS requests at a specific endpoint on the server. The input is the data sent from the terminal, and the output is the JSON data received by the server.

[0666] Step 5: Data validation and filtering

[0667] The server validates the received data and removes invalid and duplicate data. For example, it checks the data's timestamp and sensor type to eliminate invalid and duplicate data. The input is the JSON data received by the server, and the output is the filtered, accurate data.

[0668] Step 6: Save Data

[0669] The server stores the filtered data in a database. For example, it adds verified data to a MySQL database. The input is the filtered, accurate data, and the output is the data stored in the database.

[0670] Step 7: Data Analysis

[0671] The server analyzes the received and stored data to determine the scale and location of a disaster. For example, it uses FFT analysis to identify the epicenter and seismic intensity. The input is sensor data stored in a database, and the output is the analysis results.

[0672] Step 8: Obtaining external data

[0673] The server retrieves additional data from external organizations (e.g., the Japan Meteorological Agency) via a REST API. For example, it uses the Japan Meteorological Agency's API to retrieve the latest earthquake observation data. The input is the endpoint information of the external API, and the output is the retrieved external data.

[0674] Step 9: Data Integration

[0675] The server integrates acquired external and internal data to generate detailed and accurate disaster information. Inputs are analysis results and external data, while output is the integrated disaster information.

[0676] Step 10: Collecting emotional data

[0677] The server collects voice and text data that the user inputs into the terminal. For example, the user inputs "I'm scared of earthquakes" into the terminal. The input is voice or text data from the user, and the output is collected emotion data.

[0678] Step 11: Analyzing emotional data

[0679] The server uses an emotion engine to analyze voice and text data to identify the user's emotional state. For example, it can detect a stressed state from the input text "scared." The input is the collected emotion data, and the output is the emotional state identified through the analysis.

[0680] Step 12: Generate an emergency notification

[0681] The server generates emergency notifications based on analysis results and emotional states. For example, it creates notification messages that include evacuation orders and psychological support information. The input is the analysis results and emotional states, and the output is the generated emergency notification.

[0682] Step 13: Preparing for push notifications

[0683] The server prepares push notifications using services such as Firebase Cloud Messaging. For example, it uses the Firebase API to create a list of target devices. The input is the generated emergency notification, and the output is the prepared push notification.

[0684] Step 14: Sending Push Notifications

[0685] The server sends the generated emergency notification to the user's device. For example, it might prioritize sending notifications to users in a specific region. The input is a prepared push notification, and the output is the emergency notification sent to the user's device.

[0686] Step 15: Check the notification

[0687] The user receives an emergency notification from their device and checks it immediately. For example, they might check a notification that says, "An earthquake has occurred. Please evacuate." The input is the emergency notification sent to the device, and the output is the user's action upon receiving the notification.

[0688] Step 16: Evacuation Action

[0689] The user initiates evacuation actions according to the notification. For example, they might move to a safe location using the notified evacuation route. The input is the notification content, and the output is the user's evacuation actions.

[0690] Step 17: Supporting calm behavior

[0691] The user acts calmly based on the psychological support information provided by the emotion engine. For example, they reduce stress by reading a message that says, "Please act calmly." The input is the support information from the emotion engine, and the output is the user acting calmly.

[0692] (Application Example 2)

[0693] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0694] In recent years, the frequency and scale of natural disasters have increased, creating a demand for rapid and accurate information provision and support. However, conventional systems struggle to efficiently integrate and analyze sensor data and data from external organizations, and furthermore, they are unable to provide emergency responses that take into account the emotional state of users. As a result, users are unable to take appropriate actions during disasters, leading to increased psychological burden. In addition, even in autonomous vehicles, providing real-time disaster information and appropriate responses based on emotional analysis are necessary to ensure passenger safety.

[0695] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting sensor data, means for transmitting the sensor data to the server, means for analyzing the sensor data, means for acquiring and integrating additional data from other organizations, means for generating emergency notifications based on the analysis results and integrated information, means for transmitting the emergency notifications to the terminal, means for analyzing the emotional state from the user's voice and facial expressions, and means for adjusting the content and expression of the emergency notification considering the emotional state. This enables not only the real-time and accurate provision of disaster information, but also appropriate notifications and support that respond to the user's emotions. Furthermore, by applying this to autonomous vehicles, passenger safety and a sense of psychological security can be ensured.

[0696] "Sensor data" refers to physical or environmental data collected by sensors, including location information, acceleration, vibration, water level, sound, and images.

[0697] "Means of sending to a server" refers to means that have the function of transferring collected sensor data to a server via the internet or other communication networks.

[0698] "Methods for data analysis" refer to methods of analyzing sensor data received on a server and using algorithms and computational techniques to determine the scale and location of a disaster.

[0699] "Means for acquiring and integrating additional data" refers to methods for acquiring data provided by external organizations and combining it with existing sensor data to generate more detailed information.

[0700] "Means for generating emergency notifications" refers to means that have the function of creating notification messages, including warnings and instructions for users, based on analyzed data and integrated information.

[0701] "Means of sending to a terminal" refers to means that have the function of sending the generated emergency notification to the user's communication device (e.g., smartphone, tablet, PC, etc.).

[0702] "Methods for analyzing emotional states from voice and facial expressions" refers to methods that have the function of detecting and analyzing a user's emotional state based on the user's voice and facial expression data.

[0703] "Means for adjusting the content and wording of emergency notifications in consideration of emotional state" refers to means that have the function of adjusting emergency notification messages to be more appropriate and effective based on the analyzed emotional state of the user.

[0704] "Means of taking action" refers to the means by which a user takes appropriate action or response measures based on an emergency notification displayed on their device.

[0705] "Means for obtaining current location" refers to means that have the function of obtaining the user's current physical location using location information technology such as GPS.

[0706] "Means of providing optimal evacuation routes" refers to a means that has the function of suggesting and directing the safest and most efficient evacuation route based on the user's current location and disaster information.

[0707] "Means for managing the location information of support resources" refers to means that have the function of centrally managing the location information of evacuation centers and relief supplies, and providing that information as needed.

[0708] "Means of providing support resources" refers to means of delivering information on evacuation sites and relief supplies to users and providing the maximum possible support.

[0709] Modes for carrying out the invention

[0710] The system according to the present invention collects and analyzes information in real time when a natural disaster occurs and provides users with a rapid emergency notification. This system consists of the following components.

[0711] 1. Terminal components and data collection

[0712] The device is equipped with various sensors (GPS, altimeter, accelerometer, microphone, camera, etc.) and collects data from these sensors in real time. For example, recent location information, earthquake vibration data, audio data, and video data are collected.

[0713] 2. Data transmission

[0714] The collected sensor data is securely transmitted to the server using the HTTPS communication protocol. The data is packetized in JSON format and transmitted periodically through batch processing.

[0715] 3. Data Analysis and Information Integration

[0716] The server analyzes received sensor data in real time. For example, it analyzes earthquake vibration data using FFT analysis to identify the epicenter and seismic intensity. Based on these results, it confirms the location and scale of the disaster. At the same time, it acquires and integrates additional data from other organizations (e.g., external data from the Japan Meteorological Agency and river management bureaus) using REST APIs to generate more detailed and accurate disaster information.

[0717] 4. Emotion analysis

[0718] The emotion engine embedded in the server analyzes audio and video data transmitted from the terminal to identify the user's emotional state in real time. Specifically, it uses audio and facial expression analysis algorithms to detect stress levels and panic states. Machine learning libraries such as Python, TensorFlow, and Keras are used for emotion analysis.

[0719] 5. Generation and transmission of emergency notifications

[0720] The server generates emergency notifications based on analytical data, including the results of sentiment analysis. These emergency notifications include evacuation routes, psychological support information, and specific action instructions. The notifications are sent to the user's device using Firebase Cloud Messaging (FCM).

[0721] 6. User behavior and evacuation support

[0722] Users take action according to emergency notifications displayed on their devices. The device displays their current location and navigates them to the optimal evacuation route. It also provides location information for shelters and support resources to help users evacuate quickly and safely.

[0723] Specific examples

[0724] For example, in the event of an earthquake, the accelerometer in the device detects strong shaking and sends that data to a server. The server analyzes the data received from multiple devices to identify the epicenter and seismic intensity. Simultaneously, an emotion engine analyzes the user's voice and facial expressions to detect their stress level. Based on this, the server generates an emergency notification, which includes evacuation routes and psychological support information, and sends it to the user. The user can then receive this notification and take safe evacuation actions.

[0725] Examples of input prompts for a generative AI model

[0726] Write an algorithm to collect sensor data and send it to a server when an autonomous vehicle detects a strong tremor. Also, write code to generate emergency notifications based on the passenger's stress level and adjust the autonomous driving mode accordingly.

[0727] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0728] Step 1:

[0729] Data collection

[0730] The device uses sensors such as a GPS module, accelerometer, camera, and microphone to collect sensor data including current location information, vibration data, video data, and audio data. This data is collected in real time and temporarily stored in internal memory for batch processing. The main input is sensor data, and the output is batch-processed data.

[0731] Step 2:

[0732] Data transmission

[0733] The terminal transmits collected sensor data to the server using the HTTPS communication protocol. The data is packetized in JSON format and securely transferred to the server. The input is batch-processed sensor data, and the output is the data sent to the server.

[0734] Step 3:

[0735] Data reception and storage

[0736] The server receives sensor data transmitted from the terminal. Before the received data is stored in the database, it undergoes validation and filtering. Duplicate and invalid data are removed at this stage. The input is the received sensor data, and the output is the validated data stored in the database.

[0737] Step 4:

[0738] Data Analysis

[0739] The server analyzes sensor data stored in the database to determine the scale and location of a disaster. For example, it uses FFT analysis to calculate the earthquake's epicenter and intensity. The input is validated sensor data, and the output is the analysis results regarding the scale and location of the disaster.

[0740] Step 5:

[0741] Information integration

[0742] The server acquires additional data in real time from external sources. It uses a REST API to obtain weather information, river management information, and other data, and integrates it with internal sensor data. The input consists of additional data acquired from external sources and internal analysis results, and the output is integrated, detailed disaster information.

[0743] Step 6:

[0744] Emotion analysis

[0745] The emotion engine embedded in the server analyzes voice and facial expression data transmitted from the terminal. It uses machine learning models to identify the user's stress level and emotional state. The input is voice and facial expression data, and the output is the analyzed emotional state.

[0746] Step 7:

[0747] Emergency notification generation

[0748] The server generates emergency notifications based on analyzed disaster information and the user's emotional state. These notifications include evacuation orders, recommended evacuation routes, and psychological support information. The input is integrated disaster information and emotional state, while the output is the generated emergency notification.

[0749] Step 8:

[0750] Emergency notification sent

[0751] The server sends the generated emergency notification to each user's device using Firebase Cloud Messaging (FCM). The input is the emergency notification, and the output is the notification sent to the device.

[0752] Step 9:

[0753] User behavior

[0754] The user receives an emergency notification displayed on the device and takes evacuation action according to the instructions. The device displays the user's current location and the optimal evacuation route, guiding them along the way. The input is the received emergency notification, and the output is the user's evacuation action. Psychological support information is also provided to help the user act calmly.

[0755] In this way, the system can provide users with rapid and accurate information and psychological support through multi-layered data collection and analysis.

[0756] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0757] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0758] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0759] [Third Embodiment]

[0760] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0761] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0762] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0763] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0764] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0765] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0766] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0767] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0768] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0769] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0770] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0771] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0772] This invention is a system that collects and analyzes information in real time during natural disasters and provides users with rapid emergency notifications. This system consists of three main components: a terminal, a server, and a user.

[0773] System Overview

[0774] 1. Data Collection

[0775] Device: Each user's device (e.g., smartphone) is equipped with a GPS module for acquiring location information, as well as seismometers, accelerometers, water level sensors, and other sensors. These sensors periodically collect data and transmit it to a server via the network.

[0776] 2. Data transmission

[0777] Terminal: Collected sensor data is periodically batch-processed and sent to the server via a secure communication protocol (e.g., HTTPS).

[0778] 3. Data reception and storage

[0779] Server: The server receives data sent from terminals and stores it in the database. During this process, data validation and filtering are also performed to remove duplicate and invalid data.

[0780] 4. Data Analysis

[0781] Server: The server analyzes the received data to determine the scale and location of the disaster. For example, it analyzes earthquake vibration data to determine the epicenter and seismic intensity. This analysis is performed using algorithms (e.g., FFT analysis).

[0782] 5. Information Integration

[0783] Server: The server also acquires data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with its internal data. This generates more accurate and detailed disaster information.

[0784] 6. Emergency notification generation

[0785] Server: The server generates emergency notifications for each user based on the analysis results and integrated information. These emergency notifications include evacuation orders and recommended evacuation routes.

[0786] 7. Send emergency notification

[0787] Server: The generated emergency notification is sent to each user's device via a push notification service (e.g., Firebase Cloud Messaging).

[0788] 8. User behavior

[0789] User: The user receives an emergency notification from their device and begins evacuation. They check the information provided by the device (e.g., evacuation orders, evacuation routes) and move to a safe location.

[0790] Specific examples

[0791] For example, consider the case of an earthquake. When the device's accelerometer detects a strong tremor, it immediately sends this data to the server. The server analyzes the data from multiple devices to calculate the epicenter and seismic intensity. The server then obtains additional data from the Japan Meteorological Agency to further determine the extent of the disaster's impact. Finally, the server generates an emergency notification and sends a push notification to the user's device. The user checks this notification and evacuates to a safe place according to the recommended evacuation route.

[0792] Furthermore, if a flood warning is issued, the terminal collects water level sensor data and sends it to the server. The server analyzes this data and predicts the likelihood of flooding. It integrates additional data (e.g., water level data from the river management authority) to determine the extent of the flood's impact. The generated emergency notification is sent to users in the affected area as a notification including evacuation instructions. Users see this notification and take action to evacuate quickly and safely.

[0793] In this way, the present invention provides users with rapid and accurate information and supports appropriate responses during natural disasters.

[0794] The following describes the processing flow.

[0795] Step 1:

[0796] Terminals: Each terminal periodically collects sensor data (e.g., GPS location information, seismometer data). The smartphone's GPS module acquires location information, and the accelerometer collects ambient vibration data. This data is collected at regular time intervals and processed in batches.

[0797] Step 2:

[0798] Terminal: Collects sensor data and sends it to the server using a secure communication protocol (e.g., HTTPS). The data is packetized in a format such as JSON and transmitted over the network.

[0799] Step 3:

[0800] Server: The server receives data sent from the terminal. Data received via the API endpoint is immediately stored in the database. Data is validated and filtered upon receipt to remove duplicate and invalid data.

[0801] Step 4:

[0802] Server: Analyzes received data to determine the scale and location of the disaster. For example, if earthquake vibration data is transmitted, the server uses algorithms such as FFT analysis to calculate the epicenter and seismic intensity.

[0803] Step 5:

[0804] Server: Acquires additional data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with internal data. This external data is retrieved using a REST API and stored in the server's database. This generates more detailed and accurate disaster information.

[0805] Step 6:

[0806] Server: Based on analysis results and integrated information, generates emergency notifications to send to users. These emergency notifications include evacuation orders and recommended evacuation routes. The generated notifications are formatted using templates.

[0807] Step 7:

[0808] Server: Generates emergency notifications and sends them to each user's device using a push notification service (e.g., Firebase Cloud Messaging). The notification range is determined based on the target region and the user's current location.

[0809] Step 8:

[0810] User: The user receives an emergency notification from their device and checks it immediately. They begin evacuation according to the notification's contents. The smartphone app displays evacuation shelters and recommended evacuation routes to help the user evacuate safely.

[0811] The above outlines the specific flow of the system's program processing. This process enables a swift and appropriate response in the event of a disaster.

[0812] (Example 1)

[0813] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0814] Conventional disaster notification systems lacked real-time capabilities in collecting and analyzing sensor data, making it difficult to provide users with timely emergency notifications. Furthermore, the integration of data from multiple sources was insufficient, preventing improvements in the accuracy and detail of disaster information. As a result, users were unable to take appropriate evacuation actions quickly, increasing the risk of disaster damage.

[0815] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0816] In this invention, the server includes means for verifying, filtering, and storing sensor data; means for analyzing the sensor data using FFT analysis; and means for acquiring and integrating additional data from other organizations. This enables the generation of highly accurate and detailed disaster information and the rapid provision of emergency notifications to users through real-time analysis of sensor data and integration of data from multiple organizations.

[0817] "Sensor data" refers to data collected by various sensors installed in the device (e.g., GPS module, seismometer, accelerometer, water level sensor).

[0818] A "server" is a central computer system that receives, stores, and analyzes sensor data, and integrates additional data from other organizations.

[0819] "Verification and filtering" is the process of checking data sent to the server from the perspectives of accuracy, duplication, and fraud, and saving only the appropriate data.

[0820] "FFT analysis" refers to the Fast Fourier Transform algorithm used by servers to analyze sensor data, and is particularly used for vibration analysis of earthquake data.

[0821] "Additional data from other organizations" refers to supplementary data obtained from external organizations such as the Japan Meteorological Agency and river management bureaus, which are integrated with sensor data.

[0822] An "emergency notification" is a notification message, including warnings and evacuation orders, that is generated based on analysis results and integrated information when a disaster occurs.

[0823] A "communication network" is the infrastructure used to send and receive digital data, such as the internet or dedicated lines.

[0824] A "terminal" refers to a device, such as a smartphone or tablet, that collects sensor data and receives emergency notifications.

[0825] "Means of taking action" refers to the user checking emergency notifications from their device and then acting according to the designated evacuation instructions and evacuation routes.

[0826] This invention is a system that collects and analyzes information in real time during natural disasters and provides users with rapid emergency notifications. This system consists of three main components: a terminal, a server, and a user. Details of each are described below.

[0827] 1. Data Collection

[0828] Device: The user's device (e.g., smartphone) collects data using various sensors such as GPS modules, seismometers, accelerometers, and water level sensors. These sensors operate continuously and measure data at regular intervals. For example, a smartphone's accelerometer captures tremor information at a frequency of 5 times per second.

[0829] 2. Data transmission

[0830] Terminal: The collected sensor data is periodically batch-processed and securely sent to the server via the HTTPS protocol. For example, the terminal sends the data it collects to the server via HTTPS every 5 minutes.

[0831] 3. Data reception and storage

[0832] Server: The server receives data sent from the terminal and stores it in a database using Python and MySQL. During this process, it validates and filters the data to remove invalid or duplicate data.

[0833] 4. Data Analysis

[0834] Server: The server analyzes the received data to determine the scale and location of the disaster. In particular, in the case of earthquakes, it uses FFT (Fast Fourier Transform) analysis to calculate the epicenter and seismic intensity. The server identifies the epicenter and seismic intensity by performing FFT analysis on the vibration data it receives.

[0835] 5. Information Integration

[0836] Server: The server also acquires data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with its internal data. This generates more accurate and detailed information about disasters. It obtains the latest weather data from the Japan Meteorological Agency's API and integrates it with its own data.

[0837] 6. Emergency notification generation

[0838] Server: The server generates emergency notifications based on analysis results and integrated information. These notifications include evacuation orders and recommended evacuation routes. The server creates emergency notifications containing evacuation orders and routes based on the specified format.

[0839] 7. Send emergency notification

[0840] Server: Generated emergency notifications are pushed to each user's device using Firebase Cloud Messaging. The server calls Firebase Cloud Messaging to deliver the generated emergency notifications to all users' devices.

[0841] 8. User behavior

[0842] User: The user checks the emergency notification received on their device and begins evacuation. The notification includes detailed information on evacuation routes and locations, and the user evacuates to a safe place according to these instructions. The user checks the notification on their smartphone and heads to the nearest evacuation center according to the displayed evacuation instructions and evacuation route.

[0843] Specific example

[0844] For example, in the event of an earthquake, if the device's accelerometer detects strong shaking, it immediately sends this data to the server. The server analyzes the data from multiple devices to calculate the epicenter and seismic intensity. The server then obtains additional data from the Japan Meteorological Agency to further determine the extent of the disaster's impact. Finally, the server generates an emergency notification and sends a push notification to the user's device. The user checks this notification and evacuates to a safe place following the recommended evacuation route.

[0845] Example of a prompt

[0846] "An earthquake has occurred. Please evacuate to a safe place immediately. The nearest evacuation center is point A. Taking route B will ensure your safety."

[0847] Thus, the present invention provides users with rapid and accurate information and supports appropriate responses during natural disasters.

[0848] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0849] Step 1:

[0850] Data collection

[0851] Device: The user's device (smartphone) collects data in real time using its built-in GPS module, seismometer, accelerometer, and water level sensor.

[0852] Input: Measurement data from the sensor.

[0853] Data processing: Batch processing of data within the terminal (e.g., grouping data at regular intervals).

[0854] Output: Batch-processed sensor data.

[0855] Specific operation: The smartphone's accelerometer captures and temporarily stores information about shaking at a frequency of 5 times per second.

[0856] Step 2:

[0857] Data transmission

[0858] Terminal: Collected sensor data is periodically sent to the server via the HTTPS protocol.

[0859] Input: Batch-processed sensor data.

[0860] Data processing: Data encryption and secure transmission.

[0861] Output: Encrypted sensor data is sent to the server.

[0862] Specific operation: Every 5 minutes, the device sends the collected data to the server via HTTPS.

[0863] Step 3:

[0864] Data reception and storage

[0865] Server: Receives data sent from terminals, verifies and filters the data to ensure its accuracy, and then stores it in a database.

[0866] Input: Transmitted sensor data.

[0867] Data processing: Verification (e.g., checking data accuracy) and filtering (e.g., removing duplicate data).

[0868] Output: Clean sensor data is saved to the database.

[0869] Specific operation: The server uses Python and MySQL to validate the data and then save it to the database.

[0870] Step 4:

[0871] Data Analysis

[0872] Server: Analyzes stored data to determine the scale and location of disasters. In particular, in the case of earthquakes, it uses FFT (Fast Fourier Transform) analysis to calculate the epicenter and seismic intensity.

[0873] Input: Stored sensor data.

[0874] Data processing: Perform FFT analysis.

[0875] Output: Analysis results such as epicenter and seismic intensity.

[0876] Specific operation: The server applies FFT analysis to the vibration data it receives to identify the epicenter.

[0877] Step 5:

[0878] Information integration

[0879] Server: Acquires additional data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with internal data.

[0880] Input: Analysis results and additional data from external organizations.

[0881] Data processing: Data integration and analysis.

[0882] Output: Even more accurate disaster information.

[0883] Specific operation: Obtain the latest earthquake information from the Japan Meteorological Agency's API and integrate it with our own data.

[0884] Step 6:

[0885] Emergency notification generation

[0886] Server: Based on analysis results and integrated information, it generates optimized emergency notifications for each user (including evacuation orders and recommended evacuation routes).

[0887] Input: Analysis results and integrated information.

[0888] Data processing: Creating emergency notifications.

[0889] Output: Emergency notification message.

[0890] Specific operation: The server creates an emergency notification, including evacuation instructions and evacuation routes, based on the specified format.

[0891] Step 7:

[0892] Emergency notification sent

[0893] Server: The generated emergency notification is pushed to each user's device using Firebase Cloud Messaging.

[0894] Input: Generated emergency notification message.

[0895] Data processing: Sending notification messages.

[0896] Output: Emergency notification sent to the user's device.

[0897] Specific operation: The server calls Firebase Cloud Messaging and delivers the generated emergency notification to all users' devices.

[0898] Step 8:

[0899] User behavior

[0900] User: The user checks the emergency notification received on their device and begins evacuation according to the instructions in the notification.

[0901] Input: Emergency notification displayed on the device.

[0902] Output: User evacuation actions.

[0903] Specific actions: The user checks the notification on their smartphone and heads to the nearest evacuation shelter according to the displayed evacuation instructions and evacuation route.

[0904] (Application Example 1)

[0905] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0906] While systems exist that provide real-time information to help users evacuate quickly and safely during natural disasters, these systems generally assume that users will move manually. Therefore, especially in urban areas with a large number of autonomous vehicles, there is a lack of efficient and immediate means of evacuation. To overcome this drawback, a method is needed that automatically controls autonomous vehicles during natural disasters to select appropriate evacuation routes and guide them to safe locations.

[0907] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0908] In this invention, the server includes means for collecting sensor data, means for transmitting sensor data to the server, means for analyzing sensor data, means for acquiring and integrating additional data from other organizations, means for generating emergency notifications based on the analysis results and integrated information, means for transmitting emergency notifications to a terminal, means for the user to take action in response to the emergency notification displayed on the terminal, means for controlling the operation of the autonomous mobile vehicle and calculating a route for safe evacuation, and means for the autonomous mobile vehicle to move according to the calculated evacuation route. This enables the autonomous mobile vehicle to perform appropriate evacuation actions quickly and safely.

[0909] "Sensor data" refers to data collected from a wide variety of sensors (e.g., GPS, seismometers, accelerometers, water level sensors).

[0910] A "server" is a computer system that receives and analyzes sensor data via a network, and processes information by integrating additional data from other organizations.

[0911] "Means of sending to the server" refers to the function of securely and efficiently transferring collected sensor data to the server.

[0912] "Means of analysis" refers to methods of evaluating the scale, location, and impact of a disaster using collected sensor data.

[0913] "Methods for acquiring and integrating additional data from other organizations" refers to methods of collecting data from organizations such as the Japan Meteorological Agency and river management bureaus, and combining it with the data analyzed on the server.

[0914] "Means for generating emergency notifications" refers to a method of creating emergency information (e.g., evacuation orders, recommended evacuation routes) that should be conveyed to users, based on analysis results and integrated information.

[0915] "Means of sending emergency notifications to devices" refers to a function that transmits generated emergency notifications to user devices such as smartphones and tablets.

[0916] "Means of action for the user" refers to a function that facilitates actions to evacuate to a safe place based on emergency notifications displayed on the device.

[0917] An "autonomous mobile vehicle" is a vehicle that utilizes artificial intelligence and sensor technology to operate on its own judgment without external instructions.

[0918] "Means of controlling operation" refers to functions that manage the movement of autonomous mobile vehicles and ensure safe operation.

[0919] The "means for calculating routes" refer to a function that calculates the optimal evacuation route in the event of a disaster and instructs autonomous mobile vehicles accordingly.

[0920] "Means of transportation" refers to the ability of an autonomous vehicle to travel to its destination according to a calculated route.

[0921] This invention is a system that enables autonomous mobile vehicles to evacuate quickly and safely in the event of a natural disaster. Specific embodiments for realizing this system are described below.

[0922] 1. Data Collection

[0923] The terminal is equipped with various sensors, including a GPS module, seismometer, accelerometer, and water level sensor. These sensors periodically collect data and transmit it to a server via the network. For example, if an autonomous vehicle detects an earthquake, its accelerometer will acquire vibration data.

[0924] 2. Data transmission

[0925] The terminal periodically processes the collected sensor data in batches and sends it to the server via a secure communication protocol (such as HTTPS). It is recommended to use a high-speed and fault-tolerant communication method for this process.

[0926] 3. Data reception and storage

[0927] The server receives sensor data transmitted from terminals and stores it in a database. During this process, data verification and filtering are performed to remove duplicate and invalid data. For example, earthquake data transmitted simultaneously from multiple vehicles can be integrated.

[0928] 4. Data Analysis

[0929] The server analyzes the received data to determine the scale and location of the disaster. In the case of an earthquake, it uses algorithms such as FFT analysis to calculate the epicenter and seismic intensity. It also acquires data from the Japan Meteorological Agency and river management bureaus and integrates it with internal data to generate more accurate disaster information.

[0930] 5. Generate and send emergency notifications

[0931] The server generates an emergency notification based on the analysis results and integrated information. This notification includes evacuation instructions and recommended evacuation routes. The generated emergency notification is sent to each terminal via a push notification service. Based on the emergency notification received by the autonomous mobile vehicle, the vehicle's operation management system recalculates the route and selects the optimal evacuation route.

[0932] 6. Operation control of autonomous mobile vehicles

[0933] The autonomous mobile vehicle will automatically suspend its operation and move to a safe location in accordance with emergency notifications and recommended evacuation routes received from the server. During this process, in-vehicle sensors such as LiDAR and cameras will detect the surrounding environment to ensure safety during evacuation.

[0934] Specific example

[0935] For example, if an autonomous mobile vehicle detects an earthquake in central Tokyo, it would calculate the safest route to evacuate from that location and control the vehicle to move to a safe location. In this process, the system would need to monitor and avoid the conditions inside the vehicle and surrounding obstacles in real time.

[0936] Example of a prompt

[0937] "We are developing an application to enable the safe evacuation of autonomous mobile vehicles using a system that collects sensor data in real time during natural disasters and provides rapid emergency notifications. Data is collected from seismometers, accelerometers, water level sensors, etc., and transmitted to a server using a secure protocol. The server analyzes the data, generates an emergency notification, and sends it to the autonomous mobile vehicle. Based on the notification, the vehicle selects a safe evacuation route and moves to a safe location."

[0938] This embodiment enables autonomous mobile vehicles to evacuate quickly and safely in the event of a natural disaster, thereby protecting the lives and property of users.

[0939] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0940] Step 1: Data Collection

[0941] The device collects data from a GPS module, seismometer, accelerometer, and water level sensor. This allows it to obtain sensor data such as location information, seismic intensity, and surrounding water level information. The input consists of various sensor data, which is acquired in real time by the device. The output is the collected sensor data.

[0942] Step 2: Data transmission

[0943] The terminal periodically batches the collected sensor data and sends it to the server via a secure communication protocol (e.g., HTTPS). The input is the collected sensor data, which is formatted and encrypted for secure transmission to the server. The output is the transmitted sensor data arriving at the server.

[0944] Step 3: Data reception and storage

[0945] The server receives sensor data transmitted from the terminal and stores it in a database. During this process, the data is validated and filtered to remove duplicate and invalid data. The input is the transmitted sensor data, and the output is a clean and consistent dataset stored in the database.

[0946] Step 4: Data Analysis

[0947] The server analyzes sensor data stored in the database. Specifically, it analyzes earthquake vibration data using algorithms such as FFT analysis to determine the scale and location of the disaster. It also acquires additional data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with the internal data. The input is the stored sensor data and additional data, and the output is the analysis results that identify the scale and location of the disaster.

[0948] Step 5: Generate emergency notification

[0949] The server generates emergency notifications based on the analysis results and integrated information. These notifications include evacuation orders and recommended evacuation routes. The input is the data analysis results and integrated information, and the output is the generated emergency notification.

[0950] Step 6: Send emergency notification

[0951] The server sends the generated emergency notification to the device. This allows the user to receive important information in real time. The input is the generated emergency notification, and the output is displayed on the device as a push notification.

[0952] Step 7: Operation control of autonomous mobile vehicles

[0953] The terminal relays received emergency notifications to the autonomous mobile vehicle's operation management system, thereby controlling autonomous driving. The input is the received emergency notification, and the output is a recalculation of the vehicle's route, selecting a safe evacuation path.

[0954] Step 8: Evacuation Action

[0955] The user takes action based on the emergency notification displayed on their device. The autonomous mobile vehicle moves along a calculated route and evacuates to a safe location. The input is the selected evacuation route, and the output is the user's action of evacuating to a safe location.

[0956] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0957] This invention combines a system that collects and analyzes information in real time during natural disasters and provides users with rapid emergency notifications with an emotion engine that recognizes the user's emotions. This system consists of four main components: a terminal, a server, an emotion engine, and a user.

[0958] System Overview

[0959] 1. Data Collection

[0960] Device: Each user's device (e.g., smartphone) is equipped with a GPS module for acquiring location information, as well as seismometers, accelerometers, water level sensors, and other sensors. These sensors periodically collect data and transmit it to a server via the network.

[0961] 2. Data transmission

[0962] Terminal: Collected sensor data is periodically batch-processed and sent to the server via a secure communication protocol (e.g., HTTPS). The data is packetized in a format such as JSON and transmitted over the network.

[0963] 3. Data reception and storage

[0964] Server: The server receives data sent from terminals and stores it in the database. Data is validated and filtered upon receipt, removing duplicate and invalid data.

[0965] 4. Data Analysis

[0966] Server: The server analyzes the received data to determine the scale and location of the disaster. For example, if earthquake vibration data is transmitted, the server uses algorithms such as FFT analysis to calculate the epicenter and seismic intensity.

[0967] 5. Information Integration

[0968] Server: The server also acquires data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with internal data. This external data is retrieved using a REST API and stored in the server's database. This generates more detailed and accurate disaster information.

[0969] 6. Analysis using an emotion engine

[0970] Server: The emotion engine built into the server analyzes the user's voice and text messages to identify the user's emotional state. For example, it analyzes in real time what the user says to the device and the messages they type to detect stress or panic.

[0971] 7. Emergency notification generation

[0972] Server: Based on the analysis results of the emotion engine, the server generates emergency notifications considering the user's emotional state. Emergency notifications include evacuation orders, recommended evacuation routes, and psychological support information. The content and wording of the notifications are adjusted as needed.

[0973] 8. Send emergency notification

[0974] Server: Generates emergency notifications and sends them to each user's device using a push notification service (e.g., Firebase Cloud Messaging). The notification range is determined based on the target region and the user's current location.

[0975] 9. User behavior

[0976] User: The user receives an emergency notification from their device and checks it immediately. They begin evacuation actions according to the notification. The smartphone app displays evacuation shelters and recommended evacuation routes to help the user evacuate safely. Furthermore, the user can act calmly by referring to psychological support information provided by the emotion engine.

[0977] Specific examples

[0978] For example, consider the case of an earthquake. When the device's accelerometer detects a strong tremor, it immediately sends this data to the server. The server analyzes the data from multiple devices to calculate the epicenter and seismic intensity. The server then obtains additional data from the Japan Meteorological Agency to further determine the extent of the disaster's impact. When the emotion engine detects the user's stress level from their voice or text, the server generates an emergency notification that takes this into account and sends it to the user's device. The user checks this notification and evacuates to a safe place following the recommended evacuation route. At the same time, psychological support messages provided by the emotion engine help the user remain calm.

[0979] Furthermore, if a flood warning is issued, the device collects water level sensor data and sends it to the server. The server analyzes this data and predicts the likelihood of flooding. It integrates additional data (e.g., water level data from the river management authority) to determine the extent of the flood's impact. The emotion engine analyzes the user's emotional state and, if necessary, includes psychological support information to help them stay calm in the emergency notification. The user receives this notification and evacuates safely.

[0980] In this way, the present invention not only provides users with rapid and accurate information and supports appropriate responses during natural disasters, but also, by combining it with an emotion engine, takes into account the user's psychological state and provides more effective support.

[0981] The following describes the processing flow.

[0982] Step 1:

[0983] Terminals: Each terminal periodically collects sensor data (e.g., GPS location information, seismometer data). The smartphone's GPS module acquires location information, and the accelerometer collects ambient vibration data. This data is collected at regular time intervals and processed in batches.

[0984] Step 2:

[0985] Terminal: Collects sensor data and sends it to the server using a secure communication protocol (e.g., HTTPS). The data is packetized in a format such as JSON and transmitted over the network.

[0986] Step 3:

[0987] Server: The server receives data sent from the terminal. Data received via the API endpoint is immediately stored in the database. Data is validated and filtered upon receipt, removing duplicate and invalid data.

[0988] Step 4:

[0989] Server: Analyzes received data to determine the scale and location of the disaster. For example, if earthquake vibration data is transmitted, the server uses algorithms such as FFT analysis to calculate the epicenter and seismic intensity.

[0990] Step 5:

[0991] Server: Acquires additional data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with internal data. This external data is retrieved using a REST API and stored in the server's database. This generates more detailed and accurate disaster information.

[0992] Step 6:

[0993] Server: The emotion engine built into the server analyzes the user's voice and text messages to identify the user's emotional state. For example, it analyzes in real time what the user says to the device and the messages they type to detect stress or panic.

[0994] Step 7:

[0995] Server: Based on the analysis results of the emotion engine, the server generates emergency notifications considering the user's emotional state. Emergency notifications include evacuation orders, recommended evacuation routes, and psychological support information. The content and wording of the notifications are adjusted as needed.

[0996] Step 8:

[0997] Server: Generates emergency notifications and sends them to each user's device using a push notification service (e.g., Firebase Cloud Messaging). The notification range is determined based on the target region and the user's current location.

[0998] Step 9:

[0999] User: The user receives an emergency notification from their device and checks it immediately. They begin evacuation according to the notification's contents. The smartphone app displays evacuation shelters and recommended evacuation routes to help the user evacuate safely. They also act calmly, referring to psychological support information provided by the emotion engine.

[1000] The above outlines the specific program processing flow of the system that incorporates the emotion engine. This process enables a rapid and appropriate response in the event of a disaster, while simultaneously providing psychological support to the user.

[1001] (Example 2)

[1002] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1003] When natural disasters occur, there is a need for rapid and accurate information provision and appropriate responses that take into account the psychological state of users. Conventional systems do not adequately collect and analyze real-time data, making it difficult to provide support that takes into account the emotional state of users. In addition, emergency notifications are uniform, making it difficult to respond to the individual circumstances of users.

[1004] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting sensor data, means for analyzing the sensor data, means for acquiring and integrating additional data from other organizations, means for collecting user voice and text data and identifying their emotional state, means for generating an emergency notification based on the analysis results, integrated information, and the user's emotional state, and means for sending the emergency notification as a push notification to the terminal. This enables the rapid and accurate provision of disaster information and individualized responses that take into account the user's psychological state.

[1005] "Sensor data" refers to data collected from various sensors installed in the user's device (e.g., GPS module, seismometer, accelerometer, water level sensor).

[1006] A "server" is a remote computing device that receives, analyzes, and stores sensor data transmitted from terminals, acquires additional data from other organizations, and generates and transmits emergency notifications.

[1007] "Analysis" is the process of applying algorithms to received sensor data to extract meaningful information.

[1008] "Integration" refers to the process of combining data obtained from other organizations with internal data to create more detailed and accurate disaster information.

[1009] An "emotion engine" is software that analyzes a user's voice and text data to identify the user's emotional state.

[1010] An "emergency notification" is a message generated during a disaster that provides users with evacuation instructions and psychological support information.

[1011] "Push notifications" are a real-time notification format that a server sends directly to a user's device.

[1012] A "terminal" refers to a personal information terminal (e.g., a smartphone) used by a user, which is a device that collects sensor data and displays emergency notifications.

[1013] A "user" is a person who uses this system to receive emergency notifications and take action.

[1014]

[1015] This invention is a system that collects and analyzes information in real time during natural disasters and provides users with rapid emergency notifications. This system consists of four main components: a terminal, a server, an emotion engine, and a user.

[1016] First, the device is the user's smartphone, equipped with a GPS module, seismometer, accelerometer, water level sensor, and other sensors. These sensors periodically collect current location information, earthquake vibration data, and water level data. For example, when an earthquake occurs, the device's accelerometer detects the shaking and records the data.

[1017] Next, the collected sensor data is batch-processed at regular intervals and sent to the server using the HTTPS protocol. The data is sent in JSON format. Specifically, for example, seismometer data is converted into a JSON structure such as "{ 'timestamp': '2023-10-01T12:00:00Z', 'sensor': 'accelerometer', 'value': 5.6}" and sent as an HTTPS request. The server waits for reception at a specific endpoint and receives the data when it receives an HTTPS request from the terminal.

[1018] The server stores the received sensor data in a database and performs data validation and filtering. For example, it checks the timestamp and sensor type included in the received data to eliminate invalid data. The filtered data is then applied for analysis. Using FFT analysis and other algorithms, for example, it identifies the epicenter and seismic intensity from earthquake vibration data.

[1019] Furthermore, the server acquires additional data from other organizations (e.g., the Japan Meteorological Agency, river management bureaus) and integrates it with internal data. At this time, it uses a REST API to retrieve external data and generates detailed and accurate disaster information after integration. For example, it acquires earthquake reports from the Japan Meteorological Agency and integrates them with data from terminals to determine the affected area.

[1020] An emotion engine is also integrated into the server, analyzing the voice and text data that users input into their devices. This engine can identify the user's emotional state and detect stress or panic. For example, if a user inputs "I'm scared of earthquakes," the emotion engine will detect a stressed state from that message.

[1021] The server generates emergency notifications based on analysis results and the results of the emotion engine. These emergency notifications include evacuation instructions and psychological support information that takes the user's emotional state into consideration, and the content is customized as needed. For example, in addition to a basic notification such as "Please evacuate," it also sends psychological support messages such as "Please stay calm."

[1022] Finally, the server sends the generated emergency notification to the user's device using a push notification service such as Firebase Cloud Messaging. The notification is sent in real time, and the delivery range is set based on the target area and the user's current location. The user receives this notification and takes appropriate action according to the displayed evacuation route and recommended actions.

[1023] This system provides users with rapid and accurate disaster information and psychological support, assisting them in taking appropriate action during emergencies. An example of a prompt message generated using the AI ​​model is: "There is an earthquake. Please evacuate immediately. Please remain calm."

[1024] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1025] Step 1: Data Collection

[1026] The device collects sensor data from the user's smartphone, including GPS modules, seismometers, accelerometers, and water level sensors. For example, when a seismometer detects shaking, it collects the data and stores it in a buffer. The input is data acquired from the sensor modules, and the output is raw data stored in the device's internal buffer.

[1027] Step 2: Convert Data Format

[1028] The terminal processes the collected sensor data in batches at regular time intervals and converts it to JSON format. For example, it converts seismometer data into a JSON structure. The input is raw data stored in the terminal's buffer, and the output is data formatted in JSON format.

[1029] Step 3: Data transmission

[1030] The terminal sends data converted to JSON format to the server using the HTTPS protocol. For example, the converted JSON data is sent to the server's endpoint via an HTTPS request. The input is data formatted in JSON format, and the output is the data sent to the server.

[1031] Step 4: Received

[1032] The server receives data sent from the terminal. For example, it receives HTTPS requests at a specific endpoint on the server. The input is the data sent from the terminal, and the output is the JSON data received by the server.

[1033] Step 5: Data validation and filtering

[1034] The server validates the received data and removes invalid and duplicate data. For example, it checks the data's timestamp and sensor type to eliminate invalid and duplicate data. The input is the JSON data received by the server, and the output is the filtered, accurate data.

[1035] Step 6: Save Data

[1036] The server stores the filtered data in a database. For example, it adds verified data to a MySQL database. The input is the filtered, accurate data, and the output is the data stored in the database.

[1037] Step 7: Data Analysis

[1038] The server analyzes the received and stored data to determine the scale and location of a disaster. For example, it uses FFT analysis to identify the epicenter and seismic intensity. The input is sensor data stored in a database, and the output is the analysis results.

[1039] Step 8: Obtaining external data

[1040] The server retrieves additional data from external organizations (e.g., the Japan Meteorological Agency) via a REST API. For example, it uses the Japan Meteorological Agency's API to retrieve the latest earthquake observation data. The input is the endpoint information of the external API, and the output is the retrieved external data.

[1041] Step 9: Data Integration

[1042] The server integrates acquired external and internal data to generate detailed and accurate disaster information. Inputs are analysis results and external data, while output is the integrated disaster information.

[1043] Step 10: Collecting emotional data

[1044] The server collects voice and text data that the user inputs into the terminal. For example, the user inputs "I'm scared of earthquakes" into the terminal. The input is voice or text data from the user, and the output is collected emotion data.

[1045] Step 11: Analyzing emotional data

[1046] The server uses an emotion engine to analyze voice and text data to identify the user's emotional state. For example, it can detect a stressed state from the input text "scared." The input is the collected emotion data, and the output is the emotional state identified through the analysis.

[1047] Step 12: Generate an emergency notification

[1048] The server generates emergency notifications based on analysis results and emotional states. For example, it creates notification messages that include evacuation orders and psychological support information. The input is the analysis results and emotional states, and the output is the generated emergency notification.

[1049] Step 13: Preparing for push notifications

[1050] The server prepares push notifications using services such as Firebase Cloud Messaging. For example, it uses the Firebase API to create a list of target devices. The input is the generated emergency notification, and the output is the prepared push notification.

[1051] Step 14: Sending Push Notifications

[1052] The server sends the generated emergency notification to the user's device. For example, it might prioritize sending notifications to users in a specific region. The input is a prepared push notification, and the output is the emergency notification sent to the user's device.

[1053] Step 15: Check the notification

[1054] The user receives an emergency notification from their device and checks it immediately. For example, they might check a notification that says, "An earthquake has occurred. Please evacuate." The input is the emergency notification sent to the device, and the output is the user's action upon receiving the notification.

[1055] Step 16: Evacuation Action

[1056] The user initiates evacuation actions according to the notification. For example, they might move to a safe location using the notified evacuation route. The input is the notification content, and the output is the user's evacuation actions.

[1057] Step 17: Supporting calm behavior

[1058] The user acts calmly based on the psychological support information provided by the emotion engine. For example, they reduce stress by reading a message that says, "Please act calmly." The input is the support information from the emotion engine, and the output is the user acting calmly.

[1059] (Application Example 2)

[1060] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1061] In recent years, the frequency and scale of natural disasters have increased, creating a demand for rapid and accurate information provision and support. However, conventional systems struggle to efficiently integrate and analyze sensor data and data from external organizations, and furthermore, they are unable to provide emergency responses that take into account the emotional state of users. As a result, users are unable to take appropriate actions during disasters, leading to increased psychological burden. In addition, even in autonomous vehicles, providing real-time disaster information and appropriate responses based on emotional analysis are necessary to ensure passenger safety.

[1062] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting sensor data, means for transmitting the sensor data to the server, means for analyzing the sensor data, means for acquiring and integrating additional data from other organizations, means for generating emergency notifications based on the analysis results and integrated information, means for transmitting the emergency notifications to the terminal, means for analyzing the emotional state from the user's voice and facial expressions, and means for adjusting the content and expression of the emergency notification considering the emotional state. This enables not only the real-time and accurate provision of disaster information, but also appropriate notifications and support that respond to the user's emotions. Furthermore, by applying this to autonomous vehicles, passenger safety and a sense of psychological security can be ensured.

[1063] "Sensor data" refers to physical or environmental data collected by sensors, including location information, acceleration, vibration, water level, sound, and images.

[1064] "Means of sending to a server" refers to means that have the function of transferring collected sensor data to a server via the internet or other communication networks.

[1065] "Methods for data analysis" refer to methods of analyzing sensor data received on a server and using algorithms and computational techniques to determine the scale and location of a disaster.

[1066] "Means for acquiring and integrating additional data" refers to methods for acquiring data provided by external organizations and combining it with existing sensor data to generate more detailed information.

[1067] "Means for generating emergency notifications" refers to means that have the function of creating notification messages, including warnings and instructions for users, based on analyzed data and integrated information.

[1068] "Means of sending to a terminal" refers to means that have the function of sending the generated emergency notification to the user's communication device (e.g., smartphone, tablet, PC, etc.).

[1069] "Methods for analyzing emotional states from voice and facial expressions" refers to methods that have the function of detecting and analyzing a user's emotional state based on the user's voice and facial expression data.

[1070] "Means for adjusting the content and wording of emergency notifications in consideration of emotional state" refers to means that have the function of adjusting emergency notification messages to be more appropriate and effective based on the analyzed emotional state of the user.

[1071] "Means of taking action" refers to the means by which a user takes appropriate action or response measures based on an emergency notification displayed on their device.

[1072] "Means for obtaining current location" refers to means that have the function of obtaining the user's current physical location using location information technology such as GPS.

[1073] "Means of providing optimal evacuation routes" refers to a means that has the function of suggesting and directing the safest and most efficient evacuation route based on the user's current location and disaster information.

[1074] "Means for managing the location information of support resources" refers to means that have the function of centrally managing the location information of evacuation centers and relief supplies, and providing that information as needed.

[1075] "Means of providing support resources" refers to means of delivering information on evacuation sites and relief supplies to users and providing the maximum possible support.

[1076] Modes for carrying out the invention

[1077] The system according to the present invention collects and analyzes information in real time when a natural disaster occurs and provides users with a rapid emergency notification. This system consists of the following components.

[1078] 1. Terminal components and data collection

[1079] The device is equipped with various sensors (GPS, altimeter, accelerometer, microphone, camera, etc.) and collects data from these sensors in real time. For example, recent location information, earthquake vibration data, audio data, and video data are collected.

[1080] 2. Data transmission

[1081] The collected sensor data is securely transmitted to the server using the HTTPS communication protocol. The data is packetized in JSON format and transmitted periodically through batch processing.

[1082] 3. Data Analysis and Information Integration

[1083] The server analyzes received sensor data in real time. For example, it analyzes earthquake vibration data using FFT analysis to identify the epicenter and seismic intensity. Based on these results, it confirms the location and scale of the disaster. At the same time, it acquires and integrates additional data from other organizations (e.g., external data from the Japan Meteorological Agency and river management bureaus) using REST APIs to generate more detailed and accurate disaster information.

[1084] 4. Emotion analysis

[1085] The emotion engine embedded in the server analyzes audio and video data transmitted from the terminal to identify the user's emotional state in real time. Specifically, it uses audio and facial expression analysis algorithms to detect stress levels and panic states. Machine learning libraries such as Python, TensorFlow, and Keras are used for emotion analysis.

[1086] 5. Generation and transmission of emergency notifications

[1087] The server generates emergency notifications based on analytical data, including the results of sentiment analysis. These emergency notifications include evacuation routes, psychological support information, and specific action instructions. The notifications are sent to the user's device using Firebase Cloud Messaging (FCM).

[1088] 6. User behavior and evacuation support

[1089] Users take action according to emergency notifications displayed on their devices. The device displays their current location and navigates them to the optimal evacuation route. It also provides location information for shelters and support resources to help users evacuate quickly and safely.

[1090] Specific examples

[1091] For example, in the event of an earthquake, the accelerometer in the device detects strong shaking and sends that data to a server. The server analyzes the data received from multiple devices to identify the epicenter and seismic intensity. Simultaneously, an emotion engine analyzes the user's voice and facial expressions to detect their stress level. Based on this, the server generates an emergency notification, which includes evacuation routes and psychological support information, and sends it to the user. The user can then receive this notification and take safe evacuation actions.

[1092] Examples of input prompts for a generative AI model

[1093] Write an algorithm to collect sensor data and send it to a server when an autonomous vehicle detects a strong tremor. Also, write code to generate emergency notifications based on the passenger's stress level and adjust the autonomous driving mode accordingly.

[1094] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1095] Step 1:

[1096] Data collection

[1097] The device uses sensors such as a GPS module, accelerometer, camera, and microphone to collect sensor data including current location information, vibration data, video data, and audio data. This data is collected in real time and temporarily stored in internal memory for batch processing. The main input is sensor data, and the output is batch-processed data.

[1098] Step 2:

[1099] Data transmission

[1100] The terminal transmits collected sensor data to the server using the HTTPS communication protocol. The data is packetized in JSON format and securely transferred to the server. The input is batch-processed sensor data, and the output is the data sent to the server.

[1101] Step 3:

[1102] Data reception and storage

[1103] The server receives sensor data transmitted from the terminal. Before the received data is stored in the database, it undergoes validation and filtering. Duplicate and invalid data are removed at this stage. The input is the received sensor data, and the output is the validated data stored in the database.

[1104] Step 4:

[1105] Data Analysis

[1106] The server analyzes sensor data stored in the database to determine the scale and location of a disaster. For example, it uses FFT analysis to calculate the earthquake's epicenter and intensity. The input is validated sensor data, and the output is the analysis results regarding the scale and location of the disaster.

[1107] Step 5:

[1108] Information integration

[1109] The server acquires additional data in real time from external sources. It uses a REST API to obtain weather information, river management information, and other data, and integrates it with internal sensor data. The input consists of additional data acquired from external sources and internal analysis results, and the output is integrated, detailed disaster information.

[1110] Step 6:

[1111] Emotion analysis

[1112] The emotion engine embedded in the server analyzes voice and facial expression data transmitted from the terminal. It uses machine learning models to identify the user's stress level and emotional state. The input is voice and facial expression data, and the output is the analyzed emotional state.

[1113] Step 7:

[1114] Emergency notification generation

[1115] The server generates emergency notifications based on analyzed disaster information and the user's emotional state. These notifications include evacuation orders, recommended evacuation routes, and psychological support information. The input is integrated disaster information and emotional state, while the output is the generated emergency notification.

[1116] Step 8:

[1117] Emergency notification sent

[1118] The server sends the generated emergency notification to each user's device using Firebase Cloud Messaging (FCM). The input is the emergency notification, and the output is the notification sent to the device.

[1119] Step 9:

[1120] User behavior

[1121] The user receives an emergency notification displayed on the device and takes evacuation action according to the instructions. The device displays the user's current location and the optimal evacuation route, guiding them along the way. The input is the received emergency notification, and the output is the user's evacuation action. Psychological support information is also provided to help the user act calmly.

[1122] In this way, the system can provide users with rapid and accurate information and psychological support through multi-layered data collection and analysis.

[1123] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1124] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1125] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1126] [Fourth Embodiment]

[1127] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1128] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1129] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1130] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1131] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1132] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1133] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1134] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1135] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1136] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1137] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1138] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1139] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1140] This invention is a system that collects and analyzes information in real time during natural disasters and provides users with rapid emergency notifications. This system consists of three main components: a terminal, a server, and a user.

[1141] System Overview

[1142] 1. Data Collection

[1143] Device: Each user's device (e.g., smartphone) is equipped with a GPS module for acquiring location information, as well as seismometers, accelerometers, water level sensors, and other sensors. These sensors periodically collect data and transmit it to a server via the network.

[1144] 2. Data transmission

[1145] Terminal: Collected sensor data is periodically batch-processed and sent to the server via a secure communication protocol (e.g., HTTPS).

[1146] 3. Data reception and storage

[1147] Server: The server receives data sent from terminals and stores it in the database. During this process, data validation and filtering are also performed to remove duplicate and invalid data.

[1148] 4. Data Analysis

[1149] Server: The server analyzes the received data to determine the scale and location of the disaster. For example, it analyzes earthquake vibration data to determine the epicenter and seismic intensity. This analysis is performed using algorithms (e.g., FFT analysis).

[1150] 5. Information Integration

[1151] Server: The server also acquires data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with its internal data. This generates more accurate and detailed disaster information.

[1152] 6. Emergency notification generation

[1153] Server: The server generates emergency notifications for each user based on the analysis results and integrated information. These emergency notifications include evacuation orders and recommended evacuation routes.

[1154] 7. Send emergency notification

[1155] Server: The generated emergency notification is sent to each user's device via a push notification service (e.g., Firebase Cloud Messaging).

[1156] 8. User behavior

[1157] User: The user receives an emergency notification from their device and begins evacuation. They check the information provided by the device (e.g., evacuation orders, evacuation routes) and move to a safe location.

[1158] Specific examples

[1159] For example, consider the case of an earthquake. When the device's accelerometer detects a strong tremor, it immediately sends this data to the server. The server analyzes the data from multiple devices to calculate the epicenter and seismic intensity. The server then obtains additional data from the Japan Meteorological Agency to further determine the extent of the disaster's impact. Finally, the server generates an emergency notification and sends a push notification to the user's device. The user checks this notification and evacuates to a safe place according to the recommended evacuation route.

[1160] Furthermore, if a flood warning is issued, the terminal collects water level sensor data and sends it to the server. The server analyzes this data and predicts the likelihood of flooding. It integrates additional data (e.g., water level data from the river management authority) to determine the extent of the flood's impact. The generated emergency notification is sent to users in the affected area as a notification including evacuation instructions. Users see this notification and take action to evacuate quickly and safely.

[1161] In this way, the present invention provides users with rapid and accurate information and supports appropriate responses during natural disasters.

[1162] The following describes the processing flow.

[1163] Step 1:

[1164] Terminals: Each terminal periodically collects sensor data (e.g., GPS location information, seismometer data). The smartphone's GPS module acquires location information, and the accelerometer collects ambient vibration data. This data is collected at regular time intervals and processed in batches.

[1165] Step 2:

[1166] Terminal: Collects sensor data and sends it to the server using a secure communication protocol (e.g., HTTPS). The data is packetized in a format such as JSON and transmitted over the network.

[1167] Step 3:

[1168] Server: The server receives data sent from the terminal. Data received via the API endpoint is immediately stored in the database. Data is validated and filtered upon receipt to remove duplicate and invalid data.

[1169] Step 4:

[1170] Server: Analyzes received data to determine the scale and location of the disaster. For example, if earthquake vibration data is transmitted, the server uses algorithms such as FFT analysis to calculate the epicenter and seismic intensity.

[1171] Step 5:

[1172] Server: Acquires additional data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with internal data. This external data is retrieved using a REST API and stored in the server's database. This generates more detailed and accurate disaster information.

[1173] Step 6:

[1174] Server: Based on analysis results and integrated information, generates emergency notifications to send to users. These emergency notifications include evacuation orders and recommended evacuation routes. The generated notifications are formatted using templates.

[1175] Step 7:

[1176] Server: Generates emergency notifications and sends them to each user's device using a push notification service (e.g., Firebase Cloud Messaging). The notification range is determined based on the target region and the user's current location.

[1177] Step 8:

[1178] User: The user receives an emergency notification from their device and checks it immediately. They begin evacuation according to the notification's contents. The smartphone app displays evacuation shelters and recommended evacuation routes to help the user evacuate safely.

[1179] The above outlines the specific flow of the system's program processing. This process enables a swift and appropriate response in the event of a disaster.

[1180] (Example 1)

[1181] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1182] Conventional disaster notification systems lacked real-time capabilities in collecting and analyzing sensor data, making it difficult to provide users with timely emergency notifications. Furthermore, the integration of data from multiple sources was insufficient, preventing improvements in the accuracy and detail of disaster information. As a result, users were unable to take appropriate evacuation actions quickly, increasing the risk of disaster damage.

[1183] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1184] In this invention, the server includes means for verifying, filtering, and storing sensor data; means for analyzing the sensor data using FFT analysis; and means for acquiring and integrating additional data from other organizations. This enables the generation of highly accurate and detailed disaster information and the rapid provision of emergency notifications to users through real-time analysis of sensor data and integration of data from multiple organizations.

[1185] "Sensor data" refers to data collected by various sensors installed in the device (e.g., GPS module, seismometer, accelerometer, water level sensor).

[1186] A "server" is a central computer system that receives, stores, and analyzes sensor data, and integrates additional data from other organizations.

[1187] "Verification and filtering" is the process of checking data sent to the server from the perspectives of accuracy, duplication, and fraud, and saving only the appropriate data.

[1188] "FFT analysis" refers to the Fast Fourier Transform algorithm used by servers to analyze sensor data, and is particularly used for vibration analysis of earthquake data.

[1189] "Additional data from other organizations" refers to supplementary data obtained from external organizations such as the Japan Meteorological Agency and river management bureaus, which are integrated with sensor data.

[1190] An "emergency notification" is a notification message, including warnings and evacuation orders, that is generated based on analysis results and integrated information when a disaster occurs.

[1191] A "communication network" is the infrastructure used to send and receive digital data, such as the internet or dedicated lines.

[1192] A "terminal" refers to a device, such as a smartphone or tablet, that collects sensor data and receives emergency notifications.

[1193] "Means of taking action" refers to the user checking emergency notifications from their device and then acting according to the designated evacuation instructions and evacuation routes.

[1194] This invention is a system that collects and analyzes information in real time during natural disasters and provides users with rapid emergency notifications. This system consists of three main components: a terminal, a server, and a user. Details of each are described below.

[1195] 1. Data Collection

[1196] Device: The user's device (e.g., smartphone) collects data using various sensors such as GPS modules, seismometers, accelerometers, and water level sensors. These sensors operate continuously and measure data at regular intervals. For example, a smartphone's accelerometer captures tremor information at a frequency of 5 times per second.

[1197] 2. Data transmission

[1198] Terminal: The collected sensor data is periodically batch-processed and securely sent to the server via the HTTPS protocol. For example, the terminal sends the data it collects to the server via HTTPS every 5 minutes.

[1199] 3. Data reception and storage

[1200] Server: The server receives data sent from the terminal and stores it in a database using Python and MySQL. During this process, it validates and filters the data to remove invalid or duplicate data.

[1201] 4. Data Analysis

[1202] Server: The server analyzes the received data to determine the scale and location of the disaster. In particular, in the case of earthquakes, it uses FFT (Fast Fourier Transform) analysis to calculate the epicenter and seismic intensity. The server identifies the epicenter and seismic intensity by performing FFT analysis on the vibration data it receives.

[1203] 5. Information Integration

[1204] Server: The server also acquires data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with its internal data. This generates more accurate and detailed information about disasters. It obtains the latest weather data from the Japan Meteorological Agency's API and integrates it with its own data.

[1205] 6. Emergency notification generation

[1206] Server: The server generates emergency notifications based on analysis results and integrated information. These notifications include evacuation orders and recommended evacuation routes. The server creates emergency notifications containing evacuation orders and routes based on the specified format.

[1207] 7. Send emergency notification

[1208] Server: Generated emergency notifications are pushed to each user's device using Firebase Cloud Messaging. The server calls Firebase Cloud Messaging to deliver the generated emergency notifications to all users' devices.

[1209] 8. User behavior

[1210] User: The user checks the emergency notification received on their device and begins evacuation. The notification includes detailed information on evacuation routes and locations, and the user evacuates to a safe place according to these instructions. The user checks the notification on their smartphone and heads to the nearest evacuation center according to the displayed evacuation instructions and evacuation route.

[1211] Specific example

[1212] For example, in the event of an earthquake, if the device's accelerometer detects strong shaking, it immediately sends this data to the server. The server analyzes the data from multiple devices to calculate the epicenter and seismic intensity. The server then obtains additional data from the Japan Meteorological Agency to further determine the extent of the disaster's impact. Finally, the server generates an emergency notification and sends a push notification to the user's device. The user checks this notification and evacuates to a safe place following the recommended evacuation route.

[1213] Example of a prompt

[1214] "An earthquake has occurred. Please evacuate to a safe place immediately. The nearest evacuation center is point A. Taking route B will ensure your safety."

[1215] Thus, the present invention provides users with rapid and accurate information and supports appropriate responses during natural disasters.

[1216] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1217] Step 1:

[1218] Data collection

[1219] Device: The user's device (smartphone) collects data in real time using its built-in GPS module, seismometer, accelerometer, and water level sensor.

[1220] Input: Measurement data from the sensor.

[1221] Data processing: Batch processing of data within the terminal (e.g., grouping data at regular intervals).

[1222] Output: Batch-processed sensor data.

[1223] Specific operation: The smartphone's accelerometer captures and temporarily stores information about shaking at a frequency of 5 times per second.

[1224] Step 2:

[1225] Data transmission

[1226] Terminal: Collected sensor data is periodically sent to the server via the HTTPS protocol.

[1227] Input: Batch-processed sensor data.

[1228] Data processing: Data encryption and secure transmission.

[1229] Output: Encrypted sensor data is sent to the server.

[1230] Specific operation: Every 5 minutes, the device sends the collected data to the server via HTTPS.

[1231] Step 3:

[1232] Data reception and storage

[1233] Server: Receives data sent from terminals, verifies and filters the data to ensure its accuracy, and then stores it in a database.

[1234] Input: Transmitted sensor data.

[1235] Data processing: Verification (e.g., checking data accuracy) and filtering (e.g., removing duplicate data).

[1236] Output: Clean sensor data is saved to the database.

[1237] Specific operation: The server uses Python and MySQL to validate the data and then save it to the database.

[1238] Step 4:

[1239] Data Analysis

[1240] Server: Analyzes stored data to determine the scale and location of disasters. In particular, in the case of earthquakes, it uses FFT (Fast Fourier Transform) analysis to calculate the epicenter and seismic intensity.

[1241] Input: Stored sensor data.

[1242] Data processing: Perform FFT analysis.

[1243] Output: Analysis results such as epicenter and seismic intensity.

[1244] Specific operation: The server applies FFT analysis to the vibration data it receives to identify the epicenter.

[1245] Step 5:

[1246] Information integration

[1247] Server: Acquires additional data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with internal data.

[1248] Input: Analysis results and additional data from external organizations.

[1249] Data processing: Data integration and analysis.

[1250] Output: Even more accurate disaster information.

[1251] Specific operation: Obtain the latest earthquake information from the Japan Meteorological Agency's API and integrate it with our own data.

[1252] Step 6:

[1253] Emergency notification generation

[1254] Server: Based on analysis results and integrated information, it generates optimized emergency notifications for each user (including evacuation orders and recommended evacuation routes).

[1255] Input: Analysis results and integrated information.

[1256] Data processing: Creating emergency notifications.

[1257] Output: Emergency notification message.

[1258] Specific operation: The server creates an emergency notification, including evacuation instructions and evacuation routes, based on the specified format.

[1259] Step 7:

[1260] Emergency notification sent

[1261] Server: The generated emergency notification is pushed to each user's device using Firebase Cloud Messaging.

[1262] Input: Generated emergency notification message.

[1263] Data processing: Sending notification messages.

[1264] Output: Emergency notification sent to the user's device.

[1265] Specific operation: The server calls Firebase Cloud Messaging and delivers the generated emergency notification to all users' devices.

[1266] Step 8:

[1267] User behavior

[1268] User: The user checks the emergency notification received on their device and begins evacuation according to the instructions in the notification.

[1269] Input: Emergency notification displayed on the device.

[1270] Output: User evacuation actions.

[1271] Specific actions: The user checks the notification on their smartphone and heads to the nearest evacuation shelter according to the displayed evacuation instructions and evacuation route.

[1272] (Application Example 1)

[1273] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1274] While systems exist that provide real-time information to help users evacuate quickly and safely during natural disasters, these systems generally assume that users will move manually. Therefore, especially in urban areas with a large number of autonomous vehicles, there is a lack of efficient and immediate means of evacuation. To overcome this drawback, a method is needed that automatically controls autonomous vehicles during natural disasters to select appropriate evacuation routes and guide them to safe locations.

[1275] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1276] In this invention, the server includes means for collecting sensor data, means for transmitting sensor data to the server, means for analyzing sensor data, means for acquiring and integrating additional data from other organizations, means for generating emergency notifications based on the analysis results and integrated information, means for transmitting emergency notifications to a terminal, means for the user to take action in response to the emergency notification displayed on the terminal, means for controlling the operation of the autonomous mobile vehicle and calculating a route for safe evacuation, and means for the autonomous mobile vehicle to move according to the calculated evacuation route. This enables the autonomous mobile vehicle to perform appropriate evacuation actions quickly and safely.

[1277] "Sensor data" refers to data collected from a wide variety of sensors (e.g., GPS, seismometers, accelerometers, water level sensors).

[1278] A "server" is a computer system that receives and analyzes sensor data via a network, and processes information by integrating additional data from other organizations.

[1279] "Means of sending to the server" refers to the function of securely and efficiently transferring collected sensor data to the server.

[1280] "Means of analysis" refers to methods of evaluating the scale, location, and impact of a disaster using collected sensor data.

[1281] "Methods for acquiring and integrating additional data from other organizations" refers to methods of collecting data from organizations such as the Japan Meteorological Agency and river management bureaus, and combining it with the data analyzed on the server.

[1282] "Means for generating emergency notifications" refers to a method of creating emergency information (e.g., evacuation orders, recommended evacuation routes) that should be conveyed to users, based on analysis results and integrated information.

[1283] "Means of sending emergency notifications to devices" refers to a function that transmits generated emergency notifications to user devices such as smartphones and tablets.

[1284] "Means of action for the user" refers to a function that facilitates actions to evacuate to a safe place based on emergency notifications displayed on the device.

[1285] An "autonomous mobile vehicle" is a vehicle that utilizes artificial intelligence and sensor technology to operate on its own judgment without external instructions.

[1286] "Means of controlling operation" refers to functions that manage the movement of autonomous mobile vehicles and ensure safe operation.

[1287] The "means for calculating routes" refer to a function that calculates the optimal evacuation route in the event of a disaster and instructs autonomous mobile vehicles accordingly.

[1288] "Means of transportation" refers to the ability of an autonomous vehicle to travel to its destination according to a calculated route.

[1289] This invention is a system that enables autonomous mobile vehicles to evacuate quickly and safely in the event of a natural disaster. Specific embodiments for realizing this system are described below.

[1290] 1. Data Collection

[1291] The terminal is equipped with various sensors, including a GPS module, seismometer, accelerometer, and water level sensor. These sensors periodically collect data and transmit it to a server via the network. For example, if an autonomous vehicle detects an earthquake, its accelerometer will acquire vibration data.

[1292] 2. Data transmission

[1293] The terminal periodically processes the collected sensor data in batches and sends it to the server via a secure communication protocol (such as HTTPS). It is recommended to use a high-speed and fault-tolerant communication method for this process.

[1294] 3. Data reception and storage

[1295] The server receives sensor data transmitted from terminals and stores it in a database. During this process, data verification and filtering are performed to remove duplicate and invalid data. For example, earthquake data transmitted simultaneously from multiple vehicles can be integrated.

[1296] 4. Data Analysis

[1297] The server analyzes the received data to determine the scale and location of the disaster. In the case of an earthquake, it uses algorithms such as FFT analysis to calculate the epicenter and seismic intensity. It also acquires data from the Japan Meteorological Agency and river management bureaus and integrates it with internal data to generate more accurate disaster information.

[1298] 5. Generate and send emergency notifications

[1299] The server generates an emergency notification based on the analysis results and integrated information. This notification includes evacuation instructions and recommended evacuation routes. The generated emergency notification is sent to each terminal via a push notification service. Based on the emergency notification received by the autonomous mobile vehicle, the vehicle's operation management system recalculates the route and selects the optimal evacuation route.

[1300] 6. Operation control of autonomous mobile vehicles

[1301] The autonomous mobile vehicle will automatically suspend its operation and move to a safe location in accordance with emergency notifications and recommended evacuation routes received from the server. During this process, in-vehicle sensors such as LiDAR and cameras will detect the surrounding environment to ensure safety during evacuation.

[1302] Specific example

[1303] For example, if an autonomous mobile vehicle detects an earthquake in central Tokyo, it would calculate the safest route to evacuate from that location and control the vehicle to move to a safe location. In this process, the system would need to monitor and avoid the conditions inside the vehicle and surrounding obstacles in real time.

[1304] Example of a prompt

[1305] "We are developing an application to enable the safe evacuation of autonomous mobile vehicles using a system that collects sensor data in real time during natural disasters and provides rapid emergency notifications. Data is collected from seismometers, accelerometers, water level sensors, etc., and transmitted to a server using a secure protocol. The server analyzes the data, generates an emergency notification, and sends it to the autonomous mobile vehicle. Based on the notification, the vehicle selects a safe evacuation route and moves to a safe location."

[1306] This embodiment enables autonomous mobile vehicles to evacuate quickly and safely in the event of a natural disaster, thereby protecting the lives and property of users.

[1307] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1308] Step 1: Data Collection

[1309] The device collects data from a GPS module, seismometer, accelerometer, and water level sensor. This allows it to obtain sensor data such as location information, seismic intensity, and surrounding water level information. The input consists of various sensor data, which is acquired in real time by the device. The output is the collected sensor data.

[1310] Step 2: Data transmission

[1311] The terminal periodically batches the collected sensor data and sends it to the server via a secure communication protocol (e.g., HTTPS). The input is the collected sensor data, which is formatted and encrypted for secure transmission to the server. The output is the transmitted sensor data arriving at the server.

[1312] Step 3: Data reception and storage

[1313] The server receives sensor data transmitted from the terminal and stores it in a database. During this process, the data is validated and filtered to remove duplicate and invalid data. The input is the transmitted sensor data, and the output is a clean and consistent dataset stored in the database.

[1314] Step 4: Data Analysis

[1315] The server analyzes sensor data stored in the database. Specifically, it analyzes earthquake vibration data using algorithms such as FFT analysis to determine the scale and location of the disaster. It also acquires additional data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with the internal data. The input is the stored sensor data and additional data, and the output is the analysis results that identify the scale and location of the disaster.

[1316] Step 5: Generate emergency notification

[1317] The server generates emergency notifications based on the analysis results and integrated information. These notifications include evacuation orders and recommended evacuation routes. The input is the data analysis results and integrated information, and the output is the generated emergency notification.

[1318] Step 6: Send emergency notification

[1319] The server sends the generated emergency notification to the device. This allows the user to receive important information in real time. The input is the generated emergency notification, and the output is displayed on the device as a push notification.

[1320] Step 7: Operation control of autonomous mobile vehicles

[1321] The terminal relays received emergency notifications to the autonomous mobile vehicle's operation management system, thereby controlling autonomous driving. The input is the received emergency notification, and the output is a recalculation of the vehicle's route, selecting a safe evacuation path.

[1322] Step 8: Evacuation Action

[1323] The user takes action based on the emergency notification displayed on their device. The autonomous mobile vehicle moves along a calculated route and evacuates to a safe location. The input is the selected evacuation route, and the output is the user's action of evacuating to a safe location.

[1324] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1325] This invention combines a system that collects and analyzes information in real time during natural disasters and provides users with rapid emergency notifications with an emotion engine that recognizes the user's emotions. This system consists of four main components: a terminal, a server, an emotion engine, and a user.

[1326] System Overview

[1327] 1. Data Collection

[1328] Device: Each user's device (e.g., smartphone) is equipped with a GPS module for acquiring location information, as well as seismometers, accelerometers, water level sensors, and other sensors. These sensors periodically collect data and transmit it to a server via the network.

[1329] 2. Data transmission

[1330] Terminal: Collected sensor data is periodically batch-processed and sent to the server via a secure communication protocol (e.g., HTTPS). The data is packetized in a format such as JSON and transmitted over the network.

[1331] 3. Data reception and storage

[1332] Server: The server receives data sent from terminals and stores it in the database. Data is validated and filtered upon receipt, removing duplicate and invalid data.

[1333] 4. Data Analysis

[1334] Server: The server analyzes the received data to determine the scale and location of the disaster. For example, if earthquake vibration data is transmitted, the server uses algorithms such as FFT analysis to calculate the epicenter and seismic intensity.

[1335] 5. Information Integration

[1336] Server: The server also acquires data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with internal data. This external data is retrieved using a REST API and stored in the server's database. This generates more detailed and accurate disaster information.

[1337] 6. Analysis using an emotion engine

[1338] Server: The emotion engine built into the server analyzes the user's voice and text messages to identify the user's emotional state. For example, it analyzes in real time what the user says to the device and the messages they type to detect stress or panic.

[1339] 7. Emergency notification generation

[1340] Server: Based on the analysis results of the emotion engine, the server generates emergency notifications considering the user's emotional state. Emergency notifications include evacuation orders, recommended evacuation routes, and psychological support information. The content and wording of the notifications are adjusted as needed.

[1341] 8. Send emergency notification

[1342] Server: Generates emergency notifications and sends them to each user's device using a push notification service (e.g., Firebase Cloud Messaging). The notification range is determined based on the target region and the user's current location.

[1343] 9. User behavior

[1344] User: The user receives an emergency notification from their device and checks it immediately. They begin evacuation actions according to the notification. The smartphone app displays evacuation shelters and recommended evacuation routes to help the user evacuate safely. Furthermore, the user can act calmly by referring to psychological support information provided by the emotion engine.

[1345] Specific examples

[1346] For example, consider the case of an earthquake. When the device's accelerometer detects a strong tremor, it immediately sends this data to the server. The server analyzes the data from multiple devices to calculate the epicenter and seismic intensity. The server then obtains additional data from the Japan Meteorological Agency to further determine the extent of the disaster's impact. When the emotion engine detects the user's stress level from their voice or text, the server generates an emergency notification that takes this into account and sends it to the user's device. The user checks this notification and evacuates to a safe place following the recommended evacuation route. At the same time, psychological support messages provided by the emotion engine help the user remain calm.

[1347] Furthermore, if a flood warning is issued, the device collects water level sensor data and sends it to the server. The server analyzes this data and predicts the likelihood of flooding. It integrates additional data (e.g., water level data from the river management authority) to determine the extent of the flood's impact. The emotion engine analyzes the user's emotional state and, if necessary, includes psychological support information to help them stay calm in the emergency notification. The user receives this notification and evacuates safely.

[1348] In this way, the present invention not only provides users with rapid and accurate information and supports appropriate responses during natural disasters, but also, by combining it with an emotion engine, takes into account the user's psychological state and provides more effective support.

[1349] The following describes the processing flow.

[1350] Step 1:

[1351] Terminals: Each terminal periodically collects sensor data (e.g., GPS location information, seismometer data). The smartphone's GPS module acquires location information, and the accelerometer collects ambient vibration data. This data is collected at regular time intervals and processed in batches.

[1352] Step 2:

[1353] Terminal: Collects sensor data and sends it to the server using a secure communication protocol (e.g., HTTPS). The data is packetized in a format such as JSON and transmitted over the network.

[1354] Step 3:

[1355] Server: The server receives data sent from the terminal. Data received via the API endpoint is immediately stored in the database. Data is validated and filtered upon receipt, removing duplicate and invalid data.

[1356] Step 4:

[1357] Server: Analyzes received data to determine the scale and location of the disaster. For example, if earthquake vibration data is transmitted, the server uses algorithms such as FFT analysis to calculate the epicenter and seismic intensity.

[1358] Step 5:

[1359] Server: Acquires additional data from other organizations (e.g., Japan Meteorological Agency, River Management Bureau) and integrates it with internal data. This external data is retrieved using a REST API and stored in the server's database. This generates more detailed and accurate disaster information.

[1360] Step 6:

[1361] Server: The emotion engine built into the server analyzes the user's voice and text messages to identify the user's emotional state. For example, it analyzes in real time what the user says to the device and the messages they type to detect stress or panic.

[1362] Step 7:

[1363] Server: Based on the analysis results of the emotion engine, the server generates emergency notifications considering the user's emotional state. Emergency notifications include evacuation orders, recommended evacuation routes, and psychological support information. The content and wording of the notifications are adjusted as needed.

[1364] Step 8:

[1365] Server: Generates emergency notifications and sends them to each user's device using a push notification service (e.g., Firebase Cloud Messaging). The notification range is determined based on the target region and the user's current location.

[1366] Step 9:

[1367] User: The user receives an emergency notification from their device and checks it immediately. They begin evacuation according to the notification's contents. The smartphone app displays evacuation shelters and recommended evacuation routes to help the user evacuate safely. They also act calmly, referring to psychological support information provided by the emotion engine.

[1368] The above outlines the specific program processing flow of the system that incorporates the emotion engine. This process enables a rapid and appropriate response in the event of a disaster, while simultaneously providing psychological support to the user.

[1369] (Example 2)

[1370] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1371] When natural disasters occur, there is a need for rapid and accurate information provision and appropriate responses that take into account the psychological state of users. Conventional systems do not adequately collect and analyze real-time data, making it difficult to provide support that takes into account the emotional state of users. In addition, emergency notifications are uniform, making it difficult to respond to the individual circumstances of users.

[1372] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting sensor data, means for analyzing the sensor data, means for acquiring and integrating additional data from other organizations, means for collecting user voice and text data and identifying their emotional state, means for generating an emergency notification based on the analysis results, integrated information, and the user's emotional state, and means for sending the emergency notification as a push notification to the terminal. This enables the rapid and accurate provision of disaster information and individualized responses that take into account the user's psychological state.

[1373] "Sensor data" refers to data collected from various sensors installed in the user's device (e.g., GPS module, seismometer, accelerometer, water level sensor).

[1374] A "server" is a remote computing device that receives, analyzes, and stores sensor data transmitted from terminals, acquires additional data from other organizations, and generates and transmits emergency notifications.

[1375] "Analysis" is the process of applying algorithms to received sensor data to extract meaningful information.

[1376] "Integration" refers to the process of combining data obtained from other organizations with internal data to create more detailed and accurate disaster information.

[1377] An "emotion engine" is software that analyzes a user's voice and text data to identify the user's emotional state.

[1378] An "emergency notification" is a message generated during a disaster that provides users with evacuation instructions and psychological support information.

[1379] "Push notifications" are a real-time notification format that a server sends directly to a user's device.

[1380] A "terminal" refers to a personal information terminal (e.g., a smartphone) used by a user, which is a device that collects sensor data and displays emergency notifications.

[1381] A "user" is a person who uses this system to receive emergency notifications and take action.

[1382]

[1383] This invention is a system that collects and analyzes information in real time during natural disasters and provides users with rapid emergency notifications. This system consists of four main components: a terminal, a server, an emotion engine, and a user.

[1384] First, the device is the user's smartphone, equipped with a GPS module, seismometer, accelerometer, water level sensor, and other sensors. These sensors periodically collect current location information, earthquake vibration data, and water level data. For example, when an earthquake occurs, the device's accelerometer detects the shaking and records the data.

[1385] Next, the collected sensor data is batch-processed at regular intervals and sent to the server using the HTTPS protocol. The data is sent in JSON format. Specifically, for example, seismometer data is converted into a JSON structure such as "{ 'timestamp': '2023-10-01T12:00:00Z', 'sensor': 'accelerometer', 'value': 5.6}" and sent as an HTTPS request. The server waits for reception at a specific endpoint and receives the data when it receives an HTTPS request from the terminal.

[1386] The server stores the received sensor data in a database and performs data validation and filtering. For example, it checks the timestamp and sensor type included in the received data to eliminate invalid data. The filtered data is then applied for analysis. Using FFT analysis and other algorithms, for example, it identifies the epicenter and seismic intensity from earthquake vibration data.

[1387] Furthermore, the server acquires additional data from other organizations (e.g., the Japan Meteorological Agency, river management bureaus) and integrates it with internal data. At this time, it uses a REST API to retrieve external data and generates detailed and accurate disaster information after integration. For example, it acquires earthquake reports from the Japan Meteorological Agency and integrates them with data from terminals to determine the affected area.

[1388] An emotion engine is also integrated into the server, analyzing the voice and text data that users input into their devices. This engine can identify the user's emotional state and detect stress or panic. For example, if a user inputs "I'm scared of earthquakes," the emotion engine will detect a stressed state from that message.

[1389] The server generates emergency notifications based on analysis results and the results of the emotion engine. These emergency notifications include evacuation instructions and psychological support information that takes the user's emotional state into consideration, and the content is customized as needed. For example, in addition to a basic notification such as "Please evacuate," it also sends psychological support messages such as "Please stay calm."

[1390] Finally, the server sends the generated emergency notification to the user's device using a push notification service such as Firebase Cloud Messaging. The notification is sent in real time, and the delivery range is set based on the target area and the user's current location. The user receives this notification and takes appropriate action according to the displayed evacuation route and recommended actions.

[1391] This system provides users with rapid and accurate disaster information and psychological support, assisting them in taking appropriate action during emergencies. An example of a prompt message generated using the AI ​​model is: "There is an earthquake. Please evacuate immediately. Please remain calm."

[1392] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1393] Step 1: Data Collection

[1394] The device collects sensor data from the user's smartphone, including GPS modules, seismometers, accelerometers, and water level sensors. For example, when a seismometer detects shaking, it collects the data and stores it in a buffer. The input is data acquired from the sensor modules, and the output is raw data stored in the device's internal buffer.

[1395] Step 2: Convert Data Format

[1396] The terminal processes the collected sensor data in batches at regular time intervals and converts it to JSON format. For example, it converts seismometer data into a JSON structure. The input is raw data stored in the terminal's buffer, and the output is data formatted in JSON format.

[1397] Step 3: Data transmission

[1398] The terminal sends data converted to JSON format to the server using the HTTPS protocol. For example, the converted JSON data is sent to the server's endpoint via an HTTPS request. The input is data formatted in JSON format, and the output is the data sent to the server.

[1399] Step 4: Received

[1400] The server receives data sent from the terminal. For example, it receives HTTPS requests at a specific endpoint on the server. The input is the data sent from the terminal, and the output is the JSON data received by the server.

[1401] Step 5: Data validation and filtering

[1402] The server validates the received data and removes invalid and duplicate data. For example, it checks the data's timestamp and sensor type to eliminate invalid and duplicate data. The input is the JSON data received by the server, and the output is the filtered, accurate data.

[1403] Step 6: Save Data

[1404] The server stores the filtered data in a database. For example, it adds verified data to a MySQL database. The input is the filtered, accurate data, and the output is the data stored in the database.

[1405] Step 7: Data Analysis

[1406] The server analyzes the received and stored data to determine the scale and location of a disaster. For example, it uses FFT analysis to identify the epicenter and seismic intensity. The input is sensor data stored in a database, and the output is the analysis results.

[1407] Step 8: Obtaining external data

[1408] The server retrieves additional data from external organizations (e.g., the Japan Meteorological Agency) via a REST API. For example, it uses the Japan Meteorological Agency's API to retrieve the latest earthquake observation data. The input is the endpoint information of the external API, and the output is the retrieved external data.

[1409] Step 9: Data Integration

[1410] The server integrates acquired external and internal data to generate detailed and accurate disaster information. Inputs are analysis results and external data, while output is the integrated disaster information.

[1411] Step 10: Collecting emotional data

[1412] The server collects voice and text data that the user inputs into the terminal. For example, the user inputs "I'm scared of earthquakes" into the terminal. The input is voice or text data from the user, and the output is collected emotion data.

[1413] Step 11: Analyzing emotional data

[1414] The server uses an emotion engine to analyze voice and text data to identify the user's emotional state. For example, it can detect a stressed state from the input text "scared." The input is the collected emotion data, and the output is the emotional state identified through the analysis.

[1415] Step 12: Generate an emergency notification

[1416] The server generates emergency notifications based on analysis results and emotional states. For example, it creates notification messages that include evacuation orders and psychological support information. The input is the analysis results and emotional states, and the output is the generated emergency notification.

[1417] Step 13: Preparing for push notifications

[1418] The server prepares push notifications using services such as Firebase Cloud Messaging. For example, it uses the Firebase API to create a list of target devices. The input is the generated emergency notification, and the output is the prepared push notification.

[1419] Step 14: Sending Push Notifications

[1420] The server sends the generated emergency notification to the user's device. For example, it might prioritize sending notifications to users in a specific region. The input is a prepared push notification, and the output is the emergency notification sent to the user's device.

[1421] Step 15: Check the notification

[1422] The user receives an emergency notification from their device and checks it immediately. For example, they might check a notification that says, "An earthquake has occurred. Please evacuate." The input is the emergency notification sent to the device, and the output is the user's action upon receiving the notification.

[1423] Step 16: Evacuation Action

[1424] The user initiates evacuation actions according to the notification. For example, they might move to a safe location using the notified evacuation route. The input is the notification content, and the output is the user's evacuation actions.

[1425] Step 17: Supporting calm behavior

[1426] The user acts calmly based on the psychological support information provided by the emotion engine. For example, they reduce stress by reading a message that says, "Please act calmly." The input is the support information from the emotion engine, and the output is the user acting calmly.

[1427] (Application Example 2)

[1428] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1429] In recent years, the frequency and scale of natural disasters have increased, creating a demand for rapid and accurate information provision and support. However, conventional systems struggle to efficiently integrate and analyze sensor data and data from external organizations, and furthermore, they are unable to provide emergency responses that take into account the emotional state of users. As a result, users are unable to take appropriate actions during disasters, leading to increased psychological burden. In addition, even in autonomous vehicles, providing real-time disaster information and appropriate responses based on emotional analysis are necessary to ensure passenger safety.

[1430] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting sensor data, means for transmitting the sensor data to the server, means for analyzing the sensor data, means for acquiring and integrating additional data from other organizations, means for generating emergency notifications based on the analysis results and integrated information, means for transmitting the emergency notifications to the terminal, means for analyzing the emotional state from the user's voice and facial expressions, and means for adjusting the content and expression of the emergency notification considering the emotional state. This enables not only the real-time and accurate provision of disaster information, but also appropriate notifications and support that respond to the user's emotions. Furthermore, by applying this to autonomous vehicles, passenger safety and a sense of psychological security can be ensured.

[1431] "Sensor data" refers to physical or environmental data collected by sensors, including location information, acceleration, vibration, water level, sound, and images.

[1432] "Means of sending to a server" refers to means that have the function of transferring collected sensor data to a server via the internet or other communication networks.

[1433] "Methods for data analysis" refer to methods of analyzing sensor data received on a server and using algorithms and computational techniques to determine the scale and location of a disaster.

[1434] "Means for acquiring and integrating additional data" refers to methods for acquiring data provided by external organizations and combining it with existing sensor data to generate more detailed information.

[1435] "Means for generating emergency notifications" refers to means that have the function of creating notification messages, including warnings and instructions for users, based on analyzed data and integrated information.

[1436] "Means of sending to a terminal" refers to means that have the function of sending the generated emergency notification to the user's communication device (e.g., smartphone, tablet, PC, etc.).

[1437] "Methods for analyzing emotional states from voice and facial expressions" refers to methods that have the function of detecting and analyzing a user's emotional state based on the user's voice and facial expression data.

[1438] "Means for adjusting the content and wording of emergency notifications in consideration of emotional state" refers to means that have the function of adjusting emergency notification messages to be more appropriate and effective based on the analyzed emotional state of the user.

[1439] "Means of taking action" refers to the means by which a user takes appropriate action or response measures based on an emergency notification displayed on their device.

[1440] "Means for obtaining current location" refers to means that have the function of obtaining the user's current physical location using location information technology such as GPS.

[1441] "Means of providing optimal evacuation routes" refers to a means that has the function of suggesting and directing the safest and most efficient evacuation route based on the user's current location and disaster information.

[1442] "Means for managing the location information of support resources" refers to means that have the function of centrally managing the location information of evacuation centers and relief supplies, and providing that information as needed.

[1443] "Means of providing support resources" refers to means of delivering information on evacuation sites and relief supplies to users and providing the maximum possible support.

[1444] Modes for carrying out the invention

[1445] The system according to the present invention collects and analyzes information in real time when a natural disaster occurs and provides users with a rapid emergency notification. This system consists of the following components.

[1446] 1. Terminal components and data collection

[1447] The device is equipped with various sensors (GPS, altimeter, accelerometer, microphone, camera, etc.) and collects data from these sensors in real time. For example, recent location information, earthquake vibration data, audio data, and video data are collected.

[1448] 2. Data transmission

[1449] The collected sensor data is securely transmitted to the server using the HTTPS communication protocol. The data is packetized in JSON format and transmitted periodically through batch processing.

[1450] 3. Data Analysis and Information Integration

[1451] The server analyzes received sensor data in real time. For example, it analyzes earthquake vibration data using FFT analysis to identify the epicenter and seismic intensity. Based on these results, it confirms the location and scale of the disaster. At the same time, it acquires and integrates additional data from other organizations (e.g., external data from the Japan Meteorological Agency and river management bureaus) using REST APIs to generate more detailed and accurate disaster information.

[1452] 4. Emotion analysis

[1453] The emotion engine embedded in the server analyzes audio and video data transmitted from the terminal to identify the user's emotional state in real time. Specifically, it uses audio and facial expression analysis algorithms to detect stress levels and panic states. Machine learning libraries such as Python, TensorFlow, and Keras are used for emotion analysis.

[1454] 5. Generation and transmission of emergency notifications

[1455] The server generates emergency notifications based on analytical data, including the results of sentiment analysis. These emergency notifications include evacuation routes, psychological support information, and specific action instructions. The notifications are sent to the user's device using Firebase Cloud Messaging (FCM).

[1456] 6. User behavior and evacuation support

[1457] Users take action according to emergency notifications displayed on their devices. The device displays their current location and navigates them to the optimal evacuation route. It also provides location information for shelters and support resources to help users evacuate quickly and safely.

[1458] Specific examples

[1459] For example, in the event of an earthquake, the accelerometer in the device detects strong shaking and sends that data to a server. The server analyzes the data received from multiple devices to identify the epicenter and seismic intensity. Simultaneously, an emotion engine analyzes the user's voice and facial expressions to detect their stress level. Based on this, the server generates an emergency notification, which includes evacuation routes and psychological support information, and sends it to the user. The user can then receive this notification and take safe evacuation actions.

[1460] Examples of input prompts for a generative AI model

[1461] Write an algorithm to collect sensor data and send it to a server when an autonomous vehicle detects a strong tremor. Also, write code to generate emergency notifications based on the passenger's stress level and adjust the autonomous driving mode accordingly.

[1462] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1463] Step 1:

[1464] Data collection

[1465] The device uses sensors such as a GPS module, accelerometer, camera, and microphone to collect sensor data including current location information, vibration data, video data, and audio data. This data is collected in real time and temporarily stored in internal memory for batch processing. The main input is sensor data, and the output is batch-processed data.

[1466] Step 2:

[1467] Data transmission

[1468] The terminal transmits collected sensor data to the server using the HTTPS communication protocol. The data is packetized in JSON format and securely transferred to the server. The input is batch-processed sensor data, and the output is the data sent to the server.

[1469] Step 3:

[1470] Data reception and storage

[1471] The server receives sensor data transmitted from the terminal. Before the received data is stored in the database, it undergoes validation and filtering. Duplicate and invalid data are removed at this stage. The input is the received sensor data, and the output is the validated data stored in the database.

[1472] Step 4:

[1473] Data Analysis

[1474] The server analyzes sensor data stored in the database to determine the scale and location of a disaster. For example, it uses FFT analysis to calculate the earthquake's epicenter and intensity. The input is validated sensor data, and the output is the analysis results regarding the scale and location of the disaster.

[1475] Step 5:

[1476] Information integration

[1477] The server acquires additional data in real time from external sources. It uses a REST API to obtain weather information, river management information, and other data, and integrates it with internal sensor data. The input consists of additional data acquired from external sources and internal analysis results, and the output is integrated, detailed disaster information.

[1478] Step 6:

[1479] Emotion analysis

[1480] The emotion engine embedded in the server analyzes voice and facial expression data transmitted from the terminal. It uses machine learning models to identify the user's stress level and emotional state. The input is voice and facial expression data, and the output is the analyzed emotional state.

[1481] Step 7:

[1482] Emergency notification generation

[1483] The server generates emergency notifications based on analyzed disaster information and the user's emotional state. These notifications include evacuation orders, recommended evacuation routes, and psychological support information. The input is integrated disaster information and emotional state, while the output is the generated emergency notification.

[1484] Step 8:

[1485] Emergency notification sent

[1486] The server sends the generated emergency notification to each user's device using Firebase Cloud Messaging (FCM). The input is the emergency notification, and the output is the notification sent to the device.

[1487] Step 9:

[1488] User behavior

[1489] The user receives an emergency notification displayed on the device and takes evacuation action according to the instructions. The device displays the user's current location and the optimal evacuation route, guiding them along the way. The input is the received emergency notification, and the output is the user's evacuation action. Psychological support information is also provided to help the user act calmly.

[1490] In this way, the system can provide users with rapid and accurate information and psychological support through multi-layered data collection and analysis.

[1491] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1492] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1493] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1494] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1495] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1496] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1497] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1498] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1499] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1500] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1501] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1502] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1503] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1504] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1505] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1506] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1507] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1508] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1509] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1510] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1511] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1512] The following is further disclosed regarding the embodiments described above.

[1513] (Claim 1)

[1514] Means for collecting sensor data,

[1515] Means for transmitting the aforementioned sensor data to a server,

[1516] Means for analyzing the aforementioned sensor data,

[1517] A means of acquiring and integrating additional data from other institutions,

[1518] A means for generating an emergency notification based on the aforementioned analysis results and integrated information,

[1519] A means for sending the aforementioned emergency notification to the terminal,

[1520] Means by which the user takes action in response to an emergency notification displayed on the aforementioned terminal.

[1521] A system that includes this.

[1522] (Claim 2)

[1523] The system according to claim 1, comprising means for obtaining the user's current location and providing an optimal evacuation route in accordance with the emergency notification.

[1524] (Claim 3)

[1525] The system according to claim 1, comprising means for managing location information of evacuation shelters and relief supplies and providing users with the most suitable support resources.

[1526] "Example 1"

[1527] (Claim 1)

[1528] Means for collecting sensor data,

[1529] Means for transmitting the aforementioned sensor data to a server,

[1530] Means for verifying, filtering, and storing the aforementioned sensor data,

[1531] The means for analyzing the aforementioned sensor data using FFT analysis,

[1532] Means for obtaining and integrating additional data from other institutions,

[1533] A means for generating an emergency notification based on the aforementioned analysis results and integrated information,

[1534] Means for transmitting the aforementioned emergency notification to a terminal via a communication network,

[1535] Means by which the user takes action in response to an emergency notification displayed on the aforementioned terminal.

[1536] A system that includes this.

[1537] (Claim 2)

[1538] The system according to claim 1, comprising means for obtaining the user's current location and providing an optimal evacuation route in accordance with the emergency notification.

[1539] (Claim 3)

[1540] The system according to claim 1, comprising means for managing location information of evacuation shelters and relief supplies and providing users with the most suitable support resources.

[1541] "Application Example 1"

[1542] (Claim 1)

[1543] Means for collecting sensor data,

[1544] Means for transmitting the aforementioned sensor data to a server,

[1545] Means for analyzing the aforementioned sensor data,

[1546] A means of acquiring and integrating additional data from other institutions,

[1547] A means for generating an emergency notification based on the aforementioned analysis results and integrated information,

[1548] A means for sending the aforementioned emergency notification to the terminal,

[1549] A means by which the user takes action in response to an emergency notification displayed on the aforementioned terminal,

[1550] A means for controlling the operation of autonomous mobile vehicles and calculating safe evacuation routes,

[1551] Means by which the autonomous mobile vehicle moves according to the calculated evacuation route

[1552] A system that includes this.

[1553] (Claim 2)

[1554] The system according to claim 1, comprising means for obtaining the user's current location and providing an optimal evacuation route in accordance with the emergency notification.

[1555] (Claim 3)

[1556] The system according to claim 1, comprising means for managing location information of evacuation shelters and relief supplies and providing users with the most suitable support resources.

[1557] "Example 2 of combining an emotion engine"

[1558] (Claim 1)

[1559] Means for collecting sensor data,

[1560] Means for transmitting the aforementioned sensor data to a server,

[1561] Means for analyzing the aforementioned sensor data,

[1562] A means of acquiring and integrating additional data from other institutions,

[1563] A means of collecting user voice and text data to identify their emotional state,

[1564] A means for generating an emergency notification based on the aforementioned analysis results and integrated information, as well as the user's emotional state,

[1565] A means for sending the aforementioned emergency notification to the terminal as a push notification,

[1566] Means by which the user takes action in response to an emergency notification displayed on the aforementioned terminal.

[1567] A system that includes this.

[1568] (Claim 2)

[1569] The system according to claim 1, comprising means for obtaining the user's current location and providing an optimal evacuation route in accordance with the emergency notification.

[1570] (Claim 3)

[1571] The system according to claim 1, comprising means for managing location information of evacuation shelters and support resources and providing the user with the most suitable support resources.

[1572] "Application example 2 when combining with an emotional engine"

[1573] (Claim 1)

[1574] Means for collecting sensor data,

[1575] Means for transmitting the aforementioned sensor data to a server,

[1576] Means for analyzing the aforementioned sensor data,

[1577] A means of acquiring and integrating additional data from other institutions,

[1578] A means for generating an emergency notification based on the aforementioned analysis results and integrated information,

[1579] A means for sending the aforementioned emergency notification to the terminal,

[1580] A method for analyzing the emotional state from the user's voice and facial expressions,

[1581] Means for adjusting the content and wording of emergency notices in consideration of the aforementioned emotional state,

[1582] Means by which the user takes action in response to an emergency notification displayed on the aforementioned terminal.

[1583] A system that includes this.

[1584] (Claim 2)

[1585] The system according to claim 1, comprising means for obtaining the user's current location and providing an optimal evacuation route in accordance with the emergency notification.

[1586] (Claim 3)

[1587] The system according to claim 1, comprising means for managing location information of evacuation shelters and support resources and providing the user with the most suitable support resources. [Explanation of symbols]

[1588] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means for collecting sensor data, Means for transmitting the aforementioned sensor data to a server, Means for analyzing the aforementioned sensor data, A means of acquiring and integrating additional data from other institutions, A means for generating an emergency notification based on the aforementioned analysis results and integrated information, A means for sending the aforementioned emergency notification to the terminal, Means by which the user takes action in response to an emergency notification displayed on the aforementioned terminal. A system that includes this.

2. The system according to claim 1, comprising means for obtaining the user's current location and providing an optimal evacuation route in accordance with the emergency notification.

3. The system according to claim 1, comprising means for managing location information of evacuation centers and relief supplies and providing users with the most suitable support resources.

Images