system

Abstract

To provide a system that enables rapid rescue and effective support for disaster victims. [Solution] A system including: means for monitoring seismic intensity information and detecting earthquakes of intensity 7 or higher; means for sending an alert to a user's terminal based on the seismic intensity information; means for automatically activating the user's terminal that receives the alert; means for detecting the user's voice and generating and sending a rescue request message; means for switching to ad hoc communication mode if the Internet connection is lost; means for providing first aid instructions and psychological support using edge AI; and means for sending the user's location information and status information to a server and storing it in a database.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Japan is a region where earthquakes occur frequently, and when a large earthquake occurs, many people are affected and prompt rescue is required. However, with the flood of rescue requests via social media and telephone, information about people who truly need help is often buried. Furthermore, when internet connections are cut off, it becomes difficult for victims to share information with each other. Another problem is that victims are unable to receive appropriate first aid instructions while feeling psychological anxiety and fear. [Means for solving the problem]

[0005] This system monitors seismic intensity information and, if it detects an earthquake of magnitude 7 or higher, sends an alert to the user's device and automatically activates the device. The device detects the user's voice and generates and transmits a rescue request message. Furthermore, if the internet connection is lost, it switches to ad-hoc communication mode and communicates directly with other devices to share information. Edge AI is used to provide first aid instructions and psychological support, and the user's location and status information is sent to a server and stored in a database. It also provides a means for the user to record a final message, which is automatically sent to family members or designated contacts when the internet connection is restored. This system enables rapid rescue and effective support for disaster victims.

[0006] "Seismic intensity information" is data that indicates the strength of shaking observed when an earthquake occurs.

[0007] "User Device" refers to an individual mobile phone, smartphone, or other portable electronic device used during an earthquake disaster.

[0008] An "alert" is a notification or warning message sent to get the user's attention when a specific condition is met.

[0009] "Voice detection" refers to technology that detects the user's voice through a microphone and analyzes it.

[0010] A "rescue request message" refers to a message sent by a disaster victim to request rescue when a disaster occurs.

[0011] "Ad hoc communication" refers to a technology that allows devices to communicate directly with each other without using a fixed network infrastructure.

[0012] "Edge AI" refers to a technology that uses artificial intelligence (AI) to analyze data and make decisions in real time on a device, rather than relying on the cloud.

[0013] "First Aid Instructions" means instructions that provide a user with measures or procedures to follow in the event of an emergency such as injury or illness.

[0014] "Psychological support" refers to providing assistance and advice to promote mental well-being in people facing disasters and emergencies.

[0015] "Location Information" refers to data from GPS or other positioning technologies that indicates the location of a user's device.

[0016] A "database" refers to a collection of data that stores information in a systematic way so that it can be easily accessed and managed.

[0017] A "final message" refers to a voice or text message left by a disaster victim for family and friends during a crisis.

[0018] "Internet connection restored" refers to the moment when a temporarily lost Internet connection becomes available again. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] System Overview

[0041] This invention is a system for providing rapid and effective rescue and support during large-scale disasters such as earthquakes. This system automatically activates when an earthquake of magnitude 7 or higher occurs, detects the voices of victims, and issues rescue requests. It also enables information sharing using ad-hoc communication even when the internet connection is lost, and uses edge AI to provide first aid instructions and psychological support.

[0042] Server Features

[0043] The server monitors earthquake information and sends an alert to user devices in the relevant area when an earthquake of magnitude 7 or higher occurs. This alert automatically wakes up the user device and triggers the rescue request process. The server also stores and continuously updates the user's location and status information in a database to support rescue efforts.

[0044] Device Features

[0045] The device automatically wakes up when it receives an alert from the server, detects the user's voice, and generates the necessary rescue request message. The rescue request message includes the user's current location and a brief description of the situation, and is immediately sent to the server if there is an Internet connection. Furthermore, if the Internet connection is lost, the device automatically switches to ad-hoc communication mode and shares information by directly communicating with other devices in the vicinity.

[0046] The Edge AI runs on the device and provides first aid instructions and psychological support to victims. For example, if the user is injured, it will provide specific voice instructions such as "If there is bleeding, apply pressure with a clean cloth." If the user is feeling anxious or scared, it will provide psychological support such as "Please stay calm. Take a deep breath and relax."

[0047] User Roles

[0048] When an earthquake occurs, users follow the instructions provided by the device to ensure their own safety and, if necessary, request rescue by voice. When the device detects voice, it automatically generates a rescue request message and sends it to the server. Users can also record a final message on the device, which will be automatically sent to designated contacts as soon as the internet connection is restored.

[0049] Specific use cases

[0050] Use case 1: Immediately after an earthquake occurs

[0051] The server detects earthquakes of magnitude 7 or higher and sends alerts to users' devices in the affected area.

[0052] The device will automatically launch the app and enter voice detection mode.

[0053] When a user says "Help!", the device detects the voice and generates and sends a rescue request message.

[0054] Use case 2: When the Internet is down

[0055] The device checks for an Internet connection and, if it determines that it is unavailable, automatically switches to ad-hoc communication mode.

[0056] The device communicates directly with the devices of other victims in the vicinity, sharing rescue request information and location information.

[0057] Use case 3: When first aid is required

[0058] If the user is injured, the device will use edge AI to give voice instructions such as, "If there is bleeding, apply pressure with a clean cloth."

[0059] Edge AI also provides psychological support, such as "Stay calm. Take a deep breath and relax."

[0060] This system allows disaster victims to receive prompt and effective assistance and creates an environment in which local and community cooperation is facilitated.

[0061] The processing flow will be explained below.

[0062] Step 1:

[0063] The server monitors earthquake information.

[0064] Data is regularly obtained from the Japan Meteorological Agency and earthquake early warning services and the seismic intensity is analyzed.

[0065] Step 2:

[0066] If an earthquake with a magnitude of 7 or higher is detected, the server will send an alert to user devices in the relevant area.

[0067] The alert will include information on the earthquake's epicenter and magnitude.

[0068] Step 3:

[0069] The device receives an alert from the server and automatically launches the app.

[0070] When the app is launched, the microphone goes into voice detection mode.

[0071] Step 4:

[0072] The user makes a sound requesting rescue, such as "help me."

[0073] The sound is detected by the device's microphone.

[0074] Step 5:

[0075] The device analyzes the voice and generates a message requesting help.

[0076] The rescue request message will include the user's location and a brief description of the situation.

[0077] Step 6:

[0078] The terminal sends a rescue request message to the server.

[0079] If there is an internet connection, the message is sent immediately to the server.

[0080] Step 7:

[0081] The device checks for internet connectivity.

[0082] If the connection is lost, it will automatically switch to ad-hoc communication mode.

[0083] Step 8:

[0084] In the ad-hoc communication mode, a terminal communicates directly with other terminals in the vicinity.

[0085] Broadcast a rescue request message to other devices and share information.

[0086] Step 9:

[0087] Edge AI runs within the device and analyzes the user's situation.

[0088] If first aid is required, detailed instructions are provided via voice.

[0089] Step 10:

[0090] Edge AI initiates psychological support dialogue.

[0091] Providing users with messages such as "Take a deep breath and stay calm."

[0092] Step 11:

[0093] The device periodically collects the user's location and status information.

[0094] Data is collected based on GPS data and user input.

[0095] Step 12:

[0096] The terminal sends the collected information to the server and stores it in a database.

[0097] The information is continuously updated and used in rescue operations.

[0098] Step 13:

[0099] The user records a final message on the device.

[0100] The recorded message is saved on the device.

[0101] Step 14:

[0102] Once your internet connection is restored, your device will automatically send a recorded message to your designated contacts.

[0103] Your message will be sent to your family and friends.

[0104] Example 1

[0105] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0106] Providing prompt and effective rescue and assistance is crucial when large-scale natural disasters occur, especially earthquakes with a seismic intensity of 7 or higher. However, conventional methods often result in the destruction of communication infrastructure, making it difficult to quickly share information and request rescue. Furthermore, there are limitations on the means by which victims can receive instructions on how to provide appropriate first aid and psychological support. This makes it difficult for conventional systems to provide complete support.

[0107] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0108] In this invention, the server includes means for monitoring seismic intensity information and detecting earthquakes of seismic intensity 7 or higher, means for sending alerts to user terminals based on the seismic intensity information, and means for analyzing meteorological data and identifying users in the affected area. This makes it possible to quickly send alerts to users in the affected area and begin providing support when an earthquake of seismic intensity 7 or higher occurs.

[0109] The user's device also includes means for automatically activating upon receiving the alert, means for detecting the user's voice and generating and transmitting a rescue request message, means for switching to ad-hoc communication mode if the Internet connection is lost, means for analyzing the user's situation based on edge artificial intelligence and generating first aid instructions and psychological support, and means for detecting the voice of the user when the user makes a rescue request and generating the necessary message. This enables victims to quickly request rescue even if infrastructure is destroyed, and to receive appropriate first aid instructions and psychological support.

[0110] "Seismic intensity information" is data that indicates the strength of an earthquake and the degree of shaking.

[0111] "Means for detecting earthquakes of seismic intensity 7 or higher" refers to a device or program that has the function of monitoring earthquake information and detecting the occurrence of an earthquake of seismic intensity 7 or higher.

[0112] "Means for sending an alert to a user's terminal" refers to a device or program that has the function of sending a warning message to a communication device owned by a user based on seismic intensity information.

[0113] The "means for automatically starting up the user's terminal" refers to a device or program that has the function of automatically starting up the user's terminal upon receiving an alert.

[0114] "Means for detecting the user's voice and generating and transmitting a rescue request message" refers to a device or program that has the function of detecting the voice uttered by the user and creating and transmitting a message requesting rescue based on that voice.

[0115] "Means for switching to ad hoc communication mode when Internet connection is lost" refers to a device or program that has the function of automatically changing to a mode for direct communication between terminals when Internet connection is unavailable.

[0116] "Edge AI" is an artificial intelligence technology installed in a terminal, and is a system that has the ability to perform data analysis and decision-making in real time.

[0117] "Means for analyzing the user's situation and providing first aid instructions and psychological support" refers to a device or program that has the function of analyzing the situation of a disaster victim and providing necessary first aid instructions and psychological support via voice or other means.

[0118] "Means for analyzing meteorological data and identifying users in the affected area" refers to a device or program that has the function of analyzing earthquake and other meteorological information and identifying users in the affected area.

[0119] "Means for detecting voices requesting rescue and generating the necessary messages when the voice is uttered" refers to a device or program that has the function of detecting the voices when a user utters a voice requesting rescue and automatically generating a corresponding message.

[0120] This invention is a system for providing prompt and effective rescue and support in the event of a large-scale disaster, such as an earthquake with a seismic intensity of 7 or higher. This system is composed of a server, terminals, and users, each of which plays a specific role.

[0121] Server Features

[0122] The server is responsible for constantly monitoring earthquake information and analyzing meteorological data. The server obtains data in real time from earthquake early warning services such as the Japan Meteorological Agency API, and sends an alert to user devices in the relevant area when an earthquake of magnitude 7 or higher occurs. This alert automatically activates the device and triggers the rescue request process. The server also stores and continuously updates user location and status information in a database. Specific server product names that can be used are Amazon Web Services (AWS (registered trademark)) and Google (registered trademark) Cloud Platform (GCP).

[0123] Device Features

[0124] When the device receives an alert from the server, it automatically launches the app and enters voice detection mode. The device detects the user's voice and generates and sends a rescue request message. This message includes the user's current location and a brief description of the situation. If the internet connection is lost, the device automatically switches to ad-hoc communication mode and communicates directly with other devices in the vicinity. In addition, the edge artificial intelligence (Edge AI) installed in the device provides first aid instructions and psychological support to victims. For example, if the user is injured, it will issue voice instructions such as, "If there is bleeding, apply pressure with a clean cloth." If the user is feeling anxious or scared, it will provide psychological support such as, "Please stay calm. Take a deep breath and relax."

[0125] User Roles

[0126] When an earthquake occurs, users follow the instructions provided by the device to ensure their own safety. If necessary, they can request rescue by voice. When the device detects the user's voice, it automatically generates a rescue request message and sends it to the server. Users can also record a final message on the device, which will be automatically sent to the designated contacts once the internet connection is restored.

[0127] Specific use cases

[0128] Use case 1: Immediately after an earthquake occurs

[0129] The server detects earthquakes of magnitude 7 or higher and sends alerts to users' devices in the affected area.

[0130] The device will automatically launch the app and enter voice detection mode.

[0131] When a user says "Help!", the device detects the voice and generates and sends a rescue request message.

[0132] Use case 2: When the Internet is down

[0133] The device checks for an Internet connection and, if it determines that it is unavailable, automatically switches to ad-hoc communication mode.

[0134] The device communicates directly with the devices of other victims in the vicinity, sharing rescue request information and location information.

[0135] Use case 3: When first aid is required

[0136] If the user is injured, the device will use edge AI to give voice instructions such as, "If there is bleeding, apply pressure with a clean cloth."

[0137] Edge AI also provides psychological support, such as "Stay calm. Take a deep breath and relax."

[0138] Prompt Sentence Examples

[0139] By using a generative AI model in this system, it is possible to generate detailed rescue requests and first aid instructions. An example of a prompt sentence to be input to the generative AI model is as follows:

[0140] "There have been many injuries from the earthquake. What should we do if someone is bleeding?"

[0141] "If survivors are feeling anxious, what words of psychological support can you offer?"

[0142] Using this prompt, the AI ​​generates specific instructions and support words that it provides to the user via the device.

[0143] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0144] Step 1:

[0145] The server constantly monitors earthquake information and analyzes meteorological data. Specifically, it obtains data in real time from earthquake early warning services such as the Japan Meteorological Agency API. This data includes information such as seismic intensity, location, and time of occurrence. The server analyzes this data, and if an earthquake with a seismic intensity of 7 or higher occurs, it identifies users in the affected area. Based on the analysis results, it generates a list of users in the affected area.

[0146] Input: Real-time earthquake data obtained from the Japan Meteorological Agency API

[0147] Output: List of users in the affected area

[0148] Step 2:

[0149] The server sends an alert to the user's device based on the seismic intensity information. Specifically, it references a list of users in the affected area and generates an alert message based on this. The message contains earthquake information as well as a message saying, "Please launch the app immediately and follow the instructions." This alert message is sent to each user's device as a push notification.

[0150] Input: User list in affected area, seismic intensity information

[0151] Output: Alert message to the user's terminal

[0152] Step 3:

[0153] When the device receives an alert, it automatically launches the app and enters voice detection mode. Specifically, the application on the device waits in the background and is launched when the alert notification is triggered. When the app launches, the microphone is enabled and it is ready to detect the user's voice.

[0154] Input: Alert message sent from the server

[0155] Output: Device that has entered voice detection state

[0156] Step 4:

[0157] When a user utters a voice requesting rescue, such as "Help!", the device detects this. Specifically, it uses a voice recognition algorithm to recognize specific keywords (e.g., "Help!"). It then analyzes this voice data and generates a rescue request message. The message includes the user's current location (GPS information) and a brief description of the situation (voice-to-text).

[0158] Input: User's voice requesting help

[0159] Output: Help request message

[0160] Step 5:

[0161] The device generates a rescue request message and sends it to the server. If an Internet connection is available, the message is sent immediately to the server. If the Internet connection is lost, the device switches to ad-hoc communication mode and relays the message through other devices.

[0162] Input: Message requesting help

[0163] Output: A help-call message sent to the server or a message relayed via ad-hoc communication.

[0164] Step 6:

[0165] The server analyzes the received rescue request message and forwards the information to the appropriate rescue team. Specifically, it analyzes the content of the rescue request and forwards the message to the appropriate rescue team based on the urgency and required resources. It also generates optimal route information based on the location information and provides it to the rescue team.

[0166] Input: A message requesting help sent by the user

[0167] Output: Message forwarding to rescue team and optimal route information

[0168] Step 7:

[0169] The edge AI inside the device analyzes the user's situation in real time and provides first aid instructions and psychological support. Specifically, it analyzes the user's voice and behavior, and if there is an injury, for example, it will give instructions such as "If there is bleeding, apply pressure with a clean cloth." It also provides psychological support such as advice such as "Remain calm. Take a deep breath and relax."

[0170] Input: User voice and behavioral data

[0171] Output: Audio messages with first aid instructions and psychological support

[0172] (Application example 1)

[0173] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0174] In the event of a large-scale disaster, it is extremely important to carry out rapid and effective rescue operations for victims. However, conventional rescue operations rely on human resources and communication infrastructure, and have faced problems such as internet connection outages and difficulty sharing information in widespread disaster areas. Rescue operations tend to be delayed, especially in disaster-stricken areas outside of urban areas, making ensuring the safety of victims and providing early rescue a challenge. Furthermore, there is a need to simultaneously provide psychological support and first aid instructions to victims.

[0175] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0176] In this invention, the server includes: means for monitoring seismic intensity information and detecting earthquakes of seismic intensity 7 or higher; means for sending an alert to a user's device based on the seismic intensity information; means for automatically activating the user's device upon receiving the alert; means for detecting the user's voice and generating and transmitting a rescue request message; means for switching to ad hoc communication mode if the Internet connection is lost; means for providing first aid instructions and psychological support using edge AI; means for transmitting the user's location information and status information to the server and storing them in a database; means for automatically activating a terminal in the autonomous vehicle or an in-vehicle terminal; means for the terminal in the autonomous vehicle to detect the user's voice and generate and transmit a rescue request message; and means for the terminal in the autonomous vehicle to directly communicate with other autonomous vehicles and terminals using ad hoc communication to share information. This allows victims to quickly request rescue and enables efficient rescue operations using autonomous vehicles. Furthermore, edge AI can be used to provide first aid instructions and psychological support, ensuring the safety and security of victims.

[0177] "Seismic intensity information" is information that indicates the strength of shaking when an earthquake occurs.

[0178] "User devices" refers to smartphones or dedicated devices owned by disaster victims.

[0179] An "alert" is a message that notifies the user when a particular condition occurs.

[0180] "Voice detection" is a technology that detects the voices emitted by the user and recognizes them as specific commands or messages.

[0181] A "rescue request message" is a message that includes information to request rescue of a disaster victim.

[0182] "Ad hoc communication" is a method of communicating directly with other devices when there is no Internet connection.

[0183] "Edge AI" is an artificial intelligence technology that runs on a device and processes data and makes decisions in real time.

[0184] "First Aid Instructions" provide specific instructions for initial response to injuries and emergency situations.

[0185] "Psychological support" refers to providing advice and encouragement to reduce the mental anxiety and stress of disaster victims.

[0186] "Location information" is data that indicates a user's current physical location.

[0187] "Status information" is data about the user's health status and surrounding environment.

[0188] A "server" is a networked computer system that collects and processes data and provides information to other devices.

[0189] A "database" is an electronic storage device for organizing and storing information.

[0190] "Terminal in an autonomous vehicle" refers to a computing device installed in an autonomous vehicle.

[0191] "In-vehicle dedicated terminal" refers to a terminal used exclusively within an autonomous vehicle.

[0192] "Direct communication" is communication between devices without going through the Internet.

[0193] This invention is a system for carrying out rapid and effective rescue operations in the event of a large-scale disaster. The system detects the voices of victims and generates rescue request messages, enables information sharing using ad-hoc communication even when the internet connection is lost, and uses edge AI to provide first aid instructions and psychological support.

[0194] System Overview

[0195] The server monitors seismic intensity information and detects earthquakes of magnitude 7 or higher. When an earthquake occurs, an alert is sent to the user's device in the affected area, causing the device to automatically wake up. When the user's device receives the alert, it enters voice detection mode, detects the user's voice, and generates and sends a rescue request message. If the internet connection is lost, it switches to ad-hoc communication mode, communicating directly with other nearby devices to share information.

[0196] Device operation

[0197] The device automatically wakes up when it receives an alert from the server, detects the user's voice, and generates a rescue request message. The rescue request message includes the user's current location and a brief description of the situation, and is immediately sent to the server if there is an internet connection. In ad-hoc communication mode, information is shared by directly communicating with other nearby devices. In addition, the edge AI provides first aid instructions and psychological support within the device.

[0198] Use of autonomous vehicles

[0199] The system also incorporates a dedicated terminal installed in the autonomous vehicle that automatically activates when an earthquake occurs and detects the user's voice. The dedicated terminal generates a rescue request message and communicates ad hoc with other autonomous vehicles and terminals in the vicinity to share information.

[0200] Hardware and Software Details

[0201] Hardware: Dedicated terminals installed in autonomous vehicles, smartphones carried by users, microphones, and GPS sensors

[0202] Software: VoiceRecognition, GPS module (location information acquisition), EdgeAI (real-time data processing and instructions), AdHocCommunication

[0203] Program processing explanation

[0204] The server constantly monitors seismic intensity information, and if an earthquake of magnitude 7 or higher occurs, it sends an alert to devices in the relevant area. The device then wakes up and switches to voice detection mode to detect the user's voice. After detecting voice, it generates a rescue request message including GPS information and sends it to the server if there is an internet connection, or to another device via ad-hoc communication if there is not. The edge AI runs on the device and provides first aid instructions and psychological support.

[0205] Specific examples

[0206] Example 1: Immediately after an earthquake occurs

[0207] The server detects earthquakes with a magnitude of 7 or higher and sends alerts to users' devices in the affected area.

[0208] The device will automatically launch the app and enter voice detection mode.

[0209] When a user says "Help!", the device detects the voice and generates and sends a rescue request message.

[0210] Example prompt sentence:

[0211] The system automatically activates when an earthquake of magnitude 7 or higher occurs, detects the voice "Help!", and generates and sends a rescue request message. Furthermore, if the internet connection is cut off, the system shares information with other autonomous vehicles using ad-hoc communication.

[0212] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0213] Step 1:

[0214] The server constantly monitors seismic intensity information. As input, it receives real-time seismic intensity data from the Japan Meteorological Agency and seismometers, analyzes that data, and determines whether an earthquake of seismic intensity 7 or higher has occurred. As output, if an earthquake of seismic intensity 7 or higher is detected, it generates a signal to send an alert to user devices in the relevant area.

[0215] Step 2:

[0216] The user's device receives the alert from the server. As an input, it receives the alert signal sent from the server and uses it as a trigger to automatically wake up the device. As an output, it sends information to the device system to switch to voice-sensing mode.

[0217] Step 3:

[0218] The device then enters voice detection mode. The device's built-in microphone collects voice data from the user's surroundings as input, which is then fed into the VoiceRecognition software. The collected voice data is then analyzed and recognized as a rescue request, such as "Help!". The output is the voice recognition results provided to the device's first aid instruction module.

[0219] Step 4:

[0220] The user says "Help!". As input, the user's voice is collected by the device's microphone and analyzed by VoiceRecognition software. As output, if "Help!" is recognized, a signal is generated to generate a rescue request message.

[0221] Step 5:

[0222] The device generates a rescue request message. As input, it combines the voice recognition results with the current location data obtained from the GPS module. As data processing, it integrates the user's location information and the rescue request voice information to generate a detailed rescue request message. As output, it prepares the message to be sent to the server.

[0223] Step 6:

[0224] Checking Internet connection. As input, the terminal checks whether it has an Internet connection. As data processing, it performs an Internet connection test and obtains the results. As output, if there is an Internet connection, it generates a signal to send a rescue request message to the server, and if there is not, it generates a signal to switch to ad-hoc communication mode.

[0225] Step 7:

[0226] Switching to ad-hoc communication mode. As input, it receives information that there is no Internet connection. As data processing, it changes the communication mode of the terminal to ad-hoc mode. As output, it starts direct communication with another terminal and sends a rescue request message.

[0227] Step 8:

[0228] Provision of first aid instructions and psychological support using edge AI. Voice recognition results and user condition information are acquired as input. The edge AI processes the data to generate first aid instructions and psychological support messages appropriate to the situation. The output is to provide instructions and support to the user via voice or text.

[0229] Specific examples

[0230] Example prompt sentence:

[0231] The system automatically activates when an earthquake of magnitude 7 or higher occurs, detects the voice "Help!", and generates and sends a rescue request message. Furthermore, if the internet connection is cut off, the system shares information with other autonomous vehicles using ad-hoc communication.

[0232] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0233] System Overview

[0234] This invention is a system for providing prompt and effective rescue and support in the event of a large-scale disaster such as an earthquake. This system automatically activates when an earthquake of magnitude 7 or higher occurs, detects the voices of victims, and issues a rescue request. Even if the internet connection is cut off, the system can share information using ad-hoc communication, and utilizes edge AI and an emotion engine to provide first aid instructions and psychological support.

[0235] Server Features

[0236] The server monitors earthquake information and sends an alert to user devices in the relevant area when an earthquake of magnitude 7 or higher occurs. The alert automatically activates the user device and triggers the rescue request process. The server stores the user's location, status, and emotional data in a database to help optimize rescue operations.

[0237] Device Features

[0238] When the device receives an alert, it automatically launches the app and enters voice detection mode. When the user utters a voice command such as "help me," the device detects it and generates and sends a rescue request message. If the internet connection is lost, the device switches to ad-hoc communication mode and communicates directly with other nearby devices to share information.

[0239] The Edge AI runs within the device and provides first aid instructions and psychological support to disaster victims. The emotion engine recognizes emotions from the user's voice and facial expressions and adjusts the support provided based on those emotions. For example, if the user expresses strong anxiety or fear, the Edge AI will provide a customized message such as, "Don't worry. Help is on its way."

[0240] User Roles

[0241] When an earthquake occurs, users follow the instructions provided by the device to ensure their own safety and make a voice call to request rescue. When the device detects voice, it automatically generates a rescue request message and sends it to the server. Users can also record a final message on the device, which will be automatically sent to designated contacts when the internet connection is restored.

[0242] Specific use cases

[0243] Use case 1: Immediately after an earthquake occurs

[0244] The server detects earthquakes with a magnitude of 7 or higher and sends alerts to users' devices in the affected area.

[0245] The device will automatically launch the app and enter voice detection mode.

[0246] When a user says "Help!", the device detects the voice and generates and sends a rescue request message.

[0247] Use case 2: When the Internet is down

[0248] The device checks for an Internet connection and, if it determines that it is unavailable, automatically switches to ad-hoc communication mode.

[0249] The device communicates directly with the devices of other victims in the vicinity, sharing rescue request information and location information.

[0250] Use Case 3: Supported by Emotion Engine

[0251] If the user is injured and feels strong anxiety, the device will use edge AI and an emotion engine to give instructions such as "If there is bleeding, apply pressure with a clean cloth" and also provide psychological support such as "Don't worry, help will arrive shortly."

[0252] The emotion engine recognizes emotional data from the user's voice and facial expressions, sends it to the server, and stores it in a database.

[0253] What the program does

[0254] The server constantly monitors earthquake information, and when it detects an earthquake with a seismic intensity of 7 or higher, it sends an alert to the user's device in the relevant area. When the device receives the alert, it automatically launches the app and enters voice detection mode. When the user issues a voice request for rescue, the device analyzes the voice and generates a rescue request message.

[0255] If the internet connection is lost, the device will switch to ad-hoc communication mode and directly communicate with other devices to share information. Edge AI will provide first aid instructions and psychological support, and the emotion engine will analyze the user's emotions and provide appropriate support.

[0256] For example, if a user expresses strong anxiety or fear, the edge AI will generate a message saying, "Don't worry, help is on its way," and the emotion engine will report the situation to the server. This information will be stored in a database and used to optimize rescue operations.

[0257] The processing flow will be explained below.

[0258] Step 1:

[0259] The server monitors earthquake information.

[0260] Data is regularly obtained from the Japan Meteorological Agency and earthquake early warning services and the seismic intensity is analyzed.

[0261] Step 2:

[0262] If an earthquake with a magnitude of 7 or higher is detected, the server will send an alert to user devices in the relevant area.

[0263] The alert will include information on the earthquake's epicenter and magnitude.

[0264] Step 3:

[0265] The device receives an alert from the server and automatically launches the app.

[0266] When the app is launched, it switches to voice detection mode.

[0267] Step 4:

[0268] The user makes a sound requesting rescue, such as "help me."

[0269] The sound is detected by the device's microphone.

[0270] Step 5:

[0271] The device analyzes the voice and generates a message requesting help.

[0272] The rescue request message will include the user's location and a brief description of the situation.

[0273] Step 6:

[0274] The terminal sends a rescue request message to the server.

[0275] If there is an internet connection, the message is sent immediately to the server.

[0276] Step 7:

[0277] The device checks for internet connectivity.

[0278] If the connection is lost, it will automatically switch to ad-hoc communication mode.

[0279] Step 8:

[0280] In the ad-hoc communication mode, a terminal communicates directly with other terminals in the vicinity.

[0281] Broadcast a rescue request message to other devices and share information.

[0282] Step 9:

[0283] Edge AI runs within the device and analyzes the user's situation.

[0284] If first aid is required, detailed instructions are provided via voice.

[0285] Step 10:

[0286] The device's emotion engine analyzes the user's voice and facial expressions to recognize their emotions.

[0287] The recognized emotions are structured as data.

[0288] Step 11:

[0289] Edge AI tailors first aid instructions and psychological support based on emotions recognized by the emotion engine.

[0290] For example, if a user is feeling very anxious, it will generate a message such as "Don't worry, help is on its way."

[0291] Step 12:

[0292] The device transmits the user's emotional information recognized by the emotion engine to the server.

[0293] Emotional information is stored in a database and used to optimize rescue operations.

[0294] Step 13:

[0295] The device periodically collects the user's location and status information.

[0296] Data is collected based on GPS data and user input.

[0297] Step 14:

[0298] The terminal sends the collected information to the server and stores it in a database.

[0299] The information is continuously updated and used in rescue operations.

[0300] Step 15:

[0301] The user records a final message on the device.

[0302] The recorded message is saved on the device.

[0303] Step 16:

[0304] Once your internet connection is restored, your device will automatically send a recorded message to your designated contacts.

[0305] Your message will be sent to your family and friends.

[0306] Example 2

[0307] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0308] In the event of a large-scale disaster such as an earthquake, victims lack the means to quickly and effectively request rescue. There is also a need for a method to efficiently share rescue information even when internet connection is cut off. Furthermore, a support system is needed to reduce the psychological burden on victims and provide appropriate first aid.

[0309] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for monitoring seismic intensity information and detecting earthquakes of seismic intensity 7 or higher, means for sending an alert to a user's terminal based on the seismic intensity information, means for automatically activating the user's terminal that receives the alert, means for detecting the user's voice and generating and sending a rescue request message, means for switching to ad hoc communication mode when the Internet connection is lost, means for providing first aid instructions and psychological support using edge AI, means for sending the user's location information and status information to the server and storing them in a database, and means for analyzing the user's emotions and sending the analysis results to the server. This enables victims to quickly and reliably request rescue, enables information sharing even in situations where the Internet connection is lost, and further reduces the psychological burden on victims and enables appropriate first aid to be provided.

[0310] "Seismic intensity information" is data that indicates the scale of an earthquake and the strength of the vibrations, and includes information such as the location and time of the earthquake and the depth of the epicenter.

[0311] "User device" refers to a mobile information terminal such as a smartphone or tablet, and is the device to which this system sends alerts.

[0312] An "alert" is a warning message that the system sends to the user's device after detecting an earthquake, and serves as a trigger to start the rescue request process.

[0313] "Voice detection" is the process of detecting the user's voice using the device's microphone and analyzing the voice data.

[0314] A "rescue request message" is a message generated based on the user's voice requesting rescue, and is sent to the server along with the user's location information.

[0315] "Ad-hoc communication mode" is a mode in which terminals connect directly to each other and communicate data when an Internet connection is not available.

[0316] "Edge AI" is artificial intelligence that runs within a device and has the ability to provide first aid instructions and psychological support in real time.

[0317] "First aid instructions" are instructions provided by edge AI on the medical procedures that victims should take immediately.

[0318] "Psychological support" refers to messages and actions that edge AI uses to stabilize the mental state of disaster victims and provide them with a sense of security.

[0319] "Location information" is data that includes the user's current latitude and longitude, and is important during rescue operations.

[0320] "Status information" is data related to the user's health condition and the extent of the damage.

[0321] The "database" is a system installed on a server for storing collected earthquake information, user location information, status information, and emotional data.

[0322] "Emotion analysis" is the process of inferring a user's emotions from their voice and facial expressions, and is the basis for providing appropriate assistance.

[0323] "Emotion data" is data that indicates the user's emotional state, and is information that is automatically sent to the server.

[0324] MODE FOR CARRYING OUT THE INVENTION

[0325] This invention is a system for providing prompt and effective rescue and support in the event of a large-scale disaster such as an earthquake. This system automatically activates when an earthquake of magnitude 7 or higher occurs, and can detect the voices of victims and issue rescue requests. Even if the internet connection is cut off, the system can share information using ad-hoc communication and provide first aid instructions and psychological support using edge AI and an emotion engine.

[0326] Hardware and Software Configuration

[0327] server

[0328] The server has the function of constantly monitoring earthquake information. To obtain real-time earthquake data, the server obtains earthquake occurrence information using APIs from the Japan Meteorological Agency and other earthquake information providers. The server is implemented using programming languages ​​such as Python and JavaScript (registered trademark), and uses databases such as PostgreSQL and MySQL (registered trademark).

[0329] Terminal

[0330] The device is a user's smartphone or tablet. The device receives an alert from the server and automatically launches the app to enter voice sensing mode. The device runs an application compatible with the Android (registered trademark) or iOS operating system.

[0331] Edge AI

[0332] The AI ​​model runs on the edge device and uses lightweight AI frameworks such as TENSORFLOW® Lite and PyTorch Mobile to analyze the user's voice and facial expressions, and provides first aid instructions and psychological support in real time.

[0333] System Operation

[0334] server

[0335] The server analyzes the earthquake information it receives and sends an alert to user devices in the relevant area if an earthquake with a seismic intensity of 7 or higher occurs. When a user device receives the alert, it automatically launches the app and enters voice detection mode. At this time, the server stores the user's location and status information in a database to help optimize rescue operations.

[0336] Terminal

[0337] When the device receives an alert, it automatically launches the app and enters voice detection mode. When the user utters a word such as "help me," the device's microphone picks up the sound and the voice recognition engine analyzes it. A rescue request message is generated from the analyzed voice and sent to the server.

[0338] If an internet connection is unavailable, the device automatically switches to ad-hoc communication mode, allowing it to communicate directly with other devices in the vicinity and share information such as rescue requests and location information.

[0339] Edge AI and Emotion Engine

[0340] The edge AI provides first aid instructions to the user. For example, it provides voice instructions such as, "If there is bleeding, apply pressure with a clean cloth." The emotion engine also analyzes the user's emotions from their voice and facial expressions, and provides appropriate psychological support based on the analysis results. For example, if the user shows strong anxiety or fear, it can provide a message such as, "Don't worry. Help will arrive shortly."

[0341] Specific examples of operation

[0342] For example, if a user is trapped in a building that has collapsed due to an earthquake and yells "help," the system operates as follows:

[0343] 1. The server detects earthquakes with a magnitude of 7 or higher and sends an alert to user devices in the relevant area.

[0344] 2. The device will receive the alert and automatically launch the app to enter voice detection mode.

[0345] 3. When the user shouts "help," the device's microphone detects the voice and the voice recognition engine analyzes it.

[0346] 4. The device generates a rescue request message and sends it to the server.

[0347] 5. If the Internet connection is lost, the device switches to ad-hoc communication mode and shares information with other devices in the vicinity.

[0348] 6. The edge AI gives instructions such as "If there is bleeding, apply pressure with a clean cloth," and the emotion engine provides psychological support such as "Don't worry, help will arrive soon."

[0349] Prompt Sentence Examples

[0350] "An earthquake of magnitude 7 or higher has occurred. Please explain how the rescue system will respond to this situation from the perspectives of the server, the device, and the user."

[0351] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0352] The flow of this system's program processing

[0353] Step 1:

[0354] The server monitors earthquake information

[0355] The server calls the APIs of the Japan Meteorological Agency and other earthquake information providers to obtain real-time earthquake data. The input includes earthquake information obtained from external APIs, including the location, time, depth of the epicenter, and seismic intensity of the earthquake. The server analyzes this data and detects earthquakes with a seismic intensity of 7 or higher. The output is a flag when an earthquake with a seismic intensity of 7 or higher is detected.

[0356] Step 2:

[0357] The server sends an alert to the user's device in the relevant area.

[0358] The server analyzes the earthquake information and sends an alert message to the devices of users living in areas judged to have a seismic intensity of 7 or higher. The input includes the earthquake information obtained in step 1 and the user's location information in the database. Based on this, the server identifies users in the relevant area and sends an alert to their device. The output is the alert message sent to the user's device.

[0359] Step 3:

[0360] The device receives the alert and automatically launches the app

[0361] The device has a listener for receiving alerts sent from the server. The input is the alert message sent from the server. When the listener receives an alert, the device automatically launches the app. It also enters voice-sensing mode, so the device's microphone is always on. The output is the launch of the app and the transition to voice-sensing mode.

[0362] Step 4:

[0363] The device detects the user's voice and generates a message requesting help.

[0364] When a user utters a voice request for help, such as "help me," the device's microphone picks up the voice. The input is the user's voice. The speech recognition engine analyzes the voice, and a rescue request message is generated as a result of the analysis. The output is the generated rescue request message.

[0365] Step 5:

[0366] The device sends a rescue request message to the server.

[0367] The generated distress request message is sent to the server using the device's normal internet connection. The input is the distress request message generated in step 4. The server receives it and stores it in a database. The output is the distress request message sent to the server.

[0368] Step 6:

[0369] If the Internet connection is lost, the device switches to ad-hoc communication mode.

[0370] The device periodically checks for an Internet connection and automatically switches to ad-hoc communication mode if one is unavailable. The input is the result of checking the Internet connection status. Once in ad-hoc communication mode, it communicates directly with other devices in the vicinity and shares rescue request information and location information. The output is the information shared through communication with other devices.

[0371] Step 7:

[0372] Edge AI provides first aid instructions and psychological support to disaster victims

[0373] Edge AI uses data such as the user's voice and facial expressions to provide appropriate first aid instructions and psychological support. The input is the user's voice, facial expression information, location information, and emotional data. Edge AI analyzes this and provides voice instructions such as "If there is bleeding, apply pressure with a clean cloth" and psychological support such as "Don't worry, help will arrive shortly." The output is the provided first aid instructions and psychological support message.

[0374] Step 8:

[0375] The device uses an emotion engine to analyze the user's emotions.

[0376] The device's emotion engine analyzes emotions from the user's voice and facial expressions. The input is the user's voice and facial expression data. The emotion engine identifies the user's emotion based on this data and generates an analysis result. The output is the analyzed emotion data.

[0377] Step 9:

[0378] The emotion engine sends the analysis results to the server and stores them in a database.

[0379] The analysis results generated by the emotion engine are sent to the server via the Internet. The input is the emotion data generated in step 8. The server receives this and stores it in a database. The output is the emotion data stored in the database.

[0380] (Application example 2)

[0381] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0382] Conventional disaster rescue systems have had problems such as ineffective functioning when the internet connection is cut off or when psychological support for people at the disaster site is lacking. Furthermore, in-store emergency response measures lack the means for a central control unit to quickly and simultaneously transmit information to multiple devices. A system that can solve these problems, provide rescue and support quickly and effectively in the event of a disaster, and ensure the safety of customers and staff in the store, is needed.

[0383] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for monitoring seismic intensity information and detecting earthquakes of seismic intensity 7 or higher, means for sending an alert to a user's device based on the seismic intensity information, means for automatically activating the user's device that receives the alert, means for detecting the user's voice and generating and sending a rescue request message, means for switching to ad hoc communication mode when the Internet connection is lost, means for providing first aid instructions and psychological support using edge AI, means for sending user location information and status information to a central device and storing it in an information storage device, means for providing voice instructions to customers and staff in the store, and means for simultaneously sending alerts to multiple devices in the store based on the seismic intensity information. This makes it possible to share information using ad hoc communication even in situations where the Internet is lost, and to provide quick and appropriate first aid and psychological support to customers and staff in the store.

[0384] "Seismic intensity information" is data indicating the seismic intensity of an earthquake, and is usually provided by seismological observation organizations.

[0385] "Users" refer to customers and staff in physical stores.

[0386] "Terminal" refers to a mobile device such as a smartphone or smart glasses.

[0387] "Internet connection" refers to the communication means by which a terminal accesses the Internet.

[0388] An "alert" is a warning message sent to a device when an earthquake with a seismic intensity of 7 or higher occurs.

[0389] "Ad hoc communication" is a method in which terminals communicate directly with each other when the Internet connection is cut off.

[0390] "Edge AI" refers to artificial intelligence technology that runs on the device itself, allowing for real-time processing without relying on cloud servers.

[0391] "First Aid Instructions" provide appropriate medical guidelines for injured users.

[0392] "Psychological support" refers to encouraging and stabilizing messages provided to help clients achieve their emotional well-being.

[0393] A "central device" is a device that oversees and manages data within a physical store.

[0394] The "information storage device" is a database for storing data collected by the central device.

[0395] "Voice instructions" are instructions given to users and staff via voice.

[0396] A "rescue request message" is a message generated by detecting a voice calling for help made by a user in the event of a disaster.

[0397] "Multiple terminals in store" refers to multiple mobile devices located in a physical store.

[0398] System Overview

[0399] This invention is a system for providing prompt and effective rescue and support in the event of a large-scale disaster, such as an earthquake with a seismic intensity of 7 or higher, occurring within a brick-and-mortar store. This system primarily includes a server, a device equipped with edge AI, and users (customers and staff in the store).

[0400] Server Features

[0401] The server constantly monitors earthquake intensity information and, if it detects an earthquake of magnitude 7 or higher, sends an alert to the devices of users in the relevant area. At this time, the server obtains the intensity information from the meteorological agency's API using AWS IoT. The server also supports an ad-hoc communication mode that can be used even if the internet connection is cut off.

[0402] Device Features

[0403] When a device (such as a smartphone or smart glasses) receives an alert from the server, it automatically wakes up and enters voice-sensing mode. The device is equipped with a TensorFlow-based voice recognition model, which detects when a user speaks a rescue request such as "help me" and generates a rescue request message. The generated message is immediately sent to a central device or an external rescue agency.

[0404] If the device loses internet connection, it switches to ad-hoc communication mode and communicates directly with other devices in the store to share information. This communication uses Bluetooth to transmit data between devices.

[0405] Edge AI runs within the device and provides real-time first aid instructions to disaster victims. Specifically, it presents first aid methods for injuries and gives voice instructions on the necessary actions. Furthermore, the emotion engine recognizes emotions from the user's voice and facial expressions and provides psychological support based on those emotions. For example, if a user is feeling extremely anxious, it will provide a message such as, "Don't worry. Help will arrive shortly."

[0406] User Roles

[0407] When an earthquake occurs, users can follow the instructions provided by the device to ensure their own safety and make a voice call for help. Users can also record a final message on the device, which will be automatically sent to designated contacts when internet connection is restored.

[0408] Specific examples

[0409] Example 1: Immediately after an earthquake occurs

[0410] The server detects earthquakes of magnitude 7 or higher and sends an alert to all terminals in the store.

[0411] The device will automatically start up and enter voice-activated mode.

[0412] When a user shouts "Help!", the device detects the voice, generates a rescue request message, and sends it to the central device.

[0413] Example 2: Internet outage

[0414] The device checks for an Internet connection and, if it determines that it is unavailable, automatically switches to ad-hoc communication mode.

[0415] The device communicates directly with other nearby victims' devices, sharing rescue request information and location information.

[0416] Example 3: Supported by an Emotion Engine

[0417] If a user is injured and at the same time feels strong anxiety, the device's edge AI and emotion engine will instruct them to "apply pressure with a clean cloth if there is bleeding," and also provide psychological support by saying, "Don't worry, help will arrive shortly."

[0418] Example prompts to input to the generative AI model

[0419] "An earthquake of magnitude 7 or higher has occurred. Please move to a safe place. If you need first aid, please follow the steps below. If you are injured, apply pressure with a clean cloth to stop bleeding. We will also provide psychological support. Please rest assured, help will be on its way soon."

[0420] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0421] Step 1:

[0422] The server monitors seismic intensity information. Specifically, it uses AWS IoT Core to obtain real-time earthquake information from the meteorological agency's API. It receives seismic intensity data from the API as input and generates an alert signal as output when an earthquake with a seismic intensity of 7 or higher occurs.

[0423] Step 2:

[0424] When the server detects an earthquake with a seismic intensity of 7 or higher, it sends an alert to multiple users' devices. It receives the alert signal generated in step 1 as input and sends an alert message to all devices using the MQTT protocol as output.

[0425] Step 3:

[0426] When the device receives an alert from the server, it automatically wakes up and enters voice-sensitive mode, taking the alert message from the server as input and arming the voice recognition system as output.

[0427] Step 4:

[0428] The user issues a voice request such as "help me." The device detects the voice and analyzes the voice data using a TensorFlow-based speech recognition model. It receives the voice data as input and generates a text message as output.

[0429] Step 5:

[0430] The terminal transmits the generated rescue request message to a central device or an external rescue organization. The terminal receives a rescue request message in text format as input and transmits the message to a specified rescue organization as output.

[0431] Step 6:

[0432] If the Internet connection is lost, the device automatically switches to ad-hoc communication mode, checking for Internet connectivity as input and using Bluetooth as output to start communicating directly with other devices.

[0433] Step 7:

[0434] Edge AI operates within the device and provides first aid instructions to the user, receiving information about the user's condition as input and providing specific first aid instructions via voice as output.

[0435] Step 8:

[0436] The emotion engine recognizes emotions from the user's voice and facial expressions and provides psychological support. It receives voice and facial expression data as input, and generates a message according to the user's emotional state as output, which is delivered in voice.

[0437] Step 9:

[0438] The terminal sends rescue request messages and user location and status information to the central unit, which stores them in the information storage device. The terminal receives location and status information as input, sends the data to the central unit as output, and stores it in the information storage device.

[0439] Step 10:

[0440] When the internet connection is restored, the device will automatically send the last message recorded by the user to the designated contacts. It takes the recording as input and sends the voice message to the designated contacts as output.

[0441] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0442] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0443] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0444] [Second embodiment]

[0445] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0446] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0447] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0448] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0449] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0450] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0451] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0452] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0453] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0454] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0455] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0456] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0457] System Overview

[0458] This invention is a system for providing rapid and effective rescue and support during large-scale disasters such as earthquakes. This system automatically activates when an earthquake of magnitude 7 or higher occurs, detects the voices of victims, and issues rescue requests. It also enables information sharing using ad-hoc communication even when the internet connection is lost, and uses edge AI to provide first aid instructions and psychological support.

[0459] Server Features

[0460] The server monitors earthquake information and sends an alert to user devices in the relevant area when an earthquake of magnitude 7 or higher occurs. This alert automatically wakes up the user device and triggers the rescue request process. The server also stores and continuously updates the user's location and status information in a database to support rescue efforts.

[0461] Device Features

[0462] The device automatically wakes up when it receives an alert from the server, detects the user's voice, and generates the necessary rescue request message. The rescue request message includes the user's current location and a brief description of the situation, and is immediately sent to the server if there is an Internet connection. Furthermore, if the Internet connection is lost, the device automatically switches to ad-hoc communication mode and shares information by directly communicating with other devices in the vicinity.

[0463] The Edge AI runs on the device and provides first aid instructions and psychological support to victims. For example, if the user is injured, it will provide specific voice instructions such as "If there is bleeding, apply pressure with a clean cloth." If the user is feeling anxious or scared, it will provide psychological support such as "Please stay calm. Take a deep breath and relax."

[0464] User Roles

[0465] When an earthquake occurs, users follow the instructions provided by the device to ensure their own safety and, if necessary, request rescue by voice. When the device detects voice, it automatically generates a rescue request message and sends it to the server. Users can also record a final message on the device, which will be automatically sent to designated contacts as soon as the internet connection is restored.

[0466] Specific use cases

[0467] Use case 1: Immediately after an earthquake occurs

[0468] The server detects earthquakes of magnitude 7 or higher and sends alerts to users' devices in the affected area.

[0469] The device will automatically launch the app and enter voice detection mode.

[0470] When a user says "Help!", the device detects the voice and generates and sends a rescue request message.

[0471] Use case 2: When the Internet is down

[0472] The device checks for an Internet connection and, if it determines that it is unavailable, automatically switches to ad-hoc communication mode.

[0473] The device communicates directly with the devices of other victims in the vicinity, sharing rescue request information and location information.

[0474] Use case 3: When first aid is required

[0475] If the user is injured, the device will use edge AI to give voice instructions such as, "If there is bleeding, apply pressure with a clean cloth."

[0476] Edge AI also provides psychological support, such as "Stay calm. Take a deep breath and relax."

[0477] This system allows disaster victims to receive prompt and effective assistance and creates an environment in which local and community cooperation is facilitated.

[0478] The processing flow will be explained below.

[0479] Step 1:

[0480] The server monitors earthquake information.

[0481] Data is regularly obtained from the Japan Meteorological Agency and earthquake early warning services and the seismic intensity is analyzed.

[0482] Step 2:

[0483] If an earthquake with a magnitude of 7 or higher is detected, the server will send an alert to user devices in the relevant area.

[0484] The alert will include information on the earthquake's epicenter and magnitude.

[0485] Step 3:

[0486] The device receives an alert from the server and automatically launches the app.

[0487] When the app is launched, the microphone goes into voice detection mode.

[0488] Step 4:

[0489] The user makes a sound requesting rescue, such as "help me."

[0490] The sound is detected by the device's microphone.

[0491] Step 5:

[0492] The device analyzes the voice and generates a message requesting help.

[0493] The rescue request message will include the user's location and a brief description of the situation.

[0494] Step 6:

[0495] The terminal sends a rescue request message to the server.

[0496] If there is an internet connection, the message is sent immediately to the server.

[0497] Step 7:

[0498] The device checks for internet connectivity.

[0499] If the connection is lost, it will automatically switch to ad-hoc communication mode.

[0500] Step 8:

[0501] In the ad-hoc communication mode, a terminal communicates directly with other terminals in the vicinity.

[0502] Broadcast a rescue request message to other devices and share information.

[0503] Step 9:

[0504] Edge AI runs within the device and analyzes the user's situation.

[0505] If first aid is required, detailed instructions are provided via voice.

[0506] Step 10:

[0507] Edge AI initiates psychological support dialogue.

[0508] Providing users with messages such as "Take a deep breath and stay calm."

[0509] Step 11:

[0510] The device periodically collects the user's location and status information.

[0511] Data is collected based on GPS data and user input.

[0512] Step 12:

[0513] The terminal sends the collected information to the server and stores it in a database.

[0514] The information is continuously updated and used in rescue operations.

[0515] Step 13:

[0516] The user records a final message on the device.

[0517] The recorded message is saved on the device.

[0518] Step 14:

[0519] Once your internet connection is restored, your device will automatically send a recorded message to your designated contacts.

[0520] Your message will be sent to your family and friends.

[0521] Example 1

[0522] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0523] Providing prompt and effective rescue and assistance is crucial when large-scale natural disasters occur, especially earthquakes with a seismic intensity of 7 or higher. However, conventional methods often result in the destruction of communication infrastructure, making it difficult to quickly share information and request rescue. Furthermore, there are limitations on the means by which victims can receive instructions on how to provide appropriate first aid and psychological support. This makes it difficult for conventional systems to provide complete support.

[0524] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0525] In this invention, the server includes means for monitoring seismic intensity information and detecting earthquakes of seismic intensity 7 or higher, means for sending alerts to user terminals based on the seismic intensity information, and means for analyzing meteorological data and identifying users in the affected area. This makes it possible to quickly send alerts to users in the affected area and begin providing support when an earthquake of seismic intensity 7 or higher occurs.

[0526] The user's device also includes means for automatically activating upon receiving the alert, means for detecting the user's voice and generating and transmitting a rescue request message, means for switching to ad-hoc communication mode if the Internet connection is lost, means for analyzing the user's situation based on edge artificial intelligence and generating first aid instructions and psychological support, and means for detecting the voice of the user when the user makes a rescue request and generating the necessary message. This enables victims to quickly request rescue even if infrastructure is destroyed, and to receive appropriate first aid instructions and psychological support.

[0527] "Seismic intensity information" is data that indicates the strength of an earthquake and the degree of shaking.

[0528] "Means for detecting earthquakes of seismic intensity 7 or higher" refers to a device or program that has the function of monitoring earthquake information and detecting the occurrence of an earthquake of seismic intensity 7 or higher.

[0529] "Means for sending an alert to a user's terminal" refers to a device or program that has the function of sending a warning message to a communication device owned by a user based on seismic intensity information.

[0530] The "means for automatically starting up the user's terminal" refers to a device or program that has the function of automatically starting up the user's terminal upon receiving an alert.

[0531] "Means for detecting the user's voice and generating and transmitting a rescue request message" refers to a device or program that has the function of detecting the voice uttered by the user and creating and transmitting a message requesting rescue based on that voice.

[0532] "Means for switching to ad hoc communication mode when Internet connection is lost" refers to a device or program that has the function of automatically changing to a mode for direct communication between terminals when Internet connection is unavailable.

[0533] "Edge AI" is an artificial intelligence technology installed within a terminal, and is a system that has the ability to perform data analysis and decision-making in real time.

[0534] "Means for analyzing the user's situation and providing first aid instructions and psychological support" refers to a device or program that has the function of analyzing the situation of a disaster victim and providing necessary first aid instructions and psychological support via voice or other means.

[0535] "Means for analyzing meteorological data and identifying users in the affected area" refers to a device or program that has the function of analyzing earthquake and other meteorological information and identifying users in the affected area.

[0536] "Means for detecting voices requesting rescue and generating the necessary messages when the voice is uttered" refers to a device or program that has the function of detecting the voices when a user utters a voice requesting rescue and automatically generating a corresponding message.

[0537] This invention is a system for providing prompt and effective rescue and support in the event of a large-scale disaster, such as an earthquake with a seismic intensity of 7 or higher. This system is composed of a server, terminals, and users, each of which plays a specific role.

[0538] Server Features

[0539] The server is responsible for constantly monitoring earthquake information and analyzing meteorological data. The server obtains data in real time from earthquake alert services such as the Japan Meteorological Agency API, and sends an alert to user devices in the relevant area when an earthquake of magnitude 7 or higher occurs. This alert automatically activates the device and triggers the rescue request process. The server also stores and continuously updates the user's location and status information in a database. Specific server product names that can be used are Amazon Web Services (AWS) and Google Cloud Platform (GCP).

[0540] Device Features

[0541] When the device receives an alert from the server, it automatically launches the app and enters voice detection mode. The device detects the user's voice and generates and sends a rescue request message. This message includes the user's current location and a brief description of the situation. If the internet connection is lost, the device automatically switches to ad-hoc communication mode and communicates directly with other devices in the vicinity. In addition, the edge artificial intelligence (Edge AI) installed in the device provides first aid instructions and psychological support to victims. For example, if the user is injured, it will issue voice instructions such as, "If there is bleeding, apply pressure with a clean cloth." If the user is feeling anxious or scared, it will provide psychological support such as, "Please stay calm. Take a deep breath and relax."

[0542] User Roles

[0543] When an earthquake occurs, users follow the instructions provided by the device to ensure their own safety. If necessary, they can request rescue by voice. When the device detects the user's voice, it automatically generates a rescue request message and sends it to the server. Users can also record a final message on the device, which will be automatically sent to the designated contacts once the internet connection is restored.

[0544] Specific use cases

[0545] Use case 1: Immediately after an earthquake occurs

[0546] The server detects earthquakes of magnitude 7 or higher and sends alerts to users' devices in the affected area.

[0547] The device will automatically launch the app and enter voice detection mode.

[0548] When a user says "Help!", the device detects the voice and generates and sends a rescue request message.

[0549] Use case 2: When the Internet is down

[0550] The device checks for an Internet connection and, if it determines that it is unavailable, automatically switches to ad-hoc communication mode.

[0551] The device communicates directly with the devices of other victims in the vicinity, sharing rescue request information and location information.

[0552] Use case 3: When first aid is required

[0553] If the user is injured, the device will use edge AI to give voice instructions such as, "If there is bleeding, apply pressure with a clean cloth."

[0554] Edge AI also provides psychological support, such as "Stay calm. Take a deep breath and relax."

[0555] Prompt Sentence Examples

[0556] By using a generative AI model in this system, it is possible to generate detailed rescue requests and first aid instructions. An example of a prompt sentence to be input to the generative AI model is as follows:

[0557] "There have been many injuries from the earthquake. What should we do if someone is bleeding?"

[0558] "If survivors are feeling anxious, what words of psychological support can you offer?"

[0559] Using this prompt, the AI ​​generates specific instructions and support words that it provides to the user via the device.

[0560] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0561] Step 1:

[0562] The server constantly monitors earthquake information and analyzes meteorological data. Specifically, it obtains data in real time from earthquake early warning services such as the Japan Meteorological Agency API. This data includes information such as seismic intensity, location, and time of occurrence. The server analyzes this data, and if an earthquake with a seismic intensity of 7 or higher occurs, it identifies users in the affected area. Based on the analysis results, it generates a list of users in the affected area.

[0563] Input: Real-time earthquake data obtained from the Japan Meteorological Agency API

[0564] Output: List of users in the affected area

[0565] Step 2:

[0566] The server sends an alert to the user's device based on the seismic intensity information. Specifically, it references a list of users in the affected area and generates an alert message based on this. The message contains earthquake information as well as a message saying, "Please launch the app immediately and follow the instructions." This alert message is sent to each user's device as a push notification.

[0567] Input: User list in affected area, seismic intensity information

[0568] Output: Alert message to the user's terminal

[0569] Step 3:

[0570] When the device receives an alert, it automatically launches the app and enters voice detection mode. Specifically, the application on the device waits in the background and is launched when the alert notification is triggered. When the app launches, the microphone is enabled and it is ready to detect the user's voice.

[0571] Input: Alert message sent from the server

[0572] Output: Device that has entered voice detection state

[0573] Step 4:

[0574] When a user utters a voice requesting rescue, such as "Help!", the device detects this. Specifically, it uses a voice recognition algorithm to recognize specific keywords (e.g., "Help!"). It then analyzes this voice data and generates a rescue request message. The message includes the user's current location (GPS information) and a brief description of the situation (voice-to-text).

[0575] Input: User's voice requesting help

[0576] Output: Help request message

[0577] Step 5:

[0578] The device generates a rescue request message and sends it to the server. If an Internet connection is available, the message is sent immediately to the server. If the Internet connection is lost, the device switches to ad-hoc communication mode and relays the message through other devices.

[0579] Input: Message requesting help

[0580] Output: A help-call message sent to the server or a message relayed via ad-hoc communication.

[0581] Step 6:

[0582] The server analyzes the received rescue request message and forwards the information to the appropriate rescue team. Specifically, it analyzes the content of the rescue request and forwards the message to the appropriate rescue team based on the urgency and required resources. It also generates optimal route information based on the location information and provides it to the rescue team.

[0583] Input: A message requesting help sent by the user

[0584] Output: Message forwarding to rescue team and optimal route information

[0585] Step 7:

[0586] The edge AI inside the device analyzes the user's situation in real time and provides first aid instructions and psychological support. Specifically, it analyzes the user's voice and behavior, and if there is an injury, for example, it will give instructions such as "If there is bleeding, apply pressure with a clean cloth." It also provides psychological support such as advice such as "Remain calm. Take a deep breath and relax."

[0587] Input: User voice and behavioral data

[0588] Output: Audio messages with first aid instructions and psychological support

[0589] (Application example 1)

[0590] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0591] In the event of a large-scale disaster, it is extremely important to carry out rapid and effective rescue operations for victims. However, conventional rescue operations rely on human resources and communication infrastructure, and have faced problems such as internet connection outages and difficulty sharing information in widespread disaster areas. Rescue operations tend to be delayed, especially in disaster-stricken areas outside of urban areas, making ensuring the safety of victims and providing early rescue a challenge. Furthermore, there is a need to simultaneously provide psychological support and first aid instructions to victims.

[0592] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0593] In this invention, the server includes: means for monitoring seismic intensity information and detecting earthquakes of seismic intensity 7 or higher; means for sending an alert to a user's device based on the seismic intensity information; means for automatically activating the user's device upon receiving the alert; means for detecting the user's voice and generating and transmitting a rescue request message; means for switching to ad hoc communication mode if the Internet connection is lost; means for providing first aid instructions and psychological support using edge AI; means for transmitting the user's location information and status information to the server and storing them in a database; means for automatically activating a terminal in the autonomous vehicle or an in-vehicle terminal; means for the terminal in the autonomous vehicle to detect the user's voice and generate and transmit a rescue request message; and means for the terminal in the autonomous vehicle to directly communicate with other autonomous vehicles and terminals using ad hoc communication to share information. This allows victims to quickly request rescue and enables efficient rescue operations using autonomous vehicles. Furthermore, edge AI can be used to provide first aid instructions and psychological support, ensuring the safety and security of victims.

[0594] "Seismic intensity information" is information that indicates the strength of shaking when an earthquake occurs.

[0595] "User devices" refers to smartphones or dedicated devices owned by disaster victims.

[0596] An "alert" is a message that notifies the user when a particular condition occurs.

[0597] "Voice detection" is a technology that detects the voices emitted by the user and recognizes them as specific commands or messages.

[0598] A "rescue request message" is a message that includes information to request rescue of a disaster victim.

[0599] "Ad hoc communication" is a method of communicating directly with other devices when there is no Internet connection.

[0600] "Edge AI" is an artificial intelligence technology that runs on a device and processes data and makes decisions in real time.

[0601] "First Aid Instructions" provide specific instructions for initial response to injuries and emergency situations.

[0602] "Psychological support" refers to providing advice and encouragement to reduce the mental anxiety and stress of disaster victims.

[0603] "Location information" is data that indicates a user's current physical location.

[0604] "Status information" is data about the user's health status and surrounding environment.

[0605] A "server" is a networked computer system that collects and processes data and provides information to other devices.

[0606] A "database" is an electronic storage device for organizing and storing information.

[0607] "Terminal in an autonomous vehicle" refers to a computing device installed in an autonomous vehicle.

[0608] "In-vehicle dedicated terminal" refers to a terminal used exclusively within an autonomous vehicle.

[0609] "Direct communication" is communication between devices without going through the Internet.

[0610] This invention is a system for carrying out rapid and effective rescue operations in the event of a large-scale disaster. The system detects the voices of victims and generates rescue request messages, enables information sharing using ad-hoc communication even when the internet connection is lost, and uses edge AI to provide first aid instructions and psychological support.

[0611] System Overview

[0612] The server monitors seismic intensity information and detects earthquakes of magnitude 7 or higher. When an earthquake occurs, an alert is sent to the user's device in the affected area, causing the device to automatically wake up. When the user's device receives the alert, it enters voice detection mode, detects the user's voice, and generates and sends a rescue request message. If the internet connection is lost, it switches to ad-hoc communication mode, communicating directly with other nearby devices to share information.

[0613] Device operation

[0614] The device automatically wakes up when it receives an alert from the server, detects the user's voice, and generates a rescue request message. The rescue request message includes the user's current location and a brief description of the situation, and is immediately sent to the server if there is an internet connection. In ad-hoc communication mode, information is shared by directly communicating with other nearby devices. In addition, the edge AI provides first aid instructions and psychological support within the device.

[0615] Use of autonomous vehicles

[0616] The system also incorporates a dedicated terminal installed in the autonomous vehicle that automatically activates when an earthquake occurs and detects the user's voice. The dedicated terminal generates a rescue request message and communicates ad hoc with other autonomous vehicles and terminals in the vicinity to share information.

[0617] Hardware and Software Details

[0618] Hardware: Dedicated terminals installed in autonomous vehicles, smartphones carried by users, microphones, and GPS sensors

[0619] Software: VoiceRecognition, GPS module (location information acquisition), EdgeAI (real-time data processing and instructions), AdHocCommunication

[0620] Program processing explanation

[0621] The server constantly monitors seismic intensity information, and if an earthquake of magnitude 7 or higher occurs, it sends an alert to devices in the relevant area. The device then wakes up and switches to voice detection mode to detect the user's voice. After detecting voice, it generates a rescue request message including GPS information and sends it to the server if there is an internet connection, or to another device via ad-hoc communication if there is not. The edge AI runs on the device and provides first aid instructions and psychological support.

[0622] Specific examples

[0623] Example 1: Immediately after an earthquake occurs

[0624] The server detects earthquakes with a magnitude of 7 or higher and sends alerts to users' devices in the affected area.

[0625] The device will automatically launch the app and enter voice detection mode.

[0626] When a user says "Help!", the device detects the voice and generates and sends a rescue request message.

[0627] Example prompt sentence:

[0628] The system automatically activates when an earthquake of magnitude 7 or higher occurs, detects the voice "Help!", and generates and sends a rescue request message. Furthermore, if the internet connection is cut off, the system shares information with other autonomous vehicles using ad-hoc communication.

[0629] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0630] Step 1:

[0631] The server constantly monitors seismic intensity information. As input, it receives real-time seismic intensity data from the Japan Meteorological Agency and seismometers, analyzes that data, and determines whether an earthquake of seismic intensity 7 or higher has occurred. As output, if an earthquake of seismic intensity 7 or higher is detected, it generates a signal to send an alert to user devices in the relevant area.

[0632] Step 2:

[0633] The user's device receives the alert from the server. As an input, it receives the alert signal sent from the server and uses it as a trigger to automatically wake up the device. As an output, it sends information to the device system to switch to voice-sensing mode.

[0634] Step 3:

[0635] The device then enters voice detection mode. The device's built-in microphone collects voice data from the user's surroundings as input, which is then fed into the VoiceRecognition software. The collected voice data is then analyzed and recognized as a rescue request, such as "Help!". The output is the voice recognition results provided to the device's first aid instruction module.

[0636] Step 4:

[0637] The user says "Help!". As input, the user's voice is collected by the device's microphone and analyzed by VoiceRecognition software. As output, if "Help!" is recognized, a signal is generated to generate a rescue request message.

[0638] Step 5:

[0639] The device generates a rescue request message. As input, it combines the voice recognition results with the current location data obtained from the GPS module. As data processing, it integrates the user's location information and the rescue request voice information to generate a detailed rescue request message. As output, it prepares the message to be sent to the server.

[0640] Step 6:

[0641] Checking Internet connection. As input, the terminal checks whether it has an Internet connection. As data processing, it performs an Internet connection test and obtains the results. As output, if there is an Internet connection, it generates a signal to send a rescue request message to the server, and if there is not, it generates a signal to switch to ad-hoc communication mode.

[0642] Step 7:

[0643] Switching to ad-hoc communication mode. As input, it receives information that there is no Internet connection. As data processing, it changes the communication mode of the terminal to ad-hoc mode. As output, it starts direct communication with another terminal and sends a rescue request message.

[0644] Step 8:

[0645] Provision of first aid instructions and psychological support using edge AI. Voice recognition results and user condition information are acquired as input. The edge AI processes the data to generate first aid instructions and psychological support messages appropriate to the situation. The output is to provide instructions and support to the user via voice or text.

[0646] Specific examples

[0647] Example prompt sentence:

[0648] The system automatically activates when an earthquake of magnitude 7 or higher occurs, detects the voice "Help!", and generates and sends a rescue request message. Furthermore, if the internet connection is cut off, the system shares information with other autonomous vehicles using ad-hoc communication.

[0649] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0650] System Overview

[0651] This invention is a system for providing prompt and effective rescue and support in the event of a large-scale disaster such as an earthquake. This system automatically activates when an earthquake of magnitude 7 or higher occurs, detects the voices of victims, and issues a rescue request. Even if the internet connection is cut off, the system can share information using ad-hoc communication, and utilizes edge AI and an emotion engine to provide first aid instructions and psychological support.

[0652] Server Features

[0653] The server monitors earthquake information and sends an alert to user devices in the relevant area when an earthquake of magnitude 7 or higher occurs. The alert automatically activates the user device and triggers the rescue request process. The server stores the user's location, status, and emotional data in a database to help optimize rescue operations.

[0654] Device Features

[0655] When the device receives an alert, it automatically launches the app and enters voice detection mode. When the user utters a voice command such as "help me," the device detects it and generates and sends a rescue request message. If the internet connection is lost, the device switches to ad-hoc communication mode and communicates directly with other nearby devices to share information.

[0656] The Edge AI runs within the device and provides first aid instructions and psychological support to disaster victims. The emotion engine recognizes emotions from the user's voice and facial expressions and adjusts the support provided based on those emotions. For example, if the user expresses strong anxiety or fear, the Edge AI will provide a customized message such as, "Don't worry. Help is on its way."

[0657] User Roles

[0658] When an earthquake occurs, users follow the instructions provided by the device to ensure their own safety and make a voice call to request rescue. When the device detects voice, it automatically generates a rescue request message and sends it to the server. Users can also record a final message on the device, which will be automatically sent to designated contacts when the internet connection is restored.

[0659] Specific use cases

[0660] Use case 1: Immediately after an earthquake occurs

[0661] The server detects earthquakes with a magnitude of 7 or higher and sends alerts to users' devices in the affected area.

[0662] The device will automatically launch the app and enter voice detection mode.

[0663] When a user says "Help!", the device detects the voice and generates and sends a rescue request message.

[0664] Use case 2: When the Internet is down

[0665] The device checks for an Internet connection and, if it determines that it is unavailable, automatically switches to ad-hoc communication mode.

[0666] The device communicates directly with the devices of other victims in the vicinity, sharing rescue request information and location information.

[0667] Use Case 3: Supported by Emotion Engine

[0668] If the user is injured and feels strong anxiety, the device will use edge AI and an emotion engine to give instructions such as "If there is bleeding, apply pressure with a clean cloth" and also provide psychological support such as "Don't worry, help will arrive shortly."

[0669] The emotion engine recognizes emotional data from the user's voice and facial expressions, sends it to the server, and stores it in a database.

[0670] What the program does

[0671] The server constantly monitors earthquake information, and when it detects an earthquake with a seismic intensity of 7 or higher, it sends an alert to the user's device in the relevant area. When the device receives the alert, it automatically launches the app and enters voice detection mode. When the user issues a voice request for rescue, the device analyzes the voice and generates a rescue request message.

[0672] If the internet connection is lost, the device will switch to ad-hoc communication mode and directly communicate with other devices to share information. Edge AI will provide first aid instructions and psychological support, and the emotion engine will analyze the user's emotions and provide appropriate support.

[0673] For example, if a user expresses strong anxiety or fear, the edge AI will generate a message saying, "Don't worry, help is on its way," and the emotion engine will report the situation to the server. This information will be stored in a database and used to optimize rescue operations.

[0674] The processing flow will be explained below.

[0675] Step 1:

[0676] The server monitors earthquake information.

[0677] Data is regularly obtained from the Japan Meteorological Agency and earthquake early warning services and the seismic intensity is analyzed.

[0678] Step 2:

[0679] If an earthquake with a magnitude of 7 or higher is detected, the server will send an alert to user devices in the relevant area.

[0680] The alert will include information on the earthquake's epicenter and magnitude.

[0681] Step 3:

[0682] The device receives an alert from the server and automatically launches the app.

[0683] When the app is launched, it switches to voice detection mode.

[0684] Step 4:

[0685] The user makes a sound requesting rescue, such as "help me."

[0686] The sound is detected by the device's microphone.

[0687] Step 5:

[0688] The device analyzes the voice and generates a message requesting help.

[0689] The rescue request message will include the user's location and a brief description of the situation.

[0690] Step 6:

[0691] The terminal sends a rescue request message to the server.

[0692] If there is an internet connection, the message is sent immediately to the server.

[0693] Step 7:

[0694] The device checks for internet connectivity.

[0695] If the connection is lost, it will automatically switch to ad-hoc communication mode.

[0696] Step 8:

[0697] In the ad-hoc communication mode, a terminal communicates directly with other terminals in the vicinity.

[0698] Broadcast a rescue request message to other devices and share information.

[0699] Step 9:

[0700] Edge AI runs within the device and analyzes the user's situation.

[0701] If first aid is required, detailed instructions are provided via voice.

[0702] Step 10:

[0703] The device's emotion engine analyzes the user's voice and facial expressions to recognize their emotions.

[0704] The recognized emotions are structured as data.

[0705] Step 11:

[0706] Edge AI tailors first aid instructions and psychological support based on emotions recognized by the emotion engine.

[0707] For example, if a user is feeling very anxious, it will generate a message such as "Don't worry, help is on its way."

[0708] Step 12:

[0709] The device transmits the user's emotional information recognized by the emotion engine to the server.

[0710] Emotional information is stored in a database and used to optimize rescue operations.

[0711] Step 13:

[0712] The device periodically collects the user's location and status information.

[0713] Data is collected based on GPS data and user input.

[0714] Step 14:

[0715] The terminal sends the collected information to the server and stores it in a database.

[0716] The information is continuously updated and used in rescue operations.

[0717] Step 15:

[0718] The user records a final message on the device.

[0719] The recorded message is saved on the device.

[0720] Step 16:

[0721] Once your internet connection is restored, your device will automatically send a recorded message to your designated contacts.

[0722] Your message will be sent to your family and friends.

[0723] Example 2

[0724] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0725] In the event of a large-scale disaster such as an earthquake, victims lack the means to quickly and effectively request rescue. There is also a need for a method to efficiently share rescue information even when internet connection is cut off. Furthermore, a support system is needed to reduce the psychological burden on victims and provide appropriate first aid.

[0726] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for monitoring seismic intensity information and detecting earthquakes of seismic intensity 7 or higher, means for sending an alert to a user's terminal based on the seismic intensity information, means for automatically activating the user's terminal that receives the alert, means for detecting the user's voice and generating and sending a rescue request message, means for switching to ad hoc communication mode when the Internet connection is lost, means for providing first aid instructions and psychological support using edge AI, means for sending the user's location information and status information to the server and storing them in a database, and means for analyzing the user's emotions and sending the analysis results to the server. This enables victims to quickly and reliably request rescue, enables information sharing even in situations where the Internet connection is lost, and further reduces the psychological burden on victims and enables appropriate first aid to be provided.

[0727] "Seismic intensity information" is data that indicates the scale of an earthquake and the strength of the vibrations, and includes information such as the location and time of the earthquake and the depth of the epicenter.

[0728] "User device" refers to a mobile information terminal such as a smartphone or tablet, and is the device to which this system sends alerts.

[0729] An "alert" is a warning message that the system sends to the user's device after detecting an earthquake, and serves as a trigger to start the rescue request process.

[0730] "Voice detection" is the process of detecting the user's voice using the device's microphone and analyzing the voice data.

[0731] A "rescue request message" is a message generated based on the user's voice requesting rescue, and is sent to the server along with the user's location information.

[0732] "Ad-hoc communication mode" is a mode in which terminals connect directly to each other and communicate data when an Internet connection is not available.

[0733] "Edge AI" is artificial intelligence that runs within a device and has the ability to provide first aid instructions and psychological support in real time.

[0734] "First aid instructions" are instructions provided by edge AI on the medical procedures that victims should take immediately.

[0735] "Psychological support" refers to messages and actions that edge AI uses to stabilize the mental state of disaster victims and provide them with a sense of security.

[0736] "Location information" is data that includes the user's current latitude and longitude, and is important during rescue operations.

[0737] "Status information" is data related to the user's health condition and the extent of the damage.

[0738] The "database" is a system installed on a server for storing collected earthquake information, user location information, status information, and emotional data.

[0739] "Emotion analysis" is the process of inferring a user's emotions from their voice and facial expressions, and is the basis for providing appropriate assistance.

[0740] "Emotion data" is data that indicates the user's emotional state, and is information that is automatically sent to the server.

[0741] MODE FOR CARRYING OUT THE INVENTION

[0742] This invention is a system for providing prompt and effective rescue and support in the event of a large-scale disaster such as an earthquake. This system automatically activates when an earthquake of magnitude 7 or higher occurs, and can detect the voices of victims and issue rescue requests. Even if the internet connection is cut off, the system can share information using ad-hoc communication and provide first aid instructions and psychological support using edge AI and an emotion engine.

[0743] Hardware and Software Configuration

[0744] server

[0745] The server has the function of constantly monitoring earthquake information. To obtain real-time earthquake data, the server obtains earthquake occurrence information using APIs from the Japan Meteorological Agency and other earthquake information providers. The server is implemented using programming languages ​​such as Python and JavaScript, and uses databases such as PostgreSQL and MySQL.

[0746] Terminal

[0747] The device is a user's smartphone or tablet. When the device receives an alert from the server, it automatically launches the app and goes into voice-activated mode. The device runs an application compatible with the Android or iOS operating system.

[0748] Edge AI

[0749] The AI ​​model runs on the edge device and uses lightweight AI frameworks such as TensorFlow Lite and PyTorch Mobile to analyze the user's voice and facial expressions, and provides first aid instructions and psychological support in real time.

[0750] System Operation

[0751] server

[0752] The server analyzes the earthquake information it receives and sends an alert to user devices in the relevant area if an earthquake with a seismic intensity of 7 or higher occurs. When a user device receives the alert, it automatically launches the app and enters voice detection mode. At this time, the server stores the user's location and status information in a database to help optimize rescue operations.

[0753] Terminal

[0754] When the device receives an alert, it automatically launches the app and enters voice detection mode. When the user utters a word such as "help me," the device's microphone picks up the sound and the voice recognition engine analyzes it. A rescue request message is generated from the analyzed voice and sent to the server.

[0755] If an internet connection is unavailable, the device automatically switches to ad-hoc communication mode, allowing it to communicate directly with other devices in the vicinity and share information such as rescue requests and location information.

[0756] Edge AI and Emotion Engine

[0757] The edge AI provides first aid instructions to the user. For example, it provides voice instructions such as, "If there is bleeding, apply pressure with a clean cloth." The emotion engine also analyzes the user's emotions from their voice and facial expressions, and provides appropriate psychological support based on the analysis results. For example, if the user shows strong anxiety or fear, it can provide a message such as, "Don't worry. Help will arrive shortly."

[0758] Specific examples of operation

[0759] For example, if a user is trapped in a building that has collapsed due to an earthquake and yells "help," the system operates as follows:

[0760] 1. The server detects earthquakes with a magnitude of 7 or higher and sends an alert to user devices in the relevant area.

[0761] 2. The device will receive the alert and automatically launch the app to enter voice detection mode.

[0762] 3. When the user shouts "help," the device's microphone detects the voice and the voice recognition engine analyzes it.

[0763] 4. The device generates a rescue request message and sends it to the server.

[0764] 5. If the Internet connection is lost, the device switches to ad-hoc communication mode and shares information with other devices in the vicinity.

[0765] 6. The edge AI gives instructions such as "If there is bleeding, apply pressure with a clean cloth," and the emotion engine provides psychological support such as "Don't worry, help will arrive soon."

[0766] Prompt Sentence Examples

[0767] "An earthquake of magnitude 7 or higher has occurred. Please explain how the rescue system will respond to this situation from the perspectives of the server, the device, and the user."

[0768] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0769] The flow of this system's program processing

[0770] Step 1:

[0771] The server monitors earthquake information

[0772] The server calls the APIs of the Japan Meteorological Agency and other earthquake information providers to obtain real-time earthquake data. The input includes earthquake information obtained from external APIs, including the location, time, depth of the epicenter, and seismic intensity of the earthquake. The server analyzes this data and detects earthquakes with a seismic intensity of 7 or higher. The output is a flag when an earthquake with a seismic intensity of 7 or higher is detected.

[0773] Step 2:

[0774] The server sends an alert to the user's device in the relevant area.

[0775] The server analyzes the earthquake information and sends an alert message to the devices of users living in areas judged to have a seismic intensity of 7 or higher. The input includes the earthquake information obtained in step 1 and the user's location information in the database. Based on this, the server identifies users in the relevant area and sends an alert to their device. The output is the alert message sent to the user's device.

[0776] Step 3:

[0777] The device receives the alert and automatically launches the app

[0778] The device has a listener for receiving alerts sent from the server. The input is the alert message sent from the server. When the listener receives an alert, the device automatically launches the app. It also enters voice-sensing mode, so the device's microphone is always on. The output is the launch of the app and the transition to voice-sensing mode.

[0779] Step 4:

[0780] The device detects the user's voice and generates a message requesting help.

[0781] When a user utters a voice request for help, such as "help me," the device's microphone picks up the voice. The input is the user's voice. The speech recognition engine analyzes the voice, and a rescue request message is generated as a result of the analysis. The output is the generated rescue request message.

[0782] Step 5:

[0783] The device sends a rescue request message to the server.

[0784] The generated distress request message is sent to the server using the device's normal internet connection. The input is the distress request message generated in step 4. The server receives it and stores it in a database. The output is the distress request message sent to the server.

[0785] Step 6:

[0786] If the Internet connection is lost, the device switches to ad-hoc communication mode.

[0787] The device periodically checks for an Internet connection and automatically switches to ad-hoc communication mode if one is unavailable. The input is the result of checking the Internet connection status. Once in ad-hoc communication mode, it communicates directly with other devices in the vicinity and shares rescue request information and location information. The output is the information shared through communication with other devices.

[0788] Step 7:

[0789] Edge AI provides first aid instructions and psychological support to disaster victims

[0790] Edge AI uses data such as the user's voice and facial expressions to provide appropriate first aid instructions and psychological support. The input is the user's voice, facial expression information, location information, and emotional data. Edge AI analyzes this and provides voice instructions such as "If there is bleeding, apply pressure with a clean cloth" and psychological support such as "Don't worry, help will arrive shortly." The output is the provided first aid instructions and psychological support message.

[0791] Step 8:

[0792] The device uses an emotion engine to analyze the user's emotions.

[0793] The device's emotion engine analyzes emotions from the user's voice and facial expressions. The input is the user's voice and facial expression data. The emotion engine identifies the user's emotion based on this data and generates an analysis result. The output is the analyzed emotion data.

[0794] Step 9:

[0795] The emotion engine sends the analysis results to the server and stores them in a database.

[0796] The analysis results generated by the emotion engine are sent to the server via the Internet. The input is the emotion data generated in step 8. The server receives this and stores it in a database. The output is the emotion data stored in the database.

[0797] (Application example 2)

[0798] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0799] Conventional disaster rescue systems have had problems such as ineffective functioning when the internet connection is cut off or when psychological support for people at the disaster site is lacking. Furthermore, in-store emergency response measures lack the means for a central control unit to quickly and simultaneously transmit information to multiple devices. A system that can solve these problems, provide rescue and support quickly and effectively in the event of a disaster, and ensure the safety of customers and staff in the store, is needed.

[0800] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for monitoring seismic intensity information and detecting earthquakes of seismic intensity 7 or higher, means for sending an alert to a user's device based on the seismic intensity information, means for automatically activating the user's device that receives the alert, means for detecting the user's voice and generating and sending a rescue request message, means for switching to ad hoc communication mode when the Internet connection is lost, means for providing first aid instructions and psychological support using edge AI, means for sending user location information and status information to a central device and storing it in an information storage device, means for providing voice instructions to customers and staff in the store, and means for simultaneously sending alerts to multiple devices in the store based on the seismic intensity information. This makes it possible to share information using ad hoc communication even in situations where the Internet is lost, and to provide quick and appropriate first aid and psychological support to customers and staff in the store.

[0801] "Seismic intensity information" is data indicating the seismic intensity of an earthquake, and is usually provided by seismological observation organizations.

[0802] "Users" refer to customers and staff in physical stores.

[0803] "Terminal" refers to a mobile device such as a smartphone or smart glasses.

[0804] "Internet connection" refers to the communication means by which a terminal accesses the Internet.

[0805] An "alert" is a warning message sent to a device when an earthquake with a seismic intensity of 7 or higher occurs.

[0806] "Ad hoc communication" is a method in which terminals communicate directly with each other when the Internet connection is cut off.

[0807] "Edge AI" refers to artificial intelligence technology that runs on the device itself, allowing for real-time processing without relying on cloud servers.

[0808] "First Aid Instructions" provide appropriate medical guidelines for injured users.

[0809] "Psychological support" refers to encouraging and stabilizing messages provided to help clients achieve their emotional well-being.

[0810] A "central device" is a device that oversees and manages data within a physical store.

[0811] The "information storage device" is a database for storing data collected by the central device.

[0812] "Voice instructions" are instructions given to users and staff via voice.

[0813] A "rescue request message" is a message generated by detecting a voice calling for help made by a user in the event of a disaster.

[0814] "Multiple terminals in store" refers to multiple mobile devices located in a physical store.

[0815] System Overview

[0816] This invention is a system for providing prompt and effective rescue and support in the event of a large-scale disaster, such as an earthquake with a seismic intensity of 7 or higher, occurring within a brick-and-mortar store. This system primarily includes a server, a device equipped with edge AI, and users (customers and staff in the store).

[0817] Server Features

[0818] The server constantly monitors earthquake intensity information and, if it detects an earthquake of magnitude 7 or higher, sends an alert to the devices of users in the relevant area. At this time, the server obtains the intensity information from the meteorological agency's API using AWS IoT. The server also supports an ad-hoc communication mode that can be used even if the internet connection is cut off.

[0819] Device Features

[0820] When a device (such as a smartphone or smart glasses) receives an alert from the server, it automatically wakes up and enters voice-sensing mode. The device is equipped with a TensorFlow-based voice recognition model, which detects when a user speaks a rescue request such as "help me" and generates a rescue request message. The generated message is immediately sent to a central device or an external rescue agency.

[0821] If the device loses internet connection, it switches to ad-hoc communication mode and communicates directly with other devices in the store to share information. This communication uses Bluetooth to transmit data between devices.

[0822] Edge AI runs within the device and provides real-time first aid instructions to disaster victims. Specifically, it presents first aid methods for injuries and gives voice instructions on the necessary actions. Furthermore, the emotion engine recognizes emotions from the user's voice and facial expressions and provides psychological support based on those emotions. For example, if a user is feeling extremely anxious, it will provide a message such as, "Don't worry. Help will arrive shortly."

[0823] User Roles

[0824] When an earthquake occurs, users can follow the instructions provided by the device to ensure their own safety and make a voice call for help. Users can also record a final message on the device, which will be automatically sent to designated contacts when internet connection is restored.

[0825] Specific examples

[0826] Example 1: Immediately after an earthquake occurs

[0827] The server detects earthquakes of magnitude 7 or higher and sends an alert to all terminals in the store.

[0828] The device will automatically start up and enter voice-activated mode.

[0829] When a user shouts "Help!", the device detects the voice, generates a rescue request message, and sends it to the central device.

[0830] Example 2: Internet outage

[0831] The device checks for an Internet connection and, if it determines that it is unavailable, automatically switches to ad-hoc communication mode.

[0832] The device communicates directly with other nearby victims' devices, sharing rescue request information and location information.

[0833] Example 3: Supported by an Emotion Engine

[0834] If a user is injured and at the same time feels strong anxiety, the device's edge AI and emotion engine will instruct them to "apply pressure with a clean cloth if there is bleeding," and also provide psychological support by saying, "Don't worry, help will arrive shortly."

[0835] Example prompts to input to the generative AI model

[0836] "An earthquake of magnitude 7 or higher has occurred. Please move to a safe place. If you need first aid, please follow the steps below. If you are injured, apply pressure with a clean cloth to stop bleeding. We will also provide psychological support. Please rest assured, help will be on its way soon."

[0837] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0838] Step 1:

[0839] The server monitors seismic intensity information. Specifically, it uses AWS IoT Core to obtain real-time earthquake information from the meteorological agency's API. It receives seismic intensity data from the API as input and generates an alert signal as output when an earthquake with a seismic intensity of 7 or higher occurs.

[0840] Step 2:

[0841] When the server detects an earthquake with a seismic intensity of 7 or higher, it sends an alert to multiple users' devices. It receives the alert signal generated in step 1 as input and sends an alert message to all devices using the MQTT protocol as output.

[0842] Step 3:

[0843] When the device receives an alert from the server, it automatically wakes up and enters voice-sensitive mode, taking the alert message from the server as input and arming the voice recognition system as output.

[0844] Step 4:

[0845] The user issues a voice request such as "help me." The device detects the voice and analyzes the voice data using a TensorFlow-based speech recognition model. It receives the voice data as input and generates a text message as output.

[0846] Step 5:

[0847] The terminal transmits the generated rescue request message to a central device or an external rescue organization. The terminal receives a rescue request message in text format as input and transmits the message to a specified rescue organization as output.

[0848] Step 6:

[0849] If the Internet connection is lost, the device automatically switches to ad-hoc communication mode, checking for Internet connectivity as input and using Bluetooth as output to start communicating directly with other devices.

[0850] Step 7:

[0851] Edge AI operates within the device and provides first aid instructions to the user, receiving information about the user's condition as input and providing specific first aid instructions via voice as output.

[0852] Step 8:

[0853] The emotion engine recognizes emotions from the user's voice and facial expressions and provides psychological support. It receives voice and facial expression data as input, and generates a message according to the user's emotional state as output, which is delivered in voice.

[0854] Step 9:

[0855] The terminal sends rescue request messages and user location and status information to the central unit, which stores them in the information storage device. The terminal receives location and status information as input, sends the data to the central unit as output, and stores it in the information storage device.

[0856] Step 10:

[0857] When the internet connection is restored, the device will automatically send the last message recorded by the user to the designated contacts. It takes the recording as input and sends the voice message to the designated contacts as output.

[0858] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0859] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0860] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0861] [Third embodiment]

[0862] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0863] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0864] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0865] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0866] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0867] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0868] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0869] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0870] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0871] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0872] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0873] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0874] System Overview

[0875] This invention is a system for providing rapid and effective rescue and support during large-scale disasters such as earthquakes. This system automatically activates when an earthquake of magnitude 7 or higher occurs, detects the voices of victims, and issues rescue requests. It also enables information sharing using ad-hoc communication even when the internet connection is lost, and uses edge AI to provide first aid instructions and psychological support.

[0876] Server Features

[0877] The server monitors earthquake information and sends an alert to user devices in the relevant area when an earthquake of magnitude 7 or higher occurs. This alert automatically wakes up the user device and triggers the rescue request process. The server also stores and continuously updates the user's location and status information in a database to support rescue efforts.

[0878] Device Features

[0879] The device automatically wakes up when it receives an alert from the server, detects the user's voice, and generates the necessary rescue request message. The rescue request message includes the user's current location and a brief description of the situation, and is immediately sent to the server if there is an Internet connection. Furthermore, if the Internet connection is lost, the device automatically switches to ad-hoc communication mode and shares information by directly communicating with other devices in the vicinity.

[0880] The Edge AI runs on the device and provides first aid instructions and psychological support to victims. For example, if the user is injured, it will provide specific voice instructions such as "If there is bleeding, apply pressure with a clean cloth." If the user is feeling anxious or scared, it will provide psychological support such as "Please stay calm. Take a deep breath and relax."

[0881] User Roles

[0882] When an earthquake occurs, users follow the instructions provided by the device to ensure their own safety and, if necessary, request rescue by voice. When the device detects voice, it automatically generates a rescue request message and sends it to the server. Users can also record a final message on the device, which will be automatically sent to designated contacts as soon as the internet connection is restored.

[0883] Specific use cases

[0884] Use case 1: Immediately after an earthquake occurs

[0885] The server detects earthquakes of magnitude 7 or higher and sends alerts to users' devices in the affected area.

[0886] The device will automatically launch the app and enter voice detection mode.

[0887] When a user says "Help!", the device detects the voice and generates and sends a rescue request message.

[0888] Use case 2: When the Internet is down

[0889] The device checks for an Internet connection and, if it determines that it is unavailable, automatically switches to ad-hoc communication mode.

[0890] The device communicates directly with the devices of other victims in the vicinity, sharing rescue request information and location information.

[0891] Use case 3: When first aid is required

[0892] If the user is injured, the device will use edge AI to give voice instructions such as, "If there is bleeding, apply pressure with a clean cloth."

[0893] Edge AI also provides psychological support, such as "Stay calm. Take a deep breath and relax."

[0894] This system allows disaster victims to receive prompt and effective assistance and creates an environment in which local and community cooperation is facilitated.

[0895] The processing flow will be explained below.

[0896] Step 1:

[0897] The server monitors earthquake information.

[0898] Data is regularly obtained from the Japan Meteorological Agency and earthquake early warning services and the seismic intensity is analyzed.

[0899] Step 2:

[0900] If an earthquake with a magnitude of 7 or higher is detected, the server will send an alert to user devices in the relevant area.

[0901] The alert will include information on the earthquake's epicenter and magnitude.

[0902] Step 3:

[0903] The device receives an alert from the server and automatically launches the app.

[0904] When the app is launched, the microphone goes into voice detection mode.

[0905] Step 4:

[0906] The user makes a sound requesting rescue, such as "help me."

[0907] The sound is detected by the device's microphone.

[0908] Step 5:

[0909] The device analyzes the voice and generates a message requesting help.

[0910] The rescue request message will include the user's location and a brief description of the situation.

[0911] Step 6:

[0912] The terminal sends a rescue request message to the server.

[0913] If there is an internet connection, the message is sent immediately to the server.

[0914] Step 7:

[0915] The device checks for internet connectivity.

[0916] If the connection is lost, it will automatically switch to ad-hoc communication mode.

[0917] Step 8:

[0918] In the ad-hoc communication mode, a terminal communicates directly with other terminals in the vicinity.

[0919] Broadcast a rescue request message to other devices and share information.

[0920] Step 9:

[0921] Edge AI runs within the device and analyzes the user's situation.

[0922] If first aid is required, detailed instructions are provided via voice.

[0923] Step 10:

[0924] Edge AI initiates psychological support dialogue.

[0925] Providing users with messages such as "Take a deep breath and stay calm."

[0926] Step 11:

[0927] The device periodically collects the user's location and status information.

[0928] Data is collected based on GPS data and user input.

[0929] Step 12:

[0930] The terminal sends the collected information to the server and stores it in a database.

[0931] The information is continuously updated and used in rescue operations.

[0932] Step 13:

[0933] The user records a final message on the device.

[0934] The recorded message is saved on the device.

[0935] Step 14:

[0936] Once your internet connection is restored, your device will automatically send a recorded message to your designated contacts.

[0937] Your message will be sent to your family and friends.

[0938] Example 1

[0939] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0940] Providing prompt and effective rescue and assistance is crucial when large-scale natural disasters occur, especially earthquakes with a seismic intensity of 7 or higher. However, conventional methods often result in the destruction of communication infrastructure, making it difficult to quickly share information and request rescue. Furthermore, there are limitations on the means by which victims can receive instructions on how to provide appropriate first aid and psychological support. This makes it difficult for conventional systems to provide complete support.

[0941] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0942] In this invention, the server includes means for monitoring seismic intensity information and detecting earthquakes of seismic intensity 7 or higher, means for sending alerts to user terminals based on the seismic intensity information, and means for analyzing meteorological data and identifying users in the affected area. This makes it possible to quickly send alerts to users in the affected area and begin providing support when an earthquake of seismic intensity 7 or higher occurs.

[0943] The user's device also includes means for automatically activating upon receiving the alert, means for detecting the user's voice and generating and transmitting a rescue request message, means for switching to ad-hoc communication mode if the Internet connection is lost, means for analyzing the user's situation based on edge artificial intelligence and generating first aid instructions and psychological support, and means for detecting the voice of the user when the user makes a rescue request and generating the necessary message. This enables victims to quickly request rescue even if infrastructure is destroyed, and to receive appropriate first aid instructions and psychological support.

[0944] "Seismic intensity information" is data that indicates the strength of an earthquake and the degree of shaking.

[0945] "Means for detecting earthquakes of seismic intensity 7 or higher" refers to a device or program that has the function of monitoring earthquake information and detecting the occurrence of an earthquake of seismic intensity 7 or higher.

[0946] "Means for sending an alert to a user's terminal" refers to a device or program that has the function of sending a warning message to a communication device owned by a user based on seismic intensity information.

[0947] The "means for automatically starting up the user's terminal" refers to a device or program that has the function of automatically starting up the user's terminal upon receiving an alert.

[0948] "Means for detecting the user's voice and generating and transmitting a rescue request message" refers to a device or program that has the function of detecting the voice uttered by the user and creating and transmitting a message requesting rescue based on that voice.

[0949] "Means for switching to ad hoc communication mode when Internet connection is lost" refers to a device or program that has the function of automatically changing to a mode for direct communication between terminals when Internet connection is unavailable.

[0950] "Edge AI" is an artificial intelligence technology installed in a terminal, and is a system that has the ability to perform data analysis and decision-making in real time.

[0951] "Means for analyzing the user's situation and providing first aid instructions and psychological support" refers to a device or program that has the function of analyzing the situation of a disaster victim and providing necessary first aid instructions and psychological support via voice or other means.

[0952] "Means for analyzing meteorological data and identifying users in the affected area" refers to a device or program that has the function of analyzing earthquake and other meteorological information and identifying users in the affected area.

[0953] "Means for detecting voices requesting rescue and generating the necessary messages when the voice is uttered" refers to a device or program that has the function of detecting the voices when a user utters a voice requesting rescue and automatically generating a corresponding message.

[0954] This invention is a system for providing prompt and effective rescue and support in the event of a large-scale disaster, such as an earthquake with a seismic intensity of 7 or higher. This system is composed of a server, terminals, and users, each of which plays a specific role.

[0955] Server Features

[0956] The server is responsible for constantly monitoring earthquake information and analyzing meteorological data. The server obtains data in real time from earthquake alert services such as the Japan Meteorological Agency API, and sends an alert to user devices in the relevant area when an earthquake of magnitude 7 or higher occurs. This alert automatically activates the device and triggers the rescue request process. The server also stores and continuously updates the user's location and status information in a database. Specific server product names that can be used are Amazon Web Services (AWS) and Google Cloud Platform (GCP).

[0957] Device Features

[0958] When the device receives an alert from the server, it automatically launches the app and enters voice detection mode. The device detects the user's voice and generates and sends a rescue request message. This message includes the user's current location and a brief description of the situation. If the internet connection is lost, the device automatically switches to ad-hoc communication mode and communicates directly with other devices in the vicinity. In addition, the edge artificial intelligence (Edge AI) installed in the device provides first aid instructions and psychological support to victims. For example, if the user is injured, it will issue voice instructions such as, "If there is bleeding, apply pressure with a clean cloth." If the user is feeling anxious or scared, it will provide psychological support such as, "Please stay calm. Take a deep breath and relax."

[0959] User Roles

[0960] When an earthquake occurs, users follow the instructions provided by the device to ensure their own safety. If necessary, they can request rescue by voice. When the device detects the user's voice, it automatically generates a rescue request message and sends it to the server. Users can also record a final message on the device, which will be automatically sent to the designated contacts once the internet connection is restored.

[0961] Specific use cases

[0962] Use case 1: Immediately after an earthquake occurs

[0963] The server detects earthquakes of magnitude 7 or higher and sends alerts to users' devices in the affected area.

[0964] The device will automatically launch the app and enter voice detection mode.

[0965] When a user says "Help!", the device detects the voice and generates and sends a rescue request message.

[0966] Use case 2: When the Internet is down

[0967] The device checks for an Internet connection and, if it determines that it is unavailable, automatically switches to ad-hoc communication mode.

[0968] The device communicates directly with the devices of other victims in the vicinity, sharing rescue request information and location information.

[0969] Use case 3: When first aid is required

[0970] If the user is injured, the device will use edge AI to give voice instructions such as, "If there is bleeding, apply pressure with a clean cloth."

[0971] Edge AI also provides psychological support, such as "Stay calm. Take a deep breath and relax."

[0972] Prompt Sentence Examples

[0973] By using a generative AI model in this system, it is possible to generate detailed rescue requests and first aid instructions. An example of a prompt sentence to be input to the generative AI model is as follows:

[0974] "There have been many injuries from the earthquake. What should we do if someone is bleeding?"

[0975] "If survivors are feeling anxious, what words of psychological support can you offer?"

[0976] Using this prompt, the AI ​​generates specific instructions and support words that it provides to the user via the device.

[0977] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0978] Step 1:

[0979] The server constantly monitors earthquake information and analyzes meteorological data. Specifically, it obtains data in real time from earthquake early warning services such as the Japan Meteorological Agency API. This data includes information such as seismic intensity, location, and time of occurrence. The server analyzes this data, and if an earthquake with a seismic intensity of 7 or higher occurs, it identifies users in the affected area. Based on the analysis results, it generates a list of users in the affected area.

[0980] Input: Real-time earthquake data obtained from the Japan Meteorological Agency API

[0981] Output: List of users in the affected area

[0982] Step 2:

[0983] The server sends an alert to the user's device based on the seismic intensity information. Specifically, it references a list of users in the affected area and generates an alert message based on this. The message contains earthquake information as well as a message saying, "Please launch the app immediately and follow the instructions." This alert message is sent to each user's device as a push notification.

[0984] Input: User list in affected area, seismic intensity information

[0985] Output: Alert message to the user's terminal

[0986] Step 3:

[0987] When the device receives an alert, it automatically launches the app and enters voice detection mode. Specifically, the application on the device waits in the background and is launched when the alert notification is triggered. When the app launches, the microphone is enabled and it is ready to detect the user's voice.

[0988] Input: Alert message sent from the server

[0989] Output: Device that has entered voice detection state

[0990] Step 4:

[0991] When a user utters a voice requesting rescue, such as "Help!", the device detects this. Specifically, it uses a voice recognition algorithm to recognize specific keywords (e.g., "Help!"). It then analyzes this voice data and generates a rescue request message. The message includes the user's current location (GPS information) and a brief description of the situation (voice-to-text).

[0992] Input: User's voice requesting help

[0993] Output: Help request message

[0994] Step 5:

[0995] The device generates a rescue request message and sends it to the server. If an Internet connection is available, the message is sent immediately to the server. If the Internet connection is lost, the device switches to ad-hoc communication mode and relays the message through other devices.

[0996] Input: Message requesting help

[0997] Output: A help-call message sent to the server or a message relayed via ad-hoc communication.

[0998] Step 6:

[0999] The server analyzes the received rescue request message and forwards the information to the appropriate rescue team. Specifically, it analyzes the content of the rescue request and forwards the message to the appropriate rescue team based on the urgency and required resources. It also generates optimal route information based on the location information and provides it to the rescue team.

[1000] Input: A message requesting help sent by the user

[1001] Output: Message forwarding to rescue team and optimal route information

[1002] Step 7:

[1003] The edge AI inside the device analyzes the user's situation in real time and provides first aid instructions and psychological support. Specifically, it analyzes the user's voice and behavior, and if there is an injury, for example, it will give instructions such as "If there is bleeding, apply pressure with a clean cloth." It also provides psychological support such as advice such as "Remain calm. Take a deep breath and relax."

[1004] Input: User voice and behavioral data

[1005] Output: Audio messages with first aid instructions and psychological support

[1006] (Application example 1)

[1007] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1008] In the event of a large-scale disaster, it is extremely important to carry out rapid and effective rescue operations for victims. However, conventional rescue operations rely on human resources and communication infrastructure, and have faced problems such as internet connection outages and difficulty sharing information in widespread disaster areas. Rescue operations tend to be delayed, especially in disaster-stricken areas outside of urban areas, making ensuring the safety of victims and providing early rescue a challenge. Furthermore, there is a need to simultaneously provide psychological support and first aid instructions to victims.

[1009] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1010] In this invention, the server includes: means for monitoring seismic intensity information and detecting earthquakes of seismic intensity 7 or higher; means for sending an alert to a user's device based on the seismic intensity information; means for automatically activating the user's device upon receiving the alert; means for detecting the user's voice and generating and transmitting a rescue request message; means for switching to ad hoc communication mode if the Internet connection is lost; means for providing first aid instructions and psychological support using edge AI; means for transmitting the user's location information and status information to the server and storing them in a database; means for automatically activating a terminal in the autonomous vehicle or an in-vehicle terminal; means for the terminal in the autonomous vehicle to detect the user's voice and generate and transmit a rescue request message; and means for the terminal in the autonomous vehicle to directly communicate with other autonomous vehicles and terminals using ad hoc communication to share information. This allows victims to quickly request rescue and enables efficient rescue operations using autonomous vehicles. Furthermore, edge AI can be used to provide first aid instructions and psychological support, ensuring the safety and security of victims.

[1011] "Seismic intensity information" is information that indicates the strength of shaking when an earthquake occurs.

[1012] "User devices" refers to smartphones or dedicated devices owned by disaster victims.

[1013] An "alert" is a message that notifies the user when a particular condition occurs.

[1014] "Voice detection" is a technology that detects the voices emitted by the user and recognizes them as specific commands or messages.

[1015] A "rescue request message" is a message that includes information to request rescue of a disaster victim.

[1016] "Ad hoc communication" is a method of communicating directly with other devices when there is no Internet connection.

[1017] "Edge AI" is an artificial intelligence technology that runs on a device and processes data and makes decisions in real time.

[1018] "First Aid Instructions" provide specific instructions for initial response to injuries and emergency situations.

[1019] "Psychological support" refers to providing advice and encouragement to reduce the mental anxiety and stress of disaster victims.

[1020] "Location information" is data that indicates a user's current physical location.

[1021] "Status information" is data about the user's health status and surrounding environment.

[1022] A "server" is a networked computer system that collects and processes data and provides information to other devices.

[1023] A "database" is an electronic storage device for organizing and storing information.

[1024] "Terminal in an autonomous vehicle" refers to a computing device installed in an autonomous vehicle.

[1025] "In-vehicle dedicated terminal" refers to a terminal used exclusively within an autonomous vehicle.

[1026] "Direct communication" is communication between devices without going through the Internet.

[1027] This invention is a system for carrying out rapid and effective rescue operations in the event of a large-scale disaster. The system detects the voices of victims and generates rescue request messages, enables information sharing using ad-hoc communication even when the internet connection is lost, and uses edge AI to provide first aid instructions and psychological support.

[1028] System Overview

[1029] The server monitors seismic intensity information and detects earthquakes of magnitude 7 or higher. When an earthquake occurs, an alert is sent to the user's device in the affected area, causing the device to automatically wake up. When the user's device receives the alert, it enters voice detection mode, detects the user's voice, and generates and sends a rescue request message. If the internet connection is lost, it switches to ad-hoc communication mode, communicating directly with other nearby devices to share information.

[1030] Device operation

[1031] The device automatically wakes up when it receives an alert from the server, detects the user's voice, and generates a rescue request message. The rescue request message includes the user's current location and a brief description of the situation, and is immediately sent to the server if there is an internet connection. In ad-hoc communication mode, information is shared by directly communicating with other nearby devices. In addition, the edge AI provides first aid instructions and psychological support within the device.

[1032] Use of autonomous vehicles

[1033] The system also incorporates a dedicated terminal installed in the autonomous vehicle that automatically activates when an earthquake occurs and detects the user's voice. The dedicated terminal generates a rescue request message and communicates ad hoc with other autonomous vehicles and terminals in the vicinity to share information.

[1034] Hardware and Software Details

[1035] Hardware: Dedicated terminals installed in autonomous vehicles, smartphones carried by users, microphones, and GPS sensors

[1036] Software: VoiceRecognition, GPS module (location information acquisition), EdgeAI (real-time data processing and instructions), AdHocCommunication

[1037] Program processing explanation

[1038] The server constantly monitors seismic intensity information, and if an earthquake of magnitude 7 or higher occurs, it sends an alert to devices in the relevant area. The device then wakes up and switches to voice detection mode to detect the user's voice. After detecting voice, it generates a rescue request message including GPS information and sends it to the server if there is an internet connection, or to another device via ad-hoc communication if there is not. The edge AI runs on the device and provides first aid instructions and psychological support.

[1039] Specific examples

[1040] Example 1: Immediately after an earthquake occurs

[1041] The server detects earthquakes with a magnitude of 7 or higher and sends alerts to users' devices in the affected area.

[1042] The device will automatically launch the app and enter voice detection mode.

[1043] When a user says "Help!", the device detects the voice and generates and sends a rescue request message.

[1044] Example prompt sentence:

[1045] The system automatically activates when an earthquake of magnitude 7 or higher occurs, detects the voice "Help!", and generates and sends a rescue request message. Furthermore, if the internet connection is cut off, the system shares information with other autonomous vehicles using ad-hoc communication.

[1046] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1047] Step 1:

[1048] The server constantly monitors seismic intensity information. As input, it receives real-time seismic intensity data from the Japan Meteorological Agency and seismometers, analyzes that data, and determines whether an earthquake of seismic intensity 7 or higher has occurred. As output, if an earthquake of seismic intensity 7 or higher is detected, it generates a signal to send an alert to user devices in the relevant area.

[1049] Step 2:

[1050] The user's device receives the alert from the server. As an input, it receives the alert signal sent from the server and uses it as a trigger to automatically wake up the device. As an output, it sends information to the device system to switch to voice-sensing mode.

[1051] Step 3:

[1052] The device then enters voice detection mode. The device's built-in microphone collects voice data from the user's surroundings as input, which is then fed into the VoiceRecognition software. The collected voice data is then analyzed and recognized as a rescue request, such as "Help!". The output is the voice recognition results provided to the device's first aid instruction module.

[1053] Step 4:

[1054] The user says "Help!". As input, the user's voice is collected by the device's microphone and analyzed by VoiceRecognition software. As output, if "Help!" is recognized, a signal is generated to generate a rescue request message.

[1055] Step 5:

[1056] The device generates a rescue request message. As input, it combines the voice recognition results with the current location data obtained from the GPS module. As data processing, it integrates the user's location information and the rescue request voice information to generate a detailed rescue request message. As output, it prepares the message to be sent to the server.

[1057] Step 6:

[1058] Checking Internet connection. As input, the terminal checks whether it has an Internet connection. As data processing, it performs an Internet connection test and obtains the results. As output, if there is an Internet connection, it generates a signal to send a rescue request message to the server, and if there is not, it generates a signal to switch to ad-hoc communication mode.

[1059] Step 7:

[1060] Switching to ad-hoc communication mode. As input, it receives information that there is no Internet connection. As data processing, it changes the communication mode of the terminal to ad-hoc mode. As output, it starts direct communication with another terminal and sends a rescue request message.

[1061] Step 8:

[1062] Provision of first aid instructions and psychological support using edge AI. Voice recognition results and user condition information are acquired as input. The edge AI processes the data to generate first aid instructions and psychological support messages appropriate to the situation. The output is to provide instructions and support to the user via voice or text.

[1063] Specific examples

[1064] Example prompt sentence:

[1065] The system automatically activates when an earthquake of magnitude 7 or higher occurs, detects the voice "Help!", and generates and sends a rescue request message. Furthermore, if the internet connection is cut off, the system shares information with other autonomous vehicles using ad-hoc communication.

[1066] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1067] System Overview

[1068] This invention is a system for providing prompt and effective rescue and support in the event of a large-scale disaster such as an earthquake. This system automatically activates when an earthquake of magnitude 7 or higher occurs, detects the voices of victims, and issues a rescue request. Even if the internet connection is cut off, the system can share information using ad-hoc communication, and utilizes edge AI and an emotion engine to provide first aid instructions and psychological support.

[1069] Server Features

[1070] The server monitors earthquake information and sends an alert to user devices in the relevant area when an earthquake of magnitude 7 or higher occurs. The alert automatically activates the user device and triggers the rescue request process. The server stores the user's location, status, and emotional data in a database to help optimize rescue operations.

[1071] Device Features

[1072] When the device receives an alert, it automatically launches the app and enters voice detection mode. When the user utters a voice command such as "help me," the device detects it and generates and sends a rescue request message. If the internet connection is lost, the device switches to ad-hoc communication mode and communicates directly with other nearby devices to share information.

[1073] The Edge AI runs within the device and provides first aid instructions and psychological support to disaster victims. The emotion engine recognizes emotions from the user's voice and facial expressions and adjusts the support provided based on those emotions. For example, if the user expresses strong anxiety or fear, the Edge AI will provide a customized message such as, "Don't worry. Help is on its way."

[1074] User Roles

[1075] When an earthquake occurs, users follow the instructions provided by the device to ensure their own safety and make a voice call to request rescue. When the device detects voice, it automatically generates a rescue request message and sends it to the server. Users can also record a final message on the device, which will be automatically sent to designated contacts when the internet connection is restored.

[1076] Specific use cases

[1077] Use case 1: Immediately after an earthquake occurs

[1078] The server detects earthquakes with a magnitude of 7 or higher and sends alerts to users' devices in the affected area.

[1079] The device will automatically launch the app and enter voice detection mode.

[1080] When a user says "Help!", the device detects the voice and generates and sends a rescue request message.

[1081] Use case 2: When the Internet is down

[1082] The device checks for an Internet connection and, if it determines that it is unavailable, automatically switches to ad-hoc communication mode.

[1083] The device communicates directly with the devices of other victims in the vicinity, sharing rescue request information and location information.

[1084] Use Case 3: Supported by Emotion Engine

[1085] If the user is injured and feels strong anxiety, the device will use edge AI and an emotion engine to give instructions such as "If there is bleeding, apply pressure with a clean cloth" and also provide psychological support such as "Don't worry, help will arrive shortly."

[1086] The emotion engine recognizes emotional data from the user's voice and facial expressions, sends it to the server, and stores it in a database.

[1087] What the program does

[1088] The server constantly monitors earthquake information, and when it detects an earthquake with a seismic intensity of 7 or higher, it sends an alert to the user's device in the relevant area. When the device receives the alert, it automatically launches the app and enters voice detection mode. When the user issues a voice request for rescue, the device analyzes the voice and generates a rescue request message.

[1089] If the internet connection is lost, the device will switch to ad-hoc communication mode and directly communicate with other devices to share information. Edge AI will provide first aid instructions and psychological support, and the emotion engine will analyze the user's emotions and provide appropriate support.

[1090] For example, if a user expresses strong anxiety or fear, the edge AI will generate a message saying, "Don't worry, help is on its way," and the emotion engine will report the situation to the server. This information will be stored in a database and used to optimize rescue operations.

[1091] The processing flow will be explained below.

[1092] Step 1:

[1093] The server monitors earthquake information.

[1094] Data is regularly obtained from the Japan Meteorological Agency and earthquake early warning services and the seismic intensity is analyzed.

[1095] Step 2:

[1096] If an earthquake with a magnitude of 7 or higher is detected, the server will send an alert to user devices in the relevant area.

[1097] The alert will include information on the earthquake's epicenter and magnitude.

[1098] Step 3:

[1099] The device receives an alert from the server and automatically launches the app.

[1100] When the app is launched, it switches to voice detection mode.

[1101] Step 4:

[1102] The user makes a sound requesting rescue, such as "help me."

[1103] The sound is detected by the device's microphone.

[1104] Step 5:

[1105] The device analyzes the voice and generates a message requesting help.

[1106] The rescue request message will include the user's location and a brief description of the situation.

[1107] Step 6:

[1108] The terminal sends a rescue request message to the server.

[1109] If there is an internet connection, the message is sent immediately to the server.

[1110] Step 7:

[1111] The device checks for internet connectivity.

[1112] If the connection is lost, it will automatically switch to ad-hoc communication mode.

[1113] Step 8:

[1114] In the ad-hoc communication mode, a terminal communicates directly with other terminals in the vicinity.

[1115] Broadcast a rescue request message to other devices and share information.

[1116] Step 9:

[1117] Edge AI runs within the device and analyzes the user's situation.

[1118] If first aid is required, detailed instructions are provided via voice.

[1119] Step 10:

[1120] The device's emotion engine analyzes the user's voice and facial expressions to recognize their emotions.

[1121] The recognized emotions are structured as data.

[1122] Step 11:

[1123] Edge AI tailors first aid instructions and psychological support based on emotions recognized by the emotion engine.

[1124] For example, if a user is feeling very anxious, it will generate a message such as "Don't worry, help is on its way."

[1125] Step 12:

[1126] The device transmits the user's emotional information recognized by the emotion engine to the server.

[1127] Emotional information is stored in a database and used to optimize rescue operations.

[1128] Step 13:

[1129] The device periodically collects the user's location and status information.

[1130] Data is collected based on GPS data and user input.

[1131] Step 14:

[1132] The terminal sends the collected information to the server and stores it in a database.

[1133] The information is continuously updated and used in rescue operations.

[1134] Step 15:

[1135] The user records a final message on the device.

[1136] The recorded message is saved on the device.

[1137] Step 16:

[1138] Once your internet connection is restored, your device will automatically send a recorded message to your designated contacts.

[1139] Your message will be sent to your family and friends.

[1140] Example 2

[1141] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1142] In the event of a large-scale disaster such as an earthquake, victims lack the means to quickly and effectively request rescue. There is also a need for a method to efficiently share rescue information even when internet connection is cut off. Furthermore, a support system is needed to reduce the psychological burden on victims and provide appropriate first aid.

[1143] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for monitoring seismic intensity information and detecting earthquakes of seismic intensity 7 or higher, means for sending an alert to a user's terminal based on the seismic intensity information, means for automatically activating the user's terminal that receives the alert, means for detecting the user's voice and generating and sending a rescue request message, means for switching to ad hoc communication mode when the Internet connection is lost, means for providing first aid instructions and psychological support using edge AI, means for sending the user's location information and status information to the server and storing them in a database, and means for analyzing the user's emotions and sending the analysis results to the server. This enables victims to quickly and reliably request rescue, enables information sharing even in situations where the Internet connection is lost, and further reduces the psychological burden on victims and enables appropriate first aid to be provided.

[1144] "Seismic intensity information" is data that indicates the scale of an earthquake and the strength of the vibrations, and includes information such as the location and time of the earthquake and the depth of the epicenter.

[1145] "User device" refers to a mobile information terminal such as a smartphone or tablet, and is the device to which this system sends alerts.

[1146] An "alert" is a warning message that the system sends to the user's device after detecting an earthquake, and serves as a trigger to start the rescue request process.

[1147] "Voice detection" is the process of detecting the user's voice using the device's microphone and analyzing the voice data.

[1148] A "rescue request message" is a message generated based on the user's voice requesting rescue, and is sent to the server along with the user's location information.

[1149] "Ad-hoc communication mode" is a mode in which terminals connect directly to each other and communicate data when an Internet connection is not available.

[1150] "Edge AI" is artificial intelligence that runs within a device and has the ability to provide first aid instructions and psychological support in real time.

[1151] "First aid instructions" are instructions provided by edge AI on the medical procedures that victims should take immediately.

[1152] "Psychological support" refers to messages and actions that edge AI uses to stabilize the mental state of disaster victims and provide them with a sense of security.

[1153] "Location information" is data that includes the user's current latitude and longitude, and is important during rescue operations.

[1154] "Status information" is data related to the user's health condition and the extent of the damage.

[1155] The "database" is a system installed on a server for storing collected earthquake information, user location information, status information, and emotional data.

[1156] "Emotion analysis" is the process of inferring a user's emotions from their voice and facial expressions, and is the basis for providing appropriate assistance.

[1157] "Emotion data" is data that indicates the user's emotional state, and is information that is automatically sent to the server.

[1158] MODE FOR CARRYING OUT THE INVENTION

[1159] This invention is a system for providing prompt and effective rescue and support in the event of a large-scale disaster such as an earthquake. This system automatically activates when an earthquake of magnitude 7 or higher occurs, and can detect the voices of victims and issue rescue requests. Even if the internet connection is cut off, the system can share information using ad-hoc communication and provide first aid instructions and psychological support using edge AI and an emotion engine.

[1160] Hardware and Software Configuration

[1161] server

[1162] The server has the function of constantly monitoring earthquake information. To obtain real-time earthquake data, the server obtains earthquake occurrence information using APIs from the Japan Meteorological Agency and other earthquake information providers. The server is implemented using programming languages ​​such as Python and JavaScript, and uses databases such as PostgreSQL and MySQL.

[1163] Terminal

[1164] The device is a user's smartphone or tablet. When the device receives an alert from the server, it automatically launches the app and goes into voice-activated mode. The device runs an application compatible with the Android or iOS operating system.

[1165] Edge AI

[1166] The AI ​​model runs on the edge device and uses lightweight AI frameworks such as TensorFlow Lite and PyTorch Mobile to analyze the user's voice and facial expressions, and provides first aid instructions and psychological support in real time.

[1167] System Operation

[1168] server

[1169] The server analyzes the earthquake information it receives and sends an alert to user devices in the relevant area if an earthquake with a seismic intensity of 7 or higher occurs. When a user device receives the alert, it automatically launches the app and enters voice detection mode. At this time, the server stores the user's location and status information in a database to help optimize rescue operations.

[1170] Terminal

[1171] When the device receives an alert, it automatically launches the app and enters voice detection mode. When the user utters a word such as "help me," the device's microphone picks up the sound and the voice recognition engine analyzes it. A rescue request message is generated from the analyzed voice and sent to the server.

[1172] If an internet connection is unavailable, the device automatically switches to ad-hoc communication mode, allowing it to communicate directly with other devices in the vicinity and share information such as rescue requests and location information.

[1173] Edge AI and Emotion Engine

[1174] The edge AI provides first aid instructions to the user. For example, it provides voice instructions such as, "If there is bleeding, apply pressure with a clean cloth." The emotion engine also analyzes the user's emotions from their voice and facial expressions, and provides appropriate psychological support based on the analysis results. For example, if the user shows strong anxiety or fear, it can provide a message such as, "Don't worry. Help will arrive shortly."

[1175] Specific examples of operation

[1176] For example, if a user is trapped in a building that has collapsed due to an earthquake and yells "help," the system operates as follows:

[1177] 1. The server detects earthquakes with a magnitude of 7 or higher and sends an alert to user devices in the relevant area.

[1178] 2. The device will receive the alert and automatically launch the app to enter voice detection mode.

[1179] 3. When the user shouts "help," the device's microphone detects the voice and the voice recognition engine analyzes it.

[1180] 4. The device generates a rescue request message and sends it to the server.

[1181] 5. If the Internet connection is lost, the device switches to ad-hoc communication mode and shares information with other devices in the vicinity.

[1182] 6. The edge AI gives instructions such as "If there is bleeding, apply pressure with a clean cloth," and the emotion engine provides psychological support such as "Don't worry, help will arrive soon."

[1183] Prompt Sentence Examples

[1184] "An earthquake of magnitude 7 or higher has occurred. Please explain how the rescue system will respond to this situation from the perspectives of the server, the device, and the user."

[1185] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1186] The flow of this system's program processing

[1187] Step 1:

[1188] The server monitors earthquake information

[1189] The server calls the APIs of the Japan Meteorological Agency and other earthquake information providers to obtain real-time earthquake data. The input includes earthquake information obtained from external APIs, including the location, time, depth of the epicenter, and seismic intensity of the earthquake. The server analyzes this data and detects earthquakes with a seismic intensity of 7 or higher. The output is a flag when an earthquake with a seismic intensity of 7 or higher is detected.

[1190] Step 2:

[1191] The server sends an alert to the user's device in the relevant area.

[1192] The server analyzes the earthquake information and sends an alert message to the devices of users living in areas judged to have a seismic intensity of 7 or higher. The input includes the earthquake information obtained in step 1 and the user's location information in the database. Based on this, the server identifies users in the relevant area and sends an alert to their device. The output is the alert message sent to the user's device.

[1193] Step 3:

[1194] The device receives the alert and automatically launches the app

[1195] The device has a listener for receiving alerts sent from the server. The input is the alert message sent from the server. When the listener receives an alert, the device automatically launches the app. It also enters voice-sensing mode, so the device's microphone is always on. The output is the launch of the app and the transition to voice-sensing mode.

[1196] Step 4:

[1197] The device detects the user's voice and generates a message requesting help.

[1198] When a user utters a voice request for help, such as "help me," the device's microphone picks up the voice. The input is the user's voice. The speech recognition engine analyzes the voice, and a rescue request message is generated as a result of the analysis. The output is the generated rescue request message.

[1199] Step 5:

[1200] The device sends a rescue request message to the server.

[1201] The generated distress request message is sent to the server using the device's normal internet connection. The input is the distress request message generated in step 4. The server receives it and stores it in a database. The output is the distress request message sent to the server.

[1202] Step 6:

[1203] If the Internet connection is lost, the device switches to ad-hoc communication mode.

[1204] The device periodically checks for an Internet connection and automatically switches to ad-hoc communication mode if one is unavailable. The input is the result of checking the Internet connection status. Once in ad-hoc communication mode, it communicates directly with other devices in the vicinity and shares rescue request information and location information. The output is the information shared through communication with other devices.

[1205] Step 7:

[1206] Edge AI provides first aid instructions and psychological support to disaster victims

[1207] Edge AI uses data such as the user's voice and facial expressions to provide appropriate first aid instructions and psychological support. The input is the user's voice, facial expression information, location information, and emotional data. Edge AI analyzes this and provides voice instructions such as "If there is bleeding, apply pressure with a clean cloth" and psychological support such as "Don't worry, help will arrive shortly." The output is the provided first aid instructions and psychological support message.

[1208] Step 8:

[1209] The device uses an emotion engine to analyze the user's emotions.

[1210] The device's emotion engine analyzes emotions from the user's voice and facial expressions. The input is the user's voice and facial expression data. The emotion engine identifies the user's emotion based on this data and generates an analysis result. The output is the analyzed emotion data.

[1211] Step 9:

[1212] The emotion engine sends the analysis results to the server and stores them in a database.

[1213] The analysis results generated by the emotion engine are sent to the server via the Internet. The input is the emotion data generated in step 8. The server receives this and stores it in a database. The output is the emotion data stored in the database.

[1214] (Application example 2)

[1215] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1216] Conventional disaster rescue systems have had problems such as ineffective functioning when the internet connection is cut off or when psychological support for people at the disaster site is lacking. Furthermore, in-store emergency response measures lack the means for a central control unit to quickly and simultaneously transmit information to multiple devices. A system that can solve these problems, provide rescue and support quickly and effectively in the event of a disaster, and ensure the safety of customers and staff in the store, is needed.

[1217] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for monitoring seismic intensity information and detecting earthquakes of seismic intensity 7 or higher, means for sending an alert to a user's device based on the seismic intensity information, means for automatically activating the user's device that receives the alert, means for detecting the user's voice and generating and sending a rescue request message, means for switching to ad hoc communication mode when the Internet connection is lost, means for providing first aid instructions and psychological support using edge AI, means for sending user location information and status information to a central device and storing it in an information storage device, means for providing voice instructions to customers and staff in the store, and means for simultaneously sending alerts to multiple devices in the store based on the seismic intensity information. This makes it possible to share information using ad hoc communication even in situations where the Internet is lost, and to provide quick and appropriate first aid and psychological support to customers and staff in the store.

[1218] "Seismic intensity information" is data indicating the seismic intensity of an earthquake, and is usually provided by seismological observation organizations.

[1219] "Users" refer to customers and staff in physical stores.

[1220] "Terminal" refers to a mobile device such as a smartphone or smart glasses.

[1221] "Internet connection" refers to the communication means by which a terminal accesses the Internet.

[1222] An "alert" is a warning message sent to a device when an earthquake with a seismic intensity of 7 or higher occurs.

[1223] "Ad hoc communication" is a method in which terminals communicate directly with each other when the Internet connection is cut off.

[1224] "Edge AI" refers to artificial intelligence technology that runs on the device itself, allowing for real-time processing without relying on cloud servers.

[1225] "First Aid Instructions" provide appropriate medical guidelines for injured users.

[1226] "Psychological support" refers to encouraging and stabilizing messages provided to help clients achieve their emotional well-being.

[1227] A "central device" is a device that oversees and manages data within a physical store.

[1228] The "information storage device" is a database for storing data collected by the central device.

[1229] "Voice instructions" are instructions given to users and staff via voice.

[1230] A "rescue request message" is a message generated by detecting a voice calling for help made by a user in the event of a disaster.

[1231] "Multiple terminals in store" refers to multiple mobile devices located in a physical store.

[1232] System Overview

[1233] This invention is a system for providing prompt and effective rescue and support in the event of a large-scale disaster, such as an earthquake with a seismic intensity of 7 or higher, occurring within a brick-and-mortar store. This system primarily includes a server, a device equipped with edge AI, and users (customers and staff in the store).

[1234] Server Features

[1235] The server constantly monitors earthquake intensity information and, if it detects an earthquake of magnitude 7 or higher, sends an alert to the devices of users in the relevant area. At this time, the server obtains the intensity information from the meteorological agency's API using AWS IoT. The server also supports an ad-hoc communication mode that can be used even if the internet connection is cut off.

[1236] Device Features

[1237] When a device (such as a smartphone or smart glasses) receives an alert from the server, it automatically wakes up and enters voice-sensing mode. The device is equipped with a TensorFlow-based voice recognition model, which detects when a user speaks a rescue request such as "help me" and generates a rescue request message. The generated message is immediately sent to a central device or an external rescue agency.

[1238] If the device loses internet connection, it switches to ad-hoc communication mode and communicates directly with other devices in the store to share information. This communication uses Bluetooth to transmit data between devices.

[1239] Edge AI runs within the device and provides real-time first aid instructions to disaster victims. Specifically, it presents first aid methods for injuries and gives voice instructions on the necessary actions. Furthermore, the emotion engine recognizes emotions from the user's voice and facial expressions and provides psychological support based on those emotions. For example, if a user is feeling extremely anxious, it will provide a message such as, "Don't worry. Help will arrive shortly."

[1240] User Roles

[1241] When an earthquake occurs, users can follow the instructions provided by the device to ensure their own safety and make a voice call for help. Users can also record a final message on the device, which will be automatically sent to designated contacts when internet connection is restored.

[1242] Specific examples

[1243] Example 1: Immediately after an earthquake occurs

[1244] The server detects earthquakes of magnitude 7 or higher and sends an alert to all terminals in the store.

[1245] The device will automatically start up and enter voice-activated mode.

[1246] When a user shouts "Help!", the device detects the voice, generates a rescue request message, and sends it to the central device.

[1247] Example 2: Internet outage

[1248] The device checks for an Internet connection and, if it determines that it is unavailable, automatically switches to ad-hoc communication mode.

[1249] The device communicates directly with other nearby victims' devices, sharing rescue request information and location information.

[1250] Example 3: Supported by an Emotion Engine

[1251] If a user is injured and at the same time feels strong anxiety, the device's edge AI and emotion engine will instruct them to "apply pressure with a clean cloth if there is bleeding," and also provide psychological support by saying, "Don't worry, help will arrive shortly."

[1252] Example prompts to input to the generative AI model

[1253] "An earthquake of magnitude 7 or higher has occurred. Please move to a safe place. If you need first aid, please follow the steps below. If you are injured, apply pressure with a clean cloth to stop bleeding. We will also provide psychological support. Please rest assured, help will be on its way soon."

[1254] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1255] Step 1:

[1256] The server monitors seismic intensity information. Specifically, it uses AWS IoT Core to obtain real-time earthquake information from the meteorological agency's API. It receives seismic intensity data from the API as input and generates an alert signal as output when an earthquake with a seismic intensity of 7 or higher occurs.

[1257] Step 2:

[1258] When the server detects an earthquake with a seismic intensity of 7 or higher, it sends an alert to multiple users' devices. It receives the alert signal generated in step 1 as input and sends an alert message to all devices using the MQTT protocol as output.

[1259] Step 3:

[1260] When the device receives an alert from the server, it automatically wakes up and enters voice-sensitive mode, taking the alert message from the server as input and arming the voice recognition system as output.

[1261] Step 4:

[1262] The user issues a voice request such as "help me." The device detects the voice and analyzes the voice data using a TensorFlow-based speech recognition model. It receives the voice data as input and generates a text message as output.

[1263] Step 5:

[1264] The terminal transmits the generated rescue request message to a central device or an external rescue organization. The terminal receives a rescue request message in text format as input and transmits the message to a specified rescue organization as output.

[1265] Step 6:

[1266] If the Internet connection is lost, the device automatically switches to ad-hoc communication mode, checking for Internet connectivity as input and using Bluetooth as output to start communicating directly with other devices.

[1267] Step 7:

[1268] Edge AI operates within the device and provides first aid instructions to the user, receiving information about the user's condition as input and providing specific first aid instructions via voice as output.

[1269] Step 8:

[1270] The emotion engine recognizes emotions from the user's voice and facial expressions and provides psychological support. It receives voice and facial expression data as input, and generates a message according to the user's emotional state as output, which is delivered in voice.

[1271] Step 9:

[1272] The terminal sends rescue request messages and user location and status information to the central unit, which stores them in the information storage device. The terminal receives location and status information as input, sends the data to the central unit as output, and stores it in the information storage device.

[1273] Step 10:

[1274] When the internet connection is restored, the device will automatically send the last message recorded by the user to the designated contacts. It takes the recording as input and sends the voice message to the designated contacts as output.

[1275] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1276] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1277] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1278] [Fourth embodiment]

[1279] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1280] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1281] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1282] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1283] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1284] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1285] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1286] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1287] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1288] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1289] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1290] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1291] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1292] System Overview

[1293] This invention is a system for providing rapid and effective rescue and support during large-scale disasters such as earthquakes. This system automatically activates when an earthquake of magnitude 7 or higher occurs, detects the voices of victims, and issues rescue requests. It also enables information sharing using ad-hoc communication even when the internet connection is lost, and uses edge AI to provide first aid instructions and psychological support.

[1294] Server Features

[1295] The server monitors earthquake information and sends an alert to user devices in the relevant area when an earthquake of magnitude 7 or higher occurs. This alert automatically wakes up the user device and triggers the rescue request process. The server also stores and continuously updates the user's location and status information in a database to support rescue efforts.

[1296] Device Features

[1297] The device automatically wakes up when it receives an alert from the server, detects the user's voice, and generates the necessary rescue request message. The rescue request message includes the user's current location and a brief description of the situation, and is immediately sent to the server if there is an Internet connection. Furthermore, if the Internet connection is lost, the device automatically switches to ad-hoc communication mode and shares information by directly communicating with other devices in the vicinity.

[1298] The Edge AI runs on the device and provides first aid instructions and psychological support to victims. For example, if the user is injured, it will provide specific voice instructions such as "If there is bleeding, apply pressure with a clean cloth." If the user is feeling anxious or scared, it will provide psychological support such as "Please stay calm. Take a deep breath and relax."

[1299] User Roles

[1300] When an earthquake occurs, users follow the instructions provided by the device to ensure their own safety and, if necessary, request rescue by voice. When the device detects voice, it automatically generates a rescue request message and sends it to the server. Users can also record a final message on the device, which will be automatically sent to designated contacts as soon as the internet connection is restored.

[1301] Specific use cases

[1302] Use case 1: Immediately after an earthquake occurs

[1303] The server detects earthquakes of magnitude 7 or higher and sends alerts to users' devices in the affected area.

[1304] The device will automatically launch the app and enter voice detection mode.

[1305] When a user says "Help!", the device detects the voice and generates and sends a rescue request message.

[1306] Use case 2: When the Internet is down

[1307] The device checks for an Internet connection and, if it determines that it is unavailable, automatically switches to ad-hoc communication mode.

[1308] The device communicates directly with the devices of other victims in the vicinity, sharing rescue request information and location information.

[1309] Use case 3: When first aid is required

[1310] If the user is injured, the device will use edge AI to give voice instructions such as, "If there is bleeding, apply pressure with a clean cloth."

[1311] Edge AI also provides psychological support, such as "Stay calm. Take a deep breath and relax."

[1312] This system allows disaster victims to receive prompt and effective assistance and creates an environment in which local and community cooperation is facilitated.

[1313] The processing flow will be explained below.

[1314] Step 1:

[1315] The server monitors earthquake information.

[1316] Data is regularly obtained from the Japan Meteorological Agency and earthquake early warning services and the seismic intensity is analyzed.

[1317] Step 2:

[1318] If an earthquake with a magnitude of 7 or higher is detected, the server will send an alert to user devices in the relevant area.

[1319] The alert will include information on the earthquake's epicenter and magnitude.

[1320] Step 3:

[1321] The device receives an alert from the server and automatically launches the app.

[1322] When the app is launched, the microphone goes into voice detection mode.

[1323] Step 4:

[1324] The user makes a sound requesting rescue, such as "help me."

[1325] The sound is detected by the device's microphone.

[1326] Step 5:

[1327] The device analyzes the voice and generates a message requesting help.

[1328] The rescue request message will include the user's location and a brief description of the situation.

[1329] Step 6:

[1330] The terminal sends a rescue request message to the server.

[1331] If there is an internet connection, the message is sent immediately to the server.

[1332] Step 7:

[1333] The device checks for internet connectivity.

[1334] If the connection is lost, it will automatically switch to ad-hoc communication mode.

[1335] Step 8:

[1336] In the ad-hoc communication mode, a terminal communicates directly with other terminals in the vicinity.

[1337] Broadcast a rescue request message to other devices and share information.

[1338] Step 9:

[1339] Edge AI runs within the device and analyzes the user's situation.

[1340] If first aid is required, detailed instructions are provided via voice.

[1341] Step 10:

[1342] Edge AI initiates psychological support dialogue.

[1343] Providing users with messages such as "Take a deep breath and stay calm."

[1344] Step 11:

[1345] The device periodically collects the user's location and status information.

[1346] Data is collected based on GPS data and user input.

[1347] Step 12:

[1348] The terminal sends the collected information to the server and stores it in a database.

[1349] The information is continuously updated and used in rescue operations.

[1350] Step 13:

[1351] The user records a final message on the device.

[1352] The recorded message is saved on the device.

[1353] Step 14:

[1354] Once your internet connection is restored, your device will automatically send a recorded message to your designated contacts.

[1355] Your message will be sent to your family and friends.

[1356] Example 1

[1357] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1358] Providing prompt and effective rescue and assistance is crucial when large-scale natural disasters occur, especially earthquakes with a seismic intensity of 7 or higher. However, conventional methods often result in the destruction of communication infrastructure, making it difficult to quickly share information and request rescue. Furthermore, there are limitations on the means by which victims can receive instructions on how to provide appropriate first aid and psychological support. This makes it difficult for conventional systems to provide complete support.

[1359] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1360] In this invention, the server includes means for monitoring seismic intensity information and detecting earthquakes of seismic intensity 7 or higher, means for sending alerts to user terminals based on the seismic intensity information, and means for analyzing meteorological data and identifying users in the affected area. This makes it possible to quickly send alerts to users in the affected area and begin providing support when an earthquake of seismic intensity 7 or higher occurs.

[1361] The user's device also includes means for automatically activating upon receiving the alert, means for detecting the user's voice and generating and transmitting a rescue request message, means for switching to ad-hoc communication mode if the Internet connection is lost, means for analyzing the user's situation based on edge artificial intelligence and generating first aid instructions and psychological support, and means for detecting the voice of the user when the user makes a rescue request and generating the necessary message. This enables victims to quickly request rescue even if infrastructure is destroyed, and to receive appropriate first aid instructions and psychological support.

[1362] "Seismic intensity information" is data that indicates the strength of an earthquake and the degree of shaking.

[1363] "Means for detecting earthquakes of seismic intensity 7 or higher" refers to a device or program that has the function of monitoring earthquake information and detecting the occurrence of an earthquake of seismic intensity 7 or higher.

[1364] "Means for sending an alert to a user's terminal" refers to a device or program that has the function of sending a warning message to a communication device owned by a user based on seismic intensity information.

[1365] The "means for automatically starting up the user's terminal" refers to a device or program that has the function of automatically starting up the user's terminal upon receiving an alert.

[1366] "Means for detecting the user's voice and generating and transmitting a rescue request message" refers to a device or program that has the function of detecting the voice uttered by the user and creating and transmitting a message requesting rescue based on that voice.

[1367] "Means for switching to ad hoc communication mode when Internet connection is lost" refers to a device or program that has the function of automatically changing to a mode for direct communication between terminals when Internet connection is unavailable.

[1368] "Edge AI" is an artificial intelligence technology installed in a terminal, and is a system that has the ability to perform data analysis and decision-making in real time.

[1369] "Means for analyzing the user's situation and providing first aid instructions and psychological support" refers to a device or program that has the function of analyzing the situation of a disaster victim and providing necessary first aid instructions and psychological support via voice or other means.

[1370] "Means for analyzing meteorological data and identifying users in the affected area" refers to a device or program that has the function of analyzing earthquake and other meteorological information and identifying users in the affected area.

[1371] "Means for detecting voices requesting rescue and generating the necessary messages when the voice is uttered" refers to a device or program that has the function of detecting the voices when a user utters a voice requesting rescue and automatically generating a corresponding message.

[1372] This invention is a system for providing prompt and effective rescue and support in the event of a large-scale disaster, such as an earthquake with a seismic intensity of 7 or higher. This system is composed of a server, terminals, and users, each of which plays a specific role.

[1373] Server Features

[1374] The server is responsible for constantly monitoring earthquake information and analyzing meteorological data. The server obtains data in real time from earthquake alert services such as the Japan Meteorological Agency API, and sends an alert to user devices in the relevant area when an earthquake of magnitude 7 or higher occurs. This alert automatically activates the device and triggers the rescue request process. The server also stores and continuously updates the user's location and status information in a database. Specific server product names that can be used are Amazon Web Services (AWS) and Google Cloud Platform (GCP).

[1375] Device Features

[1376] When the device receives an alert from the server, it automatically launches the app and enters voice detection mode. The device detects the user's voice and generates and sends a rescue request message. This message includes the user's current location and a brief description of the situation. If the internet connection is lost, the device automatically switches to ad-hoc communication mode and communicates directly with other devices in the vicinity. In addition, the edge artificial intelligence (Edge AI) installed in the device provides first aid instructions and psychological support to victims. For example, if the user is injured, it will issue voice instructions such as, "If there is bleeding, apply pressure with a clean cloth." If the user is feeling anxious or scared, it will provide psychological support such as, "Please stay calm. Take a deep breath and relax."

[1377] User Roles

[1378] When an earthquake occurs, users follow the instructions provided by the device to ensure their own safety. If necessary, they can request rescue by voice. When the device detects the user's voice, it automatically generates a rescue request message and sends it to the server. Users can also record a final message on the device, which will be automatically sent to the designated contacts once the internet connection is restored.

[1379] Specific use cases

[1380] Use case 1: Immediately after an earthquake occurs

[1381] The server detects earthquakes of magnitude 7 or higher and sends alerts to users' devices in the affected area.

[1382] The device will automatically launch the app and enter voice detection mode.

[1383] When a user says "Help!", the device detects the voice and generates and sends a rescue request message.

[1384] Use case 2: When the Internet is down

[1385] The device checks for an Internet connection and, if it determines that it is unavailable, automatically switches to ad-hoc communication mode.

[1386] The device communicates directly with the devices of other victims in the vicinity, sharing rescue request information and location information.

[1387] Use case 3: When first aid is required

[1388] If the user is injured, the device will use edge AI to give voice instructions such as, "If there is bleeding, apply pressure with a clean cloth."

[1389] Edge AI also provides psychological support, such as "Stay calm. Take a deep breath and relax."

[1390] Prompt Sentence Examples

[1391] By using a generative AI model in this system, it is possible to generate detailed rescue requests and first aid instructions. An example of a prompt sentence to be input to the generative AI model is as follows:

[1392] "There have been many injuries from the earthquake. What should we do if someone is bleeding?"

[1393] "If survivors are feeling anxious, what words of psychological support can you offer?"

[1394] Using this prompt, the AI ​​generates specific instructions and support words that it provides to the user via the device.

[1395] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1396] Step 1:

[1397] The server constantly monitors earthquake information and analyzes meteorological data. Specifically, it obtains data in real time from earthquake early warning services such as the Japan Meteorological Agency API. This data includes information such as seismic intensity, location, and time of occurrence. The server analyzes this data, and if an earthquake with a seismic intensity of 7 or higher occurs, it identifies users in the affected area. Based on the analysis results, it generates a list of users in the affected area.

[1398] Input: Real-time earthquake data obtained from the Japan Meteorological Agency API

[1399] Output: List of users in the affected area

[1400] Step 2:

[1401] The server sends an alert to the user's device based on the seismic intensity information. Specifically, it references a list of users in the affected area and generates an alert message based on this. The message contains earthquake information as well as a message saying, "Please launch the app immediately and follow the instructions." This alert message is sent to each user's device as a push notification.

[1402] Input: User list in affected area, seismic intensity information

[1403] Output: Alert message to the user's terminal

[1404] Step 3:

[1405] When the device receives an alert, it automatically launches the app and enters voice detection mode. Specifically, the application on the device waits in the background and is launched when the alert notification is triggered. When the app launches, the microphone is enabled and it is ready to detect the user's voice.

[1406] Input: Alert message sent from the server

[1407] Output: Device that has entered voice detection state

[1408] Step 4:

[1409] When a user utters a voice requesting rescue, such as "Help!", the device detects this. Specifically, it uses a voice recognition algorithm to recognize specific keywords (e.g., "Help!"). It then analyzes this voice data and generates a rescue request message. The message includes the user's current location (GPS information) and a brief description of the situation (voice-to-text).

[1410] Input: User's voice requesting help

[1411] Output: Help request message

[1412] Step 5:

[1413] The device generates a rescue request message and sends it to the server. If an Internet connection is available, the message is sent immediately to the server. If the Internet connection is lost, the device switches to ad-hoc communication mode and relays the message through other devices.

[1414] Input: Message requesting help

[1415] Output: A help-call message sent to the server or a message relayed via ad-hoc communication.

[1416] Step 6:

[1417] The server analyzes the received rescue request message and forwards the information to the appropriate rescue team. Specifically, it analyzes the content of the rescue request and forwards the message to the appropriate rescue team based on the urgency and required resources. It also generates optimal route information based on the location information and provides it to the rescue team.

[1418] Input: A message requesting help sent by the user

[1419] Output: Message forwarding to rescue team and optimal route information

[1420] Step 7:

[1421] The edge AI inside the device analyzes the user's situation in real time and provides first aid instructions and psychological support. Specifically, it analyzes the user's voice and behavior, and if there is an injury, for example, it will give instructions such as "If there is bleeding, apply pressure with a clean cloth." It also provides psychological support such as advice such as "Remain calm. Take a deep breath and relax."

[1422] Input: User voice and behavioral data

[1423] Output: Audio messages with first aid instructions and psychological support

[1424] (Application example 1)

[1425] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1426] In the event of a large-scale disaster, it is extremely important to carry out rapid and effective rescue operations for victims. However, conventional rescue operations rely on human resources and communication infrastructure, and have faced problems such as internet connection outages and difficulty sharing information in widespread disaster areas. Rescue operations tend to be delayed, especially in disaster-stricken areas outside of urban areas, making ensuring the safety of victims and providing early rescue a challenge. Furthermore, there is a need to simultaneously provide psychological support and first aid instructions to victims.

[1427] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1428] In this invention, the server includes: means for monitoring seismic intensity information and detecting earthquakes of seismic intensity 7 or higher; means for sending an alert to a user's device based on the seismic intensity information; means for automatically activating the user's device upon receiving the alert; means for detecting the user's voice and generating and transmitting a rescue request message; means for switching to ad hoc communication mode if the Internet connection is lost; means for providing first aid instructions and psychological support using edge AI; means for transmitting the user's location information and status information to the server and storing them in a database; means for automatically activating a terminal in the autonomous vehicle or an in-vehicle terminal; means for the terminal in the autonomous vehicle to detect the user's voice and generate and transmit a rescue request message; and means for the terminal in the autonomous vehicle to directly communicate with other autonomous vehicles and terminals using ad hoc communication to share information. This allows victims to quickly request rescue and enables efficient rescue operations using autonomous vehicles. Furthermore, edge AI can be used to provide first aid instructions and psychological support, ensuring the safety and security of victims.

[1429] "Seismic intensity information" is information that indicates the strength of shaking when an earthquake occurs.

[1430] "User devices" refers to smartphones or dedicated devices owned by disaster victims.

[1431] An "alert" is a message that notifies the user when a particular condition occurs.

[1432] "Voice detection" is a technology that detects the voices emitted by the user and recognizes them as specific commands or messages.

[1433] A "rescue request message" is a message that includes information to request rescue of a disaster victim.

[1434] "Ad hoc communication" is a method of communicating directly with other devices when there is no Internet connection.

[1435] "Edge AI" is an artificial intelligence technology that runs on a device and processes data and makes decisions in real time.

[1436] "First Aid Instructions" provide specific instructions for initial response to injuries and emergency situations.

[1437] "Psychological support" refers to providing advice and encouragement to reduce the mental anxiety and stress of disaster victims.

[1438] "Location information" is data that indicates a user's current physical location.

[1439] "Status information" is data about the user's health status and surrounding environment.

[1440] A "server" is a networked computer system that collects and processes data and provides information to other devices.

[1441] A "database" is an electronic storage device for organizing and storing information.

[1442] "Terminal in an autonomous vehicle" refers to a computing device installed in an autonomous vehicle.

[1443] "In-vehicle dedicated terminal" refers to a terminal used exclusively within an autonomous vehicle.

[1444] "Direct communication" is communication between devices without going through the Internet.

[1445] This invention is a system for carrying out rapid and effective rescue operations in the event of a large-scale disaster. The system detects the voices of victims and generates rescue request messages, enables information sharing using ad-hoc communication even when the internet connection is lost, and uses edge AI to provide first aid instructions and psychological support.

[1446] System Overview

[1447] The server monitors seismic intensity information and detects earthquakes of magnitude 7 or higher. When an earthquake occurs, an alert is sent to the user's device in the affected area, causing the device to automatically wake up. When the user's device receives the alert, it enters voice detection mode, detects the user's voice, and generates and sends a rescue request message. If the internet connection is lost, it switches to ad-hoc communication mode, communicating directly with other nearby devices to share information.

[1448] Device operation

[1449] The device automatically wakes up when it receives an alert from the server, detects the user's voice, and generates a rescue request message. The rescue request message includes the user's current location and a brief description of the situation, and is immediately sent to the server if there is an internet connection. In ad-hoc communication mode, information is shared by directly communicating with other nearby devices. In addition, the edge AI provides first aid instructions and psychological support within the device.

[1450] Use of autonomous vehicles

[1451] The system also incorporates a dedicated terminal installed in the autonomous vehicle that automatically activates when an earthquake occurs and detects the user's voice. The dedicated terminal generates a rescue request message and communicates ad hoc with other autonomous vehicles and terminals in the vicinity to share information.

[1452] Hardware and Software Details

[1453] Hardware: Dedicated terminals installed in autonomous vehicles, smartphones carried by users, microphones, and GPS sensors

[1454] Software: VoiceRecognition, GPS module (location information acquisition), EdgeAI (real-time data processing and instructions), AdHocCommunication

[1455] Program processing explanation

[1456] The server constantly monitors seismic intensity information, and if an earthquake of magnitude 7 or higher occurs, it sends an alert to devices in the relevant area. The device then wakes up and switches to voice detection mode to detect the user's voice. After detecting voice, it generates a rescue request message including GPS information and sends it to the server if there is an internet connection, or to another device via ad-hoc communication if there is not. The edge AI runs on the device and provides first aid instructions and psychological support.

[1457] Specific examples

[1458] Example 1: Immediately after an earthquake occurs

[1459] The server detects earthquakes with a magnitude of 7 or higher and sends alerts to users' devices in the affected area.

[1460] The device will automatically launch the app and enter voice detection mode.

[1461] When a user says "Help!", the device detects the voice and generates and sends a rescue request message.

[1462] Example prompt sentence:

[1463] The system automatically activates when an earthquake of magnitude 7 or higher occurs, detects the voice "Help!", and generates and sends a rescue request message. Furthermore, if the internet connection is cut off, the system shares information with other autonomous vehicles using ad-hoc communication.

[1464] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1465] Step 1:

[1466] The server constantly monitors seismic intensity information. As input, it receives real-time seismic intensity data from the Japan Meteorological Agency and seismometers, analyzes that data, and determines whether an earthquake of seismic intensity 7 or higher has occurred. As output, if an earthquake of seismic intensity 7 or higher is detected, it generates a signal to send an alert to user devices in the relevant area.

[1467] Step 2:

[1468] The user's device receives the alert from the server. As an input, it receives the alert signal sent from the server and uses it as a trigger to automatically wake up the device. As an output, it sends information to the device system to switch to voice-sensing mode.

[1469] Step 3:

[1470] The device then enters voice detection mode. The device's built-in microphone collects voice data from the user's surroundings as input, which is then fed into the VoiceRecognition software. The collected voice data is then analyzed and recognized as a rescue request, such as "Help!". The output is the voice recognition results provided to the device's first aid instruction module.

[1471] Step 4:

[1472] The user says "Help!". As input, the user's voice is collected by the device's microphone and analyzed by VoiceRecognition software. As output, if "Help!" is recognized, a signal is generated to generate a rescue request message.

[1473] Step 5:

[1474] The device generates a rescue request message. As input, it combines the voice recognition results with the current location data obtained from the GPS module. As data processing, it integrates the user's location information and the rescue request voice information to generate a detailed rescue request message. As output, it prepares the message to be sent to the server.

[1475] Step 6:

[1476] Checking Internet connection. As input, the terminal checks whether it has an Internet connection. As data processing, it performs an Internet connection test and obtains the results. As output, if there is an Internet connection, it generates a signal to send a rescue request message to the server, and if there is not, it generates a signal to switch to ad-hoc communication mode.

[1477] Step 7:

[1478] Switching to ad-hoc communication mode. As input, it receives information that there is no Internet connection. As data processing, it changes the communication mode of the terminal to ad-hoc mode. As output, it starts direct communication with another terminal and sends a rescue request message.

[1479] Step 8:

[1480] Provision of first aid instructions and psychological support using edge AI. Voice recognition results and user condition information are acquired as input. The edge AI processes the data to generate first aid instructions and psychological support messages appropriate to the situation. The output is to provide instructions and support to the user via voice or text.

[1481] Specific examples

[1482] Example prompt sentence:

[1483] The system automatically activates when an earthquake of magnitude 7 or higher occurs, detects the voice "Help!", and generates and sends a rescue request message. Furthermore, if the internet connection is cut off, the system shares information with other autonomous vehicles using ad-hoc communication.

[1484] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1485] System Overview

[1486] This invention is a system for providing prompt and effective rescue and support in the event of a large-scale disaster such as an earthquake. This system automatically activates when an earthquake of magnitude 7 or higher occurs, detects the voices of victims, and issues a rescue request. Even if the internet connection is cut off, the system can share information using ad-hoc communication, and utilizes edge AI and an emotion engine to provide first aid instructions and psychological support.

[1487] Server Features

[1488] The server monitors earthquake information and sends an alert to user devices in the relevant area when an earthquake of magnitude 7 or higher occurs. The alert automatically activates the user device and triggers the rescue request process. The server stores the user's location, status, and emotional data in a database to help optimize rescue operations.

[1489] Device Features

[1490] When the device receives an alert, it automatically launches the app and enters voice detection mode. When the user utters a voice command such as "help me," the device detects it and generates and sends a rescue request message. If the internet connection is lost, the device switches to ad-hoc communication mode and communicates directly with other nearby devices to share information.

[1491] The Edge AI runs within the device and provides first aid instructions and psychological support to disaster victims. The emotion engine recognizes emotions from the user's voice and facial expressions and adjusts the support provided based on those emotions. For example, if the user expresses strong anxiety or fear, the Edge AI will provide a customized message such as, "Don't worry. Help is on its way."

[1492] User Roles

[1493] When an earthquake occurs, users follow the instructions provided by the device to ensure their own safety and make a voice call to request rescue. When the device detects voice, it automatically generates a rescue request message and sends it to the server. Users can also record a final message on the device, which will be automatically sent to designated contacts when the internet connection is restored.

[1494] Specific use cases

[1495] Use case 1: Immediately after an earthquake occurs

[1496] The server detects earthquakes with a magnitude of 7 or higher and sends alerts to users' devices in the affected area.

[1497] The device will automatically launch the app and enter voice detection mode.

[1498] When a user says "Help!", the device detects the voice and generates and sends a rescue request message.

[1499] Use case 2: When the Internet is down

[1500] The device checks for an Internet connection and, if it determines that it is unavailable, automatically switches to ad-hoc communication mode.

[1501] The device communicates directly with the devices of other victims in the vicinity, sharing rescue request information and location information.

[1502] Use Case 3: Supported by Emotion Engine

[1503] If the user is injured and feels strong anxiety, the device will use edge AI and an emotion engine to give instructions such as "If there is bleeding, apply pressure with a clean cloth" and also provide psychological support such as "Don't worry, help will arrive shortly."

[1504] The emotion engine recognizes emotional data from the user's voice and facial expressions, sends it to the server, and stores it in a database.

[1505] What the program does

[1506] The server constantly monitors earthquake information, and when it detects an earthquake with a seismic intensity of 7 or higher, it sends an alert to the user's device in the relevant area. When the device receives the alert, it automatically launches the app and enters voice detection mode. When the user issues a voice request for rescue, the device analyzes the voice and generates a rescue request message.

[1507] If the internet connection is lost, the device will switch to ad-hoc communication mode and directly communicate with other devices to share information. Edge AI will provide first aid instructions and psychological support, and the emotion engine will analyze the user's emotions and provide appropriate support.

[1508] For example, if a user expresses strong anxiety or fear, the edge AI will generate a message saying, "Don't worry, help is on its way," and the emotion engine will report the situation to the server. This information will be stored in a database and used to optimize rescue operations.

[1509] The processing flow will be explained below.

[1510] Step 1:

[1511] The server monitors earthquake information.

[1512] Data is regularly obtained from the Japan Meteorological Agency and earthquake early warning services and the seismic intensity is analyzed.

[1513] Step 2:

[1514] If an earthquake with a magnitude of 7 or higher is detected, the server will send an alert to user devices in the relevant area.

[1515] The alert will include information on the earthquake's epicenter and magnitude.

[1516] Step 3:

[1517] The device receives an alert from the server and automatically launches the app.

[1518] When the app is launched, it switches to voice detection mode.

[1519] Step 4:

[1520] The user makes a sound requesting rescue, such as "help me."

[1521] The sound is detected by the device's microphone.

[1522] Step 5:

[1523] The device analyzes the voice and generates a message requesting help.

[1524] The rescue request message will include the user's location and a brief description of the situation.

[1525] Step 6:

[1526] The terminal sends a rescue request message to the server.

[1527] If there is an internet connection, the message is sent immediately to the server.

[1528] Step 7:

[1529] The device checks for internet connectivity.

[1530] If the connection is lost, it will automatically switch to ad-hoc communication mode.

[1531] Step 8:

[1532] In the ad-hoc communication mode, a terminal communicates directly with other terminals in the vicinity.

[1533] Broadcast a rescue request message to other devices and share information.

[1534] Step 9:

[1535] Edge AI runs within the device and analyzes the user's situation.

[1536] If first aid is required, detailed instructions are provided via voice.

[1537] Step 10:

[1538] The device's emotion engine analyzes the user's voice and facial expressions to recognize their emotions.

[1539] The recognized emotions are structured as data.

[1540] Step 11:

[1541] Edge AI tailors first aid instructions and psychological support based on emotions recognized by the emotion engine.

[1542] For example, if a user is feeling very anxious, it will generate a message such as "Don't worry, help is on its way."

[1543] Step 12:

[1544] The device transmits the user's emotional information recognized by the emotion engine to the server.

[1545] Emotional information is stored in a database and used to optimize rescue operations.

[1546] Step 13:

[1547] The device periodically collects the user's location and status information.

[1548] Data is collected based on GPS data and user input.

[1549] Step 14:

[1550] The terminal sends the collected information to the server and stores it in a database.

[1551] The information is continuously updated and used in rescue operations.

[1552] Step 15:

[1553] The user records a final message on the device.

[1554] The recorded message is saved on the device.

[1555] Step 16:

[1556] Once your internet connection is restored, your device will automatically send a recorded message to your designated contacts.

[1557] Your message will be sent to your family and friends.

[1558] Example 2

[1559] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1560] In the event of a large-scale disaster such as an earthquake, victims lack the means to quickly and effectively request rescue. There is also a need for a method to efficiently share rescue information even when internet connection is cut off. Furthermore, a support system is needed to reduce the psychological burden on victims and provide appropriate first aid.

[1561] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for monitoring seismic intensity information and detecting earthquakes of seismic intensity 7 or higher, means for sending an alert to a user's terminal based on the seismic intensity information, means for automatically activating the user's terminal that receives the alert, means for detecting the user's voice and generating and sending a rescue request message, means for switching to ad hoc communication mode when the Internet connection is lost, means for providing first aid instructions and psychological support using edge AI, means for sending the user's location information and status information to the server and storing them in a database, and means for analyzing the user's emotions and sending the analysis results to the server. This enables victims to quickly and reliably request rescue, enables information sharing even in situations where the Internet connection is lost, and further reduces the psychological burden on victims and enables appropriate first aid to be provided.

[1562] "Seismic intensity information" is data that indicates the scale of an earthquake and the strength of the vibrations, and includes information such as the location and time of the earthquake and the depth of the epicenter.

[1563] "User device" refers to a mobile information terminal such as a smartphone or tablet, and is the device to which this system sends alerts.

[1564] An "alert" is a warning message that the system sends to the user's device after detecting an earthquake, and serves as a trigger to start the rescue request process.

[1565] "Voice detection" is the process of detecting the user's voice using the device's microphone and analyzing the voice data.

[1566] A "rescue request message" is a message generated based on the user's voice requesting rescue, and is sent to the server along with the user's location information.

[1567] "Ad-hoc communication mode" is a mode in which terminals connect directly to each other and communicate data when an Internet connection is not available.

[1568] "Edge AI" is artificial intelligence that runs within a device and has the ability to provide first aid instructions and psychological support in real time.

[1569] "First aid instructions" are instructions provided by edge AI on the medical procedures that victims should take immediately.

[1570] "Psychological support" refers to messages and actions that edge AI uses to stabilize the mental state of disaster victims and provide them with a sense of security.

[1571] "Location information" is data that includes the user's current latitude and longitude, and is important during rescue operations.

[1572] "Status information" is data related to the user's health condition and the extent of the damage.

[1573] The "database" is a system installed on a server for storing collected earthquake information, user location information, status information, and emotional data.

[1574] "Emotion analysis" is the process of inferring a user's emotions from their voice and facial expressions, and is the basis for providing appropriate assistance.

[1575] "Emotion data" is data that indicates the user's emotional state, and is information that is automatically sent to the server.

[1576] MODE FOR CARRYING OUT THE INVENTION

[1577] This invention is a system for providing prompt and effective rescue and support in the event of a large-scale disaster such as an earthquake. This system automatically activates when an earthquake of magnitude 7 or higher occurs, and can detect the voices of victims and issue rescue requests. Even if the internet connection is cut off, the system can share information using ad-hoc communication and provide first aid instructions and psychological support using edge AI and an emotion engine.

[1578] Hardware and Software Configuration

[1579] server

[1580] The server has the function of constantly monitoring earthquake information. To obtain real-time earthquake data, the server obtains earthquake occurrence information using APIs from the Japan Meteorological Agency and other earthquake information providers. The server is implemented using programming languages ​​such as Python and JavaScript, and uses databases such as PostgreSQL and MySQL.

[1581] Terminal

[1582] The device is a user's smartphone or tablet. When the device receives an alert from the server, it automatically launches the app and goes into voice-activated mode. The device runs an application compatible with the Android or iOS operating system.

[1583] Edge AI

[1584] The AI ​​model runs on the edge device and uses lightweight AI frameworks such as TensorFlow Lite and PyTorch Mobile to analyze the user's voice and facial expressions, and provides first aid instructions and psychological support in real time.

[1585] System Operation

[1586] server

[1587] The server analyzes the earthquake information it receives and sends an alert to user devices in the relevant area if an earthquake with a seismic intensity of 7 or higher occurs. When a user device receives the alert, it automatically launches the app and enters voice detection mode. At this time, the server stores the user's location and status information in a database to help optimize rescue operations.

[1588] Terminal

[1589] When the device receives an alert, it automatically launches the app and enters voice detection mode. When the user utters a word such as "help me," the device's microphone picks up the sound and the voice recognition engine analyzes it. A rescue request message is generated from the analyzed voice and sent to the server.

[1590] If an internet connection is unavailable, the device automatically switches to ad-hoc communication mode, allowing it to communicate directly with other devices in the vicinity and share information such as rescue requests and location information.

[1591] Edge AI and Emotion Engine

[1592] The edge AI provides first aid instructions to the user. For example, it provides voice instructions such as, "If there is bleeding, apply pressure with a clean cloth." The emotion engine also analyzes the user's emotions from their voice and facial expressions, and provides appropriate psychological support based on the analysis results. For example, if the user shows strong anxiety or fear, it can provide a message such as, "Don't worry. Help will arrive shortly."

[1593] Specific examples of operation

[1594] For example, if a user is trapped in a building that has collapsed due to an earthquake and yells "help," the system operates as follows:

[1595] 1. The server detects earthquakes with a magnitude of 7 or higher and sends an alert to user devices in the relevant area.

[1596] 2. The device will receive the alert and automatically launch the app to enter voice detection mode.

[1597] 3. When the user shouts "help," the device's microphone detects the voice and the voice recognition engine analyzes it.

[1598] 4. The device generates a rescue request message and sends it to the server.

[1599] 5. If the Internet connection is lost, the device switches to ad-hoc communication mode and shares information with other devices in the vicinity.

[1600] 6. The edge AI gives instructions such as "If there is bleeding, apply pressure with a clean cloth," and the emotion engine provides psychological support such as "Don't worry, help will arrive soon."

[1601] Prompt Sentence Examples

[1602] "An earthquake of magnitude 7 or higher has occurred. Please explain how the rescue system will respond to this situation from the perspectives of the server, the device, and the user."

[1603] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1604] The flow of this system's program processing

[1605] Step 1:

[1606] The server monitors earthquake information

[1607] The server calls the APIs of the Japan Meteorological Agency and other earthquake information providers to obtain real-time earthquake data. The input includes earthquake information obtained from external APIs, including the location, time, depth of the epicenter, and seismic intensity of the earthquake. The server analyzes this data and detects earthquakes with a seismic intensity of 7 or higher. The output is a flag when an earthquake with a seismic intensity of 7 or higher is detected.

[1608] Step 2:

[1609] The server sends an alert to the user's device in the relevant area.

[1610] The server analyzes the earthquake information and sends an alert message to the devices of users living in areas judged to have a seismic intensity of 7 or higher. The input includes the earthquake information obtained in step 1 and the user's location information in the database. Based on this, the server identifies users in the relevant area and sends an alert to their device. The output is the alert message sent to the user's device.

[1611] Step 3:

[1612] The device receives the alert and automatically launches the app

[1613] The device has a listener for receiving alerts sent from the server. The input is the alert message sent from the server. When the listener receives an alert, the device automatically launches the app. It also enters voice-sensing mode, so the device's microphone is always on. The output is the launch of the app and the transition to voice-sensing mode.

[1614] Step 4:

[1615] The device detects the user's voice and generates a message requesting help.

[1616] When a user utters a voice request for help, such as "help me," the device's microphone picks up the voice. The input is the user's voice. The speech recognition engine analyzes the voice, and a rescue request message is generated as a result of the analysis. The output is the generated rescue request message.

[1617] Step 5:

[1618] The device sends a rescue request message to the server.

[1619] The generated distress request message is sent to the server using the device's normal internet connection. The input is the distress request message generated in step 4. The server receives it and stores it in a database. The output is the distress request message sent to the server.

[1620] Step 6:

[1621] If the Internet connection is lost, the device switches to ad-hoc communication mode.

[1622] The device periodically checks for an Internet connection and automatically switches to ad-hoc communication mode if one is unavailable. The input is the result of checking the Internet connection status. Once in ad-hoc communication mode, it communicates directly with other devices in the vicinity and shares rescue request information and location information. The output is the information shared through communication with other devices.

[1623] Step 7:

[1624] Edge AI provides first aid instructions and psychological support to disaster victims

[1625] Edge AI uses data such as the user's voice and facial expressions to provide appropriate first aid instructions and psychological support. The input is the user's voice, facial expression information, location information, and emotional data. Edge AI analyzes this and provides voice instructions such as "If there is bleeding, apply pressure with a clean cloth" and psychological support such as "Don't worry, help will arrive shortly." The output is the provided first aid instructions and psychological support message.

[1626] Step 8:

[1627] The device uses an emotion engine to analyze the user's emotions.

[1628] The device's emotion engine analyzes emotions from the user's voice and facial expressions. The input is the user's voice and facial expression data. The emotion engine identifies the user's emotion based on this data and generates an analysis result. The output is the analyzed emotion data.

[1629] Step 9:

[1630] The emotion engine sends the analysis results to the server and stores them in a database.

[1631] The analysis results generated by the emotion engine are sent to the server via the Internet. The input is the emotion data generated in step 8. The server receives this and stores it in a database. The output is the emotion data stored in the database.

[1632] (Application example 2)

[1633] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1634] Conventional disaster rescue systems have had problems such as ineffective functioning when the internet connection is cut off or when psychological support for people at the disaster site is lacking. Furthermore, in-store emergency response measures lack the means for a central control unit to quickly and simultaneously transmit information to multiple devices. A system that can solve these problems, provide rescue and support quickly and effectively in the event of a disaster, and ensure the safety of customers and staff in the store, is needed.

[1635] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for monitoring seismic intensity information and detecting earthquakes of seismic intensity 7 or higher, means for sending an alert to a user's device based on the seismic intensity information, means for automatically activating the user's device that receives the alert, means for detecting the user's voice and generating and sending a rescue request message, means for switching to ad hoc communication mode when the Internet connection is lost, means for providing first aid instructions and psychological support using edge AI, means for sending user location information and status information to a central device and storing it in an information storage device, means for providing voice instructions to customers and staff in the store, and means for simultaneously sending alerts to multiple devices in the store based on the seismic intensity information. This makes it possible to share information using ad hoc communication even in situations where the Internet is lost, and to provide quick and appropriate first aid and psychological support to customers and staff in the store.

[1636] "Seismic intensity information" is data indicating the seismic intensity of an earthquake, and is usually provided by seismological observation organizations.

[1637] "Users" refer to customers and staff in physical stores.

[1638] "Terminal" refers to a mobile device such as a smartphone or smart glasses.

[1639] "Internet connection" refers to the communication means by which a terminal accesses the Internet.

[1640] An "alert" is a warning message sent to a device when an earthquake with a seismic intensity of 7 or higher occurs.

[1641] "Ad hoc communication" is a method in which terminals communicate directly with each other when the Internet connection is cut off.

[1642] "Edge AI" refers to artificial intelligence technology that runs on the device itself, allowing for real-time processing without relying on cloud servers.

[1643] "First Aid Instructions" provide appropriate medical guidelines for injured users.

[1644] "Psychological support" refers to encouraging and stabilizing messages provided to help clients achieve their emotional well-being.

[1645] A "central device" is a device that oversees and manages data within a physical store.

[1646] The "information storage device" is a database for storing data collected by the central device.

[1647] "Voice instructions" are instructions given to users and staff via voice.

[1648] A "rescue request message" is a message generated by detecting a voice calling for help made by a user in the event of a disaster.

[1649] "Multiple terminals in store" refers to multiple mobile devices located in a physical store.

[1650] System Overview

[1651] This invention is a system for providing prompt and effective rescue and support in the event of a large-scale disaster, such as an earthquake with a seismic intensity of 7 or higher, occurring within a brick-and-mortar store. This system primarily includes a server, a device equipped with edge AI, and users (customers and staff in the store).

[1652] Server Features

[1653] The server constantly monitors earthquake intensity information and, if it detects an earthquake of magnitude 7 or higher, sends an alert to the devices of users in the relevant area. At this time, the server obtains the intensity information from the meteorological agency's API using AWS IoT. The server also supports an ad-hoc communication mode that can be used even if the internet connection is cut off.

[1654] Device Features

[1655] When a device (such as a smartphone or smart glasses) receives an alert from the server, it automatically wakes up and enters voice-sensing mode. The device is equipped with a TensorFlow-based voice recognition model, which detects when a user speaks a rescue request such as "help me" and generates a rescue request message. The generated message is immediately sent to a central device or an external rescue agency.

[1656] If the device loses internet connection, it switches to ad-hoc communication mode and communicates directly with other devices in the store to share information. This communication uses Bluetooth to transmit data between devices.

[1657] Edge AI runs within the device and provides real-time first aid instructions to disaster victims. Specifically, it presents first aid methods for injuries and gives voice instructions on the necessary actions. Furthermore, the emotion engine recognizes emotions from the user's voice and facial expressions and provides psychological support based on those emotions. For example, if a user is feeling extremely anxious, it will provide a message such as, "Don't worry. Help will arrive shortly."

[1658] User Roles

[1659] When an earthquake occurs, users can follow the instructions provided by the device to ensure their own safety and make a voice call for help. Users can also record a final message on the device, which will be automatically sent to designated contacts when internet connection is restored.

[1660] Specific examples

[1661] Example 1: Immediately after an earthquake occurs

[1662] The server detects earthquakes of magnitude 7 or higher and sends an alert to all terminals in the store.

[1663] The device will automatically start up and enter voice-activated mode.

[1664] When a user shouts "Help!", the device detects the voice, generates a rescue request message, and sends it to the central device.

[1665] Example 2: Internet outage

[1666] The device checks for an Internet connection and, if it determines that it is unavailable, automatically switches to ad-hoc communication mode.

[1667] The device communicates directly with other nearby victims' devices, sharing rescue request information and location information.

[1668] Example 3: Supported by an Emotion Engine

[1669] If a user is injured and at the same time feels strong anxiety, the device's edge AI and emotion engine will instruct them to "apply pressure with a clean cloth if there is bleeding," and also provide psychological support by saying, "Don't worry, help will arrive shortly."

[1670] Example prompts to input to the generative AI model

[1671] "An earthquake of magnitude 7 or higher has occurred. Please move to a safe place. If you need first aid, please follow the steps below. If you are injured, apply pressure with a clean cloth to stop bleeding. We will also provide psychological support. Please rest assured, help will be on its way soon."

[1672] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1673] Step 1:

[1674] The server monitors seismic intensity information. Specifically, it uses AWS IoT Core to obtain real-time earthquake information from the meteorological agency's API. It receives seismic intensity data from the API as input and generates an alert signal as output when an earthquake with a seismic intensity of 7 or higher occurs.

[1675] Step 2:

[1676] When the server detects an earthquake with a seismic intensity of 7 or higher, it sends an alert to multiple users' devices. It receives the alert signal generated in step 1 as input and sends an alert message to all devices using the MQTT protocol as output.

[1677] Step 3:

[1678] When the device receives an alert from the server, it automatically wakes up and enters voice-sensitive mode, taking the alert message from the server as input and arming the voice recognition system as output.

[1679] Step 4:

[1680] The user issues a voice request such as "help me." The device detects the voice and analyzes the voice data using a TensorFlow-based speech recognition model. It receives the voice data as input and generates a text message as output.

[1681] Step 5:

[1682] The terminal transmits the generated rescue request message to a central device or an external rescue organization. The terminal receives a rescue request message in text format as input and transmits the message to a specified rescue organization as output.

[1683] Step 6:

[1684] If the Internet connection is lost, the device automatically switches to ad-hoc communication mode, checking for Internet connectivity as input and using Bluetooth as output to start communicating directly with other devices.

[1685] Step 7:

[1686] Edge AI operates within the device and provides first aid instructions to the user, receiving information about the user's condition as input and providing specific first aid instructions via voice as output.

[1687] Step 8:

[1688] The emotion engine recognizes emotions from the user's voice and facial expressions and provides psychological support. It receives voice and facial expression data as input, and generates a message according to the user's emotional state as output, which is delivered in voice.

[1689] Step 9:

[1690] The terminal sends rescue request messages and user location and status information to the central unit, which stores them in the information storage device. The terminal receives location and status information as input, sends the data to the central unit as output, and stores it in the information storage device.

[1691] Step 10:

[1692] When the internet connection is restored, the device will automatically send the last message recorded by the user to the designated contacts. It takes the recording as input and sends the voice message to the designated contacts as output.

[1693] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1694] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1695] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1696] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1697] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1698] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1699] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1700] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1701] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1702] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1703] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1704] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1705] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1706] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1707] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1708] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1709] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1710] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1711] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1712] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1713] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1714] The following is further disclosed regarding the above embodiment.

[1715] (Claim 1)

[1716] A means for monitoring seismic intensity information and detecting earthquakes of seismic intensity 7 or higher;

[1717] means for sending an alert to a user's terminal based on the seismic intensity information;

[1718] means for automatically starting up a terminal of a user who receives the alert;

[1719] means for detecting a user's voice and generating and transmitting a help request message;

[1720] A means for switching to ad-hoc communication mode if the Internet connection is lost;

[1721] A means of providing first aid instructions and psychological support using edge AI;

[1722] A means for transmitting user location and status information to a server and storing it in a database.

[1723] A system including:

[1724] (Claim 2)

[1725] 10. The system of claim 1, further comprising means for allowing the user to record a final message that is automatically sent to family members or designated contacts when internet connectivity is restored.

[1726] (Claim 3)

[1727] 2. The system according to claim 1, further comprising means for directly communicating with other terminals using ad-hoc communication when the Internet connection is cut off, and sharing rescue request information and location information.

[1728] "Example 1"

[1729] (Claim 1)

[1730] A means for monitoring seismic intensity information and detecting earthquakes of seismic intensity 7 or higher;

[1731] means for sending an alert to a user's terminal based on the seismic intensity information;

[1732] means for automatically starting up a terminal of a user who receives the alert;

[1733] means for detecting a user's voice and generating and transmitting a help request message;

[1734] A means for switching to ad-hoc communication mode if the Internet connection is lost; 【17...

Claims

1. a means for monitoring seismic intensity information and detecting earthquakes of seismic intensity 7 or greater; means for sending an alert to a user's terminal based on the seismic intensity information; means for automatically starting up a terminal of a user who receives the alert; means for detecting a user's voice and generating and transmitting a help request message; A means for switching to ad-hoc communication mode if the Internet connection is lost; A means of providing first aid instructions and psychological support using edge AI; A means for transmitting user location and status information to a server and storing it in a database. A system including:

2. 10. The system of claim 1, further comprising means for allowing the user to record a final message that is automatically sent to family members or designated contacts when internet connectivity is restored.

3. 2. The system according to claim 1, further comprising means for directly communicating with other terminals using ad-hoc communication when the Internet connection is cut off, and sharing rescue request information and location information.

Images