system

Abstract

Provide a system. 【Solution means】 Means for acquiring the user's location information, Means for receiving and storing user registration information, Means for acquiring disaster information, Means for generating an optimal evacuation action plan based on the acquired location information, registration information, and disaster information, Means for calculating an optimal evacuation route, Means for sending evacuation instructions step by step by push notification, Means for providing an evacuation action plan specialized for users who need special support, A system including the above.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] There is a problem that safety is not ensured due to insufficient appropriate evacuation behavior of residents during disasters. Also, since different actions are required depending on the type and situation of disasters, it is difficult for all residents to take uniformly effective actions. Furthermore, it is difficult to provide appropriate support to residents who require special support. As a result, there is a problem that confusion occurs during emergencies and sufficient safety cannot be ensured.

Means for Solving the Problems

[0005] This invention provides a system that automatically generates individually optimized evacuation plans in the event of a disaster by pre-registering users' addresses, cohabitation configurations, and building information, and acquiring location information in real time. This system acquires disaster information, uses AI to calculate the optimal evacuation route based on registered information and location information, and sends evacuation instructions via push notifications in stages. In addition, it provides special evacuation plans to residents who need assistance, supporting easy and safe evacuation that takes into account traffic conditions and population density.

[0006] A "user" refers to an individual or household that uses the system to receive evacuation information.

[0007] "Means for acquiring location information" refers to methods or devices that use GPS or the communication functions of a device to determine the user's current location.

[0008] "User registration information" refers to information such as the user's address, household composition, and year of construction that the user has registered in the system.

[0009] "Disaster information" refers to data obtained from public institutions or sensors regarding natural disasters such as earthquakes, typhoons, and floods.

[0010] An "evacuation action plan" refers to a series of specific guidelines for ensuring user safety in the event of a disaster.

[0011] "Means for calculating evacuation routes" refers to a method or device that uses map data and real-time traffic information to suggest the optimal evacuation route to the user.

[0012] "Push notification" refers to a communication method in which a server directly sends information to a user's device.

[0013] "Users requiring special assistance" refers to users, including the elderly and infants, who require special consideration or support during evacuation. [Brief explanation of the drawing]

[0014] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Modes for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] In this embodiment of the invention, the user first registers their address, household composition, and the construction date of their residence using a dedicated application via their smartphone or compatible device. This registration information is transmitted to a server and stored in a database.

[0036] Next, in the event of a disaster, the server acquires disaster information based on a pre-configured schedule or a trigger indicating the occurrence of a disaster. This disaster information is obtained in real time through APIs provided by weather stations and disaster information agencies. The server analyzes this information to identify the type and location of the disaster relevant to the user.

[0037] Furthermore, the user's device periodically or under specific conditions utilizes GPS to measure the user's current location and transmit it to the server. Based on this location information, the server determines the extent of any disasters that may affect the user.

[0038] The server uses AI to generate an optimal evacuation plan based on user registration information, current location information, and acquired disaster information. This plan includes the timing of evacuation, specific actions to take, and necessary preparations. In particular, it includes measures that require special consideration for households with elderly people or infants.

[0039] After an evacuation plan is generated, the server uses a map application API to calculate the optimal evacuation route, taking into account traffic conditions and population density, and sends this information to the user's device. Users can then intuitively view this detailed route information on their smartphones.

[0040] Finally, the server uses push notifications to send users step-by-step evacuation instructions, such as starting to prepare for evacuation, waiting at an evacuation site, or immediate evacuation orders. This allows users to act safely and without confusion during a disaster.

[0041] As a concrete example, consider a case where a large-scale earthquake occurs in a certain area. The server quickly acquires earthquake information and provides registered users within the affected area with the optimal action plan based on the seismic resistance of each house, taking into account building information. Furthermore, it suggests the shortest and safest route to evacuation shelters, reflecting real-time traffic information and passability. As a result, users can be expected to complete their evacuation safely.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] The user launches the smartphone app, enters personal information such as address, household composition, and construction date, and presses the registration button. The app then sends this information to the server.

[0045] Step 2:

[0046] The server receives the registration information and saves it to the database. The saved information is managed in association with the user ID.

[0047] Step 3:

[0048] The server acquires disaster information. It accesses APIs from weather stations and disaster information agencies to collect real-time disaster information and analyzes that data.

[0049] Step 4:

[0050] The device obtains the user's location information using its GPS function. The obtained location information is then sent to the server.

[0051] Step 5:

[0052] The server uses the user's location information to assess the characteristics of the area and the potential impact of disasters. By comparing this information with the acquired disaster data, it identifies disasters that the user may be affected by.

[0053] Step 6:

[0054] The server uses AI generation to create an optimal evacuation plan based on the user's registration information, location information, and acquired disaster information. This plan includes the timing of evacuation and specific action suggestions.

[0055] Step 7:

[0056] The server uses the map application's API to calculate the optimal evacuation route, taking into account current traffic conditions and population density. The calculated route information is then sent to the terminal.

[0057] Step 8:

[0058] The terminal displays evacuation route information received from the server to the user. Based on this information, the user prepares for evacuation by intuitively reviewing it.

[0059] Step 9:

[0060] The server uses push notifications to send users step-by-step evacuation instructions, such as when to begin preparing to evacuate and when to move. The instructions are adjusted according to the situation to ensure that users can evacuate safely.

[0061] (Example 1)

[0062] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0063] Providing prompt and accurate evacuation plans during disasters is crucial for saving many lives. However, providing appropriate information and instructions tailored to individual circumstances is difficult, and support for the elderly and those requiring special assistance is particularly inadequate. There is a need to address this challenge and provide users with personalized, safe, and efficient evacuation solutions.

[0064] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0065] In this invention, the server includes means for acquiring user spatial information, means for receiving and storing user registration information, and means for acquiring disaster-related information. This makes it possible to generate evacuation action plans optimized for individual users and to provide appropriate support to users who require special assistance.

[0066] A "user" refers to an individual or organization that uses the system and provides spatial information or registration information.

[0067] "Spatial information" refers to data that indicates the user's current location. This includes digital geographic information such as latitude and longitude.

[0068] "Registered information" refers to personal information that users have provided to the system in advance, such as their address, household composition, and the construction date of their residence.

[0069] "Disaster information" refers to data related to natural disasters, including real-time information provided by weather stations and disaster information agencies.

[0070] A "generative AI model" refers to an applied artificial intelligence algorithm used to generate the optimal output from specific input data. In this case, it is used to create an optimal evacuation plan.

[0071] An "evacuation action plan" refers to a plan that outlines the evacuation procedures and guidelines that users should follow in the event of a disaster.

[0072] An "evacuation route" refers to an optimized path for users to safely move to an evacuation shelter. This route is calculated taking into account real-time traffic conditions and population density.

[0073] "Notification technology" refers to a method of sending information to users in stages. This is used as part of providing evacuation instructions during disasters.

[0074] "Users requiring assistance" refers to users who require additional support during normal evacuation procedures, including the elderly and individuals with physical disabilities.

[0075] In this embodiment of the invention, the user, terminal, and server work together to achieve safe and effective evacuation during a disaster. The user uses a dedicated application via a smartphone or other compatible terminal. This application includes an interface for inputting registration information such as the user's address, household composition, and the construction date of the residence.

[0076] The terminal plays the role of transmitting information entered by the user to the server. The user's location information is periodically acquired using the terminal's GPS function and transmitted to the server. This allows the server to understand the user's spatial information in real time.

[0077] The server stores this registration information and location data in a database and then uses APIs obtained from weather stations and disaster information agencies to acquire disaster-related information. Using a generative AI model, it generates an optimal evacuation plan for each user based on prompt messages. This AI model operates by considering the user's registration information, current situation, and the type and scale of the disaster. For example, a possible prompt message might be: "Address: Shinjuku-ku, Tokyo; Years of residence: 5 years; Presence of elderly: Yes. An earthquake with a seismic intensity of 6- is currently occurring. Please generate the optimal evacuation plan."

[0078] Furthermore, the server uses the map application's API to calculate the optimal evacuation route, taking into account factors such as traffic conditions and population density. The calculated information is sent to the user's device via push notifications as phased evacuation instructions. This allows users to take intuitive and rapid evacuation action.

[0079] As a concrete example, if a user experiences an earthquake in a certain area, the server first acquires earthquake information and provides the user within the affected area with the most suitable action plan. If elderly people or infants are accompanying the user, a plan including special considerations is generated. Evacuation routes based on traffic conditions are also provided, allowing users to avoid congestion and reach evacuation shelters safely.

[0080] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0081] Step 1:

[0082] Users access a dedicated application using a smartphone or other device. There, they enter registration information such as their address, household composition, and the construction date of their residence. This input information is sent from the device to the server. The server receives this information and stores it in a database. The input consists of the user's registration information, and the output is recorded in the form of information stored in the database.

[0083] Step 2:

[0084] The server periodically retrieves disaster information from weather stations and disaster information agencies via APIs. This is done based on specified schedules or real-time triggers. The input for this process is the disaster information obtained via APIs, which is then processed into the required format for analysis. As a result, the retrieved disaster information is stored on the server.

[0085] Step 3:

[0086] The device uses its built-in GPS function to periodically obtain the user's current location information. This location information is sent to the server with the user's permission. The input is the current location data, and the server uses this information to obtain output that confirms the user's current location.

[0087] Step 4:

[0088] The server uses the acquired user registration information, disaster information, and current location information to input prompt messages into the generating AI model. An example of such a prompt message might be: "Address: Shinjuku-ku, Tokyo; Years of residence: 5 years; Presence of elderly: Yes. An earthquake of magnitude 6- is currently occurring. Please generate the optimal evacuation plan." The generating AI model then creates an optimal evacuation plan based on the input data, and the plan is output.

[0089] Step 5:

[0090] The server utilizes the API of a map application to calculate the optimal evacuation route, taking into account traffic conditions and population density. Input includes current road information and traffic conditions, and the optimal route is output based on this information.

[0091] Step 6:

[0092] The server sends an evacuation plan and evacuation route to the user's device via push notification. The user receives the notification and gets specific instructions to begin evacuating safely. The server sends the destination and content of the push notification, and the notification is output to the user as a reference for their actions.

[0093] (Application Example 1)

[0094] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0095] In the event of a natural disaster, there is a need to provide prompt and appropriate evacuation instructions tailored to the individual circumstances of each user. In particular, there is a need for individually tailored action plans for people requiring special assistance, such as the elderly and people with disabilities, and there is a lack of systems and equipment to provide this. Furthermore, providing evacuation routes that appropriately reflect real-time changes in traffic conditions and population density is another crucial issue that needs to be addressed.

[0096] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0097] In this invention, the server includes a device for receiving user location information, a device for acquiring and storing user profile information, and a device for acquiring natural disaster data. This makes it possible to provide optimal evacuation routes and action plans tailored to individual circumstances.

[0098] "User location information" refers to data that indicates the geographical coordinates of a moving object or individual at their current location.

[0099] "User profile information" refers to data that includes individual attribute information about each user, such as address, cohabitation status, and construction date.

[0100] "Natural disaster data" refers to data provided by meteorological observation agencies and disaster prevention agencies, including information on disasters such as earthquakes, typhoons, and floods.

[0101] An "evacuation action plan" is a plan that includes specific procedures and instructions for users to evacuate safely in the event of a disaster.

[0102] A "route guidance device" is a device that calculates and displays the optimal route for guiding users between specified points.

[0103] "Communication methods" refer to technologies and devices for sending and receiving information, including those that utilize the internet and mobile networks.

[0104] A "crisis management system" is a system that plans, implements, and evaluates a series of measures to protect lives and property in the event of a disaster or sudden accident.

[0105] The system for implementing this invention aims to provide evacuation action plans tailored to the individual circumstances of users, thereby supporting safer evacuation during disasters. This system is implemented using mobile terminals, including smartphones, servers, and network communication means connecting them.

[0106] First, users register profile information such as their address, cohabitation status, and the construction date of their residence using a dedicated application on their smartphone. This information is transmitted to the server via the device and stored in a server-side database. This database uses cloud data storage such as Firebase.

[0107] Next, the server uses APIs provided by weather stations and disaster prevention agencies to acquire real-time natural disaster data. This data includes information on earthquakes, typhoons, and floods, which can be used to facilitate a rapid response in the event of the next disaster.

[0108] Furthermore, the user's device utilizes GPS functionality to periodically obtain the user's current location and transmit it to the server. Location information is obtained and managed by the Google® Maps API.

[0109] Based on this location information, profile information, and natural disaster data, the server uses a generative AI model to generate individually optimized evacuation action plans for each user. These plans are dynamically generated using APIs such as OpenAI®.

[0110] Furthermore, the system uses route guidance devices to calculate the optimal evacuation route, taking into account real-time traffic conditions and population density. This function is also implemented using the Google Maps API.

[0111] Finally, Firebase Cloud Messaging (FCM) will be used as the communication method to send phased evacuation instructions to users via push notifications. These notifications will include specific action instructions tailored to the timing and progress of the evacuation.

[0112] As a concrete example, when an earthquake occurs in a certain area, the server quickly collects earthquake information and evaluates the safety of the user's home based on seismic resistance data. Then, it provides the user with the shortest and safest route to the nearest evacuation center in real time.

[0113] An example of a prompt message is: "Your current location is Minato Ward, Tokyo. You are being affected by Typhoon No. 19. Please tell me the best route to an evacuation center."

[0114] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0115] Step 1:

[0116] The user launches a dedicated application on their device and enters their address, cohabitation details, and the construction date of their residence. This entered data is sent by the device to the Firebase database and stored as user profile information.

[0117] Step 2:

[0118] The device periodically obtains the user's current location using its GPS function. The obtained location information is sent to the server as intermediate data, and the server updates the location information based on it.

[0119] Step 3:

[0120] The server acquires disaster information through APIs provided by weather stations and disaster prevention organizations. It analyzes the disaster information data received as input to understand the disaster situation in real time.

[0121] Step 4:

[0122] The server uses a generation AI model to generate an evacuation plan based on user-specific profile information, location information, and disaster information. This plan includes departure timing, necessary items to bring, and additional notes for cases requiring special assistance.

[0123] Step 5:

[0124] The server uses the Google Maps API to calculate the optimal evacuation route based on real-time traffic conditions and population density. The calculated route information is then integrated into the plan to complete comprehensive evacuation guidance.

[0125] Step 6:

[0126] The server sends the completed evacuation plan to the device using Firebase Cloud Messaging (FCM) and provides users with phased push notifications. These notifications include immediate evacuation orders and instructions for the preparation phase.

[0127] Step 7:

[0128] Users review the evacuation plan received on their device screen and begin taking action according to the instructions. This ensures a safe evacuation.

[0129] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0130] In this embodiment of the invention, an emotion engine that takes into account the user's emotional state is added to the functionality as a more personalized safety measure during disasters. This system allows the user to use a smartphone or other device to collect real-time information, including their emotions, and use it to provide advice on evacuation actions.

[0131] First, the user enters their address, cohabitation configuration, and building information via the app and registers it on the server. At this stage, the device prepares to send the user's emotional data to the emotion engine using its camera, microphone, etc. The emotion engine analyzes this emotional data and evaluates the level of stress and anxiety the user exhibits when faced with an emergency.

[0132] In the event of a disaster, the server acquires disaster information in real time and generates an appropriate evacuation plan based on location information and emotional data sent by the user. The data provided by the emotional engine is used to carefully consider the content and timing of evacuation orders and communicate them to the user in the most optimal way to reduce stress.

[0133] As a concrete example, consider the case of an earthquake. The server will determine the magnitude of the earthquake and its impact on the user's location, and decide whether evacuation is necessary. Then, the emotion engine will analyze the user's facial expressions and tone of voice, and if it determines that the user is in a high-stress state, the server will prioritize sending push notifications with messages encouraging calm behavior or simple, reassuring instructions.

[0134] Furthermore, for users requiring special assistance, the server utilizes the results of the emotion engine's evaluation to provide functions such as notifying companions and requesting rescue, ensuring they can evacuate safely. In this way, users can always obtain information in the most optimal form and act with confidence even during disasters.

[0135] The following describes the processing flow.

[0136] Step 1:

[0137] Users enter basic information such as their address, household composition, and construction date into a smartphone app, and the device sends this information to the server. This allows basic user data to be stored on the server.

[0138] Step 2:

[0139] The device activates an emotion engine and collects emotional data through facial recognition and voice analysis of the user. This data is used to estimate the user's stress level and anxiety level.

[0140] Step 3:

[0141] In the event of a disaster, the server retrieves disaster information in real time from the APIs of external disaster information providers. This information includes the type, scale, and location of the disaster.

[0142] Step 4:

[0143] The device obtains the user's current location information using GPS and sends it to the server. Based on the registration information and location information, the server identifies disasters that the user may be affected by.

[0144] Step 5:

[0145] The server combines acquired disaster information with user registration information, location data, and emotional data, and uses a generative AI to generate the optimal evacuation plan. Emotional data is used to consider the evacuation procedures and recommendations that best suit the user's mental state.

[0146] Step 6:

[0147] The server uses the map application's API to calculate the optimal evacuation route, taking into account traffic conditions and population density. This allows users to choose a safe and time-efficient route.

[0148] Step 7:

[0149] The terminal displays information received from the server to the user. This includes information the user needs, such as details of the evacuation plan, evacuation routes, and a list of items to bring.

[0150] Step 8:

[0151] The server sends phased evacuation instructions to the user via push notifications. The content of these notifications is adjusted according to the user's emotional state; for example, a calmer tone of message is sent to a user experiencing high stress.

[0152] Step 9:

[0153] For users requiring special assistance, the server sends instructions that trigger assistance requests based on their emotional state, as well as notifications to alert their companions. This creates an environment where they can evacuate with peace of mind.

[0154] (Example 2)

[0155] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0156] In recent years, the increasing frequency of disasters has highlighted the need for evacuation plans optimized for individual circumstances. However, current systems struggle to provide evacuation instructions that take into account the psychological state of users, making it difficult to make appropriate decisions, especially in emergencies. In addition, conventional methods cannot be expected to provide a swift and appropriate response to people who require special assistance.

[0157] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0158] In this invention, the server includes means for acquiring user location information, means for collecting user emotional states, and means for receiving and storing user registration information. This makes it possible to provide optimized evacuation instructions and plans that take into account each user's emotional state. This method promotes calm and appropriate actions by users even during disasters and enables a rapid response to users who require special assistance.

[0159] "User location information" refers to data indicating the user's current location and is fundamental information for issuing appropriate evacuation orders in emergencies.

[0160] "User emotional state" refers to data that indicates the user's psychological and emotional state, and is an important factor to consider in order to maximize the effectiveness of evacuation orders.

[0161] "User registration information" refers to data that includes personal information such as address, household composition, and building information provided by the user in advance, and is basic information used in formulating evacuation plans.

[0162] "Disaster information" refers to data that shows the occurrence and scale of disasters such as earthquakes and typhoons, and is external information necessary for making decisions about evacuation actions.

[0163] An "optimal evacuation plan" is a plan that takes into account the individual circumstances and emotional state of each user and designs instructions to ensure the most effective and safe evacuation.

[0164] A "generative AI model" is a learning model that uses artificial intelligence to perform data analysis and prediction, and is a technology that supports the generation of evacuation orders based on emotional states.

[0165] "Push notifications" are a method of sending information directly from a server to a user's device, and are a technology that enables rapid communication in emergencies.

[0166] A "specialized evacuation plan for users requiring special assistance" is an evacuation plan designed to meet the specific needs of users who require special consideration, such as the elderly or people with disabilities.

[0167] In this embodiment of the invention, a system is constructed to generate personalized evacuation instructions that take into account the user's emotional state in order to provide effective evacuation support during disasters.

[0168] server

[0169] The server receives basic information and real-time location data from users and is responsible for collecting the latest disaster information from reliable sources. It also integrates an emotion engine and uses a generative AI model to analyze data based on the user's emotional state and adjust the content of evacuation orders. Furthermore, it generates an optimal evacuation action plan tailored to each user's situation and sends instructions directly to the user's device via push notification.

[0170] terminal

[0171] The terminal is a portable device (e.g., a smartphone) that the user uses daily and is equipped with a camera and microphone. This is used to acquire emotional data, including the user's facial expressions and voice, and transmit it to the server. The terminal receives instructions from the server in real time and displays alerts to the user prompting appropriate evacuation actions, either visually or audibly.

[0172] User

[0173] Users register initial information such as their address, household composition, and building details through a smartphone app. Subsequently, they provide information to the system using their camera and microphone for emotional monitoring as needed. In the event of a disaster, they strive to evacuate safely and efficiently by following instructions from their device.

[0174] Specific example

[0175] For example, in the event of an earthquake, the server acquires earthquake intensity information in real time and analyzes the impact on the user's location. Using an emotion engine, it analyzes the user's facial expressions and voice, and if it detects a high-stress state, it sends a message to the device saying, "Please stay calm. Move to a safe place immediately." This kind of personalized response helps users to take calm evacuation actions.

[0176] Example of a prompt

[0177] An example of a prompt to input into the generation AI model would be: "Generate appropriate evacuation instructions based on the user's emotional state. Input data includes the user's voice tone and facial expression during an emergency." This prompt allows the AI ​​model to generate evacuation instructions that take into account the user's psychological aspects.

[0178] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0179] Step 1:

[0180] The user launches a smartphone app, enters their address, cohabitation details, and building information on the registration screen, and sends this data to the server. The server stores the received data and builds a user profile. The input is the user's personal information, and the output is the update of the registration information database on the server side. This operation provides the basis for developing evacuation plans tailored to the user's characteristics.

[0181] Step 2:

[0182] The user uses the device's camera and microphone to capture facial expressions and voice in real time. The device preprocesses this information and sends it to the server as emotion data. The input is audio and video captured from the camera and microphone, and the output is analyzable emotion data. This operation enables more precise evacuation instructions that take the user's psychological state into account.

[0183] Step 3:

[0184] The server acquires the latest disaster information from reliable external sources while continuously monitoring the user's location. It integrates location data with sentiment data and uses a generative AI model to generate an optimal evacuation plan. Inputs include disaster information, location data, and sentiment data, while output is a personalized evacuation plan. This operation enables the provision of rapid and appropriate instructions based on the given conditions.

[0185] Step 4:

[0186] The server uses a generation AI model to create evacuation instructions for individual users based on the generated evacuation action plan. These instructions are fine-tuned according to the user's emotional state and sent to their device via push notifications. The input is the generated evacuation plan, and the output is the specific evacuation instructions sent to the user. This allows users to take the necessary actions at the appropriate time.

[0187] Step 5:

[0188] The terminal notifies the user of evacuation instructions received from the server, providing on-screen display or audio guidance. It converts the information into an optimal format within the terminal to ensure it is presented in a user-friendly manner. The input is the evacuation instruction sent from the server, and the output is specific instructions prompting the user to take action. This operation encourages calm decision-making even in emergency situations.

[0189] Step 6:

[0190] Until the disaster subsides or the user's safety is confirmed, the server and terminal will maintain communication and remain ready to provide further instructions as needed. Inputs are the user's actions and changes in external disaster information, while outputs are up-to-date information relevant to the situation. This operation ensures the user's safety throughout the entire process.

[0191] (Application Example 2)

[0192] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0193] During disasters, there is a need to provide safer and more reassuring evacuation instructions that take into account the emotional state of individual users. However, conventional evacuation instruction systems lack the functionality to consider users' emotions and psychological conditions, making it difficult to alleviate stress and anxiety during emergencies. Therefore, a new system is needed that can provide appropriate instructions tailored to the individual emotional state of each user.

[0194] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0195] In this invention, the server includes means for acquiring user location information, means for receiving and storing user registration information, and means for acquiring disaster information. This makes it possible to adjust personalized evacuation instructions by analyzing the user's emotional state.

[0196] "User location information" refers to data that indicates the user's current location, and is acquired using the device's sensors and GPS function.

[0197] "User registration information" refers to various data about the user, including personal information such as address and emergency contact information.

[0198] "Disaster information" refers to information related to natural disasters such as earthquakes and typhoons, and is data that indicates the degree of risk and impact on the affected area.

[0199] An "evacuation action plan" is a plan that outlines appropriate evacuation routes and procedures during a disaster, and includes instructions necessary to ensure the safety of users.

[0200] "Emotional state" refers to information that indicates the user's current psychological state, and is analyzed based on factors such as facial expressions and voice tone.

[0201] "Analyzing emotional state" is a process that evaluates the user's psychological state based on data from their facial expressions and voice, and determines their stress and anxiety levels.

[0202] An "evacuation order" is a series of instructions provided to ensure safe evacuation during a disaster, and includes information designed to prompt users to take action.

[0203] An "optimal evacuation route" refers to the best path for users to evacuate quickly and safely during a disaster, taking into account traffic conditions and terrain.

[0204] The system for implementing this invention works in conjunction with the user's smart device to provide personalized evacuation instructions during a disaster. It mainly consists of a server, a user-owned terminal, and an emotion analysis engine.

[0205] The server has the ability to collect disaster information in real time and combine it with the user's location and registration information. Based on the acquired data, an emotion analysis engine analyzes the user's facial expressions and voice tone to assess their stress and anxiety levels. This then generates evacuation instructions optimized for each user.

[0206] The device is equipped with a camera and microphone, and its role is to capture the user's facial expressions and voice in real time and send them to an emotion analysis engine. General emotion recognition software (such as Emotion API) is used for emotion analysis. This enables safe evacuation that takes the user's psychological state into consideration.

[0207] As a concrete example, if a user experiences an earthquake, the device immediately sends its location and facial expression data to the server. The server quickly analyzes the scale of the disaster and the user's current location, and sends a push notification of an evacuation message that provides reassurance, tailored to the user's emotional state.

[0208] Examples of prompt statements include:

[0209] "Design an app that allows users of head-mounted displays to receive instructions for safe evacuation during an earthquake. It should utilize the camera and microphone, taking into account the user's real-time emotional state."

[0210] These are some of the points that can be mentioned.

[0211] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0212] Step 1:

[0213] The server collects disaster information and creates an up-to-date database. This information is obtained from various sensor networks and online sources. The input is external disaster information, and the output is an organized disaster information database. This enables real-time information updates.

[0214] Step 2:

[0215] The device uses its built-in camera and microphone to collect user facial expressions and voice data. The input is the user's real-time voice and video, and the output is collected emotion data. This data is used in subsequent analysis steps.

[0216] Step 3:

[0217] The device transmits collected emotional data to an emotion analysis engine. This engine analyzes the input data and quantifies the user's stress and anxiety levels. The output is a numerical indicator showing the user's current emotional state. This allows for a quantitative evaluation of the user's psychological state.

[0218] Step 4:

[0219] The server generates evacuation information based on the user's location and emotional state. The input is the user's location and emotional indicators, and the output is personalized evacuation instructions. A generative AI model is used to formulate an optimal evacuation plan that takes the user's emotions into consideration. This provides the user with the most suitable evacuation method.

[0220] Step 5:

[0221] The terminal sends evacuation instructions received from the server to the user in the form of push notifications. The input is the evacuation instructions from the server, and the output is specific instructions displayed on the terminal screen. This provides the user with information visually or audibly, enabling them to take immediate action.

[0222] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0223] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0224] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0225] [Second Embodiment]

[0226] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0227] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0228] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0229] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0230] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0231] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0232] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0233] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0234] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0235] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0236] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0237] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0238] In this embodiment of the invention, the user first registers their address, household composition, and the construction date of their residence using a dedicated application via their smartphone or compatible device. This registration information is transmitted to a server and stored in a database.

[0239] Next, in the event of a disaster, the server acquires disaster information based on a pre-configured schedule or a trigger indicating the occurrence of a disaster. This disaster information is obtained in real time through APIs provided by weather stations and disaster information agencies. The server analyzes this information to identify the type and location of the disaster relevant to the user.

[0240] Furthermore, the user's device periodically or under specific conditions utilizes GPS to measure the user's current location and transmit it to the server. Based on this location information, the server determines the extent of any disasters that may affect the user.

[0241] The server uses AI to generate an optimal evacuation plan based on user registration information, current location information, and acquired disaster information. This plan includes the timing of evacuation, specific actions to take, and necessary preparations. In particular, it includes measures that require special consideration for households with elderly people or infants.

[0242] After an evacuation plan is generated, the server uses a map application API to calculate the optimal evacuation route, taking into account traffic conditions and population density, and sends this information to the user's device. Users can then intuitively view this detailed route information on their smartphones.

[0243] Finally, the server uses push notifications to send users step-by-step evacuation instructions, such as starting to prepare for evacuation, waiting at an evacuation site, or immediate evacuation orders. This allows users to act safely and without confusion during a disaster.

[0244] As a concrete example, consider a case where a large-scale earthquake occurs in a certain area. The server quickly acquires earthquake information and provides registered users within the affected area with the optimal action plan based on the seismic resistance of each house, taking into account building information. Furthermore, it suggests the shortest and safest route to evacuation shelters, reflecting real-time traffic information and passability. As a result, users can be expected to complete their evacuation safely.

[0245] The following describes the processing flow.

[0246] Step 1:

[0247] The user launches the smartphone app, enters personal information such as address, household composition, and construction date, and presses the registration button. The app then sends this information to the server.

[0248] Step 2:

[0249] The server receives the registration information and saves it to the database. The saved information is managed in association with the user ID.

[0250] Step 3:

[0251] The server acquires disaster information. It accesses APIs from weather stations and disaster information agencies to collect real-time disaster information and analyzes that data.

[0252] Step 4:

[0253] The device obtains the user's location information using its GPS function. The obtained location information is then sent to the server.

[0254] Step 5:

[0255] The server uses the user's location information to assess the characteristics of the area and the potential impact of disasters. By comparing this information with the acquired disaster data, it identifies disasters that the user may be affected by.

[0256] Step 6:

[0257] The server uses AI generation to create an optimal evacuation plan based on the user's registration information, location information, and acquired disaster information. This plan includes the timing of evacuation and specific action suggestions.

[0258] Step 7:

[0259] The server uses the map application's API to calculate the optimal evacuation route, taking into account current traffic conditions and population density. The calculated route information is then sent to the terminal.

[0260] Step 8:

[0261] The terminal displays evacuation route information received from the server to the user. Based on this information, the user prepares for evacuation by intuitively reviewing it.

[0262] Step 9:

[0263] The server uses push notifications to send users step-by-step evacuation instructions, such as when to begin preparing to evacuate and when to move. The instructions are adjusted according to the situation to ensure that users can evacuate safely.

[0264] (Example 1)

[0265] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0266] Providing prompt and accurate evacuation plans during disasters is crucial for saving many lives. However, providing appropriate information and instructions tailored to individual circumstances is difficult, and support for the elderly and those requiring special assistance is particularly inadequate. There is a need to address this challenge and provide users with personalized, safe, and efficient evacuation solutions.

[0267] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0268] In this invention, the server includes means for acquiring user spatial information, means for receiving and storing user registration information, and means for acquiring disaster-related information. This makes it possible to generate evacuation action plans optimized for individual users and to provide appropriate support to users who require special assistance.

[0269] A "user" refers to an individual or organization that uses the system and provides spatial information or registration information.

[0270] "Spatial information" refers to data that indicates the user's current location. This includes digital geographic information such as latitude and longitude.

[0271] "Registered information" refers to personal information that users have provided to the system in advance, such as their address, household composition, and the construction date of their residence.

[0272] "Disaster information" refers to data related to natural disasters, including real-time information provided by weather stations and disaster information agencies.

[0273] A "generative AI model" refers to an applied artificial intelligence algorithm used to generate the optimal output from specific input data. In this case, it is used to create an optimal evacuation plan.

[0274] An "evacuation action plan" refers to a plan that outlines the evacuation procedures and guidelines that users should follow in the event of a disaster.

[0275] An "evacuation route" refers to an optimized path for users to safely move to an evacuation shelter. This route is calculated taking into account real-time traffic conditions and population density.

[0276] "Notification technology" refers to a method of sending information to users in stages. This is used as part of providing evacuation instructions during disasters.

[0277] "Users requiring assistance" refers to users who require additional support during normal evacuation procedures, including the elderly and individuals with physical disabilities.

[0278] In this embodiment of the invention, the user, terminal, and server work together to achieve safe and effective evacuation during a disaster. The user uses a dedicated application via a smartphone or other compatible terminal. This application includes an interface for inputting registration information such as the user's address, household composition, and the construction date of the residence.

[0279] The terminal plays a role of sending the information input by the user to the server. The user's location information is periodically obtained using the GPS function of the terminal and sent to the server. Thereby, the server can grasp the user's spatial information in real time.

[0280] After the server stores these registration information and location information in the database, it obtains disaster-related information using the APIs obtained from meteorological observation stations and disaster information institutions. Using the generated AI model, an optimal evacuation action plan for each user is generated based on the prompt text. This AI model operates considering the user's registration information, current situation, type and scale of the disaster. For example, possible prompt texts include "Address: Shinjuku-ku, Tokyo, Years of residence: 5 years, Presence of elderly people: Yes. Currently, an earthquake of intensity 5-weak is occurring. Please generate the optimal evacuation action."

[0281] Furthermore, the server uses the API of the map application to calculate the optimal evacuation route considering elements such as traffic conditions and population density. The calculated information is sent to the user's terminal through push notifications as step-by-step evacuation instructions. Thereby, the user can take evacuation actions intuitively and quickly.

[0282] As a specific example, if a user encounters an earthquake in a certain area, first the server obtains the earthquake information and provides an optimal action plan to the users within the affected area. If there are elderly people or infants traveling together, a plan including special considerations is generated. An evacuation route according to the traffic conditions is also provided, enabling them to avoid congestion and head safely to the evacuation shelter.

[0283] The flow of the specific process in Example 1 will be described using FIG. 11.

[0284] Step 1:

[0285] The user uses a terminal such as a smartphone to access a dedicated application. There, the user inputs registration information such as their address, cohabitation composition, and the construction date of their residence. This input information is sent from the terminal to the server. The server receives this and stores it in the database. The input is the user's registration information, and the output is recorded in the form of the information being stored in the database.

[0286] Step 2:

[0287] The server periodically obtains disaster information from meteorological observation stations and disaster information agencies via an API. This is done based on a specified schedule or a real-time trigger. The input in this process is the disaster information obtained via the API, which is processed into the required format to be useful for analysis. As a result, the obtained disaster information is accumulated on the server.

[0288] Step 3:

[0289] The terminal uses its built-in GPS function to periodically obtain the user's current location information. With the user's permission, this location information is sent to the server. The input is the data of the current location, and the server uses this information to obtain an output for confirming the user's current location.

[0290] Step 4:

[0291] The server uses the obtained user registration information, disaster information, and current location information to input a prompt sentence into the generative AI model. As an example of this prompt sentence, content such as "Address: Shinjuku-ku, Tokyo, Years of residence: 5 years, Presence of elderly people: Yes. Currently, an earthquake of intensity 5- has occurred. Please generate the optimal evacuation behavior." can be considered. The generative AI model creates an optimal evacuation plan for the input data, and the plan is output.

[0292] Step 5:

[0293] The server utilizes the API of a map application to calculate the optimal evacuation route, taking into account traffic conditions and population density. Input includes current road information and traffic conditions, and the optimal route is output based on this information.

[0294] Step 6:

[0295] The server sends an evacuation plan and evacuation route to the user's device via push notification. The user receives the notification and gets specific instructions to begin evacuating safely. The server sends the destination and content of the push notification, and the notification is output to the user as a reference for their actions.

[0296] (Application Example 1)

[0297] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0298] In the event of a natural disaster, there is a need to provide prompt and appropriate evacuation instructions tailored to the individual circumstances of each user. In particular, there is a need for individually tailored action plans for people requiring special assistance, such as the elderly and people with disabilities, and there is a lack of systems and equipment to provide this. Furthermore, providing evacuation routes that appropriately reflect real-time changes in traffic conditions and population density is another crucial issue that needs to be addressed.

[0299] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0300] In this invention, the server includes a device for receiving user location information, a device for acquiring and storing user profile information, and a device for acquiring natural disaster data. This makes it possible to provide optimal evacuation routes and action plans tailored to individual circumstances.

[0301] "User location information" refers to data that indicates the geographical coordinates of a moving object or individual at their current location.

[0302] "User profile information" is data that includes individual attribute information such as the address of an individual user, the living situation of co - residents, and the construction date of a building.

[0303] "Natural disaster data" is data that includes information on disasters such as earthquakes, typhoons, and floods provided by meteorological observation institutions and disaster prevention institutions.

[0304] "Evacuation plan" is a plan that includes specific procedures and instructions for a user to evacuate safely during a disaster.

[0305] "Route guidance device" is a device for calculating and displaying an optimal route for guiding between designated points.

[0306] "Communication means" refers to technologies and devices for transmitting and receiving information, which utilize the Internet and mobile networks.

[0307] "Crisis management system" is a system that conducts a series of plans, executions, and evaluations to protect lives and property in the event of disasters or emergencies.

[0308] The system for implementing this invention aims to provide an evacuation plan according to the individual situation of a user and support safer evacuation during a disaster. This system is realized by using a mobile terminal including a smartphone, a server, and network communication means connecting them.

[0309] First, the user uses a dedicated application on their smartphone to register profile information such as their address, the living situation of co - residents, and the construction date of their residence. This information is transmitted to the server via the terminal and stored in the database on the server side. For this database, cloud data storage such as Firebase is used, for example.

[0310] Next, the server uses APIs provided by weather stations and disaster prevention agencies to acquire real-time natural disaster data. This data includes information on earthquakes, typhoons, and floods, which can be used to facilitate a rapid response in the event of the next disaster.

[0311] Furthermore, the user's device utilizes GPS functionality to periodically obtain the user's current location and transmit it to the server. Location information is obtained and managed by the Google Maps API.

[0312] Based on this location information, profile information, and natural disaster data, the server uses a generative AI model to generate individually optimized evacuation action plans for each user. These plans are dynamically generated using APIs such as OpenAI.

[0313] Furthermore, the system uses route guidance devices to calculate the optimal evacuation route, taking into account real-time traffic conditions and population density. This function is also implemented using the Google Maps API.

[0314] Finally, Firebase Cloud Messaging (FCM) will be used as the communication method to send phased evacuation instructions to users via push notifications. These notifications will include specific action instructions tailored to the timing and progress of the evacuation.

[0315] As a concrete example, when an earthquake occurs in a certain area, the server quickly collects earthquake information and evaluates the safety of the user's home based on seismic resistance data. Then, it provides the user with the shortest and safest route to the nearest evacuation center in real time.

[0316] An example of a prompt message is: "Your current location is Minato Ward, Tokyo. You are being affected by Typhoon No. 19. Please tell me the best route to an evacuation center."

[0317] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0318] Step 1:

[0319] The user launches a dedicated application on their device and enters their address, cohabitation details, and the construction date of their residence. This entered data is sent by the device to the Firebase database and stored as user profile information.

[0320] Step 2:

[0321] The device periodically obtains the user's current location using its GPS function. The obtained location information is sent to the server as intermediate data, and the server updates the location information based on it.

[0322] Step 3:

[0323] The server acquires disaster information through APIs provided by weather stations and disaster prevention organizations. It analyzes the disaster information data received as input to understand the disaster situation in real time.

[0324] Step 4:

[0325] The server uses a generation AI model to generate an evacuation plan based on user-specific profile information, location information, and disaster information. This plan includes departure timing, necessary items to bring, and additional notes for cases requiring special assistance.

[0326] Step 5:

[0327] The server uses the Google Maps API to calculate the optimal evacuation route based on real-time traffic conditions and population density. The calculated route information is then integrated into the plan to complete comprehensive evacuation guidance.

[0328] Step 6:

[0329] The server sends the completed evacuation plan to the device using Firebase Cloud Messaging (FCM) and provides users with phased push notifications. These notifications include immediate evacuation orders and instructions for the preparation phase.

[0330] Step 7:

[0331] Users review the evacuation plan received on their device screen and begin taking action according to the instructions. This ensures a safe evacuation.

[0332] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0333] In this embodiment of the invention, an emotion engine that takes into account the user's emotional state is added to the functionality as a more personalized safety measure during disasters. This system allows the user to use a smartphone or other device to collect real-time information, including their emotions, and use it to provide advice on evacuation actions.

[0334] First, the user enters their address, cohabitation configuration, and building information via the app and registers it on the server. At this stage, the device prepares to send the user's emotional data to the emotion engine using its camera, microphone, etc. The emotion engine analyzes this emotional data and evaluates the level of stress and anxiety the user exhibits when faced with an emergency.

[0335] In the event of a disaster, the server acquires disaster information in real time and generates an appropriate evacuation plan based on location information and emotional data sent by the user. The data provided by the emotional engine is used to carefully consider the content and timing of evacuation orders and communicate them to the user in the most optimal way to reduce stress.

[0336] As a concrete example, consider the case of an earthquake. The server will determine the magnitude of the earthquake and its impact on the user's location, and decide whether evacuation is necessary. Then, the emotion engine will analyze the user's facial expressions and tone of voice, and if it determines that the user is in a high-stress state, the server will prioritize sending push notifications with messages encouraging calm behavior or simple, reassuring instructions.

[0337] Furthermore, for users requiring special assistance, the server utilizes the results of the emotion engine's evaluation to provide functions such as notifying companions and requesting rescue, ensuring they can evacuate safely. In this way, users can always obtain information in the most optimal form and act with confidence even during disasters.

[0338] The following describes the processing flow.

[0339] Step 1:

[0340] Users enter basic information such as their address, household composition, and construction date into a smartphone app, and the device sends this information to the server. This allows basic user data to be stored on the server.

[0341] Step 2:

[0342] The device activates an emotion engine and collects emotional data through facial recognition and voice analysis of the user. This data is used to estimate the user's stress level and anxiety level.

[0343] Step 3:

[0344] In the event of a disaster, the server retrieves disaster information in real time from the APIs of external disaster information providers. This information includes the type, scale, and location of the disaster.

[0345] Step 4:

[0346] The device obtains the user's current location information using GPS and sends it to the server. Based on the registration information and location information, the server identifies disasters that the user may be affected by.

[0347] Step 5:

[0348] The server combines acquired disaster information with user registration information, location data, and emotional data, and uses a generative AI to generate the optimal evacuation plan. Emotional data is used to consider the evacuation procedures and recommendations that best suit the user's mental state.

[0349] Step 6:

[0350] The server uses the map application's API to calculate the optimal evacuation route, taking into account traffic conditions and population density. This allows users to choose a safe and time-efficient route.

[0351] Step 7:

[0352] The terminal displays information received from the server to the user. This includes information the user needs, such as details of the evacuation plan, evacuation routes, and a list of items to bring.

[0353] Step 8:

[0354] The server sends phased evacuation instructions to the user via push notifications. The content of these notifications is adjusted according to the user's emotional state; for example, a calmer tone of message is sent to a user experiencing high stress.

[0355] Step 9:

[0356] For users requiring special assistance, the server sends instructions that trigger assistance requests based on their emotional state, as well as notifications to alert their companions. This creates an environment where they can evacuate with peace of mind.

[0357] (Example 2)

[0358] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0359] In recent years, the increasing frequency of disasters has highlighted the need for evacuation plans optimized for individual circumstances. However, current systems struggle to provide evacuation instructions that take into account the psychological state of users, making it difficult to make appropriate decisions, especially in emergencies. In addition, conventional methods cannot be expected to provide a swift and appropriate response to people who require special assistance.

[0360] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0361] In this invention, the server includes means for acquiring user location information, means for collecting user emotional states, and means for receiving and storing user registration information. This makes it possible to provide optimized evacuation instructions and plans that take into account each user's emotional state. This method promotes calm and appropriate actions by users even during disasters and enables a rapid response to users who require special assistance.

[0362] "User location information" refers to data indicating the user's current location and is fundamental information for issuing appropriate evacuation orders in emergencies.

[0363] "User emotional state" refers to data that indicates the user's psychological and emotional state, and is an important factor to consider in order to maximize the effectiveness of evacuation orders.

[0364] "User registration information" refers to data that includes personal information such as address, household composition, and building information provided by the user in advance, and is basic information used in formulating evacuation plans.

[0365] "Disaster information" refers to data that shows the occurrence and scale of disasters such as earthquakes and typhoons, and is external information necessary for making decisions about evacuation actions.

[0366] An "optimal evacuation plan" is a plan that takes into account the individual circumstances and emotional state of each user and designs instructions to ensure the most effective and safe evacuation.

[0367] A "generative AI model" is a learning model that uses artificial intelligence to perform data analysis and prediction, and is a technology that supports the generation of evacuation orders based on emotional states.

[0368] "Push notifications" are a method of sending information directly from a server to a user's device, and are a technology that enables rapid communication in emergencies.

[0369] A "specialized evacuation plan for users requiring special assistance" is an evacuation plan designed to meet the specific needs of users who require special consideration, such as the elderly or people with disabilities.

[0370] In this embodiment of the invention, a system is constructed to generate personalized evacuation instructions that take into account the user's emotional state in order to provide effective evacuation support during disasters.

[0371] server

[0372] The server receives basic information and real-time location data from users and is responsible for collecting the latest disaster information from reliable sources. It also integrates an emotion engine and uses a generative AI model to analyze data based on the user's emotional state and adjust the content of evacuation orders. Furthermore, it generates an optimal evacuation action plan tailored to each user's situation and sends instructions directly to the user's device via push notification.

[0373] terminal

[0374] The terminal is a portable device (e.g., a smartphone) that the user uses daily and is equipped with a camera and microphone. This is used to acquire emotional data, including the user's facial expressions and voice, and transmit it to the server. The terminal receives instructions from the server in real time and displays alerts to the user prompting appropriate evacuation actions, either visually or audibly.

[0375] User

[0376] Users register initial information such as their address, household composition, and building details through a smartphone app. Subsequently, they provide information to the system using their camera and microphone for emotional monitoring as needed. In the event of a disaster, they strive to evacuate safely and efficiently by following instructions from their device.

[0377] Specific example

[0378] For example, in the event of an earthquake, the server acquires earthquake intensity information in real time and analyzes the impact on the user's location. Using an emotion engine, it analyzes the user's facial expressions and voice, and if it detects a high-stress state, it sends a message to the device saying, "Please stay calm. Move to a safe place immediately." This kind of personalized response helps users to take calm evacuation actions.

[0379] Example of a prompt

[0380] An example of a prompt to input into the generation AI model would be: "Generate appropriate evacuation instructions based on the user's emotional state. Input data includes the user's voice tone and facial expression during an emergency." This prompt allows the AI ​​model to generate evacuation instructions that take into account the user's psychological aspects.

[0381] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0382] Step 1:

[0383] The user launches a smartphone app, enters their address, cohabitation details, and building information on the registration screen, and sends this data to the server. The server stores the received data and builds a user profile. The input is the user's personal information, and the output is the update of the registration information database on the server side. This operation provides the basis for developing evacuation plans tailored to the user's characteristics.

[0384] Step 2:

[0385] The user uses the device's camera and microphone to capture facial expressions and voice in real time. The device preprocesses this information and sends it to the server as emotion data. The input is audio and video captured from the camera and microphone, and the output is analyzable emotion data. This operation enables more precise evacuation instructions that take the user's psychological state into account.

[0386] Step 3:

[0387] The server acquires the latest disaster information from reliable external sources while continuously monitoring the user's location. It integrates location data with sentiment data and uses a generative AI model to generate an optimal evacuation plan. Inputs include disaster information, location data, and sentiment data, while output is a personalized evacuation plan. This operation enables the provision of rapid and appropriate instructions based on the given conditions.

[0388] Step 4:

[0389] The server uses a generation AI model to create evacuation instructions for individual users based on the generated evacuation action plan. These instructions are fine-tuned according to the user's emotional state and sent to their device via push notifications. The input is the generated evacuation plan, and the output is the specific evacuation instructions sent to the user. This allows users to take the necessary actions at the appropriate time.

[0390] Step 5:

[0391] The terminal notifies the user of evacuation instructions received from the server, providing on-screen display or audio guidance. It converts the information into an optimal format within the terminal to ensure it is presented in a user-friendly manner. The input is the evacuation instruction sent from the server, and the output is specific instructions prompting the user to take action. This operation encourages calm decision-making even in emergency situations.

[0392] Step 6:

[0393] Until the disaster subsides or the user's safety is confirmed, the server and terminal will maintain communication and remain ready to provide further instructions as needed. Inputs are the user's actions and changes in external disaster information, while outputs are up-to-date information relevant to the situation. This operation ensures the user's safety throughout the entire process.

[0394] (Application Example 2)

[0395] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0396] During disasters, there is a need to provide safer and more reassuring evacuation instructions that take into account the emotional state of individual users. However, conventional evacuation instruction systems lack the functionality to consider users' emotions and psychological conditions, making it difficult to alleviate stress and anxiety during emergencies. Therefore, a new system is needed that can provide appropriate instructions tailored to the individual emotional state of each user.

[0397] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0398] In this invention, the server includes means for acquiring user location information, means for receiving and storing user registration information, and means for acquiring disaster information. This makes it possible to adjust personalized evacuation instructions by analyzing the user's emotional state.

[0399] "User location information" refers to data that indicates the user's current location, and is acquired using the device's sensors and GPS function.

[0400] "User registration information" refers to various data about the user, including personal information such as address and emergency contact information.

[0401] "Disaster information" refers to information related to natural disasters such as earthquakes and typhoons, and is data that indicates the degree of risk and impact on the affected area.

[0402] An "evacuation action plan" is a plan that outlines appropriate evacuation routes and procedures during a disaster, and includes instructions necessary to ensure the safety of users.

[0403] "Emotional state" refers to information that indicates the user's current psychological state, and is analyzed based on factors such as facial expressions and voice tone.

[0404] "Analyzing emotional state" is a process that evaluates the user's psychological state based on data from their facial expressions and voice, and determines their stress and anxiety levels.

[0405] An "evacuation order" is a series of instructions provided to ensure safe evacuation during a disaster, and includes information designed to prompt users to take action.

[0406] An "optimal evacuation route" refers to the best path for users to evacuate quickly and safely during a disaster, taking into account traffic conditions and terrain.

[0407] The system for implementing this invention works in conjunction with the user's smart device to provide personalized evacuation instructions during a disaster. It mainly consists of a server, a user-owned terminal, and an emotion analysis engine.

[0408] The server has the ability to collect disaster information in real time and combine it with the user's location and registration information. Based on the acquired data, an emotion analysis engine analyzes the user's facial expressions and voice tone to assess their stress and anxiety levels. This then generates evacuation instructions optimized for each user.

[0409] The device is equipped with a camera and microphone, and its role is to capture the user's facial expressions and voice in real time and send them to an emotion analysis engine. General emotion recognition software (such as Emotion API) is used for emotion analysis. This enables safe evacuation that takes the user's psychological state into consideration.

[0410] As a concrete example, if a user experiences an earthquake, the device immediately sends its location and facial expression data to the server. The server quickly analyzes the scale of the disaster and the user's current location, and sends a push notification of an evacuation message that provides reassurance, tailored to the user's emotional state.

[0411] Examples of prompt statements include:

[0412] "Design an app that allows users of head-mounted displays to receive instructions for safe evacuation during an earthquake. It should utilize the camera and microphone, taking into account the user's real-time emotional state."

[0413] These are some of the points that were raised.

[0414] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0415] Step 1:

[0416] The server collects disaster information and creates an up-to-date database. This information is obtained from various sensor networks and online sources. The input is external disaster information, and the output is an organized disaster information database. This enables real-time information updates.

[0417] Step 2:

[0418] The device uses its built-in camera and microphone to collect user facial expressions and voice data. The input is the user's real-time voice and video, and the output is collected emotion data. This data is used in subsequent analysis steps.

[0419] Step 3:

[0420] The device transmits collected emotional data to an emotion analysis engine. This engine analyzes the input data and quantifies the user's stress and anxiety levels. The output is a numerical indicator showing the user's current emotional state. This allows for a quantitative evaluation of the user's psychological state.

[0421] Step 4:

[0422] The server generates evacuation information based on the user's location and emotional state. The input is the user's location and emotional indicators, and the output is personalized evacuation instructions. A generative AI model is used to formulate an optimal evacuation plan that takes the user's emotions into consideration. This provides the user with the most suitable evacuation method.

[0423] Step 5:

[0424] The terminal sends evacuation instructions received from the server to the user in the form of push notifications. The input is the evacuation instructions from the server, and the output is specific instructions displayed on the terminal screen. This provides the user with information visually or audibly, enabling them to take immediate action.

[0425] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0426] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0427] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0428] [Third Embodiment]

[0429] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0430] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0431] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0432] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0433] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0434] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0435] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0436] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0437] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0438] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0439] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0440] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0441] In this embodiment of the invention, the user first registers their address, household composition, and the construction date of their residence using a dedicated application via their smartphone or compatible device. This registration information is transmitted to a server and stored in a database.

[0442] Next, in the event of a disaster, the server acquires disaster information based on a pre-configured schedule or a trigger indicating the occurrence of a disaster. This disaster information is obtained in real time through APIs provided by weather stations and disaster information agencies. The server analyzes this information to identify the type and location of the disaster relevant to the user.

[0443] Furthermore, the user's device periodically or under specific conditions utilizes GPS to measure the user's current location and transmit it to the server. Based on this location information, the server determines the extent of any disasters that may affect the user.

[0444] The server uses AI to generate an optimal evacuation plan based on user registration information, current location information, and acquired disaster information. This plan includes the timing of evacuation, specific actions to take, and necessary preparations. In particular, it includes measures that require special consideration for households with elderly people or infants.

[0445] After an evacuation plan is generated, the server uses a map application API to calculate the optimal evacuation route, taking into account traffic conditions and population density, and sends this information to the user's device. Users can then intuitively view this detailed route information on their smartphones.

[0446] Finally, the server uses push notifications to send users step-by-step evacuation instructions, such as starting to prepare for evacuation, waiting at an evacuation site, or immediate evacuation orders. This allows users to act safely and without confusion during a disaster.

[0447] As a concrete example, consider a case where a large-scale earthquake occurs in a certain area. The server quickly acquires earthquake information and provides registered users within the affected area with the optimal action plan based on the seismic resistance of each house, taking into account building information. Furthermore, it suggests the shortest and safest route to evacuation shelters, reflecting real-time traffic information and passability. As a result, users can be expected to complete their evacuation safely.

[0448] The following describes the processing flow.

[0449] Step 1:

[0450] The user launches the smartphone app, enters personal information such as address, household composition, and construction date, and presses the registration button. The app then sends this information to the server.

[0451] Step 2:

[0452] The server receives the registration information and saves it to the database. The saved information is managed in association with the user ID.

[0453] Step 3:

[0454] The server acquires disaster information. It accesses APIs from weather stations and disaster information agencies to collect real-time disaster information and analyzes that data.

[0455] Step 4:

[0456] The device obtains the user's location information using its GPS function. The obtained location information is then sent to the server.

[0457] Step 5:

[0458] The server uses the user's location information to assess the characteristics of the area and the potential impact of disasters. By comparing this information with the acquired disaster data, it identifies disasters that the user may be affected by.

[0459] Step 6:

[0460] The server uses AI generation to create an optimal evacuation plan based on the user's registration information, location information, and acquired disaster information. This plan includes the timing of evacuation and specific action suggestions.

[0461] Step 7:

[0462] The server uses the map application's API to calculate the optimal evacuation route, taking into account current traffic conditions and population density. The calculated route information is then sent to the terminal.

[0463] Step 8:

[0464] The terminal displays evacuation route information received from the server to the user. Based on this information, the user prepares for evacuation by intuitively reviewing it.

[0465] Step 9:

[0466] The server uses push notifications to send users step-by-step evacuation instructions, such as when to begin preparing to evacuate and when to move. The instructions are adjusted according to the situation to ensure that users can evacuate safely.

[0467] (Example 1)

[0468] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0469] Providing prompt and accurate evacuation plans during disasters is crucial for saving many lives. However, providing appropriate information and instructions tailored to individual circumstances is difficult, and support for the elderly and those requiring special assistance is particularly inadequate. There is a need to address this challenge and provide users with personalized, safe, and efficient evacuation solutions.

[0470] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0471] In this invention, the server includes means for acquiring user spatial information, means for receiving and storing user registration information, and means for acquiring disaster-related information. This makes it possible to generate evacuation action plans optimized for individual users and to provide appropriate support to users who require special assistance.

[0472] A "user" refers to an individual or organization that uses the system and provides spatial information or registration information.

[0473] "Spatial information" refers to data that indicates the user's current location. This includes digital geographic information such as latitude and longitude.

[0474] "Registered information" refers to personal information that users have provided to the system in advance, such as their address, household composition, and the construction date of their residence.

[0475] "Disaster information" refers to data related to natural disasters, including real-time information provided by weather stations and disaster information agencies.

[0476] A "generative AI model" refers to an applied artificial intelligence algorithm used to generate the optimal output from specific input data. In this case, it is used to create an optimal evacuation plan.

[0477] An "evacuation action plan" refers to a plan that outlines the evacuation procedures and guidelines that users should follow in the event of a disaster.

[0478] An "evacuation route" refers to an optimized path for users to safely move to an evacuation shelter. This route is calculated taking into account real-time traffic conditions and population density.

[0479] "Notification technology" refers to a method of sending information to users in stages. This is used as part of providing evacuation instructions during disasters.

[0480] "Users requiring assistance" refers to users who require additional support during normal evacuation procedures, including the elderly and individuals with physical disabilities.

[0481] In this embodiment of the invention, the user, terminal, and server work together to achieve safe and effective evacuation during a disaster. The user uses a dedicated application via a smartphone or other compatible terminal. This application includes an interface for inputting registration information such as the user's address, household composition, and the construction date of the residence.

[0482] The terminal plays the role of transmitting information entered by the user to the server. The user's location information is periodically acquired using the terminal's GPS function and transmitted to the server. This allows the server to understand the user's spatial information in real time.

[0483] The server stores this registration information and location data in a database and then uses APIs obtained from weather stations and disaster information agencies to acquire disaster-related information. Using a generative AI model, it generates an optimal evacuation plan for each user based on prompt messages. This AI model operates by considering the user's registration information, current situation, and the type and scale of the disaster. For example, a possible prompt message might be: "Address: Shinjuku-ku, Tokyo; Years of residence: 5 years; Presence of elderly: Yes. An earthquake with a seismic intensity of 6- is currently occurring. Please generate the optimal evacuation plan."

[0484] Furthermore, the server uses the map application's API to calculate the optimal evacuation route, taking into account factors such as traffic conditions and population density. The calculated information is sent to the user's device via push notifications as phased evacuation instructions. This allows users to take intuitive and rapid evacuation action.

[0485] As a concrete example, if a user experiences an earthquake in a certain area, the server first acquires earthquake information and provides the user within the affected area with the most suitable action plan. If elderly people or infants are accompanying the user, a plan including special considerations is generated. Evacuation routes based on traffic conditions are also provided, allowing users to avoid congestion and reach evacuation shelters safely.

[0486] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0487] Step 1:

[0488] Users access a dedicated application using a smartphone or other device. There, they enter registration information such as their address, household composition, and the construction date of their residence. This input information is sent from the device to the server. The server receives this information and stores it in a database. The input consists of the user's registration information, and the output is recorded in the form of information stored in the database.

[0489] Step 2:

[0490] The server periodically retrieves disaster information from weather stations and disaster information agencies via APIs. This is done based on specified schedules or real-time triggers. The input for this process is the disaster information obtained via APIs, which is then processed into the required format for analysis. As a result, the retrieved disaster information is stored on the server.

[0491] Step 3:

[0492] The device uses its built-in GPS function to periodically obtain the user's current location information. This location information is sent to the server with the user's permission. The input is the current location data, and the server uses this information to obtain output that confirms the user's current location.

[0493] Step 4:

[0494] The server uses the acquired user registration information, disaster information, and current location information to input prompt messages into the generating AI model. An example of such a prompt message might be: "Address: Shinjuku-ku, Tokyo; Years of residence: 5 years; Presence of elderly: Yes. An earthquake of magnitude 6- is currently occurring. Please generate the optimal evacuation plan." The generating AI model then creates an optimal evacuation plan based on the input data, and the plan is output.

[0495] Step 5:

[0496] The server utilizes the API of a map application to calculate the optimal evacuation route, taking into account traffic conditions and population density. Input includes current road information and traffic conditions, and the optimal route is output based on this information.

[0497] Step 6:

[0498] The server sends an evacuation plan and evacuation route to the user's device via push notification. The user receives the notification and gets specific instructions to begin evacuating safely. The server sends the destination and content of the push notification, and the notification is output to the user as a reference for their actions.

[0499] (Application Example 1)

[0500] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0501] In the event of a natural disaster, there is a need to provide prompt and appropriate evacuation instructions tailored to the individual circumstances of each user. In particular, there is a need for individually tailored action plans for people requiring special assistance, such as the elderly and people with disabilities, and there is a lack of systems and equipment to provide this. Furthermore, providing evacuation routes that appropriately reflect real-time changes in traffic conditions and population density is another crucial issue that needs to be addressed.

[0502] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0503] In this invention, the server includes a device for receiving user location information, a device for acquiring and storing user profile information, and a device for acquiring natural disaster data. This makes it possible to provide optimal evacuation routes and action plans tailored to individual circumstances.

[0504] "User location information" refers to data that indicates the geographical coordinates of a moving object or individual at their current location.

[0505] "User profile information" refers to data that includes individual attribute information about each user, such as address, cohabitation status, and construction date.

[0506] "Natural disaster data" refers to data provided by meteorological observation agencies and disaster prevention agencies, including information on disasters such as earthquakes, typhoons, and floods.

[0507] An "evacuation action plan" is a plan that includes specific procedures and instructions for users to evacuate safely in the event of a disaster.

[0508] A "route guidance device" is a device that calculates and displays the optimal route for guiding users between specified points.

[0509] "Communication methods" refer to technologies and devices for sending and receiving information, including those that utilize the internet and mobile networks.

[0510] A "crisis management system" is a system that plans, implements, and evaluates a series of measures to protect lives and property in the event of a disaster or sudden accident.

[0511] The system for implementing this invention aims to provide evacuation action plans tailored to the individual circumstances of users, thereby supporting safer evacuation during disasters. This system is implemented using mobile terminals, including smartphones, servers, and network communication means connecting them.

[0512] First, users register profile information such as their address, cohabitation status, and the construction date of their residence using a dedicated application on their smartphone. This information is transmitted to the server via the device and stored in a server-side database. This database uses cloud data storage such as Firebase.

[0513] Next, the server uses APIs provided by weather stations and disaster prevention agencies to acquire real-time natural disaster data. This data includes information on earthquakes, typhoons, and floods, which can be used to facilitate a rapid response in the event of the next disaster.

[0514] Furthermore, the user's device utilizes GPS functionality to periodically obtain the user's current location and transmit it to the server. Location information is obtained and managed by the Google Maps API.

[0515] Based on this location information, profile information, and natural disaster data, the server uses a generative AI model to generate individually optimized evacuation action plans for each user. These plans are dynamically generated using APIs such as OpenAI.

[0516] Furthermore, the system uses route guidance devices to calculate the optimal evacuation route, taking into account real-time traffic conditions and population density. This function is also implemented using the Google Maps API.

[0517] Finally, Firebase Cloud Messaging (FCM) will be used as the communication method to send phased evacuation instructions to users via push notifications. These notifications will include specific action instructions tailored to the timing and progress of the evacuation.

[0518] As a concrete example, when an earthquake occurs in a certain area, the server quickly collects earthquake information and evaluates the safety of the user's home based on seismic resistance data. Then, it provides the user with the shortest and safest route to the nearest evacuation center in real time.

[0519] An example of a prompt message is: "Your current location is Minato Ward, Tokyo. You are being affected by Typhoon No. 19. Please tell me the best route to an evacuation center."

[0520] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0521] Step 1:

[0522] The user launches a dedicated application on their device and enters their address, cohabitation details, and the construction date of their residence. This entered data is sent by the device to the Firebase database and stored as user profile information.

[0523] Step 2:

[0524] The device periodically obtains the user's current location using its GPS function. The obtained location information is sent to the server as intermediate data, and the server updates the location information based on it.

[0525] Step 3:

[0526] The server acquires disaster information through APIs provided by weather stations and disaster prevention organizations. It analyzes the disaster information data received as input to understand the disaster situation in real time.

[0527] Step 4:

[0528] The server uses a generation AI model to generate an evacuation plan based on user-specific profile information, location information, and disaster information. This plan includes departure timing, necessary items to bring, and additional notes for cases requiring special assistance.

[0529] Step 5:

[0530] The server uses the Google Maps API to calculate the optimal evacuation route based on real-time traffic conditions and population density. The calculated route information is then integrated into the plan to complete comprehensive evacuation guidance.

[0531] Step 6:

[0532] The server sends the completed evacuation plan to the device using Firebase Cloud Messaging (FCM) and provides users with phased push notifications. These notifications include immediate evacuation orders and instructions for the preparation phase.

[0533] Step 7:

[0534] Users review the evacuation plan received on their device screen and begin taking action according to the instructions. This ensures a safe evacuation.

[0535] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0536] In this embodiment of the invention, an emotion engine that takes into account the user's emotional state is added to the functionality as a more personalized safety measure during disasters. This system allows the user to use a smartphone or other device to collect real-time information, including their emotions, and use it to provide advice on evacuation actions.

[0537] First, the user enters their address, cohabitation configuration, and building information via the app and registers it on the server. At this stage, the device prepares to send the user's emotional data to the emotion engine using its camera, microphone, etc. The emotion engine analyzes this emotional data and evaluates the level of stress and anxiety the user exhibits when faced with an emergency.

[0538] In the event of a disaster, the server acquires disaster information in real time and generates an appropriate evacuation plan based on location information and emotional data sent by the user. The data provided by the emotional engine is used to carefully consider the content and timing of evacuation orders and communicate them to the user in the most optimal way to reduce stress.

[0539] As a concrete example, consider the case of an earthquake. The server will determine the magnitude of the earthquake and its impact on the user's location, and decide whether evacuation is necessary. Then, the emotion engine will analyze the user's facial expressions and tone of voice, and if it determines that the user is in a high-stress state, the server will prioritize sending push notifications with messages encouraging calm behavior or simple, reassuring instructions.

[0540] Furthermore, for users requiring special assistance, the server utilizes the results of the emotion engine's evaluation to provide functions such as notifying companions and requesting rescue, ensuring they can evacuate safely. In this way, users can always obtain information in the most optimal form and act with confidence even during disasters.

[0541] The following describes the processing flow.

[0542] Step 1:

[0543] Users enter basic information such as their address, household composition, and construction date into a smartphone app, and the device sends this information to the server. This allows basic user data to be stored on the server.

[0544] Step 2:

[0545] The device activates an emotion engine and collects emotional data through facial recognition and voice analysis of the user. This data is used to estimate the user's stress level and anxiety level.

[0546] Step 3:

[0547] In the event of a disaster, the server retrieves disaster information in real time from the APIs of external disaster information providers. This information includes the type, scale, and location of the disaster.

[0548] Step 4:

[0549] The device obtains the user's current location information using GPS and sends it to the server. Based on the registration information and location information, the server identifies disasters that the user may be affected by.

[0550] Step 5:

[0551] The server combines acquired disaster information with user registration information, location data, and emotional data, and uses a generative AI to generate the optimal evacuation plan. Emotional data is used to consider the evacuation procedures and recommendations that best suit the user's mental state.

[0552] Step 6:

[0553] The server uses the map application's API to calculate the optimal evacuation route, taking into account traffic conditions and population density. This allows users to choose a safe and time-efficient route.

[0554] Step 7:

[0555] The terminal displays information received from the server to the user. This includes information the user needs, such as details of the evacuation plan, evacuation routes, and a list of items to bring.

[0556] Step 8:

[0557] The server sends phased evacuation instructions to the user via push notifications. The content of these notifications is adjusted according to the user's emotional state; for example, a calmer tone of message is sent to a user experiencing high stress.

[0558] Step 9:

[0559] For users requiring special assistance, the server sends instructions that trigger assistance requests based on their emotional state, as well as notifications to alert their companions. This creates an environment where they can evacuate with peace of mind.

[0560] (Example 2)

[0561] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0562] In recent years, the increasing frequency of disasters has highlighted the need for evacuation plans optimized for individual circumstances. However, current systems struggle to provide evacuation instructions that take into account the psychological state of users, making it difficult to make appropriate decisions, especially in emergencies. In addition, conventional methods cannot be expected to provide a swift and appropriate response to people who require special assistance.

[0563] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0564] In this invention, the server includes means for acquiring user location information, means for collecting user emotional states, and means for receiving and storing user registration information. This makes it possible to provide optimized evacuation instructions and plans that take into account each user's emotional state. This method promotes calm and appropriate actions by users even during disasters and enables a rapid response to users who require special assistance.

[0565] "User location information" refers to data indicating the user's current location and is fundamental information for issuing appropriate evacuation orders in emergencies.

[0566] "User emotional state" refers to data that indicates the user's psychological and emotional state, and is an important factor to consider in order to maximize the effectiveness of evacuation orders.

[0567] "User registration information" refers to data that includes personal information such as address, household composition, and building information provided by the user in advance, and is basic information used in formulating evacuation plans.

[0568] "Disaster information" refers to data that shows the occurrence and scale of disasters such as earthquakes and typhoons, and is external information necessary for making decisions about evacuation actions.

[0569] An "optimal evacuation plan" is a plan that takes into account the individual circumstances and emotional state of each user and designs instructions to ensure the most effective and safe evacuation.

[0570] A "generative AI model" is a learning model that uses artificial intelligence to perform data analysis and prediction, and is a technology that supports the generation of evacuation orders based on emotional states.

[0571] "Push notifications" are a method of sending information directly from a server to a user's device, and are a technology that enables rapid communication in emergencies.

[0572] A "specialized evacuation plan for users requiring special assistance" is an evacuation plan designed to meet the specific needs of users who require special consideration, such as the elderly or people with disabilities.

[0573] In this embodiment of the invention, a system is constructed to generate personalized evacuation instructions that take into account the user's emotional state in order to provide effective evacuation support during disasters.

[0574] server

[0575] The server receives basic information and real-time location data from users and is responsible for collecting the latest disaster information from reliable sources. It also integrates an emotion engine and uses a generative AI model to analyze data based on the user's emotional state and adjust the content of evacuation orders. Furthermore, it generates an optimal evacuation action plan tailored to each user's situation and sends instructions directly to the user's device via push notification.

[0576] terminal

[0577] The terminal is a portable device (e.g., a smartphone) that the user uses daily and is equipped with a camera and microphone. This is used to acquire emotional data, including the user's facial expressions and voice, and transmit it to the server. The terminal receives instructions from the server in real time and displays alerts to the user prompting appropriate evacuation actions, either visually or audibly.

[0578] User

[0579] Users register initial information such as their address, household composition, and building details through a smartphone app. Subsequently, they provide information to the system using their camera and microphone for emotional monitoring as needed. In the event of a disaster, they strive to evacuate safely and efficiently by following instructions from their device.

[0580] Specific example

[0581] For example, in the event of an earthquake, the server acquires earthquake intensity information in real time and analyzes the impact on the user's location. Using an emotion engine, it analyzes the user's facial expressions and voice, and if it detects a high-stress state, it sends a message to the device saying, "Please stay calm. Move to a safe place immediately." This kind of personalized response helps users to take calm evacuation actions.

[0582] Example of a prompt

[0583] An example of a prompt to input into the generation AI model would be: "Generate appropriate evacuation instructions based on the user's emotional state. Input data includes the user's voice tone and facial expression during an emergency." This prompt allows the AI ​​model to generate evacuation instructions that take into account the user's psychological aspects.

[0584] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0585] Step 1:

[0586] The user launches a smartphone app, enters their address, cohabitation details, and building information on the registration screen, and sends this data to the server. The server stores the received data and builds a user profile. The input is the user's personal information, and the output is the update of the registration information database on the server side. This operation provides the basis for developing evacuation plans tailored to the user's characteristics.

[0587] Step 2:

[0588] The user uses the device's camera and microphone to capture facial expressions and voice in real time. The device preprocesses this information and sends it to the server as emotion data. The input is audio and video captured from the camera and microphone, and the output is analyzable emotion data. This operation enables more precise evacuation instructions that take the user's psychological state into account.

[0589] Step 3:

[0590] The server acquires the latest disaster information from reliable external sources while continuously monitoring the user's location. It integrates location data with sentiment data and uses a generative AI model to generate an optimal evacuation plan. Inputs include disaster information, location data, and sentiment data, while output is a personalized evacuation plan. This operation enables the provision of rapid and appropriate instructions based on the given conditions.

[0591] Step 4:

[0592] The server uses a generation AI model to create evacuation instructions for individual users based on the generated evacuation action plan. These instructions are fine-tuned according to the user's emotional state and sent to their device via push notifications. The input is the generated evacuation plan, and the output is the specific evacuation instructions sent to the user. This allows users to take the necessary actions at the appropriate time.

[0593] Step 5:

[0594] The terminal notifies the user of evacuation instructions received from the server, providing on-screen display or audio guidance. It converts the information into an optimal format within the terminal to ensure it is presented in a user-friendly manner. The input is the evacuation instruction sent from the server, and the output is specific instructions prompting the user to take action. This operation encourages calm decision-making even in emergency situations.

[0595] Step 6:

[0596] Until the disaster subsides or the user's safety is confirmed, the server and terminal will maintain communication and remain ready to provide further instructions as needed. Inputs are the user's actions and changes in external disaster information, while outputs are up-to-date information relevant to the situation. This operation ensures the user's safety throughout the entire process.

[0597] (Application Example 2)

[0598] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0599] During disasters, there is a need to provide safer and more reassuring evacuation instructions that take into account the emotional state of individual users. However, conventional evacuation instruction systems lack the functionality to consider users' emotions and psychological conditions, making it difficult to alleviate stress and anxiety during emergencies. Therefore, a new system is needed that can provide appropriate instructions tailored to the individual emotional state of each user.

[0600] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0601] In this invention, the server includes means for acquiring user location information, means for receiving and storing user registration information, and means for acquiring disaster information. This makes it possible to adjust personalized evacuation instructions by analyzing the user's emotional state.

[0602] "User location information" refers to data that indicates the user's current location, and is acquired using the device's sensors and GPS function.

[0603] "User registration information" refers to various data about the user, including personal information such as address and emergency contact information.

[0604] "Disaster information" refers to information related to natural disasters such as earthquakes and typhoons, and is data that indicates the degree of risk and impact on the affected area.

[0605] An "evacuation action plan" is a plan that outlines appropriate evacuation routes and procedures during a disaster, and includes instructions necessary to ensure the safety of users.

[0606] "Emotional state" refers to information that indicates the user's current psychological state, and is analyzed based on factors such as facial expressions and voice tone.

[0607] "Analyzing emotional state" is a process that evaluates the user's psychological state based on data from their facial expressions and voice, and determines their stress and anxiety levels.

[0608] An "evacuation order" is a series of instructions provided to ensure safe evacuation during a disaster, and includes information designed to prompt users to take action.

[0609] An "optimal evacuation route" refers to the best path for users to evacuate quickly and safely during a disaster, taking into account traffic conditions and terrain.

[0610] The system for implementing this invention works in conjunction with the user's smart device to provide personalized evacuation instructions during a disaster. It mainly consists of a server, a user-owned terminal, and an emotion analysis engine.

[0611] The server has the ability to collect disaster information in real time and combine it with the user's location and registration information. Based on the acquired data, an emotion analysis engine analyzes the user's facial expressions and voice tone to assess their stress and anxiety levels. This then generates evacuation instructions optimized for each user.

[0612] The device is equipped with a camera and microphone, and its role is to capture the user's facial expressions and voice in real time and send them to an emotion analysis engine. General emotion recognition software (such as Emotion API) is used for emotion analysis. This enables safe evacuation that takes the user's psychological state into consideration.

[0613] As a concrete example, if a user experiences an earthquake, the device immediately sends its location and facial expression data to the server. The server quickly analyzes the scale of the disaster and the user's current location, and sends a push notification of an evacuation message that provides reassurance, tailored to the user's emotional state.

[0614] Examples of prompt statements include:

[0615] "Design an app that allows users of head-mounted displays to receive instructions for safe evacuation during an earthquake. It should utilize the camera and microphone, taking into account the user's real-time emotional state."

[0616] These are some of the points that were raised.

[0617] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0618] Step 1:

[0619] The server collects disaster information and creates an up-to-date database. This information is obtained from various sensor networks and online sources. The input is external disaster information, and the output is an organized disaster information database. This enables real-time information updates.

[0620] Step 2:

[0621] The device uses its built-in camera and microphone to collect user facial expressions and voice data. The input is the user's real-time voice and video, and the output is collected emotion data. This data is used in subsequent analysis steps.

[0622] Step 3:

[0623] The device transmits collected emotional data to an emotion analysis engine. This engine analyzes the input data and quantifies the user's stress and anxiety levels. The output is a numerical indicator showing the user's current emotional state. This allows for a quantitative evaluation of the user's psychological state.

[0624] Step 4:

[0625] The server generates evacuation information based on the user's location and emotional state. The input is the user's location and emotional indicators, and the output is personalized evacuation instructions. A generative AI model is used to formulate an optimal evacuation plan that takes the user's emotions into consideration. This provides the user with the most suitable evacuation method.

[0626] Step 5:

[0627] The terminal sends evacuation instructions received from the server to the user in the form of push notifications. The input is the evacuation instructions from the server, and the output is specific instructions displayed on the terminal screen. This provides the user with information visually or audibly, enabling them to take immediate action.

[0628] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0629] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0630] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0631] [Fourth Embodiment]

[0632] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0633] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0634] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0635] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0636] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0637] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0638] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0639] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0640] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0641] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0642] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0643] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0644] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0645] In this embodiment of the invention, the user first registers their address, household composition, and the construction date of their residence using a dedicated application via their smartphone or compatible device. This registration information is transmitted to a server and stored in a database.

[0646] Next, in the event of a disaster, the server acquires disaster information based on a pre-configured schedule or a trigger indicating the occurrence of a disaster. This disaster information is obtained in real time through APIs provided by weather stations and disaster information agencies. The server analyzes this information to identify the type and location of the disaster relevant to the user.

[0647] Furthermore, the user's device periodically or under specific conditions utilizes GPS to measure the user's current location and transmit it to the server. Based on this location information, the server determines the extent of any disasters that may affect the user.

[0648] The server uses AI to generate an optimal evacuation plan based on user registration information, current location information, and acquired disaster information. This plan includes the timing of evacuation, specific actions to take, and necessary preparations. In particular, it includes measures that require special consideration for households with elderly people or infants.

[0649] After an evacuation plan is generated, the server uses a map application API to calculate the optimal evacuation route, taking into account traffic conditions and population density, and sends this information to the user's device. Users can then intuitively view this detailed route information on their smartphones.

[0650] Finally, the server uses push notifications to send users step-by-step evacuation instructions, such as starting to prepare for evacuation, waiting at an evacuation site, or immediate evacuation orders. This allows users to act safely and without confusion during a disaster.

[0651] As a concrete example, consider a case where a large-scale earthquake occurs in a certain area. The server quickly acquires earthquake information and provides registered users within the affected area with the optimal action plan based on the seismic resistance of each house, taking into account building information. Furthermore, it suggests the shortest and safest route to evacuation shelters, reflecting real-time traffic information and passability. As a result, users can be expected to complete their evacuation safely.

[0652] The following describes the processing flow.

[0653] Step 1:

[0654] The user launches the smartphone app, enters personal information such as address, household composition, and construction date, and presses the registration button. The app then sends this information to the server.

[0655] Step 2:

[0656] The server receives the registration information and saves it to the database. The saved information is managed in association with the user ID.

[0657] Step 3:

[0658] The server acquires disaster information. It accesses APIs from weather stations and disaster information agencies to collect real-time disaster information and analyzes that data.

[0659] Step 4:

[0660] The device obtains the user's location information using its GPS function. The obtained location information is then sent to the server.

[0661] Step 5:

[0662] The server uses the user's location information to assess the characteristics of the area and the potential impact of disasters. By comparing this information with the acquired disaster data, it identifies disasters that the user may be affected by.

[0663] Step 6:

[0664] The server uses AI generation to create an optimal evacuation plan based on the user's registration information, location information, and acquired disaster information. This plan includes the timing of evacuation and specific action suggestions.

[0665] Step 7:

[0666] The server uses the map application's API to calculate the optimal evacuation route, taking into account current traffic conditions and population density. The calculated route information is then sent to the terminal.

[0667] Step 8:

[0668] The terminal displays evacuation route information received from the server to the user. Based on this information, the user prepares for evacuation by intuitively reviewing it.

[0669] Step 9:

[0670] The server uses push notifications to send users step-by-step evacuation instructions, such as when to begin preparing to evacuate and when to move. The instructions are adjusted according to the situation to ensure that users can evacuate safely.

[0671] (Example 1)

[0672] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0673] Providing prompt and accurate evacuation plans during disasters is crucial for saving many lives. However, providing appropriate information and instructions tailored to individual circumstances is difficult, and support for the elderly and those requiring special assistance is particularly inadequate. There is a need to address this challenge and provide users with personalized, safe, and efficient evacuation solutions.

[0674] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0675] In this invention, the server includes means for acquiring user spatial information, means for receiving and storing user registration information, and means for acquiring disaster-related information. This makes it possible to generate evacuation action plans optimized for individual users and to provide appropriate support to users who require special assistance.

[0676] A "user" refers to an individual or organization that uses the system and provides spatial information or registration information.

[0677] "Spatial information" refers to data that indicates the user's current location. This includes digital geographic information such as latitude and longitude.

[0678] "Registered information" refers to personal information that users have provided to the system in advance, such as their address, household composition, and the construction date of their residence.

[0679] "Disaster information" refers to data related to natural disasters, including real-time information provided by weather stations and disaster information agencies.

[0680] A "generative AI model" refers to an applied artificial intelligence algorithm used to generate the optimal output from specific input data. In this case, it is used to create an optimal evacuation plan.

[0681] An "evacuation action plan" refers to a plan that outlines the evacuation procedures and guidelines that users should follow in the event of a disaster.

[0682] An "evacuation route" refers to an optimized path for users to safely move to an evacuation shelter. This route is calculated taking into account real-time traffic conditions and population density.

[0683] "Notification technology" refers to a method of sending information to users in stages. This is used as part of providing evacuation instructions during disasters.

[0684] "Users requiring assistance" refers to users who require additional support during normal evacuation procedures, including the elderly and individuals with physical disabilities.

[0685] In this embodiment of the invention, the user, terminal, and server work together to achieve safe and effective evacuation during a disaster. The user uses a dedicated application via a smartphone or other compatible terminal. This application includes an interface for inputting registration information such as the user's address, household composition, and the construction date of the residence.

[0686] The terminal plays the role of transmitting information entered by the user to the server. The user's location information is periodically acquired using the terminal's GPS function and transmitted to the server. This allows the server to understand the user's spatial information in real time.

[0687] The server stores this registration information and location data in a database and then uses APIs obtained from weather stations and disaster information agencies to acquire disaster-related information. Using a generative AI model, it generates an optimal evacuation plan for each user based on prompt messages. This AI model operates by considering the user's registration information, current situation, and the type and scale of the disaster. For example, a possible prompt message might be: "Address: Shinjuku-ku, Tokyo; Years of residence: 5 years; Presence of elderly: Yes. An earthquake with a seismic intensity of 6- is currently occurring. Please generate the optimal evacuation plan."

[0688] Furthermore, the server uses the map application's API to calculate the optimal evacuation route, taking into account factors such as traffic conditions and population density. The calculated information is sent to the user's device via push notifications as phased evacuation instructions. This allows users to take intuitive and rapid evacuation action.

[0689] As a concrete example, if a user experiences an earthquake in a certain area, the server first acquires earthquake information and provides the user within the affected area with the most suitable action plan. If elderly people or infants are accompanying the user, a plan including special considerations is generated. Evacuation routes based on traffic conditions are also provided, allowing users to avoid congestion and reach evacuation shelters safely.

[0690] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0691] Step 1:

[0692] Users access a dedicated application using a smartphone or other device. There, they enter registration information such as their address, household composition, and the construction date of their residence. This input information is sent from the device to the server. The server receives this information and stores it in a database. The input consists of the user's registration information, and the output is recorded in the form of information stored in the database.

[0693] Step 2:

[0694] The server periodically retrieves disaster information from weather stations and disaster information agencies via APIs. This is done based on specified schedules or real-time triggers. The input for this process is the disaster information obtained via APIs, which is then processed into the required format for analysis. As a result, the retrieved disaster information is stored on the server.

[0695] Step 3:

[0696] The device uses its built-in GPS function to periodically obtain the user's current location information. This location information is sent to the server with the user's permission. The input is the current location data, and the server uses this information to obtain output that confirms the user's current location.

[0697] Step 4:

[0698] The server uses the acquired user registration information, disaster information, and current location information to input prompt messages into the generating AI model. An example of such a prompt message might be: "Address: Shinjuku-ku, Tokyo; Years of residence: 5 years; Presence of elderly: Yes. An earthquake of magnitude 6- is currently occurring. Please generate the optimal evacuation plan." The generating AI model then creates an optimal evacuation plan based on the input data, and the plan is output.

[0699] Step 5:

[0700] The server utilizes the API of a map application to calculate the optimal evacuation route, taking into account traffic conditions and population density. Input includes current road information and traffic conditions, and the optimal route is output based on this information.

[0701] Step 6:

[0702] The server sends an evacuation plan and evacuation route to the user's device via push notification. The user receives the notification and gets specific instructions to begin evacuating safely. The server sends the destination and content of the push notification, and the notification is output to the user as a reference for their actions.

[0703] (Application Example 1)

[0704] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0705] In the event of a natural disaster, there is a need to provide prompt and appropriate evacuation instructions tailored to the individual circumstances of each user. In particular, there is a need for individually tailored action plans for people requiring special assistance, such as the elderly and people with disabilities, and there is a lack of systems and equipment to provide this. Furthermore, providing evacuation routes that appropriately reflect real-time changes in traffic conditions and population density is another crucial issue that needs to be addressed.

[0706] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0707] In this invention, the server includes a device for receiving user location information, a device for acquiring and storing user profile information, and a device for acquiring natural disaster data. This makes it possible to provide optimal evacuation routes and action plans tailored to individual circumstances.

[0708] "User location information" refers to data that indicates the geographical coordinates of a moving object or individual at their current location.

[0709] "User profile information" refers to data that includes individual attribute information about each user, such as address, cohabitation status, and construction date.

[0710] "Natural disaster data" refers to data provided by meteorological observation agencies and disaster prevention agencies, including information on disasters such as earthquakes, typhoons, and floods.

[0711] An "evacuation action plan" is a plan that includes specific procedures and instructions for users to evacuate safely in the event of a disaster.

[0712] A "route guidance device" is a device that calculates and displays the optimal route for guiding users between specified points.

[0713] "Communication methods" refer to technologies and devices for sending and receiving information, including those that utilize the internet and mobile networks.

[0714] A "crisis management system" is a system that plans, implements, and evaluates a series of measures to protect lives and property in the event of a disaster or sudden accident.

[0715] The system for implementing this invention aims to provide evacuation action plans tailored to the individual circumstances of users, thereby supporting safer evacuation during disasters. This system is implemented using mobile terminals, including smartphones, servers, and network communication means connecting them.

[0716] First, users register profile information such as their address, cohabitation status, and the construction date of their residence using a dedicated application on their smartphone. This information is transmitted to the server via the device and stored in a server-side database. This database uses cloud data storage such as Firebase.

[0717] Next, the server uses APIs provided by weather stations and disaster prevention agencies to acquire real-time natural disaster data. This data includes information on earthquakes, typhoons, and floods, which can be used to facilitate a rapid response in the event of the next disaster.

[0718] Furthermore, the user's device utilizes GPS functionality to periodically obtain the user's current location and transmit it to the server. Location information is obtained and managed by the Google Maps API.

[0719] Based on this location information, profile information, and natural disaster data, the server uses a generative AI model to generate individually optimized evacuation action plans for each user. These plans are dynamically generated using APIs such as OpenAI.

[0720] Furthermore, the system uses route guidance devices to calculate the optimal evacuation route, taking into account real-time traffic conditions and population density. This function is also implemented using the Google Maps API.

[0721] Finally, Firebase Cloud Messaging (FCM) will be used as the communication method to send phased evacuation instructions to users via push notifications. These notifications will include specific action instructions tailored to the timing and progress of the evacuation.

[0722] As a concrete example, when an earthquake occurs in a certain area, the server quickly collects earthquake information and evaluates the safety of the user's home based on seismic resistance data. Then, it provides the user with the shortest and safest route to the nearest evacuation center in real time.

[0723] An example of a prompt message is: "Your current location is Minato Ward, Tokyo. You are being affected by Typhoon No. 19. Please tell me the best route to an evacuation center."

[0724] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0725] Step 1:

[0726] The user launches a dedicated application on their device and enters their address, cohabitation details, and the construction date of their residence. This entered data is sent by the device to the Firebase database and stored as user profile information.

[0727] Step 2:

[0728] The device periodically obtains the user's current location using its GPS function. The obtained location information is sent to the server as intermediate data, and the server updates the location information based on it.

[0729] Step 3:

[0730] The server acquires disaster information through APIs provided by weather stations and disaster prevention organizations. It analyzes the disaster information data received as input to understand the disaster situation in real time.

[0731] Step 4:

[0732] The server uses a generation AI model to generate an evacuation plan based on user-specific profile information, location information, and disaster information. This plan includes departure timing, necessary items to bring, and additional notes for cases requiring special assistance.

[0733] Step 5:

[0734] The server uses the Google Maps API to calculate the optimal evacuation route based on real-time traffic conditions and population density. The calculated route information is then integrated into the plan to complete comprehensive evacuation guidance.

[0735] Step 6:

[0736] The server sends the completed evacuation plan to the device using Firebase Cloud Messaging (FCM) and provides users with phased push notifications. These notifications include immediate evacuation orders and instructions for the preparation phase.

[0737] Step 7:

[0738] Users review the evacuation plan received on their device screen and begin taking action according to the instructions. This ensures a safe evacuation.

[0739] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0740] In this embodiment of the invention, an emotion engine that takes into account the user's emotional state is added to the functionality as a more personalized safety measure during disasters. This system allows the user to use a smartphone or other device to collect real-time information, including their emotions, and use it to provide advice on evacuation actions.

[0741] First, the user enters their address, cohabitation configuration, and building information via the app and registers it on the server. At this stage, the device prepares to send the user's emotional data to the emotion engine using its camera, microphone, etc. The emotion engine analyzes this emotional data and evaluates the level of stress and anxiety the user exhibits when faced with an emergency.

[0742] In the event of a disaster, the server acquires disaster information in real time and generates an appropriate evacuation plan based on location information and emotional data sent by the user. The data provided by the emotional engine is used to carefully consider the content and timing of evacuation orders and communicate them to the user in the most optimal way to reduce stress.

[0743] As a concrete example, consider the case of an earthquake. The server will determine the magnitude of the earthquake and its impact on the user's location, and decide whether evacuation is necessary. Then, the emotion engine will analyze the user's facial expressions and tone of voice, and if it determines that the user is in a high-stress state, the server will prioritize sending push notifications with messages encouraging calm behavior or simple, reassuring instructions.

[0744] Furthermore, for users requiring special assistance, the server utilizes the results of the emotion engine's evaluation to provide functions such as notifying companions and requesting rescue, ensuring they can evacuate safely. In this way, users can always obtain information in the most optimal form and act with confidence even during disasters.

[0745] The following describes the processing flow.

[0746] Step 1:

[0747] Users enter basic information such as their address, household composition, and construction date into a smartphone app, and the device sends this information to the server. This allows basic user data to be stored on the server.

[0748] Step 2:

[0749] The device activates an emotion engine and collects emotional data through facial recognition and voice analysis of the user. This data is used to estimate the user's stress level and anxiety level.

[0750] Step 3:

[0751] In the event of a disaster, the server retrieves disaster information in real time from the APIs of external disaster information providers. This information includes the type, scale, and location of the disaster.

[0752] Step 4:

[0753] The device obtains the user's current location information using GPS and sends it to the server. Based on the registration information and location information, the server identifies disasters that the user may be affected by.

[0754] Step 5:

[0755] The server combines acquired disaster information with user registration information, location data, and emotional data, and uses a generative AI to generate the optimal evacuation plan. Emotional data is used to consider the evacuation procedures and recommendations that best suit the user's mental state.

[0756] Step 6:

[0757] The server uses the map application's API to calculate the optimal evacuation route, taking into account traffic conditions and population density. This allows users to choose a safe and time-efficient route.

[0758] Step 7:

[0759] The terminal displays information received from the server to the user. This includes information the user needs, such as details of the evacuation plan, evacuation routes, and a list of items to bring.

[0760] Step 8:

[0761] The server sends phased evacuation instructions to the user via push notifications. The content of these notifications is adjusted according to the user's emotional state; for example, a calmer tone of message is sent to a user experiencing high stress.

[0762] Step 9:

[0763] For users requiring special assistance, the server sends instructions that trigger assistance requests based on their emotional state, as well as notifications to alert their companions. This creates an environment where they can evacuate with peace of mind.

[0764] (Example 2)

[0765] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0766] In recent years, the increasing frequency of disasters has highlighted the need for evacuation plans optimized for individual circumstances. However, current systems struggle to provide evacuation instructions that take into account the psychological state of users, making it difficult to make appropriate decisions, especially in emergencies. In addition, conventional methods cannot be expected to provide a swift and appropriate response to people who require special assistance.

[0767] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0768] In this invention, the server includes means for acquiring user location information, means for collecting user emotional states, and means for receiving and storing user registration information. This makes it possible to provide optimized evacuation instructions and plans that take into account each user's emotional state. This method promotes calm and appropriate actions by users even during disasters and enables a rapid response to users who require special assistance.

[0769] "User location information" refers to data indicating the user's current location and is fundamental information for issuing appropriate evacuation orders in emergencies.

[0770] "User emotional state" refers to data that indicates the user's psychological and emotional state, and is an important factor to consider in order to maximize the effectiveness of evacuation orders.

[0771] "User registration information" refers to data that includes personal information such as address, household composition, and building information provided by the user in advance, and is basic information used in formulating evacuation plans.

[0772] "Disaster information" refers to data that shows the occurrence and scale of disasters such as earthquakes and typhoons, and is external information necessary for making decisions about evacuation actions.

[0773] An "optimal evacuation plan" is a plan that takes into account the individual circumstances and emotional state of each user and designs instructions to ensure the most effective and safe evacuation.

[0774] A "generative AI model" is a learning model that uses artificial intelligence to perform data analysis and prediction, and is a technology that supports the generation of evacuation orders based on emotional states.

[0775] "Push notifications" are a method of sending information directly from a server to a user's device, and are a technology that enables rapid communication in emergencies.

[0776] A "specialized evacuation plan for users requiring special assistance" is an evacuation plan designed to meet the specific needs of users who require special consideration, such as the elderly or people with disabilities.

[0777] In this embodiment of the invention, a system is constructed to generate personalized evacuation instructions that take into account the user's emotional state in order to provide effective evacuation support during disasters.

[0778] server

[0779] The server receives basic information and real-time location data from users and is responsible for collecting the latest disaster information from reliable sources. It also integrates an emotion engine and uses a generative AI model to analyze data based on the user's emotional state and adjust the content of evacuation orders. Furthermore, it generates an optimal evacuation action plan tailored to each user's situation and sends instructions directly to the user's device via push notification.

[0780] terminal

[0781] The terminal is a portable device (e.g., a smartphone) that the user uses daily and is equipped with a camera and microphone. This is used to acquire emotional data, including the user's facial expressions and voice, and transmit it to the server. The terminal receives instructions from the server in real time and displays alerts to the user prompting appropriate evacuation actions, either visually or audibly.

[0782] User

[0783] Users register initial information such as their address, household composition, and building details through a smartphone app. Subsequently, they provide information to the system using their camera and microphone for emotional monitoring as needed. In the event of a disaster, they strive to evacuate safely and efficiently by following instructions from their device.

[0784] Specific example

[0785] For example, in the event of an earthquake, the server acquires earthquake intensity information in real time and analyzes the impact on the user's location. Using an emotion engine, it analyzes the user's facial expressions and voice, and if it detects a high-stress state, it sends a message to the device saying, "Please stay calm. Move to a safe place immediately." This kind of personalized response helps users to take calm evacuation actions.

[0786] Example of a prompt

[0787] An example of a prompt to input into the generation AI model would be: "Generate appropriate evacuation instructions based on the user's emotional state. Input data includes the user's voice tone and facial expression during an emergency." This prompt allows the AI ​​model to generate evacuation instructions that take into account the user's psychological aspects.

[0788] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0789] Step 1:

[0790] The user launches a smartphone app, enters their address, cohabitation details, and building information on the registration screen, and sends this data to the server. The server stores the received data and builds a user profile. The input is the user's personal information, and the output is the update of the registration information database on the server side. This operation provides the basis for developing evacuation plans tailored to the user's characteristics.

[0791] Step 2:

[0792] The user uses the device's camera and microphone to capture facial expressions and voice in real time. The device preprocesses this information and sends it to the server as emotion data. The input is audio and video captured from the camera and microphone, and the output is analyzable emotion data. This operation enables more precise evacuation instructions that take the user's psychological state into account.

[0793] Step 3:

[0794] The server acquires the latest disaster information from reliable external sources while continuously monitoring the user's location. It integrates location data with sentiment data and uses a generative AI model to generate an optimal evacuation plan. Inputs include disaster information, location data, and sentiment data, while output is a personalized evacuation plan. This operation enables the provision of rapid and appropriate instructions based on the given conditions.

[0795] Step 4:

[0796] The server uses a generation AI model to create evacuation instructions for individual users based on the generated evacuation action plan. These instructions are fine-tuned according to the user's emotional state and sent to their device via push notifications. The input is the generated evacuation plan, and the output is the specific evacuation instructions sent to the user. This allows users to take the necessary actions at the appropriate time.

[0797] Step 5:

[0798] The terminal notifies the user of evacuation instructions received from the server, providing on-screen display or audio guidance. It converts the information into an optimal format within the terminal to ensure it is presented in a user-friendly manner. The input is the evacuation instruction sent from the server, and the output is specific instructions prompting the user to take action. This operation encourages calm decision-making even in emergency situations.

[0799] Step 6:

[0800] Until the disaster subsides or the user's safety is confirmed, the server and terminal will maintain communication and remain ready to provide further instructions as needed. Inputs are the user's actions and changes in external disaster information, while outputs are up-to-date information relevant to the situation. This operation ensures the user's safety throughout the entire process.

[0801] (Application Example 2)

[0802] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0803] During disasters, there is a need to provide safer and more reassuring evacuation instructions that take into account the emotional state of individual users. However, conventional evacuation instruction systems lack the functionality to consider users' emotions and psychological conditions, making it difficult to alleviate stress and anxiety during emergencies. Therefore, a new system is needed that can provide appropriate instructions tailored to the individual emotional state of each user.

[0804] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0805] In this invention, the server includes means for acquiring user location information, means for receiving and storing user registration information, and means for acquiring disaster information. This makes it possible to adjust personalized evacuation instructions by analyzing the user's emotional state.

[0806] "User location information" refers to data that indicates the user's current location, and is acquired using the device's sensors and GPS function.

[0807] "User registration information" refers to various data about the user, including personal information such as address and emergency contact information.

[0808] "Disaster information" refers to information related to natural disasters such as earthquakes and typhoons, and is data that indicates the degree of risk and impact on the affected area.

[0809] An "evacuation action plan" is a plan that outlines appropriate evacuation routes and procedures during a disaster, and includes instructions necessary to ensure the safety of users.

[0810] "Emotional state" refers to information that indicates the user's current psychological state, and is analyzed based on factors such as facial expressions and voice tone.

[0811] "Analyzing emotional state" is a process that evaluates the user's psychological state based on data from their facial expressions and voice, and determines their stress and anxiety levels.

[0812] An "evacuation order" is a series of instructions provided to ensure safe evacuation during a disaster, and includes information designed to prompt users to take action.

[0813] An "optimal evacuation route" refers to the best path for users to evacuate quickly and safely during a disaster, taking into account traffic conditions and terrain.

[0814] The system for implementing this invention works in conjunction with the user's smart device to provide personalized evacuation instructions during a disaster. It mainly consists of a server, a user-owned terminal, and an emotion analysis engine.

[0815] The server has the ability to collect disaster information in real time and combine it with the user's location and registration information. Based on the acquired data, an emotion analysis engine analyzes the user's facial expressions and voice tone to assess their stress and anxiety levels. This then generates evacuation instructions optimized for each user.

[0816] The device is equipped with a camera and microphone, and its role is to capture the user's facial expressions and voice in real time and send them to an emotion analysis engine. General emotion recognition software (such as Emotion API) is used for emotion analysis. This enables safe evacuation that takes the user's psychological state into consideration.

[0817] As a concrete example, if a user experiences an earthquake, the device immediately sends its location and facial expression data to the server. The server quickly analyzes the scale of the disaster and the user's current location, and sends a push notification of an evacuation message that provides reassurance, tailored to the user's emotional state.

[0818] Examples of prompt statements include:

[0819] "Design an app that allows users of head-mounted displays to receive instructions for safe evacuation during an earthquake. It should utilize the camera and microphone, taking into account the user's real-time emotional state."

[0820] These are some of the points that were raised.

[0821] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0822] Step 1:

[0823] The server collects disaster information and creates an up-to-date database. This information is obtained from various sensor networks and online sources. The input is external disaster information, and the output is an organized disaster information database. This enables real-time information updates.

[0824] Step 2:

[0825] The device uses its built-in camera and microphone to collect user facial expressions and voice data. The input is the user's real-time voice and video, and the output is collected emotion data. This data is used in subsequent analysis steps.

[0826] Step 3:

[0827] The device transmits collected emotional data to an emotion analysis engine. This engine analyzes the input data and quantifies the user's stress and anxiety levels. The output is a numerical indicator showing the user's current emotional state. This allows for a quantitative evaluation of the user's psychological state.

[0828] Step 4:

[0829] The server generates evacuation information based on the user's location and emotional state. The input is the user's location and emotional indicators, and the output is personalized evacuation instructions. A generative AI model is used to formulate an optimal evacuation plan that takes the user's emotions into consideration. This provides the user with the most suitable evacuation method.

[0830] Step 5:

[0831] The terminal sends evacuation instructions received from the server to the user in the form of push notifications. The input is the evacuation instructions from the server, and the output is specific instructions displayed on the terminal screen. This provides the user with information visually or audibly, enabling them to take immediate action.

[0832] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0833] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0834] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0835] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0836] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0837] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0838] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0839] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0840] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0841] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0842] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0843] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0844] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0845] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0846] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0847] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0848] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0849] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0850] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0851] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0852] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0853] The following is further disclosed regarding the embodiments described above.

[0854] (Claim 1)

[0855] Means for obtaining the user's location information,

[0856] A means of receiving and storing user registration information,

[0857] Means of obtaining disaster information,

[0858] A means for generating an optimal evacuation action plan based on acquired location information, registration information, and disaster information,

[0859] A means of calculating the optimal evacuation route,

[0860] A means of sending evacuation orders in stages via push notifications,

[0861] A means of providing a specialized evacuation action plan for users who require special assistance,

[0862] A system that includes this.

[0863] (Claim 2)

[0864] The system according to claim 1, comprising means for optimizing evacuation routes in consideration of current traffic conditions and population density.

[0865] (Claim 3)

[0866] The system according to claim 1, comprising means for collecting location information by communicating with a device owned by the user.

[0867] "Example 1"

[0868] (Claim 1)

[0869] Means for acquiring the user's spatial information,

[0870] A means of receiving and storing user registration information,

[0871] Means of obtaining information about disasters,

[0872] To generate an optimal evacuation action plan based on acquired spatial information, registration information, and disaster information, a method using a generative AI model is employed.

[0873] A means of calculating the optimal evacuation route,

[0874] A means of transmitting evacuation orders in stages using notification technology,

[0875] A means of providing a specialized evacuation action plan for users who require special assistance,

[0876] A system that includes this.

[0877] (Claim 2)

[0878] The system according to claim 1, comprising means for optimizing evacuation routes in consideration of current movement conditions and population density.

[0879] (Claim 3)

[0880] The system according to claim 1, comprising means for collecting spatial information by communicating with a device owned by the user.

[0881] "Application Example 1"

[0882] (Claim 1)

[0883] A device that receives user location information,

[0884] A device that acquires and stores user profile information,

[0885] A device for acquiring natural disaster data,

[0886] A device that generates an optimal evacuation action plan based on acquired location information, profile information, and natural disaster data,

[0887] A device that calculates the optimal evacuation route using a route guidance system,

[0888] A device that transmits evacuation orders in stages via communication means,

[0889] A device that provides a specialized evacuation action plan for users who require special assistance,

[0890] A device that is linked to the crisis management system,

[0891] A network that includes this.

[0892] (Claim 2)

[0893] The network according to claim 1, which includes a process for optimizing travel routes in consideration of real-time traffic conditions and human movement data.

[0894] (Claim 3)

[0895] The network according to claim 1, which includes a process for collecting location information through an interface with an information terminal owned by the user.

[0896] "Example 2 of combining an emotion engine"

[0897] (Claim 1)

[0898] Means for obtaining the user's location information,

[0899] A means of collecting the user's emotional state,

[0900] A means of receiving and storing user registration information,

[0901] Means of obtaining disaster information,

[0902] A means for generating an optimal evacuation action plan based on acquired location information, registration information, disaster information, and emotional state,

[0903] A means of setting evacuation orders according to emotional states using a generative AI model,

[0904] A means of sending evacuation orders in stages via push notifications,

[0905] A means of providing a specialized evacuation action plan for users who require special assistance,

[0906] A system that includes this.

[0907] (Claim 2)

[0908] The system according to claim 1, comprising means for optimizing evacuation routes in consideration of current traffic conditions and population density.

[0909] (Claim 3)

[0910] The system according to claim 1, comprising means for collecting location information and sentiment data through communication with a device owned by the user.

[0911] "Application example 2 when combining with an emotional engine"

[0912] (Claim 1)

[0913] Means for obtaining the user's location information,

[0914] A means of receiving and storing user registration information,

[0915] Means of obtaining disaster information,

[0916] A means for generating an optimal evacuation action plan based on acquired location information, registration information, and disaster information,

[0917] A means of analyzing the user's emotional state,

[0918] A means of adjusting the optimal evacuation order based on the analyzed emotional state of the user,

[0919] A means of sending evacuation orders in stages via push notifications,

[0920] A means of providing a specialized evacuation action plan for users who require special assistance,

[0921] A system that includes this.

[0922] (Claim 2)

[0923] The system according to claim 1, comprising means for optimizing evacuation routes in consideration of current traffic conditions and population density.

[0924] (Claim 3)

[0925] The system according to claim 1, comprising means for collecting location information by communicating with a device owned by the user. [Explanation of Symbols]

[0926] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means for obtaining the user's location information, A means of receiving and storing user registration information, Means of obtaining disaster information, A means for generating an optimal evacuation action plan based on acquired location information, registration information, and disaster information, A means of calculating the optimal evacuation route, A means of sending evacuation orders in stages via push notifications, A means of providing a specialized evacuation action plan for users who require special assistance, A system that includes this.

2. The system according to claim 1, comprising means for optimizing evacuation routes in consideration of current traffic conditions and population density.

3. The system according to claim 1, comprising means for collecting location information by communicating with a terminal owned by the user.

Images