system

Abstract

Provide a system. [Solution] A means for acquiring information collected by users in a disaster area; A means of analyzing the information obtained and identifying the needs of the disaster area; a means of assessing and prioritizing identified needs; means for generating hearing contents based on the priority and presenting the contents to a user; a means for recording the results of the user's hearing; A means for generating a specific support plan based on the recorded interview results; A system that includes a means for sharing the generated support plan with support organizations.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When a disaster occurs, there is often a lack of information to grasp the local situation and quickly determine the necessary support. This makes it difficult to quickly and accurately grasp the needs of each region and area, making it difficult to accurately determine the priority of support. Furthermore, there is often no system in place to provide information to volunteers and support groups so that they can work efficiently. This leads to problems such as delayed support and inefficient measures. The purpose of this invention is to solve these problems. [Means for solving the problem]

[0005] In order to solve the above problems, the present invention provides the following means.

[0006] This system allows users to collect information in disaster areas, analyze this information, and identify needs. Specifically, it includes a means to analyze photos, videos, and text data collected by users and extract specific needs based on them. It also prioritizes the extracted needs, generates interview content based on the priorities, and presents it to the user. It records the results of the user's interviews, generates specific support plans based on these results, and shares them with support organizations, thereby achieving efficient support activities.

[0007] "Disaster area" refers to an area affected by a natural or man-made disaster.

[0008] "User" refers to a person who uses this system to gather information and conduct interviews within the disaster area.

[0009] "Information" includes photographs, videos, text data, and GPS data showing the situation within the disaster area.

[0010] "Means of acquisition" refers to the function that allows users to use their terminals to record information within the disaster area and input it into the system.

[0011] "Means of analysis" refers to the function of analyzing acquired information using text mining and image recognition technology to identify needs in disaster-stricken areas.

[0012] "Needs" refers to the assistance and supplies that are particularly needed by the victims and communities within the disaster area.

[0013] "Means of evaluation" refers to the ability to prioritize identified needs and determine which needs are most important.

[0014] "Hearing contents" include specific questions and survey items that the user will ask people in the disaster area.

[0015] "Means for recording" refers to a function that allows the user to save the results of the interview so that they can be referenced and analyzed later.

[0016] "Support plan" refers to a plan for specific support activities based on the results of the recorded interviews.

[0017] "Support groups" refer to organizations and groups that provide support to disaster victims and affected areas after a disaster occurs. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention is a system for realizing an immediate and efficient support system in a disaster area. A specific embodiment of this system will be described below.

[0040] User information collection

[0041] User: In the disaster area, the user uses a device to record the damage situation. The user takes photos of the scene and inputs videos and text notes. In addition, GPS data is also acquired during this collection process, so the location information of the information becomes clear.

[0042] Data transmission and analysis

[0043] Terminal: The collected information is packaged into a single packet and sent to the server. Security technology (e.g., HTTPS) is used for communication, ensuring data security.

[0044] Server: The server analyzes the received packets and applies natural language processing (NLP) technology to the text data and machine learning image recognition technology to the image data, thereby identifying specific needs within the disaster area (e.g., food shortages, lack of shelter, need for medical assistance).

[0045] Prioritizing needs

[0046] Server: The extracted needs are passed through an evaluation algorithm to determine priorities. For example, a list may be generated based on the scale, severity, and extent of impact of the disaster, such as "food shortages (high priority)," "need for medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[0047] User interviews

[0048] Users: Through the system interface, they select a high-priority issue, such as "food shortage," and then conduct surveys and interviews with local people. This allows them to understand specific requests and detailed local conditions.

[0049] Saving the results of the hearing

[0050] Terminal: The user's hearing results are sent to the server, including text and audio data.

[0051] Server: The server stores the received hearing results in a database for future reference and analysis.

[0052] Generate and share support plans

[0053] Server: Based on the stored data, it automatically generates specific assistance plans, such as securing food supply routes, setting up evacuation shelters, and dispatching medical teams.

[0054] Server: The generated support plans are shared with volunteer groups and support organizations, enabling them to carry out support activities quickly and effectively.

[0055] This system will enable rapid and efficient emergency response in the event of a disaster. Specifically, by immediately grasping the local situation and appropriately prioritizing the necessary assistance, it will be possible to provide assistance without delay or waste. Such a system is expected to save many lives and contribute to the early recovery of disaster-stricken areas.

[0056] The processing flow will be explained below.

[0057] Step 1:

[0058] User: In the disaster area, the user uses the device to record the damage situation. Specifically, the user takes photos and videos of the damaged buildings and inputs text notes. GPS data is also automatically acquired during this process.

[0059] Step 2:

[0060] Device: Collected photos, videos, text data, and GPS data are packaged into a single data packet, which is then sent from the device to the server using a secure communication protocol (e.g., HTTPS).

[0061] Step 3:

[0062] Server: Analyzes the received data packets, applying natural language processing (NLP) techniques to text data and image recognition techniques using machine learning to image data. This identifies specific needs in the disaster area (e.g., food shortages, lack of evacuation shelters, need for medical assistance).

[0063] Step 4:

[0064] Server: Prioritizes the identified needs based on an evaluation algorithm. Evaluation criteria include the scale, severity, and scope of impact of the disaster. For example, priorities are determined as "food shortages (high priority)," "need for medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[0065] Step 5:

[0066] Server: Generate interview content based on prioritized needs. For example, regarding "food shortages," create questions about what is lacking and what specific support is needed.

[0067] Step 6:

[0068] Users: Through the system interface, they can display the generated interview content and conduct surveys and interviews with local people, thereby gaining a better understanding of specific requirements and detailed on-site conditions.

[0069] Step 7:

[0070] Terminal: The user sends the collected hearing results back to the server, including text data and audio data.

[0071] Step 8:

[0072] Server: Stores received hearing results in a database for later reference and further analysis.

[0073] Step 9:

[0074] Server: Based on the stored interview data, the server automatically generates specific assistance plans, including, for example, securing food supply routes, setting up evacuation shelters, and dispatching medical teams.

[0075] Step 10:

[0076] Server: Shares the generated support plan with relevant volunteer groups and support organizations. Sharing includes email notifications and uploading information to a web portal. Based on this information, volunteer groups and support organizations can quickly begin responding.

[0077] Example 1

[0078] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0079] In order to provide appropriate assistance immediately at disaster sites, it is necessary to quickly and accurately grasp the detailed situation on the ground and identify specific assistance needs. However, conventional methods take time to collect information, making it difficult to quickly determine the priorities of needs. Furthermore, there has been a lack of systems that can effectively organize collected information, automatically generate specific assistance plans, and share them with assistance organizations. The objective of this invention is to solve these problems.

[0080] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0081] In this invention, the server includes: means for acquiring information collected by users using their terminals in the disaster area; means for assembling the acquired information into a single data packet and transmitting it to the server; means for analyzing text data using natural language processing technology and image data using machine learning technology; means for identifying specific needs in the disaster area from the analyzed data; means for prioritizing the identified needs using an evaluation algorithm; means for generating interview content based on the priorities and presenting it to the user; means for recording the results of the interview conducted by the user and transmitting them to the server; means for the server to generate a specific support plan based on the recorded results of the interview; and means for sharing the generated support plan with support organizations. This enables rapid and effective information collection and needs identification when a disaster occurs, automatically generating an appropriate support plan based on priorities, and providing efficient support.

[0082] A "disaster area" is a specific area that has suffered damage or suffering as a result of a natural or man-made disaster.

[0083] A "user" is a person or organization that uses a terminal to collect information within the disaster area.

[0084] A "terminal" is an electronic device such as a mobile phone, smartphone, or tablet that a user uses to gather information or conduct interviews.

[0085] A "data packet" is a single integrated data set that includes text data, media data, and location information collected by a terminal.

[0086] A "server" is a central processing unit or system that receives collected data packets and performs analysis and generates support plans.

[0087] "Natural language processing technology" is a technology that allows computers to understand and analyze language spoken by humans.

[0088] "Machine learning technology" is a technology in which a computer learns from past data and performs highly accurate analysis and predictions on new data.

[0089] "Needs" are the identified needs for goods and services in the disaster area.

[0090] "Priority" is an order determined based on the importance or urgency of identified needs.

[0091] "Hearing contents" are questions and survey items that users use when conducting questionnaires or interviews with local people.

[0092] "Hearing results" are data obtained as a result of questionnaires or interviews with users.

[0093] A "support plan" is a plan for specific support measures generated by the server, and includes food supplies, medical support, and the establishment of evacuation shelters.

[0094] "Support groups" are professional organizations or volunteer groups that provide support to disaster areas.

[0095] The present invention is a system for realizing an immediate and efficient support system in a disaster area. A specific embodiment of this system will be described below.

[0096] User information collection

[0097] Users record the damage situation in the disaster area using devices such as smartphones and tablets. Specifically, users take photos of the scene with their smartphone's camera and enter text notes using the device's application. This process also utilizes the device's GPS function, and location information is automatically included in the collected information.

[0098] Data packet generation

[0099] The device combines text notes entered by the user, photos and videos taken, and location information into a single data packet, which also contains checks to ensure that each piece of data is included correctly.

[0100] Sending data

[0101] The terminal sends the generated data packet to the server using the HTTPS protocol. Before sending, the data is encrypted to ensure the security of the transmission. It also records a log of whether the transmission was successful or not.

[0102] Data reception and analysis

[0103] The server receives data packets sent from the device. After receiving the data, it first decodes the data and then analyzes the text data using natural language processing (NLP) technology. For example, NLP technologies such as TENSORFLOW (registered trademark) and SpaCy can be used. Furthermore, image and video data are analyzed using machine learning models (e.g., ResNet and YOLO) to extract important information such as "collapsed buildings" and "injured people."

[0104] Identifying and prioritizing needs

[0105] The server uses the analyzed data to identify specific needs within the disaster area. For example, needs such as "food shortages" and "medical assistance required" are extracted. An evaluation algorithm (e.g., AHP, Point Ranking System) is then used to determine the priority of the identified needs. Here, taking into account factors such as the scale and severity of the disaster, the server lists the needs as "food shortages (high priority)," "medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[0106] User interviews

[0107] Through the system interface, users select a high-priority need (e.g., "food shortage") and then survey or interview local people. The survey and interview contents are recorded as text and audio data. Specific prompts could include questions such as, "What food is most needed in this area?"

[0108] Saving the results of the hearing

[0109] The device then sends the collected data to the server, again using the HTTPS protocol for secure and reliable data transmission. The server then stores the data in a database for future analysis and reference.

[0110] Generate and share support plans

[0111] The server generates a specific relief plan based on the results of the stored interviews. For example, the server automatically formulates things like "securing food supply routes," "location of evacuation shelters," and "medical team dispatch plans." This relief plan may also be created using a generative AI model (e.g., a GPT-based model). The generated relief plan is shared with relief groups and related organizations via email or a dedicated relief management system.

[0112] Specific examples

[0113] For example, a user in an area affected by an earthquake uses a smartphone to record the damage situation. They take photos and videos, enter a text note saying "There is a food shortage in this area," and send the data to a server. The server analyzes the received data and identifies food shortages as a high-priority need. To gain a more detailed understanding of the situation, the user uses a tablet to survey residents and sends the results to the server. The server then uses this information to generate a food supply assistance plan and shares it with relief organizations.

[0114] Prompt Sentence Examples

[0115] "Analyze the provided image and text data to identify needs within the disaster area. Also, prioritize the identified needs and generate a specific support plan."

[0116] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0117] Step 1:

[0118] Users use devices such as smartphones and tablets in disaster areas to collect disaster information. Input includes taking images and videos, entering text notes, and acquiring GPS data. Specifically, users launch the smartphone's camera app and take photos of the local situation. They also enter content such as "There is a food shortage in this area" into a text input field within the application. This records detailed information about the disaster site on the device. Output is a set of image data, video data, text notes, and GPS data.

[0119] Step 2:

[0120] The device combines the collected image data, video data, text memos, and GPS data into a single data packet. The input is the various data acquired in step 1. Specifically, the device converts the various data into a packet format and performs an error check. For example, it checks whether the data is stored correctly and records a log of the packet generation. The output is the combined data packet.

[0121] Step 3:

[0122] The terminal sends the generated data packet to the server using the HTTPS protocol. The input is the data packet generated in step 2. Specifically, the terminal first encrypts the data packet, and then starts sending it to the server using HTTPS. A log of whether the transmission was successful or failed is also recorded at the same time. The output is the data packet sent to the server and its transmission log.

[0123] Step 4:

[0124] The server receives and analyzes the data packets sent from the device. The input is the data packet sent in step 3. Specifically, the server first decodes the data and then analyzes the text data using natural language processing (NLP) techniques. For example, NLP techniques such as TensorFlow and SpaCy are used to extract important keywords and phrases from the text notes. It also analyzes images and videos using machine learning models (e.g., ResNet and YOLO) to identify important information (e.g., "collapsed building" and "injured person"). The output is needs information extracted from the analyzed text and image data.

[0125] Step 5:

[0126] The server identifies specific needs within the disaster area from the analyzed data. The input is the needs information analyzed in step 4. Specifically, it integrates the results of text analysis and image recognition to create a list of needs such as "food shortage" and "medical assistance needed." The output is a list of identified needs.

[0127] Step 6:

[0128] The server uses an evaluation algorithm to prioritize the identified needs. The input is the list of needs identified in step 5. Specifically, it applies an evaluation algorithm (e.g., AHP or Point Ranking System) to evaluate the importance of each need. For example, it determines that "food shortage" is a high priority, "medical support" is a medium priority, and "lack of shelter" is a low priority. The output is a prioritized list of needs.

[0129] Step 7:

[0130] The user selects high-priority needs through the system interface and then surveys or interviews local people. The input is the prioritized needs list generated in step 6. Specifically, the user brings in a tablet and asks questions to residents, such as "What type of food do you need most?" The answers are entered as text or voice data. The output is the survey or interview response data.

[0131] Step 8:

[0132] The terminal sends the collected interview results to the server. The input is the response data obtained in step 7. Specifically, the terminal sends the response data back to the server using the HTTPS protocol and records the transmission log. The output is the response data sent to the server and the transmission log.

[0133] Step 9:

[0134] The server stores the results of the interview in a database, making them available for future analysis and reference. The input is the response data submitted in step 8. Specific operations include storing the response data in a database and indexing it to facilitate searchability. The output is the response data stored in the database.

[0135] Step 10:

[0136] The server generates a specific support plan based on the stored data. The input is the response data stored in the database in step 9. Specifically, it uses a generative AI model (e.g., a GPT-based model) to automatically generate support plans such as "securing food supply routes," "location of evacuation shelters," and "medical team dispatch plans." The output is a specific support plan.

[0137] Step 11:

[0138] The server shares the generated support plan with support groups and related organizations. The input is the support plan generated in step 10. The specific operation is to notify the support plan using email or a dedicated support management system. The output is the support plan in a shared state.

[0139] (Application example 1)

[0140] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0141] Building a rapid and efficient support system in disaster areas is a difficult task. It is particularly important to immediately grasp the situation in the affected area and quickly formulate an appropriate support plan. However, conventional methods require time to grasp the situation on the ground and analyze data, which can delay support. Optimizing support activities using autonomous vehicles is also a challenge.

[0142] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0143] In this invention, the server includes means for acquiring information collected by users in the disaster area, means for analyzing the acquired information and identifying needs in the disaster area, means for evaluating and prioritizing the identified needs, means for receiving information collected by the autonomous vehicle, means for analyzing the collected information using image recognition and natural language processing, means for generating a trip plan for the autonomous vehicle based on the analysis results, and means for transmitting the generated trip plan to the autonomous vehicle. This enables rapid and efficient disaster relief, prevents delays in relief activities, and supports the early recovery of disaster-stricken areas.

[0144] "Disaster Area" means an area affected by a natural disaster, technological disaster, or other emergency.

[0145] "User" means an individual or entity that collects information in a disaster area and provides data for analysis and prioritization.

[0146] "Means for obtaining information" refers to devices and methods, including devices (e.g., cameras and GPS) for recording the situation in the disaster area.

[0147] "Means for analyzing data" refers to technologies for analyzing acquired data using natural language processing and machine learning techniques to identify important needs.

[0148] "Means for identifying needs" refers to techniques for extracting the types of support and supplies needed in disaster-stricken areas from the analyzed data.

[0149] A "prioritization tool" is an algorithm that evaluates identified needs and ranks them according to importance and urgency.

[0150] "Means for generating interview content" refers to a system for creating the content of questionnaires and interviews to be conducted with local people based on prioritized needs.

[0151] "Means for recording interview results" refers to technology that allows the results of user interviews and questionnaires to be saved in digital format for later analysis and reference.

[0152] The "means for generating support plans" is a system that automatically generates specific support plans and routes based on the recorded interview results.

[0153] "Means for sharing with support organizations" refers to how the generated support plans can be shared with volunteer groups and other support organizations through a digital platform.

[0154] An "autonomous vehicle" is a vehicle that uses artificial intelligence and sensor technology to move autonomously and transport supplies and collect information in disaster-stricken areas.

[0155] "Means for receiving information collected by an autonomous vehicle" refers to technology for transmitting data collected by an autonomous vehicle (e.g., images and GPS information) to a server and receiving it on the server side.

[0156] "Image recognition" is a technology that analyzes captured video and images and extracts important elements.

[0157] "Natural language processing" is a technology for analyzing and understanding text data, and involves mechanically analyzing collected notes and interview content.

[0158] "Means for generating operation plans" refers to technology that determines the optimal travel routes and schedules for autonomous vehicles based on collected data and analysis results.

[0159] The "means for transmitting the operation plan" refers to a method for transmitting the generated operation plan to the autonomously driven vehicle using wireless communication or the like.

[0160] The present invention provides a system for realizing a rapid and efficient support system in a disaster area. Specific embodiments will be described below.

[0161] User information collection

[0162] Users use a dedicated device in the disaster area to record the damage situation. The device is equipped with a high-resolution camera and a GPS module, allowing it to take photos and videos of the local situation. The device also has a text input function, allowing users to enter notes about the local situation. This allows location information to be attached to the collected information.

[0163] Data transmission and analysis

[0164] The device then packets the collected information and sends it to the server via HTTPS. The server then analyzes the received data packets, first analyzing the text data using natural language processing (NLP) technology. The image data is then analyzed using image recognition technology that utilizes machine learning. At this stage, specific needs within the disaster area (e.g., food shortages or the need for medical assistance) are identified.

[0165] Prioritizing needs

[0166] The server then evaluates and prioritizes the identified needs based on the analysis results. The evaluation algorithm considers factors such as the scale, scope, and severity of the disaster and generates a list such as "food shortages (high priority)" or "medical assistance needs (medium priority)."

[0167] User interviews

[0168] Next, the system interface presents users with high-priority items (e.g., "food shortages") and they can then conduct surveys and interviews with local people, thereby gaining a detailed understanding of their specific needs and the situation on the ground.

[0169] Saving the results of the hearing

[0170] The user's hearing results (text data and voice data) are then sent from the device to the server, which stores the data in a database for future analysis and reference.

[0171] Generate and share support plans

[0172] The server automatically generates specific relief plans based on the stored data. These plans include food supply routes, shelter locations, and medical team dispatch plans. These plans are shared with volunteer groups and relief organizations, enabling swift and effective relief efforts.

[0173] The role of autonomous vehicles

[0174] The system includes an autonomous vehicle that collects information on-site. The vehicle is equipped with a camera, GPS module, and communication module, and automatically patrols and collects information on the damage situation on-site. The vehicle periodically sends data to a server, allowing real-time analysis on the server side. The server generates a vehicle operation plan based on the collected data and transmits it to the vehicle via wireless communication. This allows the autonomous vehicle to efficiently collect information and patrol along the optimal route.

[0175] Examples:

[0176] For example, if an earthquake occurs in a certain area, an autonomous vehicle will be the first to be dispatched to the site. The video and GPS data collected by the vehicle will be sent to a server, which will analyze it and identify local needs. At the same time, users will use their devices to record the local situation from various angles and send it to the server. Based on this information, the server will quickly provide support plans to volunteer groups and aid organizations, enabling appropriate responses.

[0177] Example prompt sentence:

[0178] How can I use the Disaster Response Autonomous Driving Support System to record and transmit local damage information and generate an appropriate support plan?

[0179] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0180] Step 1:

[0181] Users use the device in the disaster area to record the damage situation. The device is equipped with a high-resolution camera and a GPS module, which allows users to take photos and videos of the disaster area and save them along with their location information. Text notes can also be entered.

[0182] Input: Local conditions (photos, videos, text notes, GPS data)

[0183] Output: Disaster situation data saved on the device

[0184] Step 2:

[0185] The device then packets the collected information and sends it to the server using HTTPS, where it is encrypted to ensure the data remains secure.

[0186] Input: Disaster situation data stored on the device

[0187] Output: Data packet sent to the server

[0188] Step 3:

[0189] The server analyzes the received data packets. First, text data is analyzed using natural language processing (NLP) technology, and image data is analyzed using image recognition technology using machine learning.

[0190] Input: Data packet sent to the server

[0191] Output: Analyzed needs data (text, images)

[0192] Step 4:

[0193] The server evaluates and prioritizes the identified needs based on the analysis results. An evaluation algorithm ranks the needs according to their importance and urgency.

[0194] Input: Parsed needs data

[0195] Output: Prioritized needs list

[0196] Step 5:

[0197] The server generates a hearing content based on the priority and presents it to the user. The hearing content is created using natural language processing technology.

[0198] Input: Prioritized Needs List

[0199] Output: The hearing content presented to the user

[0200] Step 6:

[0201] Based on the information provided, the user conducts questionnaires and interviews with local people to understand their specific needs and detailed on-site conditions, and records the results on the device.

[0202] Input: Presented hearing content, responses from local people

[0203] Output: Hearing results saved on the device

[0204] Step 7:

[0205] The device then sends the results of the hearing back to the server, including text and audio data.

[0206] Input: Hearing results saved on the device

[0207] Output: Hearing data sent to the server

[0208] Step 8:

[0209] The server stores the hearing data in a database for future analysis and reference.

[0210] Input: Hearing data sent to the server

[0211] Output: Hearing data stored in a database

[0212] Step 9:

[0213] The server automatically generates specific assistance plans based on the stored data, including food supply routes, shelter locations, and medical team dispatch plans.

[0214] Input: Hearing data stored in a database

[0215] Output: Auto-generated support plan

[0216] Step 10:

[0217] The autonomous vehicle will patrol the disaster area and collect information using cameras and GPS modules, which will then be periodically sent to a server.

[0218] Input: Current state of the disaster area (photography data taken by autonomous vehicles, GPS data)

[0219] Output: Autonomous vehicle data sent to the server

[0220] Step 11:

[0221] The server analyzes the information collected by the autonomous vehicle and analyzes the data using image recognition and natural language processing technologies.

[0222] Input: Data sent from the autonomous vehicle

[0223] Output: Parsed autonomous vehicle data

[0224] Step 12:

[0225] The server generates a driving plan for the autonomous vehicle based on the analysis results, and transmits the generated driving plan to the autonomous vehicle via wireless communication.

[0226] Input: Parsed autonomous vehicle data

[0227] Output: Trip plan sent to the autonomous vehicle

[0228] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0229] This invention is a system for quickly and effectively identifying needs in disaster areas and streamlining relief activities. Furthermore, by combining this system with an emotion engine that recognizes the user's emotions, it is possible to collect information more accurately and generate relief plans.

[0230] User information collection

[0231] Users: In the disaster area, they use their devices to record the damage situation. They take photos of the scene, record videos, and input text notes. GPS data is automatically acquired during the collection process, making the location of the information clear.

[0232] Data transmission and analysis

[0233] Device: Collected photos, videos, text data, and GPS data are packaged into a single data packet and sent to the server. Security technology (e.g., HTTPS) is used for communication, ensuring data safety.

[0234] Server: The server analyzes the received data packets, applying natural language processing (NLP) techniques to the text data and machine learning image recognition techniques to the image data, thereby identifying specific needs in the disaster area (e.g., food shortages, lack of shelter, need for medical assistance).

[0235] Prioritizing needs

[0236] Server: Prioritizes the identified needs based on an evaluation algorithm. Evaluation criteria include the scale, severity, and scope of impact of the disaster. For example, priorities are determined as "food shortages (high priority)," "need for medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[0237] Utilizing the Emotion Engine

[0238] Server: Uses an emotion engine to analyze the voice data and facial expression data collected by the user. Voice analysis technology is used to identify emotions (e.g., anxiety, fear, relief) from the voice data, and image analysis technology is used to recognize changes in facial expressions to identify emotions.

[0239] User interviews

[0240] User: The user displays the generated interview content through the system interface and conducts surveys and interviews with local people. The interview content also takes into account the results of the emotion engine, and appropriate questions and dialogue are conducted according to the user's emotional state.

[0241] Saving the results of the hearing

[0242] Terminal: The user's hearing results are sent back to the server, including text data and voice data.

[0243] Server: The server stores the received hearing results in a database for later reference and further analysis.

[0244] Generate and share support plans

[0245] Server: Based on the stored interview data and emotional data, the server automatically generates specific support plans, such as securing food supply routes, setting up evacuation shelters, and dispatching medical teams.

[0246] Server: The generated support plan is shared with relevant volunteer groups and support organizations. This information is shared via email notifications and a dedicated information sharing system. Based on this information, volunteer groups and support organizations can quickly begin responding.

[0247] Specific examples

[0248] As a concrete scenario, consider a case where there is a serious food shortage in a disaster area. A user takes a photo of the food shortage situation on-site and inputs specific items in a text memo. This information, along with the emotional data of local residents (e.g., anxiety or impatience), is analyzed by an emotion engine and sent to the server. Based on this data, the server creates an "emergency food supply plan" and simultaneously generates a comprehensive support plan that includes counseling support to alleviate anxiety. This plan is shared with volunteer organizations, allowing for swift and accurate support activities to be initiated.

[0249] In this way, the system can comprehensively grasp the local situation and the emotional state of residents, and provide optimal assistance, greatly improving the efficiency and effectiveness of disaster response.

[0250] The processing flow will be explained below.

[0251] Step 1:

[0252] User: In the disaster area, the user uses the device to record the damage situation. For example, the user takes photos and videos of damaged buildings and enters text notes. At this time, the device automatically acquires GPS data.

[0253] Step 2:

[0254] Device: Collected photos, videos, text data, and GPS data are packaged into a single data packet, which is then sent from the device to the server using a secure communication protocol (e.g., HTTPS).

[0255] Step 3:

[0256] Server: Analyzes the received data packets, applying natural language processing (NLP) techniques to text data and image recognition techniques using machine learning to image data. This identifies specific needs in the disaster area (e.g., food shortages, lack of evacuation shelters, need for medical assistance).

[0257] Step 4:

[0258] Server: Prioritizes the identified needs based on an evaluation algorithm. Evaluation criteria include the scale, severity, and scope of impact of the disaster. For example, priorities are determined as "food shortages (high priority)," "need for medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[0259] Step 5:

[0260] Server: Analyzes the user's voice data and facial expression data using an emotion engine. Voice data is analyzed using voice analysis technology to identify emotions (e.g., anxiety, fear, relief), and facial expression data is analyzed using image analysis technology to recognize changes in facial expressions and identify emotions.

[0261] Step 6:

[0262] Server: Generates interview content based on priorities. For example, regarding "food shortages," it sets questions about what exactly is in short supply, how much is needed, and how anxious residents are.

[0263] Step 7:

[0264] User: Through the system interface, the user displays the generated interview content and conducts surveys and interviews with local people. Emotional data is also taken into account here, and appropriate dialogue is conducted. For example, if a resident is feeling anxious, questions that will reassure them are added.

[0265] Step 8:

[0266] Terminal: The user sends the collected hearing results back to the server. The transmission includes text data and audio data.

[0267] Step 9:

[0268] Server: Stores received hearing results and emotion data in a database for future reference and further analysis.

[0269] Step 10:

[0270] Server: Based on the stored interview results and emotional data, the server automatically generates specific support plans, such as securing food supply routes, setting up evacuation shelters, and dispatching medical teams, as well as providing counseling support to reduce residents' anxiety.

[0271] Step 11:

[0272] Server: The generated support plan is shared with relevant volunteer groups and support organizations. This is done via email notifications and a dedicated information sharing system. Based on this information, volunteer groups and support organizations can quickly begin responding.

[0273] Example 2

[0274] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0275] Conventional disaster relief systems have difficulty quickly and accurately grasping local needs and the emotional state of disaster victims, resulting in delays in the creation and implementation of relief plans. Furthermore, the accuracy of the analysis and prioritization of collected data is low, making it difficult to provide optimal relief. Furthermore, there are issues with security and efficiency when sharing relief plans. The present invention aims to solve these issues, streamline relief activities in disaster areas, and provide rapid and accurate relief.

[0276] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0277] In this invention, the server includes means for analyzing acquired information using natural language processing technology and image recognition technology to identify needs in the disaster area, means for prioritizing the identified needs based on an evaluation algorithm, and means for recording the results of user interviews in accordance with the generated interview content. This makes it possible to quickly and accurately grasp local needs and create and implement optimal support plans.

[0278] An "information recording device" is a device used by users to record the situation in a disaster area, and includes smartphones, tablets, and the like.

[0279] "Location information" is data indicating the geographic location of information collected in the disaster area, and is obtained by GPS.

[0280] A "data packet" is a unit of data that allows collected photos, videos, text, GPS data, etc. to be handled as a single unit.

[0281] "Communication path" refers to the network and protocols used to send and receive data packets between a terminal and a server.

[0282] "Natural language processing technology" is a technology for analyzing text data, understanding its meaning, and extracting information, and includes NLP technology.

[0283] "Image recognition technology" refers to technology for analyzing image data and video data and identifying the objects and situations contained therein.

[0284] An "evaluation algorithm" is a calculation method that determines priorities based on identified needs, according to evaluation criteria such as their importance and urgency.

[0285] "Hearing content" refers to the questions and questionnaires given to local people, and includes items to gain a detailed understanding of the disaster situation and needs.

[0286] An "emotion analysis engine" is a software technology for identifying emotions by analyzing voice data and facial expression data, and includes voice analysis technology and image analysis technology.

[0287] A "support plan" is a plan for specific support activities in the disaster area, including food supply, setting up shelters, and dispatching medical teams.

[0288] "Support groups" refers to organizations and groups that provide support in disaster areas, including volunteer groups and government agencies.

[0289] This invention is a system for quickly and effectively identifying needs in disaster areas and streamlining relief activities. It includes specific means for users to acquire and analyze information collected in disaster areas, and generate and share relief plans based on the results. Furthermore, by combining this system with an emotion engine, the invention achieves more accurate information collection and relief plan generation.

[0290] Hardware and Software Configuration

[0291] User information collection

[0292] Users record the local situation in the disaster area using information recording devices such as smartphones and tablets. Users take photos and videos and enter detailed information in text memos. During this process, the device automatically acquires GPS data and records location information.

[0293] Creating and sending data packets

[0294] The device assembles the information collected by the user into a single data packet. The data packet includes photos, videos, text, GPS data, user ID, and a timestamp. The device then sends this data packet to the server using HTTPS. To ensure data security, the communication is encrypted using SSL / TLS.

[0295] Data analysis and emotion recognition

[0296] The server analyzes the received data packets. First, it uses natural language processing technology to analyze the text data and perform keyword extraction and sentiment analysis. Specifically, it uses natural language processing technologies such as spaCy and BERT. Next, it uses image recognition technology to analyze the photo and video data and identify the specific needs of the disaster area. In this process, it uses machine learning frameworks such as TensorFlow and PyTorch. Furthermore, it uses an emotion engine to analyze the voice data and facial expression data and identify the user's emotional state. For emotion analysis, it uses Google® Cloud Speech-to-Text and image analysis technology.

[0297] Prioritizing needs

[0298] The server uses the analysis results to prioritize the needs of the disaster area based on an evaluation algorithm. Evaluation criteria include the scale, severity, and scope of impact of the disaster. For example, priorities are determined as "food shortages (high priority)," "need for medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[0299] Conducting interviews and sending results

[0300] The user checks the generated interview content through the system interface and conducts surveys and interviews with local people. The system generates an appropriate list of questions based on the data collected by the user. The user then asks residents questions based on these questions and collects their responses. This data (voice data and text data) is also repackaged into data packets and sent to the server.

[0301] Generate and share support plans

[0302] The server stores the received interview results and emotional data in a database and generates a specific support plan based on that. Using a generative AI model (e.g., OpenAI's (registered trademark) GPT series), a specific support plan is created based on the input data. Examples include an "emergency food supply plan," "shelter locations," and "medical team dispatch plan." The generated support plan is shared with related support and volunteer organizations via email notifications and a dedicated information-sharing system.

[0303] Specific examples

[0304] As a concrete scenario, consider a case where there is a serious food shortage in a disaster area. The user takes photos of the food shortage situation on-site and enters specific items in text memos. This information, along with the emotional data of local residents (e.g., anxiety and impatience), is analyzed by an emotion engine and sent to the server. The server uses this data to create an "emergency food supply plan" and generate a comprehensive support plan, including counseling support to alleviate anxiety. This plan is shared with volunteer organizations, allowing for swift and accurate support activities to be initiated.

[0305] Prompt Sentence Examples

[0306] "Identify specific needs in disaster areas and use the emotion engine to generate assistance plans. Suggest comprehensive assistance proposals based on user-collected information and emotion data."

[0307] As described above, this system comprehensively grasps the local situation and the emotional state of residents, and provides optimal support, thereby significantly improving the efficiency and effectiveness of disaster response.

[0308] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0309] Step 1: Gather information

[0310] User:

[0311] Users record the situation in the disaster area using an information recording device such as a smartphone or tablet.

[0312] Input: Photos, videos, text notes, GPS data

[0313] How it works: Users take photos and videos of the disaster site and write detailed descriptions of the situation in text notes. The device automatically acquires GPS data and records the location information.

[0314] Output: All collected data (photos, videos, text notes, GPS data)

[0315] Step 2: Generate and send data packets

[0316] Device:

[0317] The terminal assembles the information collected by the user into a single data packet.

[0318] Input: Photos, videos, text notes, GPS data, user ID, timestamp

[0319] How it works: The device assembles all collected data into data packets and sends them to the server using HTTPS, encrypting them with SSL / TLS to ensure secure communications.

[0320] Output: Data packet sent to the server

[0321] Step 3: Receiving and analyzing data

[0322] server:

[0323] The server analyzes the received data packets.

[0324] Input: Data packets (photos, videos, text notes, GPS data, user ID, timestamp)

[0325] Specific behavior:

[0326] Natural Language Processing: The server applies natural language processing techniques (e.g., spaCy or BERT) to the text notes to perform keyword extraction and sentiment analysis.

[0327] Image Recognition: Apply image recognition techniques (e.g., TensorFlow or PyTorch) to photos and videos to identify specific needs in disaster areas.

[0328] Emotion Analysis: An emotion engine is used to analyze voice and facial expression data to identify emotional states. Google Cloud Speech-to-Text is used for voice analysis, and image analysis technology is applied to recognize changes in facial expressions.

[0329] Output: List of needs, user's emotional state

[0330] Step 4: Prioritize your needs

[0331] server:

[0332] The server prioritizes the identified needs based on a rating algorithm.

[0333] Input: List of needs, assessment criteria (scale, severity, scope of impact, etc.)

[0334] Specific behavior: The server scores the identified needs according to the evaluation criteria and determines their priorities.

[0335] Output: A prioritized list of needs

[0336] Step 5: Generate and present the interview

[0337] server:

[0338] The server generates hearing content based on the priority and presents it to the user.

[0339] Input: List of prioritized needs, user information

[0340] Specific operation: The server uses natural language processing technology to automatically generate a list of appropriate questions, which are then sent to the terminal and displayed to the user.

[0341] Output: Terminal displaying the interview contents

[0342] Step 6: Conducting an interview and sending the results

[0343] User:

[0344] The user checks the generated interview content and conducts surveys and interviews with local people.

[0345] Input: Interview details, residents' responses (audio data, text data)

[0346] Specific operation: The user asks questions to residents and collects their answers. This data is then repackaged into a data packet and sent to the server.

[0347] Output: Data packet of the hearing results sent to the server

[0348] Step 7: Analyze and save the results of the interview

[0349] server:

[0350] The server analyzes the received hearing results and stores the data in a database.

[0351] Input: Data packet of hearing results (audio data, text data)

[0352] What it does: The server performs speech and text analysis and stores the results of the hearing in a database for later analysis and reference.

[0353] Output: Hearing results stored in a database

[0354] Step 8: Create and share your support plan

[0355] server:

[0356] The server automatically generates a specific support plan based on the stored hearing data and emotional data.

[0357] Input: Hearing results and emotion data stored in the database

[0358] How it works: Using generative AI models (e.g., OpenAI's GPT series), it generates relief plans tailored to the needs of disaster-stricken areas. The relief plans are then shared with relevant relief and volunteer organizations.

[0359] Output: A support plan shared with support groups and volunteer organizations

[0360] (Application example 2)

[0361] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0362] Conventional disaster response systems did not adequately improve the efficiency of information gathering and support activities in disaster areas, making it difficult to quickly and accurately identify and respond to risks. Furthermore, support plans that properly considered the emotional state of victims were often insufficient, resulting in delays in appropriate responses based on local needs. This resulted in issues such as delays in providing necessary support, and delays in rescue efforts for victims and risk reduction measures.

[0363] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring information collected by users in the disaster area, means for analyzing the acquired information to identify needs in the disaster area, means for evaluating the identified needs and prioritizing them, means for generating interview content based on the priorities and presenting it to the user, means for recording the user's interview results, means for generating a specific support plan based on the recorded interview results, means for sharing the generated support plan with support organizations, emotion analysis means for analyzing voice data and facial expression data collected by the user to identify emotions, means for presenting the generated interview content to the user taking the emotion data into consideration, risk assessment means for identifying and prioritizing risks in the disaster area, means for generating a risk response plan based on the input data, and means for sharing the generated risk response plan with related organizations. This enables rapid and accurate risk identification and support plan generation, as well as providing appropriate support that takes into account the emotional state of the disaster victims.

[0364] A "disaster area" is an area affected by a natural or man-made disaster.

[0365] "User" means an individual or entity that collects information in the disaster area and operates the system.

[0366] "Means for acquiring" refers to the method or device for collecting and storing the information collected by the user.

[0367] "Means of analysis" refers to the techniques and algorithms used to interpret the acquired information and understand its content.

[0368] "Needs" refers to the assistance and services required in the disaster area.

[0369] A "prioritization tool" is an evaluation method or criteria used to select the most important needs from among those identified.

[0370] "Hearing content" refers to questions and interview items used to hear the opinions and requests of local people.

[0371] "Presenting means" refers to the technology or device for displaying the generated hearing content to the user.

[0372] "Recording means" refers to a method or device for saving the results of a user's hearing.

[0373] A "support plan" refers to the plans and policies for specific support activities in the disaster area.

[0374] "Means of sharing" refers to the techniques and methods for communicating the generated support plans to relevant support groups and organizations.

[0375] "Voice data" refers to digital data that records the voices of users and disaster victims.

[0376] "Facial expression data" refers to image data or video data that captures the facial expressions of disaster victims or users.

[0377] "Emotion analysis means" refers to technologies and algorithms for analyzing voice data and facial expression data to identify emotions.

[0378] "Risk" refers to potential dangers or problems in the disaster area.

[0379] "Risk assessment tools" are methods and techniques for assessing the importance and priority of risks identified in a disaster area.

[0380] A "risk response plan" is a specific action plan or policy for dealing with identified risks.

[0381] "Related organizations" include volunteer groups and government agencies that carry out disaster relief activities.

[0382] This invention is a system that supports rapid and effective risk identification and support plan generation in disaster areas. In particular, it takes into account user emotion data to provide more accurate information collection and risk response.

[0383] Information gathering

[0384] User:

[0385] Users use their smartphones in disaster areas to collect local information, including photos, videos, and text notes, and GPS data is automatically added to the collected information, making the location of the collected information clear.

[0386] Data transmission and analysis

[0387] Device:

[0388] Photos, videos, text data, and GPS data collected by users are packaged into a single data packet and sent to a server using HTTPS, ensuring data security.

[0389] server:

[0390] The server analyzes the received data packets, applying natural language processing (NLP) techniques to the text data and machine learning image recognition techniques to the image data. This allows specific needs in the disaster area to be identified, such as food shortages and the need for medical assistance.

[0391] Prioritizing needs

[0392] server:

[0393] The identified needs are prioritized based on an assessment algorithm, which includes criteria such as the scale, severity, and scope of impact of the disaster. For example, priorities are determined as follows: food shortages (high priority), medical assistance needs (medium priority), and lack of shelter (low priority).

[0394] Utilizing the Emotion Engine

[0395] server:

[0396] An emotion analysis method is used to analyze the voice data and facial expression data collected by the user. The voice data is analyzed using voice analysis technology to identify emotions, and the facial expression data is analyzed using image analysis technology to recognize changes in facial expressions to identify emotions. This allows the emotional state of the victim to be properly taken into account.

[0397] User interviews

[0398] User:

[0399] The system displays the generated interview content through the system interface, and conducts surveys and interviews with local people. Emotional data is also reflected in the interview content, and appropriate questions and dialogue are conducted according to the user's emotional state.

[0400] Saving the results of the hearing

[0401] Device:

[0402] The user's hearing results are sent to the server again. The data sent includes text data and voice data.

[0403] server:

[0404] The received interview results are stored in a database for later reference and further analysis.

[0405] Generate and share support plans

[0406] server:

[0407] Based on the stored interview data and emotional data, the system automatically generates specific support plans, such as securing food supply routes, setting up evacuation shelters, and planning the dispatch of medical teams.

[0408] server:

[0409] The generated support plan is shared with relevant volunteer groups and support organizations via email notifications and a dedicated information sharing system.

[0410] Specific examples

[0411] As a concrete scenario, consider a case where there is a serious food shortage in a disaster area. A user takes a photo of the food shortage situation on-site and inputs specific items in a text memo. This information, along with the emotional data of local residents (e.g., anxiety or impatience), is analyzed using an emotion analysis tool and sent to the server. Based on this data, the server creates an "emergency food supply plan" and generates a comprehensive support plan that includes counseling support to alleviate anxiety. This plan is shared with volunteer organizations, allowing for swift and accurate support activities to be initiated.

[0412] Prompt Sentence Examples

[0413] Generate a plan for:

[0414] On-site image and video data

[0415] Text note: "Rubble hazard location confirmed."

[0416] GPS data: Latitude 35.6895, Longitude 139.6917

[0417] Voice data (emotion analysis): Anxiety

[0418] This allows the system to comprehensively grasp the local situation and the emotional state of residents, and provide optimal assistance, significantly improving the efficiency and effectiveness of disaster response.

[0419] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0420] Step 1:

[0421] Information gathering

[0422] Users use their smartphones in disaster areas to collect local information, including photos, videos, text notes, and automatically acquired GPS data.

[0423] The device combines the collected photos, videos, text data, and GPS data into a single data packet, and the output data is the combined data packet.

[0424] Step 2:

[0425] Data transmission

[0426] The terminal sends data packets to the server using HTTPS communication. The input data is the integrated data packet. The output data is the data packet sent over the secure communication channel.

[0427] As a specific operation, the terminal encrypts the data and sends a transmission request to the server.

[0428] Step 3:

[0429] Data analysis

[0430] The server analyzes the received data packets. The input data includes photos, videos, text data, and GPS data.

[0431] Natural language processing (NLP) technology is applied to text data, and image recognition technology using machine learning is applied to image data. The output data is the analyzed specific needs.

[0432] Specifically, the server tokenizes the text data, inputs it into an analysis model (e.g., BERT) to extract needs, and classifies image data using a machine learning model to identify the state of the disaster.

[0433] Step 4:

[0434] Prioritizing needs

[0435] The server prioritizes the identified needs based on an evaluation algorithm. The input data are the analyzed needs. The output data is a prioritized list of needs.

[0436] Specifically, the server scores the severity and scope of the need and ranks it from high priority to low priority.

[0437] Step 5:

[0438] Utilizing the Emotion Engine

[0439] The server analyzes the voice and facial expression data collected by the user. The input data is the voice and facial expression data. The output data is the identified emotional state.

[0440] The voice data is analyzed using voice analysis technology to identify emotions, and the facial expression data is analyzed using image analysis technology to recognize changes in facial expressions. Specifically, the voice data is input into a voice recognition model and an emotion label is attached. The facial expression data is input into an image recognition model and emotions are similarly identified.

[0441] Step 6:

[0442] Generate hearing content

[0443] The server generates a hearing based on the prioritized needs and emotional data. The input data is the prioritized needs list and the identified emotional state. The output data is the generated hearing.

[0444] Specifically, the server uses a generative AI model to create optimal questions and dialogue content based on prioritized needs and emotional state.

[0445] Step 7:

[0446] Hearings

[0447] The user checks the generated interview content through the terminal interface and conducts questionnaires and interviews with local people. The input data is the generated interview content. The output data is the interview results.

[0448] Specifically, the user asks a question displayed on the interface and inputs the answer.

[0449] Step 8:

[0450] Saving the results of the hearing

[0451] The terminal transmits the user's hearing result to the server. The input data is the hearing result. The output data is the transmitted hearing result.

[0452] The server stores the received hearing results in a database. Specifically, the terminal assembles the hearing results into a data packet and sends it to the server, which then stores it in the database.

[0453] Step 9:

[0454] Generate a support plan

[0455] The server automatically generates a specific support plan based on the stored hearing data and emotion data. The input data are the hearing data and emotion data. The output data is a specific support plan.

[0456] Specifically, the server analyzes the hearing data and emotional data and uses a generative AI model to formulate support plans, such as food supply plans and medical team dispatch plans.

[0457] Step 10:

[0458] Sharing support plans

[0459] The server shares the generated support plan with related support groups and organizations. The input data is the specific support plan. The output data is the support plan notified to the support group.

[0460] Specifically, the server uses email notifications and a dedicated information sharing system to quickly communicate the generated support plan to relevant parties.

[0461] Through the above processing steps, the system enables rapid and accurate risk identification in disaster areas and the generation of support plans, enabling effective support activities.

[0462] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0463] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0464] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0465] [Second embodiment]

[0466] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0467] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0468] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0469] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0470] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0471] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0472] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0473] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0474] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0475] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0476] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0477] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0478] The present invention is a system for realizing an immediate and efficient support system in a disaster area. A specific embodiment of this system will be described below.

[0479] User information collection

[0480] User: In the disaster area, the user uses a device to record the damage situation. The user takes photos of the scene and inputs videos and text notes. In addition, GPS data is also acquired during this collection process, so the location information of the information becomes clear.

[0481] Data transmission and analysis

[0482] Terminal: The collected information is packaged into a single packet and sent to the server. Security technology (e.g., HTTPS) is used for communication, ensuring data security.

[0483] Server: The server analyzes the received packets and applies natural language processing (NLP) technology to the text data and machine learning image recognition technology to the image data, thereby identifying specific needs within the disaster area (e.g., food shortages, lack of shelter, need for medical assistance).

[0484] Prioritizing needs

[0485] Server: The extracted needs are passed through an evaluation algorithm to determine priorities. For example, a list may be generated based on the scale, severity, and extent of impact of the disaster, such as "food shortages (high priority)," "need for medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[0486] User interviews

[0487] Users: Through the system interface, they select a high-priority issue, such as "food shortage," and then conduct surveys and interviews with local people. This allows them to understand specific requests and detailed local conditions.

[0488] Saving the results of the hearing

[0489] Terminal: The user's hearing results are sent to the server, including text and audio data.

[0490] Server: The server stores the received hearing results in a database for future reference and analysis.

[0491] Generate and share support plans

[0492] Server: Based on the stored data, it automatically generates specific assistance plans, such as securing food supply routes, setting up evacuation shelters, and dispatching medical teams.

[0493] Server: The generated support plans are shared with volunteer groups and support organizations, enabling them to carry out support activities quickly and effectively.

[0494] This system will enable rapid and efficient emergency response in the event of a disaster. Specifically, by immediately grasping the local situation and appropriately prioritizing the necessary assistance, it will be possible to provide assistance without delay or waste. Such a system is expected to save many lives and contribute to the early recovery of disaster-stricken areas.

[0495] The processing flow will be explained below.

[0496] Step 1:

[0497] User: In the disaster area, the user uses the device to record the damage situation. Specifically, the user takes photos and videos of the damaged buildings and inputs text notes. GPS data is also automatically acquired during this process.

[0498] Step 2:

[0499] Device: Collected photos, videos, text data, and GPS data are packaged into a single data packet, which is then sent from the device to the server using a secure communication protocol (e.g., HTTPS).

[0500] Step 3:

[0501] Server: Analyzes the received data packets, applying natural language processing (NLP) techniques to text data and image recognition techniques using machine learning to image data. This identifies specific needs in the disaster area (e.g., food shortages, lack of evacuation shelters, need for medical assistance).

[0502] Step 4:

[0503] Server: Prioritizes the identified needs based on an evaluation algorithm. Evaluation criteria include the scale, severity, and scope of impact of the disaster. For example, priorities are determined as "food shortages (high priority)," "need for medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[0504] Step 5:

[0505] Server: Generate interview content based on prioritized needs. For example, regarding "food shortages," create questions about what is lacking and what specific support is needed.

[0506] Step 6:

[0507] Users: Through the system interface, they can display the generated interview content and conduct surveys and interviews with local people, thereby gaining a better understanding of specific requirements and detailed on-site conditions.

[0508] Step 7:

[0509] Terminal: The user sends the collected hearing results back to the server, including text data and audio data.

[0510] Step 8:

[0511] Server: Stores received hearing results in a database for later reference and further analysis.

[0512] Step 9:

[0513] Server: Based on the stored interview data, the server automatically generates specific assistance plans, including, for example, securing food supply routes, setting up evacuation shelters, and dispatching medical teams.

[0514] Step 10:

[0515] Server: Shares the generated support plan with relevant volunteer groups and support organizations. Sharing includes email notifications and uploading information to a web portal. Based on this information, volunteer groups and support organizations can quickly begin responding.

[0516] Example 1

[0517] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0518] In order to provide appropriate assistance immediately at disaster sites, it is necessary to quickly and accurately grasp the detailed situation on the ground and identify specific assistance needs. However, conventional methods take time to collect information, making it difficult to quickly determine the priorities of needs. Furthermore, there has been a lack of systems that can effectively organize collected information, automatically generate specific assistance plans, and share them with assistance organizations. The objective of this invention is to solve these problems.

[0519] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0520] In this invention, the server includes: means for acquiring information collected by users using their terminals in the disaster area; means for assembling the acquired information into a single data packet and transmitting it to the server; means for analyzing text data using natural language processing technology and image data using machine learning technology; means for identifying specific needs in the disaster area from the analyzed data; means for prioritizing the identified needs using an evaluation algorithm; means for generating interview content based on the priorities and presenting it to the user; means for recording the results of the interview conducted by the user and transmitting them to the server; means for the server to generate a specific support plan based on the recorded results of the interview; and means for sharing the generated support plan with support organizations. This enables rapid and effective information collection and needs identification when a disaster occurs, automatically generating an appropriate support plan based on priorities, and providing efficient support.

[0521] A "disaster area" is a specific area that has suffered damage or suffering as a result of a natural or man-made disaster.

[0522] A "user" is a person or organization that uses a terminal to collect information within the disaster area.

[0523] A "terminal" is an electronic device such as a mobile phone, smartphone, or tablet that a user uses to gather information or conduct interviews.

[0524] A "data packet" is a single integrated data set that includes text data, media data, and location information collected by a terminal.

[0525] A "server" is a central processing unit or system that receives collected data packets and performs analysis and generates support plans.

[0526] "Natural language processing technology" is a technology that allows computers to understand and analyze language spoken by humans.

[0527] "Machine learning technology" is a technology in which a computer learns from past data and performs highly accurate analysis and predictions on new data.

[0528] "Needs" are the identified needs for goods and services in the disaster area.

[0529] "Priority" is an order determined based on the importance or urgency of identified needs.

[0530] "Hearing contents" are questions and survey items that users use when conducting questionnaires or interviews with local people.

[0531] "Hearing results" are data obtained as a result of questionnaires or interviews with users.

[0532] A "support plan" is a plan for specific support measures generated by the server, and includes food supplies, medical support, and the establishment of evacuation shelters.

[0533] "Support groups" are professional organizations or volunteer groups that provide support to disaster areas.

[0534] The present invention is a system for realizing an immediate and efficient support system in a disaster area. A specific embodiment of this system will be described below.

[0535] User information collection

[0536] Users record the damage situation in the disaster area using devices such as smartphones and tablets. Specifically, users take photos of the scene with their smartphone's camera and enter text notes using the device's application. This process also utilizes the device's GPS function, and location information is automatically included in the collected information.

[0537] Data packet generation

[0538] The device combines text notes entered by the user, photos and videos taken, and location information into a single data packet, which also contains checks to ensure that each piece of data is included correctly.

[0539] Sending data

[0540] The terminal sends the generated data packet to the server using the HTTPS protocol. Before sending, the data is encrypted to ensure the security of the transmission. It also records a log of whether the transmission was successful or not.

[0541] Data reception and analysis

[0542] The server receives data packets sent from the device. After receiving the data, it first decodes the data and then analyzes the text data using natural language processing (NLP) techniques. For example, NLP techniques such as TensorFlow and SpaCy can be used. Furthermore, image and video data are analyzed using machine learning models (e.g., ResNet and YOLO) to extract important information such as "collapsed buildings" and "injured people."

[0543] Identifying and prioritizing needs

[0544] The server uses the analyzed data to identify specific needs within the disaster area. For example, needs such as "food shortages" and "medical assistance required" are extracted. An evaluation algorithm (e.g., AHP, Point Ranking System) is then used to determine the priority of the identified needs. Here, taking into account factors such as the scale and severity of the disaster, the server lists the needs as "food shortages (high priority)," "medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[0545] User interviews

[0546] Through the system interface, users select a high-priority need (e.g., "food shortage") and then survey or interview local people. The survey and interview contents are recorded as text and audio data. Specific prompts could include questions such as, "What food is most needed in this area?"

[0547] Saving the results of the hearing

[0548] The device then sends the collected data to the server, again using the HTTPS protocol for secure and reliable data transmission. The server then stores the data in a database for future analysis and reference.

[0549] Generate and share support plans

[0550] The server generates a specific relief plan based on the results of the stored interviews. For example, the server automatically formulates things like "securing food supply routes," "location of evacuation shelters," and "medical team dispatch plans." This relief plan may also be created using a generative AI model (e.g., a GPT-based model). The generated relief plan is shared with relief groups and related organizations via email or a dedicated relief management system.

[0551] Specific examples

[0552] For example, a user in an area affected by an earthquake uses a smartphone to record the damage situation. They take photos and videos, enter a text note saying "There is a food shortage in this area," and send the data to a server. The server analyzes the received data and identifies food shortages as a high-priority need. To gain a more detailed understanding of the situation, the user uses a tablet to survey residents and sends the results to the server. The server then uses this information to generate a food supply assistance plan and shares it with relief organizations.

[0553] Prompt Sentence Examples

[0554] "Analyze the provided image and text data to identify needs within the disaster area. Also, prioritize the identified needs and generate a specific support plan."

[0555] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0556] Step 1:

[0557] Users use devices such as smartphones and tablets in disaster areas to collect disaster information. Input includes taking images and videos, entering text notes, and acquiring GPS data. Specifically, users launch the smartphone's camera app and take photos of the local situation. They also enter content such as "There is a food shortage in this area" into a text input field within the application. This records detailed information about the disaster site on the device. Output is a set of image data, video data, text notes, and GPS data.

[0558] Step 2:

[0559] The device combines the collected image data, video data, text memos, and GPS data into a single data packet. The input is the various data acquired in step 1. Specifically, the device converts the various data into a packet format and performs an error check. For example, it checks whether the data is stored correctly and records a log of the packet generation. The output is the combined data packet.

[0560] Step 3:

[0561] The terminal sends the generated data packet to the server using the HTTPS protocol. The input is the data packet generated in step 2. Specifically, the terminal first encrypts the data packet, and then starts sending it to the server using HTTPS. A log of whether the transmission was successful or failed is also recorded at the same time. The output is the data packet sent to the server and its transmission log.

[0562] Step 4:

[0563] The server receives and analyzes the data packets sent from the device. The input is the data packet sent in step 3. Specifically, the server first decodes the data and then analyzes the text data using natural language processing (NLP) techniques. For example, NLP techniques such as TensorFlow and SpaCy are used to extract important keywords and phrases from the text notes. It also analyzes images and videos using machine learning models (e.g., ResNet and YOLO) to identify important information (e.g., "collapsed building" and "injured person"). The output is needs information extracted from the analyzed text and image data.

[0564] Step 5:

[0565] The server identifies specific needs within the disaster area from the analyzed data. The input is the needs information analyzed in step 4. Specifically, it integrates the results of text analysis and image recognition to create a list of needs such as "food shortage" and "medical assistance needed." The output is a list of identified needs.

[0566] Step 6:

[0567] The server uses an evaluation algorithm to prioritize the identified needs. The input is the list of needs identified in step 5. Specifically, it applies an evaluation algorithm (e.g., AHP or Point Ranking System) to evaluate the importance of each need. For example, it determines that "food shortage" is a high priority, "medical support" is a medium priority, and "lack of shelter" is a low priority. The output is a prioritized list of needs.

[0568] Step 7:

[0569] The user selects high-priority needs through the system interface and then surveys or interviews local people. The input is the prioritized needs list generated in step 6. Specifically, the user brings in a tablet and asks questions to residents, such as "What type of food do you need most?" The answers are entered as text or voice data. The output is the survey or interview response data.

[0570] Step 8:

[0571] The terminal sends the collected interview results to the server. The input is the response data obtained in step 7. Specifically, the terminal sends the response data back to the server using the HTTPS protocol and records the transmission log. The output is the response data sent to the server and the transmission log.

[0572] Step 9:

[0573] The server stores the results of the interview in a database, making them available for future analysis and reference. The input is the response data submitted in step 8. Specific operations include storing the response data in a database and indexing it to facilitate searchability. The output is the response data stored in the database.

[0574] Step 10:

[0575] The server generates a specific support plan based on the stored data. The input is the response data stored in the database in step 9. Specifically, it uses a generative AI model (e.g., a GPT-based model) to automatically generate support plans such as "securing food supply routes," "location of evacuation shelters," and "medical team dispatch plans." The output is a specific support plan.

[0576] Step 11:

[0577] The server shares the generated support plan with support groups and related organizations. The input is the support plan generated in step 10. The specific operation is to notify the support plan using email or a dedicated support management system. The output is the support plan in a shared state.

[0578] (Application example 1)

[0579] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0580] Building a rapid and efficient support system in disaster areas is a difficult task. It is particularly important to immediately grasp the situation in the affected area and quickly formulate an appropriate support plan. However, conventional methods require time to grasp the situation on the ground and analyze data, which can delay support. Optimizing support activities using autonomous vehicles is also a challenge.

[0581] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0582] In this invention, the server includes means for acquiring information collected by users in the disaster area, means for analyzing the acquired information and identifying needs in the disaster area, means for evaluating and prioritizing the identified needs, means for receiving information collected by the autonomous vehicle, means for analyzing the collected information using image recognition and natural language processing, means for generating a trip plan for the autonomous vehicle based on the analysis results, and means for transmitting the generated trip plan to the autonomous vehicle. This enables rapid and efficient disaster relief, prevents delays in relief activities, and supports the early recovery of disaster-stricken areas.

[0583] "Disaster Area" means an area affected by a natural disaster, technological disaster, or other emergency.

[0584] "User" means an individual or entity that collects information in a disaster area and provides data for analysis and prioritization.

[0585] "Means for obtaining information" refers to devices and methods, including devices (e.g., cameras and GPS) for recording the situation in the disaster area.

[0586] "Means for analyzing data" refers to technologies for analyzing acquired data using natural language processing and machine learning techniques to identify important needs.

[0587] "Means for identifying needs" refers to techniques for extracting the types of support and supplies needed in disaster-stricken areas from the analyzed data.

[0588] A "prioritization tool" is an algorithm that evaluates identified needs and ranks them according to importance and urgency.

[0589] "Means for generating interview content" refers to a system for creating the content of questionnaires and interviews to be conducted with local people based on prioritized needs.

[0590] "Means for recording interview results" refers to technology that allows the results of user interviews and questionnaires to be saved in digital format for later analysis and reference.

[0591] The "means for generating support plans" is a system that automatically generates specific support plans and routes based on the recorded interview results.

[0592] "Means for sharing with support organizations" refers to how the generated support plans can be shared with volunteer groups and other support organizations through a digital platform.

[0593] An "autonomous vehicle" is a vehicle that uses artificial intelligence and sensor technology to move autonomously and transport supplies and collect information in disaster-stricken areas.

[0594] "Means for receiving information collected by an autonomous vehicle" refers to technology for transmitting data collected by an autonomous vehicle (e.g., images and GPS information) to a server and receiving it on the server side.

[0595] "Image recognition" is a technology that analyzes captured video and images and extracts important elements.

[0596] "Natural language processing" is a technology for analyzing and understanding text data, and involves mechanically analyzing collected notes and interview content.

[0597] "Means for generating operation plans" refers to technology that determines the optimal travel routes and schedules for autonomous vehicles based on collected data and analysis results.

[0598] The "means for transmitting the operation plan" refers to a method for transmitting the generated operation plan to the autonomously driven vehicle using wireless communication or the like.

[0599] The present invention provides a system for realizing a rapid and efficient support system in a disaster area. Specific embodiments will be described below.

[0600] User information collection

[0601] Users use a dedicated device in the disaster area to record the damage situation. The device is equipped with a high-resolution camera and a GPS module, allowing it to take photos and videos of the local situation. The device also has a text input function, allowing users to enter notes about the local situation. This allows location information to be attached to the collected information.

[0602] Data transmission and analysis

[0603] The device then packets the collected information and sends it to the server via HTTPS. The server then analyzes the received data packets, first analyzing the text data using natural language processing (NLP) technology. The image data is then analyzed using image recognition technology that utilizes machine learning. At this stage, specific needs within the disaster area (e.g., food shortages or the need for medical assistance) are identified.

[0604] Prioritizing needs

[0605] The server then evaluates and prioritizes the identified needs based on the analysis results. The evaluation algorithm considers factors such as the scale, scope, and severity of the disaster and generates a list such as "food shortages (high priority)" or "medical assistance needs (medium priority)."

[0606] User interviews

[0607] Next, the system interface presents users with high-priority items (e.g., "food shortages") and they can then conduct surveys and interviews with local people, thereby gaining a detailed understanding of their specific needs and the situation on the ground.

[0608] Saving the results of the hearing

[0609] The user's hearing results (text data and voice data) are then sent from the device to the server, which stores the data in a database for future analysis and reference.

[0610] Generate and share support plans

[0611] The server automatically generates specific relief plans based on the stored data. These plans include food supply routes, shelter locations, and medical team dispatch plans. These plans are shared with volunteer groups and relief organizations, enabling swift and effective relief efforts.

[0612] The role of autonomous vehicles

[0613] The system includes an autonomous vehicle that collects information on-site. The vehicle is equipped with a camera, GPS module, and communication module, and automatically patrols and collects information on the damage situation on-site. The vehicle periodically sends data to a server, allowing real-time analysis on the server side. The server generates a vehicle operation plan based on the collected data and transmits it to the vehicle via wireless communication. This allows the autonomous vehicle to efficiently collect information and patrol along the optimal route.

[0614] Examples:

[0615] For example, if an earthquake occurs in a certain area, an autonomous vehicle will be the first to be dispatched to the site. The video and GPS data collected by the vehicle will be sent to a server, which will analyze it and identify local needs. At the same time, users will use their devices to record the local situation from various angles and send it to the server. Based on this information, the server will quickly provide support plans to volunteer groups and aid organizations, enabling appropriate responses.

[0616] Example prompt sentence:

[0617] How can I use the Disaster Response Autonomous Driving Support System to record and transmit local damage information and generate an appropriate support plan?

[0618] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0619] Step 1:

[0620] Users use the device in the disaster area to record the damage situation. The device is equipped with a high-resolution camera and a GPS module, which allows users to take photos and videos of the disaster area and save them along with their location information. Text notes can also be entered.

[0621] Input: Local conditions (photos, videos, text notes, GPS data)

[0622] Output: Disaster situation data saved on the device

[0623] Step 2:

[0624] The device then packets the collected information and sends it to the server using HTTPS, where it is encrypted to ensure the data remains secure.

[0625] Input: Disaster situation data stored on the device

[0626] Output: Data packet sent to the server

[0627] Step 3:

[0628] The server analyzes the received data packets. First, text data is analyzed using natural language processing (NLP) technology, and image data is analyzed using image recognition technology using machine learning.

[0629] Input: Data packet sent to the server

[0630] Output: Analyzed needs data (text, images)

[0631] Step 4:

[0632] The server evaluates and prioritizes the identified needs based on the analysis results. An evaluation algorithm ranks the needs according to their importance and urgency.

[0633] Input: Parsed needs data

[0634] Output: Prioritized needs list

[0635] Step 5:

[0636] The server generates a hearing content based on the priority and presents it to the user. The hearing content is created using natural language processing technology.

[0637] Input: Prioritized Needs List

[0638] Output: The hearing content presented to the user

[0639] Step 6:

[0640] Based on the information provided, the user conducts questionnaires and interviews with local people to understand their specific needs and detailed on-site conditions, and records the results on the device.

[0641] Input: Presented hearing content, responses from local people

[0642] Output: Hearing results saved on the device

[0643] Step 7:

[0644] The device then sends the results of the hearing back to the server, including text and audio data.

[0645] Input: Hearing results saved on the device

[0646] Output: Hearing data sent to the server

[0647] Step 8:

[0648] The server stores the hearing data in a database for future analysis and reference.

[0649] Input: Hearing data sent to the server

[0650] Output: Hearing data stored in a database

[0651] Step 9:

[0652] The server automatically generates specific assistance plans based on the stored data, including food supply routes, shelter locations, and medical team dispatch plans.

[0653] Input: Hearing data stored in a database

[0654] Output: Auto-generated support plan

[0655] Step 10:

[0656] The autonomous vehicle will patrol the disaster area and collect information using cameras and GPS modules, which will then be periodically sent to a server.

[0657] Input: Current state of the disaster area (photography data taken by autonomous vehicles, GPS data)

[0658] Output: Autonomous vehicle data sent to the server

[0659] Step 11:

[0660] The server analyzes the information collected by the autonomous vehicle and analyzes the data using image recognition and natural language processing technologies.

[0661] Input: Data sent from the autonomous vehicle

[0662] Output: Parsed autonomous vehicle data

[0663] Step 12:

[0664] The server generates a driving plan for the autonomous vehicle based on the analysis results, and transmits the generated driving plan to the autonomous vehicle via wireless communication.

[0665] Input: Parsed autonomous vehicle data

[0666] Output: Trip plan sent to the autonomous vehicle

[0667] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0668] This invention is a system for quickly and effectively identifying needs in disaster areas and streamlining relief activities. Furthermore, by combining this system with an emotion engine that recognizes the user's emotions, it is possible to collect information more accurately and generate relief plans.

[0669] User information collection

[0670] Users: In the disaster area, they use their devices to record the damage situation. They take photos of the scene, record videos, and input text notes. GPS data is automatically acquired during the collection process, making the location of the information clear.

[0671] Data transmission and analysis

[0672] Device: Collected photos, videos, text data, and GPS data are packaged into a single data packet and sent to the server. Security technology (e.g., HTTPS) is used for communication, ensuring data safety.

[0673] Server: The server analyzes the received data packets, applying natural language processing (NLP) techniques to the text data and machine learning image recognition techniques to the image data, thereby identifying specific needs in the disaster area (e.g., food shortages, lack of shelter, need for medical assistance).

[0674] Prioritizing needs

[0675] Server: Prioritizes the identified needs based on an evaluation algorithm. Evaluation criteria include the scale, severity, and scope of impact of the disaster. For example, priorities are determined as "food shortages (high priority)," "need for medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[0676] Utilizing the Emotion Engine

[0677] Server: Uses an emotion engine to analyze the voice data and facial expression data collected by the user. Voice analysis technology is used to identify emotions (e.g., anxiety, fear, relief) from the voice data, and image analysis technology is used to recognize changes in facial expressions to identify emotions.

[0678] User interviews

[0679] User: The user displays the generated interview content through the system interface and conducts surveys and interviews with local people. The interview content also takes into account the results of the emotion engine, and appropriate questions and dialogue are conducted according to the user's emotional state.

[0680] Saving the results of the hearing

[0681] Terminal: The user's hearing results are sent back to the server, including text data and voice data.

[0682] Server: The server stores the received hearing results in a database for later reference and further analysis.

[0683] Generate and share support plans

[0684] Server: Based on the stored interview data and emotional data, the server automatically generates specific support plans, such as securing food supply routes, setting up evacuation shelters, and dispatching medical teams.

[0685] Server: The generated support plan is shared with relevant volunteer groups and support organizations. This information is shared via email notifications and a dedicated information sharing system. Based on this information, volunteer groups and support organizations can quickly begin responding.

[0686] Specific examples

[0687] As a concrete scenario, consider a case where there is a serious food shortage in a disaster area. A user takes a photo of the food shortage situation on-site and inputs specific items in a text memo. This information, along with the emotional data of local residents (e.g., anxiety or impatience), is analyzed by an emotion engine and sent to the server. Based on this data, the server creates an "emergency food supply plan" and simultaneously generates a comprehensive support plan that includes counseling support to alleviate anxiety. This plan is shared with volunteer organizations, allowing for swift and accurate support activities to be initiated.

[0688] In this way, the system can comprehensively grasp the local situation and the emotional state of residents, and provide optimal assistance, greatly improving the efficiency and effectiveness of disaster response.

[0689] The processing flow will be explained below.

[0690] Step 1:

[0691] User: In the disaster area, the user uses the device to record the damage situation. For example, the user takes photos and videos of damaged buildings and enters text notes. At this time, the device automatically acquires GPS data.

[0692] Step 2:

[0693] Device: Collected photos, videos, text data, and GPS data are packaged into a single data packet, which is then sent from the device to the server using a secure communication protocol (e.g., HTTPS).

[0694] Step 3:

[0695] Server: Analyzes the received data packets, applying natural language processing (NLP) techniques to text data and image recognition techniques using machine learning to image data. This identifies specific needs in the disaster area (e.g., food shortages, lack of evacuation shelters, need for medical assistance).

[0696] Step 4:

[0697] Server: Prioritizes the identified needs based on an evaluation algorithm. Evaluation criteria include the scale, severity, and scope of impact of the disaster. For example, priorities are determined as "food shortages (high priority)," "need for medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[0698] Step 5:

[0699] Server: Analyzes the user's voice data and facial expression data using an emotion engine. Voice data is analyzed using voice analysis technology to identify emotions (e.g., anxiety, fear, relief), and facial expression data is analyzed using image analysis technology to recognize changes in facial expressions and identify emotions.

[0700] Step 6:

[0701] Server: Generates interview content based on priorities. For example, regarding "food shortages," it sets questions about what exactly is in short supply, how much is needed, and how anxious residents are.

[0702] Step 7:

[0703] User: Through the system interface, the user displays the generated interview content and conducts surveys and interviews with local people. Emotional data is also taken into account here, and appropriate dialogue is conducted. For example, if a resident is feeling anxious, questions that will reassure them are added.

[0704] Step 8:

[0705] Terminal: The user sends the collected hearing results back to the server. The transmission includes text data and audio data.

[0706] Step 9:

[0707] Server: Stores received hearing results and emotion data in a database for future reference and further analysis.

[0708] Step 10:

[0709] Server: Based on the stored interview results and emotional data, the server automatically generates specific support plans, such as securing food supply routes, setting up evacuation shelters, and dispatching medical teams, as well as providing counseling support to reduce residents' anxiety.

[0710] Step 11:

[0711] Server: The generated support plan is shared with relevant volunteer groups and support organizations. This is done via email notifications and a dedicated information sharing system. Based on this information, volunteer groups and support organizations can quickly begin responding.

[0712] Example 2

[0713] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0714] Conventional disaster relief systems have difficulty quickly and accurately grasping local needs and the emotional state of disaster victims, resulting in delays in the creation and implementation of relief plans. Furthermore, the accuracy of the analysis and prioritization of collected data is low, making it difficult to provide optimal relief. Furthermore, there are issues with security and efficiency when sharing relief plans. The present invention aims to solve these issues, streamline relief activities in disaster areas, and provide rapid and accurate relief.

[0715] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0716] In this invention, the server includes means for analyzing acquired information using natural language processing technology and image recognition technology to identify needs in the disaster area, means for prioritizing the identified needs based on an evaluation algorithm, and means for recording the results of user interviews in accordance with the generated interview content. This makes it possible to quickly and accurately grasp local needs and create and implement optimal support plans.

[0717] An "information recording device" is a device used by users to record the situation in a disaster area, and includes smartphones, tablets, and the like.

[0718] "Location information" is data indicating the geographic location of information collected in the disaster area, and is obtained by GPS.

[0719] A "data packet" is a unit of data that allows collected photos, videos, text, GPS data, etc. to be handled as a single unit.

[0720] "Communication path" refers to the network and protocols used to send and receive data packets between a terminal and a server.

[0721] "Natural language processing technology" is a technology for analyzing text data, understanding its meaning, and extracting information, and includes NLP technology.

[0722] "Image recognition technology" refers to technology for analyzing image data and video data and identifying the objects and situations contained therein.

[0723] An "evaluation algorithm" is a calculation method that determines priorities based on identified needs, according to evaluation criteria such as their importance and urgency.

[0724] "Hearing content" refers to the questions and questionnaires given to local people, and includes items to gain a detailed understanding of the disaster situation and needs.

[0725] An "emotion analysis engine" is a software technology for identifying emotions by analyzing voice data and facial expression data, and includes voice analysis technology and image analysis technology.

[0726] A "support plan" is a plan for specific support activities in the disaster area, including food supply, setting up shelters, and dispatching medical teams.

[0727] "Support groups" refers to organizations and groups that provide support in disaster areas, including volunteer groups and government agencies.

[0728] This invention is a system for quickly and effectively identifying needs in disaster areas and streamlining relief activities. It includes specific means for users to acquire and analyze information collected in disaster areas, and generate and share relief plans based on the results. Furthermore, by combining this system with an emotion engine, the invention achieves more accurate information collection and relief plan generation.

[0729] Hardware and Software Configuration

[0730] User information collection

[0731] Users record the local situation in the disaster area using information recording devices such as smartphones and tablets. Users take photos and videos and enter detailed information in text memos. During this process, the device automatically acquires GPS data and records location information.

[0732] Creating and sending data packets

[0733] The device assembles the information collected by the user into a single data packet. The data packet includes photos, videos, text, GPS data, user ID, and a timestamp. The device then sends this data packet to the server using HTTPS. To ensure data security, the communication is encrypted using SSL / TLS.

[0734] Data analysis and emotion recognition

[0735] The server analyzes the received data packets. First, it uses natural language processing technology to analyze the text data and perform keyword extraction and sentiment analysis. Specifically, it uses natural language processing technologies such as spaCy and BERT. Next, it uses image recognition technology to analyze the photo and video data and identify the specific needs of the disaster area. During this process, it uses machine learning frameworks such as TensorFlow and PyTorch. Furthermore, it uses an emotion engine to analyze the voice data and facial expression data and identify the user's emotional state. Google Cloud Speech-to-Text and image analysis technology are used for emotion analysis.

[0736] Prioritizing needs

[0737] The server uses the analysis results to prioritize the needs of the disaster area based on an evaluation algorithm. Evaluation criteria include the scale, severity, and scope of impact of the disaster. For example, priorities are determined as "food shortages (high priority)," "need for medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[0738] Conducting interviews and sending results

[0739] The user checks the generated interview content through the system interface and conducts surveys and interviews with local people. The system generates an appropriate list of questions based on the data collected by the user. The user then asks residents questions based on these questions and collects their responses. This data (voice data and text data) is also repackaged into data packets and sent to the server.

[0740] Generate and share support plans

[0741] The server stores the received interview results and emotional data in a database and generates a specific support plan based on that information. Using a generative AI model (e.g., OpenAI's GPT series), a specific support plan is created based on the input data. Examples of support plans include an "emergency food supply plan," "shelter locations," and "medical team dispatch plan." The generated support plan is shared with relevant support and volunteer organizations via email notifications and a dedicated information-sharing system.

[0742] Specific examples

[0743] As a concrete scenario, consider a case where there is a serious food shortage in a disaster area. The user takes photos of the food shortage situation on-site and enters specific items in text memos. This information, along with the emotional data of local residents (e.g., anxiety and impatience), is analyzed by an emotion engine and sent to the server. The server uses this data to create an "emergency food supply plan" and generate a comprehensive support plan, including counseling support to alleviate anxiety. This plan is shared with volunteer organizations, allowing for swift and accurate support activities to be initiated.

[0744] Prompt Sentence Examples

[0745] "Identify specific needs in disaster areas and use the emotion engine to generate assistance plans. Suggest comprehensive assistance proposals based on user-collected information and emotion data."

[0746] As described above, this system comprehensively grasps the local situation and the emotional state of residents, and provides optimal support, thereby significantly improving the efficiency and effectiveness of disaster response.

[0747] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0748] Step 1: Gather information

[0749] User:

[0750] Users record the situation in the disaster area using an information recording device such as a smartphone or tablet.

[0751] Input: Photos, videos, text notes, GPS data

[0752] How it works: Users take photos and videos of the disaster site and write detailed descriptions of the situation in text notes. The device automatically acquires GPS data and records the location information.

[0753] Output: All collected data (photos, videos, text notes, GPS data)

[0754] Step 2: Generate and send data packets

[0755] Device:

[0756] The terminal assembles the information collected by the user into a single data packet.

[0757] Input: Photos, videos, text notes, GPS data, user ID, timestamp

[0758] How it works: The device assembles all collected data into data packets and sends them to the server using HTTPS, encrypting them with SSL / TLS to ensure secure communications.

[0759] Output: Data packet sent to the server

[0760] Step 3: Receiving and analyzing data

[0761] server:

[0762] The server analyzes the received data packets.

[0763] Input: Data packets (photos, videos, text notes, GPS data, user ID, timestamp)

[0764] Specific behavior:

[0765] Natural Language Processing: The server applies natural language processing techniques (e.g., spaCy or BERT) to the text notes to perform keyword extraction and sentiment analysis.

[0766] Image Recognition: Apply image recognition techniques (e.g., TensorFlow or PyTorch) to photos and videos to identify specific needs in disaster areas.

[0767] Emotion Analysis: An emotion engine is used to analyze voice and facial expression data to identify emotional states. Google Cloud Speech-to-Text is used for voice analysis, and image analysis technology is applied to recognize changes in facial expressions.

[0768] Output: List of needs, user's emotional state

[0769] Step 4: Prioritize your needs

[0770] server:

[0771] The server prioritizes the identified needs based on a rating algorithm.

[0772] Input: List of needs, assessment criteria (scale, severity, scope of impact, etc.)

[0773] Specific behavior: The server scores the identified needs according to the evaluation criteria and determines their priorities.

[0774] Output: A prioritized list of needs

[0775] Step 5: Generate and present the interview

[0776] server:

[0777] The server generates hearing content based on the priority and presents it to the user.

[0778] Input: List of prioritized needs, user information

[0779] Specific operation: The server uses natural language processing technology to automatically generate a list of appropriate questions, which are then sent to the terminal and displayed to the user.

[0780] Output: Terminal displaying the interview contents

[0781] Step 6: Conducting an interview and sending the results

[0782] User:

[0783] The user checks the generated interview content and conducts surveys and interviews with local people.

[0784] Input: Interview details, residents' responses (audio data, text data)

[0785] Specific operation: The user asks questions to residents and collects their answers. This data is then repackaged into a data packet and sent to the server.

[0786] Output: Data packet of the hearing results sent to the server

[0787] Step 7: Analyze and save the results of the interview

[0788] server:

[0789] The server analyzes the received hearing results and stores the data in a database.

[0790] Input: Data packet of hearing results (audio data, text data)

[0791] What it does: The server performs speech and text analysis and stores the results of the hearing in a database for later analysis and reference.

[0792] Output: Hearing results stored in a database

[0793] Step 8: Create and share your support plan

[0794] server:

[0795] The server automatically generates a specific support plan based on the stored hearing data and emotional data.

[0796] Input: Hearing results and emotion data stored in the database

[0797] How it works: Using generative AI models (e.g., OpenAI's GPT series), it generates relief plans tailored to the needs of disaster-stricken areas. The relief plans are then shared with relevant relief and volunteer organizations.

[0798] Output: A support plan shared with support groups and volunteer organizations

[0799] (Application example 2)

[0800] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0801] Conventional disaster response systems did not adequately improve the efficiency of information gathering and support activities in disaster areas, making it difficult to quickly and accurately identify and respond to risks. Furthermore, support plans that properly considered the emotional state of victims were often insufficient, resulting in delays in appropriate responses based on local needs. This resulted in issues such as delays in providing necessary support, and delays in rescue efforts for victims and risk reduction measures.

[0802] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring information collected by users in the disaster area, means for analyzing the acquired information to identify needs in the disaster area, means for evaluating the identified needs and prioritizing them, means for generating interview content based on the priorities and presenting it to the user, means for recording the user's interview results, means for generating a specific support plan based on the recorded interview results, means for sharing the generated support plan with support organizations, emotion analysis means for analyzing voice data and facial expression data collected by the user to identify emotions, means for presenting the generated interview content to the user taking the emotion data into consideration, risk assessment means for identifying and prioritizing risks in the disaster area, means for generating a risk response plan based on the input data, and means for sharing the generated risk response plan with related organizations. This enables rapid and accurate risk identification and support plan generation, as well as providing appropriate support that takes into account the emotional state of the disaster victims.

[0803] A "disaster area" is an area affected by a natural or man-made disaster.

[0804] "User" means an individual or entity that collects information in the disaster area and operates the system.

[0805] "Means for acquiring" refers to the method or device for collecting and storing the information collected by the user.

[0806] "Means of analysis" refers to the techniques and algorithms used to interpret the acquired information and understand its content.

[0807] "Needs" refers to the assistance and services required in the disaster area.

[0808] A "prioritization tool" is an evaluation method or criteria used to select the most important needs from among those identified.

[0809] "Hearing content" refers to questions and interview items used to hear the opinions and requests of local people.

[0810] "Presenting means" refers to the technology or device for displaying the generated hearing content to the user.

[0811] "Recording means" refers to a method or device for saving the results of a user's hearing.

[0812] A "support plan" refers to the plans and policies for specific support activities in the disaster area.

[0813] "Means of sharing" refers to the techniques and methods for communicating the generated support plans to relevant support groups and organizations.

[0814] "Voice data" refers to digital data that records the voices of users and disaster victims.

[0815] "Facial expression data" refers to image data or video data that captures the facial expressions of disaster victims or users.

[0816] "Emotion analysis means" refers to technologies and algorithms for analyzing voice data and facial expression data to identify emotions.

[0817] "Risk" refers to potential dangers or problems in the disaster area.

[0818] "Risk assessment tools" are methods and techniques for assessing the importance and priority of risks identified in a disaster area.

[0819] A "risk response plan" is a specific action plan or policy for dealing with identified risks.

[0820] "Related organizations" include volunteer groups and government agencies that carry out disaster relief activities.

[0821] This invention is a system that supports rapid and effective risk identification and support plan generation in disaster areas. In particular, it takes into account user emotion data to provide more accurate information collection and risk response.

[0822] Information gathering

[0823] User:

[0824] Users use their smartphones in disaster areas to collect local information, including photos, videos, and text notes, and GPS data is automatically added to the collected information, making the location of the collected information clear.

[0825] Data transmission and analysis

[0826] Device:

[0827] Photos, videos, text data, and GPS data collected by users are packaged into a single data packet and sent to a server using HTTPS, ensuring data security.

[0828] server:

[0829] The server analyzes the received data packets, applying natural language processing (NLP) techniques to the text data and machine learning image recognition techniques to the image data. This allows specific needs in the disaster area to be identified, such as food shortages and the need for medical assistance.

[0830] Prioritizing needs

[0831] server:

[0832] The identified needs are prioritized based on an assessment algorithm, which includes criteria such as the scale, severity, and scope of impact of the disaster. For example, priorities are determined as follows: food shortages (high priority), medical assistance needs (medium priority), and lack of shelter (low priority).

[0833] Utilizing the Emotion Engine

[0834] server:

[0835] An emotion analysis method is used to analyze the voice data and facial expression data collected by the user. The voice data is analyzed using voice analysis technology to identify emotions, and the facial expression data is analyzed using image analysis technology to recognize changes in facial expressions to identify emotions. This allows the emotional state of the victim to be properly taken into account.

[0836] User interviews

[0837] User:

[0838] The system displays the generated interview content through the system interface, and conducts surveys and interviews with local people. Emotional data is also reflected in the interview content, and appropriate questions and dialogue are conducted according to the user's emotional state.

[0839] Saving the results of the hearing

[0840] Device:

[0841] The user's hearing results are sent to the server again. The data sent includes text data and voice data.

[0842] server:

[0843] The received interview results are stored in a database for later reference and further analysis.

[0844] Generate and share support plans

[0845] server:

[0846] Based on the stored interview data and emotional data, the system automatically generates specific support plans, such as securing food supply routes, setting up evacuation shelters, and planning the dispatch of medical teams.

[0847] server:

[0848] The generated support plan is shared with relevant volunteer groups and support organizations via email notifications and a dedicated information sharing system.

[0849] Specific examples

[0850] As a concrete scenario, consider a case where there is a serious food shortage in a disaster area. A user takes a photo of the food shortage situation on-site and inputs specific items in a text memo. This information, along with the emotional data of local residents (e.g., anxiety or impatience), is analyzed using an emotion analysis tool and sent to the server. Based on this data, the server creates an "emergency food supply plan" and generates a comprehensive support plan that includes counseling support to alleviate anxiety. This plan is shared with volunteer organizations, allowing for swift and accurate support activities to be initiated.

[0851] Prompt Sentence Examples

[0852] Generate a plan for:

[0853] On-site image and video data

[0854] Text note: "Rubble hazard location confirmed."

[0855] GPS data: Latitude 35.6895, Longitude 139.6917

[0856] Voice data (emotion analysis): Anxiety

[0857] This allows the system to comprehensively grasp the local situation and the emotional state of residents, and provide optimal assistance, significantly improving the efficiency and effectiveness of disaster response.

[0858] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0859] Step 1:

[0860] Information gathering

[0861] Users use their smartphones in disaster areas to collect local information, including photos, videos, text notes, and automatically acquired GPS data.

[0862] The device combines the collected photos, videos, text data, and GPS data into a single data packet, and the output data is the combined data packet.

[0863] Step 2:

[0864] Data transmission

[0865] The terminal sends data packets to the server using HTTPS communication. The input data is the integrated data packet. The output data is the data packet sent over the secure communication channel.

[0866] As a specific operation, the terminal encrypts the data and sends a transmission request to the server.

[0867] Step 3:

[0868] Data analysis

[0869] The server analyzes the received data packets. The input data includes photos, videos, text data, and GPS data.

[0870] Natural language processing (NLP) technology is applied to text data, and image recognition technology using machine learning is applied to image data. The output data is the analyzed specific needs.

[0871] Specifically, the server tokenizes the text data, inputs it into an analysis model (e.g., BERT) to extract needs, and classifies image data using a machine learning model to identify the state of the disaster.

[0872] Step 4:

[0873] Prioritizing needs

[0874] The server prioritizes the identified needs based on an evaluation algorithm. The input data are the analyzed needs. The output data is a prioritized list of needs.

[0875] Specifically, the server scores the severity and scope of the need and ranks it from high priority to low priority.

[0876] Step 5:

[0877] Utilizing the Emotion Engine

[0878] The server analyzes the voice and facial expression data collected by the user. The input data is the voice and facial expression data. The output data is the identified emotional state.

[0879] The voice data is analyzed using voice analysis technology to identify emotions, and the facial expression data is analyzed using image analysis technology to recognize changes in facial expressions. Specifically, the voice data is input into a voice recognition model and an emotion label is attached. The facial expression data is input into an image recognition model and emotions are similarly identified.

[0880] Step 6:

[0881] Generate hearing content

[0882] The server generates a hearing based on the prioritized needs and emotional data. The input data is the prioritized needs list and the identified emotional state. The output data is the generated hearing.

[0883] Specifically, the server uses a generative AI model to create optimal questions and dialogue content based on prioritized needs and emotional state.

[0884] Step 7:

[0885] Hearings

[0886] The user checks the generated interview content through the terminal interface and conducts questionnaires and interviews with local people. The input data is the generated interview content. The output data is the interview results.

[0887] Specifically, the user asks a question displayed on the interface and inputs the answer.

[0888] Step 8:

[0889] Saving the results of the hearing

[0890] The terminal transmits the user's hearing result to the server. The input data is the hearing result. The output data is the transmitted hearing result.

[0891] The server stores the received hearing results in a database. Specifically, the terminal assembles the hearing results into a data packet and sends it to the server, which then stores it in the database.

[0892] Step 9:

[0893] Generate a support plan

[0894] The server automatically generates a specific support plan based on the stored hearing data and emotion data. The input data are the hearing data and emotion data. The output data is a specific support plan.

[0895] Specifically, the server analyzes the hearing data and emotional data and uses a generative AI model to formulate support plans, such as food supply plans and medical team dispatch plans.

[0896] Step 10:

[0897] Sharing support plans

[0898] The server shares the generated support plan with related support groups and organizations. The input data is the specific support plan. The output data is the support plan notified to the support group.

[0899] Specifically, the server uses email notifications and a dedicated information sharing system to quickly communicate the generated support plan to relevant parties.

[0900] Through the above processing steps, the system enables rapid and accurate risk identification in disaster areas and the generation of support plans, enabling effective support activities.

[0901] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0902] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0903] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0904] [Third embodiment]

[0905] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0906] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0907] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0908] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0909] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0910] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0911] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0912] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0913] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0914] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0915] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0916] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0917] The present invention is a system for realizing an immediate and efficient support system in a disaster area. A specific embodiment of this system will be described below.

[0918] User information collection

[0919] User: In the disaster area, the user uses a device to record the damage situation. The user takes photos of the scene and inputs videos and text notes. In addition, GPS data is also acquired during this collection process, so the location information of the information becomes clear.

[0920] Data transmission and analysis

[0921] Terminal: The collected information is packaged into a single packet and sent to the server. Security technology (e.g., HTTPS) is used for communication, ensuring data security.

[0922] Server: The server analyzes the received packets and applies natural language processing (NLP) technology to the text data and machine learning image recognition technology to the image data, thereby identifying specific needs within the disaster area (e.g., food shortages, lack of shelter, need for medical assistance).

[0923] Prioritizing needs

[0924] Server: The extracted needs are passed through an evaluation algorithm to determine priorities. For example, a list may be generated based on the scale, severity, and extent of impact of the disaster, such as "food shortages (high priority)," "need for medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[0925] User interviews

[0926] Users: Through the system interface, they select a high-priority issue, such as "food shortage," and then conduct surveys and interviews with local people. This allows them to understand specific requests and detailed local conditions.

[0927] Saving the results of the hearing

[0928] Terminal: The user's hearing results are sent to the server, including text and audio data.

[0929] Server: The server stores the received hearing results in a database for future reference and analysis.

[0930] Generate and share support plans

[0931] Server: Based on the stored data, it automatically generates specific assistance plans, such as securing food supply routes, setting up evacuation shelters, and dispatching medical teams.

[0932] Server: The generated support plans are shared with volunteer groups and support organizations, enabling them to carry out support activities quickly and effectively.

[0933] This system will enable rapid and efficient emergency response in the event of a disaster. Specifically, by immediately grasping the local situation and appropriately prioritizing the necessary assistance, it will be possible to provide assistance without delay or waste. Such a system is expected to save many lives and contribute to the early recovery of disaster-stricken areas.

[0934] The processing flow will be explained below.

[0935] Step 1:

[0936] User: In the disaster area, the user uses the device to record the damage situation. Specifically, the user takes photos and videos of the damaged buildings and inputs text notes. GPS data is also automatically acquired during this process.

[0937] Step 2:

[0938] Device: Collected photos, videos, text data, and GPS data are packaged into a single data packet, which is then sent from the device to the server using a secure communication protocol (e.g., HTTPS).

[0939] Step 3:

[0940] Server: Analyzes the received data packets, applying natural language processing (NLP) techniques to text data and image recognition techniques using machine learning to image data. This identifies specific needs in the disaster area (e.g., food shortages, lack of evacuation shelters, need for medical assistance).

[0941] Step 4:

[0942] Server: Prioritizes the identified needs based on an evaluation algorithm. Evaluation criteria include the scale, severity, and scope of impact of the disaster. For example, priorities are determined as "food shortages (high priority)," "need for medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[0943] Step 5:

[0944] Server: Generate interview content based on prioritized needs. For example, regarding "food shortages," create questions about what is lacking and what specific support is needed.

[0945] Step 6:

[0946] Users: Through the system interface, they can display the generated interview content and conduct surveys and interviews with local people, thereby gaining a better understanding of specific requirements and detailed on-site conditions.

[0947] Step 7:

[0948] Terminal: The user sends the collected hearing results back to the server, including text data and audio data.

[0949] Step 8:

[0950] Server: Stores received hearing results in a database for later reference and further analysis.

[0951] Step 9:

[0952] Server: Based on the stored interview data, the server automatically generates specific assistance plans, including, for example, securing food supply routes, setting up evacuation shelters, and dispatching medical teams.

[0953] Step 10:

[0954] Server: Shares the generated support plan with relevant volunteer groups and support organizations. Sharing includes email notifications and uploading information to a web portal. Based on this information, volunteer groups and support organizations can quickly begin responding.

[0955] Example 1

[0956] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0957] In order to provide appropriate assistance immediately at disaster sites, it is necessary to quickly and accurately grasp the detailed situation on the ground and identify specific assistance needs. However, conventional methods take time to collect information, making it difficult to quickly determine the priorities of needs. Furthermore, there has been a lack of systems that can effectively organize collected information, automatically generate specific assistance plans, and share them with assistance organizations. The objective of this invention is to solve these problems.

[0958] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0959] In this invention, the server includes: means for acquiring information collected by users using their terminals in the disaster area; means for assembling the acquired information into a single data packet and transmitting it to the server; means for analyzing text data using natural language processing technology and image data using machine learning technology; means for identifying specific needs in the disaster area from the analyzed data; means for prioritizing the identified needs using an evaluation algorithm; means for generating interview content based on the priorities and presenting it to the user; means for recording the results of the interview conducted by the user and transmitting them to the server; means for the server to generate a specific support plan based on the recorded results of the interview; and means for sharing the generated support plan with support organizations. This enables rapid and effective information collection and needs identification when a disaster occurs, automatically generating an appropriate support plan based on priorities, and providing efficient support.

[0960] A "disaster area" is a specific area that has suffered damage or suffering as a result of a natural or man-made disaster.

[0961] A "user" is a person or organization that uses a terminal to collect information within the disaster area.

[0962] A "terminal" is an electronic device such as a mobile phone, smartphone, or tablet that a user uses to gather information or conduct interviews.

[0963] A "data packet" is a single integrated data set that includes text data, media data, and location information collected by a terminal.

[0964] A "server" is a central processing unit or system that receives collected data packets and performs analysis and generates support plans.

[0965] "Natural language processing technology" is a technology that allows computers to understand and analyze language spoken by humans.

[0966] "Machine learning technology" is a technology in which a computer learns from past data and performs highly accurate analysis and predictions on new data.

[0967] "Needs" are the identified needs for goods and services in the disaster area.

[0968] "Priority" is an order determined based on the importance or urgency of identified needs.

[0969] "Hearing contents" are questions and survey items that users use when conducting questionnaires or interviews with local people.

[0970] "Hearing results" are data obtained as a result of questionnaires or interviews with users.

[0971] A "support plan" is a plan for specific support measures generated by the server, and includes food supplies, medical support, and the establishment of evacuation shelters.

[0972] "Support groups" are professional organizations or volunteer groups that provide support to disaster areas.

[0973] The present invention is a system for realizing an immediate and efficient support system in a disaster area. A specific embodiment of this system will be described below.

[0974] User information collection

[0975] Users record the damage situation in the disaster area using devices such as smartphones and tablets. Specifically, users take photos of the scene with their smartphone's camera and enter text notes using the device's application. This process also utilizes the device's GPS function, and location information is automatically included in the collected information.

[0976] Data packet generation

[0977] The device combines text notes entered by the user, photos and videos taken, and location information into a single data packet, which also contains checks to ensure that each piece of data is included correctly.

[0978] Sending data

[0979] The terminal sends the generated data packet to the server using the HTTPS protocol. Before sending, the data is encrypted to ensure the security of the transmission. It also records a log of whether the transmission was successful or not.

[0980] Data reception and analysis

[0981] The server receives data packets sent from the device. After receiving the data, it first decodes the data and then analyzes the text data using natural language processing (NLP) techniques. For example, NLP techniques such as TensorFlow and SpaCy can be used. Furthermore, image and video data are analyzed using machine learning models (e.g., ResNet and YOLO) to extract important information such as "collapsed buildings" and "injured people."

[0982] Identifying and prioritizing needs

[0983] The server uses the analyzed data to identify specific needs within the disaster area. For example, needs such as "food shortages" and "medical assistance required" are extracted. An evaluation algorithm (e.g., AHP, Point Ranking System) is then used to determine the priority of the identified needs. Here, taking into account factors such as the scale and severity of the disaster, the server lists the needs as "food shortages (high priority)," "medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[0984] User interviews

[0985] Through the system interface, users select a high-priority need (e.g., "food shortage") and then survey or interview local people. The survey and interview contents are recorded as text and audio data. Specific prompts could include questions such as, "What food is most needed in this area?"

[0986] Saving the results of the hearing

[0987] The device then sends the collected data to the server, again using the HTTPS protocol for secure and reliable data transmission. The server then stores the data in a database for future analysis and reference.

[0988] Generate and share support plans

[0989] The server generates a specific relief plan based on the results of the stored interviews. For example, the server automatically formulates things like "securing food supply routes," "location of evacuation shelters," and "medical team dispatch plans." This relief plan may also be created using a generative AI model (e.g., a GPT-based model). The generated relief plan is shared with relief groups and related organizations via email or a dedicated relief management system.

[0990] Specific examples

[0991] For example, a user in an area affected by an earthquake uses a smartphone to record the damage situation. They take photos and videos, enter a text note saying "There is a food shortage in this area," and send the data to a server. The server analyzes the received data and identifies food shortages as a high-priority need. To gain a more detailed understanding of the situation, the user uses a tablet to survey residents and sends the results to the server. The server then uses this information to generate a food supply assistance plan and shares it with relief organizations.

[0992] Prompt Sentence Examples

[0993] "Analyze the provided image and text data to identify needs within the disaster area. Also, prioritize the identified needs and generate a specific support plan."

[0994] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0995] Step 1:

[0996] Users use devices such as smartphones and tablets in disaster areas to collect disaster information. Input includes taking images and videos, entering text notes, and acquiring GPS data. Specifically, users launch the smartphone's camera app and take photos of the local situation. They also enter content such as "There is a food shortage in this area" into a text input field within the application. This records detailed information about the disaster site on the device. Output is a set of image data, video data, text notes, and GPS data.

[0997] Step 2:

[0998] The device combines the collected image data, video data, text memos, and GPS data into a single data packet. The input is the various data acquired in step 1. Specifically, the device converts the various data into a packet format and performs an error check. For example, it checks whether the data is stored correctly and records a log of the packet generation. The output is the combined data packet.

[0999] Step 3:

[1000] The terminal sends the generated data packet to the server using the HTTPS protocol. The input is the data packet generated in step 2. Specifically, the terminal first encrypts the data packet, and then starts sending it to the server using HTTPS. A log of whether the transmission was successful or failed is also recorded at the same time. The output is the data packet sent to the server and its transmission log.

[1001] Step 4:

[1002] The server receives and analyzes the data packets sent from the device. The input is the data packet sent in step 3. Specifically, the server first decodes the data and then analyzes the text data using natural language processing (NLP) techniques. For example, NLP techniques such as TensorFlow and SpaCy are used to extract important keywords and phrases from the text notes. It also analyzes images and videos using machine learning models (e.g., ResNet and YOLO) to identify important information (e.g., "collapsed building" and "injured person"). The output is needs information extracted from the analyzed text and image data.

[1003] Step 5:

[1004] The server identifies specific needs within the disaster area from the analyzed data. The input is the needs information analyzed in step 4. Specifically, it integrates the results of text analysis and image recognition to create a list of needs such as "food shortage" and "medical assistance needed." The output is a list of identified needs.

[1005] Step 6:

[1006] The server uses an evaluation algorithm to prioritize the identified needs. The input is the list of needs identified in step 5. Specifically, it applies an evaluation algorithm (e.g., AHP or Point Ranking System) to evaluate the importance of each need. For example, it determines that "food shortage" is a high priority, "medical support" is a medium priority, and "lack of shelter" is a low priority. The output is a prioritized list of needs.

[1007] Step 7:

[1008] The user selects high-priority needs through the system interface and then surveys or interviews local people. The input is the prioritized needs list generated in step 6. Specifically, the user brings in a tablet and asks questions to residents, such as "What type of food do you need most?" The answers are entered as text or voice data. The output is the survey or interview response data.

[1009] Step 8:

[1010] The terminal sends the collected interview results to the server. The input is the response data obtained in step 7. Specifically, the terminal sends the response data back to the server using the HTTPS protocol and records the transmission log. The output is the response data sent to the server and the transmission log.

[1011] Step 9:

[1012] The server stores the results of the interview in a database, making them available for future analysis and reference. The input is the response data submitted in step 8. Specific operations include storing the response data in a database and indexing it to facilitate searchability. The output is the response data stored in the database.

[1013] Step 10:

[1014] The server generates a specific support plan based on the stored data. The input is the response data stored in the database in step 9. Specifically, it uses a generative AI model (e.g., a GPT-based model) to automatically generate support plans such as "securing food supply routes," "location of evacuation shelters," and "medical team dispatch plans." The output is a specific support plan.

[1015] Step 11:

[1016] The server shares the generated support plan with support groups and related organizations. The input is the support plan generated in step 10. The specific operation is to notify the support plan using email or a dedicated support management system. The output is the support plan in a shared state.

[1017] (Application example 1)

[1018] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1019] Building a rapid and efficient support system in disaster areas is a difficult task. It is particularly important to immediately grasp the situation in the affected area and quickly formulate an appropriate support plan. However, conventional methods require time to grasp the situation on the ground and analyze data, which can delay support. Optimizing support activities using autonomous vehicles is also a challenge.

[1020] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1021] In this invention, the server includes means for acquiring information collected by users in the disaster area, means for analyzing the acquired information and identifying needs in the disaster area, means for evaluating and prioritizing the identified needs, means for receiving information collected by the autonomous vehicle, means for analyzing the collected information using image recognition and natural language processing, means for generating a trip plan for the autonomous vehicle based on the analysis results, and means for transmitting the generated trip plan to the autonomous vehicle. This enables rapid and efficient disaster relief, prevents delays in relief activities, and supports the early recovery of disaster-stricken areas.

[1022] "Disaster Area" means an area affected by a natural disaster, technological disaster, or other emergency.

[1023] "User" means an individual or entity that collects information in a disaster area and provides data for analysis and prioritization.

[1024] "Means for obtaining information" refers to devices and methods, including devices (e.g., cameras and GPS) for recording the situation in the disaster area.

[1025] "Means for analyzing data" refers to technologies for analyzing acquired data using natural language processing and machine learning techniques to identify important needs.

[1026] "Means for identifying needs" refers to techniques for extracting the types of support and supplies needed in disaster-stricken areas from the analyzed data.

[1027] A "prioritization tool" is an algorithm that evaluates identified needs and ranks them according to importance and urgency.

[1028] "Means for generating interview content" refers to a system for creating the content of questionnaires and interviews to be conducted with local people based on prioritized needs.

[1029] "Means for recording interview results" refers to technology that allows the results of user interviews and questionnaires to be saved in digital format for later analysis and reference.

[1030] The "means for generating support plans" is a system that automatically generates specific support plans and routes based on the recorded interview results.

[1031] "Means for sharing with support organizations" refers to how the generated support plans can be shared with volunteer groups and other support organizations through a digital platform.

[1032] An "autonomous vehicle" is a vehicle that uses artificial intelligence and sensor technology to move autonomously and transport supplies and collect information in disaster-stricken areas.

[1033] "Means for receiving information collected by an autonomous vehicle" refers to technology for transmitting data collected by an autonomous vehicle (e.g., images and GPS information) to a server and receiving it on the server side.

[1034] "Image recognition" is a technology that analyzes captured video and images and extracts important elements.

[1035] "Natural language processing" is a technology for analyzing and understanding text data, and involves mechanically analyzing collected notes and interview content.

[1036] "Means for generating operation plans" refers to technology that determines the optimal travel routes and schedules for autonomous vehicles based on collected data and analysis results.

[1037] The "means for transmitting the operation plan" refers to a method for transmitting the generated operation plan to the autonomously driven vehicle using wireless communication or the like.

[1038] The present invention provides a system for realizing a rapid and efficient support system in a disaster area. Specific embodiments will be described below.

[1039] User information collection

[1040] Users use a dedicated device in the disaster area to record the damage situation. The device is equipped with a high-resolution camera and a GPS module, allowing it to take photos and videos of the local situation. The device also has a text input function, allowing users to enter notes about the local situation. This allows location information to be attached to the collected information.

[1041] Data transmission and analysis

[1042] The device then packets the collected information and sends it to the server via HTTPS. The server then analyzes the received data packets, first analyzing the text data using natural language processing (NLP) technology. The image data is then analyzed using image recognition technology that utilizes machine learning. At this stage, specific needs within the disaster area (e.g., food shortages or the need for medical assistance) are identified.

[1043] Prioritizing needs

[1044] The server then evaluates and prioritizes the identified needs based on the analysis results. The evaluation algorithm considers factors such as the scale, scope, and severity of the disaster and generates a list such as "food shortages (high priority)" or "medical assistance needs (medium priority)."

[1045] User interviews

[1046] Next, the system interface presents users with high-priority items (e.g., "food shortages") and they can then conduct surveys and interviews with local people, thereby gaining a detailed understanding of their specific needs and the situation on the ground.

[1047] Saving the results of the hearing

[1048] The user's hearing results (text data and voice data) are then sent from the device to the server, which stores the data in a database for future analysis and reference.

[1049] Generate and share support plans

[1050] The server automatically generates specific relief plans based on the stored data. These plans include food supply routes, shelter locations, and medical team dispatch plans. These plans are shared with volunteer groups and relief organizations, enabling swift and effective relief efforts.

[1051] The role of autonomous vehicles

[1052] The system includes an autonomous vehicle that collects information on-site. The vehicle is equipped with a camera, GPS module, and communication module, and automatically patrols and collects information on the damage situation on-site. The vehicle periodically sends data to a server, allowing real-time analysis on the server side. The server generates a vehicle operation plan based on the collected data and transmits it to the vehicle via wireless communication. This allows the autonomous vehicle to efficiently collect information and patrol along the optimal route.

[1053] Examples:

[1054] For example, if an earthquake occurs in a certain area, an autonomous vehicle will be the first to be dispatched to the site. The video and GPS data collected by the vehicle will be sent to a server, which will analyze it and identify local needs. At the same time, users will use their devices to record the local situation from various angles and send it to the server. Based on this information, the server will quickly provide support plans to volunteer groups and aid organizations, enabling appropriate responses.

[1055] Example prompt sentence:

[1056] How can I use the Disaster Response Autonomous Driving Support System to record and transmit local damage information and generate an appropriate support plan?

[1057] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1058] Step 1:

[1059] Users use the device in the disaster area to record the damage situation. The device is equipped with a high-resolution camera and a GPS module, which allows users to take photos and videos of the disaster area and save them along with their location information. Text notes can also be entered.

[1060] Input: Local conditions (photos, videos, text notes, GPS data)

[1061] Output: Disaster situation data saved on the device

[1062] Step 2:

[1063] The device then packets the collected information and sends it to the server using HTTPS, where it is encrypted to ensure the data remains secure.

[1064] Input: Disaster situation data stored on the device

[1065] Output: Data packet sent to the server

[1066] Step 3:

[1067] The server analyzes the received data packets. First, text data is analyzed using natural language processing (NLP) technology, and image data is analyzed using image recognition technology using machine learning.

[1068] Input: Data packet sent to the server

[1069] Output: Analyzed needs data (text, images)

[1070] Step 4:

[1071] The server evaluates and prioritizes the identified needs based on the analysis results. An evaluation algorithm ranks the needs according to their importance and urgency.

[1072] Input: Parsed needs data

[1073] Output: Prioritized needs list

[1074] Step 5:

[1075] The server generates a hearing content based on the priority and presents it to the user. The hearing content is created using natural language processing technology.

[1076] Input: Prioritized Needs List

[1077] Output: The hearing content presented to the user

[1078] Step 6:

[1079] Based on the information provided, the user conducts questionnaires and interviews with local people to understand their specific needs and detailed on-site conditions, and records the results on the device.

[1080] Input: Presented hearing content, responses from local people

[1081] Output: Hearing results saved on the device

[1082] Step 7:

[1083] The device then sends the results of the hearing back to the server, including text and audio data.

[1084] Input: Hearing results saved on the device

[1085] Output: Hearing data sent to the server

[1086] Step 8:

[1087] The server stores the hearing data in a database for future analysis and reference.

[1088] Input: Hearing data sent to the server

[1089] Output: Hearing data stored in a database

[1090] Step 9:

[1091] The server automatically generates specific assistance plans based on the stored data, including food supply routes, shelter locations, and medical team dispatch plans.

[1092] Input: Hearing data stored in a database

[1093] Output: Auto-generated support plan

[1094] Step 10:

[1095] The autonomous vehicle will patrol the disaster area and collect information using cameras and GPS modules, which will then be periodically sent to a server.

[1096] Input: Current state of the disaster area (photography data taken by autonomous vehicles, GPS data)

[1097] Output: Autonomous vehicle data sent to the server

[1098] Step 11:

[1099] The server analyzes the information collected by the autonomous vehicle and analyzes the data using image recognition and natural language processing technologies.

[1100] Input: Data sent from the autonomous vehicle

[1101] Output: Parsed autonomous vehicle data

[1102] Step 12:

[1103] The server generates a driving plan for the autonomous vehicle based on the analysis results, and transmits the generated driving plan to the autonomous vehicle via wireless communication.

[1104] Input: Parsed autonomous vehicle data

[1105] Output: Trip plan sent to the autonomous vehicle

[1106] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1107] This invention is a system for quickly and effectively identifying needs in disaster areas and streamlining relief activities. Furthermore, by combining this system with an emotion engine that recognizes the user's emotions, it is possible to collect information more accurately and generate relief plans.

[1108] User information collection

[1109] Users: In the disaster area, they use their devices to record the damage situation. They take photos of the scene, record videos, and input text notes. GPS data is automatically acquired during the collection process, making the location of the information clear.

[1110] Data transmission and analysis

[1111] Device: Collected photos, videos, text data, and GPS data are packaged into a single data packet and sent to the server. Security technology (e.g., HTTPS) is used for communication, ensuring data safety.

[1112] Server: The server analyzes the received data packets, applying natural language processing (NLP) techniques to the text data and machine learning image recognition techniques to the image data, thereby identifying specific needs in the disaster area (e.g., food shortages, lack of shelter, need for medical assistance).

[1113] Prioritizing needs

[1114] Server: Prioritizes the identified needs based on an evaluation algorithm. Evaluation criteria include the scale, severity, and scope of impact of the disaster. For example, priorities are determined as "food shortages (high priority)," "need for medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[1115] Utilizing the Emotion Engine

[1116] Server: Uses an emotion engine to analyze the voice data and facial expression data collected by the user. Voice analysis technology is used to identify emotions (e.g., anxiety, fear, relief) from the voice data, and image analysis technology is used to recognize changes in facial expressions to identify emotions.

[1117] User interviews

[1118] User: The user displays the generated interview content through the system interface and conducts surveys and interviews with local people. The interview content also takes into account the results of the emotion engine, and appropriate questions and dialogue are conducted according to the user's emotional state.

[1119] Saving the results of the hearing

[1120] Terminal: The user's hearing results are sent back to the server, including text data and voice data.

[1121] Server: The server stores the received hearing results in a database for later reference and further analysis.

[1122] Generate and share support plans

[1123] Server: Based on the stored interview data and emotional data, the server automatically generates specific support plans, such as securing food supply routes, setting up evacuation shelters, and dispatching medical teams.

[1124] Server: The generated support plan is shared with relevant volunteer groups and support organizations. This information is shared via email notifications and a dedicated information sharing system. Based on this information, volunteer groups and support organizations can quickly begin responding.

[1125] Specific examples

[1126] As a concrete scenario, consider a case where there is a serious food shortage in a disaster area. A user takes a photo of the food shortage situation on-site and inputs specific items in a text memo. This information, along with the emotional data of local residents (e.g., anxiety or impatience), is analyzed by an emotion engine and sent to the server. Based on this data, the server creates an "emergency food supply plan" and simultaneously generates a comprehensive support plan that includes counseling support to alleviate anxiety. This plan is shared with volunteer organizations, allowing for swift and accurate support activities to be initiated.

[1127] In this way, the system can comprehensively grasp the local situation and the emotional state of residents, and provide optimal assistance, greatly improving the efficiency and effectiveness of disaster response.

[1128] The processing flow will be explained below.

[1129] Step 1:

[1130] User: In the disaster area, the user uses the device to record the damage situation. For example, the user takes photos and videos of damaged buildings and enters text notes. At this time, the device automatically acquires GPS data.

[1131] Step 2:

[1132] Device: Collected photos, videos, text data, and GPS data are packaged into a single data packet, which is then sent from the device to the server using a secure communication protocol (e.g., HTTPS).

[1133] Step 3:

[1134] Server: Analyzes the received data packets, applying natural language processing (NLP) techniques to text data and image recognition techniques using machine learning to image data. This identifies specific needs in the disaster area (e.g., food shortages, lack of evacuation shelters, need for medical assistance).

[1135] Step 4:

[1136] Server: Prioritizes the identified needs based on an evaluation algorithm. Evaluation criteria include the scale, severity, and scope of impact of the disaster. For example, priorities are determined as "food shortages (high priority)," "need for medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[1137] Step 5:

[1138] Server: Analyzes the user's voice data and facial expression data using an emotion engine. Voice data is analyzed using voice analysis technology to identify emotions (e.g., anxiety, fear, relief), and facial expression data is analyzed using image analysis technology to recognize changes in facial expressions and identify emotions.

[1139] Step 6:

[1140] Server: Generates interview content based on priorities. For example, regarding "food shortages," it sets questions about what exactly is in short supply, how much is needed, and how anxious residents are.

[1141] Step 7:

[1142] User: Through the system interface, the user displays the generated interview content and conducts surveys and interviews with local people. Emotional data is also taken into account here, and appropriate dialogue is conducted. For example, if a resident is feeling anxious, questions that will reassure them are added.

[1143] Step 8:

[1144] Terminal: The user sends the collected hearing results back to the server. The transmission includes text data and audio data.

[1145] Step 9:

[1146] Server: Stores received hearing results and emotion data in a database for future reference and further analysis.

[1147] Step 10:

[1148] Server: Based on the stored interview results and emotional data, the server automatically generates specific support plans, such as securing food supply routes, setting up evacuation shelters, and dispatching medical teams, as well as providing counseling support to reduce residents' anxiety.

[1149] Step 11:

[1150] Server: The generated support plan is shared with relevant volunteer groups and support organizations. This is done via email notifications and a dedicated information sharing system. Based on this information, volunteer groups and support organizations can quickly begin responding.

[1151] Example 2

[1152] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1153] Conventional disaster relief systems have difficulty quickly and accurately grasping local needs and the emotional state of disaster victims, resulting in delays in the creation and implementation of relief plans. Furthermore, the accuracy of the analysis and prioritization of collected data is low, making it difficult to provide optimal relief. Furthermore, there are issues with security and efficiency when sharing relief plans. The present invention aims to solve these issues, streamline relief activities in disaster areas, and provide rapid and accurate relief.

[1154] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1155] In this invention, the server includes means for analyzing acquired information using natural language processing technology and image recognition technology to identify needs in the disaster area, means for prioritizing the identified needs based on an evaluation algorithm, and means for recording the results of user interviews in accordance with the generated interview content. This makes it possible to quickly and accurately grasp local needs and create and implement optimal support plans.

[1156] An "information recording device" is a device used by users to record the situation in a disaster area, and includes smartphones, tablets, and the like.

[1157] "Location information" is data indicating the geographic location of information collected in the disaster area, and is obtained by GPS.

[1158] A "data packet" is a unit of data that allows collected photos, videos, text, GPS data, etc. to be handled as a single unit.

[1159] "Communication path" refers to the network and protocols used to send and receive data packets between a terminal and a server.

[1160] "Natural language processing technology" is a technology for analyzing text data, understanding its meaning, and extracting information, and includes NLP technology.

[1161] "Image recognition technology" refers to technology for analyzing image data and video data and identifying the objects and situations contained therein.

[1162] An "evaluation algorithm" is a calculation method that determines priorities based on identified needs, according to evaluation criteria such as their importance and urgency.

[1163] "Hearing content" refers to the questions and questionnaires given to local people, and includes items to gain a detailed understanding of the disaster situation and needs.

[1164] An "emotion analysis engine" is a software technology for identifying emotions by analyzing voice data and facial expression data, and includes voice analysis technology and image analysis technology.

[1165] A "support plan" is a plan for specific support activities in the disaster area, including food supply, setting up shelters, and dispatching medical teams.

[1166] "Support groups" refers to organizations and groups that provide support in disaster areas, including volunteer groups and government agencies.

[1167] This invention is a system for quickly and effectively identifying needs in disaster areas and streamlining relief activities. It includes specific means for users to acquire and analyze information collected in disaster areas, and generate and share relief plans based on the results. Furthermore, by combining this system with an emotion engine, the invention achieves more accurate information collection and relief plan generation.

[1168] Hardware and Software Configuration

[1169] User information collection

[1170] Users record the local situation in the disaster area using information recording devices such as smartphones and tablets. Users take photos and videos and enter detailed information in text memos. During this process, the device automatically acquires GPS data and records location information.

[1171] Creating and sending data packets

[1172] The device assembles the information collected by the user into a single data packet. The data packet includes photos, videos, text, GPS data, user ID, and a timestamp. The device then sends this data packet to the server using HTTPS. To ensure data security, the communication is encrypted using SSL / TLS.

[1173] Data analysis and emotion recognition

[1174] The server analyzes the received data packets. First, it uses natural language processing technology to analyze the text data and perform keyword extraction and sentiment analysis. Specifically, it uses natural language processing technologies such as spaCy and BERT. Next, it uses image recognition technology to analyze the photo and video data and identify the specific needs of the disaster area. During this process, it uses machine learning frameworks such as TensorFlow and PyTorch. Furthermore, it uses an emotion engine to analyze the voice data and facial expression data and identify the user's emotional state. Google Cloud Speech-to-Text and image analysis technology are used for emotion analysis.

[1175] Prioritizing needs

[1176] The server uses the analysis results to prioritize the needs of the disaster area based on an evaluation algorithm. Evaluation criteria include the scale, severity, and scope of impact of the disaster. For example, priorities are determined as "food shortages (high priority)," "need for medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[1177] Conducting interviews and sending results

[1178] The user checks the generated interview content through the system interface and conducts surveys and interviews with local people. The system generates an appropriate list of questions based on the data collected by the user. The user then asks residents questions based on these questions and collects their responses. This data (voice data and text data) is also repackaged into data packets and sent to the server.

[1179] Generate and share support plans

[1180] The server stores the received interview results and emotional data in a database and generates a specific support plan based on that information. Using a generative AI model (e.g., OpenAI's GPT series), a specific support plan is created based on the input data. Examples of support plans include an "emergency food supply plan," "shelter locations," and "medical team dispatch plan." The generated support plan is shared with relevant support and volunteer organizations via email notifications and a dedicated information-sharing system.

[1181] Specific examples

[1182] As a concrete scenario, consider a case where there is a serious food shortage in a disaster area. The user takes photos of the food shortage situation on-site and enters specific items in text memos. This information, along with the emotional data of local residents (e.g., anxiety and impatience), is analyzed by an emotion engine and sent to the server. The server uses this data to create an "emergency food supply plan" and generate a comprehensive support plan, including counseling support to alleviate anxiety. This plan is shared with volunteer organizations, allowing for swift and accurate support activities to be initiated.

[1183] Prompt Sentence Examples

[1184] "Identify specific needs in disaster areas and use the emotion engine to generate assistance plans. Suggest comprehensive assistance proposals based on user-collected information and emotion data."

[1185] As described above, this system comprehensively grasps the local situation and the emotional state of residents, and provides optimal support, thereby significantly improving the efficiency and effectiveness of disaster response.

[1186] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1187] Step 1: Gather information

[1188] User:

[1189] Users record the situation in the disaster area using an information recording device such as a smartphone or tablet.

[1190] Input: Photos, videos, text notes, GPS data

[1191] How it works: Users take photos and videos of the disaster site and write detailed descriptions of the situation in text notes. The device automatically acquires GPS data and records the location information.

[1192] Output: All collected data (photos, videos, text notes, GPS data)

[1193] Step 2: Generate and send data packets

[1194] Device:

[1195] The terminal assembles the information collected by the user into a single data packet.

[1196] Input: Photos, videos, text notes, GPS data, user ID, timestamp

[1197] How it works: The device assembles all collected data into data packets and sends them to the server using HTTPS, encrypting them with SSL / TLS to ensure secure communications.

[1198] Output: Data packet sent to the server

[1199] Step 3: Receiving and analyzing data

[1200] server:

[1201] The server analyzes the received data packets.

[1202] Input: Data packets (photos, videos, text notes, GPS data, user ID, timestamp)

[1203] Specific behavior:

[1204] Natural Language Processing: The server applies natural language processing techniques (e.g., spaCy or BERT) to the text notes to perform keyword extraction and sentiment analysis.

[1205] Image Recognition: Apply image recognition techniques (e.g., TensorFlow or PyTorch) to photos and videos to identify specific needs in disaster areas.

[1206] Emotion Analysis: An emotion engine is used to analyze voice and facial expression data to identify emotional states. Google Cloud Speech-to-Text is used for voice analysis, and image analysis technology is applied to recognize changes in facial expressions.

[1207] Output: List of needs, user's emotional state

[1208] Step 4: Prioritize your needs

[1209] server:

[1210] The server prioritizes the identified needs based on a rating algorithm.

[1211] Input: List of needs, assessment criteria (scale, severity, scope of impact, etc.)

[1212] Specific behavior: The server scores the identified needs according to the evaluation criteria and determines their priorities.

[1213] Output: A prioritized list of needs

[1214] Step 5: Generate and present the interview

[1215] server:

[1216] The server generates hearing content based on the priority and presents it to the user.

[1217] Input: List of prioritized needs, user information

[1218] Specific operation: The server uses natural language processing technology to automatically generate a list of appropriate questions, which are then sent to the terminal and displayed to the user.

[1219] Output: Terminal displaying the interview contents

[1220] Step 6: Conducting an interview and sending the results

[1221] User:

[1222] The user checks the generated interview content and conducts surveys and interviews with local people.

[1223] Input: Interview details, residents' responses (audio data, text data)

[1224] Specific operation: The user asks questions to residents and collects their answers. This data is then repackaged into a data packet and sent to the server.

[1225] Output: Data packet of the hearing results sent to the server

[1226] Step 7: Analyze and save the results of the interview

[1227] server:

[1228] The server analyzes the received hearing results and stores the data in a database.

[1229] Input: Data packet of hearing results (audio data, text data)

[1230] What it does: The server performs speech and text analysis and stores the results of the hearing in a database for later analysis and reference.

[1231] Output: Hearing results stored in a database

[1232] Step 8: Create and share your support plan

[1233] server:

[1234] The server automatically generates a specific support plan based on the stored hearing data and emotional data.

[1235] Input: Hearing results and emotion data stored in the database

[1236] How it works: Using generative AI models (e.g., OpenAI's GPT series), it generates relief plans tailored to the needs of disaster-stricken areas. The relief plans are then shared with relevant relief and volunteer organizations.

[1237] Output: A support plan shared with support groups and volunteer organizations

[1238] (Application example 2)

[1239] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1240] Conventional disaster response systems did not adequately improve the efficiency of information gathering and support activities in disaster areas, making it difficult to quickly and accurately identify and respond to risks. Furthermore, support plans that properly considered the emotional state of victims were often insufficient, resulting in delays in appropriate responses based on local needs. This resulted in issues such as delays in providing necessary support, and delays in rescue efforts for victims and risk reduction measures.

[1241] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring information collected by users in the disaster area, means for analyzing the acquired information to identify needs in the disaster area, means for evaluating the identified needs and prioritizing them, means for generating interview content based on the priorities and presenting it to the user, means for recording the user's interview results, means for generating a specific support plan based on the recorded interview results, means for sharing the generated support plan with support organizations, emotion analysis means for analyzing voice data and facial expression data collected by the user to identify emotions, means for presenting the generated interview content to the user taking the emotion data into consideration, risk assessment means for identifying and prioritizing risks in the disaster area, means for generating a risk response plan based on the input data, and means for sharing the generated risk response plan with related organizations. This enables rapid and accurate risk identification and support plan generation, as well as providing appropriate support that takes into account the emotional state of the disaster victims.

[1242] A "disaster area" is an area affected by a natural or man-made disaster.

[1243] "User" means an individual or entity that collects information in the disaster area and operates the system.

[1244] "Means for acquiring" refers to the method or device for collecting and storing the information collected by the user.

[1245] "Means of analysis" refers to the techniques and algorithms used to interpret the acquired information and understand its content.

[1246] "Needs" refers to the assistance and services required in the disaster area.

[1247] A "prioritization tool" is an evaluation method or criteria used to select the most important needs from among those identified.

[1248] "Hearing content" refers to questions and interview items used to hear the opinions and requests of local people.

[1249] "Presenting means" refers to the technology or device for displaying the generated hearing content to the user.

[1250] "Recording means" refers to a method or device for saving the results of a user's hearing.

[1251] A "support plan" refers to the plans and policies for specific support activities in the disaster area.

[1252] "Means of sharing" refers to the techniques and methods for communicating the generated support plans to relevant support groups and organizations.

[1253] "Voice data" refers to digital data that records the voices of users and disaster victims.

[1254] "Facial expression data" refers to image data or video data that captures the facial expressions of disaster victims or users.

[1255] "Emotion analysis means" refers to technologies and algorithms for analyzing voice data and facial expression data to identify emotions.

[1256] "Risk" refers to potential dangers or problems in the disaster area.

[1257] "Risk assessment tools" are methods and techniques for assessing the importance and priority of risks identified in a disaster area.

[1258] A "risk response plan" is a specific action plan or policy for dealing with identified risks.

[1259] "Related organizations" include volunteer groups and government agencies that carry out disaster relief activities.

[1260] This invention is a system that supports rapid and effective risk identification and support plan generation in disaster areas. In particular, it takes into account user emotion data to provide more accurate information collection and risk response.

[1261] Information gathering

[1262] User:

[1263] Users use their smartphones in disaster areas to collect local information, including photos, videos, and text notes, and GPS data is automatically added to the collected information, making the location of the collected information clear.

[1264] Data transmission and analysis

[1265] Device:

[1266] Photos, videos, text data, and GPS data collected by users are packaged into a single data packet and sent to a server using HTTPS, ensuring data security.

[1267] server:

[1268] The server analyzes the received data packets, applying natural language processing (NLP) techniques to the text data and machine learning image recognition techniques to the image data. This allows specific needs in the disaster area to be identified, such as food shortages and the need for medical assistance.

[1269] Prioritizing needs

[1270] server:

[1271] The identified needs are prioritized based on an assessment algorithm, which includes criteria such as the scale, severity, and scope of impact of the disaster. For example, priorities are determined as follows: food shortages (high priority), medical assistance needs (medium priority), and lack of shelter (low priority).

[1272] Utilizing the Emotion Engine

[1273] server:

[1274] An emotion analysis method is used to analyze the voice data and facial expression data collected by the user. The voice data is analyzed using voice analysis technology to identify emotions, and the facial expression data is analyzed using image analysis technology to recognize changes in facial expressions to identify emotions. This allows the emotional state of the victim to be properly taken into account.

[1275] User interviews

[1276] User:

[1277] The system displays the generated interview content through the system interface, and conducts surveys and interviews with local people. Emotional data is also reflected in the interview content, and appropriate questions and dialogue are conducted according to the user's emotional state.

[1278] Saving the results of the hearing

[1279] Device:

[1280] The user's hearing results are sent to the server again. The data sent includes text data and voice data.

[1281] server:

[1282] The received interview results are stored in a database for later reference and further analysis.

[1283] Generate and share support plans

[1284] server:

[1285] Based on the stored interview data and emotional data, the system automatically generates specific support plans, such as securing food supply routes, setting up evacuation shelters, and planning the dispatch of medical teams.

[1286] server:

[1287] The generated support plan is shared with relevant volunteer groups and support organizations via email notifications and a dedicated information sharing system.

[1288] Specific examples

[1289] As a concrete scenario, consider a case where there is a serious food shortage in a disaster area. A user takes a photo of the food shortage situation on-site and inputs specific items in a text memo. This information, along with the emotional data of local residents (e.g., anxiety or impatience), is analyzed using an emotion analysis tool and sent to the server. Based on this data, the server creates an "emergency food supply plan" and generates a comprehensive support plan that includes counseling support to alleviate anxiety. This plan is shared with volunteer organizations, allowing for swift and accurate support activities to be initiated.

[1290] Prompt Sentence Examples

[1291] Generate a plan for:

[1292] On-site image and video data

[1293] Text note: "Rubble hazard location confirmed."

[1294] GPS data: Latitude 35.6895, Longitude 139.6917

[1295] Voice data (emotion analysis): Anxiety

[1296] This allows the system to comprehensively grasp the local situation and the emotional state of residents, and provide optimal assistance, significantly improving the efficiency and effectiveness of disaster response.

[1297] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1298] Step 1:

[1299] Information gathering

[1300] Users use their smartphones in disaster areas to collect local information, including photos, videos, text notes, and automatically acquired GPS data.

[1301] The device combines the collected photos, videos, text data, and GPS data into a single data packet, and the output data is the combined data packet.

[1302] Step 2:

[1303] Data transmission

[1304] The terminal sends data packets to the server using HTTPS communication. The input data is the integrated data packet. The output data is the data packet sent over the secure communication channel.

[1305] As a specific operation, the terminal encrypts the data and sends a transmission request to the server.

[1306] Step 3:

[1307] Data analysis

[1308] The server analyzes the received data packets. The input data includes photos, videos, text data, and GPS data.

[1309] Natural language processing (NLP) technology is applied to text data, and image recognition technology using machine learning is applied to image data. The output data is the analyzed specific needs.

[1310] Specifically, the server tokenizes the text data, inputs it into an analysis model (e.g., BERT) to extract needs, and classifies image data using a machine learning model to identify the state of the disaster.

[1311] Step 4:

[1312] Prioritizing needs

[1313] The server prioritizes the identified needs based on an evaluation algorithm. The input data are the analyzed needs. The output data is a prioritized list of needs.

[1314] Specifically, the server scores the severity and scope of the need and ranks it from high priority to low priority.

[1315] Step 5:

[1316] Utilizing the Emotion Engine

[1317] The server analyzes the voice and facial expression data collected by the user. The input data is the voice and facial expression data. The output data is the identified emotional state.

[1318] The voice data is analyzed using voice analysis technology to identify emotions, and the facial expression data is analyzed using image analysis technology to recognize changes in facial expressions. Specifically, the voice data is input into a voice recognition model and an emotion label is attached. The facial expression data is input into an image recognition model and emotions are similarly identified.

[1319] Step 6:

[1320] Generate hearing content

[1321] The server generates a hearing based on the prioritized needs and emotional data. The input data is the prioritized needs list and the identified emotional state. The output data is the generated hearing.

[1322] Specifically, the server uses a generative AI model to create optimal questions and dialogue content based on prioritized needs and emotional state.

[1323] Step 7:

[1324] Hearings

[1325] The user checks the generated interview content through the terminal interface and conducts questionnaires and interviews with local people. The input data is the generated interview content. The output data is the interview results.

[1326] Specifically, the user asks a question displayed on the interface and inputs the answer.

[1327] Step 8:

[1328] Saving the results of the hearing

[1329] The terminal transmits the user's hearing result to the server. The input data is the hearing result. The output data is the transmitted hearing result.

[1330] The server stores the received hearing results in a database. Specifically, the terminal assembles the hearing results into a data packet and sends it to the server, which then stores it in the database.

[1331] Step 9:

[1332] Generate a support plan

[1333] The server automatically generates a specific support plan based on the stored hearing data and emotion data. The input data are the hearing data and emotion data. The output data is a specific support plan.

[1334] Specifically, the server analyzes the hearing data and emotional data and uses a generative AI model to formulate support plans, such as food supply plans and medical team dispatch plans.

[1335] Step 10:

[1336] Sharing support plans

[1337] The server shares the generated support plan with related support groups and organizations. The input data is the specific support plan. The output data is the support plan notified to the support group.

[1338] Specifically, the server uses email notifications and a dedicated information sharing system to quickly communicate the generated support plan to relevant parties.

[1339] Through the above processing steps, the system enables rapid and accurate risk identification in disaster areas and the generation of support plans, enabling effective support activities.

[1340] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1341] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1342] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1343] [Fourth embodiment]

[1344] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1345] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1346] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1347] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1348] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1349] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1350] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1351] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1352] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1353] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1354] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1355] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1356] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1357] The present invention is a system for realizing an immediate and efficient support system in a disaster area. A specific embodiment of this system will be described below.

[1358] User information collection

[1359] User: In the disaster area, the user uses a device to record the damage situation. The user takes photos of the scene and inputs videos and text notes. In addition, GPS data is also acquired during this collection process, so the location information of the information becomes clear.

[1360] Data transmission and analysis

[1361] Terminal: The collected information is packaged into a single packet and sent to the server. Security technology (e.g., HTTPS) is used for communication, ensuring data security.

[1362] Server: The server analyzes the received packets and applies natural language processing (NLP) technology to the text data and machine learning image recognition technology to the image data, thereby identifying specific needs within the disaster area (e.g., food shortages, lack of shelter, need for medical assistance).

[1363] Prioritizing needs

[1364] Server: The extracted needs are passed through an evaluation algorithm to determine priorities. For example, a list may be generated based on the scale, severity, and extent of impact of the disaster, such as "food shortages (high priority)," "need for medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[1365] User interviews

[1366] Users: Through the system interface, they select a high-priority issue, such as "food shortage," and then conduct surveys and interviews with local people. This allows them to understand specific requests and detailed local conditions.

[1367] Saving the results of the hearing

[1368] Terminal: The user's hearing results are sent to the server, including text and audio data.

[1369] Server: The server stores the received hearing results in a database for future reference and analysis.

[1370] Generate and share support plans

[1371] Server: Based on the stored data, it automatically generates specific assistance plans, such as securing food supply routes, setting up evacuation shelters, and dispatching medical teams.

[1372] Server: The generated support plans are shared with volunteer groups and support organizations, enabling them to carry out support activities quickly and effectively.

[1373] This system will enable rapid and efficient emergency response in the event of a disaster. Specifically, by immediately grasping the local situation and appropriately prioritizing the necessary assistance, it will be possible to provide assistance without delay or waste. Such a system is expected to save many lives and contribute to the early recovery of disaster-stricken areas.

[1374] The processing flow will be explained below.

[1375] Step 1:

[1376] User: In the disaster area, the user uses the device to record the damage situation. Specifically, the user takes photos and videos of the damaged buildings and inputs text notes. GPS data is also automatically acquired during this process.

[1377] Step 2:

[1378] Device: Collected photos, videos, text data, and GPS data are packaged into a single data packet, which is then sent from the device to the server using a secure communication protocol (e.g., HTTPS).

[1379] Step 3:

[1380] Server: Analyzes the received data packets, applying natural language processing (NLP) techniques to text data and image recognition techniques using machine learning to image data. This identifies specific needs in the disaster area (e.g., food shortages, lack of evacuation shelters, need for medical assistance).

[1381] Step 4:

[1382] Server: Prioritizes the identified needs based on an evaluation algorithm. Evaluation criteria include the scale, severity, and scope of impact of the disaster. For example, priorities are determined as "food shortages (high priority)," "need for medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[1383] Step 5:

[1384] Server: Generate interview content based on prioritized needs. For example, regarding "food shortages," create questions about what is lacking and what specific support is needed.

[1385] Step 6:

[1386] Users: Through the system interface, they can display the generated interview content and conduct surveys and interviews with local people, thereby gaining a better understanding of specific requirements and detailed on-site conditions.

[1387] Step 7:

[1388] Terminal: The user sends the collected hearing results back to the server, including text data and audio data.

[1389] Step 8:

[1390] Server: Stores received hearing results in a database for later reference and further analysis.

[1391] Step 9:

[1392] Server: Based on the stored interview data, the server automatically generates specific assistance plans, including, for example, securing food supply routes, setting up evacuation shelters, and dispatching medical teams.

[1393] Step 10:

[1394] Server: Shares the generated support plan with relevant volunteer groups and support organizations. Sharing includes email notifications and uploading information to a web portal. Based on this information, volunteer groups and support organizations can quickly begin responding.

[1395] Example 1

[1396] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1397] In order to provide appropriate assistance immediately at disaster sites, it is necessary to quickly and accurately grasp the detailed situation on the ground and identify specific assistance needs. However, conventional methods take time to collect information, making it difficult to quickly determine the priorities of needs. Furthermore, there has been a lack of systems that can effectively organize collected information, automatically generate specific assistance plans, and share them with assistance organizations. The objective of this invention is to solve these problems.

[1398] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1399] In this invention, the server includes: means for acquiring information collected by users using their terminals in the disaster area; means for assembling the acquired information into a single data packet and transmitting it to the server; means for analyzing text data using natural language processing technology and image data using machine learning technology; means for identifying specific needs in the disaster area from the analyzed data; means for prioritizing the identified needs using an evaluation algorithm; means for generating interview content based on the priorities and presenting it to the user; means for recording the results of the interview conducted by the user and transmitting them to the server; means for the server to generate a specific support plan based on the recorded results of the interview; and means for sharing the generated support plan with support organizations. This enables rapid and effective information collection and needs identification when a disaster occurs, automatically generating an appropriate support plan based on priorities, and providing efficient support.

[1400] A "disaster area" is a specific area that has suffered damage or suffering as a result of a natural or man-made disaster.

[1401] A "user" is a person or organization that uses a terminal to collect information within the disaster area.

[1402] A "terminal" is an electronic device such as a mobile phone, smartphone, or tablet that a user uses to gather information or conduct interviews.

[1403] A "data packet" is a single integrated data set that includes text data, media data, and location information collected by a terminal.

[1404] A "server" is a central processing unit or system that receives collected data packets and performs analysis and generates support plans.

[1405] "Natural language processing technology" is a technology that allows computers to understand and analyze language spoken by humans.

[1406] "Machine learning technology" is a technology in which a computer learns from past data and performs highly accurate analysis and predictions on new data.

[1407] "Needs" are the identified needs for goods and services in the disaster area.

[1408] "Priority" is an order determined based on the importance or urgency of identified needs.

[1409] "Hearing contents" are questions and survey items that users use when conducting questionnaires or interviews with local people.

[1410] "Hearing results" are data obtained as a result of questionnaires or interviews with users.

[1411] A "support plan" is a plan for specific support measures generated by the server, and includes food supplies, medical support, and the establishment of evacuation shelters.

[1412] "Support groups" are professional organizations or volunteer groups that provide support to disaster areas.

[1413] The present invention is a system for realizing an immediate and efficient support system in a disaster area. A specific embodiment of this system will be described below.

[1414] User information collection

[1415] Users record the damage situation in the disaster area using devices such as smartphones and tablets. Specifically, users take photos of the scene with their smartphone's camera and enter text notes using the device's application. This process also utilizes the device's GPS function, and location information is automatically included in the collected information.

[1416] Data packet generation

[1417] The device combines text notes entered by the user, photos and videos taken, and location information into a single data packet, which also contains checks to ensure that each piece of data is included correctly.

[1418] Sending data

[1419] The terminal sends the generated data packet to the server using the HTTPS protocol. Before sending, the data is encrypted to ensure the security of the transmission. It also records a log of whether the transmission was successful or not.

[1420] Data reception and analysis

[1421] The server receives data packets sent from the device. After receiving the data, it first decodes the data and then analyzes the text data using natural language processing (NLP) techniques. For example, NLP techniques such as TensorFlow and SpaCy can be used. Furthermore, image and video data are analyzed using machine learning models (e.g., ResNet and YOLO) to extract important information such as "collapsed buildings" and "injured people."

[1422] Identifying and prioritizing needs

[1423] The server uses the analyzed data to identify specific needs within the disaster area. For example, needs such as "food shortages" and "medical assistance required" are extracted. An evaluation algorithm (e.g., AHP, Point Ranking System) is then used to determine the priority of the identified needs. Here, taking into account factors such as the scale and severity of the disaster, the server lists the needs as "food shortages (high priority)," "medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[1424] User interviews

[1425] Through the system interface, users select a high-priority need (e.g., "food shortage") and then survey or interview local people. The survey and interview contents are recorded as text and audio data. Specific prompts could include questions such as, "What food is most needed in this area?"

[1426] Saving the results of the hearing

[1427] The device then sends the collected data to the server, again using the HTTPS protocol for secure and reliable data transmission. The server then stores the data in a database for future analysis and reference.

[1428] Generate and share support plans

[1429] The server generates a specific relief plan based on the results of the stored interviews. For example, the server automatically formulates things like "securing food supply routes," "location of evacuation shelters," and "medical team dispatch plans." This relief plan may also be created using a generative AI model (e.g., a GPT-based model). The generated relief plan is shared with relief groups and related organizations via email or a dedicated relief management system.

[1430] Specific examples

[1431] For example, a user in an area affected by an earthquake uses a smartphone to record the damage situation. They take photos and videos, enter a text note saying "There is a food shortage in this area," and send the data to a server. The server analyzes the received data and identifies food shortages as a high-priority need. To gain a more detailed understanding of the situation, the user uses a tablet to survey residents and sends the results to the server. The server then uses this information to generate a food supply assistance plan and shares it with relief organizations.

[1432] Prompt Sentence Examples

[1433] "Analyze the provided image and text data to identify needs within the disaster area. Also, prioritize the identified needs and generate a specific support plan."

[1434] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1435] Step 1:

[1436] Users use devices such as smartphones and tablets in disaster areas to collect disaster information. Input includes taking images and videos, entering text notes, and acquiring GPS data. Specifically, users launch the smartphone's camera app and take photos of the local situation. They also enter content such as "There is a food shortage in this area" into a text input field within the application. This records detailed information about the disaster site on the device. Output is a set of image data, video data, text notes, and GPS data.

[1437] Step 2:

[1438] The device combines the collected image data, video data, text memos, and GPS data into a single data packet. The input is the various data acquired in step 1. Specifically, the device converts the various data into a packet format and performs an error check. For example, it checks whether the data is stored correctly and records a log of the packet generation. The output is the combined data packet.

[1439] Step 3:

[1440] The terminal sends the generated data packet to the server using the HTTPS protocol. The input is the data packet generated in step 2. Specifically, the terminal first encrypts the data packet, and then starts sending it to the server using HTTPS. A log of whether the transmission was successful or failed is also recorded at the same time. The output is the data packet sent to the server and its transmission log.

[1441] Step 4:

[1442] The server receives and analyzes the data packets sent from the device. The input is the data packet sent in step 3. Specifically, the server first decodes the data and then analyzes the text data using natural language processing (NLP) techniques. For example, NLP techniques such as TensorFlow and SpaCy are used to extract important keywords and phrases from the text notes. It also analyzes images and videos using machine learning models (e.g., ResNet and YOLO) to identify important information (e.g., "collapsed building" and "injured person"). The output is needs information extracted from the analyzed text and image data.

[1443] Step 5:

[1444] The server identifies specific needs within the disaster area from the analyzed data. The input is the needs information analyzed in step 4. Specifically, it integrates the results of text analysis and image recognition to create a list of needs such as "food shortage" and "medical assistance needed." The output is a list of identified needs.

[1445] Step 6:

[1446] The server uses an evaluation algorithm to prioritize the identified needs. The input is the list of needs identified in step 5. Specifically, it applies an evaluation algorithm (e.g., AHP or Point Ranking System) to evaluate the importance of each need. For example, it determines that "food shortage" is a high priority, "medical support" is a medium priority, and "lack of shelter" is a low priority. The output is a prioritized list of needs.

[1447] Step 7:

[1448] The user selects high-priority needs through the system interface and then surveys or interviews local people. The input is the prioritized needs list generated in step 6. Specifically, the user brings in a tablet and asks questions to residents, such as "What type of food do you need most?" The answers are entered as text or voice data. The output is the survey or interview response data.

[1449] Step 8:

[1450] The terminal sends the collected interview results to the server. The input is the response data obtained in step 7. Specifically, the terminal sends the response data back to the server using the HTTPS protocol and records the transmission log. The output is the response data sent to the server and the transmission log.

[1451] Step 9:

[1452] The server stores the results of the interview in a database, making them available for future analysis and reference. The input is the response data submitted in step 8. Specific operations include storing the response data in a database and indexing it to facilitate searchability. The output is the response data stored in the database.

[1453] Step 10:

[1454] The server generates a specific support plan based on the stored data. The input is the response data stored in the database in step 9. Specifically, it uses a generative AI model (e.g., a GPT-based model) to automatically generate support plans such as "securing food supply routes," "location of evacuation shelters," and "medical team dispatch plans." The output is a specific support plan.

[1455] Step 11:

[1456] The server shares the generated support plan with support groups and related organizations. The input is the support plan generated in step 10. The specific operation is to notify the support plan using email or a dedicated support management system. The output is the support plan in a shared state.

[1457] (Application example 1)

[1458] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1459] Building a rapid and efficient support system in disaster areas is a difficult task. It is particularly important to immediately grasp the situation in the affected area and quickly formulate an appropriate support plan. However, conventional methods require time to grasp the situation on the ground and analyze data, which can delay support. Optimizing support activities using autonomous vehicles is also a challenge.

[1460] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1461] In this invention, the server includes means for acquiring information collected by users in the disaster area, means for analyzing the acquired information and identifying needs in the disaster area, means for evaluating and prioritizing the identified needs, means for receiving information collected by the autonomous vehicle, means for analyzing the collected information using image recognition and natural language processing, means for generating a trip plan for the autonomous vehicle based on the analysis results, and means for transmitting the generated trip plan to the autonomous vehicle. This enables rapid and efficient disaster relief, prevents delays in relief activities, and supports the early recovery of disaster-stricken areas.

[1462] "Disaster Area" means an area affected by a natural disaster, technological disaster, or other emergency.

[1463] "User" means an individual or entity that collects information in a disaster area and provides data for analysis and prioritization.

[1464] "Means for obtaining information" refers to devices and methods, including devices (e.g., cameras and GPS) for recording the situation in the disaster area.

[1465] "Means for analyzing data" refers to technologies for analyzing acquired data using natural language processing and machine learning techniques to identify important needs.

[1466] "Means for identifying needs" refers to techniques for extracting the types of support and supplies needed in disaster-stricken areas from the analyzed data.

[1467] A "prioritization tool" is an algorithm that evaluates identified needs and ranks them according to importance and urgency.

[1468] "Means for generating interview content" refers to a system for creating the content of questionnaires and interviews to be conducted with local people based on prioritized needs.

[1469] "Means for recording interview results" refers to technology that allows the results of user interviews and questionnaires to be saved in digital format for later analysis and reference.

[1470] The "means for generating support plans" is a system that automatically generates specific support plans and routes based on the recorded interview results.

[1471] "Means for sharing with support organizations" refers to how the generated support plans can be shared with volunteer groups and other support organizations through a digital platform.

[1472] An "autonomous vehicle" is a vehicle that uses artificial intelligence and sensor technology to move autonomously and transport supplies and collect information in disaster-stricken areas.

[1473] "Means for receiving information collected by an autonomous vehicle" refers to technology for transmitting data collected by an autonomous vehicle (e.g., images and GPS information) to a server and receiving it on the server side.

[1474] "Image recognition" is a technology that analyzes captured video and images and extracts important elements.

[1475] "Natural language processing" is a technology for analyzing and understanding text data, and involves mechanically analyzing collected notes and interview content.

[1476] "Means for generating operation plans" refers to technology that determines the optimal travel routes and schedules for autonomous vehicles based on collected data and analysis results.

[1477] The "means for transmitting the operation plan" refers to a method for transmitting the generated operation plan to the autonomously driven vehicle using wireless communication or the like.

[1478] The present invention provides a system for realizing a rapid and efficient support system in a disaster area. Specific embodiments will be described below.

[1479] User information collection

[1480] Users use a dedicated device in the disaster area to record the damage situation. The device is equipped with a high-resolution camera and a GPS module, allowing it to take photos and videos of the local situation. The device also has a text input function, allowing users to enter notes about the local situation. This allows location information to be attached to the collected information.

[1481] Data transmission and analysis

[1482] The device then packets the collected information and sends it to the server via HTTPS. The server then analyzes the received data packets, first analyzing the text data using natural language processing (NLP) technology. The image data is then analyzed using image recognition technology that utilizes machine learning. At this stage, specific needs within the disaster area (e.g., food shortages or the need for medical assistance) are identified.

[1483] Prioritizing needs

[1484] The server then evaluates and prioritizes the identified needs based on the analysis results. The evaluation algorithm considers factors such as the scale, scope, and severity of the disaster and generates a list such as "food shortages (high priority)" or "medical assistance needs (medium priority)."

[1485] User interviews

[1486] Next, the system interface presents users with high-priority items (e.g., "food shortages") and they can then conduct surveys and interviews with local people, thereby gaining a detailed understanding of their specific needs and the situation on the ground.

[1487] Saving the results of the hearing

[1488] The user's hearing results (text data and voice data) are then sent from the device to the server, which stores the data in a database for future analysis and reference.

[1489] Generate and share support plans

[1490] The server automatically generates specific relief plans based on the stored data. These plans include food supply routes, shelter locations, and medical team dispatch plans. These plans are shared with volunteer groups and relief organizations, enabling swift and effective relief efforts.

[1491] The role of autonomous vehicles

[1492] The system includes an autonomous vehicle that collects information on-site. The vehicle is equipped with a camera, GPS module, and communication module, and automatically patrols and collects information on the damage situation on-site. The vehicle periodically sends data to a server, allowing real-time analysis on the server side. The server generates a vehicle operation plan based on the collected data and transmits it to the vehicle via wireless communication. This allows the autonomous vehicle to efficiently collect information and patrol along the optimal route.

[1493] Examples:

[1494] For example, if an earthquake occurs in a certain area, an autonomous vehicle will be the first to be dispatched to the site. The video and GPS data collected by the vehicle will be sent to a server, which will analyze it and identify local needs. At the same time, users will use their devices to record the local situation from various angles and send it to the server. Based on this information, the server will quickly provide support plans to volunteer groups and aid organizations, enabling appropriate responses.

[1495] Example prompt sentence:

[1496] How can I use the Disaster Response Autonomous Driving Support System to record and transmit local damage information and generate an appropriate support plan?

[1497] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1498] Step 1:

[1499] Users use the device in the disaster area to record the damage situation. The device is equipped with a high-resolution camera and a GPS module, which allows users to take photos and videos of the disaster area and save them along with their location information. Text notes can also be entered.

[1500] Input: Local conditions (photos, videos, text notes, GPS data)

[1501] Output: Disaster situation data saved on the device

[1502] Step 2:

[1503] The device then packets the collected information and sends it to the server using HTTPS, where it is encrypted to ensure the data remains secure.

[1504] Input: Disaster situation data stored on the device

[1505] Output: Data packet sent to the server

[1506] Step 3:

[1507] The server analyzes the received data packets. First, text data is analyzed using natural language processing (NLP) technology, and image data is analyzed using image recognition technology using machine learning.

[1508] Input: Data packet sent to the server

[1509] Output: Analyzed needs data (text, images)

[1510] Step 4:

[1511] The server evaluates and prioritizes the identified needs based on the analysis results. An evaluation algorithm ranks the needs according to their importance and urgency.

[1512] Input: Parsed needs data

[1513] Output: Prioritized needs list

[1514] Step 5:

[1515] The server generates a hearing content based on the priority and presents it to the user. The hearing content is created using natural language processing technology.

[1516] Input: Prioritized Needs List

[1517] Output: The hearing content presented to the user

[1518] Step 6:

[1519] Based on the information provided, the user conducts questionnaires and interviews with local people to understand their specific needs and detailed on-site conditions, and records the results on the device.

[1520] Input: Presented hearing content, responses from local people

[1521] Output: Hearing results saved on the device

[1522] Step 7:

[1523] The device then sends the results of the hearing back to the server, including text and audio data.

[1524] Input: Hearing results saved on the device

[1525] Output: Hearing data sent to the server

[1526] Step 8:

[1527] The server stores the hearing data in a database for future analysis and reference.

[1528] Input: Hearing data sent to the server

[1529] Output: Hearing data stored in a database

[1530] Step 9:

[1531] The server automatically generates specific assistance plans based on the stored data, including food supply routes, shelter locations, and medical team dispatch plans.

[1532] Input: Hearing data stored in a database

[1533] Output: Auto-generated support plan

[1534] Step 10:

[1535] The autonomous vehicle will patrol the disaster area and collect information using cameras and GPS modules, which will then be periodically sent to a server.

[1536] Input: Current state of the disaster area (photography data taken by autonomous vehicles, GPS data)

[1537] Output: Autonomous vehicle data sent to the server

[1538] Step 11:

[1539] The server analyzes the information collected by the autonomous vehicle and analyzes the data using image recognition and natural language processing technologies.

[1540] Input: Data sent from the autonomous vehicle

[1541] Output: Parsed autonomous vehicle data

[1542] Step 12:

[1543] The server generates a driving plan for the autonomous vehicle based on the analysis results, and transmits the generated driving plan to the autonomous vehicle via wireless communication.

[1544] Input: Parsed autonomous vehicle data

[1545] Output: Trip plan sent to the autonomous vehicle

[1546] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1547] This invention is a system for quickly and effectively identifying needs in disaster areas and streamlining relief activities. Furthermore, by combining this system with an emotion engine that recognizes the user's emotions, it is possible to collect information more accurately and generate relief plans.

[1548] User information collection

[1549] Users: In the disaster area, they use their devices to record the damage situation. They take photos of the scene, record videos, and input text notes. GPS data is automatically acquired during the collection process, making the location of the information clear.

[1550] Data transmission and analysis

[1551] Device: Collected photos, videos, text data, and GPS data are packaged into a single data packet and sent to the server. Security technology (e.g., HTTPS) is used for communication, ensuring data safety.

[1552] Server: The server analyzes the received data packets, applying natural language processing (NLP) techniques to the text data and machine learning image recognition techniques to the image data, thereby identifying specific needs in the disaster area (e.g., food shortages, lack of shelter, need for medical assistance).

[1553] Prioritizing needs

[1554] Server: Prioritizes the identified needs based on an evaluation algorithm. Evaluation criteria include the scale, severity, and scope of impact of the disaster. For example, priorities are determined as "food shortages (high priority)," "need for medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[1555] Utilizing the Emotion Engine

[1556] Server: Uses an emotion engine to analyze the voice data and facial expression data collected by the user. Voice analysis technology is used to identify emotions (e.g., anxiety, fear, relief) from the voice data, and image analysis technology is used to recognize changes in facial expressions to identify emotions.

[1557] User interviews

[1558] User: The user displays the generated interview content through the system interface and conducts surveys and interviews with local people. The interview content also takes into account the results of the emotion engine, and appropriate questions and dialogue are conducted according to the user's emotional state.

[1559] Saving the results of the hearing

[1560] Terminal: The user's hearing results are sent back to the server, including text data and voice data.

[1561] Server: The server stores the received hearing results in a database for later reference and further analysis.

[1562] Generate and share support plans

[1563] Server: Based on the stored interview data and emotional data, the server automatically generates specific support plans, such as securing food supply routes, setting up evacuation shelters, and dispatching medical teams.

[1564] Server: The generated support plan is shared with relevant volunteer groups and support organizations. This information is shared via email notifications and a dedicated information sharing system. Based on this information, volunteer groups and support organizations can quickly begin responding.

[1565] Specific examples

[1566] As a concrete scenario, consider a case where there is a serious food shortage in a disaster area. A user takes a photo of the food shortage situation on-site and inputs specific items in a text memo. This information, along with the emotional data of local residents (e.g., anxiety or impatience), is analyzed by an emotion engine and sent to the server. Based on this data, the server creates an "emergency food supply plan" and simultaneously generates a comprehensive support plan that includes counseling support to alleviate anxiety. This plan is shared with volunteer organizations, allowing for swift and accurate support activities to be initiated.

[1567] In this way, the system can comprehensively grasp the local situation and the emotional state of residents, and provide optimal assistance, greatly improving the efficiency and effectiveness of disaster response.

[1568] The processing flow will be explained below.

[1569] Step 1:

[1570] User: In the disaster area, the user uses the device to record the damage situation. For example, the user takes photos and videos of damaged buildings and enters text notes. At this time, the device automatically acquires GPS data.

[1571] Step 2:

[1572] Device: Collected photos, videos, text data, and GPS data are packaged into a single data packet, which is then sent from the device to the server using a secure communication protocol (e.g., HTTPS).

[1573] Step 3:

[1574] Server: Analyzes the received data packets, applying natural language processing (NLP) techniques to text data and image recognition techniques using machine learning to image data. This identifies specific needs in the disaster area (e.g., food shortages, lack of evacuation shelters, need for medical assistance).

[1575] Step 4:

[1576] Server: Prioritizes the identified needs based on an evaluation algorithm. Evaluation criteria include the scale, severity, and scope of impact of the disaster. For example, priorities are determined as "food shortages (high priority)," "need for medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[1577] Step 5:

[1578] Server: Analyzes the user's voice data and facial expression data using an emotion engine. Voice data is analyzed using voice analysis technology to identify emotions (e.g., anxiety, fear, relief), and facial expression data is analyzed using image analysis technology to recognize changes in facial expressions and identify emotions.

[1579] Step 6:

[1580] Server: Generates interview content based on priorities. For example, regarding "food shortages," it sets questions about what exactly is in short supply, how much is needed, and how anxious residents are.

[1581] Step 7:

[1582] User: Through the system interface, the user displays the generated interview content and conducts surveys and interviews with local people. Emotional data is also taken into account here, and appropriate dialogue is conducted. For example, if a resident is feeling anxious, questions that will reassure them are added.

[1583] Step 8:

[1584] Terminal: The user sends the collected hearing results back to the server. The transmission includes text data and audio data.

[1585] Step 9:

[1586] Server: Stores received hearing results and emotion data in a database for future reference and further analysis.

[1587] Step 10:

[1588] Server: Based on the stored interview results and emotional data, the server automatically generates specific support plans, such as securing food supply routes, setting up evacuation shelters, and dispatching medical teams, as well as providing counseling support to reduce residents' anxiety.

[1589] Step 11:

[1590] Server: The generated support plan is shared with relevant volunteer groups and support organizations. This is done via email notifications and a dedicated information sharing system. Based on this information, volunteer groups and support organizations can quickly begin responding.

[1591] Example 2

[1592] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1593] Conventional disaster relief systems have difficulty quickly and accurately grasping local needs and the emotional state of disaster victims, resulting in delays in the creation and implementation of relief plans. Furthermore, the accuracy of the analysis and prioritization of collected data is low, making it difficult to provide optimal relief. Furthermore, there are issues with security and efficiency when sharing relief plans. The present invention aims to solve these issues, streamline relief activities in disaster areas, and provide rapid and accurate relief.

[1594] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1595] In this invention, the server includes means for analyzing acquired information using natural language processing technology and image recognition technology to identify needs in the disaster area, means for prioritizing the identified needs based on an evaluation algorithm, and means for recording the results of user interviews in accordance with the generated interview content. This makes it possible to quickly and accurately grasp local needs and create and implement optimal support plans.

[1596] An "information recording device" is a device used by users to record the situation in a disaster area, and includes smartphones, tablets, and the like.

[1597] "Location information" is data indicating the geographic location of information collected in the disaster area, and is obtained by GPS.

[1598] A "data packet" is a unit of data that allows collected photos, videos, text, GPS data, etc. to be handled as a single unit.

[1599] "Communication path" refers to the network and protocols used to send and receive data packets between a terminal and a server.

[1600] "Natural language processing technology" is a technology for analyzing text data, understanding its meaning, and extracting information, and includes NLP technology.

[1601] "Image recognition technology" refers to technology for analyzing image data and video data and identifying the objects and situations contained therein.

[1602] An "evaluation algorithm" is a calculation method that determines priorities based on identified needs, according to evaluation criteria such as their importance and urgency.

[1603] "Hearing content" refers to the questions and questionnaires given to local people, and includes items to gain a detailed understanding of the disaster situation and needs.

[1604] An "emotion analysis engine" is a software technology for identifying emotions by analyzing voice data and facial expression data, and includes voice analysis technology and image analysis technology.

[1605] A "support plan" is a plan for specific support activities in the disaster area, including food supply, setting up shelters, and dispatching medical teams.

[1606] "Support groups" refers to organizations and groups that provide support in disaster areas, including volunteer groups and government agencies.

[1607] This invention is a system for quickly and effectively identifying needs in disaster areas and streamlining relief activities. It includes specific means for users to acquire and analyze information collected in disaster areas, and generate and share relief plans based on the results. Furthermore, by combining this system with an emotion engine, the invention achieves more accurate information collection and relief plan generation.

[1608] Hardware and Software Configuration

[1609] User information collection

[1610] Users record the local situation in the disaster area using information recording devices such as smartphones and tablets. Users take photos and videos and enter detailed information in text memos. During this process, the device automatically acquires GPS data and records location information.

[1611] Creating and sending data packets

[1612] The device assembles the information collected by the user into a single data packet. The data packet includes photos, videos, text, GPS data, user ID, and a timestamp. The device then sends this data packet to the server using HTTPS. To ensure data security, the communication is encrypted using SSL / TLS.

[1613] Data analysis and emotion recognition

[1614] The server analyzes the received data packets. First, it uses natural language processing technology to analyze the text data and perform keyword extraction and sentiment analysis. Specifically, it uses natural language processing technologies such as spaCy and BERT. Next, it uses image recognition technology to analyze the photo and video data and identify the specific needs of the disaster area. During this process, it uses machine learning frameworks such as TensorFlow and PyTorch. Furthermore, it uses an emotion engine to analyze the voice data and facial expression data and identify the user's emotional state. Google Cloud Speech-to-Text and image analysis technology are used for emotion analysis.

[1615] Prioritizing needs

[1616] The server uses the analysis results to prioritize the needs of the disaster area based on an evaluation algorithm. Evaluation criteria include the scale, severity, and scope of impact of the disaster. For example, priorities are determined as "food shortages (high priority)," "need for medical assistance (medium priority)," and "lack of evacuation shelters (low priority)."

[1617] Conducting interviews and sending results

[1618] The user checks the generated interview content through the system interface and conducts surveys and interviews with local people. The system generates an appropriate list of questions based on the data collected by the user. The user then asks residents questions based on these questions and collects their responses. This data (voice data and text data) is also repackaged into data packets and sent to the server.

[1619] Generate and share support plans

[1620] The server stores the received interview results and emotional data in a database and generates a specific support plan based on that information. Using a generative AI model (e.g., OpenAI's GPT series), a specific support plan is created based on the input data. Examples of support plans include an "emergency food supply plan," "shelter locations," and "medical team dispatch plan." The generated support plan is shared with relevant support and volunteer organizations via email notifications and a dedicated information-sharing system.

[1621] Specific examples

[1622] As a concrete scenario, consider a case where there is a serious food shortage in a disaster area. The user takes photos of the food shortage situation on-site and enters specific items in text memos. This information, along with the emotional data of local residents (e.g., anxiety and impatience), is analyzed by an emotion engine and sent to the server. The server uses this data to create an "emergency food supply plan" and generate a comprehensive support plan, including counseling support to alleviate anxiety. This plan is shared with volunteer organizations, allowing for swift and accurate support activities to be initiated.

[1623] Prompt Sentence Examples

[1624] "Identify specific needs in disaster areas and use the emotion engine to generate assistance plans. Suggest comprehensive assistance proposals based on user-collected information and emotion data."

[1625] As described above, this system comprehensively grasps the local situation and the emotional state of residents, and provides optimal support, thereby significantly improving the efficiency and effectiveness of disaster response.

[1626] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1627] Step 1: Gather information

[1628] User:

[1629] Users record the situation in the disaster area using an information recording device such as a smartphone or tablet.

[1630] Input: Photos, videos, text notes, GPS data

[1631] How it works: Users take photos and videos of the disaster site and write detailed descriptions of the situation in text notes. The device automatically acquires GPS data and records the location information.

[1632] Output: All collected data (photos, videos, text notes, GPS data)

[1633] Step 2: Generate and send data packets

[1634] Device:

[1635] The terminal assembles the information collected by the user into a single data packet.

[1636] Input: Photos, videos, text notes, GPS data, user ID, timestamp

[1637] How it works: The device assembles all collected data into data packets and sends them to the server using HTTPS, encrypting them with SSL / TLS to ensure secure communications.

[1638] Output: Data packet sent to the server

[1639] Step 3: Receiving and analyzing data

[1640] server:

[1641] The server analyzes the received data packets.

[1642] Input: Data packets (photos, videos, text notes, GPS data, user ID, timestamp)

[1643] Specific behavior:

[1644] Natural Language Processing: The server applies natural language processing techniques (e.g., spaCy or BERT) to the text notes to perform keyword extraction and sentiment analysis.

[1645] Image Recognition: Apply image recognition techniques (e.g., TensorFlow or PyTorch) to photos and videos to identify specific needs in disaster areas.

[1646] Emotion Analysis: An emotion engine is used to analyze voice and facial expression data to identify emotional states. Google Cloud Speech-to-Text is used for voice analysis, and image analysis technology is applied to recognize changes in facial expressions.

[1647] Output: List of needs, user's emotional state

[1648] Step 4: Prioritize your needs

[1649] server:

[1650] The server prioritizes the identified needs based on a rating algorithm.

[1651] Input: List of needs, assessment criteria (scale, severity, scope of impact, etc.)

[1652] Specific behavior: The server scores the identified needs according to the evaluation criteria and determines their priorities.

[1653] Output: A prioritized list of needs

[1654] Step 5: Generate and present the interview

[1655] server:

[1656] The server generates hearing content based on the priority and presents it to the user.

[1657] Input: List of prioritized needs, user information

[1658] Specific operation: The server uses natural language processing technology to automatically generate a list of appropriate questions, which are then sent to the terminal and displayed to the user.

[1659] Output: Terminal displaying the interview contents

[1660] Step 6: Conducting an interview and sending the results

[1661] User:

[1662] The user checks the generated interview content and conducts surveys and interviews with local people.

[1663] Input: Interview details, residents' responses (audio data, text data)

[1664] Specific operation: The user asks questions to residents and collects their answers. This data is then repackaged into a data packet and sent to the server.

[1665] Output: Data packet of the hearing results sent to the server

[1666] Step 7: Analyze and save the results of the interview

[1667] server:

[1668] The server analyzes the received hearing results and stores the data in a database.

[1669] Input: Data packet of hearing results (audio data, text data)

[1670] What it does: The server performs speech and text analysis and stores the results of the hearing in a database for later analysis and reference.

[1671] Output: Hearing results stored in a database

[1672] Step 8: Create and share your support plan

[1673] server:

[1674] The server automatically generates a specific support plan based on the stored hearing data and emotional data.

[1675] Input: Hearing results and emotion data stored in the database

[1676] How it works: Using generative AI models (e.g., OpenAI's GPT series), it generates relief plans tailored to the needs of disaster-stricken areas. The relief plans are then shared with relevant relief and volunteer organizations.

[1677] Output: A support plan shared with support groups and volunteer organizations

[1678] (Application example 2)

[1679] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1680] Conventional disaster response systems did not adequately improve the efficiency of information gathering and support activities in disaster areas, making it difficult to quickly and accurately identify and respond to risks. Furthermore, support plans that properly considered the emotional state of victims were often insufficient, resulting in delays in appropriate responses based on local needs. This resulted in issues such as delays in providing necessary support, and delays in rescue efforts for victims and risk reduction measures.

[1681] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring information collected by users in the disaster area, means for analyzing the acquired information to identify needs in the disaster area, means for evaluating the identified needs and prioritizing them, means for generating interview content based on the priorities and presenting it to the user, means for recording the user's interview results, means for generating a specific support plan based on the recorded interview results, means for sharing the generated support plan with support organizations, emotion analysis means for analyzing voice data and facial expression data collected by the user to identify emotions, means for presenting the generated interview content to the user taking the emotion data into consideration, risk assessment means for identifying and prioritizing risks in the disaster area, means for generating a risk response plan based on the input data, and means for sharing the generated risk response plan with related organizations. This enables rapid and accurate risk identification and support plan generation, as well as providing appropriate support that takes into account the emotional state of the disaster victims.

[1682] A "disaster area" is an area affected by a natural or man-made disaster.

[1683] "User" means an individual or entity that collects information in the disaster area and operates the system.

[1684] "Means for acquiring" refers to the method or device for collecting and storing the information collected by the user.

[1685] "Means of analysis" refers to the techniques and algorithms used to interpret the acquired information and understand its content.

[1686] "Needs" refers to the assistance and services required in the disaster area.

[1687] A "prioritization tool" is an evaluation method or criteria used to select the most important needs from among those identified.

[1688] "Hearing content" refers to questions and interview items used to hear the opinions and requests of local people.

[1689] "Presenting means" refers to the technology or device for displaying the generated hearing content to the user.

[1690] "Recording means" refers to a method or device for saving the results of a user's hearing.

[1691] A "support plan" refers to the plans and policies for specific support activities in the disaster area.

[1692] "Means of sharing" refers to the techniques and methods for communicating the generated support plans to relevant support groups and organizations.

[1693] "Voice data" refers to digital data that records the voices of users and disaster victims.

[1694] "Facial expression data" refers to image data or video data that captures the facial expressions of disaster victims or users.

[1695] "Emotion analysis means" refers to technologies and algorithms for analyzing voice data and facial expression data to identify emotions.

[1696] "Risk" refers to potential dangers or problems in the disaster area.

[1697] "Risk assessment tools" are methods and techniques for assessing the importance and priority of risks identified in a disaster area.

[1698] A "risk response plan" is a specific action plan or policy for dealing with identified risks.

[1699] "Related organizations" include volunteer groups and government agencies that carry out disaster relief activities.

[1700] This invention is a system that supports rapid and effective risk identification and support plan generation in disaster areas. In particular, it takes into account user emotion data to provide more accurate information collection and risk response.

[1701] Information gathering

[1702] User:

[1703] Users use their smartphones in disaster areas to collect local information, including photos, videos, and text notes, and GPS data is automatically added to the collected information, making the location of the collected information clear.

[1704] Data transmission and analysis

[1705] Device:

[1706] Photos, videos, text data, and GPS data collected by users are packaged into a single data packet and sent to a server using HTTPS, ensuring data security.

[1707] server:

[1708] The server analyzes the received data packets, applying natural language processing (NLP) techniques to the text data and machine learning image recognition techniques to the image data. This allows specific needs in the disaster area to be identified, such as food shortages and the need for medical assistance.

[1709] Prioritizing needs

[1710] server:

[1711] The identified needs are prioritized based on an assessment algorithm, which includes criteria such as the scale, severity, and scope of impact of the disaster. For example, priorities are determined as follows: food shortages (high priority), medical assistance needs (medium priority), and lack of shelter (low priority).

[1712] Utilizing the Emotion Engine

[1713] server:

[1714] An emotion analysis method is used to analyze the voice data and facial expression data collected by the user. The voice data is analyzed using voice analysis technology to identify emotions, and the facial expression data is analyzed using image analysis technology to recognize changes in facial expressions to identify emotions. This allows the emotional state of the victim to be properly taken into account.

[1715] User interviews

[1716] User:

[1717] The system displays the generated interview content through the system interface, and conducts surveys and interviews with local people. Emotional data is also reflected in the interview content, and appropriate questions and dialogue are conducted according to the user's emotional state.

[1718] Saving the results of the hearing

[1719] Device:

[1720] The user's hearing results are sent to the server again. The data sent includes text data and voice data.

[1721] server:

[1722] The received interview results are stored in a database for later reference and further analysis.

[1723] Generate and share support plans

[1724] server:

[1725] Based on the stored interview data and emotional data, the system automatically generates specific support plans, such as securing food supply routes, setting up evacuation shelters, and planning the dispatch of medical teams.

[1726] server:

[1727] The generated support plan is shared with relevant volunteer groups and support organizations via email notifications and a dedicated information sharing system.

[1728] Specific examples

[1729] As a concrete scenario, consider a case where there is a serious food shortage in a disaster area. A user takes a photo of the food shortage situation on-site and inputs specific items in a text memo. This information, along with the emotional data of local residents (e.g., anxiety or impatience), is analyzed using an emotion analysis tool and sent to the server. Based on this data, the server creates an "emergency food supply plan" and generates a comprehensive support plan that includes counseling support to alleviate anxiety. This plan is shared with volunteer organizations, allowing for swift and accurate support activities to be initiated.

[1730] Prompt Sentence Examples

[1731] Generate a plan for:

[1732] On-site image and video data

[1733] Text note: "Rubble hazard location confirmed."

[1734] GPS data: Latitude 35.6895, Longitude 139.6917

[1735] Voice data (emotion analysis): Anxiety

[1736] This allows the system to comprehensively grasp the local situation and the emotional state of residents, and provide optimal assistance, significantly improving the efficiency and effectiveness of disaster response.

[1737] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1738] Step 1:

[1739] Information gathering

[1740] Users use their smartphones in disaster areas to collect local information, including photos, videos, text notes, and automatically acquired GPS data.

[1741] The device combines the collected photos, videos, text data, and GPS data into a single data packet, and the output data is the combined data packet.

[1742] Step 2:

[1743] Data transmission

[1744] The terminal sends data packets to the server using HTTPS communication. The input data is the integrated data packet. The output data is the data packet sent over the secure communication channel.

[1745] As a specific operation, the terminal encrypts the data and sends a transmission request to the server.

[1746] Step 3:

[1747] Data analysis

[1748] The server analyzes the received data packets. The input data includes photos, videos, text data, and GPS data.

[1749] Natural language processing (NLP) technology is applied to text data, and image recognition technology using machine learning is applied to image data. The output data is the analyzed specific needs.

[1750] Specifically, the server tokenizes the text data, inputs it into an analysis model (e.g., BERT) to extract needs, and classifies image data using a machine learning model to identify the state of the disaster.

[1751] Step 4:

[1752] Prioritizing needs

[1753] The server prioritizes the identified needs based on an evaluation algorithm. The input data are the analyzed needs. The output data is a prioritized list of needs.

[1754] Specifically, the server scores the severity and scope of the need and ranks it from high priority to low priority.

[1755] Step 5:

[1756] Utilizing the Emotion Engine

[1757] The server analyzes the voice and facial expression data collected by the user. The input data is the voice and facial expression data. The output data is the identified emotional state.

[1758] The voice data is analyzed using voice analysis technology to identify emotions, and the facial expression data is analyzed using image analysis technology to recognize changes in facial expressions. Specifically, the voice data is input into a voice recognition model and an emotion label is attached. The facial expression data is input into an image recognition model and emotions are similarly identified.

[1759] Step 6:

[1760] Generate hearing content

[1761] The server generates a hearing based on the prioritized needs and emotional data. The input data is the prioritized needs list and the identified emotional state. The output data is the generated hearing.

[1762] Specifically, the server uses a generative AI model to create optimal questions and dialogue content based on prioritized needs and emotional state.

[1763] Step 7:

[1764] Hearings

[1765] The user checks the generated interview content through the terminal interface and conducts questionnaires and interviews with local people. The input data is the generated interview content. The output data is the interview results.

[1766] Specifically, the user asks a question displayed on the interface and inputs the answer.

[1767] Step 8:

[1768] Saving the results of the hearing

[1769] The terminal transmits the user's hearing result to the server. The input data is the hearing result. The output data is the transmitted hearing result.

[1770] The server stores the received hearing results in a database. Specifically, the terminal assembles the hearing results into a data packet and sends it to the server, which then stores it in the database.

[1771] Step 9:

[1772] Generate a support plan

[1773] The server automatically generates a specific support plan based on the stored hearing data and emotion data. The input data are the hearing data and emotion data. The output data is a specific support plan.

[1774] Specifically, the server analyzes the hearing data and emotional data and uses a generative AI model to formulate support plans, such as food supply plans and medical team dispatch plans.

[1775] Step 10:

[1776] Sharing support plans

[1777] The server shares the generated support plan with related support groups and organizations. The input data is the specific support plan. The output data is the support plan notified to the support group.

[1778] Specifically, the server uses email notifications and a dedicated information sharing system to quickly communicate the generated support plan to relevant parties.

[1779] Through the above processing steps, the system enables rapid and accurate risk identification in disaster areas and the generation of support plans, enabling effective support activities.

[1780] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1781] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1782] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1783] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1784] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1785] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1786] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1787] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1788] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1789] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1790] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1791] In the above embodi...

Claims

1. A means for acquiring information collected by users in a disaster area; A means of analyzing the information obtained and identifying the needs of the disaster area; a means of assessing and prioritizing identified needs; means for generating hearing contents based on the priority and presenting the contents to a user; a means for recording the results of the user's hearing; A means for generating a specific support plan based on the recorded interview results; A system that includes a means for sharing the generated support plan with support organizations.

2. 2. The system according to claim 1, wherein the means for generating the hearing content uses natural language processing technology.

3. 2. The system of claim 1, wherein the means for analyzing information and identifying needs uses machine learning technology.

Images