System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting image and sensor data of a disaster-stricken area; means for transmitting the collected data to a central server; means for integrating the data transmitted to the central server; means for predicting a probability of presence of a missing person based on the integrated data; and means for displaying and notifying a prediction result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] During natural disasters such as earthquakes, many people go missing, requiring rapid search efforts with limited time and resources. However, conventional methods have difficulty understanding the local situation and identifying missing people, resulting in inefficient search efforts. The present invention aims to solve these problems and realize the rapid discovery of missing people and the efficiency of rescue efforts. [Means for solving the problem]

[0005] The present invention is a system that includes a means for collecting images and sensor data of a disaster area, a means for transmitting the collected data to a central server, a means for integrating the data transmitted to the central server, a means for predicting the probability of the presence of missing persons based on the integrated data, and a means for displaying and notifying the prediction results.

[0006] First, images and sensor data from the disaster area are collected using drones and environmental sensors, and the data is sent to a central server in real time. The central server then integrates the received data and uses machine learning algorithms to predict the probability of missing persons. The prediction results are then visually displayed and promptly notified to rescue teams and the Self-Defense Forces, enabling efficient search operations. In this way, the present invention can optimally allocate limited resources and improve the efficiency of search operations.

[0007] "Disaster area" refers to an area affected by an earthquake or other natural disaster.

[0008] "Imagery and sensor data" refers to a broad range of data, including visual data obtained from drones and environmental sensors, physical observation data, and location information.

[0009] "Collection means" refers to devices and technological methods capable of acquiring imagery and sensor data, including, for example, drones and various environmental sensors.

[0010] "Transmission means" refers to the communication means or technical methods for transporting collected data to a central server, including, for example, wireless communication and 4G / 5G networks.

[0011] "Central Server" means a centralized computer system for consolidating, storing, and analyzing collected data.

[0012] "Means of integration" refers to a technical method for processing multiple collected data as a single entity and converting it into a format suitable for data analysis.

[0013] "Means for predicting the probability of the presence of missing persons" refers to technical methods that use machine learning or other algorithms to identify the likely location of missing persons based on collected data.

[0014] "Means for displaying and notifying" refers to the technical methods for visually displaying the prediction results and promptly notifying the necessary parties (e.g., the Self-Defense Forces and rescue teams). [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This invention relates to a system for efficiently and quickly searching for missing persons after natural disasters such as earthquakes. This system includes a means for collecting images and sensor data from the disaster area, a means for transmitting the collected data to a central server, a means for integrating and analyzing the data, and a means for predicting the probability of the presence of missing persons based on the analysis results, and displaying and notifying the user.

[0037] Program processing overview

[0038] The program in this system includes the following steps as its main processing steps.

[0039] 1. Data collection in the affected areas

[0040] Terminal

[0041] The devices include drones and environmental sensors. Drones operating over the affected area use high-resolution cameras to capture images of the affected area, while sensors on the ground and in the air collect seismic intensity, temperature, humidity, and other important environmental data.

[0042] A specific example would be a drone flying over a disaster area, photographing the damage to buildings, and sensors recording seismic intensity and temperature data on the ground surface.

[0043] 2. Data transmission and storage

[0044] Terminal

[0045] The collected data is transmitted in real time to a central server via wireless communication or 4G / 5G networks. Image data captured by the drone's camera is uploaded to the server as a large file. Data from sensors is also transmitted in the same way using IoT protocols.

[0046] For example, images taken by drones can be instantly uploaded to a central server via a 5G network, while data collected by sensors on the ground can be transmitted via the MQTT protocol.

[0047] 3. Data integration and analysis

[0048] server

[0049] The central server consolidates all the data it receives, applies image processing algorithms to the image data, and analyzes the damage to buildings. The sensor data is processed in real time, and the results are used to generate an environmental map of the entire disaster area.

[0050] Specifically, the server analyzes images received from the drone to identify damaged areas of buildings, and maps seismic intensity and temperature data to provide a visual understanding of the situation in the affected areas.

[0051] 4. Missing Person Prediction

[0052] server

[0053] Based on the analyzed data, an AI algorithm is used to predict the probability of missing persons being present, and by referencing past data and the current situation in the disaster area, it identifies locations in specific areas or buildings where missing persons are likely to be present.

[0054] For example, the server inputs data into a trained AI model to calculate areas where there are likely to be many missing people, and based on this result, analyzes patterns in which missing people are likely to occur.

[0055] 5. Display and notification of results

[0056] server

[0057] The prediction results are sent to search and rescue teams in real time. The server generates a map showing areas where there is a high probability of missing people and sends it to relevant parties. These parties can use this map to develop effective search plans.

[0058] For example, the server highlights areas on a map where there is a high probability of missing persons being found in red and sends this information to the Self-Defense Forces via email. It also displays the information in real time on a dedicated application, enabling a rapid response.

[0059] Specific examples

[0060] One example would be a drone flying over a disaster area, capturing images of the entire town with a high-resolution camera. Along with the captured images, ground sensors collect seismic intensity information, which is then immediately sent to a central server. The server analyzes the received data, and an AI model accurately predicts the probability of the presence of missing persons. The predictions are then displayed on a map and notified to the Self-Defense Forces and rescue teams, allowing for efficient search operations.

[0061] The processing flow will be explained below.

[0062] Step 1: Collect data

[0063] Terminal

[0064] Action 1: A drone flies over the affected area and takes high-resolution images with its camera.

[0065] Action 2: Environmental sensors collect data such as seismic intensity, temperature, humidity, and gas concentration in real time.

[0066] Action 3: The victim's smartphone obtains location information and collects data.

[0067] Step 2: Sending data

[0068] Terminal

[0069] Operation 1: The image data captured by the drone is transmitted to a central server via wireless communication or 4G / 5G networks.

[0070] Action 2: The data collected by the environmental sensors is sent to a central server using an IoT protocol (e.g., MQTT).

[0071] Action 3: The location information obtained by the victim's smartphone is sent to a central server via GPS and the Internet.

[0072] Step 3: Integrate the data

[0073] server

[0074] Operation 1: The server temporarily saves the received image data and stores it in a database.

[0075] Action 2: The server receives the sensor data and synthesizes it into an appropriate format.

[0076] Action 3: The server combines different types of data, such as images, sensors, and location information, into a single dataset.

[0077] Step 4: Analyze the data

[0078] server

[0079] Action 1: The server applies image processing algorithms to analyze the integrated data and identify the damage to the building.

[0080] Action 2: The server analyzes the sensor data in real time and grasps the environmental situation of the entire disaster area.

[0081] Action 3: The server analyzes the location data of the victims and plots the distribution of the victims on a map.

[0082] Step 5: Predict missing persons

[0083] server

[0084] Action 1: The server inputs the analyzed data into an AI model to predict areas where the missing person is likely to be located.

[0085] Step 2: The server takes past data into account and filters the output prediction results to further improve accuracy.

[0086] Action 3: The server lists the final prediction results and saves the list.

[0087] Step 6: Notification and display of results

[0088] server

[0089] Action 1: The server visualizes the prediction results and displays them on a map.

[0090] Action 2: The server notifies the search and rescue team of the prediction results.

[0091] Action 3: Receive feedback based on the information notified by the server and take action to reflect it in the next prediction.

[0092] Step 7: Conduct a search operation

[0093] User

[0094] Action 1: The rescue team formulates a search plan based on the prediction results received from the server.

[0095] Action 2: Rescue teams carry out specific search operations on site.

[0096] Action 3: The rescue team reports the results of the on-site search to the server, so that they can be reflected as data for the next step.

[0097] Example 1

[0098] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0099] Rapid and efficient search for missing persons in natural disasters such as earthquakes is a key challenge in rescue operations. However, understanding the situation in widespread disaster areas and identifying locations with a high probability of missing persons taking time and requiring significant human resources. Conventional methods have difficulty in accurately collecting and analyzing data in real time, limiting their effectiveness in situations where a rapid response is required.

[0100] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0101] In this invention, the server includes means for collecting images and environmental data of the disaster area, means for transmitting the collected data to a central server, means for storing the data transmitted to the central server, means for integrating the stored data, means for analyzing the integrated data, means for predicting the probability of the presence of missing persons based on the analyzed data, and means for displaying and notifying the results of the prediction. This enables real-time data collection and analysis over a wide area of ​​the disaster area, enabling rapid and efficient search activities for missing persons.

[0102] "Images" refer to photographs and video data used to obtain visual information about the disaster area.

[0103] "Environmental data" refers to data used to measure environmental conditions in the affected areas, such as seismic intensity, temperature, and humidity.

[0104] "Collection means" refers to devices that acquire data from disaster areas using equipment such as drones and environmental sensors.

[0105] "Transmission means" refers to a mechanism for transmitting collected data to a central server using wireless communication or a network.

[0106] The "storage means" is a storage system for storing collected data in the server.

[0107] "Integration" is the process of combining multiple stored data sets and processing them as a unified data set.

[0108] "Analysis tools" are algorithms and programs used to analyze the integrated data and grasp the situation in the disaster-stricken areas.

[0109] A "predictive tool" is a generative AI model or other algorithm used to calculate the probability of a missing person's existence based on analyzed data.

[0110] "Display means" refers to a device or interface for visually presenting the analysis and prediction results.

[0111] "Notification means" is a system for communicating prediction results to relevant parties in real time.

[0112] An "autonomous mobile object" is a device that can move automatically without external operation, such as a drone or robot.

[0113] An "environmental information acquisition device" is a sensor or measuring instrument used to collect environmental data such as temperature, humidity, and seismic intensity.

[0114] A "data processing algorithm" is a calculation method or program used to analyze acquired data.

[0115] The present invention relates to a system for quickly and efficiently searching for missing persons in the event of a natural disaster such as an earthquake. The system includes a data collection means, a data transmission means, a data storage means, a data integration means, a data analysis means, a means for predicting the probability of the presence of missing persons, a result display means, and a notification means.

[0116] Data collection methods

[0117] Terminal

[0118] The devices used include drones and environmental sensors. Drones are equipped with high-resolution cameras and fly over affected areas to collect image data. Environmental sensors collect environmental data such as seismic intensity, temperature, and humidity in real time. Drones use automatic navigation functions to fly over affected areas along pre-set routes. For example, drones may take aerial photographs of the damage to an entire town, while ground sensors measure seismic intensity and temperature.

[0119] Data transmission method

[0120] Terminal

[0121] The collected data is transmitted to a central server in real time via wireless communication or 4G / 5G networks. The large amount of image data collected by the drone is instantly uploaded using the drone's built-in communication module. Data from the sensors is also transmitted to the server using IoT protocols (e.g., MQTT). Specifically, the images captured by the drone are uploaded to the central server via the 5G network, and the ground sensors transmit seismic intensity data via the MQTT protocol.

[0122] Data storage means

[0123] server

[0124] The server first stores the data it receives. Image data sent from the drone is stored on a dedicated storage server, and data from the sensors is stored in a database system (e.g., MongoDB). Each piece of data is given a timestamp to clarify the time the data was collected.

[0125] Data Integration Methods

[0126] server

[0127] The server integrates multiple stored data sets. For example, it combines image data and sensor data to generate an environmental map of the entire disaster area. By overlaying the image data and displaying seismic intensity and temperature information in color, the current situation in the disaster area can be visually grasped. Image processing algorithms and data mapping techniques are used.

[0128] Data Analysis Methods

[0129] server

[0130] The server analyzes the integrated data, using image processing algorithms to assess the damage to buildings and quantify the degree of damage. It also analyzes collected seismic intensity and temperature data, integrating this information to understand the detailed situation in the affected area. For example, images of damaged buildings can be input into an AI model to classify the damage state in detail.

[0131] A method for predicting the probability of a missing person's existence

[0132] server

[0133] Based on the analyzed data, the server uses a generative AI model to predict the probability of missing persons. It uses past data and the current situation in the disaster area as a reference to calculate the possibility of missing persons in a specific area or building. Specifically, the latest data is input into an AI model trained on data from specific areas where many missing persons have occurred in the past, and the server outputs the probability of missing persons being present.

[0134] Result display means and notification means

[0135] server

[0136] The server notifies search teams and rescue teams of the prediction results. The prediction results are displayed on a map, highlighting areas with a high probability of missing people in color. The results are also sent to the Self-Defense Forces and rescue teams via email or a dedicated app. For example, areas with a high probability of missing people being present on a map can be displayed in red, and detailed information about the area can be attached to send a real-time notification to the rescue team.

[0137] Prompt Sentence Examples

[0138] "Please provide an overview of the data collection and analysis system for predicting the probability of missing persons in disaster areas."

[0139] The above is a specific embodiment for carrying out the present invention. This system enables real-time data collection and analysis in a wide area of ​​a disaster, enabling rapid and efficient search activities for missing persons.

[0140] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0141] Step 1: Data collection

[0142] Terminal

[0143] The device uses a drone and environmental sensors. The drone uses a high-resolution camera to capture images of the sky over the affected area, while environmental sensors on the ground measure environmental data such as seismic intensity, temperature, and humidity. Specifically, the drone flies automatically along a pre-set route, capturing detailed images of the damage to buildings and the terrain. The environmental sensors also collect seismic intensity and temperature data measured every five seconds and store it in their internal memory.

[0144] Input: Visual and environmental information of the affected area

[0145] Output: High-resolution disaster image data and environmental sensor data

[0146] Step 2: Send data

[0147] Terminal

[0148] The collected data is transmitted to a central server in real time via wireless communication or 4G / 5G networks. Image data captured by drones is transmitted as large files, while data from environmental sensors is transmitted every second using IoT protocols (such as MQTT). For example, images captured by drones can be immediately uploaded using the 5G network, and seismic intensity data recorded by sensors can be sent to the server in batches every minute.

[0149] Input: High-resolution disaster image data and environmental sensor data

[0150] Output: Data sent to the central server

[0151] Step 3: Save Data

[0152] server

[0153] The central server stores the received data. Specifically, image data sent from the drone is stored on a dedicated storage server, and data from the sensors is stored in a database system (e.g., MongoDB). Each piece of data is given a timestamp, and the time the data was collected is clearly managed.

[0154] Input: Data sent to the central server

[0155] Output: Time-stamped saved data

[0156] Step 4: Data Integration

[0157] server

[0158] The server integrates the various stored data. Specifically, it combines image data and sensor data to generate an environmental map of the entire disaster area. In doing so, it makes full use of image processing algorithms, overlays the image data, and color-codes information such as seismic intensity and temperature to visually display the situation in the disaster area.

[0159] Input: Time-stamped stored data

[0160] Output: Unified environment map

[0161] Step 5: Data analysis

[0162] server

[0163] The server analyzes the integrated data, using image processing algorithms to assess the damage to buildings and quantify the degree of damage. It also analyzes collected seismic intensity and temperature data, integrating this information to understand the current situation in the affected area. For example, images of damaged buildings can be input into an AI model, which then classifies the damage state in detail (completely damaged, partially damaged, undamaged, etc.).

[0164] Input: Unified environment map

[0165] Output: Analysis results (evaluation of damage state, etc.)

[0166] Step 6: Predict the probability of the missing person being present

[0167] server

[0168] The server uses a generative AI model based on the analyzed data to predict the probability of missing persons. It calculates the possibility of missing persons in specific areas or buildings by referring to past data and the current situation in the disaster area. It inputs the latest analysis results into an AI model trained on data from specific areas where many people have gone missing in the past, and outputs the probability of missing persons.

[0169] Input: Analysis results

[0170] Output: Prediction result of the probability of the missing person being present

[0171] Step 7: Results display and notification

[0172] server

[0173] The server notifies search teams and rescue teams of the prediction results. The prediction results are displayed on a map, and areas with a high probability of missing persons being present are highlighted in color. This information is also sent to the Self-Defense Forces and rescue teams via email or a dedicated app. For example, areas with a high probability of missing persons being present can be displayed in red on a map, and detailed information about the area can be attached to send a real-time notification.

[0174] Input: Missing person existence probability prediction result

[0175] Output: Display and notification of prediction results

[0176] (Application example 1)

[0177] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0178] Searches for missing people during natural disasters must be carried out quickly and efficiently. However, conventional methods tend to be slow in data collection, information integration, and analysis, resulting in a lack of speed and accuracy in rescue operations. Data accuracy and real-time reporting are also issues. To solve these problems, a new search system utilizing the latest technology is needed.

[0179] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0180] In this invention, the server includes means for collecting images and sensor data of the disaster area, means for transmitting the collected data to a central server, means for integrating the data transmitted to the central server, means for predicting the probability of the presence of missing persons based on the integrated data, means for displaying and notifying the prediction results, means for using data collected by cameras and sensors mounted on the autonomous vehicle, means for performing analysis and prediction in real time using an AI model, and means for displaying and notifying the prediction results on a smartphone app, thereby enabling a fast and accurate search for missing persons.

[0181] "Images and sensor data from disaster areas" refers to visual information and wireless communication data collected using devices such as cameras and environmental sensors in areas where natural disasters have occurred.

[0182] "Collection methods" refers to the devices and methods used to gather important data from disaster areas, including drones, autonomous vehicles, and environmental sensors.

[0183] The "central server" is a computer system that receives and integrates all collected data and performs analysis and predictions.

[0184] "Integration means" refers to a method or device that brings together multiple collected data and prepares them for analysis.

[0185] A "prediction method" is an algorithm or machine learning model that calculates and predicts the probability of a missing person's presence based on the integrated data.

[0186] "Display and notification means" refers to devices and applications that display prediction results to users and notify them in real time. Specifically, this includes smartphone apps and dedicated display devices.

[0187] "Autonomous vehicle-mounted cameras and sensors" refers to data collection devices, such as high-resolution cameras and LiDAR sensors, mounted on an autonomous vehicle.

[0188] An "AI model" is an artificial intelligence algorithm that analyzes data and makes predictions based on the knowledge it has learned.

[0189] "Smartphone app" refers to dedicated application software that runs on a smartphone and is accessible to users.

[0190] In this invention, in order to build a system that can efficiently and quickly search for missing people in the event of a natural disaster, images and sensor data from the disaster area are collected, and based on that, an AI model is used to analyze, predict, display, and notify. The specific configuration and operation of the system are described below.

[0191] System configuration

[0192] Hardware:

[0193] Autonomous vehicles: Equipped with high-resolution cameras and LiDAR sensors, they will collect images and environmental data of the affected area.

[0194] Drones: Equipped with high-resolution cameras, they take images of the affected areas.

[0195] Environmental sensors: Installed on the ground surface, they collect environmental data such as seismic intensity, temperature, and humidity.

[0196] Central Server: A computer system that receives, stores, and analyzes all collected data.

[0197] Smartphone: A device that runs an application that displays and notifies users of prediction results.

[0198] software:

[0199] Image processing algorithm: Use libraries such as OpenCV to analyze image data from the disaster area.

[0200] AI model: A deep learning algorithm that uses frameworks such as TensorFlow to predict the probability of a missing person's presence.

[0201] IoT protocols: Use MQTT or similar to send sensor data to a server in real time.

[0202] Smartphone app: A dedicated application for displaying and notifying prediction results in real time.

[0203] Data collection and transmission

[0204] Autonomous vehicles, drones, and environmental sensors will collect data from the disaster area. Cameras mounted on autonomous vehicles and drones will capture high-resolution images, and LiDAR sensors will collect terrain data. Environmental sensors will capture important environmental data in real time, such as seismic intensity, temperature, and humidity. This data will be transmitted to a central server via wireless communication technology (e.g., 5G network).

[0205] Data integration and analysis

[0206] The central server consolidates all the data it receives. Image processing algorithms are applied to the image data using OpenCV to identify damaged areas of buildings. Sensor data is integrated in real time to generate an environmental map of the entire affected area.

[0207] Missing person prediction and notification

[0208] Based on the combined data, an AI model predicts the probability of a missing person being found. Deep learning algorithms using TensorFlow perform this analysis. The predictions are displayed in real time on a smartphone app and sent to the search team.

[0209] Examples of specific examples and AI prompts

[0210] As a concrete example, immediately after a disaster occurs, autonomous vehicles patrol the affected area and collect data using cameras and LiDAR sensors. This data is immediately transmitted to a central server via 5G communication. The server uses OpenCV and TensorFlow to analyze the images and sensor data and predict the probability of the presence of missing persons. This prediction result is displayed on a smartphone app and notified to the search team.

[0211] Example of a generated AI prompt:

[0212] Use the latest images and sensor data from the disaster area to predict areas where missing people are likely to be found. Use OpenCV for image processing and TensorFlow for data analysis. Use a deep learning model to run an algorithm to locate missing people in real time.

[0213] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0214] Step 1:

[0215] Autonomous vehicles and drones collect data from the affected areas. Cameras capture high-resolution image data, LiDAR sensors measure topographical data, and environmental sensors simultaneously collect environmental data such as seismic intensity, temperature, and humidity.

[0216] Input: Real-time images of the affected area, topographical data, and environmental data

[0217] Output: High-resolution image data, terrain data, environmental sensor data

[0218] How it works: Autonomous vehicles patrol the affected area, drones collect data from the air, cameras take images, LiDAR sensors collect topographical data, and environmental sensors capture various seismic and weather data.

[0219] Step 2:

[0220] The collected data is transmitted to a central server via wireless communication technology (e.g., 5G networks).

[0221] Input: High-resolution image data, terrain data, environmental sensor data

[0222] Output: Consolidated data sent to a central server

[0223] How it works: All collected data is sent to a central server in real time, using wireless communication and the MQTT protocol for instant data upload.

[0224] Step 3:

[0225] The server consolidates and stores the received data.

[0226] Input: Consolidated data sent to a central server

[0227] Output: A consolidated dataset

[0228] What it does: The server receives the input data, adjusts it to fit each format (e.g., decompresses image data, normalizes sensor data), and stores it as a single integrated dataset.

[0229] Step 4:

[0230] The combined data is then analyzed using AI models: image data is analyzed using OpenCV, and sensor data is processed using deep learning models.

[0231] Input: Integrated dataset

[0232] Output: Analysis result (probability of missing person existence)

[0233] How it works: The integrated data is fed into an AI model using TensorFlow. Image data is analyzed using OpenCV to identify damaged areas and critical points, and environmental data is analyzed using a deep learning model. This calculates the probability of the presence of missing people.

[0234] Step 5:

[0235] Based on the analysis results, the location information and probability of the missing person's presence are predicted and sent to a smartphone app, where they are displayed and notified in real time.

[0236] Input: Analysis result (probability of missing person existence)

[0237] Output: Display and notification on smartphone app

[0238] What it does: The server sends the analysis results to a smartphone app. The app receives the results, displays them visually, and sends notifications. Areas with a high probability of missing people are highlighted on a map for the search team user.

[0239] Step 6:

[0240] Search teams using a smartphone app can efficiently carry out search activities based on the displayed prediction results.

[0241] Input: Display and notification on smartphone app

[0242] Output: Efficient search operations

[0243] What it does: Search team users use the information displayed in the app to optimize their missing person search plans and respond quickly.

[0244] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0245] This invention relates to a system for efficiently and quickly searching for missing persons after natural disasters such as earthquakes. This system includes a means for collecting images and sensor data from the disaster area, a means for transmitting the collected data to a central server, a means for integrating and analyzing the data, and a means for predicting the probability of the presence of missing persons based on the analysis results, and displaying and notifying the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the efficiency of search activities is further improved.

[0246] Program processing overview

[0247] The program in this system includes the following main processes:

[0248] 1. Data collection in the affected areas

[0249] Terminal

[0250] Drones and environmental sensors will collect data from the disaster area. Specifically, drones will take images with high-resolution cameras, and environmental sensors will collect real-time data on seismic intensity, temperature, humidity, gas concentration, etc. Location information will also be acquired from the smartphones of disaster victims, which will then be provided as data.

[0251] 2. Data transmission and storage

[0252] Terminal

[0253] The collected data is sent to a central server via wireless communication or 4G / 5G networks. Image data captured by drones is sent as large files, while data from sensors is sent using IoT protocols (e.g., MQTT). Location information from victims' smartphones is also sent via GPS and the Internet.

[0254] 3. Data integration and analysis

[0255] server

[0256] The central server consolidates all the data it receives. It applies image processing algorithms to the image data to identify the extent of damage to buildings. The sensor data is processed in real time, and the results are used to generate an environmental map of the entire affected area. It also analyzes the location data of victims and plots their distribution on the map.

[0257] 4. Missing Person Prediction

[0258] server

[0259] Based on the analyzed data, an AI algorithm is used to predict the probability of missing persons being present. Past data and the current situation in the disaster area are referenced to identify areas or buildings where missing persons are likely to be present. The prediction results are listed and saved.

[0260] 5. Operation of the Emotion Engine

[0261] Terminals and Servers

[0262] The emotion engine recognizes emotions by analyzing the voices and facial expressions of users participating in search operations. Emotion data is sent in real time to a central server for analysis. For example, if a user's stress level is high, that information can be used to adjust the priority of search operations.

[0263] 6. Display and notification of results

[0264] server

[0265] The server notifies search and rescue teams of the results of its predictions of missing persons, visualizes the results, and displays them on a map. It also displays emotion data collected by the emotion engine, helping to plan and coordinate the search.

[0266] Specific examples

[0267] For example, a drone flies over a disaster area and photographs the entire town with a high-resolution camera. Along with the captured images, a ground sensor collects seismic intensity information, and this data is sent to a central server. The server analyzes the received data, and an AI model predicts with high accuracy the probability of the presence of missing persons. The prediction results are then displayed on a map and notified to the Self-Defense Forces and rescue teams. Search team users also use an emotion engine to send their own stress and fatigue to the server. The server uses this information to suggest areas to prioritize search and efficient resource allocation.

[0268] As a result, the present invention supports rapid and efficient search activities during disasters, contributing to the early discovery and rescue of missing persons.

[0269] The processing flow will be explained below.

[0270] Step 1: Collect data

[0271] Terminal

[0272] Action 1: A drone flies over the affected area and takes images with a high-resolution camera.

[0273] Action 2: Environmental sensors collect data such as seismic intensity, temperature, humidity, and gas concentration in real time at the installation location.

[0274] Action 3: The victim's smartphone obtains location information and records that data.

[0275] Step 2: Send and store data

[0276] Terminal

[0277] Operation 1: The image data captured by the drone is transmitted to a central server via wireless communication or 4G / 5G networks.

[0278] Action 2: The data acquired by the environmental sensors is sent to a central server using an IoT protocol (e.g., MQTT).

[0279] Action 3: The location information acquired by the victim's smartphone is sent to a central server using GPS.

[0280] Step 3: Integrate the data

[0281] server

[0282] Action 1: The server centrally stores all received data.

[0283] Action 2: The server combines image data, sensor data, and location information and converts them into a single dataset.

[0284] Action 3: The server converts the consolidated data into a usable format for subsequent analysis.

[0285] Step 4: Analyze the data

[0286] server

[0287] Action 1: Based on the integrated data, the server applies image processing algorithms to identify the extent of damage to the building.

[0288] Step 2: The server generates an environment map in real time based on the sensor data.

[0289] Step 3: The server analyzes the location information of the victims and visualizes their distribution on a map.

[0290] Step 5: Predict missing persons

[0291] server

[0292] Action 1: The server inputs the analyzed data into an AI model to predict areas where the missing person is likely to be located.

[0293] Operation 2: The server compares past data with current data and filters the output prediction results to further improve accuracy.

[0294] Action 3: The server lists the final prediction results and saves the list.

[0295] Step 6: Emotion Recognition with the Emotion Engine

[0296] Terminals and Servers

[0297] Action 1: The device analyzes the user's voice and facial expressions to collect emotional data.

[0298] Action 2: The device sends the collected emotion data to the central server.

[0299] Action 3: The server analyzes the emotional data and evaluates the user's stress level and fatigue.

[0300] Step 7: View and notify results

[0301] server

[0302] Action 1: The server visualizes the missing person prediction results and displays them on a map.

[0303] Action 2: The server also displays the emotion data collected by the emotion engine.

[0304] Action 3: The server notifies search and rescue teams of the prediction results in real time.

[0305] Step 8: Conduct a search operation

[0306] User

[0307] Action 1: The rescue team receives the prediction results from the server and formulates a search plan.

[0308] Action 2: Rescue teams carry out specific search operations in the disaster area.

[0309] Action 3: The rescue team feeds back the results of their on-site search to the server, which helps improve the accuracy of their next prediction.

[0310] The above is a detailed explanation of the specific processing steps of the system for searching for missing persons in the event of a disaster that combines an emotion engine, and its operation.

[0311] Example 2

[0312] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0313] Rapid and efficient search for missing persons during disasters is difficult with current technology. Not only must image and sensor data from the disaster area be collected, but that data must also be processed quickly to accurately predict the probability of missing persons being found. It is also necessary to allocate resources efficiently while taking into account the emotions of users participating in the search operation. A system that can solve these problems is needed.

[0314] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0315] In this invention, the server includes means for collecting images and sensor data of the disaster area, means for transmitting the collected data to a central server, means for integrating the data transmitted to the central server, means for predicting the probability of the presence of missing persons based on the integrated data, means for displaying and notifying the user of the prediction result, and means for recognizing the user's emotions and adjusting the priority of search activities, thereby enabling a fast and efficient search for missing persons in a disaster.

[0316] "Disaster area" refers to an area affected by a natural disaster.

[0317] "Images" refers to photographs and video data taken to visually record the situation in the disaster area.

[0318] "Sensor data" refers to various types of data acquired by sensors to collect environmental information in disaster-stricken areas, including seismic intensity, temperature, humidity, gas concentration, etc.

[0319] "Central server" refers to a computer system that stores and analyzes collected data and provides information useful for search operations.

[0320] "Integrating" refers to the process of collecting, organizing, and analyzing different types of data in a centralized manner.

[0321] "AI algorithm" refers to artificial intelligence technology that automatically learns patterns and trends in data and predicts behavior.

[0322] "Probability of presence" refers to the probability that a missing person may be present in a particular location.

[0323] "Prediction result" refers to the probability of a missing person being found calculated using an AI algorithm.

[0324] "Display and notification" refers to the process of visually displaying the analysis results and communicating information to interested parties.

[0325] An "emotion engine" refers to technology that analyzes a user's voice and facial expressions to recognize their emotional state, such as stress or fatigue.

[0326] "Autonomous flying devices" refer to devices that fly automatically and collect images and data without human control. Drones generally fall into this category.

[0327] "Environmental sensors" refer to devices used to measure environmental information in disaster-stricken areas, such as seismometers, temperature sensors, and gas detectors.

[0328] "Users" refers to people engaged in search and rescue activities during disasters, such as members of search and rescue teams.

[0329] The present invention relates to a system for efficiently and quickly searching for missing persons in the event of a natural disaster such as an earthquake. This system collects images and sensor data from the disaster area, analyzes them to predict the probability of the presence of missing persons, and visualizes and notifies the user.

[0330] 1. Data collection in the affected areas

[0331] Terminal

[0332] The system uses drones, environmental sensors, and the victims' smartphones. The drones are equipped with high-resolution cameras to collect images of the entire disaster area. The environmental sensors collect data such as seismic intensity, temperature, humidity, and gas concentration in real time. The victims' smartphones use GPS to obtain location information.

[0333] 2. Data transmission and storage

[0334] Terminal

[0335] The collected data is transmitted to a central server via wireless communication or 4G / 5G networks. The large amount of image data collected by the drone is transmitted using a dedicated bandwidth, and sensor data is transmitted in real time using the MQTT protocol. Smartphone location information is also sent to the central server via the Internet.

[0336] 3. Data integration and analysis

[0337] server

[0338] A central server consolidates all data. High-resolution images are stored in a database, and data from environmental sensors and location information are also managed centrally. Image processing algorithms are applied to analyze the damage to buildings, and sensor data is processed in real time to generate a map of the affected area. The data is plotted on the map based on the location information of victims.

[0339] 4. Missing Person Prediction

[0340] server

[0341] AI algorithms, such as deep learning and machine learning models (CNN, LSTM, etc.), are used to predict the probability of missing persons being found. Based on past data and the current situation in the disaster area, locations with a high probability of missing persons are identified, and the results are listed and saved.

[0342] 5. Operation of the Emotion Engine

[0343] Terminals and Servers

[0344] The emotion engine analyzes the user's voice and facial expressions in real time and sends emotional data to a central server, which then adjusts the priority of search activities if the user's stress level is high.

[0345] 6. Display and notification of results

[0346] server

[0347] Based on the analyzed data, prediction results are visually displayed on a map. Information is then sent to search and rescue teams via a dedicated app, SMS, email, etc. Emotional data is also displayed to assist in the planning and adjustment of search plans.

[0348] Specific examples

[0349] For example, a drone flies over a disaster-hit area, capturing images of the entire town with a high-resolution camera. At the same time, sensors on the ground collect seismic intensity information and gas concentrations, and all data is sent to a central server. The server then integrates and analyzes this data, and an AI model predicts the probability of the presence of missing persons. The results are displayed on a map and communicated to the Self-Defense Forces and rescue teams. Search team members can also use an emotion engine to send their own stress and fatigue levels to the server, which then prioritizes search areas and resource allocations.

[0350] Example prompts for generative AI models

[0351] "In order to quickly search for missing people in disaster areas, please tell me how to collect data, what algorithms should be used to make predictions and analyses, and how to display and notify the results."

[0352] As a result, the present invention supports rapid and efficient search activities in the event of a disaster, contributing to the early discovery and rescue of missing persons.

[0353] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0354] Step 1: Collecting data from affected areas

[0355] The device uses drones and environmental sensors to collect data on the affected area. Specifically, the drone takes images with a high-resolution camera, and the environmental sensors collect data such as seismic intensity, temperature, humidity, and gas concentration. The victim's smartphone also uses GPS to obtain location information.

[0356] Inputs include image data from high-resolution cameras, environmental data from environmental sensors, and smartphone location data.

[0357] As an output, these collected data sets are produced.

[0358] Step 2: Data transmission and storage

[0359] The terminal transmits the collected data to a central server via wireless communication or 4G / 5G networks. Specifically, drone image data is transmitted as a large file, environmental sensor data is transmitted using the MQTT protocol, and smartphone location data is also transmitted via the Internet.

[0360] The input is all the data collected in step 1.

[0361] The output is a data store stored on a central server.

[0362] Step 3: Data integration and analysis

[0363] The server consolidates all the data it receives. Specifically, it stores high-resolution image data in a database, and also centrally manages environmental sensor data and smartphone location information. It applies image processing algorithms to extract and analyze information such as the extent of building damage. It processes the sensor data in real time to generate an environmental map of the entire disaster area.

[0364] The input is the data store saved in step 2.

[0365] The output is a consolidated and analyzed dataset that provides a situation map of the affected area.

[0366] Step 4: Predict missing persons

[0367] The server uses an AI algorithm to predict the probability of missing people based on the integrated data set. Specifically, it uses deep learning models (e.g., CNN, LSTM) to analyze past data and the current situation in the disaster area and identify locations where missing people are likely to be.

[0368] The input is the analyzed dataset generated in step 3.

[0369] The output is a prediction result that lists the probability of the missing person being present.

[0370] Step 5: Emotion Engine in Action

[0371] The device and server analyze the user's voice and facial expressions in real time and send emotional data to a central server, which then aggregates the emotional data and analyzes the user's stress and fatigue levels. Priorities are adjusted based on this information.

[0372] Inputs include voice and facial expression data obtained from the user.

[0373] As an output, the analyzed emotional data is sent to a central server, which adjusts the priorities of search operations.

[0374] Step 6: View and notify results

[0375] The server notifies search and rescue teams of the predicted probability of a missing person's presence. The results are sent via a dedicated app, SMS, or email. Furthermore, the prediction results and emotion data are visually displayed on a map to assist in the planning and coordination of search plans.

[0376] The inputs are the prediction results obtained in step 4 and the emotion data collected in step 5.

[0377] The output is a visualized map and a notification.

[0378] (Application example 2)

[0379] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0380] Efficiently and quickly searching for missing persons during natural disasters is difficult and requires accurate data collection and analysis across a wide area of ​​the disaster area. Furthermore, failure to consider the emotional state of the search team can reduce search efficiency and increase the risk of stress and fatigue. Therefore, a system is needed that enables fast and efficient search operations and enables the early discovery and rescue of missing persons.

[0381] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means including an autonomously driving vehicle that autonomously collects data on the disaster area, means for collecting images and sensor data using drones and environmental sensors, means for transmitting the collected data to a central server, means for integrating and analyzing the data transmitted to the central server, means for predicting the probability of the presence of missing persons based on the integrated data, means for displaying the prediction results on a map and notifying search teams and rescue teams, means including an emotion engine that collects and analyzes emotion data from the search team, and means for adjusting the search plan based on the emotion data. This enables fast and efficient search activities and realizes the early discovery and rescue of missing persons.

[0382] An "autonomous vehicle" refers to a vehicle that can drive autonomously without a driver and travel to a specific destination.

[0383] A "drone" refers to an unmanned aircraft that flies and collects images and sensor data.

[0384] "Environmental sensor" refers to a sensor device for detecting environmental data such as seismic intensity, temperature, humidity, and gas concentration.

[0385] "Central server" refers to a central management system for integrating and analyzing data collected from disaster-stricken areas.

[0386] "Data integration" refers to the process of centrally organizing data collected from multiple sources and making it ready for analysis.

[0387] "Probability of presence" refers to the probability that a missing person may be present in a particular location.

[0388] The "emotion engine" refers to a system that analyzes the voices and facial expressions of the search team and recognizes their emotional state.

[0389] "Emotion data" refers to information collected by the emotion engine that indicates the user's emotional state, such as stress or fatigue.

[0390] "Search plan" refers to a plan drawn up to efficiently search for a missing person.

[0391] A "generative AI model" refers to an artificial intelligence algorithm that learns from large amounts of data and predicts the probability of a missing person's existence, etc.

[0392] The present invention relates to a system for efficiently and quickly searching for missing people during natural disasters. This system is composed of a combination of autonomous vehicles, drones, environmental sensors, a central server, an emotion engine, and a generative AI model. Each part of the present invention is described in detail below.

[0393] Data collection

[0394] The server collects data using autonomous, unmanned vehicles that travel across a wide area of ​​the disaster-stricken area. The autonomous vehicles are equipped with drones and environmental sensors, and take images using high-resolution cameras. Environmental data such as seismic intensity, temperature, humidity, and gas concentration are also collected at the same time. Drones installed in various locations in the disaster-stricken area collect images from an aerial perspective, allowing the entire situation in the disaster-stricken area to be grasped.

[0395] Data transmission

[0396] The devices transmit the collected data to a central server in real time via wireless communication or 4G / 5G networks. Image data is sent as large files, and sensor data is transmitted using IoT protocols (e.g., MQTT). Location information from the victim's smartphone is also sent to the server via GPS and the Internet.

[0397] Data Integration and Analysis

[0398] The server integrates and analyzes all the received data. Using a generative AI model trained using TensorFlow and Keras, it uses image processing algorithms to identify the extent of damage to buildings. Sensor data is also processed in real time, and the results are used to generate an environmental map of the entire disaster area. Furthermore, it analyzes the location data of victims and plots their distribution on the map.

[0399] Missing Persons Prediction

[0400] Based on the analyzed data, the server uses a generative AI model to predict the probability of missing persons being present. It references past data and the current situation in the disaster area to identify locations in specific areas or buildings where missing persons are likely to be present. The prediction results are then listed and saved.

[0401] Emotion Engine Operation

[0402] The server uses an emotion engine to analyze the voices and facial expressions of users participating in the search operation to recognize their emotions. Emotional data is sent to a central server in real time and analyzed there. For example, if a user's stress level is high, that information can be used to adjust the priority of the search operation.

[0403] Displaying and notifying results

[0404] The server notifies search and rescue teams of the predicted missing persons, visualizes the results, and displays them on a map. It also displays emotional data collected by the emotion engine, helping them plan and coordinate their search.

[0405] Specific examples

[0406] For example, in a disaster area where a magnitude 7 earthquake has occurred, autonomous vehicles begin collecting data. High-resolution image data captured by the autonomous vehicles, aerial video data collected by drones, and seismic intensity information from environmental sensors are transmitted in real time to a central server. The server integrates this data, and a generative AI model predicts the probability of missing persons being present. As a result, specific areas are predicted to have a high probability of missing persons, and the search team is notified. Furthermore, the emotional state of the search team is monitored, and the search plan is adjusted if stress levels are high.

[0407] Here are some example prompts:

[0408] “In the event of a natural disaster, predict the probability of missing persons in the affected area using high-resolution images captured by drones and environmental sensor data. Consider variables such as building damage, temperature, humidity, and gas concentrations.”

[0409] As a result, the present invention enables fast and efficient search operations, contributing to the early discovery and rescue of missing persons.

[0410] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0411] Step 1:

[0412] The device uses autonomous vehicles and drones to collect data from disaster-stricken areas. The autonomous vehicles drive autonomously through the disaster-stricken areas and take images with high-resolution cameras. The drones collect aerial image data, while environmental sensors collect real-time data such as seismic intensity, temperature, humidity, and gas concentration. This allows for detailed environmental information to be obtained.

[0413] Input: On-site environmental and image data from the affected areas

[0414] Output: High-resolution image data, seismic intensity data, temperature data, humidity data, gas concentration data

[0415] Step 2:

[0416] The devices transmit the collected data to a central server via wireless communication or 4G / 5G networks. Image data is sent as large files, and sensor data is transmitted using IoT protocols. Location information collected by victims' smartphones is also transmitted via GPS and the internet.

[0417] Input: High-resolution image data, seismic intensity data, temperature data, humidity data, gas concentration data, location information of victims

[0418] Output: Sending data to a central server

[0419] Step 3:

[0420] The server integrates all the received data. Specifically, it applies image processing algorithms to the image data using TensorFlow and Keras to identify the extent of damage to buildings. It also processes the sensor data in real time and generates an environmental map of the entire disaster area based on the results. The location data of the victims is analyzed and the distribution of victims is plotted on the map.

[0421] Input: All data sent to the central server (high-resolution image data, sensor data, location information of victims)

[0422] Output: Image data analysis results, environmental map, disaster victim distribution map

[0423] Step 4:

[0424] The server uses a generative AI model to predict the probability of missing persons being present. It references past data and the current situation in the disaster area to identify locations in specific areas or buildings where missing persons are likely to be present. The prediction results are listed and saved.

[0425] Input: Integrated analysis data (image data analysis results, environmental map, disaster victim distribution map)

[0426] Output: List of predicted probability of missing person existence

[0427] Step 5:

[0428] The server uses an emotion engine to analyze the voices and facial expressions of users participating in the search operation to recognize their emotions. Emotional data is sent to the server in real time for analysis. For example, if a user's stress level is high, the server can adjust the priority of the search operation based on that information.

[0429] Input: User's voice data and facial expression data

[0430] Output: Parsed emotion data

[0431] Step 6:

[0432] The server notifies search and rescue teams of predicted missing persons and displays the results on a map. It also displays emotion data collected by the emotion engine, helping to plan and coordinate the search.

[0433] Input: List of predicted probability of missing persons, analyzed emotion data

[0434] Output: Map predictions, notifications to search and rescue teams, coordinated search plans

[0435] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0436] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0437] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0438] [Second embodiment]

[0439] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0440] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0441] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0442] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0443] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0444] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0445] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0446] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0447] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0448] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0449] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0450] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0451] This invention relates to a system for efficiently and quickly searching for missing persons after natural disasters such as earthquakes. This system includes a means for collecting images and sensor data from the disaster area, a means for transmitting the collected data to a central server, a means for integrating and analyzing the data, and a means for predicting the probability of the presence of missing persons based on the analysis results, and displaying and notifying the user.

[0452] Program processing overview

[0453] The program in this system includes the following steps as its main processing steps.

[0454] 1. Data collection in the affected areas

[0455] Terminal

[0456] The devices include drones and environmental sensors. Drones operating over the affected area use high-resolution cameras to capture images of the affected area, while sensors on the ground and in the air collect seismic intensity, temperature, humidity, and other important environmental data.

[0457] A specific example would be a drone flying over a disaster area, photographing the damage to buildings, and sensors recording seismic intensity and temperature data on the ground surface.

[0458] 2. Data transmission and storage

[0459] Terminal

[0460] The collected data is transmitted in real time to a central server via wireless communication or 4G / 5G networks. Image data captured by the drone's camera is uploaded to the server as a large file. Data from sensors is also transmitted in the same way using IoT protocols.

[0461] For example, images taken by drones can be instantly uploaded to a central server via a 5G network, while data collected by sensors on the ground can be transmitted via the MQTT protocol.

[0462] 3. Data integration and analysis

[0463] server

[0464] The central server consolidates all the data it receives, applies image processing algorithms to the image data, and analyzes the damage to buildings. The sensor data is processed in real time, and the results are used to generate an environmental map of the entire disaster area.

[0465] Specifically, the server analyzes images received from the drone to identify damaged areas of buildings, and maps seismic intensity and temperature data to provide a visual understanding of the situation in the affected areas.

[0466] 4. Missing Person Prediction

[0467] server

[0468] Based on the analyzed data, an AI algorithm is used to predict the probability of missing persons being present, and by referencing past data and the current situation in the disaster area, it identifies locations in specific areas or buildings where missing persons are likely to be present.

[0469] For example, the server inputs data into a trained AI model to calculate areas where there are likely to be many missing people, and based on this result, analyzes patterns in which missing people are likely to occur.

[0470] 5. Display and notification of results

[0471] server

[0472] The prediction results are sent to search and rescue teams in real time. The server generates a map showing areas where there is a high probability of missing people and sends it to relevant parties. These parties can use this map to develop effective search plans.

[0473] For example, the server highlights areas on a map where there is a high probability of missing persons being found in red and sends this information to the Self-Defense Forces via email. It also displays the information in real time on a dedicated application, enabling a rapid response.

[0474] Specific examples

[0475] One example would be a drone flying over a disaster area, capturing images of the entire town with a high-resolution camera. Along with the captured images, ground sensors collect seismic intensity information, which is then immediately sent to a central server. The server analyzes the received data, and an AI model accurately predicts the probability of the presence of missing persons. The predictions are then displayed on a map and notified to the Self-Defense Forces and rescue teams, allowing for efficient search operations.

[0476] The processing flow will be explained below.

[0477] Step 1: Collect data

[0478] Terminal

[0479] Action 1: A drone flies over the affected area and takes high-resolution images with its camera.

[0480] Action 2: Environmental sensors collect data such as seismic intensity, temperature, humidity, and gas concentration in real time.

[0481] Action 3: The victim's smartphone obtains location information and collects data.

[0482] Step 2: Sending data

[0483] Terminal

[0484] Operation 1: The image data captured by the drone is transmitted to a central server via wireless communication or 4G / 5G networks.

[0485] Action 2: The data collected by the environmental sensors is sent to a central server using an IoT protocol (e.g., MQTT).

[0486] Action 3: The location information obtained by the victim's smartphone is sent to a central server via GPS and the Internet.

[0487] Step 3: Integrate the data

[0488] server

[0489] Operation 1: The server temporarily saves the received image data and stores it in a database.

[0490] Action 2: The server receives the sensor data and synthesizes it into an appropriate format.

[0491] Action 3: The server combines different types of data, such as images, sensors, and location information, into a single dataset.

[0492] Step 4: Analyze the data

[0493] server

[0494] Action 1: The server applies image processing algorithms to analyze the integrated data and identify the damage to the building.

[0495] Action 2: The server analyzes the sensor data in real time and grasps the environmental situation of the entire disaster area.

[0496] Action 3: The server analyzes the location data of the victims and plots the distribution of the victims on a map.

[0497] Step 5: Predict missing persons

[0498] server

[0499] Action 1: The server inputs the analyzed data into an AI model to predict areas where the missing person is likely to be located.

[0500] Step 2: The server takes past data into account and filters the output prediction results to further improve accuracy.

[0501] Action 3: The server lists the final prediction results and saves the list.

[0502] Step 6: Notification and display of results

[0503] server

[0504] Action 1: The server visualizes the prediction results and displays them on a map.

[0505] Action 2: The server notifies the search and rescue team of the prediction results.

[0506] Action 3: Receive feedback based on the information notified by the server and take action to reflect it in the next prediction.

[0507] Step 7: Conduct a search operation

[0508] User

[0509] Action 1: The rescue team formulates a search plan based on the prediction results received from the server.

[0510] Action 2: Rescue teams carry out specific search operations on site.

[0511] Action 3: The rescue team reports the results of the on-site search to the server, so that they can be reflected as data for the next step.

[0512] Example 1

[0513] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0514] Rapid and efficient search for missing persons in natural disasters such as earthquakes is a key challenge in rescue operations. However, understanding the situation in widespread disaster areas and identifying locations with a high probability of missing persons taking time and requiring significant human resources. Conventional methods have difficulty in accurately collecting and analyzing data in real time, limiting their effectiveness in situations where a rapid response is required.

[0515] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0516] In this invention, the server includes means for collecting images and environmental data of the disaster area, means for transmitting the collected data to a central server, means for storing the data transmitted to the central server, means for integrating the stored data, means for analyzing the integrated data, means for predicting the probability of the presence of missing persons based on the analyzed data, and means for displaying and notifying the results of the prediction. This enables real-time data collection and analysis over a wide area of ​​the disaster area, enabling rapid and efficient search activities for missing persons.

[0517] "Images" refer to photographs and video data used to obtain visual information about the disaster area.

[0518] "Environmental data" refers to data used to measure environmental conditions in the affected areas, such as seismic intensity, temperature, and humidity.

[0519] "Collection means" refers to devices that acquire data from disaster areas using equipment such as drones and environmental sensors.

[0520] "Transmission means" refers to a mechanism for transmitting collected data to a central server using wireless communication or a network.

[0521] The "storage means" is a storage system for storing collected data in the server.

[0522] "Integration" is the process of combining multiple stored data sets and processing them as a unified data set.

[0523] "Analysis tools" are algorithms and programs used to analyze the integrated data and grasp the situation in the disaster-stricken areas.

[0524] A "predictive tool" is a generative AI model or other algorithm used to calculate the probability of a missing person's existence based on analyzed data.

[0525] "Display means" refers to a device or interface for visually presenting the analysis and prediction results.

[0526] "Notification means" is a system for communicating prediction results to relevant parties in real time.

[0527] An "autonomous mobile object" is a device that can move automatically without external operation, such as a drone or robot.

[0528] An "environmental information acquisition device" is a sensor or measuring instrument used to collect environmental data such as temperature, humidity, and seismic intensity.

[0529] A "data processing algorithm" is a calculation method or program used to analyze acquired data.

[0530] The present invention relates to a system for quickly and efficiently searching for missing persons in the event of a natural disaster such as an earthquake. The system includes a data collection means, a data transmission means, a data storage means, a data integration means, a data analysis means, a means for predicting the probability of the presence of missing persons, a result display means, and a notification means.

[0531] Data collection methods

[0532] Terminal

[0533] The devices used include drones and environmental sensors. Drones are equipped with high-resolution cameras and fly over affected areas to collect image data. Environmental sensors collect environmental data such as seismic intensity, temperature, and humidity in real time. Drones use automatic navigation functions to fly over affected areas along pre-set routes. For example, drones may take aerial photographs of the damage to an entire town, while ground sensors measure seismic intensity and temperature.

[0534] Data transmission method

[0535] Terminal

[0536] The collected data is transmitted to a central server in real time via wireless communication or 4G / 5G networks. The large amount of image data collected by the drone is instantly uploaded using the drone's built-in communication module. Data from the sensors is also transmitted to the server using IoT protocols (e.g., MQTT). Specifically, the images captured by the drone are uploaded to the central server via the 5G network, and the ground sensors transmit seismic intensity data via the MQTT protocol.

[0537] Data storage means

[0538] server

[0539] The server first stores the data it receives. Image data sent from the drone is stored on a dedicated storage server, and data from the sensors is stored in a database system (e.g., MongoDB). Each piece of data is given a timestamp to clarify the time the data was collected.

[0540] Data Integration Methods

[0541] server

[0542] The server integrates multiple stored data sets. For example, it combines image data and sensor data to generate an environmental map of the entire disaster area. By overlaying the image data and displaying seismic intensity and temperature information in color, the current situation in the disaster area can be visually grasped. Image processing algorithms and data mapping techniques are used.

[0543] Data Analysis Methods

[0544] server

[0545] The server analyzes the integrated data, using image processing algorithms to assess the damage to buildings and quantify the degree of damage. It also analyzes collected seismic intensity and temperature data, integrating this information to understand the detailed situation in the affected area. For example, images of damaged buildings can be input into an AI model to classify the damage state in detail.

[0546] A method for predicting the probability of a missing person's existence

[0547] server

[0548] Based on the analyzed data, the server uses a generative AI model to predict the probability of missing persons. It uses past data and the current situation in the disaster area as a reference to calculate the possibility of missing persons in a specific area or building. Specifically, the latest data is input into an AI model trained on data from specific areas where many missing persons have occurred in the past, and the server outputs the probability of missing persons being present.

[0549] Result display means and notification means

[0550] server

[0551] The server notifies search teams and rescue teams of the prediction results. The prediction results are displayed on a map, highlighting areas with a high probability of missing people in color. The results are also sent to the Self-Defense Forces and rescue teams via email or a dedicated app. For example, areas with a high probability of missing people being present on a map can be displayed in red, and detailed information about the area can be attached to send a real-time notification to the rescue team.

[0552] Prompt Sentence Examples

[0553] "Please provide an overview of the data collection and analysis system for predicting the probability of missing persons in disaster areas."

[0554] The above is a specific embodiment for carrying out the present invention. This system enables real-time data collection and analysis in a wide area of ​​a disaster, enabling rapid and efficient search activities for missing persons.

[0555] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0556] Step 1: Data collection

[0557] Terminal

[0558] The device uses a drone and environmental sensors. The drone uses a high-resolution camera to capture images of the sky over the affected area, while environmental sensors on the ground measure environmental data such as seismic intensity, temperature, and humidity. Specifically, the drone flies automatically along a pre-set route, capturing detailed images of the damage to buildings and the terrain. The environmental sensors also collect seismic intensity and temperature data measured every five seconds and store it in their internal memory.

[0559] Input: Visual and environmental information of the affected area

[0560] Output: High-resolution disaster image data and environmental sensor data

[0561] Step 2: Send data

[0562] Terminal

[0563] The collected data is transmitted to a central server in real time via wireless communication or 4G / 5G networks. Image data captured by drones is transmitted as large files, while data from environmental sensors is transmitted every second using IoT protocols (such as MQTT). For example, images captured by drones can be immediately uploaded using the 5G network, and seismic intensity data recorded by sensors can be sent to the server in batches every minute.

[0564] Input: High-resolution disaster image data and environmental sensor data

[0565] Output: Data sent to the central server

[0566] Step 3: Save Data

[0567] server

[0568] The central server stores the received data. Specifically, image data sent from the drone is stored on a dedicated storage server, and data from the sensors is stored in a database system (e.g., MongoDB). Each piece of data is given a timestamp, and the time the data was collected is clearly managed.

[0569] Input: Data sent to the central server

[0570] Output: Time-stamped saved data

[0571] Step 4: Data Integration

[0572] server

[0573] The server integrates the various stored data. Specifically, it combines image data and sensor data to generate an environmental map of the entire disaster area. In doing so, it makes full use of image processing algorithms, overlays the image data, and color-codes information such as seismic intensity and temperature to visually display the situation in the disaster area.

[0574] Input: Time-stamped stored data

[0575] Output: Unified environment map

[0576] Step 5: Data analysis

[0577] server

[0578] The server analyzes the integrated data, using image processing algorithms to assess the damage to buildings and quantify the degree of damage. It also analyzes collected seismic intensity and temperature data, integrating this information to understand the current situation in the affected area. For example, images of damaged buildings can be input into an AI model, which then classifies the damage state in detail (completely damaged, partially damaged, undamaged, etc.).

[0579] Input: Unified environment map

[0580] Output: Analysis results (evaluation of damage state, etc.)

[0581] Step 6: Predict the probability of the missing person being present

[0582] server

[0583] The server uses a generative AI model based on the analyzed data to predict the probability of missing persons. It calculates the possibility of missing persons in specific areas or buildings by referring to past data and the current situation in the disaster area. It inputs the latest analysis results into an AI model trained on data from specific areas where many people have gone missing in the past, and outputs the probability of missing persons.

[0584] Input: Analysis results

[0585] Output: Prediction result of the probability of the missing person being present

[0586] Step 7: Results display and notification

[0587] server

[0588] The server notifies search teams and rescue teams of the prediction results. The prediction results are displayed on a map, and areas with a high probability of missing persons being present are highlighted in color. This information is also sent to the Self-Defense Forces and rescue teams via email or a dedicated app. For example, areas with a high probability of missing persons being present can be displayed in red on a map, and detailed information about the area can be attached to send a real-time notification.

[0589] Input: Missing person existence probability prediction result

[0590] Output: Display and notification of prediction results

[0591] (Application example 1)

[0592] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0593] Searches for missing people during natural disasters must be carried out quickly and efficiently. However, conventional methods tend to be slow in data collection, information integration, and analysis, resulting in a lack of speed and accuracy in rescue operations. Data accuracy and real-time reporting are also issues. To solve these problems, a new search system utilizing the latest technology is needed.

[0594] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0595] In this invention, the server includes means for collecting images and sensor data of the disaster area, means for transmitting the collected data to a central server, means for integrating the data transmitted to the central server, means for predicting the probability of the presence of missing persons based on the integrated data, means for displaying and notifying the prediction results, means for using data collected by cameras and sensors mounted on the autonomous vehicle, means for performing analysis and prediction in real time using an AI model, and means for displaying and notifying the prediction results on a smartphone app, thereby enabling a fast and accurate search for missing persons.

[0596] "Images and sensor data from disaster areas" refers to visual information and wireless communication data collected using devices such as cameras and environmental sensors in areas where natural disasters have occurred.

[0597] "Collection methods" refers to the devices and methods used to gather important data from disaster areas, including drones, autonomous vehicles, and environmental sensors.

[0598] The "central server" is a computer system that receives and integrates all collected data and performs analysis and predictions.

[0599] "Integration means" refers to a method or device that brings together multiple collected data and prepares them for analysis.

[0600] A "prediction method" is an algorithm or machine learning model that calculates and predicts the probability of a missing person's presence based on the integrated data.

[0601] "Display and notification means" refers to devices and applications that display prediction results to users and notify them in real time. Specifically, this includes smartphone apps and dedicated display devices.

[0602] "Autonomous vehicle-mounted cameras and sensors" refers to data collection devices, such as high-resolution cameras and LiDAR sensors, mounted on an autonomous vehicle.

[0603] An "AI model" is an artificial intelligence algorithm that analyzes data and makes predictions based on the knowledge it has learned.

[0604] "Smartphone app" refers to dedicated application software that runs on a smartphone and is accessible to users.

[0605] In this invention, in order to build a system that can efficiently and quickly search for missing people in the event of a natural disaster, images and sensor data from the disaster area are collected, and based on that, an AI model is used to analyze, predict, display, and notify. The specific configuration and operation of the system are described below.

[0606] System configuration

[0607] Hardware:

[0608] Autonomous vehicles: Equipped with high-resolution cameras and LiDAR sensors, they will collect images and environmental data of the affected area.

[0609] Drones: Equipped with high-resolution cameras, they take images of the affected areas.

[0610] Environmental sensors: Installed on the ground surface, they collect environmental data such as seismic intensity, temperature, and humidity.

[0611] Central Server: A computer system that receives, stores, and analyzes all collected data.

[0612] Smartphone: A device that runs an application that displays and notifies users of prediction results.

[0613] software:

[0614] Image processing algorithm: Use libraries such as OpenCV to analyze image data from the disaster area.

[0615] AI model: A deep learning algorithm that uses frameworks such as TensorFlow to predict the probability of a missing person's presence.

[0616] IoT protocols: Use MQTT or similar to send sensor data to a server in real time.

[0617] Smartphone app: A dedicated application for displaying and notifying prediction results in real time.

[0618] Data collection and transmission

[0619] Autonomous vehicles, drones, and environmental sensors will collect data from the disaster area. Cameras mounted on autonomous vehicles and drones will capture high-resolution images, and LiDAR sensors will collect terrain data. Environmental sensors will capture important environmental data in real time, such as seismic intensity, temperature, and humidity. This data will be transmitted to a central server via wireless communication technology (e.g., 5G network).

[0620] Data integration and analysis

[0621] The central server consolidates all the data it receives. Image processing algorithms are applied to the image data using OpenCV to identify damaged areas of buildings. Sensor data is integrated in real time to generate an environmental map of the entire affected area.

[0622] Missing person prediction and notification

[0623] Based on the combined data, an AI model predicts the probability of a missing person being found. Deep learning algorithms using TensorFlow perform this analysis. The predictions are displayed in real time on a smartphone app and sent to the search team.

[0624] Examples of specific examples and AI prompts

[0625] As a concrete example, immediately after a disaster occurs, autonomous vehicles patrol the affected area and collect data using cameras and LiDAR sensors. This data is immediately transmitted to a central server via 5G communication. The server uses OpenCV and TensorFlow to analyze the images and sensor data and predict the probability of the presence of missing persons. This prediction result is displayed on a smartphone app and notified to the search team.

[0626] Example of a generated AI prompt:

[0627] Use the latest images and sensor data from the disaster area to predict areas where missing people are likely to be found. Use OpenCV for image processing and TensorFlow for data analysis. Use a deep learning model to run an algorithm to locate missing people in real time.

[0628] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0629] Step 1:

[0630] Autonomous vehicles and drones collect data from the affected areas. Cameras capture high-resolution image data, LiDAR sensors measure topographical data, and environmental sensors simultaneously collect environmental data such as seismic intensity, temperature, and humidity.

[0631] Input: Real-time images of the affected area, topographical data, and environmental data

[0632] Output: High-resolution image data, terrain data, environmental sensor data

[0633] How it works: Autonomous vehicles patrol the affected area, drones collect data from the air, cameras take images, LiDAR sensors collect topographical data, and environmental sensors capture various seismic and weather data.

[0634] Step 2:

[0635] The collected data is transmitted to a central server via wireless communication technology (e.g., 5G networks).

[0636] Input: High-resolution image data, terrain data, environmental sensor data

[0637] Output: Consolidated data sent to a central server

[0638] How it works: All collected data is sent to a central server in real time, using wireless communication and the MQTT protocol for instant data upload.

[0639] Step 3:

[0640] The server consolidates and stores the received data.

[0641] Input: Consolidated data sent to a central server

[0642] Output: A consolidated dataset

[0643] What it does: The server receives the input data, adjusts it to fit each format (e.g., decompresses image data, normalizes sensor data), and stores it as a single integrated dataset.

[0644] Step 4:

[0645] The combined data is then analyzed using AI models: image data is analyzed using OpenCV, and sensor data is processed using deep learning models.

[0646] Input: Integrated dataset

[0647] Output: Analysis result (probability of missing person existence)

[0648] How it works: The integrated data is fed into an AI model using TensorFlow. Image data is analyzed using OpenCV to identify damaged areas and critical points, and environmental data is analyzed using a deep learning model. This calculates the probability of the presence of missing people.

[0649] Step 5:

[0650] Based on the analysis results, the location information and probability of the missing person's presence are predicted and sent to a smartphone app, where they are displayed and notified in real time.

[0651] Input: Analysis result (probability of missing person existence)

[0652] Output: Display and notification on smartphone app

[0653] What it does: The server sends the analysis results to a smartphone app. The app receives the results, displays them visually, and sends notifications. Areas with a high probability of missing people are highlighted on a map for the search team user.

[0654] Step 6:

[0655] Search teams using a smartphone app can efficiently carry out search activities based on the displayed prediction results.

[0656] Input: Display and notification on smartphone app

[0657] Output: Efficient search operations

[0658] What it does: Search team users use the information displayed in the app to optimize their missing person search plans and respond quickly.

[0659] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0660] This invention relates to a system for efficiently and quickly searching for missing persons after natural disasters such as earthquakes. This system includes a means for collecting images and sensor data from the disaster area, a means for transmitting the collected data to a central server, a means for integrating and analyzing the data, and a means for predicting the probability of the presence of missing persons based on the analysis results, and displaying and notifying the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the efficiency of search activities is further improved.

[0661] Program processing overview

[0662] The program in this system includes the following main processes:

[0663] 1. Data collection in the affected areas

[0664] Terminal

[0665] Drones and environmental sensors will collect data from the disaster area. Specifically, drones will take images with high-resolution cameras, and environmental sensors will collect real-time data on seismic intensity, temperature, humidity, gas concentration, etc. Location information will also be acquired from the smartphones of disaster victims, which will then be provided as data.

[0666] 2. Data transmission and storage

[0667] Terminal

[0668] The collected data is sent to a central server via wireless communication or 4G / 5G networks. Image data captured by drones is sent as large files, while data from sensors is sent using IoT protocols (e.g., MQTT). Location information from victims' smartphones is also sent via GPS and the Internet.

[0669] 3. Data integration and analysis

[0670] server

[0671] The central server consolidates all the data it receives. It applies image processing algorithms to the image data to identify the extent of damage to buildings. The sensor data is processed in real time, and the results are used to generate an environmental map of the entire affected area. It also analyzes the location data of victims and plots their distribution on the map.

[0672] 4. Missing Person Prediction

[0673] server

[0674] Based on the analyzed data, an AI algorithm is used to predict the probability of missing persons being present. Past data and the current situation in the disaster area are referenced to identify areas or buildings where missing persons are likely to be present. The prediction results are listed and saved.

[0675] 5. Operation of the Emotion Engine

[0676] Terminals and Servers

[0677] The emotion engine recognizes emotions by analyzing the voices and facial expressions of users participating in search operations. Emotion data is sent in real time to a central server for analysis. For example, if a user's stress level is high, that information can be used to adjust the priority of search operations.

[0678] 6. Display and notification of results

[0679] server

[0680] The server notifies search and rescue teams of the results of its predictions of missing persons, visualizes the results, and displays them on a map. It also displays emotion data collected by the emotion engine, helping to plan and coordinate the search.

[0681] Specific examples

[0682] For example, a drone flies over a disaster area and photographs the entire town with a high-resolution camera. Along with the captured images, a ground sensor collects seismic intensity information, and this data is sent to a central server. The server analyzes the received data, and an AI model predicts with high accuracy the probability of the presence of missing persons. The prediction results are then displayed on a map and notified to the Self-Defense Forces and rescue teams. Search team users also use an emotion engine to send their own stress and fatigue to the server. The server uses this information to suggest areas to prioritize search and efficient resource allocation.

[0683] As a result, the present invention supports rapid and efficient search activities during disasters, contributing to the early discovery and rescue of missing persons.

[0684] The processing flow will be explained below.

[0685] Step 1: Collect data

[0686] Terminal

[0687] Action 1: A drone flies over the affected area and takes images with a high-resolution camera.

[0688] Action 2: Environmental sensors collect data such as seismic intensity, temperature, humidity, and gas concentration in real time at the installation location.

[0689] Action 3: The victim's smartphone obtains location information and records that data.

[0690] Step 2: Send and store data

[0691] Terminal

[0692] Operation 1: The image data captured by the drone is transmitted to a central server via wireless communication or 4G / 5G networks.

[0693] Action 2: The data acquired by the environmental sensors is sent to a central server using an IoT protocol (e.g., MQTT).

[0694] Action 3: The location information acquired by the victim's smartphone is sent to a central server using GPS.

[0695] Step 3: Integrate the data

[0696] server

[0697] Action 1: The server centrally stores all received data.

[0698] Action 2: The server combines image data, sensor data, and location information and converts them into a single dataset.

[0699] Action 3: The server converts the consolidated data into a usable format for subsequent analysis.

[0700] Step 4: Analyze the data

[0701] server

[0702] Action 1: Based on the integrated data, the server applies image processing algorithms to identify the extent of damage to the building.

[0703] Step 2: The server generates an environment map in real time based on the sensor data.

[0704] Step 3: The server analyzes the location information of the victims and visualizes their distribution on a map.

[0705] Step 5: Predict missing persons

[0706] server

[0707] Action 1: The server inputs the analyzed data into an AI model to predict areas where the missing person is likely to be located.

[0708] Operation 2: The server compares past data with current data and filters the output prediction results to further improve accuracy.

[0709] Action 3: The server lists the final prediction results and saves the list.

[0710] Step 6: Emotion Recognition with the Emotion Engine

[0711] Terminals and Servers

[0712] Action 1: The device analyzes the user's voice and facial expressions to collect emotional data.

[0713] Action 2: The device sends the collected emotion data to the central server.

[0714] Action 3: The server analyzes the emotional data and evaluates the user's stress level and fatigue.

[0715] Step 7: View and notify results

[0716] server

[0717] Action 1: The server visualizes the missing person prediction results and displays them on a map.

[0718] Action 2: The server also displays the emotion data collected by the emotion engine.

[0719] Action 3: The server notifies search and rescue teams of the prediction results in real time.

[0720] Step 8: Conduct a search operation

[0721] User

[0722] Action 1: The rescue team receives the prediction results from the server and formulates a search plan.

[0723] Action 2: Rescue teams carry out specific search operations in the disaster area.

[0724] Action 3: The rescue team feeds back the results of their on-site search to the server, which helps improve the accuracy of their next prediction.

[0725] The above is a detailed explanation of the specific processing steps of the system for searching for missing persons in the event of a disaster that combines an emotion engine, and its operation.

[0726] Example 2

[0727] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0728] Rapid and efficient search for missing persons during disasters is difficult with current technology. Not only must image and sensor data from the disaster area be collected, but that data must also be processed quickly to accurately predict the probability of missing persons being found. It is also necessary to allocate resources efficiently while taking into account the emotions of users participating in the search operation. A system that can solve these problems is needed.

[0729] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0730] In this invention, the server includes means for collecting images and sensor data of the disaster area, means for transmitting the collected data to a central server, means for integrating the data transmitted to the central server, means for predicting the probability of the presence of missing persons based on the integrated data, means for displaying and notifying the user of the prediction result, and means for recognizing the user's emotions and adjusting the priority of search activities, thereby enabling a fast and efficient search for missing persons in a disaster.

[0731] "Disaster area" refers to an area affected by a natural disaster.

[0732] "Images" refers to photographs and video data taken to visually record the situation in the disaster area.

[0733] "Sensor data" refers to various types of data acquired by sensors to collect environmental information in disaster-stricken areas, including seismic intensity, temperature, humidity, gas concentration, etc.

[0734] "Central server" refers to a computer system that stores and analyzes collected data and provides information useful for search operations.

[0735] "Integrating" refers to the process of collecting, organizing, and analyzing different types of data in a centralized manner.

[0736] "AI algorithm" refers to artificial intelligence technology that automatically learns patterns and trends in data and predicts behavior.

[0737] "Probability of presence" refers to the probability that a missing person may be present in a particular location.

[0738] "Prediction result" refers to the probability of a missing person being found calculated using an AI algorithm.

[0739] "Display and notification" refers to the process of visually displaying the analysis results and communicating information to interested parties.

[0740] An "emotion engine" refers to technology that analyzes a user's voice and facial expressions to recognize their emotional state, such as stress or fatigue.

[0741] "Autonomous flying devices" refer to devices that fly automatically and collect images and data without human control. Drones generally fall into this category.

[0742] "Environmental sensors" refer to devices used to measure environmental information in disaster-stricken areas, such as seismometers, temperature sensors, and gas detectors.

[0743] "Users" refers to people engaged in search and rescue activities during disasters, such as members of search and rescue teams.

[0744] The present invention relates to a system for efficiently and quickly searching for missing persons in the event of a natural disaster such as an earthquake. This system collects images and sensor data from the disaster area, analyzes them to predict the probability of the presence of missing persons, and visualizes and notifies the user.

[0745] 1. Data collection in the affected areas

[0746] Terminal

[0747] The system uses drones, environmental sensors, and the victims' smartphones. The drones are equipped with high-resolution cameras to collect images of the entire disaster area. The environmental sensors collect data such as seismic intensity, temperature, humidity, and gas concentration in real time. The victims' smartphones use GPS to obtain location information.

[0748] 2. Data transmission and storage

[0749] Terminal

[0750] The collected data is transmitted to a central server via wireless communication or 4G / 5G networks. The large amount of image data collected by the drone is transmitted using a dedicated bandwidth, and sensor data is transmitted in real time using the MQTT protocol. Smartphone location information is also sent to the central server via the Internet.

[0751] 3. Data integration and analysis

[0752] server

[0753] A central server consolidates all data. High-resolution images are stored in a database, and data from environmental sensors and location information are also managed centrally. Image processing algorithms are applied to analyze the damage to buildings, and sensor data is processed in real time to generate a map of the affected area. The data is plotted on the map based on the location information of victims.

[0754] 4. Missing Person Prediction

[0755] server

[0756] AI algorithms, such as deep learning and machine learning models (CNN, LSTM, etc.), are used to predict the probability of missing persons being found. Based on past data and the current situation in the disaster area, locations with a high probability of missing persons are identified, and the results are listed and saved.

[0757] 5. Operation of the Emotion Engine

[0758] Terminals and Servers

[0759] The emotion engine analyzes the user's voice and facial expressions in real time and sends emotional data to a central server, which then adjusts the priority of search activities if the user's stress level is high.

[0760] 6. Display and notification of results

[0761] server

[0762] Based on the analyzed data, prediction results are visually displayed on a map. Information is then sent to search and rescue teams via a dedicated app, SMS, email, etc. Emotional data is also displayed to assist in the planning and adjustment of search plans.

[0763] Specific examples

[0764] For example, a drone flies over a disaster-hit area, capturing images of the entire town with a high-resolution camera. At the same time, sensors on the ground collect seismic intensity information and gas concentrations, and all data is sent to a central server. The server then integrates and analyzes this data, and an AI model predicts the probability of the presence of missing persons. The results are displayed on a map and communicated to the Self-Defense Forces and rescue teams. Search team members can also use an emotion engine to send their own stress and fatigue levels to the server, which then prioritizes search areas and resource allocations.

[0765] Example prompts for generative AI models

[0766] "In order to quickly search for missing people in disaster areas, please tell me how to collect data, what algorithms should be used to make predictions and analyses, and how to display and notify the results."

[0767] As a result, the present invention supports rapid and efficient search activities in the event of a disaster, contributing to the early discovery and rescue of missing persons.

[0768] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0769] Step 1: Collecting data from affected areas

[0770] The device uses drones and environmental sensors to collect data on the affected area. Specifically, the drone takes images with a high-resolution camera, and the environmental sensors collect data such as seismic intensity, temperature, humidity, and gas concentration. The victim's smartphone also uses GPS to obtain location information.

[0771] Inputs include image data from high-resolution cameras, environmental data from environmental sensors, and smartphone location data.

[0772] As an output, these collected data sets are produced.

[0773] Step 2: Data transmission and storage

[0774] The terminal transmits the collected data to a central server via wireless communication or 4G / 5G networks. Specifically, drone image data is transmitted as a large file, environmental sensor data is transmitted using the MQTT protocol, and smartphone location data is also transmitted via the Internet.

[0775] The input is all the data collected in step 1.

[0776] The output is a data store stored on a central server.

[0777] Step 3: Data integration and analysis

[0778] The server consolidates all the data it receives. Specifically, it stores high-resolution image data in a database, and also centrally manages environmental sensor data and smartphone location information. It applies image processing algorithms to extract and analyze information such as the extent of building damage. It processes the sensor data in real time to generate an environmental map of the entire disaster area.

[0779] The input is the data store saved in step 2.

[0780] The output is a consolidated and analyzed dataset that provides a situation map of the affected area.

[0781] Step 4: Predict missing persons

[0782] The server uses an AI algorithm to predict the probability of missing people based on the integrated data set. Specifically, it uses deep learning models (e.g., CNN, LSTM) to analyze past data and the current situation in the disaster area and identify locations where missing people are likely to be.

[0783] The input is the analyzed dataset generated in step 3.

[0784] The output is a prediction result that lists the probability of the missing person being present.

[0785] Step 5: Emotion Engine in Action

[0786] The device and server analyze the user's voice and facial expressions in real time and send emotional data to a central server, which then aggregates the emotional data and analyzes the user's stress and fatigue levels. Priorities are adjusted based on this information.

[0787] Inputs include voice and facial expression data obtained from the user.

[0788] As an output, the analyzed emotional data is sent to a central server, which adjusts the priorities of search operations.

[0789] Step 6: View and notify results

[0790] The server notifies search and rescue teams of the predicted probability of a missing person's presence. The results are sent via a dedicated app, SMS, or email. Furthermore, the prediction results and emotion data are visually displayed on a map to assist in the planning and coordination of search plans.

[0791] The inputs are the prediction results obtained in step 4 and the emotion data collected in step 5.

[0792] The output is a visualized map and a notification.

[0793] (Application example 2)

[0794] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0795] Efficiently and quickly searching for missing persons during natural disasters is difficult and requires accurate data collection and analysis across a wide area of ​​the disaster area. Furthermore, failure to consider the emotional state of the search team can reduce search efficiency and increase the risk of stress and fatigue. Therefore, a system is needed that enables fast and efficient search operations and enables the early discovery and rescue of missing persons.

[0796] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means including an autonomously driving vehicle that autonomously collects data on the disaster area, means for collecting images and sensor data using drones and environmental sensors, means for transmitting the collected data to a central server, means for integrating and analyzing the data transmitted to the central server, means for predicting the probability of the presence of missing persons based on the integrated data, means for displaying the prediction results on a map and notifying search teams and rescue teams, means including an emotion engine that collects and analyzes emotion data from the search team, and means for adjusting the search plan based on the emotion data. This enables fast and efficient search activities and realizes the early discovery and rescue of missing persons.

[0797] An "autonomous vehicle" refers to a vehicle that can drive autonomously without a driver and travel to a specific destination.

[0798] A "drone" refers to an unmanned aircraft that flies and collects images and sensor data.

[0799] "Environmental sensor" refers to a sensor device for detecting environmental data such as seismic intensity, temperature, humidity, and gas concentration.

[0800] "Central server" refers to a central management system for integrating and analyzing data collected from disaster-stricken areas.

[0801] "Data integration" refers to the process of centrally organizing data collected from multiple sources and making it ready for analysis.

[0802] "Probability of presence" refers to the probability that a missing person may be present in a particular location.

[0803] The "emotion engine" refers to a system that analyzes the voices and facial expressions of the search team and recognizes their emotional state.

[0804] "Emotion data" refers to information collected by the emotion engine that indicates the user's emotional state, such as stress or fatigue.

[0805] "Search plan" refers to a plan drawn up to efficiently search for a missing person.

[0806] A "generative AI model" refers to an artificial intelligence algorithm that learns from large amounts of data and predicts the probability of a missing person's existence, etc.

[0807] The present invention relates to a system for efficiently and quickly searching for missing people during natural disasters. This system is composed of a combination of autonomous vehicles, drones, environmental sensors, a central server, an emotion engine, and a generative AI model. Each part of the present invention is described in detail below.

[0808] Data collection

[0809] The server collects data using autonomous, unmanned vehicles that travel across a wide area of ​​the disaster-stricken area. The autonomous vehicles are equipped with drones and environmental sensors, and take images using high-resolution cameras. Environmental data such as seismic intensity, temperature, humidity, and gas concentration are also collected at the same time. Drones installed in various locations in the disaster-stricken area collect images from an aerial perspective, allowing the entire situation in the disaster-stricken area to be grasped.

[0810] Data transmission

[0811] The devices transmit the collected data to a central server in real time via wireless communication or 4G / 5G networks. Image data is sent as large files, and sensor data is transmitted using IoT protocols (e.g., MQTT). Location information from the victim's smartphone is also sent to the server via GPS and the Internet.

[0812] Data Integration and Analysis

[0813] The server integrates and analyzes all the received data. Using a generative AI model trained using TensorFlow and Keras, it uses image processing algorithms to identify the extent of damage to buildings. Sensor data is also processed in real time, and the results are used to generate an environmental map of the entire disaster area. Furthermore, it analyzes the location data of victims and plots their distribution on the map.

[0814] Missing Persons Prediction

[0815] Based on the analyzed data, the server uses a generative AI model to predict the probability of missing persons being present. It references past data and the current situation in the disaster area to identify locations in specific areas or buildings where missing persons are likely to be present. The prediction results are then listed and saved.

[0816] Emotion Engine Operation

[0817] The server uses an emotion engine to analyze the voices and facial expressions of users participating in the search operation to recognize their emotions. Emotional data is sent to a central server in real time and analyzed there. For example, if a user's stress level is high, that information can be used to adjust the priority of the search operation.

[0818] Displaying and notifying results

[0819] The server notifies search and rescue teams of the predicted missing persons, visualizes the results, and displays them on a map. It also displays emotional data collected by the emotion engine, helping them plan and coordinate their search.

[0820] Specific examples

[0821] For example, in a disaster area where a magnitude 7 earthquake has occurred, autonomous vehicles begin collecting data. High-resolution image data captured by the autonomous vehicles, aerial video data collected by drones, and seismic intensity information from environmental sensors are transmitted in real time to a central server. The server integrates this data, and a generative AI model predicts the probability of missing persons being present. As a result, specific areas are predicted to have a high probability of missing persons, and the search team is notified. Furthermore, the emotional state of the search team is monitored, and the search plan is adjusted if stress levels are high.

[0822] Here are some example prompts:

[0823] “In the event of a natural disaster, predict the probability of missing persons in the affected area using high-resolution images captured by drones and environmental sensor data. Consider variables such as building damage, temperature, humidity, and gas concentrations.”

[0824] As a result, the present invention enables fast and efficient search operations, contributing to the early discovery and rescue of missing persons.

[0825] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0826] Step 1:

[0827] The device uses autonomous vehicles and drones to collect data from disaster-stricken areas. The autonomous vehicles drive autonomously through the disaster-stricken areas and take images with high-resolution cameras. The drones collect aerial image data, while environmental sensors collect real-time data such as seismic intensity, temperature, humidity, and gas concentration. This allows for detailed environmental information to be obtained.

[0828] Input: On-site environmental and image data from the affected areas

[0829] Output: High-resolution image data, seismic intensity data, temperature data, humidity data, gas concentration data

[0830] Step 2:

[0831] The devices transmit the collected data to a central server via wireless communication or 4G / 5G networks. Image data is sent as large files, and sensor data is transmitted using IoT protocols. Location information collected by victims' smartphones is also transmitted via GPS and the internet.

[0832] Input: High-resolution image data, seismic intensity data, temperature data, humidity data, gas concentration data, location information of victims

[0833] Output: Sending data to a central server

[0834] Step 3:

[0835] The server integrates all the received data. Specifically, it applies image processing algorithms to the image data using TensorFlow and Keras to identify the extent of damage to buildings. It also processes the sensor data in real time and generates an environmental map of the entire disaster area based on the results. The location data of the victims is analyzed and the distribution of victims is plotted on the map.

[0836] Input: All data sent to the central server (high-resolution image data, sensor data, location information of victims)

[0837] Output: Image data analysis results, environmental map, disaster victim distribution map

[0838] Step 4:

[0839] The server uses a generative AI model to predict the probability of missing persons being present. It references past data and the current situation in the disaster area to identify locations in specific areas or buildings where missing persons are likely to be present. The prediction results are listed and saved.

[0840] Input: Integrated analysis data (image data analysis results, environmental map, disaster victim distribution map)

[0841] Output: List of predicted probability of missing person existence

[0842] Step 5:

[0843] The server uses an emotion engine to analyze the voices and facial expressions of users participating in the search operation to recognize their emotions. Emotional data is sent to the server in real time for analysis. For example, if a user's stress level is high, the server can adjust the priority of the search operation based on that information.

[0844] Input: User's voice data and facial expression data

[0845] Output: Parsed emotion data

[0846] Step 6:

[0847] The server notifies search and rescue teams of predicted missing persons and displays the results on a map. It also displays emotion data collected by the emotion engine, helping to plan and coordinate the search.

[0848] Input: List of predicted probability of missing persons, analyzed emotion data

[0849] Output: Map predictions, notifications to search and rescue teams, coordinated search plans

[0850] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0851] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0852] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0853] [Third embodiment]

[0854] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0855] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0856] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0857] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0858] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0859] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0860] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0861] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0862] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0863] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0864] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0865] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0866] This invention relates to a system for efficiently and quickly searching for missing persons after natural disasters such as earthquakes. This system includes a means for collecting images and sensor data from the disaster area, a means for transmitting the collected data to a central server, a means for integrating and analyzing the data, and a means for predicting the probability of the presence of missing persons based on the analysis results, and displaying and notifying the user.

[0867] Program processing overview

[0868] The program in this system includes the following steps as its main processing steps.

[0869] 1. Data collection in the affected areas

[0870] Terminal

[0871] The devices include drones and environmental sensors. Drones operating over the affected area use high-resolution cameras to capture images of the affected area, while sensors on the ground and in the air collect seismic intensity, temperature, humidity, and other important environmental data.

[0872] A specific example would be a drone flying over a disaster area, photographing the damage to buildings, and sensors recording seismic intensity and temperature data on the ground surface.

[0873] 2. Data transmission and storage

[0874] Terminal

[0875] The collected data is transmitted in real time to a central server via wireless communication or 4G / 5G networks. Image data captured by the drone's camera is uploaded to the server as a large file. Data from sensors is also transmitted in the same way using IoT protocols.

[0876] For example, images taken by drones can be instantly uploaded to a central server via a 5G network, while data collected by sensors on the ground can be transmitted via the MQTT protocol.

[0877] 3. Data integration and analysis

[0878] server

[0879] The central server consolidates all the data it receives, applies image processing algorithms to the image data, and analyzes the damage to buildings. The sensor data is processed in real time, and the results are used to generate an environmental map of the entire disaster area.

[0880] Specifically, the server analyzes images received from the drone to identify damaged areas of buildings, and maps seismic intensity and temperature data to provide a visual understanding of the situation in the affected areas.

[0881] 4. Missing Person Prediction

[0882] server

[0883] Based on the analyzed data, an AI algorithm is used to predict the probability of missing persons being present, and by referencing past data and the current situation in the disaster area, it identifies locations in specific areas or buildings where missing persons are likely to be present.

[0884] For example, the server inputs data into a trained AI model to calculate areas where there are likely to be many missing people, and based on this result, analyzes patterns in which missing people are likely to occur.

[0885] 5. Display and notification of results

[0886] server

[0887] The prediction results are sent to search and rescue teams in real time. The server generates a map showing areas where there is a high probability of missing people and sends it to relevant parties. These parties can use this map to develop effective search plans.

[0888] For example, the server highlights areas on a map where there is a high probability of missing persons being found in red and sends this information to the Self-Defense Forces via email. It also displays the information in real time on a dedicated application, enabling a rapid response.

[0889] Specific examples

[0890] One example would be a drone flying over a disaster area, capturing images of the entire town with a high-resolution camera. Along with the captured images, ground sensors collect seismic intensity information, which is then immediately sent to a central server. The server analyzes the received data, and an AI model accurately predicts the probability of the presence of missing persons. The predictions are then displayed on a map and notified to the Self-Defense Forces and rescue teams, allowing for efficient search operations.

[0891] The processing flow will be explained below.

[0892] Step 1: Collect data

[0893] Terminal

[0894] Action 1: A drone flies over the affected area and takes high-resolution images with its camera.

[0895] Action 2: Environmental sensors collect data such as seismic intensity, temperature, humidity, and gas concentration in real time.

[0896] Action 3: The victim's smartphone obtains location information and collects data.

[0897] Step 2: Sending data

[0898] Terminal

[0899] Operation 1: The image data captured by the drone is transmitted to a central server via wireless communication or 4G / 5G networks.

[0900] Action 2: The data collected by the environmental sensors is sent to a central server using an IoT protocol (e.g., MQTT).

[0901] Action 3: The location information obtained by the victim's smartphone is sent to a central server via GPS and the Internet.

[0902] Step 3: Integrate the data

[0903] server

[0904] Operation 1: The server temporarily saves the received image data and stores it in a database.

[0905] Action 2: The server receives the sensor data and synthesizes it into an appropriate format.

[0906] Action 3: The server combines different types of data, such as images, sensors, and location information, into a single dataset.

[0907] Step 4: Analyze the data

[0908] server

[0909] Action 1: The server applies image processing algorithms to analyze the integrated data and identify the damage to the building.

[0910] Action 2: The server analyzes the sensor data in real time and grasps the environmental situation of the entire disaster area.

[0911] Action 3: The server analyzes the location data of the victims and plots the distribution of the victims on a map.

[0912] Step 5: Predict missing persons

[0913] server

[0914] Action 1: The server inputs the analyzed data into an AI model to predict areas where the missing person is likely to be located.

[0915] Step 2: The server takes past data into account and filters the output prediction results to further improve accuracy.

[0916] Action 3: The server lists the final prediction results and saves the list.

[0917] Step 6: Notification and display of results

[0918] server

[0919] Action 1: The server visualizes the prediction results and displays them on a map.

[0920] Action 2: The server notifies the search and rescue team of the prediction results.

[0921] Action 3: Receive feedback based on the information notified by the server and take action to reflect it in the next prediction.

[0922] Step 7: Conduct a search operation

[0923] User

[0924] Action 1: The rescue team formulates a search plan based on the prediction results received from the server.

[0925] Action 2: Rescue teams carry out specific search operations on site.

[0926] Action 3: The rescue team reports the results of the on-site search to the server, so that they can be reflected as data for the next step.

[0927] Example 1

[0928] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0929] Rapid and efficient search for missing persons in natural disasters such as earthquakes is a key challenge in rescue operations. However, understanding the situation in widespread disaster areas and identifying locations with a high probability of missing persons taking time and requiring significant human resources. Conventional methods have difficulty in accurately collecting and analyzing data in real time, limiting their effectiveness in situations where a rapid response is required.

[0930] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0931] In this invention, the server includes means for collecting images and environmental data of the disaster area, means for transmitting the collected data to a central server, means for storing the data transmitted to the central server, means for integrating the stored data, means for analyzing the integrated data, means for predicting the probability of the presence of missing persons based on the analyzed data, and means for displaying and notifying the results of the prediction. This enables real-time data collection and analysis over a wide area of ​​the disaster area, enabling rapid and efficient search activities for missing persons.

[0932] "Images" refer to photographs and video data used to obtain visual information about the disaster area.

[0933] "Environmental data" refers to data used to measure environmental conditions in the affected areas, such as seismic intensity, temperature, and humidity.

[0934] "Collection means" refers to devices that acquire data from disaster areas using equipment such as drones and environmental sensors.

[0935] "Transmission means" refers to a mechanism for transmitting collected data to a central server using wireless communication or a network.

[0936] The "storage means" is a storage system for storing collected data in the server.

[0937] "Integration" is the process of combining multiple stored data sets and processing them as a unified data set.

[0938] "Analysis tools" are algorithms and programs used to analyze the integrated data and grasp the situation in the disaster-stricken areas.

[0939] A "predictive tool" is a generative AI model or other algorithm used to calculate the probability of a missing person's existence based on analyzed data.

[0940] "Display means" refers to a device or interface for visually presenting the analysis and prediction results.

[0941] "Notification means" is a system for communicating prediction results to relevant parties in real time.

[0942] An "autonomous mobile object" is a device that can move automatically without external operation, such as a drone or robot.

[0943] An "environmental information acquisition device" is a sensor or measuring instrument used to collect environmental data such as temperature, humidity, and seismic intensity.

[0944] A "data processing algorithm" is a calculation method or program used to analyze acquired data.

[0945] The present invention relates to a system for quickly and efficiently searching for missing persons in the event of a natural disaster such as an earthquake. The system includes a data collection means, a data transmission means, a data storage means, a data integration means, a data analysis means, a means for predicting the probability of the presence of missing persons, a result display means, and a notification means.

[0946] Data collection methods

[0947] Terminal

[0948] The devices used include drones and environmental sensors. Drones are equipped with high-resolution cameras and fly over affected areas to collect image data. Environmental sensors collect environmental data such as seismic intensity, temperature, and humidity in real time. Drones use automatic navigation functions to fly over affected areas along pre-set routes. For example, drones may take aerial photographs of the damage to an entire town, while ground sensors measure seismic intensity and temperature.

[0949] Data transmission method

[0950] Terminal

[0951] The collected data is transmitted to a central server in real time via wireless communication or 4G / 5G networks. The large amount of image data collected by the drone is instantly uploaded using the drone's built-in communication module. Data from the sensors is also transmitted to the server using IoT protocols (e.g., MQTT). Specifically, the images captured by the drone are uploaded to the central server via the 5G network, and the ground sensors transmit seismic intensity data via the MQTT protocol.

[0952] Data storage means

[0953] server

[0954] The server first stores the data it receives. Image data sent from the drone is stored on a dedicated storage server, and data from the sensors is stored in a database system (e.g., MongoDB). Each piece of data is given a timestamp to clarify the time the data was collected.

[0955] Data Integration Methods

[0956] server

[0957] The server integrates multiple stored data sets. For example, it combines image data and sensor data to generate an environmental map of the entire disaster area. By overlaying the image data and displaying seismic intensity and temperature information in color, the current situation in the disaster area can be visually grasped. Image processing algorithms and data mapping techniques are used.

[0958] Data Analysis Methods

[0959] server

[0960] The server analyzes the integrated data, using image processing algorithms to assess the damage to buildings and quantify the degree of damage. It also analyzes collected seismic intensity and temperature data, integrating this information to understand the detailed situation in the affected area. For example, images of damaged buildings can be input into an AI model to classify the damage state in detail.

[0961] A method for predicting the probability of a missing person's existence

[0962] server

[0963] Based on the analyzed data, the server uses a generative AI model to predict the probability of missing persons. It uses past data and the current situation in the disaster area as a reference to calculate the possibility of missing persons in a specific area or building. Specifically, the latest data is input into an AI model trained on data from specific areas where many missing persons have occurred in the past, and the server outputs the probability of missing persons being present.

[0964] Result display means and notification means

[0965] server

[0966] The server notifies search teams and rescue teams of the prediction results. The prediction results are displayed on a map, highlighting areas with a high probability of missing people in color. The results are also sent to the Self-Defense Forces and rescue teams via email or a dedicated app. For example, areas with a high probability of missing people being present on a map can be displayed in red, and detailed information about the area can be attached to send a real-time notification to the rescue team.

[0967] Prompt Sentence Examples

[0968] "Please provide an overview of the data collection and analysis system for predicting the probability of missing persons in disaster areas."

[0969] The above is a specific embodiment for carrying out the present invention. This system enables real-time data collection and analysis in a wide area of ​​a disaster, enabling rapid and efficient search activities for missing persons.

[0970] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0971] Step 1: Data collection

[0972] Terminal

[0973] The device uses a drone and environmental sensors. The drone uses a high-resolution camera to capture images of the sky over the affected area, while environmental sensors on the ground measure environmental data such as seismic intensity, temperature, and humidity. Specifically, the drone flies automatically along a pre-set route, capturing detailed images of the damage to buildings and the terrain. The environmental sensors also collect seismic intensity and temperature data measured every five seconds and store it in their internal memory.

[0974] Input: Visual and environmental information of the affected area

[0975] Output: High-resolution disaster image data and environmental sensor data

[0976] Step 2: Send data

[0977] Terminal

[0978] The collected data is transmitted to a central server in real time via wireless communication or 4G / 5G networks. Image data captured by drones is transmitted as large files, while data from environmental sensors is transmitted every second using IoT protocols (such as MQTT). For example, images captured by drones can be immediately uploaded using the 5G network, and seismic intensity data recorded by sensors can be sent to the server in batches every minute.

[0979] Input: High-resolution disaster image data and environmental sensor data

[0980] Output: Data sent to the central server

[0981] Step 3: Save Data

[0982] server

[0983] The central server stores the received data. Specifically, image data sent from the drone is stored on a dedicated storage server, and data from the sensors is stored in a database system (e.g., MongoDB). Each piece of data is given a timestamp, and the time the data was collected is clearly managed.

[0984] Input: Data sent to the central server

[0985] Output: Time-stamped saved data

[0986] Step 4: Data Integration

[0987] server

[0988] The server integrates the various stored data. Specifically, it combines image data and sensor data to generate an environmental map of the entire disaster area. In doing so, it makes full use of image processing algorithms, overlays the image data, and color-codes information such as seismic intensity and temperature to visually display the situation in the disaster area.

[0989] Input: Time-stamped stored data

[0990] Output: Unified environment map

[0991] Step 5: Data analysis

[0992] server

[0993] The server analyzes the integrated data, using image processing algorithms to assess the damage to buildings and quantify the degree of damage. It also analyzes collected seismic intensity and temperature data, integrating this information to understand the current situation in the affected area. For example, images of damaged buildings can be input into an AI model, which then classifies the damage state in detail (completely damaged, partially damaged, undamaged, etc.).

[0994] Input: Unified environment map

[0995] Output: Analysis results (evaluation of damage state, etc.)

[0996] Step 6: Predict the probability of the missing person being present

[0997] server

[0998] The server uses a generative AI model based on the analyzed data to predict the probability of missing persons. It calculates the possibility of missing persons in specific areas or buildings by referring to past data and the current situation in the disaster area. It inputs the latest analysis results into an AI model trained on data from specific areas where many people have gone missing in the past, and outputs the probability of missing persons.

[0999] Input: Analysis results

[1000] Output: Prediction result of the probability of the missing person being present

[1001] Step 7: Results display and notification

[1002] server

[1003] The server notifies search teams and rescue teams of the prediction results. The prediction results are displayed on a map, and areas with a high probability of missing persons being present are highlighted in color. This information is also sent to the Self-Defense Forces and rescue teams via email or a dedicated app. For example, areas with a high probability of missing persons being present can be displayed in red on a map, and detailed information about the area can be attached to send a real-time notification.

[1004] Input: Missing person existence probability prediction result

[1005] Output: Display and notification of prediction results

[1006] (Application example 1)

[1007] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1008] Searches for missing people during natural disasters must be carried out quickly and efficiently. However, conventional methods tend to be slow in data collection, information integration, and analysis, resulting in a lack of speed and accuracy in rescue operations. Data accuracy and real-time reporting are also issues. To solve these problems, a new search system utilizing the latest technology is needed.

[1009] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1010] In this invention, the server includes means for collecting images and sensor data of the disaster area, means for transmitting the collected data to a central server, means for integrating the data transmitted to the central server, means for predicting the probability of the presence of missing persons based on the integrated data, means for displaying and notifying the prediction results, means for using data collected by cameras and sensors mounted on the autonomous vehicle, means for performing analysis and prediction in real time using an AI model, and means for displaying and notifying the prediction results on a smartphone app, thereby enabling a fast and accurate search for missing persons.

[1011] "Images and sensor data from disaster areas" refers to visual information and wireless communication data collected using devices such as cameras and environmental sensors in areas where natural disasters have occurred.

[1012] "Collection methods" refers to the devices and methods used to gather important data from disaster areas, including drones, autonomous vehicles, and environmental sensors.

[1013] The "central server" is a computer system that receives and integrates all collected data and performs analysis and predictions.

[1014] "Integration means" refers to a method or device that brings together multiple collected data and prepares them for analysis.

[1015] A "prediction method" is an algorithm or machine learning model that calculates and predicts the probability of a missing person's presence based on the integrated data.

[1016] "Display and notification means" refers to devices and applications that display prediction results to users and notify them in real time. Specifically, this includes smartphone apps and dedicated display devices.

[1017] "Autonomous vehicle-mounted cameras and sensors" refers to data collection devices, such as high-resolution cameras and LiDAR sensors, mounted on an autonomous vehicle.

[1018] An "AI model" is an artificial intelligence algorithm that analyzes data and makes predictions based on the knowledge it has learned.

[1019] "Smartphone app" refers to dedicated application software that runs on a smartphone and is accessible to users.

[1020] In this invention, in order to build a system that can efficiently and quickly search for missing people in the event of a natural disaster, images and sensor data from the disaster area are collected, and based on that, an AI model is used to analyze, predict, display, and notify. The specific configuration and operation of the system are described below.

[1021] System configuration

[1022] Hardware:

[1023] Autonomous vehicles: Equipped with high-resolution cameras and LiDAR sensors, they will collect images and environmental data of the affected area.

[1024] Drones: Equipped with high-resolution cameras, they take images of the affected areas.

[1025] Environmental sensors: Installed on the ground surface, they collect environmental data such as seismic intensity, temperature, and humidity.

[1026] Central Server: A computer system that receives, stores, and analyzes all collected data.

[1027] Smartphone: A device that runs an application that displays and notifies users of prediction results.

[1028] software:

[1029] Image processing algorithm: Use libraries such as OpenCV to analyze image data from the disaster area.

[1030] AI model: A deep learning algorithm that uses frameworks such as TensorFlow to predict the probability of a missing person's presence.

[1031] IoT protocols: Use MQTT or similar to send sensor data to a server in real time.

[1032] Smartphone app: A dedicated application for displaying and notifying prediction results in real time.

[1033] Data collection and transmission

[1034] Autonomous vehicles, drones, and environmental sensors will collect data from the disaster area. Cameras mounted on autonomous vehicles and drones will capture high-resolution images, and LiDAR sensors will collect terrain data. Environmental sensors will capture important environmental data in real time, such as seismic intensity, temperature, and humidity. This data will be transmitted to a central server via wireless communication technology (e.g., 5G network).

[1035] Data integration and analysis

[1036] The central server consolidates all the data it receives. Image processing algorithms are applied to the image data using OpenCV to identify damaged areas of buildings. Sensor data is integrated in real time to generate an environmental map of the entire affected area.

[1037] Missing person prediction and notification

[1038] Based on the combined data, an AI model predicts the probability of a missing person being found. Deep learning algorithms using TensorFlow perform this analysis. The predictions are displayed in real time on a smartphone app and sent to the search team.

[1039] Examples of specific examples and AI prompts

[1040] As a concrete example, immediately after a disaster occurs, autonomous vehicles patrol the affected area and collect data using cameras and LiDAR sensors. This data is immediately transmitted to a central server via 5G communication. The server uses OpenCV and TensorFlow to analyze the images and sensor data and predict the probability of the presence of missing persons. This prediction result is displayed on a smartphone app and notified to the search team.

[1041] Example of a generated AI prompt:

[1042] Use the latest images and sensor data from the disaster area to predict areas where missing people are likely to be found. Use OpenCV for image processing and TensorFlow for data analysis. Use a deep learning model to run an algorithm to locate missing people in real time.

[1043] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1044] Step 1:

[1045] Autonomous vehicles and drones collect data from the affected areas. Cameras capture high-resolution image data, LiDAR sensors measure topographical data, and environmental sensors simultaneously collect environmental data such as seismic intensity, temperature, and humidity.

[1046] Input: Real-time images of the affected area, topographical data, and environmental data

[1047] Output: High-resolution image data, terrain data, environmental sensor data

[1048] How it works: Autonomous vehicles patrol the affected area, drones collect data from the air, cameras take images, LiDAR sensors collect topographical data, and environmental sensors capture various seismic and weather data.

[1049] Step 2:

[1050] The collected data is transmitted to a central server via wireless communication technology (e.g., 5G networks).

[1051] Input: High-resolution image data, terrain data, environmental sensor data

[1052] Output: Consolidated data sent to a central server

[1053] How it works: All collected data is sent to a central server in real time, using wireless communication and the MQTT protocol for instant data upload.

[1054] Step 3:

[1055] The server consolidates and stores the received data.

[1056] Input: Consolidated data sent to a central server

[1057] Output: A consolidated dataset

[1058] What it does: The server receives the input data, adjusts it to fit each format (e.g., decompresses image data, normalizes sensor data), and stores it as a single integrated dataset.

[1059] Step 4:

[1060] The combined data is then analyzed using AI models: image data is analyzed using OpenCV, and sensor data is processed using deep learning models.

[1061] Input: Integrated dataset

[1062] Output: Analysis result (probability of missing person existence)

[1063] How it works: The integrated data is fed into an AI model using TensorFlow. Image data is analyzed using OpenCV to identify damaged areas and critical points, and environmental data is analyzed using a deep learning model. This calculates the probability of the presence of missing people.

[1064] Step 5:

[1065] Based on the analysis results, the location information and probability of the missing person's presence are predicted and sent to a smartphone app, where they are displayed and notified in real time.

[1066] Input: Analysis result (probability of missing person existence)

[1067] Output: Display and notification on smartphone app

[1068] What it does: The server sends the analysis results to a smartphone app. The app receives the results, displays them visually, and sends notifications. Areas with a high probability of missing people are highlighted on a map for the search team user.

[1069] Step 6:

[1070] Search teams using a smartphone app can efficiently carry out search activities based on the displayed prediction results.

[1071] Input: Display and notification on smartphone app

[1072] Output: Efficient search operations

[1073] What it does: Search team users use the information displayed in the app to optimize their missing person search plans and respond quickly.

[1074] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1075] This invention relates to a system for efficiently and quickly searching for missing persons after natural disasters such as earthquakes. This system includes a means for collecting images and sensor data from the disaster area, a means for transmitting the collected data to a central server, a means for integrating and analyzing the data, and a means for predicting the probability of the presence of missing persons based on the analysis results, and displaying and notifying the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the efficiency of search activities is further improved.

[1076] Program processing overview

[1077] The program in this system includes the following main processes:

[1078] 1. Data collection in the affected areas

[1079] Terminal

[1080] Drones and environmental sensors will collect data from the disaster area. Specifically, drones will take images with high-resolution cameras, and environmental sensors will collect real-time data on seismic intensity, temperature, humidity, gas concentration, etc. Location information will also be acquired from the smartphones of disaster victims, which will then be provided as data.

[1081] 2. Data transmission and storage

[1082] Terminal

[1083] The collected data is sent to a central server via wireless communication or 4G / 5G networks. Image data captured by drones is sent as large files, while data from sensors is sent using IoT protocols (e.g., MQTT). Location information from victims' smartphones is also sent via GPS and the Internet.

[1084] 3. Data integration and analysis

[1085] server

[1086] The central server consolidates all the data it receives. It applies image processing algorithms to the image data to identify the extent of damage to buildings. The sensor data is processed in real time, and the results are used to generate an environmental map of the entire affected area. It also analyzes the location data of victims and plots their distribution on the map.

[1087] 4. Missing Person Prediction

[1088] server

[1089] Based on the analyzed data, an AI algorithm is used to predict the probability of missing persons being present. Past data and the current situation in the disaster area are referenced to identify areas or buildings where missing persons are likely to be present. The prediction results are listed and saved.

[1090] 5. Operation of the Emotion Engine

[1091] Terminals and Servers

[1092] The emotion engine recognizes emotions by analyzing the voices and facial expressions of users participating in search operations. Emotion data is sent in real time to a central server for analysis. For example, if a user's stress level is high, that information can be used to adjust the priority of search operations.

[1093] 6. Display and notification of results

[1094] server

[1095] The server notifies search and rescue teams of the results of its predictions of missing persons, visualizes the results, and displays them on a map. It also displays emotion data collected by the emotion engine, helping to plan and coordinate the search.

[1096] Specific examples

[1097] For example, a drone flies over a disaster area and photographs the entire town with a high-resolution camera. Along with the captured images, a ground sensor collects seismic intensity information, and this data is sent to a central server. The server analyzes the received data, and an AI model predicts with high accuracy the probability of the presence of missing persons. The prediction results are then displayed on a map and notified to the Self-Defense Forces and rescue teams. Search team users also use an emotion engine to send their own stress and fatigue to the server. The server uses this information to suggest areas to prioritize search and efficient resource allocation.

[1098] As a result, the present invention supports rapid and efficient search activities during disasters, contributing to the early discovery and rescue of missing persons.

[1099] The processing flow will be explained below.

[1100] Step 1: Collect data

[1101] Terminal

[1102] Action 1: A drone flies over the affected area and takes images with a high-resolution camera.

[1103] Action 2: Environmental sensors collect data such as seismic intensity, temperature, humidity, and gas concentration in real time at the installation location.

[1104] Action 3: The victim's smartphone obtains location information and records that data.

[1105] Step 2: Send and store data

[1106] Terminal

[1107] Operation 1: The image data captured by the drone is transmitted to a central server via wireless communication or 4G / 5G networks.

[1108] Action 2: The data acquired by the environmental sensors is sent to a central server using an IoT protocol (e.g., MQTT).

[1109] Action 3: The location information acquired by the victim's smartphone is sent to a central server using GPS.

[1110] Step 3: Integrate the data

[1111] server

[1112] Action 1: The server centrally stores all received data.

[1113] Action 2: The server combines image data, sensor data, and location information and converts them into a single dataset.

[1114] Action 3: The server converts the consolidated data into a usable format for subsequent analysis.

[1115] Step 4: Analyze the data

[1116] server

[1117] Action 1: Based on the integrated data, the server applies image processing algorithms to identify the extent of damage to the building.

[1118] Step 2: The server generates an environment map in real time based on the sensor data.

[1119] Step 3: The server analyzes the location information of the victims and visualizes their distribution on a map.

[1120] Step 5: Predict missing persons

[1121] server

[1122] Action 1: The server inputs the analyzed data into an AI model to predict areas where the missing person is likely to be located.

[1123] Operation 2: The server compares past data with current data and filters the output prediction results to further improve accuracy.

[1124] Action 3: The server lists the final prediction results and saves the list.

[1125] Step 6: Emotion Recognition with the Emotion Engine

[1126] Terminals and Servers

[1127] Action 1: The device analyzes the user's voice and facial expressions to collect emotional data.

[1128] Action 2: The device sends the collected emotion data to the central server.

[1129] Action 3: The server analyzes the emotional data and evaluates the user's stress level and fatigue.

[1130] Step 7: View and notify results

[1131] server

[1132] Action 1: The server visualizes the missing person prediction results and displays them on a map.

[1133] Action 2: The server also displays the emotion data collected by the emotion engine.

[1134] Action 3: The server notifies search and rescue teams of the prediction results in real time.

[1135] Step 8: Conduct a search operation

[1136] User

[1137] Action 1: The rescue team receives the prediction results from the server and formulates a search plan.

[1138] Action 2: Rescue teams carry out specific search operations in the disaster area.

[1139] Action 3: The rescue team feeds back the results of their on-site search to the server, which helps improve the accuracy of their next prediction.

[1140] The above is a detailed explanation of the specific processing steps of the system for searching for missing persons in the event of a disaster that combines an emotion engine, and its operation.

[1141] Example 2

[1142] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1143] Rapid and efficient search for missing persons during disasters is difficult with current technology. Not only must image and sensor data from the disaster area be collected, but that data must also be processed quickly to accurately predict the probability of missing persons being found. It is also necessary to allocate resources efficiently while taking into account the emotions of users participating in the search operation. A system that can solve these problems is needed.

[1144] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1145] In this invention, the server includes means for collecting images and sensor data of the disaster area, means for transmitting the collected data to a central server, means for integrating the data transmitted to the central server, means for predicting the probability of the presence of missing persons based on the integrated data, means for displaying and notifying the user of the prediction result, and means for recognizing the user's emotions and adjusting the priority of search activities, thereby enabling a fast and efficient search for missing persons in a disaster.

[1146] "Disaster area" refers to an area affected by a natural disaster.

[1147] "Images" refers to photographs and video data taken to visually record the situation in the disaster area.

[1148] "Sensor data" refers to various types of data acquired by sensors to collect environmental information in disaster-stricken areas, including seismic intensity, temperature, humidity, gas concentration, etc.

[1149] "Central server" refers to a computer system that stores and analyzes collected data and provides information useful for search operations.

[1150] "Integrating" refers to the process of collecting, organizing, and analyzing different types of data in a centralized manner.

[1151] "AI algorithm" refers to artificial intelligence technology that automatically learns patterns and trends in data and predicts behavior.

[1152] "Probability of presence" refers to the probability that a missing person may be present in a particular location.

[1153] "Prediction result" refers to the probability of a missing person being found calculated using an AI algorithm.

[1154] "Display and notification" refers to the process of visually displaying the analysis results and communicating information to interested parties.

[1155] An "emotion engine" refers to technology that analyzes a user's voice and facial expressions to recognize their emotional state, such as stress or fatigue.

[1156] "Autonomous flying devices" refer to devices that fly automatically and collect images and data without human control. Drones generally fall into this category.

[1157] "Environmental sensors" refer to devices used to measure environmental information in disaster-stricken areas, such as seismometers, temperature sensors, and gas detectors.

[1158] "Users" refers to people engaged in search and rescue activities during disasters, such as members of search and rescue teams.

[1159] The present invention relates to a system for efficiently and quickly searching for missing persons in the event of a natural disaster such as an earthquake. This system collects images and sensor data from the disaster area, analyzes them to predict the probability of the presence of missing persons, and visualizes and notifies the user.

[1160] 1. Data collection in the affected areas

[1161] Terminal

[1162] The system uses drones, environmental sensors, and the victims' smartphones. The drones are equipped with high-resolution cameras to collect images of the entire disaster area. The environmental sensors collect data such as seismic intensity, temperature, humidity, and gas concentration in real time. The victims' smartphones use GPS to obtain location information.

[1163] 2. Data transmission and storage

[1164] Terminal

[1165] The collected data is transmitted to a central server via wireless communication or 4G / 5G networks. The large amount of image data collected by the drone is transmitted using a dedicated bandwidth, and sensor data is transmitted in real time using the MQTT protocol. Smartphone location information is also sent to the central server via the Internet.

[1166] 3. Data integration and analysis

[1167] server

[1168] A central server consolidates all data. High-resolution images are stored in a database, and data from environmental sensors and location information are also managed centrally. Image processing algorithms are applied to analyze the damage to buildings, and sensor data is processed in real time to generate a map of the affected area. The data is plotted on the map based on the location information of victims.

[1169] 4. Missing Person Prediction

[1170] server

[1171] AI algorithms, such as deep learning and machine learning models (CNN, LSTM, etc.), are used to predict the probability of missing persons being found. Based on past data and the current situation in the disaster area, locations with a high probability of missing persons are identified, and the results are listed and saved.

[1172] 5. Operation of the Emotion Engine

[1173] Terminals and Servers

[1174] The emotion engine analyzes the user's voice and facial expressions in real time and sends emotional data to a central server, which then adjusts the priority of search activities if the user's stress level is high.

[1175] 6. Display and notification of results

[1176] server

[1177] Based on the analyzed data, prediction results are visually displayed on a map. Information is then sent to search and rescue teams via a dedicated app, SMS, email, etc. Emotional data is also displayed to assist in the planning and adjustment of search plans.

[1178] Specific examples

[1179] For example, a drone flies over a disaster-hit area, capturing images of the entire town with a high-resolution camera. At the same time, sensors on the ground collect seismic intensity information and gas concentrations, and all data is sent to a central server. The server then integrates and analyzes this data, and an AI model predicts the probability of the presence of missing persons. The results are displayed on a map and communicated to the Self-Defense Forces and rescue teams. Search team members can also use an emotion engine to send their own stress and fatigue levels to the server, which then prioritizes search areas and resource allocations.

[1180] Example prompts for generative AI models

[1181] "In order to quickly search for missing people in disaster areas, please tell me how to collect data, what algorithms should be used to make predictions and analyses, and how to display and notify the results."

[1182] As a result, the present invention supports rapid and efficient search activities in the event of a disaster, contributing to the early discovery and rescue of missing persons.

[1183] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1184] Step 1: Collecting data from affected areas

[1185] The device uses drones and environmental sensors to collect data on the affected area. Specifically, the drone takes images with a high-resolution camera, and the environmental sensors collect data such as seismic intensity, temperature, humidity, and gas concentration. The victim's smartphone also uses GPS to obtain location information.

[1186] Inputs include image data from high-resolution cameras, environmental data from environmental sensors, and smartphone location data.

[1187] As an output, these collected data sets are produced.

[1188] Step 2: Data transmission and storage

[1189] The terminal transmits the collected data to a central server via wireless communication or 4G / 5G networks. Specifically, drone image data is transmitted as a large file, environmental sensor data is transmitted using the MQTT protocol, and smartphone location data is also transmitted via the Internet.

[1190] The input is all the data collected in step 1.

[1191] The output is a data store stored on a central server.

[1192] Step 3: Data integration and analysis

[1193] The server consolidates all the data it receives. Specifically, it stores high-resolution image data in a database, and also centrally manages environmental sensor data and smartphone location information. It applies image processing algorithms to extract and analyze information such as the extent of building damage. It processes the sensor data in real time to generate an environmental map of the entire disaster area.

[1194] The input is the data store saved in step 2.

[1195] The output is a consolidated and analyzed dataset that provides a situation map of the affected area.

[1196] Step 4: Predict missing persons

[1197] The server uses an AI algorithm to predict the probability of missing people based on the integrated data set. Specifically, it uses deep learning models (e.g., CNN, LSTM) to analyze past data and the current situation in the disaster area and identify locations where missing people are likely to be.

[1198] The input is the analyzed dataset generated in step 3.

[1199] The output is a prediction result that lists the probability of the missing person being present.

[1200] Step 5: Emotion Engine in Action

[1201] The device and server analyze the user's voice and facial expressions in real time and send emotional data to a central server, which then aggregates the emotional data and analyzes the user's stress and fatigue levels. Priorities are adjusted based on this information.

[1202] Inputs include voice and facial expression data obtained from the user.

[1203] As an output, the analyzed emotional data is sent to a central server, which adjusts the priorities of search operations.

[1204] Step 6: View and notify results

[1205] The server notifies search and rescue teams of the predicted probability of a missing person's presence. The results are sent via a dedicated app, SMS, or email. Furthermore, the prediction results and emotion data are visually displayed on a map to assist in the planning and coordination of search plans.

[1206] The inputs are the prediction results obtained in step 4 and the emotion data collected in step 5.

[1207] The output is a visualized map and a notification.

[1208] (Application example 2)

[1209] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1210] Efficiently and quickly searching for missing persons during natural disasters is difficult and requires accurate data collection and analysis across a wide area of ​​the disaster area. Furthermore, failure to consider the emotional state of the search team can reduce search efficiency and increase the risk of stress and fatigue. Therefore, a system is needed that enables fast and efficient search operations and enables the early discovery and rescue of missing persons.

[1211] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means including an autonomously driving vehicle that autonomously collects data on the disaster area, means for collecting images and sensor data using drones and environmental sensors, means for transmitting the collected data to a central server, means for integrating and analyzing the data transmitted to the central server, means for predicting the probability of the presence of missing persons based on the integrated data, means for displaying the prediction results on a map and notifying search teams and rescue teams, means including an emotion engine that collects and analyzes emotion data from the search team, and means for adjusting the search plan based on the emotion data. This enables fast and efficient search activities and realizes the early discovery and rescue of missing persons.

[1212] An "autonomous vehicle" refers to a vehicle that can drive autonomously without a driver and travel to a specific destination.

[1213] A "drone" refers to an unmanned aircraft that flies and collects images and sensor data.

[1214] "Environmental sensor" refers to a sensor device for detecting environmental data such as seismic intensity, temperature, humidity, and gas concentration.

[1215] "Central server" refers to a central management system for integrating and analyzing data collected from disaster-stricken areas.

[1216] "Data integration" refers to the process of centrally organizing data collected from multiple sources and making it ready for analysis.

[1217] "Probability of presence" refers to the probability that a missing person may be present in a particular location.

[1218] The "emotion engine" refers to a system that analyzes the voices and facial expressions of the search team and recognizes their emotional state.

[1219] "Emotion data" refers to information collected by the emotion engine that indicates the user's emotional state, such as stress or fatigue.

[1220] "Search plan" refers to a plan drawn up to efficiently search for a missing person.

[1221] A "generative AI model" refers to an artificial intelligence algorithm that learns from large amounts of data and predicts the probability of a missing person's existence, etc.

[1222] The present invention relates to a system for efficiently and quickly searching for missing people during natural disasters. This system is composed of a combination of autonomous vehicles, drones, environmental sensors, a central server, an emotion engine, and a generative AI model. Each part of the present invention is described in detail below.

[1223] Data collection

[1224] The server collects data using autonomous, unmanned vehicles that travel across a wide area of ​​the disaster-stricken area. The autonomous vehicles are equipped with drones and environmental sensors, and take images using high-resolution cameras. Environmental data such as seismic intensity, temperature, humidity, and gas concentration are also collected at the same time. Drones installed in various locations in the disaster-stricken area collect images from an aerial perspective, allowing the entire situation in the disaster-stricken area to be grasped.

[1225] Data transmission

[1226] The devices transmit the collected data to a central server in real time via wireless communication or 4G / 5G networks. Image data is sent as large files, and sensor data is transmitted using IoT protocols (e.g., MQTT). Location information from the victim's smartphone is also sent to the server via GPS and the Internet.

[1227] Data Integration and Analysis

[1228] The server integrates and analyzes all the received data. Using a generative AI model trained using TensorFlow and Keras, it uses image processing algorithms to identify the extent of damage to buildings. Sensor data is also processed in real time, and the results are used to generate an environmental map of the entire disaster area. Furthermore, it analyzes the location data of victims and plots their distribution on the map.

[1229] Missing Persons Prediction

[1230] Based on the analyzed data, the server uses a generative AI model to predict the probability of missing persons being present. It references past data and the current situation in the disaster area to identify locations in specific areas or buildings where missing persons are likely to be present. The prediction results are then listed and saved.

[1231] Emotion Engine Operation

[1232] The server uses an emotion engine to analyze the voices and facial expressions of users participating in the search operation to recognize their emotions. Emotional data is sent to a central server in real time and analyzed there. For example, if a user's stress level is high, that information can be used to adjust the priority of the search operation.

[1233] Displaying and notifying results

[1234] The server notifies search and rescue teams of the predicted missing persons, visualizes the results, and displays them on a map. It also displays emotional data collected by the emotion engine, helping them plan and coordinate their search.

[1235] Specific examples

[1236] For example, in a disaster area where a magnitude 7 earthquake has occurred, autonomous vehicles begin collecting data. High-resolution image data captured by the autonomous vehicles, aerial video data collected by drones, and seismic intensity information from environmental sensors are transmitted in real time to a central server. The server integrates this data, and a generative AI model predicts the probability of missing persons being present. As a result, specific areas are predicted to have a high probability of missing persons, and the search team is notified. Furthermore, the emotional state of the search team is monitored, and the search plan is adjusted if stress levels are high.

[1237] Here are some example prompts:

[1238] “In the event of a natural disaster, predict the probability of missing persons in the affected area using high-resolution images captured by drones and environmental sensor data. Consider variables such as building damage, temperature, humidity, and gas concentrations.”

[1239] As a result, the present invention enables fast and efficient search operations, contributing to the early discovery and rescue of missing persons.

[1240] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1241] Step 1:

[1242] The device uses autonomous vehicles and drones to collect data from disaster-stricken areas. The autonomous vehicles drive autonomously through the disaster-stricken areas and take images with high-resolution cameras. The drones collect aerial image data, while environmental sensors collect real-time data such as seismic intensity, temperature, humidity, and gas concentration. This allows for detailed environmental information to be obtained.

[1243] Input: On-site environmental and image data from the affected areas

[1244] Output: High-resolution image data, seismic intensity data, temperature data, humidity data, gas concentration data

[1245] Step 2:

[1246] The devices transmit the collected data to a central server via wireless communication or 4G / 5G networks. Image data is sent as large files, and sensor data is transmitted using IoT protocols. Location information collected by victims' smartphones is also transmitted via GPS and the internet.

[1247] Input: High-resolution image data, seismic intensity data, temperature data, humidity data, gas concentration data, location information of victims

[1248] Output: Sending data to a central server

[1249] Step 3:

[1250] The server integrates all the received data. Specifically, it applies image processing algorithms to the image data using TensorFlow and Keras to identify the extent of damage to buildings. It also processes the sensor data in real time and generates an environmental map of the entire disaster area based on the results. The location data of the victims is analyzed and the distribution of victims is plotted on the map.

[1251] Input: All data sent to the central server (high-resolution image data, sensor data, location information of victims)

[1252] Output: Image data analysis results, environmental map, disaster victim distribution map

[1253] Step 4:

[1254] The server uses a generative AI model to predict the probability of missing persons being present. It references past data and the current situation in the disaster area to identify locations in specific areas or buildings where missing persons are likely to be present. The prediction results are listed and saved.

[1255] Input: Integrated analysis data (image data analysis results, environmental map, disaster victim distribution map)

[1256] Output: List of predicted probability of missing person existence

[1257] Step 5:

[1258] The server uses an emotion engine to analyze the voices and facial expressions of users participating in the search operation to recognize their emotions. Emotional data is sent to the server in real time for analysis. For example, if a user's stress level is high, the server can adjust the priority of the search operation based on that information.

[1259] Input: User's voice data and facial expression data

[1260] Output: Parsed emotion data

[1261] Step 6:

[1262] The server notifies search and rescue teams of predicted missing persons and displays the results on a map. It also displays emotion data collected by the emotion engine, helping to plan and coordinate the search.

[1263] Input: List of predicted probability of missing persons, analyzed emotion data

[1264] Output: Map predictions, notifications to search and rescue teams, coordinated search plans

[1265] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1266] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1267] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1268] [Fourth embodiment]

[1269] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1270] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1271] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1272] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1273] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1274] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1275] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1276] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1277] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1278] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1279] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1280] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1281] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1282] This invention relates to a system for efficiently and quickly searching for missing persons after natural disasters such as earthquakes. This system includes a means for collecting images and sensor data from the disaster area, a means for transmitting the collected data to a central server, a means for integrating and analyzing the data, and a means for predicting the probability of the presence of missing persons based on the analysis results, and displaying and notifying the user.

[1283] Program processing overview

[1284] The program in this system includes the following steps as its main processing steps.

[1285] 1. Data collection in the affected areas

[1286] Terminal

[1287] The devices include drones and environmental sensors. Drones operating over the affected area use high-resolution cameras to capture images of the affected area, while sensors on the ground and in the air collect seismic intensity, temperature, humidity, and other important environmental data.

[1288] A specific example would be a drone flying over a disaster area, photographing the damage to buildings, and sensors recording seismic intensity and temperature data on the ground surface.

[1289] 2. Data transmission and storage

[1290] Terminal

[1291] The collected data is transmitted in real time to a central server via wireless communication or 4G / 5G networks. Image data captured by the drone's camera is uploaded to the server as a large file. Data from sensors is also transmitted in the same way using IoT protocols.

[1292] For example, images taken by drones can be instantly uploaded to a central server via a 5G network, while data collected by sensors on the ground can be transmitted via the MQTT protocol.

[1293] 3. Data integration and analysis

[1294] server

[1295] The central server consolidates all the data it receives, applies image processing algorithms to the image data, and analyzes the damage to buildings. The sensor data is processed in real time, and the results are used to generate an environmental map of the entire disaster area.

[1296] Specifically, the server analyzes images received from the drone to identify damaged areas of buildings, and maps seismic intensity and temperature data to provide a visual understanding of the situation in the affected areas.

[1297] 4. Missing Person Prediction

[1298] server

[1299] Based on the analyzed data, an AI algorithm is used to predict the probability of missing persons being present, and by referencing past data and the current situation in the disaster area, it identifies locations in specific areas or buildings where missing persons are likely to be present.

[1300] For example, the server inputs data into a trained AI model to calculate areas where there are likely to be many missing people, and based on this result, analyzes patterns in which missing people are likely to occur.

[1301] 5. Display and notification of results

[1302] server

[1303] The prediction results are sent to search and rescue teams in real time. The server generates a map showing areas where there is a high probability of missing people and sends it to relevant parties. These parties can use this map to develop effective search plans.

[1304] For example, the server highlights areas on a map where there is a high probability of missing persons being found in red and sends this information to the Self-Defense Forces via email. It also displays the information in real time on a dedicated application, enabling a rapid response.

[1305] Specific examples

[1306] One example would be a drone flying over a disaster area, capturing images of the entire town with a high-resolution camera. Along with the captured images, ground sensors collect seismic intensity information, which is then immediately sent to a central server. The server analyzes the received data, and an AI model accurately predicts the probability of the presence of missing persons. The predictions are then displayed on a map and notified to the Self-Defense Forces and rescue teams, allowing for efficient search operations.

[1307] The processing flow will be explained below.

[1308] Step 1: Collect data

[1309] Terminal

[1310] Action 1: A drone flies over the affected area and takes high-resolution images with its camera.

[1311] Action 2: Environmental sensors collect data such as seismic intensity, temperature, humidity, and gas concentration in real time.

[1312] Action 3: The victim's smartphone obtains location information and collects data.

[1313] Step 2: Sending data

[1314] Terminal

[1315] Operation 1: The image data captured by the drone is transmitted to a central server via wireless communication or 4G / 5G networks.

[1316] Action 2: The data collected by the environmental sensors is sent to a central server using an IoT protocol (e.g., MQTT).

[1317] Action 3: The location information obtained by the victim's smartphone is sent to a central server via GPS and the Internet.

[1318] Step 3: Integrate the data

[1319] server

[1320] Operation 1: The server temporarily saves the received image data and stores it in a database.

[1321] Action 2: The server receives the sensor data and synthesizes it into an appropriate format.

[1322] Action 3: The server combines different types of data, such as images, sensors, and location information, into a single dataset.

[1323] Step 4: Analyze the data

[1324] server

[1325] Action 1: The server applies image processing algorithms to analyze the integrated data and identify the damage to the building.

[1326] Action 2: The server analyzes the sensor data in real time and grasps the environmental situation of the entire disaster area.

[1327] Action 3: The server analyzes the location data of the victims and plots the distribution of the victims on a map.

[1328] Step 5: Predict missing persons

[1329] server

[1330] Action 1: The server inputs the analyzed data into an AI model to predict areas where the missing person is likely to be located.

[1331] Step 2: The server takes past data into account and filters the output prediction results to further improve accuracy.

[1332] Action 3: The server lists the final prediction results and saves the list.

[1333] Step 6: Notification and display of results

[1334] server

[1335] Action 1: The server visualizes the prediction results and displays them on a map.

[1336] Action 2: The server notifies the search and rescue team of the prediction results.

[1337] Action 3: Receive feedback based on the information notified by the server and take action to reflect it in the next prediction.

[1338] Step 7: Conduct a search operation

[1339] User

[1340] Action 1: The rescue team formulates a search plan based on the prediction results received from the server.

[1341] Action 2: Rescue teams carry out specific search operations on site.

[1342] Action 3: The rescue team reports the results of the on-site search to the server, so that they can be reflected as data for the next step.

[1343] Example 1

[1344] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1345] Rapid and efficient search for missing persons in natural disasters such as earthquakes is a key challenge in rescue operations. However, understanding the situation in widespread disaster areas and identifying locations with a high probability of missing persons taking time and requiring significant human resources. Conventional methods have difficulty in accurately collecting and analyzing data in real time, limiting their effectiveness in situations where a rapid response is required.

[1346] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1347] In this invention, the server includes means for collecting images and environmental data of the disaster area, means for transmitting the collected data to a central server, means for storing the data transmitted to the central server, means for integrating the stored data, means for analyzing the integrated data, means for predicting the probability of the presence of missing persons based on the analyzed data, and means for displaying and notifying the results of the prediction. This enables real-time data collection and analysis over a wide area of ​​the disaster area, enabling rapid and efficient search activities for missing persons.

[1348] "Images" refer to photographs and video data used to obtain visual information about the disaster area.

[1349] "Environmental data" refers to data used to measure environmental conditions in the affected areas, such as seismic intensity, temperature, and humidity.

[1350] "Collection means" refers to devices that acquire data from disaster areas using equipment such as drones and environmental sensors.

[1351] "Transmission means" refers to a mechanism for transmitting collected data to a central server using wireless communication or a network.

[1352] The "storage means" is a storage system for storing collected data in the server.

[1353] "Integration" is the process of combining multiple stored data sets and processing them as a unified data set.

[1354] "Analysis tools" are algorithms and programs used to analyze the integrated data and grasp the situation in the disaster-stricken areas.

[1355] A "predictive tool" is a generative AI model or other algorithm used to calculate the probability of a missing person's existence based on analyzed data.

[1356] "Display means" refers to a device or interface for visually presenting the analysis and prediction results.

[1357] "Notification means" is a system for communicating prediction results to relevant parties in real time.

[1358] An "autonomous mobile object" is a device that can move automatically without external operation, such as a drone or robot.

[1359] An "environmental information acquisition device" is a sensor or measuring instrument used to collect environmental data such as temperature, humidity, and seismic intensity.

[1360] A "data processing algorithm" is a calculation method or program used to analyze acquired data.

[1361] The present invention relates to a system for quickly and efficiently searching for missing persons in the event of a natural disaster such as an earthquake. The system includes a data collection means, a data transmission means, a data storage means, a data integration means, a data analysis means, a means for predicting the probability of the presence of missing persons, a result display means, and a notification means.

[1362] Data collection methods

[1363] Terminal

[1364] The devices used include drones and environmental sensors. Drones are equipped with high-resolution cameras and fly over affected areas to collect image data. Environmental sensors collect environmental data such as seismic intensity, temperature, and humidity in real time. Drones use automatic navigation functions to fly over affected areas along pre-set routes. For example, drones may take aerial photographs of the damage to an entire town, while ground sensors measure seismic intensity and temperature.

[1365] Data transmission method

[1366] Terminal

[1367] The collected data is transmitted to a central server in real time via wireless communication or 4G / 5G networks. The large amount of image data collected by the drone is instantly uploaded using the drone's built-in communication module. Data from the sensors is also transmitted to the server using IoT protocols (e.g., MQTT). Specifically, the images captured by the drone are uploaded to the central server via the 5G network, and the ground sensors transmit seismic intensity data via the MQTT protocol.

[1368] Data storage means

[1369] server

[1370] The server first stores the data it receives. Image data sent from the drone is stored on a dedicated storage server, and data from the sensors is stored in a database system (e.g., MongoDB). Each piece of data is given a timestamp to clarify the time the data was collected.

[1371] Data Integration Methods

[1372] server

[1373] The server integrates multiple stored data sets. For example, it combines image data and sensor data to generate an environmental map of the entire disaster area. By overlaying the image data and displaying seismic intensity and temperature information in color, the current situation in the disaster area can be visually grasped. Image processing algorithms and data mapping techniques are used.

[1374] Data Analysis Methods

[1375] server

[1376] The server analyzes the integrated data, using image processing algorithms to assess the damage to buildings and quantify the degree of damage. It also analyzes collected seismic intensity and temperature data, integrating this information to understand the detailed situation in the affected area. For example, images of damaged buildings can be input into an AI model to classify the damage state in detail.

[1377] A method for predicting the probability of a missing person's existence

[1378] server

[1379] Based on the analyzed data, the server uses a generative AI model to predict the probability of missing persons. It uses past data and the current situation in the disaster area as a reference to calculate the possibility of missing persons in a specific area or building. Specifically, the latest data is input into an AI model trained on data from specific areas where many missing persons have occurred in the past, and the server outputs the probability of missing persons being present.

[1380] Result display means and notification means

[1381] server

[1382] The server notifies search teams and rescue teams of the prediction results. The prediction results are displayed on a map, highlighting areas with a high probability of missing people in color. The results are also sent to the Self-Defense Forces and rescue teams via email or a dedicated app. For example, areas with a high probability of missing people being present on a map can be displayed in red, and detailed information about the area can be attached to send a real-time notification to the rescue team.

[1383] Prompt Sentence Examples

[1384] "Please provide an overview of the data collection and analysis system for predicting the probability of missing persons in disaster areas."

[1385] The above is a specific embodiment for carrying out the present invention. This system enables real-time data collection and analysis in a wide area of ​​a disaster, enabling rapid and efficient search activities for missing persons.

[1386] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1387] Step 1: Data collection

[1388] Terminal

[1389] The device uses a drone and environmental sensors. The drone uses a high-resolution camera to capture images of the sky over the affected area, while environmental sensors on the ground measure environmental data such as seismic intensity, temperature, and humidity. Specifically, the drone flies automatically along a pre-set route, capturing detailed images of the damage to buildings and the terrain. The environmental sensors also collect seismic intensity and temperature data measured every five seconds and store it in their internal memory.

[1390] Input: Visual and environmental information of the affected area

[1391] Output: High-resolution disaster image data and environmental sensor data

[1392] Step 2: Send data

[1393] Terminal

[1394] The collected data is transmitted to a central server in real time via wireless communication or 4G / 5G networks. Image data captured by drones is transmitted as large files, while data from environmental sensors is transmitted every second using IoT protocols (such as MQTT). For example, images captured by drones can be immediately uploaded using the 5G network, and seismic intensity data recorded by sensors can be sent to the server in batches every minute.

[1395] Input: High-resolution disaster image data and environmental sensor data

[1396] Output: Data sent to the central server

[1397] Step 3: Save Data

[1398] server

[1399] The central server stores the received data. Specifically, image data sent from the drone is stored on a dedicated storage server, and data from the sensors is stored in a database system (e.g., MongoDB). Each piece of data is given a timestamp, and the time the data was collected is clearly managed.

[1400] Input: Data sent to the central server

[1401] Output: Time-stamped saved data

[1402] Step 4: Data Integration

[1403] server

[1404] The server integrates the various stored data. Specifically, it combines image data and sensor data to generate an environmental map of the entire disaster area. In doing so, it makes full use of image processing algorithms, overlays the image data, and color-codes information such as seismic intensity and temperature to visually display the situation in the disaster area.

[1405] Input: Time-stamped stored data

[1406] Output: Unified environment map

[1407] Step 5: Data analysis

[1408] server

[1409] The server analyzes the integrated data, using image processing algorithms to assess the damage to buildings and quantify the degree of damage. It also analyzes collected seismic intensity and temperature data, integrating this information to understand the current situation in the affected area. For example, images of damaged buildings can be input into an AI model, which then classifies the damage state in detail (completely damaged, partially damaged, undamaged, etc.).

[1410] Input: Unified environment map

[1411] Output: Analysis results (evaluation of damage state, etc.)

[1412] Step 6: Predict the probability of the missing person being present

[1413] server

[1414] The server uses a generative AI model based on the analyzed data to predict the probability of missing persons. It calculates the possibility of missing persons in specific areas or buildings by referring to past data and the current situation in the disaster area. It inputs the latest analysis results into an AI model trained on data from specific areas where many people have gone missing in the past, and outputs the probability of missing persons.

[1415] Input: Analysis results

[1416] Output: Prediction result of the probability of the missing person being present

[1417] Step 7: Results display and notification

[1418] server

[1419] The server notifies search teams and rescue teams of the prediction results. The prediction results are displayed on a map, and areas with a high probability of missing persons being present are highlighted in color. This information is also sent to the Self-Defense Forces and rescue teams via email or a dedicated app. For example, areas with a high probability of missing persons being present can be displayed in red on a map, and detailed information about the area can be attached to send a real-time notification.

[1420] Input: Missing person existence probability prediction result

[1421] Output: Display and notification of prediction results

[1422] (Application example 1)

[1423] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1424] Searches for missing people during natural disasters must be carried out quickly and efficiently. However, conventional methods tend to be slow in data collection, information integration, and analysis, resulting in a lack of speed and accuracy in rescue operations. Data accuracy and real-time reporting are also issues. To solve these problems, a new search system utilizing the latest technology is needed.

[1425] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1426] In this invention, the server includes means for collecting images and sensor data of the disaster area, means for transmitting the collected data to a central server, means for integrating the data transmitted to the central server, means for predicting the probability of the presence of missing persons based on the integrated data, means for displaying and notifying the prediction results, means for using data collected by cameras and sensors mounted on the autonomous vehicle, means for performing analysis and prediction in real time using an AI model, and means for displaying and notifying the prediction results on a smartphone app, thereby enabling a fast and accurate search for missing persons.

[1427] "Images and sensor data from disaster areas" refers to visual information and wireless communication data collected using devices such as cameras and environmental sensors in areas where natural disasters have occurred.

[1428] "Collection methods" refers to the devices and methods used to gather important data from disaster areas, including drones, autonomous vehicles, and environmental sensors.

[1429] The "central server" is a computer system that receives and integrates all collected data and performs analysis and predictions.

[1430] "Integration means" refers to a method or device that brings together multiple collected data and prepares them for analysis.

[1431] A "prediction method" is an algorithm or machine learning model that calculates and predicts the probability of a missing person's presence based on the integrated data.

[1432] "Display and notification means" refers to devices and applications that display prediction results to users and notify them in real time. Specifically, this includes smartphone apps and dedicated display devices.

[1433] "Autonomous vehicle-mounted cameras and sensors" refers to data collection devices, such as high-resolution cameras and LiDAR sensors, mounted on an autonomous vehicle.

[1434] An "AI model" is an artificial intelligence algorithm that analyzes data and makes predictions based on the knowledge it has learned.

[1435] "Smartphone app" refers to dedicated application software that runs on a smartphone and is accessible to users.

[1436] In this invention, in order to build a system that can efficiently and quickly search for missing people in the event of a natural disaster, images and sensor data from the disaster area are collected, and based on that, an AI model is used to analyze, predict, display, and notify. The specific configuration and operation of the system are described below.

[1437] System configuration

[1438] Hardware:

[1439] Autonomous vehicles: Equipped with high-resolution cameras and LiDAR sensors, they will collect images and environmental data of the affected area.

[1440] Drones: Equipped with high-resolution cameras, they take images of the affected areas.

[1441] Environmental sensors: Installed on the ground surface, they collect environmental data such as seismic intensity, temperature, and humidity.

[1442] Central Server: A computer system that receives, stores, and analyzes all collected data.

[1443] Smartphone: A device that runs an application that displays and notifies users of prediction results.

[1444] software:

[1445] Image processing algorithm: Use libraries such as OpenCV to analyze image data from the disaster area.

[1446] AI model: A deep learning algorithm that uses frameworks such as TensorFlow to predict the probability of a missing person's presence.

[1447] IoT protocols: Use MQTT or similar to send sensor data to a server in real time.

[1448] Smartphone app: A dedicated application for displaying and notifying prediction results in real time.

[1449] Data collection and transmission

[1450] Autonomous vehicles, drones, and environmental sensors will collect data from the disaster area. Cameras mounted on autonomous vehicles and drones will capture high-resolution images, and LiDAR sensors will collect terrain data. Environmental sensors will capture important environmental data in real time, such as seismic intensity, temperature, and humidity. This data will be transmitted to a central server via wireless communication technology (e.g., 5G network).

[1451] Data integration and analysis

[1452] The central server consolidates all the data it receives. Image processing algorithms are applied to the image data using OpenCV to identify damaged areas of buildings. Sensor data is integrated in real time to generate an environmental map of the entire affected area.

[1453] Missing person prediction and notification

[1454] Based on the combined data, an AI model predicts the probability of a missing person being found. Deep learning algorithms using TensorFlow perform this analysis. The predictions are displayed in real time on a smartphone app and sent to the search team.

[1455] Examples of specific examples and AI prompts

[1456] As a concrete example, immediately after a disaster occurs, autonomous vehicles patrol the affected area and collect data using cameras and LiDAR sensors. This data is immediately transmitted to a central server via 5G communication. The server uses OpenCV and TensorFlow to analyze the images and sensor data and predict the probability of the presence of missing persons. This prediction result is displayed on a smartphone app and notified to the search team.

[1457] Example of a generated AI prompt:

[1458] Use the latest images and sensor data from the disaster area to predict areas where missing people are likely to be found. Use OpenCV for image processing and TensorFlow for data analysis. Use a deep learning model to run an algorithm to locate missing people in real time.

[1459] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1460] Step 1:

[1461] Autonomous vehicles and drones collect data from the affected areas. Cameras capture high-resolution image data, LiDAR sensors measure topographical data, and environmental sensors simultaneously collect environmental data such as seismic intensity, temperature, and humidity.

[1462] Input: Real-time images of the affected area, topographical data, and environmental data

[1463] Output: High-resolution image data, terrain data, environmental sensor data

[1464] How it works: Autonomous vehicles patrol the affected area, drones collect data from the air, cameras take images, LiDAR sensors collect topographical data, and environmental sensors capture various seismic and weather data.

[1465] Step 2:

[1466] The collected data is transmitted to a central server via wireless communication technology (e.g., 5G networks).

[1467] Input: High-resolution image data, terrain data, environmental sensor data

[1468] Output: Consolidated data sent to a central server

[1469] How it works: All collected data is sent to a central server in real time, using wireless communication and the MQTT protocol for instant data upload.

[1470] Step 3:

[1471] The server consolidates and stores the received data.

[1472] Input: Consolidated data sent to a central server

[1473] Output: A consolidated dataset

[1474] What it does: The server receives the input data, adjusts it to fit each format (e.g., decompresses image data, normalizes sensor data), and stores it as a single integrated dataset.

[1475] Step 4:

[1476] The combined data is then analyzed using AI models: image data is analyzed using OpenCV, and sensor data is processed using deep learning models.

[1477] Input: Integrated dataset

[1478] Output: Analysis result (probability of missing person existence)

[1479] How it works: The integrated data is fed into an AI model using TensorFlow. Image data is analyzed using OpenCV to identify damaged areas and critical points, and environmental data is analyzed using a deep learning model. This calculates the probability of the presence of missing people.

[1480] Step 5:

[1481] Based on the analysis results, the location information and probability of the missing person's presence are predicted and sent to a smartphone app, where they are displayed and notified in real time.

[1482] Input: Analysis result (probability of missing person existence)

[1483] Output: Display and notification on smartphone app

[1484] What it does: The server sends the analysis results to a smartphone app. The app receives the results, displays them visually, and sends notifications. Areas with a high probability of missing people are highlighted on a map for the search team user.

[1485] Step 6:

[1486] Search teams using a smartphone app can efficiently carry out search activities based on the displayed prediction results.

[1487] Input: Display and notification on smartphone app

[1488] Output: Efficient search operations

[1489] What it does: Search team users use the information displayed in the app to optimize their missing person search plans and respond quickly.

[1490] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1491] This invention relates to a system for efficiently and quickly searching for missing persons after natural disasters such as earthquakes. This system includes a means for collecting images and sensor data from the disaster area, a means for transmitting the collected data to a central server, a means for integrating and analyzing the data, and a means for predicting the probability of the presence of missing persons based on the analysis results, and displaying and notifying the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the efficiency of search activities is further improved.

[1492] Program processing overview

[1493] The program in this system includes the following main processes:

[1494] 1. Data collection in the affected areas

[1495] Terminal

[1496] Drones and environmental sensors will collect data from the disaster area. Specifically, drones will take images with high-resolution cameras, and environmental sensors will collect real-time data on seismic intensity, temperature, humidity, gas concentration, etc. Location information will also be acquired from the smartphones of disaster victims, which will then be provided as data.

[1497] 2. Data transmission and storage

[1498] Terminal

[1499] The collected data is sent to a central server via wireless communication or 4G / 5G networks. Image data captured by drones is sent as large files, while data from sensors is sent using IoT protocols (e.g., MQTT). Location information from victims' smartphones is also sent via GPS and the Internet.

[1500] 3. Data integration and analysis

[1501] server

[1502] The central server consolidates all the data it receives. It applies image processing algorithms to the image data to identify the extent of damage to buildings. The sensor data is processed in real time, and the results are used to generate an environmental map of the entire affected area. It also analyzes the location data of victims and plots their distribution on the map.

[1503] 4. Missing Person Prediction

[1504] server

[1505] Based on the analyzed data, an AI algorithm is used to predict the probability of missing persons being present. Past data and the current situation in the disaster area are referenced to identify areas or buildings where missing persons are likely to be present. The prediction results are listed and saved.

[1506] 5. Operation of the Emotion Engine

[1507] Terminals and Servers

[1508] The emotion engine recognizes emotions by analyzing the voices and facial expressions of users participating in search operations. Emotion data is sent in real time to a central server for analysis. For example, if a user's stress level is high, that information can be used to adjust the priority of search operations.

[1509] 6. Display and notification of results

[1510] server

[1511] The server notifies search and rescue teams of the results of its predictions of missing persons, visualizes the results, and displays them on a map. It also displays emotion data collected by the emotion engine, helping to plan and coordinate the search.

[1512] Specific examples

[1513] For example, a drone flies over a disaster area and photographs the entire town with a high-resolution camera. Along with the captured images, a ground sensor collects seismic intensity information, and this data is sent to a central server. The server analyzes the received data, and an AI model predicts with high accuracy the probability of the presence of missing persons. The prediction results are then displayed on a map and notified to the Self-Defense Forces and rescue teams. Search team users also use an emotion engine to send their own stress and fatigue to the server. The server uses this information to suggest areas to prioritize search and efficient resource allocation.

[1514] As a result, the present invention supports rapid and efficient search activities during disasters, contributing to the early discovery and rescue of missing persons.

[1515] The processing flow will be explained below.

[1516] Step 1: Collect data

[1517] Terminal

[1518] Action 1: A drone flies over the affected area and takes images with a high-resolution camera.

[1519] Action 2: Environmental sensors collect data such as seismic intensity, temperature, humidity, and gas concentration in real time at the installation location.

[1520] Action 3: The victim's smartphone obtains location information and records that data.

[1521] Step 2: Send and store data

[1522] Terminal

[1523] Operation 1: The image data captured by the drone is transmitted to a central server via wireless communication or 4G / 5G networks.

[1524] Action 2: The data acquired by the environmental sensors is sent to a central server using an IoT protocol (e.g., MQTT).

[1525] Action 3: The location information acquired by the victim's smartphone is sent to a central server using GPS.

[1526] Step 3: Integrate the data

[1527] server

[1528] Action 1: The server centrally stores all received data.

[1529] Action 2: The server combines image data, sensor data, and location information and converts them into a single dataset.

[1530] Action 3: The server converts the consolidated data into a usable format for subsequent analysis.

[1531] Step 4: Analyze the data

[1532] server

[1533] Action 1: Based on the integrated data, the server applies image processing algorithms to identify the extent of damage to the building.

[1534] Step 2: The server generates an environment map in real time based on the sensor data.

[1535] Step 3: The server analyzes the location information of the victims and visualizes their distribution on a map.

[1536] Step 5: Predict missing persons

[1537] server

[1538] Action 1: The server inputs the analyzed data into an AI model to predict areas where the missing person is likely to be located.

[1539] Operation 2: The server compares past data with current data and filters the output prediction results to further improve accuracy.

[1540] Action 3: The server lists the final prediction results and saves the list.

[1541] Step 6: Emotion Recognition with the Emotion Engine

[1542] Terminals and Servers

[1543] Action 1: The device analyzes the user's voice and facial expressions to collect emotional data.

[1544] Action 2: The device sends the collected emotion data to the central server.

[1545] Action 3: The server analyzes the emotional data and evaluates the user's stress level and fatigue.

[1546] Step 7: View and notify results

[1547] server

[1548] Action 1: The server visualizes the missing person prediction results and displays them on a map.

[1549] Action 2: The server also displays the emotion data collected by the emotion engine.

[1550] Action 3: The server notifies search and rescue teams of the prediction results in real time.

[1551] Step 8: Conduct a search operation

[1552] User

[1553] Action 1: The rescue team receives the prediction results from the server and formulates a search plan.

[1554] Action 2: Rescue teams carry out specific search operations in the disaster area.

[1555] Action 3: The rescue team feeds back the results of their on-site search to the server, which helps improve the accuracy of their next prediction.

[1556] The above is a detailed explanation of the specific processing steps of the system for searching for missing persons in the event of a disaster that combines an emotion engine, and its operation.

[1557] Example 2

[1558] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1559] Rapid and efficient search for missing persons during disasters is difficult with current technology. Not only must image and sensor data from the disaster area be collected, but that data must also be processed quickly to accurately predict the probability of missing persons being found. It is also necessary to allocate resources efficiently while taking into account the emotions of users participating in the search operation. A system that can solve these problems is needed.

[1560] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1561] In this invention, the server includes means for collecting images and sensor data of the disaster area, means for transmitting the collected data to a central server, means for integrating the data transmitted to the central server, means for predicting the probability of the presence of missing persons based on the integrated data, means for displaying and notifying the user of the prediction result, and means for recognizing the user's emotions and adjusting the priority of search activities, thereby enabling a fast and efficient search for missing persons in a disaster.

[1562] "Disaster area" refers to an area affected by a natural disaster.

[1563] "Images" refers to photographs and video data taken to visually record the situation in the disaster area.

[1564] "Sensor data" refers to various types of data acquired by sensors to collect environmental information in disaster-stricken areas, including seismic intensity, temperature, humidity, gas concentration, etc.

[1565] "Central server" refers to a computer system that stores and analyzes collected data and provides information useful for search operations.

[1566] "Integrating" refers to the process of collecting, organizing, and analyzing different types of data in a centralized manner.

[1567] "AI algorithm" refers to artificial intelligence technology that automatically learns patterns and trends in data and predicts behavior.

[1568] "Probability of presence" refers to the probability that a missing person may be present in a particular location.

[1569] "Prediction result" refers to the probability of a missing person being found calculated using an AI algorithm.

[1570] "Display and notification" refers to the process of visually displaying the analysis results and communicating information to interested parties.

[1571] An "emotion engine" refers to technology that analyzes a user's voice and facial expressions to recognize their emotional state, such as stress or fatigue.

[1572] "Autonomous flying devices" refer to devices that fly automatically and collect images and data without human control. Drones generally fall into this category.

[1573] "Environmental sensors" refer to devices used to measure environmental information in disaster-stricken areas, such as seismometers, temperature sensors, and gas detectors.

[1574] "Users" refers to people engaged in search and rescue activities during disasters, such as members of search and rescue teams.

[1575] The present invention relates to a system for efficiently and quickly searching for missing persons in the event of a natural disaster such as an earthquake. This system collects images and sensor data from the disaster area, analyzes them to predict the probability of the presence of missing persons, and visualizes and notifies the user.

[1576] 1. Data collection in the affected areas

[1577] Terminal

[1578] The system uses drones, environmental sensors, and the victims' smartphones. The drones are equipped with high-resolution cameras to collect images of the entire disaster area. The environmental sensors collect data such as seismic intensity, temperature, humidity, and gas concentration in real time. The victims' smartphones use GPS to obtain location information.

[1579] 2. Data transmission and storage

[1580] Terminal

[1581] The collected data is transmitted to a central server via wireless communication or 4G / 5G networks. The large amount of image data collected by the drone is transmitted using a dedicated bandwidth, and sensor data is transmitted in real time using the MQTT protocol. Smartphone location information is also sent to the central server via the Internet.

[1582] 3. Data integration and analysis

[1583] server

[1584] A central server consolidates all data. High-resolution images are stored in a database, and data from environmental sensors and location information are also managed centrally. Image processing algorithms are applied to analyze the damage to buildings, and sensor data is processed in real time to generate a map of the affected area. The data is plotted on the map based on the location information of victims.

[1585] 4. Missing Person Prediction

[1586] server

[1587] AI algorithms, such as deep learning and machine learning models (CNN, LSTM, etc.), are used to predict the probability of missing persons being found. Based on past data and the current situation in the disaster area, locations with a high probability of missing persons are identified, and the results are listed and saved.

[1588] 5. Operation of the Emotion Engine

[1589] Terminals and Servers

[1590] The emotion engine analyzes the user's voice and facial expressions in real time and sends emotional data to a central server, which then adjusts the priority of search activities if the user's stress level is high.

[1591] 6. Display and notification of results

[1592] server

[1593] Based on the analyzed data, prediction results are visually displayed on a map. Information is then sent to search and rescue teams via a dedicated app, SMS, email, etc. Emotional data is also displayed to assist in the planning and adjustment of search plans.

[1594] Specific examples

[1595] For example, a drone flies over a disaster-hit area, capturing images of the entire town with a high-resolution camera. At the same time, sensors on the ground collect seismic intensity information and gas concentrations, and all data is sent to a central server. The server then integrates and analyzes this data, and an AI model predicts the probability of the presence of missing persons. The results are displayed on a map and communicated to the Self-Defense Forces and rescue teams. Search team members can also use an emotion engine to send their own stress and fatigue levels to the server, which then prioritizes search areas and resource allocations.

[1596] Example prompts for generative AI models

[1597] "In order to quickly search for missing people in disaster areas, please tell me how to collect data, what algorithms should be used to make predictions and analyses, and how to display and notify the results."

[1598] As a result, the present invention supports rapid and efficient search activities in the event of a disaster, contributing to the early discovery and rescue of missing persons.

[1599] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1600] Step 1: Collecting data from affected areas

[1601] The device uses drones and environmental sensors to collect data on the affected area. Specifically, the drone takes images with a high-resolution camera, and the environmental sensors collect data such as seismic intensity, temperature, humidity, and gas concentration. The victim's smartphone also uses GPS to obtain location information.

[1602] Inputs include image data from high-resolution cameras, environmental data from environmental sensors, and smartphone location data.

[1603] As an output, these collected data sets are produced.

[1604] Step 2: Data transmission and storage

[1605] The terminal transmits the collected data to a central server via wireless communication or 4G / 5G networks. Specifically, drone image data is transmitted as a large file, environmental sensor data is transmitted using the MQTT protocol, and smartphone location data is also transmitted via the Internet.

[1606] The input is all the data collected in step 1.

[1607] The output is a data store stored on a central server.

[1608] Step 3: Data integration and analysis

[1609] The server consolidates all the data it receives. Specifically, it stores high-resolution image data in a database, and also centrally manages environmental sensor data and smartphone location information. It applies image processing algorithms to extract and analyze information such as the extent of building damage. It processes the sensor data in real time to generate an environmental map of the entire disaster area.

[1610] The input is the data store saved in step 2.

[1611] The output is a consolidated and analyzed dataset that provides a situation map of the affected area.

[1612] Step 4: Predict missing persons

[1613] The server uses an AI algorithm to predict the probability of missing people based on the integrated data set. Specifically, it uses deep learning models (e.g., CNN, LSTM) to analyze past data and the current situation in the disaster area and identify locations where missing people are likely to be.

[1614] The input is the analyzed dataset generated in step 3.

[1615] The output is a prediction result that lists the probability of the missing person being present.

[1616] Step 5: Emotion Engine in Action

[1617] The device and server analyze the user's voice and facial expressions in real time and send emotional data to a central server, which then aggregates the emotional data and analyzes the user's stress and fatigue levels. Priorities are adjusted based on this information.

[1618] Inputs include voice and facial expression data obtained from the user.

[1619] As an output, the analyzed emotional data is sent to a central server, which adjusts the priorities of search operations.

[1620] Step 6: View and notify results

[1621] The server notifies search and rescue teams of the predicted probability of a missing person's presence. The results are sent via a dedicated app, SMS, or email. Furthermore, the prediction results and emotion data are visually displayed on a map to assist in the planning and coordination of search plans.

[1622] The inputs are the prediction results obtained in step 4 and the emotion data collected in step 5.

[1623] The output is a visualized map and a notification.

[1624] (Application example 2)

[1625] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1626] Efficiently and quickly searching for missing persons during natural disasters is difficult and requires accurate data collection and analysis across a wide area of ​​the disaster area. Furthermore, failure to consider the emotional state of the search team can reduce search efficiency and increase the risk of stress and fatigue. Therefore, a system is needed that enables fast and efficient search operations and enables the early discovery and rescue of missing persons.

[1627] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means including an autonomously driving vehicle that autonomously collects data on the disaster area, means for collecting images and sensor data using drones and environmental sensors, means for transmitting the collected data to a central server, means for integrating and analyzing the data transmitted to the central server, means for predicting the probability of the presence of missing persons based on the integrated data, means for displaying the prediction results on a map and notifying search teams and rescue teams, means including an emotion engine that collects and analyzes emotion data from the search team, and means for adjusting the search plan based on the emotion data. This enables fast and efficient search activities and realizes the early discovery and rescue of missing persons.

[1628] An "autonomous vehicle" refers to a vehicle that can drive autonomously without a driver and travel to a specific destination.

[1629] A "drone" refers to an unmanned aircraft that flies and collects images and sensor data.

[1630] "Environmental sensor" refers to a sensor device for detecting environmental data such as seismic intensity, temperature, humidity, and gas concentration.

[1631] "Central server" refers to a central management system for integrating and analyzing data collected from disaster-stricken areas.

[1632] "Data integration" refers to the process of centrally organizing data collected from multiple sources and making it ready for analysis.

[1633] "Probability of presence" refers to the probability that a missing person may be present in a particular location.

[1634] The "emotion engine" refers to a system that analyzes the voices and facial expressions of the search team and recognizes their emotional state.

[1635] "Emotion data" refers to information collected by the emotion engine that indicates the user's emotional state, such as stress or fatigue.

[1636] "Search plan" refers to a plan drawn up to efficiently search for a missing person.

[1637] A "generative AI model" refers to an artificial intelligence algorithm that learns from large amounts of data and predicts the probability of a missing person's existence, etc.

[1638] The present invention relates to a system for efficiently and quickly searching for missing people during natural disasters. This system is composed of a combination of autonomous vehicles, drones, environmental sensors, a central server, an emotion engine, and a generative AI model. Each part of the present invention is described in detail below.

[1639] Data collection

[1640] The server collects data using autonomous, unmanned vehicles that travel across a wide area of ​​the disaster-stricken area. The autonomous vehicles are equipped with drones and environmental sensors, and take images using high-resolution cameras. Environmental data such as seismic intensity, temperature, humidity, and gas concentration are also collected at the same time. Drones installed in various locations in the disaster-stricken area collect images from an aerial perspective, allowing the entire situation in the disaster-stricken area to be grasped.

[1641] Data transmission

[1642] The devices transmit the collected data to a central server in real time via wireless communication or 4G / 5G networks. Image data is sent as large files, and sensor data is transmitted using IoT protocols (e.g., MQTT). Location information from the victim's smartphone is also sent to the server via GPS and the Internet.

[1643] Data Integration and Analysis

[1644] The server integrates and analyzes all the received data. Using a generative AI model trained using TensorFlow and Keras, it uses image processing algorithms to identify the extent of damage to buildings. Sensor data is also processed in real time, and the results are used to generate an environmental map of the entire disaster area. Furthermore, it analyzes the location data of victims and plots their distribution on the map.

[1645] Missing Persons Prediction

[1646] Based on the analyzed data, the server uses a generative AI model to predict the probability of missing persons being present. It references past data and the current situation in the disaster area to identify locations in specific areas or buildings where missing persons are likely to be present. The prediction results are then listed and saved.

[1647] Emotion Engine Operation

[1648] The server uses an emotion engine to analyze the voices and facial expressions of users participating in the search operation to recognize their emotions. Emotional data is sent to a central server in real time and analyzed there. For example, if a user's stress level is high, that information can be used to adjust the priority of the search operation.

[1649] Displaying and notifying results

[1650] The server notifies search and rescue teams of the predicted missing persons, visualizes the results, and displays them on a map. It also displays emotional data collected by the emotion engine, helping them plan and coordinate their search.

[1651] Specific examples

[1652] For example, in a disaster area where a magnitude 7 earthquake has occurred, autonomous vehicles begin collecting data. High-resolution image data captured by the autonomous vehicles, aerial video data collected by drones, and seismic intensity information from environmental sensors are transmitted in real time to a central server. The server integrates this data, and a generative AI model predicts the probability of missing persons being present. As a result, specific areas are predicted to have a high probability of missing persons, and the search team is notified. Furthermore, the emotional state of the search team is monitored, and the search plan is adjusted if stress levels are high.

[1653] Here are some example prompts:

[1654] “In the event of a natural disaster, predict the probability of missing persons in the affected area using high-resolution images captured by drones and environmental sensor data. Consider variables such as building damage, temperature, humidity, and gas concentrations.”

[1655] As a result, the present invention enables fast and efficient search operations, contributing to the early discovery and rescue of missing persons.

[1656] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1657] Step 1:

[1658] The device uses autonomous vehicles and drones to collect data from disaster-stricken areas. The autonomous vehicles drive autonomously through the disaster-stricken areas and take images with high-resolution cameras. The drones collect aerial image data, while environmental sensors collect real-time data such as seismic intensity, temperature, humidity, and gas concentration. This allows for detailed environmental information to be obtained.

[1659] Input: On-site environmental and image data from the affected areas

[1660] Output: High-resolution image data, seismic intensity data, temperature data, humidity data, gas concentration data

[1661] Step 2:

[1662] The devices transmit the collected data to a central server via wireless communication or 4G / 5G networks. Image data is sent as large files, and sensor data is transmitted using IoT protocols. Location information collected by victims' smartphones is also transmitted via GPS and the internet.

[1663] Input: High-resolution image data, seismic intensity data, temperature data, humidity data, gas concentration data, location information of victims

[1664] Output: Sending data to a central server

[1665] Step 3:

[1666] The server integrates all the received data. Specifically, it applies image processing algorithms to the image data using TensorFlow and Keras to identify the extent of damage to buildings. It also processes the sensor data in real time and generates an environmental map of the entire disaster area based on the results. The location data of the victims is analyzed and the distribution of victims is plotted on the map.

[1667] Input: All data sent to the central server (high-resolution image data, sensor data, location information of victims)

[1668] Output: Image data analysis results, environmental map, disaster victim distribution map

[1669] Step 4:

[1670] The server uses a generative AI model to predict the probability of missing persons being present. It references past data and the current situation in the disaster area to identify locations in specific areas or buildings where missing persons are likely to be present. The prediction results are listed and saved.

[1671] Input: Integrated analysis data (image data analysis results, environmental map, disaster victim distribution map)

[1672] Output: List of predicted probability of missing person existence

[1673] Step 5:

[1674] The server uses an emotion engine to analyze the voices and facial expressions of users participating in the search operation to recognize their emotions. Emotional data is sent to the server in real time for analysis. For example, if a user's stress level is high, the server can adjust the priority of the search operation based on that information.

[1675] Input: User's voice data and facial expression data

[1676] Output: Parsed emotion data

[1677] Step 6:

[1678] The server notifies search and rescue teams of predicted missing persons and displays the results on a map. It also displays emotion data collected by the emotion engine, helping to plan and coordinate the search.

[1679] Input: List of predicted probability of missing persons, analyzed emotion data

[1680] Output: Map predictions, notifications to search and rescue teams, coordinated search plans

[1681] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1682] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1683] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1684] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1685] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1686] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1687] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1688] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1689] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1690] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1691] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1692] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1693] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1694] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1695] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1696] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1697] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1698] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1699] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1700] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1701] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1702] The following is further disclosed regarding the above embodiment.

[1703] (Claim 1)

[1704] a means of collecting imagery and sensor data from the affected area;

[1705] means for transmitting the collected data to a central server;

[1706] means for aggregating the data transmitted to a central server;

[1707] a means for predicting the probability of the presence of a missing person based on the integrated data;

[1708] A means for displaying and notifying the prediction results;

[1709] A system including:

[1710] (Claim 2)

[1711] 10. The system of claim 1, wherein the data collection means comprises a plurality of unmanned aerial vehicles and environmental sensors.

[1712] (Claim 3)

[1713] The system of claim 1 , wherein the predicting means predicts the probability of a missing person using a machine learning algorithm.

[1714] "Example 1"

[1715] (Claim 1)

[1716] a means of collecting imagery and environmental data of the affected area;

[1717] means for transmitting the collected data to a central server;

[1718] means for storing the transmitted data on a central server;

[1719] a means for integrating the stored data;

[1720] a means for analyzing the integrated data;

[1721] a means for predicting the probability of the presence of a missing person based on the analyzed data;

[1722] A means for displaying and notifying the prediction results;

[1723] A system including:

[1724] (Claim 2)

[1725] 2. The system according to claim 1, wherein the data collection means comprises a plurality of autonomous moving bodies and an environmental information acquisition device.

[1726] (Claim 3)

[1727] 2. The system of claim 1, wherein the analysis means uses a data processing algorithm to assess the damage status of the building.

[1728] "Application Example 1"

[1729] (Claim 1)

[1730] a means of collecting imagery and sensor data from the affected area;

[1731] means for transmitting the collected data to a central server;

[1732] means for aggregating the data transmitted to a central server;

[1733] a means for predicting the probability of the presence of a missing person based on the integrated data;

[1734] A means for displaying and notifying the prediction results;

[1735] using data collected by cameras and sensors on board the autonomous vehicle;

[1736] A means of using AI models to perform real-time analysis and predictions;

[1737] A means for displaying and notifying prediction results on a smartphone app;

[1738] A system including:

[1739] (Claim 2)

[1740] 10. The system of claim 1, wherein the data collection means comprises a plurality of unmanned aerial vehicles, autonomous vehicles, and environmental sensors.

[1741] (Claim 3)

[1742] 2. The system of claim 1, wherein the predicting means predicts the probability of a missing person using a deep learning algorithm.

[1743] "Example 2: Combining Emotion Engines"

[1744] (Claim 1)

[1745] a means of collecting imagery and sensor data from the affected area;

[1746] means for transmitting the collected data to a central server;

[1747] means for aggregating the data transmitted to a central server;

[1748] a means for predicting the probability of the presence of a missing person based on the integrated data;

[1749] A means for displaying and notifying the prediction results;

[1750] a means for recognizing user emotions and adjusting search activity priorities;

[1751] A system including:

[1752] (Claim 2)

[1753] 10. The system of claim 1, wherein the data collection means comprises a plurality of autonomous flying devices and environmental sensors.

[1754] (Claim 3)

[1755] The system of claim 1, wherein the predicting means uses an artificial intelligence algorithm to predict the probability of a missing person being present.

[1756] "Application example 2 when combining emotion engines"

[1757] (Claim 1)

[1758] a means including an autonomous vehicle for autonomously collecting data from the affected area;

[1759] means for collecting imagery and sensor data using drones and environmental sensors;

[1760] means for transmitting the collected data to a central server;

[1761] means for aggregating and analyzing the data transmitted to the central server;

[1762] a means for predicting the probability of the presence of a missing person based on the integrated data;

[1763] A means of displaying the prediction results on a map and notifying search and rescue teams;

[1764] means including an emotion engine for collecting and analyzing emotion data of the search team;

[1765] means for adjusting a search plan based on the emotion data;

[1766] A system including:

[1767] (Claim 2)

[1768] 10. The system of claim 1, wherein the data collection means comprises a plurality of autonomous vehicles, unmanned aerial vehicles, and environmental sensors.

[1769] (Claim 3)

[1770] The system of claim 1, wherein the predicting means predicts the probability of a missing person's presence using a generative AI model. [Explanation of symbols]

[1771] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means of collecting imagery and sensor data from the affected area; means for transmitting the collected data to a central server; means for aggregating the data transmitted to a central server; a means for predicting the probability of the presence of a missing person based on the integrated data; A means for displaying and notifying the prediction results; A system including:

2. The system of claim 1 , wherein the data collection means comprises a plurality of unmanned vehicles and environmental sensors.

3. The system of claim 1 , wherein the predicting means predicts the probability of a missing person using a machine learning algorithm.

Images