system

Abstract

The system according to the embodiment aims to quickly and accurately grasp the damage situation when a disaster occurs. [Solution] A system according to an embodiment includes a launching unit, a data collection unit, an analysis unit, and a visualization unit. The launching unit launches a drone. The data collection unit processes data collected by photographing the damage situation using a camera mounted on the drone launched by the launching unit. The analysis unit uses a generation AI to analyze the data collected by the data collection unit. The visualization unit uses a generation AI to visualize the damage situation as a 3D map using the data analyzed by the analysis unit.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technology has had the problem of making it difficult to quickly and accurately grasp the extent of damage when a disaster occurs.

[0005] The system according to the embodiment aims to quickly and accurately grasp the damage situation when a disaster occurs. [Means for solving the problem]

[0006] A system according to an embodiment includes a launching unit, a data collection unit, an analysis unit, and a visualization unit. The launching unit launches a drone. The data collection unit processes data collected by the drone launched by the launching unit. The analysis unit analyzes the data collected by the data collection unit. The visualization unit visualizes the data analyzed by the analysis unit. [Effects of the Invention]

[0007] The system according to the embodiment can quickly and accurately grasp the damage situation when a disaster occurs. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) A disaster damage assessment system according to an embodiment of the present invention is a system for quickly assessing the damage situation when a disaster occurs. This system simultaneously launches multiple drones within one hour of the occurrence of a disaster, and the drones, equipped with cameras, map the damage situation. The massive amount of damage situation data collected by the drones is then analyzed by a generation AI and visualized as a 3D map. The generation AI processes this data at an incredible speed, allowing for a real-time assessment of the damage situation at the site. For example, when a disaster occurs, the system automatically launches multiple drones. These drones are equipped with cameras and capture the damage situation. For example, they capture detailed images of collapsed buildings and damaged roads. The damage situation data collected by the drones is then transmitted to the generation AI. The generation AI analyzes this data and visualizes it as a 3D map. For example, the collapsed buildings can be displayed on a 3D map, allowing for a quick understanding of which buildings have sustained damage and to what extent. Furthermore, the generation AI processes this data at an incredible speed. This allows for a real-time assessment of the damage situation at the site. For example, within one hour of the occurrence of a disaster, a detailed assessment of the damage situation can be achieved, enabling a rapid response. This allows the disaster damage confirmation system to quickly grasp the extent of damage when a disaster occurs, improving the efficiency of rescue and recovery efforts. For example, by prioritizing rescue efforts in areas with the most damage, damage can be minimized. In addition, recovery work plans can be made quickly, which can lead to early recovery.

[0029] A disaster damage confirmation system according to an embodiment includes a launching unit, a data collection unit, an analysis unit, and a visualization unit. The launching unit launches drones. For example, the launching unit simultaneously launches multiple drones within one hour of a disaster occurrence. The launching unit can launch the drones manually, remotely, or by timer activation. The data collection unit processes data collected by the drones launched by the launching unit. For example, the data collection unit photographs the damage situation using a drone equipped with a camera. Specific specifications and performance of the camera include resolution, field of view, and zoom function. The analysis unit analyzes the data collected by the data collection unit. For example, the analysis unit analyzes the damage situation data using a generation AI. The generation AI analyzes the data using a specific algorithm and training dataset. The visualization unit visualizes the data analyzed by the analysis unit. For example, the visualization unit visualizes the damage situation as a 3D map using the generation AI. The method for generating and displaying the 3D map includes the software and data format used. This allows the disaster damage confirmation system according to an embodiment to quickly confirm the damage situation when a disaster occurs and grasp the damage situation at the site in real time.

[0030] The launching unit can launch multiple drones simultaneously within one hour after a disaster occurs. The specific time measurement method and criteria for within one hour after a disaster occurs include the definition of a disaster occurrence and the starting point for time measurement. For example, after receiving a signal indicating a disaster has occurred, the launching unit starts a timer and launches multiple drones simultaneously within one hour. The launching unit can also automatically select the number and type of drones to launch depending on the type and scale of the disaster. This allows drones to be launched quickly after a disaster occurs and the damage situation to be assessed.

[0031] The data collection unit can capture images of the damage using a drone equipped with a camera. Specific specifications and performance of a drone equipped with a camera include resolution, field of view, zoom function, etc. For example, the data collection unit uses a high-resolution camera to capture detailed images of collapsed buildings. The data collection unit can also use a wide-angle camera to check the damage over a wide area. Furthermore, the data collection unit can use the zoom function to check specific damaged areas in detail. This allows the drone's camera to capture detailed images of the damage.

[0032] The analysis unit can analyze the damage situation data using the generative AI. Specific types and implementation methods of the generative AI include specific algorithms and training data sets. For example, the analysis unit can analyze the damage situation data using an image analysis algorithm. The analysis unit can also analyze the damage situation data using data mining technology. Furthermore, the analysis unit can analyze the damage situation data using a machine learning algorithm. As a result, the use of the generative AI improves the accuracy of the analysis of the damage situation data.

[0033] The visualization unit can use the generation AI to visualize the damage situation as a 3D map. The specific generation method and display format of the 3D map include the software and data format used. For example, the visualization unit can use the generation AI to display the collapsed building situation on a 3D map. The visualization unit can also use the generation AI to display the damaged road situation on a 3D map. Furthermore, the visualization unit can also use the generation AI to display information related to human life on a 3D map. In this way, the damage situation can be visualized as a 3D map by using the generation AI.

[0034] The analysis unit can quickly process the damage situation data. Specific standards and methods for rapid processing include the estimated processing time and the performance of the hardware used. For example, the analysis unit can quickly process the damage situation data using a high-performance processor. The analysis unit can also quickly process the damage situation data using parallel processing technology. Furthermore, the analysis unit can quickly process the damage situation data using cloud computing technology. This allows the damage situation data to be quickly processed and the situation on site to be grasped in real time.

[0035] The visualization unit can grasp the damage situation at the site in real time. Specific definitions and standards of real time include the frequency of data updates and the acceptable range of delay time. For example, the visualization unit can grasp the damage situation in real time by increasing the frequency of data updates. The visualization unit can also grasp the damage situation in real time by minimizing delay time. Furthermore, the visualization unit can process data in real time and immediately display the damage situation at the site. This allows the damage situation at the site to be grasped in real time.

[0036] The launching unit can automatically select the number and type of drones to launch depending on the type and scale of the disaster. Specific classification criteria and evaluation methods for the type and scale of disasters include classifications such as earthquakes, floods, and fires, as well as evaluation criteria for the extent of damage. For example, in the case of an earthquake, the launching unit will prioritize launching drones equipped with high-resolution cameras to check the collapse of buildings. In the case of a flood, the launching unit can also launch additional underwater drones to measure water levels. Furthermore, in the case of a fire, the launching unit can launch drones equipped with thermal sensors to check the spread of the fire. This allows the optimal drone to be selected depending on the type and scale of the disaster.

[0037] The launch unit can set an optimal launch schedule taking into account the drone's remaining battery power and remaining flight time. Specific methods and standards for measuring remaining battery power and remaining flight time include the percentage of remaining battery power and the amount of available flight time. For example, the launch unit may use a drone with a low remaining battery power to check damage over short distances. The launch unit may also use a drone with a long remaining flight time to check damage over a wide area. Furthermore, the launch unit may set a schedule to allow drones with easy battery replacement to be used continuously. This allows for efficient operation by taking into account the drone's remaining battery power and remaining flight time.

[0038] When launching the drone, the launch unit can monitor environmental conditions such as weather and wind speed in real time and set the optimal flight route. Specific measurement methods and standards for environmental conditions such as weather and wind speed include the sensors used and the frequency of data updates. For example, the launch unit can set a stable flight route taking wind speed into account when there is strong wind. The launch unit can also prioritize launching waterproof drones when it is raining. Furthermore, when the temperature is low, the launch unit can set a short flight route to reduce battery consumption. This allows the optimal flight route to be set taking environmental conditions into account.

[0039] When launching the drone, the launch unit can select priority areas for flight, taking into account geographical damage prediction data. Specific methods and standards for acquiring geographical damage prediction data include the data source and prediction algorithms used. For example, the launch unit can prioritize flights over areas predicted to suffer heavy damage. The launch unit can also prioritize flights over areas with important infrastructure (hospitals, fire stations, etc.). Furthermore, the launch unit can prioritize flights over areas with high population densities. This allows the launch unit to select priority areas for flight, taking into account geographical damage prediction data.

[0040] When launching a drone, the launching unit can set the optimal launch timing, taking into account coordination with other disaster response systems. Specific methods and standards for coordination with other disaster response systems include data sharing protocols and the types of systems to be coordinated. For example, the launching unit can coordinate the drone launch timing in coordination with the activities of the fire department and police. The launching unit can also coordinate the drone launch timing in coordination with the activities of emergency medical teams. Furthermore, the launching unit can coordinate the drone launch timing in coordination with the local government's disaster response headquarters. This allows the optimal launch timing to be set, taking into account coordination with other disaster response systems.

[0041] When the drone is launched, the launch unit can refer to past disaster data to learn and apply the optimal flight pattern. Specific methods and standards for acquiring past disaster data include the data source and type of data. For example, the launch unit can prioritize flying over areas where building collapse is predicted based on past earthquake data. The launch unit can also prioritize flying over areas where water levels are likely to rise based on past flood data. Furthermore, the launch unit can prioritize flying over areas where fires are likely to spread based on past fire data. This allows the drone to learn and apply the optimal flight pattern by referring to past disaster data.

[0042] The data collection unit can automatically adjust the resolution and shooting angle of the drone's camera according to the damage situation. Specific adjustment methods and standards for the camera's resolution and shooting angle include the numerical resolution and shooting angle range. For example, the data collection unit takes high-resolution images to check the collapsed state of a building in detail. The data collection unit can also take wide-angle images to check the damage situation over a wide area. Furthermore, the data collection unit can use the zoom function to check specific damaged areas in detail. This allows the camera's resolution and shooting angle to be automatically adjusted according to the damage situation.

[0043] The data collection unit can filter data collected by the drone's camera in real time to extract only important information. Specific filtering methods and criteria include filtering algorithms and definitions of important information. For example, the data collection unit can prioritize extracting information about collapsed buildings. The data collection unit can also prioritize extracting information about damaged roads. Furthermore, the data collection unit can prioritize extracting information related to human life. This allows only important information to be extracted in real time.

[0044] The data collection unit can integrate data collected by the drone's camera with other sensors (temperature, humidity, gas concentration, etc.) for multifaceted analysis. Specific types and uses of other sensors include temperature sensors, humidity sensors, and gas concentration sensors. For example, the data collection unit can integrate data from a temperature sensor to confirm the spread of a fire. The data collection unit can also integrate data from a humidity sensor to confirm the impact of flooding. Furthermore, the data collection unit can integrate data from a gas concentration sensor to confirm the generation status of harmful gases. This allows for multifaceted analysis by integrating data with other sensors.

[0045] The data collection unit can link the data collected by the drone's camera with a geographic information system (GIS) to display it on a map in real time. The specific type and usage of a geographic information system (GIS) includes the software and data format used. For example, the data collection unit can display the damage situation on a map in real time to confirm the extent of the damage. The data collection unit can also display the damage situation on a map in real time to confirm the severity of the damage. Furthermore, the data collection unit can display the damage situation on a map in real time to determine the priority of rescue operations. This allows the damage situation to be displayed on a map in real time.

[0046] The data collection unit can automatically upload data collected by the drone's camera to cloud storage and share it with other disaster response teams. The specific types and usage of cloud storage include the cloud service used and the method of uploading data. For example, the data collection unit uploads damage situation data to cloud storage and shares it with other disaster response teams. The data collection unit can also upload damage situation data to cloud storage and share it in real time. Furthermore, the data collection unit can upload damage situation data to cloud storage and share it quickly. This allows data to be uploaded to cloud storage and shared with other disaster response teams.

[0047] The data collection unit uses AI to perform initial analysis of the data collected by the drone's camera, allowing it to quickly extract important information. The specific methods and standards for initial analysis by AI include the algorithms used and the data to be analyzed. For example, the data collection unit uses AI to quickly extract the state of collapsed buildings. The data collection unit can also use AI to quickly extract the state of damage to roads. Furthermore, the data collection unit can also use AI to quickly extract information related to human life. This allows important information to be extracted quickly through initial analysis by AI.

[0048] During analysis, the analysis unit can evaluate the reliability of the damage situation data and exclude unreliable data. Specific evaluation criteria and methods for reliability include the origin of the data and the consistency of the data. For example, the analysis unit excludes data if the source of data collection is unclear. The analysis unit can also exclude data if the time of data collection is unclear. Furthermore, the analysis unit can also exclude data if the method of data collection is unclear. This eliminates unreliable data, improving the accuracy of the analysis results.

[0049] During analysis, the analysis unit can analyze the damage situation data in chronological order to predict the progress of the damage. Specific methods and standards for time series analysis include the time interval and the algorithm used. For example, the analysis unit can analyze the damage situation data in chronological order to predict the progress of the damage. The analysis unit can also analyze the damage situation data in chronological order to predict the expansion of the damage. Furthermore, the analysis unit can analyze the damage situation data in chronological order to predict the convergence of the damage. This makes it possible to analyze the damage situation data in chronological order to predict the progress of the damage.

[0050] During analysis, the analysis unit can integrate the damage situation data with other disaster data (seismic waves, meteorological data, etc.) for a comprehensive analysis. Specific types and acquisition methods of other disaster data include seismic wave data and meteorological data. For example, the analysis unit can integrate the data with seismic wave data to comprehensively analyze the state of building collapse. The analysis unit can also integrate the data with meteorological data to comprehensively analyze the impact of flooding. Furthermore, the analysis unit can integrate the data with other disaster data to comprehensively analyze the overall picture of the damage. This allows for integration with other disaster data for a comprehensive analysis.

[0051] During analysis, the analysis unit can cluster the damage situation data and classify it according to the type and extent of damage. Specific clustering methods and criteria include the algorithm to be used and the number of clusters. For example, the analysis unit can cluster the collapsed building status and classify it according to the extent of damage. The analysis unit can also cluster the damaged road status and classify it according to the extent of damage. Furthermore, the analysis unit can cluster information related to human life and classify it according to the extent of damage. This allows the damage situation data to be clustered and classified according to the type and extent of damage.

[0052] During analysis, the analysis unit can link damage status data with other disaster response systems to conduct comprehensive damage assessments. Specific methods and standards for linking with other disaster response systems include data sharing protocols and the types of systems to link with. For example, the analysis unit can link with data from the fire department and police to conduct comprehensive damage assessments. The analysis unit can also link with data from emergency medical teams to conduct comprehensive damage assessments. Furthermore, the analysis unit can link with data from the local government's disaster response headquarters to conduct comprehensive damage assessments. This allows for comprehensive damage assessments to be conducted in collaboration with other disaster response systems.

[0053] During analysis, the analysis unit can compare the damage situation data with past disaster data to identify similar damage patterns. Specific methods and standards for acquiring past disaster data include the data source and data type. For example, the analysis unit can compare the data with past earthquake data to identify similar damage patterns. The analysis unit can also compare the data with past flood data to identify similar damage patterns. Furthermore, the analysis unit can compare the data with past fire data to identify similar damage patterns. This makes it possible to identify similar damage patterns by comparing with past disaster data.

[0054] During visualization, the visualization unit can display the damage situation data on a 3D map and color-code the extent of the damage. The specific generation method and display format of the 3D map include the software and data format used. For example, the visualization unit can display the collapsed building status on a 3D map and color-code the extent of the damage. The visualization unit can also display the damaged road status on a 3D map and color-code the extent of the damage. Furthermore, the visualization unit can display information related to human life on a 3D map and color-code the extent of the damage. This allows the damage situation data to be displayed color-coded on a 3D map.

[0055] The visualization unit allows interactive manipulation of the damage situation data during visualization, and can display detailed information. Specific methods and criteria for interactive manipulation include the interface used and the range of manipulation. For example, when a specific building is clicked on the 3D map, the visualization unit can display detailed damage to that building. The visualization unit can also display detailed damage to a specific road when that road is clicked on the 3D map. Furthermore, the visualization unit can display detailed damage to a specific area when that area is clicked on the 3D map. This allows interactive manipulation of the damage situation data and displays detailed information.

[0056] During visualization, the visualization unit can display the damage situation data by overlaying it with other map data (roads, buildings, etc.). Specific types and acquisition methods of the other map data include road data and building data. For example, the visualization unit can display the damage situation data by overlaying it with a road map to confirm the damage status of roads. The visualization unit can also display the damage situation data by overlaying it with a building map to confirm the collapsed status of buildings. Furthermore, the visualization unit can display the damage situation data by overlaying it with a topographical map to confirm changes in the topography. This allows the damage situation data to be displayed by overlaying it with other map data.

[0057] During visualization, the visualization unit can display the damage situation data on a time axis and display the progress of the damage using animation. Specific methods and standards for displaying the time axis include time intervals and display formats. For example, the visualization unit can display the damage situation data on a time axis and display the progress of the damage using animation. The visualization unit can also display the damage situation data on a time axis and display the expansion of the damage using animation. Furthermore, the visualization unit can display the damage situation data on a time axis and display the convergence of the damage using animation. This makes it possible to display the damage situation data on a time axis and display the progress of the damage using animation.

[0058] When visualizing, the Visualization Department can share the damage situation data with other disaster response teams to jointly consider response measures. Specific methods and standards for sharing with other disaster response teams include the data sharing protocol and the platform used. For example, the Visualization Department can share the damage situation data with other disaster response teams to jointly consider response measures. The Visualization Department can also share the damage situation data with other disaster response teams to quickly consider response measures. Furthermore, the Visualization Department can also share the damage situation data with other disaster response teams to consider effective response measures. This allows the damage situation data to be shared with other disaster response teams to jointly consider response measures.

[0059] The visualization unit can make the damage situation data visible on a mobile device such as a smartphone or tablet when visualizing the data. Specific types and uses of the mobile device include a smartphone or a tablet. For example, the visualization unit can display the damage situation data on a smartphone to facilitate on-site confirmation. The visualization unit can also display the damage situation data on a tablet to confirm detailed information. Furthermore, the visualization unit can display the damage situation data on a mobile device to enable a rapid response. This allows the damage situation data to be displayed on a mobile device to facilitate on-site confirmation.

[0060] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0061] When a disaster occurs, the launching unit can check the status of the surrounding communication infrastructure and prioritize launching drones in areas where communication is ensured. For example, in an area where communication infrastructure is damaged, the drone can be flown from an area where communication is ensured to check the damage situation. Also, once the communication infrastructure is restored, the drone can immediately resume operation in that area. Furthermore, the drone's flight route can be optimized according to the status of the communication infrastructure, allowing for efficient confirmation of the damage situation. This allows the drone's launch and flight route to be optimized taking into account the status of the communication infrastructure.

[0062] The data collection unit can compare the data collected by the drone with data from before the disaster and automatically extract the difference in damage. For example, it can compare the data with the condition of a building before the disaster to identify collapsed parts. It can also compare the data with the condition of a road before the disaster to identify damaged parts. It can also compare the data with topographical data from before the disaster to identify changes in the topography. This makes it possible to compare data before and after a disaster and automatically extract the difference in damage.

[0063] When analyzing damage situation data, the analysis unit can prioritize the analysis according to the severity of the damage. For example, it can prioritize analysis of areas where buildings have collapsed severely. It can also prioritize analysis of areas where roads have been severely damaged. It can also prioritize analysis of areas that contain information related to human life. This allows analysis to be prioritized according to the severity of the damage, enabling a rapid response.

[0064] When visualizing the damage situation data, the visualization unit can take into account the user's location information and prioritize displaying the nearest damage situation. For example, if the user is at the site, the damage situation in the surrounding area can be prioritized. Also, if the user is in a remote location, the entire damage situation can be displayed at a bird's-eye view. Furthermore, the displayed damage situation can be updated in real time according to the user's movement. This allows the nearest damage situation to be prioritized and displayed taking into account the user's location information.

[0065] When analyzing damage situation data, the analysis unit can incorporate data from other disaster response systems in real time to perform comprehensive damage assessments. For example, it can incorporate data from the fire department and police to comprehensively assess the extent of building collapses. It can also incorporate data from emergency medical teams to comprehensively assess the status of injured people. It can also incorporate data from local government disaster response headquarters to comprehensively assess the overall damage picture. This allows data from other disaster response systems to be incorporated in real time to perform comprehensive damage assessments.

[0066] The processing flow of the first embodiment will be briefly explained below.

[0067] Step 1: The launch unit launches the drones. For example, the launch unit can launch multiple drones simultaneously within one hour of a disaster occurring. The launch unit can also launch drones manually, remotely, or by timer. Step 2: The data collection unit processes the data collected by the drone activated by the activation unit. For example, the data collection unit may use a drone equipped with a camera to capture images of the damage. The specific specifications and performance of the camera include resolution, field of view, zoom function, etc. Step 3: The analysis unit analyzes the data collected by the data collection unit. For example, the analysis unit analyzes the damage situation data using the generation AI. The generation AI analyzes the data using specific algorithms and training data sets. Step 4: The visualization unit visualizes the data analyzed by the analysis unit. For example, the visualization unit uses generation AI to visualize the damage situation as a 3D map. The method for generating the 3D map and its display format include the software and data format used.

[0068] (Example 2) A disaster damage assessment system according to an embodiment of the present invention is a system for quickly assessing the damage situation when a disaster occurs. This system simultaneously launches multiple drones within one hour of the occurrence of a disaster, and the drones, equipped with cameras, map the damage situation. The massive amount of damage situation data collected by the drones is then analyzed by a generation AI and visualized as a 3D map. The generation AI processes this data at an incredible speed, allowing for a real-time assessment of the damage situation at the site. For example, when a disaster occurs, the system automatically launches multiple drones. These drones are equipped with cameras and capture the damage situation. For example, they capture detailed images of collapsed buildings and damaged roads. The damage situation data collected by the drones is then transmitted to the generation AI. The generation AI analyzes this data and visualizes it as a 3D map. For example, the collapsed buildings can be displayed on a 3D map, allowing for a quick understanding of which buildings have sustained damage and to what extent. Furthermore, the generation AI processes this data at an incredible speed. This allows for a real-time assessment of the damage situation at the site. For example, within one hour of the occurrence of a disaster, a detailed assessment of the damage situation can be achieved, enabling a rapid response. This allows the disaster damage confirmation system to quickly grasp the extent of damage when a disaster occurs, improving the efficiency of rescue and recovery efforts. For example, by prioritizing rescue efforts in areas with the most damage, damage can be minimized. In addition, recovery work plans can be made quickly, which can lead to early recovery.

[0069] A disaster damage confirmation system according to an embodiment includes a launching unit, a data collection unit, an analysis unit, and a visualization unit. The launching unit launches drones. For example, the launching unit simultaneously launches multiple drones within one hour of a disaster occurrence. The launching unit can launch the drones manually, remotely, or by timer activation. The data collection unit processes data collected by the drones launched by the launching unit. For example, the data collection unit photographs the damage situation using a drone equipped with a camera. Specific specifications and performance of the camera include resolution, field of view, and zoom function. The analysis unit analyzes the data collected by the data collection unit. For example, the analysis unit analyzes the damage situation data using a generation AI. The generation AI analyzes the data using a specific algorithm and training dataset. The visualization unit visualizes the data analyzed by the analysis unit. For example, the visualization unit visualizes the damage situation as a 3D map using the generation AI. The method for generating and displaying the 3D map includes the software and data format used. This allows the disaster damage confirmation system according to an embodiment to quickly confirm the damage situation when a disaster occurs and grasp the damage situation at the site in real time.

[0070] The launching unit can launch multiple drones simultaneously within one hour after a disaster occurs. The specific time measurement method and criteria for within one hour after a disaster occurs include the definition of a disaster occurrence and the starting point for time measurement. For example, after receiving a signal indicating a disaster has occurred, the launching unit starts a timer and launches multiple drones simultaneously within one hour. The launching unit can also automatically select the number and type of drones to launch depending on the type and scale of the disaster. This allows drones to be launched quickly after a disaster occurs and the damage situation to be assessed.

[0071] The data collection unit can capture images of the damage using a drone equipped with a camera. Specific specifications and performance of a drone equipped with a camera include resolution, field of view, zoom function, etc. For example, the data collection unit uses a high-resolution camera to capture detailed images of collapsed buildings. The data collection unit can also use a wide-angle camera to check the damage over a wide area. Furthermore, the data collection unit can use the zoom function to check specific damaged areas in detail. This allows the drone's camera to capture detailed images of the damage.

[0072] The analysis unit can analyze the damage situation data using the generative AI. Specific types and implementation methods of the generative AI include specific algorithms and training data sets. For example, the analysis unit can analyze the damage situation data using an image analysis algorithm. The analysis unit can also analyze the damage situation data using data mining technology. Furthermore, the analysis unit can analyze the damage situation data using a machine learning algorithm. As a result, the use of the generative AI improves the accuracy of the analysis of the damage situation data.

[0073] The visualization unit can use the generation AI to visualize the damage situation as a 3D map. The specific generation method and display format of the 3D map include the software and data format used. For example, the visualization unit can use the generation AI to display the collapsed building situation on a 3D map. The visualization unit can also use the generation AI to display the damaged road situation on a 3D map. Furthermore, the visualization unit can also use the generation AI to display information related to human life on a 3D map. In this way, the damage situation can be visualized as a 3D map by using the generation AI.

[0074] The analysis unit can quickly process the damage situation data. Specific standards and methods for rapid processing include the estimated processing time and the performance of the hardware used. For example, the analysis unit can quickly process the damage situation data using a high-performance processor. The analysis unit can also quickly process the damage situation data using parallel processing technology. Furthermore, the analysis unit can quickly process the damage situation data using cloud computing technology. This allows the damage situation data to be quickly processed and the situation on site to be grasped in real time.

[0075] The visualization unit can grasp the damage situation at the site in real time. Specific definitions and standards of real time include the frequency of data updates and the acceptable range of delay time. For example, the visualization unit can grasp the damage situation in real time by increasing the frequency of data updates. The visualization unit can also grasp the damage situation in real time by minimizing delay time. Furthermore, the visualization unit can process data in real time and immediately display the damage situation at the site. This allows the damage situation at the site to be grasped in real time.

[0076] The launch unit can estimate the user's emotions and adjust the drone's launch timing based on the estimated user emotions. Specific methods and criteria for estimating the user's emotions include emotion recognition algorithms and sensors. For example, the launch unit can capture the user's facial expressions with a camera and estimate the emotion using an emotion recognition algorithm. The launch unit can also record the user's voice and estimate the emotion using voice analysis technology. Furthermore, the launch unit can collect the user's biometric data (heart rate and electrodermal activity) with a sensor and estimate the emotion using an emotion recognition algorithm. This allows the drone's launch timing to be adjusted according to the user's emotions. For example, if the user is nervous, the drone's launch timing can be advanced to quickly assess the damage situation. If the user is calm, the drone can be launched at the optimal timing to efficiently assess the damage situation. If the user is panicking, the drone's launch can be automated to minimize user operation.

[0077] The launching unit can automatically select the number and type of drones to launch depending on the type and scale of the disaster. Specific classification criteria and evaluation methods for the type and scale of disasters include classifications such as earthquakes, floods, and fires, as well as evaluation criteria for the extent of damage. For example, in the case of an earthquake, the launching unit will prioritize launching drones equipped with high-resolution cameras to check the collapse of buildings. In the case of a flood, the launching unit can also launch additional underwater drones to measure water levels. Furthermore, in the case of a fire, the launching unit can launch drones equipped with thermal sensors to check the spread of the fire. This allows the optimal drone to be selected depending on the type and scale of the disaster.

[0078] The launch unit can set an optimal launch schedule taking into account the drone's remaining battery power and remaining flight time. Specific methods and standards for measuring remaining battery power and remaining flight time include the percentage of remaining battery power and the amount of available flight time. For example, the launch unit may use a drone with a low remaining battery power to check damage over short distances. The launch unit may also use a drone with a long remaining flight time to check damage over a wide area. Furthermore, the launch unit may set a schedule to allow drones with easy battery replacement to be used continuously. This allows for efficient operation by taking into account the drone's remaining battery power and remaining flight time.

[0079] When launching the drone, the launch unit can monitor environmental conditions such as weather and wind speed in real time and set the optimal flight route. Specific measurement methods and standards for environmental conditions such as weather and wind speed include the sensors used and the frequency of data updates. For example, the launch unit can set a stable flight route taking wind speed into account when there is strong wind. The launch unit can also prioritize launching waterproof drones when it is raining. Furthermore, when the temperature is low, the launch unit can set a short flight route to reduce battery consumption. This allows the optimal flight route to be set taking environmental conditions into account.

[0080] The launching unit can estimate the user's emotions and determine the priority of drones to launch based on the estimated user emotions. Specific methods and criteria for estimating the user's emotions include emotion recognition algorithms and sensors. For example, the launching unit can capture the user's facial expressions with a camera and estimate their emotions using an emotion recognition algorithm. The launching unit can also record the user's voice and estimate their emotions using voice analysis technology. Furthermore, the launching unit can collect the user's biometric data (heart rate and electrodermal activity) with sensors and estimate their emotions using an emotion recognition algorithm. This allows the priority of drones to be determined according to the user's emotions. For example, if the user is nervous, the most reliable drone is launched first. If the user is calm, multiple drones are launched simultaneously efficiently. If the user is panicking, all drones are launched simultaneously to quickly assess the damage situation.

[0081] When launching the drone, the launch unit can select priority areas for flight, taking into account geographical damage prediction data. Specific methods and standards for acquiring geographical damage prediction data include the data source and prediction algorithms used. For example, the launch unit can prioritize flights over areas predicted to suffer heavy damage. The launch unit can also prioritize flights over areas with important infrastructure (hospitals, fire stations, etc.). Furthermore, the launch unit can prioritize flights over areas with high population densities. This allows the launch unit to select priority areas for flight, taking into account geographical damage prediction data.

[0082] When launching a drone, the launching unit can set the optimal launch timing, taking into account coordination with other disaster response systems. Specific methods and standards for coordination with other disaster response systems include data sharing protocols and the types of systems to be coordinated. For example, the launching unit can coordinate the drone launch timing in coordination with the activities of the fire department and police. The launching unit can also coordinate the drone launch timing in coordination with the activities of emergency medical teams. Furthermore, the launching unit can coordinate the drone launch timing in coordination with the local government's disaster response headquarters. This allows the optimal launch timing to be set, taking into account coordination with other disaster response systems.

[0083] When the drone is launched, the launch unit can refer to past disaster data to learn and apply the optimal flight pattern. Specific methods and standards for acquiring past disaster data include the data source and type of data. For example, the launch unit can prioritize flying over areas where building collapse is predicted based on past earthquake data. The launch unit can also prioritize flying over areas where water levels are likely to rise based on past flood data. Furthermore, the launch unit can prioritize flying over areas where fires are likely to spread based on past fire data. This allows the drone to learn and apply the optimal flight pattern by referring to past disaster data.

[0084] The data collection unit can estimate the user's emotions and determine the priority of the damage situations to be captured based on the estimated user emotions. Specific methods and criteria for estimating the user's emotions include emotion recognition algorithms and sensors. For example, the data collection unit can capture the user's facial expressions with a camera and estimate the emotion using an emotion recognition algorithm. The data collection unit can also record the user's voice and estimate the emotion using voice analysis technology. Furthermore, the data collection unit can collect the user's biometric data (heart rate and electrodermal activity) with a sensor and estimate the emotion using an emotion recognition algorithm. This allows the priority of the damage situations to be captured based on the user's emotions. For example, if the user is nervous, the most important damage situations can be captured first. If the user is calm, multiple damage situations can be captured efficiently at the same time. If the user is panicked, all damage situations can be captured simultaneously.

[0085] The data collection unit can automatically adjust the resolution and shooting angle of the drone's camera according to the damage situation. Specific adjustment methods and standards for the camera's resolution and shooting angle include the numerical resolution and shooting angle range. For example, the data collection unit takes high-resolution images to check the collapsed state of a building in detail. The data collection unit can also take wide-angle images to check the damage situation over a wide area. Furthermore, the data collection unit can use the zoom function to check specific damaged areas in detail. This allows the camera's resolution and shooting angle to be automatically adjusted according to the damage situation.

[0086] The data collection unit can filter data collected by the drone's camera in real time to extract only important information. Specific filtering methods and criteria include filtering algorithms and definitions of important information. For example, the data collection unit can prioritize extracting information about collapsed buildings. The data collection unit can also prioritize extracting information about damaged roads. Furthermore, the data collection unit can prioritize extracting information related to human life. This allows only important information to be extracted in real time.

[0087] The data collection unit can integrate data collected by the drone's camera with other sensors (temperature, humidity, gas concentration, etc.) for multifaceted analysis. Specific types and uses of other sensors include temperature sensors, humidity sensors, and gas concentration sensors. For example, the data collection unit can integrate data from a temperature sensor to confirm the spread of a fire. The data collection unit can also integrate data from a humidity sensor to confirm the impact of flooding. Furthermore, the data collection unit can integrate data from a gas concentration sensor to confirm the generation status of harmful gases. This allows for multifaceted analysis by integrating data with other sensors.

[0088] The data collection unit can estimate the user's emotions and adjust the level of detail of the damage situation to be captured based on the estimated user's emotions. Specific methods and criteria for estimating the user's emotions include emotion recognition algorithms and sensors used. For example, the data collection unit can capture the user's facial expressions with a camera and estimate the emotion using an emotion recognition algorithm. The data collection unit can also record the user's voice and estimate the emotion using voice analysis technology. Furthermore, the data collection unit can collect the user's biometric data (heart rate and electrodermal activity) with a sensor and estimate the emotion using an emotion recognition algorithm. This allows the level of detail of the damage situation to be captured to be adjusted depending on the user's emotions. For example, if the user is nervous, detailed damage situations will be captured. If the user is calm, multiple damage situations will be captured efficiently. If the user is panicked, detailed images of all damage situations will be captured.

[0089] The data collection unit can link the data collected by the drone's camera with a geographic information system (GIS) to display it on a map in real time. The specific type and usage of a geographic information system (GIS) includes the software and data format used. For example, the data collection unit can display the damage situation on a map in real time to confirm the extent of the damage. The data collection unit can also display the damage situation on a map in real time to confirm the severity of the damage. Furthermore, the data collection unit can display the damage situation on a map in real time to determine the priority of rescue operations. This allows the damage situation to be displayed on a map in real time.

[0090] The data collection unit can automatically upload data collected by the drone's camera to cloud storage and share it with other disaster response teams. The specific types and usage of cloud storage include the cloud service used and the method of uploading data. For example, the data collection unit uploads damage situation data to cloud storage and shares it with other disaster response teams. The data collection unit can also upload damage situation data to cloud storage and share it in real time. Furthermore, the data collection unit can upload damage situation data to cloud storage and share it quickly. This allows data to be uploaded to cloud storage and shared with other disaster response teams.

[0091] The data collection unit uses AI to perform initial analysis of the data collected by the drone's camera, allowing it to quickly extract important information. The specific methods and standards for initial analysis by AI include the algorithms used and the data to be analyzed. For example, the data collection unit uses AI to quickly extract the state of collapsed buildings. The data collection unit can also use AI to quickly extract the state of damage to roads. Furthermore, the data collection unit can also use AI to quickly extract information related to human life. This allows important information to be extracted quickly through initial analysis by AI.

[0092] The analysis unit can estimate the user's emotions and adjust the display method of the analysis results based on the estimated user emotions. Specific methods and criteria for estimating the user's emotions include emotion recognition algorithms and sensors used. For example, the analysis unit can capture the user's facial expressions with a camera and estimate the emotion using an emotion recognition algorithm. The analysis unit can also record the user's voice and estimate the emotion using voice analysis technology. Furthermore, the analysis unit can collect the user's biometric data (heart rate and electrodermal activity) with a sensor and estimate the emotion using an emotion recognition algorithm. This allows the display method of the analysis results to be adjusted according to the user's emotions. For example, if the user is nervous, a simple, highly visible display method is provided. If the user is relaxed, a display method including detailed information is provided. If the user is in a hurry, a display method that focuses on the main points is provided.

[0093] During analysis, the analysis unit can evaluate the reliability of the damage situation data and exclude unreliable data. Specific evaluation criteria and methods for reliability include the origin of the data and the consistency of the data. For example, the analysis unit excludes data if the source of data collection is unclear. The analysis unit can also exclude data if the time of data collection is unclear. Furthermore, the analysis unit can also exclude data if the method of data collection is unclear. This eliminates unreliable data, improving the accuracy of the analysis results.

[0094] During analysis, the analysis unit can analyze the damage situation data in chronological order to predict the progress of the damage. Specific methods and standards for time series analysis include the time interval and the algorithm used. For example, the analysis unit can analyze the damage situation data in chronological order to predict the progress of the damage. The analysis unit can also analyze the damage situation data in chronological order to predict the expansion of the damage. Furthermore, the analysis unit can analyze the damage situation data in chronological order to predict the convergence of the damage. This makes it possible to analyze the damage situation data in chronological order to predict the progress of the damage.

[0095] During analysis, the analysis unit can integrate the damage situation data with other disaster data (seismic waves, meteorological data, etc.) for a comprehensive analysis. Specific types and acquisition methods of other disaster data include seismic wave data and meteorological data. For example, the analysis unit can integrate the data with seismic wave data to comprehensively analyze the state of building collapse. The analysis unit can also integrate the data with meteorological data to comprehensively analyze the impact of flooding. Furthermore, the analysis unit can integrate the data with other disaster data to comprehensively analyze the overall picture of the damage. This allows for integration with other disaster data for a comprehensive analysis.

[0096] The analysis unit can estimate the user's emotions and adjust the level of detail of the analysis results based on the estimated user emotions. Specific methods and criteria for estimating the user's emotions include emotion recognition algorithms and sensors used. For example, the analysis unit can capture the user's facial expressions with a camera and estimate the emotion using an emotion recognition algorithm. The analysis unit can also record the user's voice and estimate the emotion using voice analysis technology. Furthermore, the analysis unit can collect the user's biometric data (heart rate and electrodermal activity) with a sensor and estimate the emotion using an emotion recognition algorithm. This allows the level of detail of the analysis results to be adjusted according to the user's emotions. For example, if the user is nervous, detailed analysis results can be provided. If the user is relaxed, efficient analysis results can be provided. If the user is in a hurry, analysis results that focus on the main points can be provided.

[0097] During analysis, the analysis unit can cluster the damage situation data and classify it according to the type and extent of damage. Specific clustering methods and criteria include the algorithm to be used and the number of clusters. For example, the analysis unit can cluster the collapsed building status and classify it according to the extent of damage. The analysis unit can also cluster the damaged road status and classify it according to the extent of damage. Furthermore, the analysis unit can cluster information related to human life and classify it according to the extent of damage. This allows the damage situation data to be clustered and classified according to the type and extent of damage.

[0098] During analysis, the analysis unit can link damage status data with other disaster response systems to conduct comprehensive damage assessments. Specific methods and standards for linking with other disaster response systems include data sharing protocols and the types of systems to link with. For example, the analysis unit can link with data from the fire department and police to conduct comprehensive damage assessments. The analysis unit can also link with data from emergency medical teams to conduct comprehensive damage assessments. Furthermore, the analysis unit can link with data from the local government's disaster response headquarters to conduct comprehensive damage assessments. This allows for comprehensive damage assessments to be conducted in collaboration with other disaster response systems.

[0099] During analysis, the analysis unit can compare the damage situation data with past disaster data to identify similar damage patterns. Specific methods and standards for acquiring past disaster data include the data source and data type. For example, the analysis unit can compare the data with past earthquake data to identify similar damage patterns. The analysis unit can also compare the data with past flood data to identify similar damage patterns. Furthermore, the analysis unit can compare the data with past fire data to identify similar damage patterns. This makes it possible to identify similar damage patterns by comparing with past disaster data.

[0100] The visualization unit can estimate the user's emotions and adjust the visualization display method based on the estimated user emotions. Specific methods and criteria for estimating the user's emotions include emotion recognition algorithms and sensors used. For example, the visualization unit can capture the user's facial expressions with a camera and estimate the emotion using an emotion recognition algorithm. The visualization unit can also record the user's voice and estimate the emotion using voice analysis technology. Furthermore, the visualization unit can collect the user's biometric data (heart rate and electrodermal activity) with a sensor and estimate the emotion using an emotion recognition algorithm. This allows the visualization display method to be adjusted according to the user's emotions. For example, if the user is nervous, a simple, highly visible display method can be provided. If the user is relaxed, a display method including detailed information can be provided. If the user is in a hurry, a display method that focuses on the main points can be provided.

[0101] During visualization, the visualization unit can display the damage situation data on a 3D map and color-code the extent of the damage. The specific generation method and display format of the 3D map include the software and data format used. For example, the visualization unit can display the collapsed building status on a 3D map and color-code the extent of the damage. The visualization unit can also display the damaged road status on a 3D map and color-code the extent of the damage. Furthermore, the visualization unit can display information related to human life on a 3D map and color-code the extent of the damage. This allows the damage situation data to be displayed color-coded on a 3D map.

[0102] The visualization unit allows interactive manipulation of the damage situation data during visualization, and can display detailed information. Specific methods and criteria for interactive manipulation include the interface used and the range of manipulation. For example, when a specific building is clicked on the 3D map, the visualization unit can display detailed damage to that building. The visualization unit can also display detailed damage to a specific road when that road is clicked on the 3D map. Furthermore, the visualization unit can display detailed damage to a specific area when that area is clicked on the 3D map. This allows interactive manipulation of the damage situation data and displays detailed information.

[0103] During visualization, the visualization unit can display the damage situation data by overlaying it with other map data (roads, buildings, etc.). Specific types and acquisition methods of the other map data include road data and building data. For example, the visualization unit can display the damage situation data by overlaying it with a road map to confirm the damage status of roads. The visualization unit can also display the damage situation data by overlaying it with a building map to confirm the collapsed status of buildings. Furthermore, the visualization unit can display the damage situation data by overlaying it with a topographical map to confirm changes in the topography. This allows the damage situation data to be displayed by overlaying it with other map data.

[0104] The visualization unit can estimate the user's emotions and determine the visualization priority based on the estimated user emotions. Specific methods and criteria for estimating the user's emotions include emotion recognition algorithms and sensors used. For example, the visualization unit can capture the user's facial expressions with a camera and estimate the emotion using an emotion recognition algorithm. The visualization unit can also record the user's voice and estimate the emotion using voice analysis technology. Furthermore, the visualization unit can collect the user's biometric data (heart rate and electrodermal activity) with a sensor and estimate the emotion using an emotion recognition algorithm. This allows the visualization priority to be determined according to the user's emotions. For example, if the user is nervous, the most important damage status can be displayed first. If the user is calm, multiple damage statuses can be efficiently displayed simultaneously. If the user is panicking, all damage statuses can be displayed simultaneously.

[0105] During visualization, the visualization unit can display the damage situation data on a time axis and display the progress of the damage using animation. Specific methods and standards for displaying the time axis include time intervals and display formats. For example, the visualization unit can display the damage situation data on a time axis and display the progress of the damage using animation. The visualization unit can also display the damage situation data on a time axis and display the expansion of the damage using animation. Furthermore, the visualization unit can display the damage situation data on a time axis and display the convergence of the damage using animation. This makes it possible to display the damage situation data on a time axis and display the progress of the damage using animation.

[0106] When visualizing, the Visualization Department can share the damage situation data with other disaster response teams to jointly consider response measures. Specific methods and standards for sharing with other disaster response teams include the data sharing protocol and the platform used. For example, the Visualization Department can share the damage situation data with other disaster response teams to jointly consider response measures. The Visualization Department can also share the damage situation data with other disaster response teams to quickly consider response measures. Furthermore, the Visualization Department can also share the damage situation data with other disaster response teams to consider effective response measures. This allows the damage situation data to be shared with other disaster response teams to jointly consider response measures.

[0107] The visualization unit can make the damage situation data visible on a mobile device such as a smartphone or tablet when visualizing the data. Specific types and uses of the mobile device include a smartphone or a tablet. For example, the visualization unit can display the damage situation data on a smartphone to facilitate on-site confirmation. The visualization unit can also display the damage situation data on a tablet to confirm detailed information. Furthermore, the visualization unit can display the damage situation data on a mobile device to enable a rapid response. This allows the damage situation data to be displayed on a mobile device to facilitate on-site confirmation. === Hard Collateral 1-1 === Each of the multiple elements including the above-mentioned activation unit, data collection unit, analysis unit, and visualization unit is realized, for example, by at least one of the smart device 14 and the data processing device 12. For example, the activation unit activates a drone via the control unit 46A of the smart device 14. The data collection unit photographs the damage situation using the camera 42 of the smart device 14 and transmits the photographs to the data processing device 12. The analysis unit analyzes the damage situation data using AI generated by the specific processing unit 290 of the data processing device 12. The visualization unit visualizes the data analyzed by the specific processing unit 290 of the data processing device 12 as a 3D map. === Hard Collateral 1-2 === Each of the multiple elements including the activation unit, data collection unit, analysis unit, and visualization unit described above is realized, for example, in at least one of the smart glasses 214 and the data processing device 12. For example, the activation unit activates a drone via the control unit 46A of the smart glasses 214. The data collection unit photographs the damage situation using the camera 42 of the smart glasses 214 and transmits the photographs to the data processing device 12. The analysis unit analyzes the damage situation data using AI generated by the specific processing unit 290 of the data processing device 12. The visualization unit visualizes the data analyzed by the specific processing unit 290 of the data processing device 12 as a 3D map. === Hard Collateral 1-3 === Each of the multiple elements including the above-mentioned activation unit, data collection unit, analysis unit, and visualization unit is realized, for example, by at least one of the headset type terminal 314 and the data processing device 12. For example, the activation unit activates a drone via the control unit 46A of the headset type terminal 314. The data collection unit photographs the damage situation using the camera 42 of the headset type terminal 314 and transmits the photographs to the data processing device 12. The analysis unit analyzes the damage situation data using the generated AI by the specific processing unit 290 of the data processing device 12. The visualization unit visualizes the data analyzed by the specific processing unit 290 of the data processing device 12 as a 3D map. === Hard Collateral 1-4 === Each of the multiple elements including the activation unit, data collection unit, analysis unit, and visualization unit described above is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the activation unit activates a drone via the control unit 46A of the robot 414. The data collection unit photographs the damage situation using the camera 42 of the robot 414 and transmits the photographs to the data processing device 12. The analysis unit analyzes the damage situation data using AI generated by the specific processing unit 290 of the data processing device 12. The visualization unit visualizes the data analyzed by the specific processing unit 290 of the data processing device 12 as a 3D map.

[0108] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0109] When a disaster occurs, the launching unit can check the status of the surrounding communication infrastructure and prioritize launching drones in areas where communication is ensured. For example, in an area where communication infrastructure is damaged, the drone can be flown from an area where communication is ensured to check the damage situation. Also, once the communication infrastructure is restored, the drone can immediately resume operation in that area. Furthermore, the drone's flight route can be optimized according to the status of the communication infrastructure, allowing for efficient confirmation of the damage situation. This allows the drone's launch and flight route to be optimized taking into account the status of the communication infrastructure.

[0110] The data collection unit can compare the data collected by the drone with data from before the disaster and automatically extract the difference in damage. For example, it can compare the data with the condition of a building before the disaster to identify collapsed parts. It can also compare the data with the condition of a road before the disaster to identify damaged parts. It can also compare the data with topographical data from before the disaster to identify changes in the topography. This makes it possible to compare data before and after a disaster and automatically extract the difference in damage.

[0111] When analyzing damage situation data, the analysis unit can prioritize the analysis according to the severity of the damage. For example, it can prioritize analysis of areas where buildings have collapsed severely. It can also prioritize analysis of areas where roads have been severely damaged. It can also prioritize analysis of areas that contain information related to human life. This allows analysis to be prioritized according to the severity of the damage, enabling a rapid response.

[0112] When visualizing the damage situation data, the visualization unit can take into account the user's location information and prioritize displaying the nearest damage situation. For example, if the user is at the site, the damage situation in the surrounding area can be prioritized. Also, if the user is in a remote location, the entire damage situation can be displayed at a bird's-eye view. Furthermore, the displayed damage situation can be updated in real time according to the user's movement. This allows the nearest damage situation to be prioritized and displayed taking into account the user's location information.

[0113] The launch unit can estimate the user's emotions and adjust the drone's flight altitude based on the estimated user's emotions. For example, if the user is nervous, the drone will fly at a low altitude to check the detailed damage situation. If the user is calm, the drone will fly at a high altitude to check the widespread damage situation. If the user is in a panic, the drone will fly at an optimal altitude to quickly check the damage situation. This allows the drone's flight altitude to be adjusted according to the user's emotions.

[0114] The data collection unit can estimate the user's emotions and adjust the drone's flight speed based on the estimated user's emotions. For example, if the user is nervous, the drone will fly at a slow speed to check the detailed damage situation. If the user is calm, the drone will fly at a high speed to check the widespread damage situation. If the user is panicking, the drone will fly at an optimal speed to quickly check the damage situation. This allows the drone's flight speed to be adjusted according to the user's emotions.

[0115] The analysis unit can estimate the user's emotions and adjust the notification method of the analysis results based on the estimated user emotions. For example, if the user is nervous, a simple, highly visible notification method is provided. If the user is relaxed, a notification method including detailed information is provided. If the user is in a hurry, a notification method that focuses on the main points is provided. This makes it possible to adjust the notification method of the analysis results according to the user's emotions.

[0116] The visualization unit can estimate the user's emotions and adjust the colors of the visualization based on the estimated user's emotions. For example, if the user is nervous, the colors are displayed in calming tones. If the user is relaxed, the colors are displayed in vivid tones. If the user is in a hurry, the colors are displayed in highly visible tones. In this way, the colors of the visualization can be adjusted according to the user's emotions.

[0117] The visualization unit can estimate the user's emotions and adjust the visualization layout based on the estimated user's emotions. For example, if the user is nervous, a simple, highly visible layout is provided. If the user is relaxed, a layout including detailed information is provided. If the user is in a hurry, a layout that focuses on the main points is provided. In this way, the visualization layout can be adjusted according to the user's emotions.

[0118] When analyzing damage situation data, the analysis unit can incorporate data from other disaster response systems in real time to perform comprehensive damage assessments. For example, it can incorporate data from the fire department and police to comprehensively assess the extent of building collapses. It can also incorporate data from emergency medical teams to comprehensively assess the status of injured people. It can also incorporate data from local government disaster response headquarters to comprehensively assess the overall damage picture. This allows data from other disaster response systems to be incorporated in real time to perform comprehensive damage assessments.

[0119] The processing flow of the second embodiment will be briefly explained below.

[0120] Step 1: The launch unit launches the drones. For example, the launch unit can launch multiple drones simultaneously within one hour of a disaster occurring. The launch unit can also launch drones manually, remotely, or by timer. Step 2: The data collection unit processes the data collected by the drone activated by the activation unit. For example, the data collection unit may use a drone equipped with a camera to capture images of the damage. The specific specifications and performance of the camera include resolution, field of view, zoom function, etc. Step 3: The analysis unit analyzes the data collected by the data collection unit. For example, the analysis unit analyzes the damage situation data using the generation AI. The generation AI analyzes the data using specific algorithms and training data sets. Step 4: The visualization unit visualizes the data analyzed by the analysis unit. For example, the visualization unit uses generation AI to visualize the damage situation as a 3D map. The method for generating the 3D map and its display format include the software and data format used.

[0121] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0122] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AIs include the data generation model 58, such as a neural network model (e.g., a neural network model), and a neural network model (e.g., a neural network model). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input to the data generation model 58. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0123] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0124] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0125] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0126] 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0127] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0128] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0129] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0130] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0131] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0132] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0133] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0134] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0135] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0136] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0137] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0138] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0139] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0140] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0141] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0142] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0143] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0144] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0145] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0146] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0147] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0148] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0149] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0150] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0151] In the headset type terminal 314, the identification process is performed by the processor 46. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0152] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0153] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0154] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0155] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0156] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0157] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0158] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0159] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0160] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0161] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0162] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0163] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0164] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0165] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0166] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0167] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0168] In the robot 414, the processor 46 performs the identification process. The storage 50 stores the identification program 60. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as the control unit 46A in accordance with the identification program 60 executed on the RAM 48. The robot 414 also has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform the same process as the identification processing unit 290 using these models.

[0169] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0170] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0171] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0172] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0173] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0174] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0175] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0176] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0177] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0178] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[0179] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0180] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0181] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0182] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0183] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0184] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0185] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A processor also includes a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0186] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0187] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0188] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0189] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0190] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0191] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0192] [Explanation of symbols]

[0193] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. A launch unit that launches the drone; a data collection unit that processes data collected by the drone activated by the activation unit; an analysis unit that analyzes the data collected by the data collection unit; a visualization unit that visualizes the data analyzed by the analysis unit; Equipped with A system characterized by:

2. The startup unit Launch multiple drones simultaneously within one hour of a disaster occurring.

2. The system of claim 1.

3. The data collection unit A drone equipped with a camera will be used to capture images of the damage.

2. The system of claim 1.

4. The analysis unit Analyzing damage data using generative AI 2. The system of claim 1.

5. The visualization unit Visualizing the damage situation as a 3D map using generative AI 2. The system of claim 1.

6. The analysis unit Quickly process damage data 2. The system of claim 1.

7. The visualization unit Grasp the damage situation on site in real time 2. The system of claim 1.

8. The startup unit Estimate the user's emotions and adjust the drone's launch timing based on the estimated user emotions.

2. The system of claim 1.

Images