System

Abstract

The purpose of the present embodiment is to enable a disaster situation to be quickly and accurately ascertained when a disaster occurs. The system comprises a starting part, a data acquisition part, an analysis part and a visualization part. The starting part is used for starting the unmanned aerial vehicle. And the data acquisition part is used for processing data acquired by the unmanned aerial vehicle started by the starting part. The analysis part is used for analyzing the data collected by the data collection part. The visualization unit is configured to visualize the data analyzed by the analysis unit.

Description

Technical Field

[0001] The technology disclosed herein relates to a system. Background Technology

[0002] Patent Document 1 discloses a personalized chatbot control method executed by at least one processor, the method comprising: receiving user speech; adding the user speech to a prompt containing instructions related to a chatbot role; encoding the prompt; and inputting the encoded prompt into a language model to generate chatbot speech in response to the user speech.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2022-180282 Summary of the Invention

[0004] In existing technologies, there is a problem that it is difficult to grasp the disaster situation in a timely and accurate manner when a disaster occurs.

[0005] The purpose of the system involved in this technical solution is to enable a rapid and accurate understanding of the disaster situation when a disaster occurs.

[0006] The system involved in this technical solution includes a startup unit, a data acquisition unit, a parsing unit, and a visualization unit. The startup unit is used to start the drone. The data acquisition unit is used to process the data collected by the drone started by the startup unit. The parsing unit is used to parse the data collected by the data acquisition unit. The visualization unit is used to visualize the data parsed by the parsing unit.

[0007] The system involved in this technical solution can quickly and accurately grasp the disaster situation when a disaster occurs. Attached Figure Description

[0008] Figure 1 This is a conceptual diagram illustrating an example of the configuration of a data processing system according to the first embodiment.

[0009] Figure 2 This is a conceptual diagram illustrating an example of the main functions of the data processing apparatus and smart device according to the first embodiment.

[0010] Figure 3 This is a conceptual diagram illustrating an example of the data processing system configuration in the second embodiment.

[0011] Figure 4 This is a conceptual diagram illustrating an example of the main functions of the data processing device and smart glasses according to the second embodiment.

[0012] Figure 5 This is a conceptual diagram illustrating an example of the data processing system configuration in the third embodiment.

[0013] Figure 6 This is a conceptual diagram illustrating an example of the main functions of the data processing device and head-mounted terminal according to the third embodiment.

[0014] Figure 7 This is a conceptual diagram illustrating an example of the data processing system configuration in the fourth embodiment.

[0015] Figure 8 This is a conceptual diagram illustrating an example of the functions of the main parts of the data processing device and robot according to the fourth embodiment.

[0016] Figure 9 It represents an emotion graph that maps multiple emotions.

[0017] Figure 10 It represents an emotion graph that maps multiple emotions.

[0018] Explanation of reference numerals in the attached figures

[0019] Data processing systems 10, 210, 310, and 410

[0020] 12 Data processing device

[0021] 14 Smart devices

[0022] 214 Smart Glasses

[0023] 314 Head-mounted terminal

[0024] 414 Robot Detailed Implementation

[0025] Hereinafter, an example of an implementation of the system involved in this disclosure will be described with reference to the accompanying drawings.

[0026] First, let's explain the terms used in the following description.

[0027] In the following embodiments, the processor (hereinafter referred to as "processor"), as indicated by the reference numerals, can be a single computing device or a combination of multiple computing devices. Furthermore, a processor can be a single computing device or a combination of multiple computing devices. Examples of computing devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit), etc.

[0028] In the following embodiments, RAM (Random Access Memory), as indicated in the figures, is a memory for temporary information storage that is used by the processor as working memory.

[0029] In the following embodiments, the memory, as indicated by the reference numerals, is one or more non-volatile storage devices used to store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), disk (e.g., hard disk), or magnetic tape, etc.

[0030] In the following embodiments, the communication I / F (Interface) with reference numerals is an interface including a communication processor and an antenna, etc. The communication I / F is responsible for communication between multiple computers. Examples of communication standards applicable to the communication I / F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0031] In the following implementation, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it can be only A, only B, or a combination of A and B. Furthermore, in this specification, when "and / or" connects more than three items, the same approach as "A and / or B" applies.

[0032] First Implementation Method

[0033] Figure 1 An example of the configuration of the data processing system 10 according to the first embodiment is shown.

[0034] like Figure 1 As shown, the data processing system 10 includes a data processing device 12 and an intelligent device 14. An example of the data processing device 12 is a server.

[0035] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. An example of the network 54 includes a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0036] The smart device 14 includes a computer 36, a receiver 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the receiver 38, output device 40, and camera 42 are also connected to the bus 52.

[0037] The receiving device 38 includes a touchscreen 38A and a microphone 38B, etc., for receiving user input. The touchscreen 38A receives user input generated by contact with an indicator (e.g., a pen or finger). The microphone 38B receives user input generated by sound by detecting the user's voice. The control unit 46A sends data representing user input received via the touchscreen 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, a specific processing unit 290 (see reference...) Figure 2 Get the data that represents the user input.

[0038] The output device 40 includes a display 40A and a speaker 40B, which present data to the user by outputting data in a user-perceptible format (e.g., sound and / or text). The display 40A displays visual information such as text and images according to instructions from the processor 46. The speaker 40B outputs sound according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0039] Communication I / F 44 is connected to network 54. Communication I / F 44 and 26 are responsible for sending and receiving various information between processor 46 and processor 28 via network 54.

[0040] Figure 2 An example of the main functions of the data processing device 12 and the smart device 14 is shown.

[0041] like Figure 2 As shown, in the data processing apparatus 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the memory 32. The specific processing program 56 is an example of a "program" as understood in this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 based on the specific processing program 56 executed on the RAM 30.

[0042] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. The emotion inference function using the emotion-specific model 59 (emotion-specific function) includes inferring and predicting the user's emotions, performing various inferences and predictions related to the user's emotions, but is not limited to this example. Furthermore, emotion inference and prediction also include, for example, emotion analysis (analysis).

[0043] In the smart device 14, specific processing is performed by the processor 46. A specific processing program 60 is stored in the memory 50. The specific processing program 60 is used in conjunction with the data processing system 10. The processor 46 reads the specific processing program 60 from the memory 50 and executes the read specific processing program 60 on the RAM 48. Specific processing is implemented by the processor 46 operating as a control unit 46A based on the specific processing program 60 executed on the RAM 48. Furthermore, the smart device 14 may also have the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and use these models to perform the same processing as the specific processing unit 290.

[0044] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. Furthermore, the data processing device 12 may be a server device or a user-owned terminal device (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of the processing performed by the data processing system 10 of the first embodiment will be described.

[0045] Implementation Method 1

[0046] The disaster damage confirmation system described in this invention is used to quickly confirm the damage situation when a disaster occurs. This system can simultaneously launch multiple drones within one hour of a disaster, and these drones, equipped with cameras, map the damage situation. Subsequently, the large amount of disaster data collected by the drones is analyzed by generative AI and visualized as a 3D map. Generative AI can process this data at extremely high speed, enabling real-time monitoring of the disaster situation on-site. For example, when a disaster occurs, the system automatically launches multiple drones. These drones are equipped with cameras to photograph the damage situation, such as detailed images of collapsed buildings and damaged roads. The disaster data collected by the drones is then sent to the generative AI. The generative AI analyzes this data and visualizes it as a 3D map. For example, the 3D map displays the collapse status of buildings, providing a clear understanding of the extent of damage to each building. Furthermore, generative AI can process this data at extremely high speed, thereby enabling real-time monitoring of the disaster situation on-site. For example, within one hour of a disaster, a detailed understanding of the damage situation can be obtained, allowing for rapid response. Therefore, disaster impact assessment systems can quickly assess the extent of damage when a disaster occurs, improving the efficiency of rescue and recovery efforts. For example, by prioritizing aid to severely affected areas, losses can be minimized. Furthermore, recovery plans can be rapidly developed, enabling early recovery.

[0047] The disaster damage confirmation system according to this embodiment includes a startup unit, a data acquisition unit, an analysis unit, and a visualization unit. The startup unit is used to launch drones. For example, the startup unit can launch multiple drones simultaneously within one hour of a disaster. Furthermore, the startup unit can launch drones manually, remotely, or by a timer. The data acquisition unit is used to process the data collected by the drones launched by the startup unit. For example, the data acquisition unit uses drones equipped with cameras to photograph the disaster situation. The specific specifications and performance of the cameras include resolution, field of view, and zoom capabilities. The analysis unit is used to analyze the data collected by the data acquisition unit. For example, the analysis unit uses generative AI to analyze the disaster situation data. Generative AI analyzes the data using specific algorithms and training datasets. The visualization unit is used to visualize the data analyzed by the analysis unit. For example, the visualization unit uses generative AI to visualize the disaster situation as a 3D map. The method for generating the 3D map and its display format include the software used and the data format. Therefore, the disaster damage confirmation system according to this embodiment can quickly confirm the disaster situation when a disaster occurs and monitor the disaster status on-site in real time.

[0048] The activation unit can simultaneously launch multiple drones within one hour of a disaster. The specific time measurement methods and benchmarks for this one-hour period include the definition of a disaster and the starting point for time measurement. For example, upon receiving a disaster signal, the activation unit starts a timer and simultaneously launches multiple drones within one hour. Furthermore, the activation unit can automatically select the number and type of drones to launch based on the type and scale of the disaster. This allows for the rapid activation of drones after a disaster to assess the damage.

[0049] The data acquisition unit can use drones equipped with cameras to photograph the disaster situation. The specific specifications and performance of these camera-equipped drones include resolution, field of view, and zoom capabilities. For example, the data acquisition unit uses high-resolution cameras to capture detailed images of collapsed buildings. Furthermore, it can use wide-angle cameras to confirm the extent of the damage over a large area. Moreover, it can use zoom capabilities to pinpoint specific damaged areas. Thus, the data acquisition unit can capture detailed images of the disaster situation using drone cameras.

[0050] The analysis department can utilize generative AI to analyze disaster situation data. Specific types and implementation methods of generative AI include specific algorithms and training datasets. For example, the analysis department uses image analysis algorithms to analyze disaster situation data. Furthermore, the analysis department can also utilize data mining techniques to analyze disaster situation data. Further, the analysis department can also utilize machine learning algorithms to analyze disaster situation data. Therefore, by utilizing generative AI, the accuracy of disaster situation data analysis can be improved.

[0051] The visualization department can utilize generative AI to visualize disaster situations in the form of 3D maps. The specific methods for generating and displaying these 3D maps include the software and data formats used. For example, the visualization department uses generative AI to display building collapses on a 3D map. Furthermore, it can use generative AI to display road damage on a 3D map. Moreover, it can use generative AI to display life-or-death information on a 3D map. Thus, by utilizing generative AI, disaster situations can be visualized in the form of 3D maps.

[0052] The analysis unit is capable of rapidly processing disaster situation data. Specific benchmarks and methods for rapid processing include reference values ​​for processing time and the performance of the hardware used. For example, the analysis unit utilizes high-performance processors to rapidly process disaster situation data. Furthermore, the analysis unit can also utilize parallel processing techniques to rapidly process disaster situation data. Moreover, the analysis unit can also utilize cloud computing technology to rapidly process disaster situation data. Thus, it is able to rapidly process disaster situation data and monitor the on-site situation in real time.

[0053] The visualization department can monitor the disaster situation on-site in real time. The specific definition and benchmark for "real-time" include the data update frequency and the allowable range of latency. For example, the visualization department achieves real-time monitoring of the disaster situation by increasing the data update frequency. Furthermore, the visualization department can also achieve real-time monitoring of the disaster situation by minimizing latency. Moreover, the visualization department can process data in real time and immediately display the disaster situation on-site. Thus, it is possible to monitor the disaster situation on-site in real time.

[0054] The activation unit can automatically select the number and type of drones to deploy based on the type and scale of the disaster. Specific classification criteria and evaluation methods for disaster types and scales include classifications for earthquakes, floods, and fires, as well as evaluation criteria for the degree of damage. For example, during an earthquake, the activation unit prioritizes deploying drones equipped with high-resolution cameras to confirm building collapse. Furthermore, during a flood, the activation unit can additionally deploy underwater drones to measure water levels. Further, during a fire, the activation unit deploys drones equipped with thermal sensors to confirm the spread of the fire. Thus, the most suitable drones can be selected based on the type and scale of the disaster.

[0055] The startup unit can set an optimal startup plan taking into account the remaining battery level and flight time of the drones. Specific measurement methods and benchmarks for remaining battery level and flight time include the remaining percentage and available time. For example, the startup unit can use drones with low remaining battery levels for nearby disaster situation verification. Furthermore, it can use drones with long flight times for large-scale disaster situation verification. Moreover, the startup unit can also set plans for continuous use of drones with easily replaceable batteries. Thus, efficient utilization of drones is possible while considering their remaining battery level and flight time.

[0056] The launch unit can monitor environmental conditions such as weather and wind speed in real time when the drone starts up and set the optimal flight route. Specific measurement methods and benchmarks for environmental conditions such as weather and wind speed are included, along with the sensors used and the data update frequency. For example, in strong winds, the launch unit takes wind speed into account to set a stable flight route. Furthermore, in rainy weather, the launch unit can prioritize launching drones with waterproof capabilities. Moreover, in low temperatures, the launch unit can set a short-duration flight route to minimize battery consumption. Thus, it is possible to set the optimal flight route while taking environmental conditions into account.

[0057] The launch unit can select priority flight areas by combining geographic disaster prediction data when the drone is launched. The specific methods and benchmarks for acquiring geographic disaster prediction data include the data sources and prediction algorithms used. For example, the launch unit may prioritize flights to areas predicted to be severely affected by disasters. Furthermore, the launch unit can also prioritize flights to areas with important infrastructure (such as hospitals, fire stations, etc.). Further, the launch unit can also prioritize flights to areas with high population density. Thus, it is possible to select priority flight areas by combining geographic disaster prediction data.

[0058] The activation unit can consider coordination with other disaster response systems when activating a drone, setting the optimal activation timing. Specific methods and benchmarks for coordination with other disaster response systems include data sharing protocols and the types of systems involved. For example, the activation unit can coordinate with fire departments and police to adjust the drone's activation timing. Furthermore, it can coordinate with emergency medical teams. Moreover, it can coordinate with local government disaster response headquarters to adjust the drone's activation timing. Thus, it is possible to set the optimal activation timing while considering coordination with other disaster response systems.

[0059] The launch unit can learn and apply optimal flight patterns by referencing past disaster data when the drone starts up. The specific methods and benchmarks for acquiring past disaster data include data sources and data types. For example, based on past earthquake data, the launch unit can prioritize flying over areas where building collapses are predicted. Furthermore, it can prioritize flying over areas prone to rising water levels based on past flood data. Additionally, it can prioritize flying over areas prone to fire spread based on past fire data. Thus, it can learn and apply optimal flight patterns by referencing past disaster data.

[0060] The data acquisition unit can automatically adjust the resolution and shooting angle of the drone's camera based on the disaster situation. Specific adjustment methods and benchmarks for camera resolution and shooting angle include resolution values ​​and shooting angle ranges. For example, the data acquisition unit may use high-resolution images to accurately assess building collapse. Furthermore, it can use wide-angle images to assess widespread damage. Additionally, it can utilize zoom functionality to accurately assess specific damaged areas. Thus, the camera's resolution and shooting angle can be automatically adjusted according to the disaster situation.

[0061] The data acquisition unit can filter data collected by drone cameras in real time, extracting only important information. Specific filtering methods and criteria include filtering algorithms and the definition of important information. For example, the data acquisition unit may prioritize extracting information about building collapses. Furthermore, it can prioritize extracting information about road damage. Moreover, it can prioritize extracting information related to human lives. Thus, it can extract important information in real time.

[0062] The data acquisition unit can integrate data collected by the drone's camera with other sensors (temperature, humidity, gas concentration, etc.) for multi-faceted analysis. Specific types and usage methods of these other sensors include temperature sensors, humidity sensors, and gas concentration sensors. For example, integrating data from a temperature sensor can help determine the spread of a fire. Furthermore, it can integrate data from a humidity sensor to assess the impact of floods. Moreover, it can integrate data from a gas concentration sensor to determine the presence of harmful gases. This allows for multi-faceted analysis by integrating with other sensors.

[0063] The data acquisition department can link data collected by drone cameras with a Geographic Information System (GIS) and display it on a map in real time. This includes the specific type of GIS and its usage, as well as the software and data format used. For example, the data acquisition department can display the disaster situation on the map in real time, confirming the affected area. Furthermore, it can also display the disaster situation on the map in real time, confirming the severity of the disaster. Moreover, it can determine the priority of rescue activities by displaying the disaster situation on the map in real time. Thus, it can display the disaster situation on the map in real time.

[0064] The data acquisition department can automatically upload data collected by drone cameras to cloud storage and share it with other disaster response teams. Specific details regarding the type and usage of cloud storage, including the cloud services used and data upload methods, are provided. For example, the data acquisition department can upload disaster situation data to cloud storage and share it with other disaster response teams. Furthermore, the data acquisition department can also upload and share disaster situation data in real time. Moreover, the data acquisition department can rapidly upload and share disaster situation data. Thus, it is possible to upload data to cloud storage and share it with other disaster response teams.

[0065] The data acquisition department can perform preliminary AI analysis on data collected by drone cameras to quickly extract important information. Specific methods and benchmarks for this preliminary AI analysis include the algorithms used and the data being analyzed. For example, the data acquisition department can use AI to quickly extract information about building collapses. Furthermore, it can use AI to quickly extract information about road damage. Moreover, the data acquisition department can use AI to quickly extract information related to human lives. Thus, it is possible to quickly extract important information through preliminary AI analysis.

[0066] The analysis department is able to assess the reliability of disaster situation data during analysis and exclude data with low reliability. Specific evaluation criteria and methods for reliability include data source and data consistency. For example, the analysis department excludes data when the data collection source is unclear. Furthermore, the analysis department can exclude data when the data collection time is unclear. Even further, the analysis department can exclude data when the data collection method is unclear. Thus, by excluding data with low reliability, the accuracy of the analysis results is improved.

[0067] The analysis unit is capable of performing time-series analysis on disaster situation data during analysis to predict the progress of the disaster. Specific methods and benchmarks for time-series analysis include time intervals and algorithms used. For example, the analysis unit performs time-series analysis on disaster situation data to predict the progress of the disaster. Furthermore, the analysis unit can also perform time-series analysis on disaster situation data to predict the expansion of the disaster. Further, the analysis unit can also perform time-series analysis on disaster situation data to predict the convergence of the disaster. Thus, it is possible to perform time-series analysis on disaster situation data to predict the progress of the disaster.

[0068] The analysis unit can integrate disaster data with other disaster data (seismic waves, meteorological data, etc.) for comprehensive analysis. The specific types and acquisition methods of these other disaster data include seismic wave data and meteorological data. For example, the analysis unit can integrate seismic wave data to comprehensively analyze building collapse conditions. Furthermore, it can integrate with meteorological data to comprehensively analyze the impact of floods. Moreover, the analysis unit can integrate with other disaster data to comprehensively analyze the overall disaster situation. Thus, it can integrate with other disaster data for comprehensive analysis.

[0069] The analysis unit can cluster disaster data during analysis, classifying it according to the type and severity of the disaster. Specific methods and benchmarks for clustering are detailed, including the algorithms used and the number of clusters. For example, the analysis unit can cluster building collapse data and classify it according to the severity of the disaster. Furthermore, it can cluster road damage data and classify it according to the severity of the disaster. Moreover, it can cluster information related to human lives and classify it according to the severity of the disaster. Thus, it is possible to cluster disaster data and classify it according to the type and severity of the disaster.

[0070] The data analysis department can collaborate with other disaster response systems during analysis to conduct comprehensive disaster assessments. Specific methods and benchmarks for collaboration with other disaster response systems include data sharing protocols and the types of systems involved. For example, the analysis department can collaborate with fire departments and police to conduct comprehensive disaster assessments. Furthermore, it can collaborate with emergency medical teams to conduct comprehensive disaster assessments. Moreover, it can collaborate with local government disaster response headquarters to conduct comprehensive disaster assessments. Thus, it is possible to conduct comprehensive disaster assessments in collaboration with other disaster response systems.

[0071] The analysis unit can compare disaster data with past disaster data during analysis to identify specific similar disaster patterns. The specific methods and benchmarks for acquiring past disaster data include data sources and data types. For example, the analysis unit can compare data with past earthquake data to identify specific similar disaster patterns. Furthermore, it can compare data with past flood data to identify specific similar disaster patterns. Even further, it can compare data with past fire data to identify specific similar disaster patterns. Thus, it is possible to compare disaster data with past disaster data to identify specific similar disaster patterns.

[0072] The visualization department can display disaster data on a 3D map during visualization, using color coding to differentiate between different levels of damage. The specific methods for generating and displaying the 3D map include the software used and the data format. For example, the visualization department can display collapsed buildings on the 3D map, using color coding to differentiate between different levels of damage. Furthermore, it can display road damage on the 3D map, also using color coding to differentiate between different levels of damage. Moreover, the visualization department can display life-or-death information on the 3D map, again using color coding to differentiate between different levels of damage. Thus, disaster data can be displayed on a 3D map using color coding.

[0073] The visualization department enables interactive manipulation of disaster data and displays detailed information during visualization. Specific methods and benchmarks for interactive operation include the interface used and the scope of operation. For example, clicking on a specific building on a 3D map displays detailed damage information for that building. Similarly, clicking on a specific road on a 3D map displays detailed damage information for that road. Furthermore, clicking on a specific area on a 3D map displays detailed damage information for that area. This allows for interactive manipulation of disaster data and the display of detailed information.

[0074] The visualization department can overlay disaster damage data with other map data (roads, buildings, etc.) during visualization. The specific types and acquisition methods of the other map data include road data and building data. For example, the visualization department can overlay disaster damage data with road maps to confirm road damage. Furthermore, it can overlay disaster damage data with building maps to confirm building collapse. Moreover, it can overlay disaster damage data with topographic maps to confirm terrain changes. Thus, it is possible to overlay disaster damage data with other map data.

[0075] The visualization department can display disaster data along a timeline and animate its progress during visualization. Specific methods and benchmarks for timeline display include time intervals and display formats. For example, the visualization department can display disaster data along a timeline and animate its progress. Furthermore, it can also display disaster data along a timeline and animate the expansion of the disaster. Moreover, it can also display disaster data along a timeline and animate the containment of the disaster. Thus, it is possible to display disaster data along a timeline and animate its progress.

[0076] The Visualization Department can share disaster data with other disaster response teams during visualization, facilitating joint discussions on response measures. Specific methods and benchmarks for sharing with other disaster response teams are outlined, including data sharing protocols and the platforms used. For example, the Visualization Department can share disaster data with other disaster response teams to jointly discuss response measures. Furthermore, the Visualization Department can also share disaster data with other disaster response teams to quickly discuss response measures. Moreover, the Visualization Department can also share disaster data with other disaster response teams to discuss effective response measures. Thus, it is possible to share disaster data with other disaster response teams and jointly discuss response measures.

[0077] The visualization department enables the display of disaster data on mobile devices such as smartphones and tablets during visualization. Specific types and usage methods of mobile devices are included, specifically smartphones and tablets. For example, the visualization department displays disaster data on smartphones for easy on-site verification. Furthermore, the visualization department can also display disaster data on tablets to confirm detailed information. Moreover, the visualization department can display disaster data on mobile devices to enable rapid response. Thus, the ability to display disaster data on mobile devices facilitates on-site verification.

[0078] The system involved in this embodiment is not limited to the above examples. For example, various modifications can be made as follows.

[0079] The launch unit can assess the status of surrounding communication infrastructure during a disaster and prioritize launching drones in areas with uninterrupted communication. For example, in areas with damaged communication infrastructure, drones can take off from areas with uninterrupted communication to assess the damage. Furthermore, once communication infrastructure is restored, drone launches in that area can be resumed immediately. Moreover, drone flight routes can be optimized based on the status of communication infrastructure for efficient disaster assessment. Thus, the launch and flight routes of drones can be optimized by considering the status of communication infrastructure.

[0080] The data acquisition unit can compare data collected by drones with data collected before a disaster, automatically extracting differences in damage. For example, it can compare the data with the condition of buildings before the disaster to identify specific collapsed sections. Furthermore, it can compare the data with the condition of roads before the disaster to identify specific damaged sections. Additionally, it can compare the data with terrain data before the disaster to identify specific terrain changes. Thus, it can compare data before and after a disaster and automatically extract differences in damage.

[0081] The analysis unit can prioritize disaster data analysis based on the severity of the damage. For example, it can prioritize analyzing areas with severe building collapses, areas with severe road damage, and areas containing information related to human lives. This prioritized analysis based on disaster severity enables a rapid response.

[0082] When visualizing disaster data, the visualization department can prioritize displaying the most recent disaster situation based on the user's location. For example, if the user is on-site, the disaster situation in their surroundings will be displayed first. Furthermore, when the user is remotely viewing the data, they can also get a bird's-eye view of the overall disaster situation. Moreover, the displayed disaster situation can be updated in real time based on the user's movement. Thus, it is possible to prioritize displaying the most recent disaster situation based on the user's location.

[0083] When analyzing disaster data, the analysis department can import data from other disaster response systems in real time for comprehensive disaster assessment. For example, it can import data from fire departments and police to comprehensively assess building collapse conditions. Furthermore, it can import data from emergency medical teams to comprehensively assess the condition of the injured. Additionally, it can import data from local government disaster response headquarters to comprehensively assess the overall disaster situation. Thus, it can import data from other disaster response systems in real time for comprehensive disaster assessment.

[0084] The following is a brief description of the processing flow of Implementation Method 1.

[0085] Step 1: The launch unit starts the drones. For example, the launch unit can start multiple drones simultaneously within one hour of a disaster. In addition, the launch unit can also start drones manually, remotely, or by a timer.

[0086] Step 2: The data acquisition unit processes the data collected by the drone initiated by the launch unit. For example, the data acquisition unit uses a drone equipped with a camera to photograph the disaster situation. The specific specifications and performance of the camera include resolution, field of view, zoom function, etc.

[0087] Step 3: The analysis department analyzes the data collected by the data acquisition department. For example, the analysis department uses generative AI to analyze disaster situation data. Generative AI analyzes the data using specific algorithms and training datasets.

[0088] Step 4: The Visualization Department visualizes the data analyzed by the Analysis Department. For example, the Visualization Department uses generative AI to visualize the disaster situation in the form of a 3D map. The generation method and display format of the 3D map include the software used and the data format.

[0089] Implementation Method 2

[0090] The disaster damage confirmation system described in this invention is used to quickly confirm the damage situation when a disaster occurs. This system can simultaneously launch multiple drones within one hour of a disaster, and these drones, equipped with cameras, map the damage situation. Subsequently, the large amount of disaster data collected by the drones is analyzed by generative AI and visualized as a 3D map. Generative AI can process this data at extremely high speed, enabling real-time monitoring of the disaster situation on-site. For example, when a disaster occurs, the system automatically launches multiple drones. These drones are equipped with cameras to photograph the damage situation, such as detailed images of collapsed buildings and damaged roads. The disaster data collected by the drones is then sent to the generative AI. The generative AI analyzes this data and visualizes it as a 3D map. For example, the 3D map displays the collapse status of buildings, providing a clear understanding of the extent of damage to each building. Furthermore, generative AI can process this data at extremely high speed, thereby enabling real-time monitoring of the disaster situation on-site. For example, within one hour of a disaster, a detailed understanding of the damage situation can be obtained, allowing for rapid response. Therefore, disaster impact assessment systems can quickly assess the extent of damage when a disaster occurs, improving the efficiency of rescue and recovery efforts. For example, by prioritizing aid to severely affected areas, losses can be minimized. Furthermore, recovery plans can be rapidly developed, enabling early recovery.

[0091] The disaster damage confirmation system according to this embodiment includes a startup unit, a data acquisition unit, an analysis unit, and a visualization unit. The startup unit is used to launch drones. For example, the startup unit can launch multiple drones simultaneously within one hour of a disaster. Furthermore, the startup unit can launch drones manually, remotely, or by a timer. The data acquisition unit is used to process the data collected by the drones launched by the startup unit. For example, the data acquisition unit uses drones equipped with cameras to photograph the disaster situation. The specific specifications and performance of the cameras include resolution, field of view, and zoom capabilities. The analysis unit is used to analyze the data collected by the data acquisition unit. For example, the analysis unit uses generative AI to analyze the disaster situation data. Generative AI analyzes the data using specific algorithms and training datasets. The visualization unit is used to visualize the data analyzed by the analysis unit. For example, the visualization unit uses generative AI to visualize the disaster situation as a 3D map. The method for generating the 3D map and its display format include the software used and the data format. Therefore, the disaster damage confirmation system according to this embodiment can quickly confirm the disaster situation when a disaster occurs and monitor the disaster status on-site in real time.

[0092] The activation unit can simultaneously launch multiple drones within one hour of a disaster. The specific time measurement methods and benchmarks for this one-hour period include the definition of a disaster and the starting point for time measurement. For example, upon receiving a disaster signal, the activation unit starts a timer and simultaneously launches multiple drones within one hour. Furthermore, the activation unit can automatically select the number and type of drones to launch based on the type and scale of the disaster. This allows for the rapid activation of drones after a disaster to assess the damage.

[0093] The data acquisition unit can use drones equipped with cameras to photograph the disaster situation. The specific specifications and performance of these camera-equipped drones include resolution, field of view, and zoom capabilities. For example, the data acquisition unit uses high-resolution cameras to capture detailed images of collapsed buildings. Furthermore, it can use wide-angle cameras to confirm the extent of the damage over a large area. Moreover, it can use zoom capabilities to pinpoint specific damaged areas. Thus, the data acquisition unit can capture detailed images of the disaster situation using drone cameras.

[0094] The analysis department can utilize generative AI to analyze disaster situation data. Specific types and implementation methods of generative AI include specific algorithms and training datasets. For example, the analysis department uses image analysis algorithms to analyze disaster situation data. Furthermore, the analysis department can also utilize data mining techniques to analyze disaster situation data. Further, the analysis department can also utilize machine learning algorithms to analyze disaster situation data. Therefore, by utilizing generative AI, the accuracy of disaster situation data analysis can be improved.

[0095] The visualization department can utilize generative AI to visualize disaster situations in the form of 3D maps. The specific methods for generating and displaying these 3D maps include the software and data formats used. For example, the visualization department uses generative AI to display building collapses on a 3D map. Furthermore, it can use generative AI to display road damage on a 3D map. Moreover, it can use generative AI to display life-or-death information on a 3D map. Thus, by utilizing generative AI, disaster situations can be visualized in the form of 3D maps.

[0096] The analysis unit is capable of rapidly processing disaster situation data. Specific benchmarks and methods for rapid processing include reference values ​​for processing time and the performance of the hardware used. For example, the analysis unit utilizes high-performance processors to rapidly process disaster situation data. Furthermore, the analysis unit can also utilize parallel processing techniques to rapidly process disaster situation data. Moreover, the analysis unit can also utilize cloud computing technology to rapidly process disaster situation data. Thus, it is able to rapidly process disaster situation data and monitor the on-site situation in real time.

[0097] The visualization department can monitor the disaster situation on-site in real time. The specific definition and benchmark for "real-time" include the data update frequency and the allowable range of latency. For example, the visualization department achieves real-time monitoring of the disaster situation by increasing the data update frequency. Furthermore, the visualization department can also achieve real-time monitoring of the disaster situation by minimizing latency. Moreover, the visualization department can process data in real time and immediately display the disaster situation on-site. Thus, it is possible to monitor the disaster situation on-site in real time.

[0098] The launch unit can infer the user's emotions and adjust the drone's launch timing accordingly. Specific methods and benchmarks for inferring user emotions include emotion recognition algorithms and the sensors used. For example, the launch unit captures the user's facial expressions using a camera and uses emotion recognition algorithms to infer emotions. Furthermore, the launch unit can record the user's voice and use speech analysis technology to infer emotions. Additionally, the launch unit can collect the user's physiological data (heart rate, skin conductance) through sensors and use emotion recognition algorithms to infer emotions. Thus, the drone's launch timing can be adjusted based on the user's emotions. For example, when the user is anxious, the drone can be launched earlier to quickly assess the disaster situation. When the user is calm, the drone can be launched at the optimal time to efficiently assess the disaster situation. When the user is panicked, the drone launch can be automated to minimize user intervention.

[0099] The activation unit can automatically select the number and type of drones to deploy based on the type and scale of the disaster. Specific classification criteria and evaluation methods for disaster types and scales include classifications for earthquakes, floods, and fires, as well as evaluation criteria for the degree of damage. For example, during an earthquake, the activation unit prioritizes deploying drones equipped with high-resolution cameras to confirm building collapse. Furthermore, during a flood, the activation unit can additionally deploy underwater drones to measure water levels. Further, during a fire, the activation unit deploys drones equipped with thermal sensors to confirm the spread of the fire. Thus, the most suitable drones can be selected based on the type and scale of the disaster.

[0100] The startup unit can set an optimal startup plan taking into account the remaining battery level and flight time of the drones. Specific measurement methods and benchmarks for remaining battery level and flight time include the remaining percentage and available time. For example, the startup unit can use drones with low remaining battery levels for nearby disaster situation verification. Furthermore, it can use drones with long flight times for large-scale disaster situation verification. Moreover, the startup unit can also set plans for continuous use of drones with easily replaceable batteries. Thus, efficient utilization of drones is possible while considering their remaining battery level and flight time.

[0101] The launch unit can monitor environmental conditions such as weather and wind speed in real time when the drone starts up and set the optimal flight route. Specific measurement methods and benchmarks for environmental conditions such as weather and wind speed are included, along with the sensors used and the data update frequency. For example, in strong winds, the launch unit takes wind speed into account to set a stable flight route. Furthermore, in rainy weather, the launch unit can prioritize launching drones with waterproof capabilities. Moreover, in low temperatures, the launch unit can set a short-duration flight route to minimize battery consumption. Thus, it is possible to set the optimal flight route while taking environmental conditions into account.

[0102] The activation unit can infer the user's emotions and determine the priority of drone activation based on these inferences. Specific methods and benchmarks for inferring user emotions include emotion recognition algorithms and the sensors used. For example, the activation unit captures the user's facial expressions using a camera and uses emotion recognition algorithms to infer emotions. Furthermore, the activation unit can record the user's voice and use speech analysis technology to infer emotions. Additionally, the activation unit can collect the user's physiological data (heart rate, skin conductance) using sensors and use emotion recognition algorithms to infer emotions. Thus, the priority of drones can be determined based on the user's emotions. For example, when the user is anxious, the most reliable drone is activated first. When the user is calm, multiple drones are activated simultaneously and efficiently. When the user is panicked, all drones are activated simultaneously to quickly assess the disaster situation.

[0103] The launch unit can select priority flight areas by combining geographic disaster prediction data when the drone is launched. The specific methods and benchmarks for acquiring geographic disaster prediction data include the data sources and prediction algorithms used. For example, the launch unit may prioritize flights to areas predicted to be severely affected by disasters. Furthermore, the launch unit can also prioritize flights to areas with important infrastructure (such as hospitals, fire stations, etc.). Further, the launch unit can also prioritize flights to areas with high population density. Thus, it is possible to select priority flight areas by combining geographic disaster prediction data.

[0104] The activation unit can consider coordination with other disaster response systems when activating a drone, setting the optimal activation timing. Specific methods and benchmarks for coordination with other disaster response systems include data sharing protocols and the types of systems involved. For example, the activation unit can coordinate with fire departments and police to adjust the drone's activation timing. Furthermore, it can coordinate with emergency medical teams. Moreover, it can coordinate with local government disaster response headquarters to adjust the drone's activation timing. Thus, it is possible to set the optimal activation timing while considering coordination with other disaster response systems.

[0105] The launch unit can learn and apply optimal flight patterns by referencing past disaster data when the drone starts up. The specific methods and benchmarks for acquiring past disaster data include data sources and data types. For example, based on past earthquake data, the launch unit can prioritize flying over areas where building collapses are predicted. Furthermore, it can prioritize flying over areas prone to rising water levels based on past flood data. Additionally, it can prioritize flying over areas prone to fire spread based on past fire data. Thus, it can learn and apply optimal flight patterns by referencing past disaster data.

[0106] The data acquisition unit can infer users' emotions and prioritize the recording of disaster scenes based on these inferences. Specific methods and benchmarks for inferring user emotions include emotion recognition algorithms and the sensors used. For example, the data acquisition unit captures user facial expressions using a camera and uses emotion recognition algorithms to infer emotions. Furthermore, the data acquisition unit can record user voice and use speech analysis technology to infer emotions. Additionally, the data acquisition unit can collect user physiological data (heart rate, skin conductance) using sensors and use emotion recognition algorithms to infer emotions. Therefore, the priority of recording disaster scenes can be determined based on the user's emotions. For example, when the user is anxious, the most important disaster scenes are recorded first. When the user is calm, multiple disaster scenes are recorded simultaneously and efficiently. When the user is in a state of panic, all disaster scenes are recorded simultaneously.

[0107] The data acquisition unit can automatically adjust the resolution and shooting angle of the drone's camera based on the disaster situation. Specific adjustment methods and benchmarks for camera resolution and shooting angle include resolution values ​​and shooting angle ranges. For example, the data acquisition unit may use high-resolution images to accurately assess building collapse. Furthermore, it can use wide-angle images to assess widespread damage. Additionally, it can utilize zoom functionality to accurately assess specific damaged areas. Thus, the camera's resolution and shooting angle can be automatically adjusted according to the disaster situation.

[0108] The data acquisition unit can filter data collected by drone cameras in real time, extracting only important information. Specific filtering methods and criteria include filtering algorithms and the definition of important information. For example, the data acquisition unit may prioritize extracting information about building collapses. Furthermore, it can prioritize extracting information about road damage. Moreover, it can prioritize extracting information related to human lives. Thus, it can extract important information in real time.

[0109] The data acquisition unit can integrate data collected by the drone's camera with other sensors (temperature, humidity, gas concentration, etc.) for multi-faceted analysis. Specific types and usage methods of these other sensors include temperature sensors, humidity sensors, and gas concentration sensors. For example, integrating data from a temperature sensor can help determine the spread of a fire. Furthermore, it can integrate data from a humidity sensor to assess the impact of floods. Moreover, it can integrate data from a gas concentration sensor to determine the presence of harmful gases. This allows for multi-faceted analysis by integrating with other sensors.

[0110] The data acquisition unit can infer the user's emotions and adjust the level of detail in the disaster situation footage based on this inference. Specific methods and benchmarks for inferring user emotions include emotion recognition algorithms and the sensors used. For example, the data acquisition unit captures the user's facial expressions using a camera and uses emotion recognition algorithms to infer emotions. Furthermore, the data acquisition unit can record the user's voice and use speech analysis technology to infer emotions. Additionally, the data acquisition unit can collect the user's physiological data (heart rate, skin conductance) using sensors and use emotion recognition algorithms to infer emotions. Therefore, the level of detail in the disaster situation footage can be adjusted based on the user's emotions. For example, when the user is anxious, detailed disaster situation footage is captured. When the user is calm, multiple disaster situations are captured efficiently. When the user is panicked, all disaster situations are captured in detail.

[0111] The data acquisition department can link data collected by drone cameras with a Geographic Information System (GIS) and display it on a map in real time. This includes the specific type of GIS and its usage, as well as the software and data format used. For example, the data acquisition department can display the disaster situation on the map in real time, confirming the affected area. Furthermore, it can also display the disaster situation on the map in real time, confirming the severity of the disaster. Moreover, it can determine the priority of rescue activities by displaying the disaster situation on the map in real time. Thus, it can display the disaster situation on the map in real time.

[0112] The data acquisition department can automatically upload data collected by drone cameras to cloud storage and share it with other disaster response teams. Specific details regarding the type and usage of cloud storage, including the cloud services used and data upload methods, are provided. For example, the data acquisition department can upload disaster situation data to cloud storage and share it with other disaster response teams. Furthermore, the data acquisition department can also upload and share disaster situation data in real time. Moreover, the data acquisition department can rapidly upload and share disaster situation data. Thus, it is possible to upload data to cloud storage and share it with other disaster response teams.

[0113] The data acquisition department can perform preliminary AI analysis on data collected by drone cameras to quickly extract important information. Specific methods and benchmarks for this preliminary AI analysis include the algorithms used and the data being analyzed. For example, the data acquisition department can use AI to quickly extract information about building collapses. Furthermore, it can use AI to quickly extract information about road damage. Moreover, the data acquisition department can use AI to quickly extract information related to human lives. Thus, it is possible to quickly extract important information through preliminary AI analysis.

[0114] The analysis unit can infer a user's emotions and adjust the display of the analysis results accordingly. Specific methods and benchmarks for inferring user emotions include emotion recognition algorithms and the sensors used. For example, the analysis unit captures the user's facial expressions using a camera and uses emotion recognition algorithms to infer emotions. Furthermore, the analysis unit can record the user's voice and use voice analysis technology to infer emotions. Additionally, the analysis unit can collect the user's physiological data (heart rate, skin conductance) using sensors and use emotion recognition algorithms to infer emotions. Therefore, the display of the analysis results can be adjusted according to the user's emotions. For example, when the user is nervous, a simple and highly visual display is provided. When the user is relaxed, a display containing detailed information is provided. When the user is anxious, a display highlighting key points is provided.

[0115] The analysis department is able to assess the reliability of disaster situation data during analysis and exclude data with low reliability. Specific evaluation criteria and methods for reliability include data source and data consistency. For example, the analysis department excludes data when the data collection source is unclear. Furthermore, the analysis department can exclude data when the data collection time is unclear. Even further, the analysis department can exclude data when the data collection method is unclear. Thus, by excluding data with low reliability, the accuracy of the analysis results is improved.

[0116] The analysis unit is capable of performing time-series analysis on disaster situation data during analysis to predict the progress of the disaster. Specific methods and benchmarks for time-series analysis include time intervals and algorithms used. For example, the analysis unit performs time-series analysis on disaster situation data to predict the progress of the disaster. Furthermore, the analysis unit can also perform time-series analysis on disaster situation data to predict the expansion of the disaster. Further, the analysis unit can also perform time-series analysis on disaster situation data to predict the convergence of the disaster. Thus, it is possible to perform time-series analysis on disaster situation data to predict the progress of the disaster.

[0117] The analysis unit can integrate disaster data with other disaster data (seismic waves, meteorological data, etc.) for comprehensive analysis. The specific types and acquisition methods of these other disaster data include seismic wave data and meteorological data. For example, the analysis unit can integrate seismic wave data to comprehensively analyze building collapse conditions. Furthermore, it can integrate with meteorological data to comprehensively analyze the impact of floods. Moreover, the analysis unit can integrate with other disaster data to comprehensively analyze the overall disaster situation. Thus, it can integrate with other disaster data for comprehensive analysis.

[0118] The analysis unit can infer a user's emotions and adjust the level of detail in the analysis results based on the inferred emotions. Specific methods and benchmarks for inferring user emotions include emotion recognition algorithms and the sensors used. For example, the analysis unit captures the user's facial expressions using a camera and uses emotion recognition algorithms to infer emotions. Furthermore, the analysis unit can record the user's voice and use voice analysis technology to infer emotions. Additionally, the analysis unit can collect the user's physiological data (heart rate, skin conductance) using sensors and use emotion recognition algorithms to infer emotions. Thus, the level of detail in the analysis results can be adjusted according to the user's emotions. For example, detailed analysis results are provided when the user is nervous; efficient analysis results are provided when the user is relaxed; and analysis results highlighting key points are provided when the user is anxious.

[0119] The analysis unit can cluster disaster data during analysis, classifying it according to the type and severity of the disaster. Specific methods and benchmarks for clustering are detailed, including the algorithms used and the number of clusters. For example, the analysis unit can cluster building collapse data and classify it according to the severity of the disaster. Furthermore, it can cluster road damage data and classify it according to the severity of the disaster. Moreover, it can cluster information related to human lives and classify it according to the severity of the disaster. Thus, it is possible to cluster disaster data and classify it according to the type and severity of the disaster.

[0120] The data analysis department can collaborate with other disaster response systems during analysis to conduct comprehensive disaster assessments. Specific methods and benchmarks for collaboration with other disaster response systems include data sharing protocols and the types of systems involved. For example, the analysis department can collaborate with fire departments and police to conduct comprehensive disaster assessments. Furthermore, it can collaborate with emergency medical teams to conduct comprehensive disaster assessments. Moreover, it can collaborate with local government disaster response headquarters to conduct comprehensive disaster assessments. Thus, it is possible to conduct comprehensive disaster assessments in collaboration with other disaster response systems.

[0121] The analysis unit can compare disaster data with past disaster data during analysis to identify specific similar disaster patterns. The specific methods and benchmarks for acquiring past disaster data include data sources and data types. For example, the analysis unit can compare data with past earthquake data to identify specific similar disaster patterns. Furthermore, it can compare data with past flood data to identify specific similar disaster patterns. Even further, it can compare data with past fire data to identify specific similar disaster patterns. Thus, it is possible to compare disaster data with past disaster data to identify specific similar disaster patterns.

[0122] The visualization department can infer a user's emotions and adjust the visualization display accordingly. Specific methods and benchmarks for inferring user emotions include emotion recognition algorithms and the sensors used. For example, the visualization department captures the user's facial expressions using a camera and uses emotion recognition algorithms to infer emotions. Furthermore, the visualization department can record the user's voice and use speech analysis technology to infer emotions. Additionally, the visualization department can collect the user's physiological data (heart rate, skin conductance) using sensors and use emotion recognition algorithms to infer emotions. Thus, the visualization display can be adjusted according to the user's emotions. For example, when the user is nervous, a simple and highly visual display is provided. When the user is relaxed, a display containing detailed information is provided. When the user is anxious, a display highlighting key points is provided.

[0123] The visualization department can display disaster data on a 3D map during visualization, using color coding to differentiate between different levels of damage. The specific methods for generating and displaying the 3D map include the software used and the data format. For example, the visualization department can display collapsed buildings on the 3D map, using color coding to differentiate between different levels of damage. Furthermore, it can display road damage on the 3D map, also using color coding to differentiate between different levels of damage. Moreover, the visualization department can display life-or-death information on the 3D map, again using color coding to differentiate between different levels of damage. Thus, disaster data can be displayed on a 3D map using color coding.

[0124] The visualization department enables interactive manipulation of disaster data and displays detailed information during visualization. Specific methods and benchmarks for interactive operation include the interface used and the scope of operation. For example, clicking on a specific building on a 3D map displays detailed damage information for that building. Similarly, clicking on a specific road on a 3D map displays detailed damage information for that road. Furthermore, clicking on a specific area on a 3D map displays detailed damage information for that area. This allows for interactive manipulation of disaster data and the display of detailed information.

[0125] The visualization department can overlay disaster damage data with other map data (roads, buildings, etc.) during visualization. The specific types and acquisition methods of the other map data include road data and building data. For example, the visualization department can overlay disaster damage data with road maps to confirm road damage. Furthermore, it can overlay disaster damage data with building maps to confirm building collapse. Moreover, it can overlay disaster damage data with topographic maps to confirm terrain changes. Thus, it is possible to overlay disaster damage data with other map data.

[0126] The visualization department can infer users' emotions and determine visualization priorities based on these inferences. Specific methods and benchmarks for inferring user emotions include emotion recognition algorithms and the sensors used. For example, the visualization department captures user facial expressions using a camera and uses emotion recognition algorithms to infer emotions. Furthermore, the visualization department can record user speech and use speech analysis technology to infer emotions. Additionally, the visualization department can collect user physiological data (heart rate, skin conductance) using sensors and use emotion recognition algorithms to infer emotions. Thus, the visualization priority can be determined based on the user's emotions. For example, when a user is anxious, the most important disaster situations are displayed first. When a user is calm, multiple disaster situations are displayed simultaneously and efficiently. When a user is panicked, all disaster situations are displayed simultaneously.

[0127] The visualization department can display disaster data along a timeline and animate its progress during visualization. Specific methods and benchmarks for timeline display include time intervals and display formats. For example, the visualization department can display disaster data along a timeline and animate its progress. Furthermore, it can also display disaster data along a timeline and animate the expansion of the disaster. Moreover, it can also display disaster data along a timeline and animate the containment of the disaster. Thus, it is possible to display disaster data along a timeline and animate its progress.

[0128] The Visualization Department can share disaster data with other disaster response teams during visualization, facilitating joint discussions on response measures. Specific methods and benchmarks for sharing with other disaster response teams are outlined, including data sharing protocols and the platforms used. For example, the Visualization Department can share disaster data with other disaster response teams to jointly discuss response measures. Furthermore, the Visualization Department can also share disaster data with other disaster response teams to quickly discuss response measures. Moreover, the Visualization Department can also share disaster data with other disaster response teams to discuss effective response measures. Thus, it is possible to share disaster data with other disaster response teams and jointly discuss response measures.

[0129] The visualization department enables the display of disaster data on mobile devices such as smartphones and tablets during visualization. Specific types and usage methods of mobile devices are included, specifically smartphones and tablets. For example, the visualization department displays disaster data on smartphones for easy on-site verification. Furthermore, the visualization department can also display disaster data on tablets to confirm detailed information. Moreover, the visualization department can display disaster data on mobile devices to enable rapid response. Thus, the ability to display disaster data on mobile devices facilitates on-site verification.

[0130] ===Hardware Implementation 1-1===

[0131] Each of the aforementioned elements—the activation unit, data acquisition unit, analysis unit, and visualization unit—is implemented, for example, by at least one of the smart device 14 and the data processing device 12. For instance, the activation unit activates the drone via the control unit 46A of the smart device 14. The data acquisition unit captures images of the disaster situation using the camera 42 of the smart device 14 and sends them to the data processing device 12. The analysis unit analyzes the disaster situation data using generative AI via the specific processing unit 290 of the data processing device 12. The visualization unit visualizes the analyzed data as a 3D map using the specific processing unit 290 of the data processing device 12.

[0132] ===Hardware Implementation 1-2===

[0133] Each of the aforementioned elements—the activation unit, data acquisition unit, analysis unit, and visualization unit—is implemented, for example, by at least one of the smart glasses 214 and the data processing device 12. For instance, the activation unit activates the drone via the control unit 46A of the smart glasses 214. The data acquisition unit captures images of the disaster situation via the camera 42 of the smart glasses 214 and sends them to the data processing device 12. The analysis unit analyzes the disaster situation data using generative AI via the specific processing unit 290 of the data processing device 12. The visualization unit visualizes the analyzed data as a 3D map via the specific processing unit 290 of the data processing device 12.

[0134] ===Hardware Implementation 1-3===

[0135] Each of the aforementioned elements—the activation unit, data acquisition unit, analysis unit, and visualization unit—is implemented, for example, by at least one of the head-mounted terminal 314 and the data processing device 12. For instance, the activation unit activates the drone via the control unit 46A of the head-mounted terminal 314. The data acquisition unit captures images of the disaster situation via the camera 42 of the head-mounted terminal 314 and sends them to the data processing device 12. The analysis unit analyzes the disaster situation data using generative AI via the specific processing unit 290 of the data processing device 12. The visualization unit visualizes the analyzed data as a 3D map via the specific processing unit 290 of the data processing device 12.

[0136] ===Hardware Implementation 1-4===

[0137] Each of the aforementioned elements—the activation unit, data acquisition unit, analysis unit, and visualization unit—is implemented, for example, by at least one of the robot 414 and the data processing device 12. For instance, the activation unit activates the drone via the control unit 46A of the robot 414. The data acquisition unit captures images of the disaster situation using the camera 42 of the robot 414 and sends them to the data processing device 12. The analysis unit analyzes the disaster situation data using generative AI via the specific processing unit 290 of the data processing device 12. The visualization unit visualizes the analyzed data as a 3D map using the specific processing unit 290 of the data processing device 12.

[0138] The system involved in this embodiment is not limited to the above examples. For example, various modifications can be made as follows.

[0139] The launch unit can assess the status of surrounding communication infrastructure during a disaster and prioritize launching drones in areas with uninterrupted communication. For example, in areas with damaged communication infrastructure, drones can take off from areas with uninterrupted communication to assess the damage. Furthermore, once communication infrastructure is restored, drone launches in that area can be resumed immediately. Moreover, drone flight routes can be optimized based on the status of communication infrastructure for efficient disaster assessment. Thus, the launch and flight routes of drones can be optimized by considering the status of communication infrastructure.

[0140] The data acquisition unit can compare data collected by drones with data collected before a disaster, automatically extracting differences in damage. For example, it can compare the data with the condition of buildings before the disaster to identify specific collapsed sections. Furthermore, it can compare the data with the condition of roads before the disaster to identify specific damaged sections. Additionally, it can compare the data with terrain data before the disaster to identify specific terrain changes. Thus, it can compare data before and after a disaster and automatically extract differences in damage.

[0141] The analysis unit can prioritize disaster data analysis based on the severity of the damage. For example, it can prioritize analyzing areas with severe building collapses, areas with severe road damage, and areas containing information related to human lives. This prioritized analysis based on disaster severity enables a rapid response.

[0142] When visualizing disaster data, the visualization department can prioritize displaying the most recent disaster situation based on the user's location. For example, if the user is on-site, the disaster situation in their surroundings will be displayed first. Furthermore, when the user is remotely viewing the data, they can also get a bird's-eye view of the overall disaster situation. Moreover, the displayed disaster situation can be updated in real time based on the user's movement. Thus, it is possible to prioritize displaying the most recent disaster situation based on the user's location.

[0143] The startup unit can anticipate the user's emotions and adjust the drone's flight altitude accordingly. For example, when the user is anxious, it flies at a low altitude to assess the detailed extent of the disaster. When the user is calm, it flies at a high altitude to assess the broader damage. When the user is panicked, it flies at the optimal altitude to quickly assess the damage. Thus, the drone's flight altitude can be adjusted based on the user's emotions.

[0144] The data acquisition unit can infer the user's emotions and adjust the drone's flight speed accordingly. For example, when the user is anxious, it flies at a low speed to confirm the detailed extent of the disaster. When the user is calm, it flies at a high speed to confirm the large-scale extent of the disaster. When the user is panicked, it flies at the optimal speed to quickly confirm the extent of the disaster. Thus, the drone's flight speed can be adjusted based on the user's emotions.

[0145] The analysis department can infer a user's emotions and adjust the notification method for the analysis results based on these inferences. For example, when a user is nervous, a concise and highly visual notification is provided. When a user is relaxed, a notification containing detailed information is provided. When a user is anxious, a notification highlighting key points is provided. Thus, the notification method for analysis results can be adjusted according to the user's emotions.

[0146] The visualization department can infer users' emotions and adjust the colors of the visualizations accordingly. For example, when users are tense, the visualizations are displayed in calming colors; when users are relaxed, they are displayed in vibrant colors; and when users are anxious, they are displayed in highly visible colors. Thus, the visualization colors can be adjusted based on the user's emotions.

[0147] The visualization department can infer users' emotions and adjust the layout of the visualizations accordingly. For example, when users are nervous, a simple and highly visual layout is provided. When users are relaxed, a layout containing detailed information is provided. When users are anxious, a layout highlighting key points is provided. Thus, the visualization layout can be adjusted based on the user's emotions.

[0148] When analyzing disaster data, the analysis department can import data from other disaster response systems in real time for comprehensive disaster assessment. For example, it can import data from fire departments and police to comprehensively assess building collapse conditions. Furthermore, it can import data from emergency medical teams to comprehensively assess the condition of the injured. Additionally, it can import data from local government disaster response headquarters to comprehensively assess the overall disaster situation. Thus, it can import data from other disaster response systems in real time for comprehensive disaster assessment.

[0149] The following is a brief description of the processing flow of Implementation Method 2.

[0150] Step 1: The launch unit starts the drones. For example, the launch unit can start multiple drones simultaneously within one hour of a disaster. In addition, the launch unit can also start drones manually, remotely, or by a timer.

[0151] Step 2: The data acquisition unit processes the data collected by the drone initiated by the launch unit. For example, the data acquisition unit uses a drone equipped with a camera to photograph the disaster situation. The specific specifications and performance of the camera include resolution, field of view, zoom function, etc.

[0152] Step 3: The analysis department analyzes the data collected by the data acquisition department. For example, the analysis department uses generative AI to analyze disaster situation data. Generative AI analyzes the data using specific algorithms and training datasets.

[0153] Step 4: The Visualization Department visualizes the data analyzed by the Analysis Department. For example, the Visualization Department uses generative AI to visualize the disaster situation in the form of a 3D map. The generation method and display format of the 3D map include the software used and the data format.

[0154] The specific processing unit 290 sends the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires voice representing the user's input to the result of the specific processing. The control unit 46A sends the voice data representing the user's input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0155] Data generation model 58 is what is known as generative AI (Artificial Intelligence). An example of data generation model 58 includes ChatGPT (registered trademark) (Internet search).<URL:https: / / openai.com / blog / chatgpt> Generative AI, such as data generation model 58, is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, as well as inference data such as speech data (representing speech), text data (representing text), and image data (representing images). The data generation model 58 infers the input inference data based on the instructions shown in the prompts and outputs the inference results in the form of speech data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the aforementioned specific processing while using the data generation model 58. The data generation model 58 can also be a fine-tuned model to output inference results from prompts without instructions; in this case, the data generation model 58 can output inference results from prompts without instructions. The data processing device 12, etc., includes various data generation models 58, including AI other than generative AI. AI other than generative AI includes, but is not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes. Furthermore, AI can also act as an AI agent. Additionally, when the processing described above is performed by AI, this processing may be partially or entirely performed by AI, but is not limited to this example. Moreover, processing performed by AI, including generative AI, can be replaced by rule-based processing, and vice versa.

[0156] Furthermore, the processing performed by the aforementioned data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it can also be executed jointly by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or external devices, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or external devices.

[0157] The correspondence between the various parts and the devices or control units is not limited to the examples mentioned above, and various changes can be made.

[0158] Second Implementation Method

[0159] Figure 3 An example of the configuration of the data processing system 210 according to the second embodiment is shown.

[0160] like Figure 3 As shown, the data processing system 210 includes a data processing device 12 and smart glasses 214. One example of the data processing device 12 is a server.

[0161] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. An example of the network 54 includes a WAN and / or LAN, etc.

[0162] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0163] Microphone 238 receives user commands by receiving the user's voice. Microphone 238 captures the user's voice, converts the captured sound into speech data, and outputs it to processor 46. Speaker 240 outputs sound according to commands from processor 46.

[0164] The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, used to photograph the user's surroundings (e.g., the shooting range defined by an angle of view equivalent to the field of vision of an average healthy person).

[0165] Communication I / F 44 is connected to network 54. Communication I / F 44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F 44 and 26 is performed in a secure state.

[0166] Figure 4 An example of the main functions of the data processing device 12 and the smart glasses 214 is shown. Figure 4 As shown, in the data processing device 12, specific processing is performed by the processor 28. The memory 32 stores a specific processing program 56.

[0167] The processor 28 reads a specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is implemented by the processor 28 operating as a specific processing unit 290 based on the specific processing program 56 executed on the RAM 30.

[0168] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. The emotion inference function using the emotion-specific model 59 (emotion-specific function) includes inferring and predicting the user's emotions, performing various inferences and predictions related to the user's emotions, but is not limited to this example. Furthermore, emotion inference and prediction also include, for example, emotion analysis (analysis).

[0169] In the smart glasses 214, specific processing is performed by the processor 46. A specific processing program 60 is stored in the memory 50. The processor 46 reads the specific processing program 60 from the memory 50 and executes the read specific processing program 60 on the RAM 48. Specific processing is implemented by the processor 46 operating as a control unit 46A based on the specific processing program 60 executed on the RAM 48. Furthermore, the smart glasses 214 may also have the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and use these models to perform the same processing as the specific processing unit 290.

[0170] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. In addition, the data processing device 12 may be a server device or a user-owned terminal device (e.g., a mobile phone, robot, home appliance, etc.).

[0171] The specific processing unit 290 sends the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires voice input representing the user's input to the specific processing result. The control unit 46A sends the voice data representing the user's input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0172] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 includes generative AIs such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing instructions is input to the data generation model 58, along with inference data such as speech data representing speech, text data representing text, and image data representing images. The data generation model 58 infers the inference data based on the instructions shown in the prompt and outputs the inference result in the form of data such as speech data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the aforementioned specific processing while using the data generation model 58. The data generation model 58 can also be a fine-tuned model to output inference results from prompts without instructions; in this case, the data generation model 58 can output inference results from prompts without instructions. The data processing device 12, etc., includes various data generation models 58, including AI other than generative AI. AI other than generative AI includes, but is not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes. Furthermore, AI can also act as an AI agent. Additionally, when the processing described above is performed by AI, this processing may be partially or entirely performed by AI, but is not limited to this example. Moreover, processing performed by AI, including generative AI, can be replaced by rule-based processing, and vice versa.

[0173] The data processing system 210 of the second embodiment performs the same processing as the data processing system 10 of the first embodiment. The processing performed by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it can also be executed jointly by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or external devices, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or external devices.

[0174] The correspondence between the various parts and the devices or control units is not limited to the examples mentioned above, and various changes can be made.

[0175] Third Implementation Method

[0176] Figure 5 An example of the configuration of the data processing system 310 according to the third embodiment is shown.

[0177] like Figure 5 As shown, the data processing system 310 includes a data processing device 12 and a head-mounted terminal 314. An example of the data processing device 12 is a server.

[0178] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. An example of the network 54 includes a WAN and / or LAN, etc.

[0179] The head-mounted terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0180] Microphone 238 receives user commands by receiving the user's voice. Microphone 238 captures the user's voice, converts the captured sound into speech data, and outputs it to processor 46. Speaker 240 outputs sound according to commands from processor 46.

[0181] The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, used to photograph the user's surroundings (e.g., the shooting range defined by an angle of view equivalent to the field of vision of an average healthy person).

[0182] Communication I / F 44 is connected to network 54. Communication I / F 44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F 44 and 26 is performed in a secure state.

[0183] Figure 6 An example of the main functions of the data processing device 12 and the head-mounted terminal 314 is shown. Figure 6 As shown, in the data processing device 12, specific processing is performed by the processor 28. The memory 32 stores a specific processing program 56.

[0184] The processor 28 reads a specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is implemented by the processor 28 operating as a specific processing unit 290 based on the specific processing program 56 executed on the RAM 30.

[0185] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. The emotion inference function using the emotion-specific model 59 (emotion-specific function) includes inferring and predicting the user's emotions, performing various inferences and predictions related to the user's emotions, but is not limited to this example. Furthermore, emotion inference and prediction also include, for example, emotion analysis (analysis).

[0186] In the head-mounted terminal 314, specific processing is performed by the processor 46. A specific program 60 is stored in the memory 50. The processor 46 reads the specific program 60 from the memory 50 and executes the read specific program 60 on the RAM 48. Specific processing is implemented by the processor 46 operating as a control unit 46A based on the specific program 60 executed on the RAM 48. Furthermore, the head-mounted terminal 314 may also have the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and use these models to perform the same processing as the specific processing unit 290.

[0187] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. In addition, the data processing device 12 may be a server device or a user-owned terminal device (e.g., a mobile phone, robot, home appliance, etc.).

[0188] The specific processing unit 290 sends the result of the specific processing to the head-mounted terminal 314. In the head-mounted terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires voice representing the user's input to the specific processing result. The control unit 46A sends the voice data representing the user's input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0189] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 includes generative AIs such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing instructions is input to the data generation model 58, along with inference data such as speech data representing speech, text data representing text, and image data representing images. The data generation model 58 infers the inference data based on the instructions shown in the prompt and outputs the inference result in the form of data such as speech data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the aforementioned specific processing while using the data generation model 58. The data generation model 58 can also be a fine-tuned model to output inference results from prompts without instructions; in this case, the data generation model 58 can output inference results from prompts without instructions. The data processing device 12, etc., includes various data generation models 58, including AI other than generative AI. AI other than generative AI includes, but is not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes. Furthermore, AI can also act as an AI agent. Additionally, when the processing described above is performed by AI, this processing may be partially or entirely performed by AI, but is not limited to this example. Moreover, processing performed by AI, including generative AI, can be replaced by rule-based processing, and vice versa.

[0190] The data processing system 310 of the third embodiment performs the same processing as the data processing system 10 of the first embodiment. The processing performed by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the head-mounted terminal 314, but it can also be executed jointly by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the head-mounted terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the head-mounted terminal 314 or external devices, and the head-mounted terminal 314 acquires or collects information required for processing from the data processing device 12 or external devices.

[0191] The correspondence between the various parts and the devices or control units is not limited to the examples mentioned above, and various changes can be made.

[0192] Fourth Implementation Method

[0193] Figure 7 An example of the configuration of the data processing system 410 according to the fourth embodiment is shown.

[0194] like Figure 7 As shown, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0195] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. An example of the network 54 includes a WAN and / or LAN, etc.

[0196] Robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control object 443. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, and control object 443 are also connected to the bus 52.

[0197] Microphone 238 receives user commands by receiving the user's voice. Microphone 238 captures the user's voice, converts the captured sound into speech data, and outputs it to processor 46. Speaker 240 outputs sound according to commands from processor 46.

[0198] The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, used to photograph the user's surroundings (e.g., the shooting range defined by an angle of view equivalent to the field of vision of an average healthy person).

[0199] Communication I / F 44 is connected to network 54. Communication I / F 44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F 44 and 26 is performed in a secure state.

[0200] The controlled object 443 includes a display device, LEDs for the eyes, and motors for driving the arms, hands, and feet. The posture and movements of the robot 414 are controlled by controlling the motors for the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. In addition, facial expressions of the robot 414 can also be expressed by controlling the illumination state of the LEDs for the robot 414's eyes.

[0201] Figure 8 An example of the main functions of the data processing device 12 and the robot 414 is shown. Figure 8 As shown, in the data processing device 12, specific processing is performed by the processor 28. The memory 32 stores a specific processing program 56.

[0202] The processor 28 reads a specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is implemented by the processor 28 operating as a specific processing unit 290 based on the specific processing program 56 executed on the RAM 30.

[0203] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. The emotion inference function using the emotion-specific model 59 (emotion-specific function) includes inferring and predicting the user's emotions, performing various inferences and predictions related to the user's emotions, but is not limited to this example. Furthermore, emotion inference and prediction also include, for example, emotion analysis (analysis).

[0204] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in memory 50. Processor 46 reads the specific program 60 from memory 50 and executes the read specific program 60 on RAM 48. Specific processing is achieved by processor 46 acting as control unit 46A based on the specific program 60 executed on RAM 48. Furthermore, robot 414 may also have the same data generation model and emotion-specific model as data generation model 58 and emotion-specific model 59, and use these models to perform the same processing as specific processing unit 290.

[0205] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. In addition, the data processing device 12 may be a server device or a user-owned terminal device (e.g., a mobile phone, robot, home appliance, etc.).

[0206] The specific processing unit 290 sends the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires voice representing the user's input regarding the result of the specific processing. The control unit 46A sends the voice data representing the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0207] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 includes generative AIs such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing instructions is input to the data generation model 58, along with inference data such as speech data representing speech, text data representing text, and image data representing images. The data generation model 58 infers the inference data based on the instructions shown in the prompt and outputs the inference result in the form of data such as speech data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the aforementioned specific processing while using the data generation model 58. The data generation model 58 can also be a fine-tuned model to output inference results from prompts without instructions; in this case, the data generation model 58 can output inference results from prompts without instructions. The data processing device 12, etc., includes various data generation models 58, including AI other than generative AI. AI other than generative AI includes, but is not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes. Furthermore, AI can also act as an AI agent. Additionally, when the processing described above is performed by AI, this processing may be partially or entirely performed by AI, but is not limited to this example. Moreover, processing performed by AI, including generative AI, can be replaced by rule-based processing, and vice versa.

[0208] The data processing system 410 of the fourth embodiment performs the same processing as the data processing system 10 of the first embodiment. The processing performed by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it can also be executed jointly by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or external devices, and the robot 414 acquires or collects information required for processing from the data processing device 12 or external devices.

[0209] The correspondence between the various parts and the devices or control units is not limited to the examples mentioned above, and various changes can be made.

[0210] Furthermore, the emotion-specific model 59, serving as an emotion engine, can determine the user's emotion based on a specific mapping. Specifically, the emotion-specific model 59 can determine the user's emotion based on an emotion graph that serves as a specific mapping (see...). Figure 9 The robot's emotions can be determined by the emotion-specific model 59. In addition, the emotion-specific model 59 can also determine the robot's emotions in the same way, and the specific processing unit 290 can also perform specific processing using the robot's emotions.

[0211] Figure 9 This is a diagram representing an emotion map 400 that maps various emotions. In the emotion map 400, emotions are arranged radially from the center in concentric circles. The closer to the center of the concentric circles, the more primitive the emotion is. Further out on the concentric circles, emotions are arranged representing states or actions arising from mood. Emotion is a concept that includes both feelings and mental states. To the left of the concentric circles, emotions generated by reactions occurring in the brain are arranged roughly. To the right of the concentric circles, emotions guided by situational judgments are arranged roughly. Above and below the concentric circles, emotions generated by reactions occurring in the brain and guided by situational judgments are arranged roughly. Furthermore, the emotion of "pleasure" is arranged above the concentric circles, and the emotion of "unpleasantness" is arranged below. Thus, in the emotion map 400, various emotions are mapped according to the structure of emotion generation, while easily generated emotions are mapped nearby.

[0212] These emotions are distributed at the 3 o'clock position on the Emotion Chart 400, and usually fluctuate between peace and unease. In the right half of the Emotion Chart 400, because situational awareness is more dominant than internal feelings, it gives a sense of calm.

[0213] The inner side of the emotion diagram 400 represents the mind, and the outer side of the emotion diagram 400 represents actions. Therefore, the further you go to the outer side of the emotion diagram 400, the more the emotion can be seen (manifested in actions).

[0214] Here, human emotions are based on a balance of various factors such as posture and blood sugar levels. When these balances deviate from the ideal, it indicates unhappiness; when they approach the ideal, it indicates pleasure. In robots, cars, and motorcycles, emotions can also be created based on a balance of factors such as posture and remaining battery power. When these balances deviate from the ideal, it indicates unhappiness; when they approach the ideal, it indicates pleasure. Emotion maps can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Speech Emotion Recognition and Brain Physiological Signal Analysis Systems for Emotion, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). In the left half of the emotion map, emotions belonging to the "Reaction" domain, where sensation is dominant, are arranged. Furthermore, in the right half of the emotion map, emotions belonging to the "Situation" domain, where situational cognition is dominant, are arranged.

[0215] The emotion map defines two types of emotions that promote learning. One is a negative emotion located near the middle of "repentance" or "reflection" on the situation side. That is, when the robot experiences negative emotions such as "I never want to feel this way again" or "I never want to be scolded again." The other is a positive emotion located near "desire" on the response side. That is, when the robot experiences positive feelings such as "wanting more" or "wanting to know more."

[0216] The emotion-specific model 59 feeds user input into a pre-learned neural network to obtain emotion values ​​representing each emotion shown in the emotion graph 400, and determines the user's emotion. This neural network is pre-learned based on multiple learning data sets that combine user input with emotion values ​​representing each emotion shown in the emotion graph 400. Furthermore, this neural network is learned to... Figure 10 As shown in sentiment graph 900, nearby sentiment values ​​are similar to each other. Figure 10 Examples show that multiple emotions such as "peace of mind", "stability", and "reassurance" have similar emotional values.

[0217] In the above embodiments, a specific processing is described by a single computer 22, but the technology disclosed herein is not limited to this, and distributed processing by multiple computers, including computer 22, is also possible.

[0218] In the above embodiments, an example of storing a specific processing program 56 in memory 32 is illustrated, but the technology disclosed herein is not limited thereto. For example, the specific processing program 56 may also be stored in a portable computer-readable non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed into the computer 22 of the data processing device 12. The processor 28 performs specific processing according to the specific processing program 56.

[0219] Alternatively, the specific processing program 56 can be stored in a storage device such as a server connected to the data processing device 12 via a network 54, and the specific processing program 56 can be downloaded and installed into the computer 22 upon request from the data processing device 12.

[0220] Furthermore, it is not necessary to store the entire specific process 56 in a storage device such as a server connected to the data processing device 12 via the network 54, nor is it necessary to store the entire specific process 56 in the memory 32; a portion of the specific process 56 may also be stored.

[0221] As a hardware resource for performing specific processing, various processors can be used. For example, a CPU is a general-purpose processor that functions as a hardware resource for performing specific processing by executing software, i.e., programs. Additionally, a dedicated circuit can be listed as a processor; it is a processor with a circuit structure specifically designed for performing specific processing, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit). Every processor has built-in or connected memory, and every processor executes specific processing by using memory.

[0222] The hardware resources for performing a specific process can consist of one of these various processors, or a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Furthermore, the hardware resources for performing a specific process can also be a single processor.

[0223] As an example of a single processor, the first type consists of a combination of one or more CPUs and software, which functions as a hardware resource to perform specific processing. The second type uses a processor, such as a System-on-a-chip (SoC), which implements the entire system functionality, including multiple hardware resources for performing specific processing, using a single IC chip. In this case, the specific processing is implemented using one or more of the aforementioned processors as hardware resources.

[0224] Furthermore, as the hardware architecture of these various processors, more specifically, circuits combining semiconductor elements and other circuit components can be used. Moreover, the specific process described above is merely an example. Therefore, it goes without saying that, without departing from the main point, unnecessary steps can be removed, new steps can be added, or the processing order can be changed.

[0225] Furthermore, although the above examples have been described in terms of first to fourth embodiments, some or all of these embodiments can be combined. Additionally, the smart device 14, smart glasses 214, head-mounted terminal 314, and robot 414 are just examples and can be combined separately, or other devices may be used. Furthermore, although the above examples have been described in terms of morphological example 1 and morphological example 2, these can also be combined.

[0226] The foregoing descriptions and illustrations are detailed explanations of the parts covered by this disclosure and are merely one example of this disclosure. For instance, the descriptions of the above-described structure, function, role, and effect are just one example of the structure, function, role, and effect of the parts covered by this disclosure. Therefore, it goes without saying that, without departing from the spirit of this disclosure, unnecessary parts can be deleted, new elements can be added, or replacements can be made to the foregoing descriptions and illustrations. Furthermore, to avoid confusion and facilitate understanding of the parts covered by this disclosure, explanations of technical common sense that does not require special explanation for implementing this disclosure have been omitted from the foregoing descriptions and illustrations.

[0227] All documents, patent applications and technical standards described in this specification are incorporated herein by reference as if they were specifically and individually described as incorporated by reference.

Claims

1. A system, characterized in that, include: The starter unit is used to launch the drone. A data acquisition unit is used to process data collected by the drone initiated by the launch unit; The parsing unit is used to parse the data acquired by the data acquisition unit; The visualization unit is used to visualize the data parsed by the analysis unit.

2. The system as described in claim 1, characterized in that, The launch unit can simultaneously launch multiple drones within one hour of a disaster.

3. The system as described in claim 1, characterized in that, The data acquisition unit uses drones equipped with cameras to photograph the disaster situation.

4. The system as described in claim 1, characterized in that, The analysis unit utilizes generative AI to analyze disaster data.

5. The system as described in claim 1, characterized in that, The visualization department uses generative AI to visualize the disaster situation in the form of a 3D map.

6. The system as described in claim 1, characterized in that, The analysis unit processes disaster data rapidly.

7. The system as described in claim 1, characterized in that, The visualization department monitors the disaster situation on-site in real time.

8. The system as described in claim 1, characterized in that, The activation unit anticipates the user's emotions and adjusts the drone's activation timing based on those anticipations.

Images