System

Abstract

A system is provided.SOLUTION: A system comprising: means for capturing an image of a ground surface using an unmanned aerial vehicle equipped with a camera and a communication function; means for receiving and storing image data transmitted from the unmanned aerial vehicle; generative artificial intelligence means for analyzing the stored image data and determining a state of a terrain or a road; means for generating a passage availability map based on passage availability information determined by the generative artificial intelligence; and means for providing the generated passage availability map.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When natural disasters such as earthquakes, tsunamis, and landslides occur, transportation infrastructure is severely damaged, often making vehicle movement difficult. In such situations, rapid evacuation of victims and land transportation of relief supplies are required, but the current situation is that it is difficult to quickly and accurately obtain information on passable roads. Conventional methods take time to determine whether a road is passable, and there is an issue in that they cannot provide information in real time. [Means for solving the problem]

[0005] The present invention is a system that includes a means for capturing images of the ground using an unmanned aerial vehicle equipped with a camera and communication capabilities, a means for receiving and storing image data transmitted from the unmanned aerial vehicle, a generating AI means for analyzing the stored image data and determining the condition of terrain and roads, a means for generating a passability map based on the passability information determined by the generating AI, and a means for providing the generated passability map. This system provides real-time information on road passability, enabling smooth evacuation and logistics support during disasters. Furthermore, by using deep learning technology, the generating AI can accurately determine the state of road damage and the presence of obstacles from the image data. The passability map displays passable roads in a predetermined first color, impassable roads in a predetermined second color, and roads requiring caution in a predetermined third color, providing users with intuitively easy-to-understand information.

[0006] Understood. Below are definitions of important words.

[0007] An "unmanned aerial vehicle" is a flying device capable of remote control or autonomous flight and equipped with a camera and communication capabilities.

[0008] "Image data" is digital data containing visual information of the ground and roads photographed using a camera on an unmanned aerial vehicle.

[0009] "Generative AI" is an AI technology that analyzes image data and automatically determines road conditions and whether or not a road is passable.

[0010] "Deep learning technology" is a form of artificial intelligence that uses multi-layered neural networks to perform complex data analysis and pattern recognition.

[0011] "Passability information" is information about whether a road is passable, impassable, or requires caution.

[0012] A "passability map" is a visual display of passability information on a map, provided in a format that is intuitively easy for users to understand.

[0013] The "storing means" is a memory or database for storing the received image data in a digital format.

[0014] The "means for providing" refers to a web service or application for delivering the generated passable / unpassable map to users in a viewable format. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] ---

[0037] The present invention is a system for quickly determining whether traffic infrastructure is passable during a disaster, and is configured as follows.

[0038] This system first uses an unmanned aerial vehicle (drone) equipped with a camera and communication functions to capture images of the ground surface. The unmanned aerial vehicle automatically flies over a designated area according to a flight plan, taking high-resolution photos of the ground surface and roads at regular intervals. The captured photo data is sent to a server in real time.

[0039] The server receives image data sent from the unmanned aerial vehicle and stores it in a database in a specific format. The received image data is analyzed sequentially and input into the generative AI. The generative AI then analyzes the image data using deep learning technology to automatically determine the condition of the terrain and roads. This allows it to accurately detect the extent of road damage and the presence or absence of obstacles, and obtain information on whether the road is passable.

[0040] Based on the analysis results, the server generates a passable / unpassable map. This map displays passable roads in a predetermined first color (e.g., green), impassable roads in a predetermined second color (e.g., red), and roads requiring caution in a predetermined third color (e.g., yellow). The generated passable / unpassable map is immediately provided to the user via a web service or dedicated application.

[0041] Next, the program processing of this system will be explained in natural language.

[0042] 1. Setting up a flight plan:

[0043] Using an application on a dedicated device, the user refers to a map of the disaster area and specifies the flight path and shooting points for the unmanned aerial vehicle.

[0044] 2. Automatic flight and photography of unmanned aerial vehicles:

[0045] The terminal transmits the set flight plan to the unmanned aerial vehicle.

[0046] The unmanned aerial vehicle follows a specified flight plan, automatically flies over a designated area, and periodically takes photographs of the ground and roads.

[0047] The unmanned aerial vehicle transmits the captured photo data to a server in real time.

[0048] 3. Receiving and storing data:

[0049] The server receives image data sent from the unmanned aerial vehicle and stores it in a database in a specific format.

[0050] 4. Image Analysis:

[0051] The server sequentially inputs the stored image data into the generative artificial intelligence, which then analyzes the road conditions using deep learning technology.

[0052] For example, if a landslide is visible in some images, the generative AI will determine that road is "impassable."

[0053] 5. Generate a passability map:

[0054] Based on the analysis results, the server generates a passability map that visually displays passability information on a map.

[0055] The road availability map shows passable roads in green, impassable roads in red, and roads requiring caution in yellow.

[0056] 6. Information provision:

[0057] The server publishes the generated passability map to web services or dedicated applications and provides it to users.

[0058] Users can use a web browser or dedicated application to access real-time updated road safety maps and plan evacuation routes and movements.

[0059] As a specific example, immediately after a disaster occurs, an unmanned aerial vehicle flies over the affected area and takes photos of the roads. The image data is sent to a server, and the artificial intelligence analyzes the road for damage and obstacles. For example, if a landslide is confirmed at a certain point on a major highway, the road will be determined to be "impassable" and displayed in red on the passability map. If the road is confirmed to be safe at other points, those parts will be displayed in green. In this way, the latest passability information is provided in real time, helping users to travel quickly and safely.

[0060] As described above, the present invention is a system that quickly and accurately determines whether transportation infrastructure is passable during a disaster, and efficiently supports the evacuation of disaster victims and rescue operations.

[0061] The processing flow will be explained below.

[0062] ---

[0063] Step 1:

[0064] Users use an application on a dedicated device to refer to a map of the disaster-affected area and specify the flight path and shooting points for the unmanned aerial vehicle (drone).

[0065] Step 2:

[0066] The terminal transmits the set flight plan to the unmanned aerial vehicle, and the unmanned aerial vehicle automatically flies in the designated area according to the received flight plan.

[0067] Step 3:

[0068] The drone periodically takes high-resolution photographs of the ground and roads along its flight path.

[0069] Step 4:

[0070] The unmanned aerial vehicle transmits the captured photo data to a server in real time.

[0071] Step 5:

[0072] The server receives the image data sent from the unmanned aerial vehicle and stores it in a database in a specific format.

[0073] Step 6:

[0074] The server sequentially inputs the stored image data into the generative artificial intelligence.

[0075] Step 7:

[0076] The server uses generative artificial intelligence to analyze the image data, using deep learning technology to automatically determine the terrain and road conditions.

[0077] Step 8:

[0078] The server then determines whether a road is passable based on the analysis results. For example, it may classify a point in the image where a landslide has been confirmed as "impassable."

[0079] Step 9:

[0080] The server generates a passability map based on the passability information.

[0081] Step 10:

[0082] The server makes the passability map available to users via a web service or dedicated application.

[0083] Step 11:

[0084] Users can access the road closure map using a web browser or a dedicated application, and create evacuation routes and travel plans based on information updated in real time.

[0085] ---

[0086] These are the specific processing steps of the program for this system. This flow enables rapid and accurate understanding of the state of transportation infrastructure during a disaster, and supports effective evacuation and relief efforts.

[0087] Example 1

[0088] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0089] With conventional systems, it is difficult to quickly and accurately determine whether roads are passable during a disaster. Specifically, there is a lack of means to analyze road conditions in real time and provide that information to users immediately. As a result, evacuation plans and rescue operations cannot be carried out quickly in an emergency, and there is a risk that the damage will worsen.

[0090] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0091] In this invention, the server includes a means for a user to set a flight plan for the unmanned aerial vehicle via a dedicated terminal, a means for the unmanned aerial vehicle to automatically fly according to the set flight plan and transmit captured data to the server in real time, and a means for presenting a passability map at startup based on the passability information generated by the server. This makes it possible to quickly and accurately determine whether roads are passable during a disaster and provide the information to the user in real time.

[0092] An "unmanned aerial vehicle" is a device equipped with a camera and communication capabilities that flies automatically according to a specified flight plan.

[0093] "Image data" refers to data that includes photographic information of the ground and roads taken by unmanned aerial vehicles.

[0094] "Analysis" is the process of determining the terrain and road conditions based on image data stored on the server.

[0095] "Generative AI" is an AI system that uses image data to analyze terrain and road conditions using deep learning technology.

[0096] A "flight plan" is information about the flight path and photographing points of an unmanned aerial vehicle that is set by the user via a dedicated terminal.

[0097] A "dedicated terminal" is a device that a user uses to set a flight plan for an unmanned aerial vehicle, and includes smartphones, tablets, etc.

[0098] The "server" is a computer system that receives and stores image data transmitted from unmanned aerial vehicles, then analyzes the data and provides the generated passable / unpassable map.

[0099] A "passable roads map" is a map that visually displays passable roads, impassable roads, and roads requiring caution, using different colors based on the analysis results.

[0100] "Real-time" means that image data captured by an unmanned aerial vehicle is sent to a server almost immediately, with an extremely short time lag for analysis and provision of information.

[0101] "User" refers to a person who uses a dedicated terminal to set up a flight plan for an unmanned aerial vehicle and view a passability map.

[0102] To specifically implement this invention, an unmanned aerial vehicle (drone), a dedicated terminal (smartphone, tablet, etc.), and a server are required. The purpose of this system is to quickly determine whether transportation infrastructure is passable during a disaster and provide users with information in real time.

[0103] System Overview

[0104] Hardware Configuration

[0105] Unmanned aerial vehicle (drone): Equipped with a camera and communication capabilities, it flies automatically according to a specified flight plan.

[0106] Dedicated terminal: A device used by a user to set up a flight plan for an unmanned aerial vehicle, including a smartphone or tablet.

[0107] Server: A computer system that receives, stores, and analyzes image data sent from unmanned aerial vehicles.

[0108] Software Configuration

[0109] Dedicated terminal application: An application that allows users to set up flight plans.

[0110] Server-side analysis software: Software that uses deep learning technology to analyze image data.

[0111] Web service / dedicated application: Software that provides the generated accessibility map to the user.

[0112] Processing flow

[0113] Setting up a flight plan

[0114] The user launches the application on the dedicated device and displays a map of the affected area. The user then specifies the flight path and shooting points for the unmanned aircraft. For example, the user might "set a flight path from the center of Sendai City to the main road in the east."

[0115] Automated flight and data transmission

[0116] The terminal sends the set flight plan to the unmanned aerial vehicle. The unmanned aerial vehicle automatically flies over the specified area according to the specified flight plan, periodically taking photos of the ground and roads. The captured photo data is sent to the server in real time. Example: The unmanned aerial vehicle "takes photos every 10 minutes while flying along the specified route, and immediately sends the image data to the server after taking the photos."

[0117] Receiving and storing data

[0118] The server receives image data sent from the unmanned aerial vehicle in real time and stores it in a database in a specific format. Example: The server "receives image data in real time and stores the image data in a database with date and location information."

[0119] Image analysis and generation of passability maps

[0120] The server sequentially inputs the stored image data into generative artificial intelligence (AI), which uses deep learning technology to analyze the condition of the road. The AI ​​analyzes the image data and determines whether the road is damaged or has any obstacles. For example, if a landslide is confirmed in some of the images, the AI ​​will determine that the road is "impassable." The server generates a passable / unpassable map based on the analysis results. The passable / unpassable map shows passable roads in green, impassable roads in red, and roads requiring caution in yellow. Example: The server "generates a map that shows areas where landslides have been confirmed in red, and other safe areas in green."

[0121] Providing information

[0122] The server publishes the generated route availability map to a web service or dedicated application and provides it to users. Users can use a web browser or dedicated application to access the route availability map, which is updated in real time, and plan evacuation routes and movements. Example: A user can "check the traffic conditions in real time and plan evacuation routes and movements."

[0123] Prompt Sentence Examples

[0124] "After a disaster occurs in central Sendai, please use a drone to photograph the traffic conditions of major roads in the eastern part of the city, send the data to a server for analysis, and generate a map of whether roads are passable or not."

[0125] As described above, this system quickly and accurately determines whether transportation infrastructure is passable during a disaster, and provides users with useful information in real time.

[0126] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0127] Step 1: Set up a flight plan

[0128] User:

[0129] The user launches the application on the dedicated device and displays a map of the disaster area.

[0130] The user specifies the flight path and shooting points of the unmanned aerial vehicle on the map.

[0131] Input: Map information of the disaster area, flight path and photo points

[0132] Output: Flight plan data for unmanned aerial vehicles

[0133] Specific actions: The user "tap the tablet screen to mark the shooting point of the unmanned aerial vehicle," "confirm the flight plan, and press the start button."

[0134] Step 2: Submit your flight plan

[0135] Device:

[0136] The configured flight plan is transmitted to the unmanned aerial vehicle.

[0137] Input: Flight plan data

[0138] Output: Flight plan sent to the unmanned aerial vehicle

[0139] Specific operation: The device "wirelessly transmits flight plan data to the unmanned aerial vehicle" and "displays a notification confirming receipt."

[0140] Step 3: Automated flight of the unmanned aerial vehicle

[0141] Unmanned Aerial Vehicles:

[0142] The unmanned aerial vehicle will automatically begin flying based on the received flight plan.

[0143] Input: Flight plan

[0144] Output: Captured image data

[0145] Specific operation: The unmanned aircraft "turns on its engines and ascends to a specified altitude," then "flies automatically along a set route, repeatedly taking photographs and transmitting data."

[0146] Step 4: Sending data

[0147] Unmanned Aerial Vehicles:

[0148] The captured photo data is sent to the server in real time.

[0149] Input: Captured image data

[0150] Output: Image data sent to the server

[0151] Specific operation: The unmanned aircraft "sends image data to the server immediately after taking the image."

[0152] Step 5: Receiving the data

[0153] server:

[0154] Receive image data transmitted from the unmanned aerial vehicle.

[0155] Input: Submitted image data

[0156] Output: Received image data

[0157] Specific operation: The server receives image data in real time.

[0158] Step 6: Save your data

[0159] server:

[0160] The received image data is stored in a database in a specific format.

[0161] Input: Received image data

[0162] Output: Image data stored in a database

[0163] Specific operation: The server "saves the image data in a database with date and location information."

[0164] Step 7: Selecting the image to analyze

[0165] server:

[0166] A new image is selected from the stored image data.

[0167] Input: Image data stored in a database

[0168] Output: Image data selected for analysis

[0169] Specific operation: The server selects the latest image for analysis.

[0170] Step 8: AI analysis of images

[0171] server:

[0172] The selected image data is input into the generative artificial intelligence (AI).

[0173] The AI ​​uses deep learning technology to analyze terrain and road conditions.

[0174] Input: Image data to be analyzed

[0175] Output: Analysis results (passability information)

[0176] Specific operation: The generative AI "analyzes the presence or absence of road damage and obstacles in the image."

[0177] Step 9: Generate a passable map

[0178] server:

[0179] Based on the analysis results, a passability map is generated that visually displays passability information on a map.

[0180] Input: Analysis results (passability information)

[0181] Output: Passability map

[0182] Specific operation: The server generates a map that shows passable roads in green and impassable roads in red.

[0183] Step 10: Timely updates

[0184] server:

[0185] The generated passability map is published to a web service or dedicated application.

[0186] Input: Passability map

[0187] Output: A public accessibility map for the user

[0188] Specific operation: The server "uploads the accessibility map to a user-accessible web portal."

[0189] Step 11: Real-time access

[0190] User:

[0191] Access real-time updated road availability maps using a web browser or dedicated application.

[0192] Input: Web browser or dedicated application

[0193] Output: Real-time road availability map

[0194] Specific operation: The user "checks traffic conditions in real time and plans evacuation routes and travel plans."

[0195] Through the above steps, this system quickly and accurately determines whether transportation infrastructure is passable during a disaster and provides users with useful information in real time.

[0196] (Application example 1)

[0197] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0198] During disasters, it is extremely important to quickly and accurately determine whether transportation infrastructure is passable. In particular, traffic information updated in real time is essential for autonomous vehicles to safely transport evacuees. However, with current technology, it is difficult to determine the extent of road damage and the presence of obstacles in real time and immediately reflect this information in the navigation system of autonomous vehicles. For this reason, improving the efficiency and ensuring safety of autonomous vehicle operations during disasters is a challenge.

[0199] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0200] In this invention, the server includes means for capturing images of the ground using an unmanned aerial vehicle equipped with a camera and communication capabilities, means for receiving and storing image data transmitted from the unmanned aerial vehicle, means for analyzing the stored image data and determining the condition of the terrain and roads, means for generating a passability map based on the passability information determined by the generating artificial intelligence, and means for providing the generated passability map to the navigation system of the autonomous vehicle. This enables the passability of transportation infrastructure in the event of a disaster to be quickly and accurately determined, enabling the autonomous vehicle to operate safely and efficiently.

[0201] An "unmanned aerial vehicle" is a device equipped with a camera and communication capabilities, which flies remotely or automatically and takes images of the earth's surface.

[0202] "Image data" is digital data that includes photographic information of the ground and roads taken by unmanned aerial vehicles.

[0203] "Generative AI" is an AI system that uses deep learning technology to analyze stored image data and automatically determine the condition of terrain and roads.

[0204] "Passability information" is information indicating whether a road is passable or not and whether there are any obstacles, obtained from the results of analysis by the generating artificial intelligence.

[0205] A "passable road map" is a map generated based on passable road information, which indicates passable roads in a predetermined color and is provided to the navigation system of an autonomous vehicle.

[0206] An "autonomous vehicle" is a vehicle that is controlled by a navigation system and can drive autonomously.

[0207] A "navigation system" is a system that uses information such as road availability maps to control the operation of autonomous vehicles and determine the optimal route to a destination.

[0208] This invention is a system for quickly determining whether transportation infrastructure is passable during disasters, and aims to provide real-time traffic information to the navigation systems of self-driving vehicles. This system is composed of unmanned aerial vehicles (drones), servers, generative artificial intelligence (AI), and self-driving vehicles.

[0209] First, the user sets up a flight plan for the unmanned aerial vehicle using a dedicated terminal. The flight plan targets the disaster-stricken area and specifies the route and photography points that the unmanned aerial vehicle will fly automatically. For example, it can be set to prioritize coverage of major highways and evacuation routes.

[0210] According to the set flight plan, the unmanned aerial vehicle automatically flies within a designated area and takes images of the ground and roads. The captured image data is sent from the unmanned aerial vehicle to a server in real time. The server stores the received image data in a specific format.

[0211] The saved image data is input into the generative AI and analyzed using deep learning technology. The generative AI automatically determines road conditions and changes in the terrain, and obtains passability information. For example, it determines the state of road damage and the presence of obstacles, and if it determines that a road is impassable, it analyzes the road as "impassable."

[0212] Next, the server generates a passable / unpassable map based on the analysis results obtained by the AI. On this map, passable roads are displayed in green, impassable roads in red, and roads requiring caution in yellow. The generated passable / unpassable map is provided in real time to the navigation system of the autonomous vehicle, which then uses this information to select the optimal route.

[0213] For example, if a major highway is deemed impassable due to a landslide, an autonomous vehicle will check the road availability map in real time and automatically select an alternative passable route. This information will also be presented to the driver and rescue teams, ensuring safe and efficient travel.

[0214] An example prompt might be: "Analyze traffic conditions of transportation infrastructure in real time during disasters and provide optimal route information for autonomous vehicles. Specifically, use a deep learning model to analyze road photos taken by a drone and detect landslides and road damage."

[0215] As a result, by using this system, it will be possible to quickly and accurately determine whether transportation infrastructure is passable during a disaster, enabling self-driving vehicles to operate safely and efficiently.

[0216] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0217] Step 1:

[0218] The user sets up a flight plan for the unmanned aerial vehicle using a dedicated terminal. The user refers to a map of the disaster area and specifies the route (input) that the unmanned aerial vehicle should take and the shooting points. This flight plan (output) is sent to the unmanned aerial vehicle.

[0219] Step 2:

[0220] The unmanned aerial vehicle flies automatically according to a set flight plan and periodically takes images of the ground and roads (specific operation). The image data taken by the unmanned aerial vehicle (input) is sent to a server in real time (output).

[0221] Step 3:

[0222] The server receives image data sent from the unmanned aerial vehicle (input) and stores it in a database in a specific format (output). The stored image data is then analyzed (specific operation).

[0223] Step 4:

[0224] The server inputs the stored image data into a generative AI model (generative AI model) (input) and analyzes it using deep learning technology (specific operation). The generative AI analyzes the image data and automatically determines the terrain and road conditions (data processing). The analysis results (output) include information on whether the road is passable.

[0225] Step 5:

[0226] The server generates a passable / non-passable map based on the analysis results obtained by the generation AI (specific operation). The input is the analysis results, and the output is a passable / non-passable map. On this map, passable roads are shown in green, impassable roads in red, and roads requiring caution in yellow.

[0227] Step 6:

[0228] The server provides the generated route availability map to the navigation system of the autonomous vehicle (output). The autonomous vehicle selects a safe and efficient route based on the route availability map (action) and starts moving to the destination (specific action).

[0229] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0230] ---

[0231] The present invention is a system that quickly determines whether traffic infrastructure is passable during a disaster, and further recognizes the user's emotions and provides support information.

[0232] System Overview

[0233] This system uses an unmanned aerial vehicle (drone) equipped with a camera and communication functions to capture images of the ground surface and send the image data to a server. The server stores the received image data and uses generative artificial intelligence (hereafter referred to as generative AI) to analyze the terrain and road conditions. Based on the analysis results, a passable / unpassable map is generated and provided to the user. The system also incorporates an emotion engine that recognizes the user's emotions and provides support information according to the user's emotional state.

[0234] Program processing

[0235] 1. Setting up a flight plan:

[0236] Using an application on a dedicated device, the user refers to a map of the disaster area and specifies the flight path and shooting points for the unmanned aerial vehicle.

[0237] 2. Automatic flight and photography of unmanned aerial vehicles:

[0238] The terminal transmits the set flight plan to the unmanned aerial vehicle, and the unmanned aerial vehicle automatically flies in the designated area according to the received flight plan.

[0239] The unmanned aerial vehicle periodically takes high-resolution photographs of the ground and roads along its flight path.

[0240] The captured photo data is sent to the server in real time.

[0241] 3. Receiving and storing data:

[0242] The server receives the image data sent from the unmanned aerial vehicle and stores it in a database.

[0243] 4. Image Analysis:

[0244] The server sequentially inputs the stored image data into the generating AI, which then analyzes the road conditions using deep learning technology.

[0245] For example, it can detect obstacles such as landslides in an image and determine that the location is "impassable."

[0246] 5. Generate a passability map:

[0247] The server determines passability information based on the analysis results and generates a passability map.

[0248] The road availability map shows passable roads in green, impassable roads in red, and roads requiring caution in yellow.

[0249] 6. Leveraging the Emotion Engine:

[0250] When providing a passability map, the server analyzes the user's facial expressions and voice using an emotion engine to recognize the user's emotions.

[0251] For example, if the system recognizes that a user is feeling anxious or stressed, it will provide not only a map of whether a route is passable, but also additional support information such as the location of evacuation shelters and information on the distribution of relief supplies.

[0252] 7. Information provision:

[0253] The server makes available to users road availability maps and additional assistance information via web services or dedicated applications.

[0254] Users can use a web browser or dedicated application to access real-time updated route availability maps and assistance information, and plan evacuation routes and travel plans.

[0255] Specific examples

[0256] For example, immediately after an earthquake occurs, an unmanned aerial vehicle flies over the affected area and takes photos of the roads. The image data is then sent to a server, where the AI ​​analyzes the road for damage and obstacles. If a landslide is confirmed at a point on a major highway, the location will be deemed "impassable" and displayed in red on a road access map.

[0257] At the same time, the emotion engine analyzes the facial expressions and voice of users viewing the map, and if it detects anxiety, it also provides information on the locations of evacuation shelters and relief supply distribution points. In this way, users can obtain appropriate routes and support information to evacuate quickly and safely.

[0258] As described above, the present invention is a system that quickly and accurately determines whether transportation infrastructure is passable during a disaster, and provides support information that takes the user's emotions into consideration, thereby effectively supporting the evacuation of disaster victims and rescue efforts.

[0259] The processing flow will be explained below.

[0260] ---

[0261] Step 1:

[0262] Users use an application on a dedicated device to refer to a map of the disaster-affected area and set the flight path and shooting points for the unmanned aerial vehicle (drone).

[0263] Step 2:

[0264] The terminal transmits the set flight plan to the unmanned aerial vehicle, and the unmanned aerial vehicle automatically flies in the designated area according to the received flight plan.

[0265] Step 3:

[0266] The drone periodically takes high-resolution photographs of the ground and roads along its flight path.

[0267] Step 4:

[0268] The unmanned aerial vehicle transmits the captured photo data to a server in real time.

[0269] Step 5:

[0270] The server receives the image data sent from the unmanned aerial vehicle and stores it in a database in a specific format.

[0271] Step 6:

[0272] The server sequentially inputs the stored image data into the generating AI, which then analyzes the road conditions using deep learning technology.

[0273] Step 7:

[0274] Based on the results of the analysis by the generation AI, the server determines the condition of the terrain and roads and determines whether each road is passable.

[0275] Step 8:

[0276] The server generates a passable road map based on the determined passable road information, showing passable roads in green, impassable roads in red, and roads requiring caution in yellow.

[0277] Step 9:

[0278] The server activates an emotion engine in the process of providing the user with a passable / unpassable map. The emotion engine analyzes the user's facial expressions and voice to recognize the user's emotions.

[0279] Step 10:

[0280] If the server determines that the user is feeling anxious or stressed based on the analysis results of the emotion engine, it will provide additional support information, such as a passable / unpassable road map, the location of evacuation shelters, and information on the distribution of relief supplies.

[0281] Step 11:

[0282] Using a web browser or dedicated application, users can access real-time updated route availability maps and assistance information to plan safe and appropriate evacuation routes and travel plans.

[0283] ---

[0284] The above are the specific processing steps for carrying out the present invention. These processing steps enable quick and accurate determination of whether an area is passable during a disaster, and provide appropriate support information according to the user's emotional state, thereby effectively supporting evacuation and rescue efforts.

[0285] Example 2

[0286] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0287] When a natural disaster occurs, it is difficult to quickly and accurately grasp information on whether the transportation infrastructure in the affected area is passable, which hinders the efficiency of evacuation and rescue efforts.In addition, support information that takes into account the emotional state of the victims is not provided, which increases the mental burden and hinders appropriate evacuation behavior.

[0288] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for capturing images of the ground surface using an unmanned aerial vehicle equipped with a camera and communication function, means for receiving and storing image data transmitted from the unmanned aerial vehicle, means for generating an AI generation system for analyzing the stored image data and determining the condition of the terrain and roads, means for generating a passability map based on the passability information determined by the AI ​​generation system, means for providing the generated passability map to the user, and means for analyzing the user's facial expressions and voice and providing support information based on the user's emotional state. This makes it possible to quickly and accurately provide passability information for disaster-stricken areas and to provide support information that takes the user's emotional state into consideration.

[0289] An "unmanned aerial vehicle equipped with a camera and communication capabilities" is a flying device with autonomous flight capabilities that is equipped with a camera and communication module for capturing image data and transmitting it wirelessly.

[0290] "Image data" refers to digital information such as photographs and videos taken by unmanned aerial vehicles of the conditions of the ground and roads.

[0291] "Generative AI" is a software system equipped with advanced machine learning technology that analyzes large amounts of image data and determines the condition of terrain and roads.

[0292] "Passability information" is digital information that indicates whether a road is passable or not, based on image data analyzed by the generating AI.

[0293] A "passability map" is a map for visually displaying passability information, and indicates passable roads, impassable roads, and roads requiring caution using different colors.

[0294] "Emotional state" refers to the emotional state of the user that is inferred based on facial expressions and voice analysis.

[0295] "Support information" is additional information that is useful in the event of a disaster, such as information on the location of evacuation shelters and distribution of relief supplies, provided according to the user's emotional state.

[0296] The present invention is a system that quickly determines whether traffic infrastructure is passable during a disaster, and further recognizes the user's emotions and provides support information. How the present invention is specifically implemented will be described below.

[0297] System Overview

[0298] The system of the present invention uses an unmanned aerial vehicle (drone) equipped with a camera and communication capabilities. The unmanned aerial vehicle takes high-resolution images of the ground and roads and transmits them to a server in real time. The server stores the received image data and analyzes the terrain and road conditions using a generative AI model. Based on the analysis results, a passability map is generated and provided to the user. The system also analyzes the user's facial expressions and voice to recognize their emotional state and provides support information based on the user's emotional state.

[0299] Hardware and software used

[0300] Unmanned aerial vehicle (drone): Equipped with a high-resolution camera and communication module.

[0301] Server: Equipped with a large database and high-performance processors to run generative AI models.

[0302] User device: A smartphone or tablet capable of running a web browser and dedicated applications.

[0303] Examples and prompts

[0304] For example, immediately after an earthquake, an unmanned aerial vehicle flies over the affected area and takes photographs of the condition of major roads and bridges. The captured image data is sent to a server in real time, where a generative AI model analyzes the road for damage and obstacles. For example, if a landslide is detected in the image, the location is determined to be "impassable" and displayed in red on a passability map.

[0305] Example prompt sentence:

[0306] Example prompt for analyzing the accessibility of transportation infrastructure:

[0307] Analyze road image data of disaster-stricken areas taken by unmanned aerial vehicles to determine whether roads are passable. Detect obstacles such as landslides and cracks in the roads, and create a passable map based on the analysis results.

[0308] Example prompt for user sentiment analysis:

[0309] Analyze the facial expressions and voice of users viewing the road availability map to detect their emotional state of anxiety or stress. Depending on their emotional state, provide them with additional support information, such as the location of evacuation shelters and information on the distribution of relief supplies.

[0310] When a user checks the passability map using a web browser or a dedicated application, the emotion engine analyzes the user's facial expressions and voice at the time and displays appropriate support information. For example, if the emotion engine detects the user's anxiety, it will add information about the location of evacuation shelters and relief supplies.

[0311] As described above, the present invention can quickly and accurately provide information on whether a disaster area is passable, and can provide support information that takes into account the user's emotional state, thereby effectively supporting evacuation and rescue operations.

[0312] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0313] Step 1: Set up a flight plan

[0314] User

[0315] The user launches the application on the dedicated device and displays a map of the disaster-affected area. Next, the user sets the flight path and shooting points for the drone on the map. For example, the user selects important areas such as major roads and bridges in the disaster area. Once the settings are complete, the user finalizes the flight plan and sends it from the device to the drone.

[0316] Input: Map of disaster area, flight route and photo points

[0317] Output: Flight plan data sent to unmanned aerial vehicles

[0318] Specific actions: Pointing and clicking on the map, pressing the flight plan confirmation button

[0319] Step 2: Automatic flight and photography of the unmanned aerial vehicle

[0320] unmanned aerial vehicle

[0321] The drone begins automatic flight based on the flight plan received from the device. It uses a GPS system to follow an accurate flight path, and when it reaches a designated shooting point, it takes photos of the ground and roads with a high-resolution camera. The captured image data is sent to a server in real time.

[0322] Input: Flight plan data

[0323] Output: The captured image data is sent to the server.

[0324] Specific operations: Obtaining location information using GPS, taking photos using a high-resolution camera, and transmitting data using a wireless communication module

[0325] Step 3: Receiving and storing data

[0326] server

[0327] The server receives image data sent from the unmanned aerial vehicle in real time and stores it in a database. The image data also contains metadata such as the date and time of the photo and GPS coordinates, so it is stored together with these.

[0328] Input: Captured image data (including metadata)

[0329] Output: Image data stored in a database

[0330] Specific operation: Receiving data via wireless communication and writing it to a database

[0331] Step 4: Image analysis

[0332] server

[0333] The server inputs the stored image data into a generative AI model and analyzes the images using deep learning technology. Specifically, it detects road damage and obstacles (e.g., landslides and fallen trees). Based on the results of this analysis, it determines whether the road is passable.

[0334] Input: Image data stored in a database

[0335] Output: Analyzed passability information (e.g. passable, not passable)

[0336] Specific operations: Data input to generative AI model, image analysis using deep learning

[0337] Step 5: Generate a passability map

[0338] server

[0339] The server generates a passable road map based on the results of image analysis, showing passable roads in green, impassable roads in red, and roads requiring caution in yellow.

[0340] Input: Parsed passability information

[0341] Output: Passability map

[0342] Specific operation: Visualization of passability information, color-coded display on the map

[0343] Step 6: Leverage your emotional engine

[0344] server

[0345] The server uses an emotion engine to analyze the user's facial expressions and voice as they browse the passability map. The emotion engine uses facial recognition software and tone analysis tools to determine the user's emotional state, and if it detects anxiety or stress, it provides additional support information.

[0346] Input: User's facial expression data, voice data

[0347] Output: Support information based on the user's emotional state

[0348] Specific operations: facial expression recognition and voice analysis, emotional state estimation, and provision of support information

[0349] Step 7: Provide information

[0350] server

[0351] The server provides users with road availability maps and additional assistance information via web services and dedicated applications. The information is updated in real time and can be used whenever needed.

[0352] User

[0353] Users can use a web browser or a dedicated application to check the passability map and support information, and plan evacuation routes and travel plans.

[0354] Input: Passability map, additional assistance information

[0355] Output: User viewing and use of information

[0356] Specific actions: Operating a web browser or application, planning an evacuation route

[0357] In this way, this system can quickly and accurately determine whether transportation infrastructure is passable during a disaster, and by providing support information that takes into account the user's emotional state, it can effectively support the evacuation of disaster victims and rescue efforts.

[0358] (Application example 2)

[0359] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0360] In the past, it was difficult to quickly and accurately determine whether transportation infrastructure was passable during disasters. Furthermore, when providing evacuation and support information, appropriate support was not provided that took into account the user's feelings. Therefore, there is a need for an information provision system that allows disaster victims to evacuate safely and receive appropriate support.

[0361] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0362] In this invention, the server includes means for capturing images of the ground surface using an unmanned aerial vehicle equipped with a camera and communication capabilities, means for receiving and storing image data transmitted from the unmanned aerial vehicle, artificial intelligence generation means for analyzing the stored image data and determining the condition of the terrain and roads, means for generating a passability map, means for providing the generated passability map, emotion recognition engine means for analyzing a user's facial expressions and voice and recognizing emotions, and means for providing support information based on the user's emotions. This makes it possible to quickly and accurately determine the passability of transportation infrastructure in the event of a disaster and to provide appropriate support information that takes the user's emotions into consideration.

[0363] A "camera" is a device that captures images or videos.

[0364] "Communication function" means the function of transmitting and receiving data wirelessly or via wire.

[0365] An "unmanned aerial vehicle" is an aircraft that flies remotely or autonomously to conduct observations and collect data.

[0366] "Image data" is digital information of still or moving images captured by a camera.

[0367] "Generative AI" refers to an AI technology that recognizes patterns based on large amounts of data and performs data analysis and judgments.

[0368] "Deep learning technology" is a technology in the field of artificial intelligence that uses multi-layer neural networks to analyze data with high precision.

[0369] A "passable roads map" displays information on a map in different colors, indicating which roads are passable, which roads are impassable, and which roads require caution.

[0370] An "emotion recognition engine" is a technology that analyzes a user's facial expressions and voice to identify their emotional state.

[0371] "Support information" refers to information for supporting users' evacuation and daily life in the event of a disaster.

[0372] This invention is a system that quickly determines whether traffic infrastructure is passable during a disaster, and further recognizes the user's emotions and provides support information. This system is configured as follows.

[0373] System Overview

[0374] This system uses an unmanned aerial vehicle equipped with a camera and communications capabilities to capture images of the ground surface and send the image data to a server. The server stores the received image data and uses generative artificial intelligence to analyze the terrain and road conditions. Based on the analysis results, a passable / unpassable map is generated and provided to the user. The system also incorporates an emotion engine that recognizes the user's emotions and provides support information according to the user's emotional state.

[0375] Hardware and software used

[0376] The system's main hardware consists of an unmanned aerial vehicle (drone) equipped with a camera and communication functions, a high-performance server, and a smartphone used by the user. The unmanned aerial vehicle is equipped with a high-resolution camera and communication functions that transmit image data to the server in real time.

[0377] The server uses Python, OpenCV, TensorFlow, Keras, and other deep learning libraries to analyze image data. The backend system uses Django, and the frontend uses React Native. OpenCV and a custom model are used for emotion recognition.

[0378] Data analysis and provision details

[0379] The server receives image data sent from the unmanned aerial vehicle and stores it in a database. The stored image data is then input into the artificial intelligence that generates it, which uses deep learning technology to analyze the terrain and road conditions. It detects obstacles such as landslides and damaged roads, classifies those areas as "impassable," and generates a passability map based on the results.

[0380] The generated road availability map shows passable roads in green, impassable roads in red, and roads requiring caution in yellow, and is updated in real time. Users can access the map and assistance information using a dedicated smartphone application.

[0381] The emotion recognition engine analyzes facial expressions and voice using the camera and microphone on the user's smartphone. For example, it uses OpenCV and a custom model to recognize emotions from the user's facial expressions and provides appropriate support information. For users who are feeling anxious or stressed, it provides information on evacuation shelters and relief supply distribution locations along with a map showing whether roads are passable.

[0382] Specific examples

[0383] For example, immediately after an earthquake occurs, an unmanned aerial vehicle flies over the affected area and takes photos of the roads. The image data is then sent to a server, where the AI ​​analyzes the road for damage and obstacles. If a landslide is confirmed at a point on a major highway, the location is deemed "impassable" and displayed in red on a passability map. At the same time, the emotion engine analyzes the facial expressions and voices of users viewing the passability map, and if it detects anxiety, it also provides information on the locations of evacuation centers and relief supply distribution points. In this way, users can obtain appropriate routes and support information to evacuate quickly and safely.

[0384] Prompt Sentence Examples

[0385] For post-disaster traffic analysis, high-resolution image data is sent from an unmanned aerial vehicle to a server and road conditions are determined using a deep learning model. We would like to build a system that analyzes the user's facial expressions and voice and provides appropriate support information based on their emotions. First, we train an AI model to identify obstacles such as landslides from image data, then develop an emotion recognition model and customize the support information based on that. Please suggest an appropriate deep learning library and model structure.

[0386] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0387] Step 1:

[0388] The user uses an application on a dedicated device to refer to a map of the disaster-stricken area and specify the flight path and shooting points of the unmanned aerial vehicle. The input is the map data specified by the user, and the output is a flight plan for the unmanned aerial vehicle. In concrete terms, the user operates the map on the application, placing pins to set shooting points.

[0389] Step 2:

[0390] The terminal transmits the set flight plan to the unmanned aerial vehicle, which then automatically flies the designated area according to the received flight plan. The input is flight plan data, and the output is flight instructions for the unmanned aerial vehicle. The unmanned aerial vehicle flies along the specified route and periodically takes photos of the ground and roads.

[0391] Step 3:

[0392] The unmanned aerial vehicle sends the photographic data it captures to a server in real time. The input is the captured image data, and the output is the image data sent to the server. Specifically, the images captured by the unmanned aerial vehicle's camera are uploaded to the server via the communication function.

[0393] Step 4:

[0394] The server receives image data sent from the unmanned aerial vehicle and stores it in a database. The input is the image data received from the unmanned aerial vehicle, and the output is the image data stored in the database. Specifically, the server converts the data into an appropriate format and stores it in storage.

[0395] Step 5:

[0396] The server sequentially inputs the stored image data into the generative AI model and uses deep learning technology to analyze the road conditions. The stored image data is used as input, and the analysis results are obtained as output. Specifically, the image is input into the model using TensorFlow and Keras, and obstacles are detected and the condition is determined.

[0397] Step 6:

[0398] The server generates a passable / non-passable map based on the analysis results. The analysis results are input, and the passable / non-passable map is generated as output. Specifically, it uses a map generation algorithm to color-code passable roads in green, impassable roads in red, and roads requiring caution in yellow.

[0399] Step 7:

[0400] The server makes the generated passability map available to users via a web service or dedicated application. The generated passability map is the input, and the passability map provided to the user is generated as the output. Specifically, the map information is displayed on a web page using Django and updated in real time.

[0401] Step 8:

[0402] Users use a smartphone application to access real-time updated route availability maps and support information to plan evacuation routes and travel plans. A route availability map is available as input, and the user's confirmed evacuation route and support information are output. Specifically, a React Native application is used to enable users to easily obtain information.

[0403] Step 9:

[0404] When providing a passability map, the server analyzes the user's facial expressions and voice using an emotion recognition engine to recognize the user's emotions. The input is the user's facial expression image and voice data, and the output is the user's emotional state. Specifically, the server analyzes the user's emotions using OpenCV and a custom emotion recognition model.

[0405] Step 10:

[0406] The server provides additional support information, such as the location of evacuation shelters and information on the distribution of relief supplies, depending on the user's emotional state. The user's emotional state is input, and appropriate support information is provided as output. Specifically, the server selects appropriate support information based on the emotion recognition results and notifies the user.

[0407] Through the above processing steps, the system can quickly and accurately determine whether transportation infrastructure is passable during a disaster, and can also provide support information that takes into account the user's emotions.

[0408] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0409] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0410] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0411] [Second embodiment]

[0412] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0413] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0414] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0415] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0416] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0417] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0418] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0419] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0420] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0421] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0422] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0423] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0424] ---

[0425] The present invention is a system for quickly determining whether traffic infrastructure is passable during a disaster, and is configured as follows.

[0426] This system first uses an unmanned aerial vehicle (drone) equipped with a camera and communication functions to capture images of the ground surface. The unmanned aerial vehicle automatically flies over a designated area according to a flight plan, taking high-resolution photos of the ground surface and roads at regular intervals. The captured photo data is sent to a server in real time.

[0427] The server receives image data sent from the unmanned aerial vehicle and stores it in a database in a specific format. The received image data is analyzed sequentially and input into the generative AI. The generative AI then analyzes the image data using deep learning technology to automatically determine the condition of the terrain and roads. This allows it to accurately detect the extent of road damage and the presence or absence of obstacles, and obtain information on whether the road is passable.

[0428] Based on the analysis results, the server generates a passable / unpassable map. This map displays passable roads in a predetermined first color (e.g., green), impassable roads in a predetermined second color (e.g., red), and roads requiring caution in a predetermined third color (e.g., yellow). The generated passable / unpassable map is immediately provided to the user via a web service or dedicated application.

[0429] Next, the program processing of this system will be explained in natural language.

[0430] 1. Setting up a flight plan:

[0431] Using an application on a dedicated device, the user refers to a map of the disaster area and specifies the flight path and shooting points for the unmanned aerial vehicle.

[0432] 2. Automatic flight and photography of unmanned aerial vehicles:

[0433] The terminal transmits the set flight plan to the unmanned aerial vehicle.

[0434] The unmanned aerial vehicle follows a specified flight plan, automatically flies over a designated area, and periodically takes photographs of the ground and roads.

[0435] The unmanned aerial vehicle transmits the captured photo data to a server in real time.

[0436] 3. Receiving and storing data:

[0437] The server receives image data sent from the unmanned aerial vehicle and stores it in a database in a specific format.

[0438] 4. Image Analysis:

[0439] The server sequentially inputs the stored image data into the generative artificial intelligence, which then analyzes the road conditions using deep learning technology.

[0440] For example, if a landslide is visible in some images, the generative AI will determine that road is "impassable."

[0441] 5. Generate a passability map:

[0442] Based on the analysis results, the server generates a passability map that visually displays passability information on a map.

[0443] The road availability map shows passable roads in green, impassable roads in red, and roads requiring caution in yellow.

[0444] 6. Information provision:

[0445] The server publishes the generated passability map to web services or dedicated applications and provides it to users.

[0446] Users can use a web browser or dedicated application to access real-time updated road safety maps and plan evacuation routes and movements.

[0447] As a specific example, immediately after a disaster occurs, an unmanned aerial vehicle flies over the affected area and takes photos of the roads. The image data is sent to a server, and the artificial intelligence analyzes the road for damage and obstacles. For example, if a landslide is confirmed at a certain point on a major highway, the road will be determined to be "impassable" and displayed in red on the passability map. If the road is confirmed to be safe at other points, those parts will be displayed in green. In this way, the latest passability information is provided in real time, helping users to travel quickly and safely.

[0448] As described above, the present invention is a system that quickly and accurately determines whether transportation infrastructure is passable during a disaster, and efficiently supports the evacuation of disaster victims and rescue operations.

[0449] The processing flow will be explained below.

[0450] ---

[0451] Step 1:

[0452] Users use an application on a dedicated device to refer to a map of the disaster-affected area and specify the flight path and shooting points for the unmanned aerial vehicle (drone).

[0453] Step 2:

[0454] The terminal transmits the set flight plan to the unmanned aerial vehicle, and the unmanned aerial vehicle automatically flies in the designated area according to the received flight plan.

[0455] Step 3:

[0456] The drone periodically takes high-resolution photographs of the ground and roads along its flight path.

[0457] Step 4:

[0458] The unmanned aerial vehicle transmits the captured photo data to a server in real time.

[0459] Step 5:

[0460] The server receives the image data sent from the unmanned aerial vehicle and stores it in a database in a specific format.

[0461] Step 6:

[0462] The server sequentially inputs the stored image data into the generative artificial intelligence.

[0463] Step 7:

[0464] The server uses generative artificial intelligence to analyze the image data, using deep learning technology to automatically determine the terrain and road conditions.

[0465] Step 8:

[0466] The server then determines whether a road is passable based on the analysis results. For example, it may classify a point in the image where a landslide has been confirmed as "impassable."

[0467] Step 9:

[0468] The server generates a passability map based on the passability information.

[0469] Step 10:

[0470] The server makes the passability map available to users via a web service or dedicated application.

[0471] Step 11:

[0472] Users can access the road closure map using a web browser or a dedicated application, and create evacuation routes and travel plans based on information updated in real time.

[0473] ---

[0474] These are the specific processing steps of the program for this system. This flow enables rapid and accurate understanding of the state of transportation infrastructure during a disaster, and supports effective evacuation and relief efforts.

[0475] Example 1

[0476] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0477] With conventional systems, it is difficult to quickly and accurately determine whether roads are passable during a disaster. Specifically, there is a lack of means to analyze road conditions in real time and provide that information to users immediately. As a result, evacuation plans and rescue operations cannot be carried out quickly in an emergency, and there is a risk that the damage will worsen.

[0478] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0479] In this invention, the server includes a means for a user to set a flight plan for the unmanned aerial vehicle via a dedicated terminal, a means for the unmanned aerial vehicle to automatically fly according to the set flight plan and transmit captured data to the server in real time, and a means for presenting a passability map at startup based on the passability information generated by the server. This makes it possible to quickly and accurately determine whether roads are passable during a disaster and provide the information to the user in real time.

[0480] An "unmanned aerial vehicle" is a device equipped with a camera and communication capabilities that flies automatically according to a specified flight plan.

[0481] "Image data" refers to data that includes photographic information of the ground and roads taken by unmanned aerial vehicles.

[0482] "Analysis" is the process of determining the terrain and road conditions based on image data stored on the server.

[0483] "Generative AI" is an AI system that uses image data to analyze terrain and road conditions using deep learning technology.

[0484] A "flight plan" is information about the flight path and photographing points of an unmanned aerial vehicle that is set by the user via a dedicated terminal.

[0485] A "dedicated terminal" is a device that a user uses to set a flight plan for an unmanned aerial vehicle, and includes smartphones, tablets, etc.

[0486] The "server" is a computer system that receives and stores image data transmitted from unmanned aerial vehicles, then analyzes the data and provides the generated passable / unpassable map.

[0487] A "passable roads map" is a map that visually displays passable roads, impassable roads, and roads requiring caution, using different colors based on the analysis results.

[0488] "Real-time" means that image data captured by an unmanned aerial vehicle is sent to a server almost immediately, with an extremely short time lag for analysis and provision of information.

[0489] "User" refers to a person who uses a dedicated terminal to set up a flight plan for an unmanned aerial vehicle and view a passability map.

[0490] To specifically implement this invention, an unmanned aerial vehicle (drone), a dedicated terminal (smartphone, tablet, etc.), and a server are required. The purpose of this system is to quickly determine whether transportation infrastructure is passable during a disaster and provide users with information in real time.

[0491] System Overview

[0492] Hardware Configuration

[0493] Unmanned aerial vehicle (drone): Equipped with a camera and communication capabilities, it flies automatically according to a specified flight plan.

[0494] Dedicated terminal: A device used by a user to set up a flight plan for an unmanned aerial vehicle, including a smartphone or tablet.

[0495] Server: A computer system that receives, stores, and analyzes image data sent from unmanned aerial vehicles.

[0496] Software Configuration

[0497] Dedicated terminal application: An application that allows users to set up flight plans.

[0498] Server-side analysis software: Software that uses deep learning technology to analyze image data.

[0499] Web service / dedicated application: Software that provides the generated accessibility map to the user.

[0500] Processing flow

[0501] Setting up a flight plan

[0502] The user launches the application on the dedicated device and displays a map of the affected area. The user then specifies the flight path and shooting points for the unmanned aircraft. For example, the user might "set a flight path from the center of Sendai City to the main road in the east."

[0503] Automated flight and data transmission

[0504] The terminal sends the set flight plan to the unmanned aerial vehicle. The unmanned aerial vehicle automatically flies over the specified area according to the specified flight plan, periodically taking photos of the ground and roads. The captured photo data is sent to the server in real time. Example: The unmanned aerial vehicle "takes photos every 10 minutes while flying along the specified route, and immediately sends the image data to the server after taking the photos."

[0505] Receiving and storing data

[0506] The server receives image data sent from the unmanned aerial vehicle in real time and stores it in a database in a specific format. Example: The server "receives image data in real time and stores the image data in a database with date and location information."

[0507] Image analysis and generation of passability maps

[0508] The server sequentially inputs the stored image data into generative artificial intelligence (AI), which uses deep learning technology to analyze the condition of the road. The AI ​​analyzes the image data and determines whether the road is damaged or has any obstacles. For example, if a landslide is confirmed in some of the images, the AI ​​will determine that the road is "impassable." The server generates a passable / unpassable map based on the analysis results. The passable / unpassable map shows passable roads in green, impassable roads in red, and roads requiring caution in yellow. Example: The server "generates a map that shows areas where landslides have been confirmed in red, and other safe areas in green."

[0509] Providing information

[0510] The server publishes the generated route availability map to a web service or dedicated application and provides it to users. Users can use a web browser or dedicated application to access the route availability map, which is updated in real time, and plan evacuation routes and movements. Example: A user can "check the traffic conditions in real time and plan evacuation routes and movements."

[0511] Prompt Sentence Examples

[0512] "After a disaster occurs in central Sendai, please use a drone to photograph the traffic conditions of major roads in the eastern part of the city, send the data to a server for analysis, and generate a map of whether roads are passable or not."

[0513] As described above, this system quickly and accurately determines whether transportation infrastructure is passable during a disaster, and provides users with useful information in real time.

[0514] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0515] Step 1: Set up a flight plan

[0516] User:

[0517] The user launches the application on the dedicated device and displays a map of the disaster area.

[0518] The user specifies the flight path and shooting points of the unmanned aerial vehicle on the map.

[0519] Input: Map information of the disaster area, flight path and photo points

[0520] Output: Flight plan data for unmanned aerial vehicles

[0521] Specific actions: The user "tap the tablet screen to mark the shooting point of the unmanned aerial vehicle," "confirm the flight plan, and press the start button."

[0522] Step 2: Submit your flight plan

[0523] Device:

[0524] The configured flight plan is transmitted to the unmanned aerial vehicle.

[0525] Input: Flight plan data

[0526] Output: Flight plan sent to the unmanned aerial vehicle

[0527] Specific operation: The device "wirelessly transmits flight plan data to the unmanned aerial vehicle" and "displays a notification confirming receipt."

[0528] Step 3: Automated flight of the unmanned aerial vehicle

[0529] Unmanned Aerial Vehicles:

[0530] The unmanned aerial vehicle will automatically begin flying based on the received flight plan.

[0531] Input: Flight plan

[0532] Output: Captured image data

[0533] Specific operation: The unmanned aircraft "turns on its engines and ascends to a specified altitude," then "flies automatically along a set route, repeatedly taking photographs and transmitting data."

[0534] Step 4: Sending data

[0535] Unmanned Aerial Vehicles:

[0536] The captured photo data is sent to the server in real time.

[0537] Input: Captured image data

[0538] Output: Image data sent to the server

[0539] Specific operation: The unmanned aircraft "sends image data to the server immediately after taking the image."

[0540] Step 5: Receiving the data

[0541] server:

[0542] Receive image data transmitted from the unmanned aerial vehicle.

[0543] Input: Submitted image data

[0544] Output: Received image data

[0545] Specific operation: The server receives image data in real time.

[0546] Step 6: Save your data

[0547] server:

[0548] The received image data is stored in a database in a specific format.

[0549] Input: Received image data

[0550] Output: Image data stored in a database

[0551] Specific operation: The server "saves the image data in a database with date and location information."

[0552] Step 7: Selecting the image to analyze

[0553] server:

[0554] A new image is selected from the stored image data.

[0555] Input: Image data stored in a database

[0556] Output: Image data selected for analysis

[0557] Specific operation: The server selects the latest image for analysis.

[0558] Step 8: AI analysis of images

[0559] server:

[0560] The selected image data is input into the generative artificial intelligence (AI).

[0561] The AI ​​uses deep learning technology to analyze terrain and road conditions.

[0562] Input: Image data to be analyzed

[0563] Output: Analysis results (passability information)

[0564] Specific operation: The generative AI "analyzes the presence or absence of road damage and obstacles in the image."

[0565] Step 9: Generate a passable map

[0566] server:

[0567] Based on the analysis results, a passability map is generated that visually displays passability information on a map.

[0568] Input: Analysis results (passability information)

[0569] Output: Passability map

[0570] Specific operation: The server generates a map that shows passable roads in green and impassable roads in red.

[0571] Step 10: Timely updates

[0572] server:

[0573] The generated passability map is published to a web service or dedicated application.

[0574] Input: Passability map

[0575] Output: A public accessibility map for the user

[0576] Specific operation: The server "uploads the accessibility map to a user-accessible web portal."

[0577] Step 11: Real-time access

[0578] User:

[0579] Access real-time updated road availability maps using a web browser or dedicated application.

[0580] Input: Web browser or dedicated application

[0581] Output: Real-time road availability map

[0582] Specific operation: The user "checks traffic conditions in real time and plans evacuation routes and travel plans."

[0583] Through the above steps, this system quickly and accurately determines whether transportation infrastructure is passable during a disaster and provides users with useful information in real time.

[0584] (Application example 1)

[0585] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0586] During disasters, it is extremely important to quickly and accurately determine whether transportation infrastructure is passable. In particular, traffic information updated in real time is essential for autonomous vehicles to safely transport evacuees. However, with current technology, it is difficult to determine the extent of road damage and the presence of obstacles in real time and immediately reflect this information in the navigation system of autonomous vehicles. For this reason, improving the efficiency and ensuring safety of autonomous vehicle operations during disasters is a challenge.

[0587] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0588] In this invention, the server includes means for capturing images of the ground using an unmanned aerial vehicle equipped with a camera and communication capabilities, means for receiving and storing image data transmitted from the unmanned aerial vehicle, means for analyzing the stored image data and determining the condition of the terrain and roads, means for generating a passability map based on the passability information determined by the generating artificial intelligence, and means for providing the generated passability map to the navigation system of the autonomous vehicle. This enables the passability of transportation infrastructure in the event of a disaster to be quickly and accurately determined, enabling the autonomous vehicle to operate safely and efficiently.

[0589] An "unmanned aerial vehicle" is a device equipped with a camera and communication capabilities, which flies remotely or automatically and takes images of the earth's surface.

[0590] "Image data" is digital data that includes photographic information of the ground and roads taken by unmanned aerial vehicles.

[0591] "Generative AI" is an AI system that uses deep learning technology to analyze stored image data and automatically determine the condition of terrain and roads.

[0592] "Passability information" is information indicating whether a road is passable or not and whether there are any obstacles, obtained from the results of analysis by the generating artificial intelligence.

[0593] A "passable road map" is a map generated based on passable road information, which indicates passable roads in a predetermined color and is provided to the navigation system of an autonomous vehicle.

[0594] An "autonomous vehicle" is a vehicle that is controlled by a navigation system and can drive autonomously.

[0595] A "navigation system" is a system that uses information such as road availability maps to control the operation of autonomous vehicles and determine the optimal route to a destination.

[0596] This invention is a system for quickly determining whether transportation infrastructure is passable during disasters, and aims to provide real-time traffic information to the navigation systems of self-driving vehicles. This system is composed of unmanned aerial vehicles (drones), servers, generative artificial intelligence (AI), and self-driving vehicles.

[0597] First, the user sets up a flight plan for the unmanned aerial vehicle using a dedicated terminal. The flight plan targets the disaster-stricken area and specifies the route and photography points that the unmanned aerial vehicle will fly automatically. For example, it can be set to prioritize coverage of major highways and evacuation routes.

[0598] According to the set flight plan, the unmanned aerial vehicle automatically flies within a designated area and takes images of the ground and roads. The captured image data is sent from the unmanned aerial vehicle to a server in real time. The server stores the received image data in a specific format.

[0599] The saved image data is input into the generative AI and analyzed using deep learning technology. The generative AI automatically determines road conditions and changes in the terrain, and obtains passability information. For example, it determines the state of road damage and the presence of obstacles, and if it determines that a road is impassable, it analyzes the road as "impassable."

[0600] Next, the server generates a passable / unpassable map based on the analysis results obtained by the AI. On this map, passable roads are displayed in green, impassable roads in red, and roads requiring caution in yellow. The generated passable / unpassable map is provided in real time to the navigation system of the autonomous vehicle, which then uses this information to select the optimal route.

[0601] For example, if a major highway is deemed impassable due to a landslide, an autonomous vehicle will check the road availability map in real time and automatically select an alternative passable route. This information will also be presented to the driver and rescue teams, ensuring safe and efficient travel.

[0602] An example prompt might be: "Analyze traffic conditions of transportation infrastructure in real time during disasters and provide optimal route information for autonomous vehicles. Specifically, use a deep learning model to analyze road photos taken by a drone and detect landslides and road damage."

[0603] As a result, by using this system, it will be possible to quickly and accurately determine whether transportation infrastructure is passable during a disaster, enabling self-driving vehicles to operate safely and efficiently.

[0604] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0605] Step 1:

[0606] The user sets up a flight plan for the unmanned aerial vehicle using a dedicated terminal. The user refers to a map of the disaster area and specifies the route (input) that the unmanned aerial vehicle should take and the shooting points. This flight plan (output) is sent to the unmanned aerial vehicle.

[0607] Step 2:

[0608] The unmanned aerial vehicle flies automatically according to a set flight plan and periodically takes images of the ground and roads (specific operation). The image data taken by the unmanned aerial vehicle (input) is sent to a server in real time (output).

[0609] Step 3:

[0610] The server receives image data sent from the unmanned aerial vehicle (input) and stores it in a database in a specific format (output). The stored image data is then analyzed (specific operation).

[0611] Step 4:

[0612] The server inputs the stored image data into a generative AI model (generative AI model) (input) and analyzes it using deep learning technology (specific operation). The generative AI analyzes the image data and automatically determines the terrain and road conditions (data processing). The analysis results (output) include information on whether the road is passable.

[0613] Step 5:

[0614] The server generates a passable / non-passable map based on the analysis results obtained by the generation AI (specific operation). The input is the analysis results, and the output is a passable / non-passable map. On this map, passable roads are shown in green, impassable roads in red, and roads requiring caution in yellow.

[0615] Step 6:

[0616] The server provides the generated route availability map to the navigation system of the autonomous vehicle (output). The autonomous vehicle selects a safe and efficient route based on the route availability map (action) and starts moving to the destination (specific action).

[0617] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0618] ---

[0619] The present invention is a system that quickly determines whether traffic infrastructure is passable during a disaster, and further recognizes the user's emotions and provides support information.

[0620] System Overview

[0621] This system uses an unmanned aerial vehicle (drone) equipped with a camera and communication functions to capture images of the ground surface and send the image data to a server. The server stores the received image data and uses generative artificial intelligence (hereafter referred to as generative AI) to analyze the terrain and road conditions. Based on the analysis results, a passable / unpassable map is generated and provided to the user. The system also incorporates an emotion engine that recognizes the user's emotions and provides support information according to the user's emotional state.

[0622] Program processing

[0623] 1. Setting up a flight plan:

[0624] Using an application on a dedicated device, the user refers to a map of the disaster area and specifies the flight path and shooting points for the unmanned aerial vehicle.

[0625] 2. Automatic flight and photography of unmanned aerial vehicles:

[0626] The terminal transmits the set flight plan to the unmanned aerial vehicle, and the unmanned aerial vehicle automatically flies in the designated area according to the received flight plan.

[0627] The unmanned aerial vehicle periodically takes high-resolution photographs of the ground and roads along its flight path.

[0628] The captured photo data is sent to the server in real time.

[0629] 3. Receiving and storing data:

[0630] The server receives the image data sent from the unmanned aerial vehicle and stores it in a database.

[0631] 4. Image Analysis:

[0632] The server sequentially inputs the stored image data into the generating AI, which then analyzes the road conditions using deep learning technology.

[0633] For example, it can detect obstacles such as landslides in an image and determine that the location is "impassable."

[0634] 5. Generate a passability map:

[0635] The server determines passability information based on the analysis results and generates a passability map.

[0636] The road availability map shows passable roads in green, impassable roads in red, and roads requiring caution in yellow.

[0637] 6. Leveraging the Emotion Engine:

[0638] When providing a passability map, the server analyzes the user's facial expressions and voice using an emotion engine to recognize the user's emotions.

[0639] For example, if the system recognizes that a user is feeling anxious or stressed, it will provide not only a map of whether a route is passable, but also additional support information such as the location of evacuation shelters and information on the distribution of relief supplies.

[0640] 7. Information provision:

[0641] The server makes available to users road availability maps and additional assistance information via web services or dedicated applications.

[0642] Users can use a web browser or dedicated application to access real-time updated route availability maps and assistance information, and plan evacuation routes and travel plans.

[0643] Specific examples

[0644] For example, immediately after an earthquake occurs, an unmanned aerial vehicle flies over the affected area and takes photos of the roads. The image data is then sent to a server, where the AI ​​analyzes the road for damage and obstacles. If a landslide is confirmed at a point on a major highway, the location will be deemed "impassable" and displayed in red on a road access map.

[0645] At the same time, the emotion engine analyzes the facial expressions and voice of users viewing the map, and if it detects anxiety, it also provides information on the locations of evacuation shelters and relief supply distribution points. In this way, users can obtain appropriate routes and support information to evacuate quickly and safely.

[0646] As described above, the present invention is a system that quickly and accurately determines whether transportation infrastructure is passable during a disaster, and provides support information that takes the user's emotions into consideration, thereby effectively supporting the evacuation of disaster victims and rescue efforts.

[0647] The processing flow will be explained below.

[0648] ---

[0649] Step 1:

[0650] Users use an application on a dedicated device to refer to a map of the disaster-affected area and set the flight path and shooting points for the unmanned aerial vehicle (drone).

[0651] Step 2:

[0652] The terminal transmits the set flight plan to the unmanned aerial vehicle, and the unmanned aerial vehicle automatically flies in the designated area according to the received flight plan.

[0653] Step 3:

[0654] The drone periodically takes high-resolution photographs of the ground and roads along its flight path.

[0655] Step 4:

[0656] The unmanned aerial vehicle transmits the captured photo data to a server in real time.

[0657] Step 5:

[0658] The server receives the image data sent from the unmanned aerial vehicle and stores it in a database in a specific format.

[0659] Step 6:

[0660] The server sequentially inputs the stored image data into the generating AI, which then analyzes the road conditions using deep learning technology.

[0661] Step 7:

[0662] Based on the results of the analysis by the generation AI, the server determines the condition of the terrain and roads and determines whether each road is passable.

[0663] Step 8:

[0664] The server generates a passable road map based on the determined passable road information, showing passable roads in green, impassable roads in red, and roads requiring caution in yellow.

[0665] Step 9:

[0666] The server activates an emotion engine in the process of providing the user with a passable / unpassable map. The emotion engine analyzes the user's facial expressions and voice to recognize the user's emotions.

[0667] Step 10:

[0668] If the server determines that the user is feeling anxious or stressed based on the analysis results of the emotion engine, it will provide additional support information, such as a passable / unpassable road map, the location of evacuation shelters, and information on the distribution of relief supplies.

[0669] Step 11:

[0670] Using a web browser or dedicated application, users can access real-time updated route availability maps and assistance information to plan safe and appropriate evacuation routes and travel plans.

[0671] ---

[0672] The above are the specific processing steps for carrying out the present invention. These processing steps enable quick and accurate determination of whether an area is passable during a disaster, and provide appropriate support information according to the user's emotional state, thereby effectively supporting evacuation and rescue efforts.

[0673] Example 2

[0674] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0675] When a natural disaster occurs, it is difficult to quickly and accurately grasp information on whether the transportation infrastructure in the affected area is passable, which hinders the efficiency of evacuation and rescue efforts.In addition, support information that takes into account the emotional state of the victims is not provided, which increases the mental burden and hinders appropriate evacuation behavior.

[0676] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for capturing images of the ground surface using an unmanned aerial vehicle equipped with a camera and communication function, means for receiving and storing image data transmitted from the unmanned aerial vehicle, means for generating an AI generation system for analyzing the stored image data and determining the condition of the terrain and roads, means for generating a passability map based on the passability information determined by the AI ​​generation system, means for providing the generated passability map to the user, and means for analyzing the user's facial expressions and voice and providing support information based on the user's emotional state. This makes it possible to quickly and accurately provide passability information for disaster-stricken areas and to provide support information that takes the user's emotional state into consideration.

[0677] An "unmanned aerial vehicle equipped with a camera and communication capabilities" is a flying device with autonomous flight capabilities that is equipped with a camera and communication module for capturing image data and transmitting it wirelessly.

[0678] "Image data" refers to digital information such as photographs and videos taken by unmanned aerial vehicles of the conditions of the ground and roads.

[0679] "Generative AI" is a software system equipped with advanced machine learning technology that analyzes large amounts of image data and determines the condition of terrain and roads.

[0680] "Passability information" is digital information that indicates whether a road is passable or not, based on image data analyzed by the generating AI.

[0681] A "passability map" is a map for visually displaying passability information, and indicates passable roads, impassable roads, and roads requiring caution using different colors.

[0682] "Emotional state" refers to the emotional state of the user that is inferred based on facial expressions and voice analysis.

[0683] "Support information" is additional information that is useful in the event of a disaster, such as information on the location of evacuation shelters and distribution of relief supplies, provided according to the user's emotional state.

[0684] The present invention is a system that quickly determines whether traffic infrastructure is passable during a disaster, and further recognizes the user's emotions and provides support information. How the present invention is specifically implemented will be described below.

[0685] System Overview

[0686] The system of the present invention uses an unmanned aerial vehicle (drone) equipped with a camera and communication capabilities. The unmanned aerial vehicle takes high-resolution images of the ground and roads and transmits them to a server in real time. The server stores the received image data and analyzes the terrain and road conditions using a generative AI model. Based on the analysis results, a passability map is generated and provided to the user. The system also analyzes the user's facial expressions and voice to recognize their emotional state and provides support information based on the user's emotional state.

[0687] Hardware and software used

[0688] Unmanned aerial vehicle (drone): Equipped with a high-resolution camera and communication module.

[0689] Server: Equipped with a large database and high-performance processors to run generative AI models.

[0690] User device: A smartphone or tablet capable of running a web browser and dedicated applications.

[0691] Examples and prompts

[0692] For example, immediately after an earthquake, an unmanned aerial vehicle flies over the affected area and takes photographs of the condition of major roads and bridges. The captured image data is sent to a server in real time, where a generative AI model analyzes the road for damage and obstacles. For example, if a landslide is detected in the image, the location is determined to be "impassable" and displayed in red on a passability map.

[0693] Example prompt sentence:

[0694] Example prompt for analyzing the accessibility of transportation infrastructure:

[0695] Analyze road image data of disaster-stricken areas taken by unmanned aerial vehicles to determine whether roads are passable. Detect obstacles such as landslides and cracks in the roads, and create a passable map based on the analysis results.

[0696] Example prompt for user sentiment analysis:

[0697] Analyze the facial expressions and voice of users viewing the road availability map to detect their emotional state of anxiety or stress. Depending on their emotional state, provide them with additional support information, such as the location of evacuation shelters and information on the distribution of relief supplies.

[0698] When a user checks the passability map using a web browser or a dedicated application, the emotion engine analyzes the user's facial expressions and voice at the time and displays appropriate support information. For example, if the emotion engine detects the user's anxiety, it will add information about the location of evacuation shelters and relief supplies.

[0699] As described above, the present invention can quickly and accurately provide information on whether a disaster area is passable, and can provide support information that takes into account the user's emotional state, thereby effectively supporting evacuation and rescue operations.

[0700] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0701] Step 1: Set up a flight plan

[0702] User

[0703] The user launches the application on the dedicated device and displays a map of the disaster-affected area. Next, the user sets the flight path and shooting points for the drone on the map. For example, the user selects important areas such as major roads and bridges in the disaster area. Once the settings are complete, the user finalizes the flight plan and sends it from the device to the drone.

[0704] Input: Map of disaster area, flight route and photo points

[0705] Output: Flight plan data sent to unmanned aerial vehicles

[0706] Specific actions: Pointing and clicking on the map, pressing the flight plan confirmation button

[0707] Step 2: Automatic flight and photography of the unmanned aerial vehicle

[0708] unmanned aerial vehicle

[0709] The drone begins automatic flight based on the flight plan received from the device. It uses a GPS system to follow an accurate flight path, and when it reaches a designated shooting point, it takes photos of the ground and roads with a high-resolution camera. The captured image data is sent to a server in real time.

[0710] Input: Flight plan data

[0711] Output: The captured image data is sent to the server.

[0712] Specific operations: Obtaining location information using GPS, taking photos using a high-resolution camera, and transmitting data using a wireless communication module

[0713] Step 3: Receiving and storing data

[0714] server

[0715] The server receives image data sent from the unmanned aerial vehicle in real time and stores it in a database. The image data also contains metadata such as the date and time of the photo and GPS coordinates, so it is stored together with these.

[0716] Input: Captured image data (including metadata)

[0717] Output: Image data stored in a database

[0718] Specific operation: Receiving data via wireless communication and writing it to a database

[0719] Step 4: Image analysis

[0720] server

[0721] The server inputs the stored image data into a generative AI model and analyzes the images using deep learning technology. Specifically, it detects road damage and obstacles (e.g., landslides and fallen trees). Based on the results of this analysis, it determines whether the road is passable.

[0722] Input: Image data stored in a database

[0723] Output: Analyzed passability information (e.g. passable, not passable)

[0724] Specific operations: Data input to generative AI model, image analysis using deep learning

[0725] Step 5: Generate a passability map

[0726] server

[0727] The server generates a passable road map based on the results of image analysis, showing passable roads in green, impassable roads in red, and roads requiring caution in yellow.

[0728] Input: Parsed passability information

[0729] Output: Passability map

[0730] Specific operation: Visualization of passability information, color-coded display on the map

[0731] Step 6: Leverage your emotional engine

[0732] server

[0733] The server uses an emotion engine to analyze the user's facial expressions and voice as they browse the passability map. The emotion engine uses facial recognition software and tone analysis tools to determine the user's emotional state, and if it detects anxiety or stress, it provides additional support information.

[0734] Input: User's facial expression data, voice data

[0735] Output: Support information based on the user's emotional state

[0736] Specific operations: facial expression recognition and voice analysis, emotional state estimation, and provision of support information

[0737] Step 7: Provide information

[0738] server

[0739] The server provides users with road availability maps and additional assistance information via web services and dedicated applications. The information is updated in real time and can be used whenever needed.

[0740] User

[0741] Users can use a web browser or a dedicated application to check the passability map and support information, and plan evacuation routes and travel plans.

[0742] Input: Passability map, additional assistance information

[0743] Output: User viewing and use of information

[0744] Specific actions: Operating a web browser or application, planning an evacuation route

[0745] In this way, this system can quickly and accurately determine whether transportation infrastructure is passable during a disaster, and by providing support information that takes into account the user's emotional state, it can effectively support the evacuation of disaster victims and rescue efforts.

[0746] (Application example 2)

[0747] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0748] In the past, it was difficult to quickly and accurately determine whether transportation infrastructure was passable during disasters. Furthermore, when providing evacuation and support information, appropriate support was not provided that took into account the user's feelings. Therefore, there is a need for an information provision system that allows disaster victims to evacuate safely and receive appropriate support.

[0749] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0750] In this invention, the server includes means for capturing images of the ground surface using an unmanned aerial vehicle equipped with a camera and communication capabilities, means for receiving and storing image data transmitted from the unmanned aerial vehicle, artificial intelligence generation means for analyzing the stored image data and determining the condition of the terrain and roads, means for generating a passability map, means for providing the generated passability map, emotion recognition engine means for analyzing a user's facial expressions and voice and recognizing emotions, and means for providing support information based on the user's emotions. This makes it possible to quickly and accurately determine the passability of transportation infrastructure in the event of a disaster and to provide appropriate support information that takes the user's emotions into consideration.

[0751] A "camera" is a device that captures images or videos.

[0752] "Communication function" means the function of transmitting and receiving data wirelessly or via wire.

[0753] An "unmanned aerial vehicle" is an aircraft that flies remotely or autonomously to conduct observations and collect data.

[0754] "Image data" is digital information of still or moving images captured by a camera.

[0755] "Generative AI" refers to an AI technology that recognizes patterns based on large amounts of data and performs data analysis and judgments.

[0756] "Deep learning technology" is a technology in the field of artificial intelligence that uses multi-layer neural networks to analyze data with high precision.

[0757] A "passable roads map" displays information on a map in different colors, indicating which roads are passable, which roads are impassable, and which roads require caution.

[0758] An "emotion recognition engine" is a technology that analyzes a user's facial expressions and voice to identify their emotional state.

[0759] "Support information" refers to information for supporting users' evacuation and daily life in the event of a disaster.

[0760] This invention is a system that quickly determines whether traffic infrastructure is passable during a disaster, and further recognizes the user's emotions and provides support information. This system is configured as follows.

[0761] System Overview

[0762] This system uses an unmanned aerial vehicle equipped with a camera and communications capabilities to capture images of the ground surface and send the image data to a server. The server stores the received image data and uses generative artificial intelligence to analyze the terrain and road conditions. Based on the analysis results, a passable / unpassable map is generated and provided to the user. The system also incorporates an emotion engine that recognizes the user's emotions and provides support information according to the user's emotional state.

[0763] Hardware and software used

[0764] The system's main hardware consists of an unmanned aerial vehicle (drone) equipped with a camera and communication functions, a high-performance server, and a smartphone used by the user. The unmanned aerial vehicle is equipped with a high-resolution camera and communication functions that transmit image data to the server in real time.

[0765] The server uses Python, OpenCV, TensorFlow, Keras, and other deep learning libraries to analyze image data. The backend system uses Django, and the frontend uses React Native. OpenCV and a custom model are used for emotion recognition.

[0766] Data analysis and provision details

[0767] The server receives image data sent from the unmanned aerial vehicle and stores it in a database. The stored image data is then input into the artificial intelligence that generates it, which uses deep learning technology to analyze the terrain and road conditions. It detects obstacles such as landslides and damaged roads, classifies those areas as "impassable," and generates a passability map based on the results.

[0768] The generated road availability map shows passable roads in green, impassable roads in red, and roads requiring caution in yellow, and is updated in real time. Users can access the map and assistance information using a dedicated smartphone application.

[0769] The emotion recognition engine analyzes facial expressions and voice using the camera and microphone on the user's smartphone. For example, it uses OpenCV and a custom model to recognize emotions from the user's facial expressions and provides appropriate support information. For users who are feeling anxious or stressed, it provides information on evacuation shelters and relief supply distribution locations along with a map showing whether roads are passable.

[0770] Specific examples

[0771] For example, immediately after an earthquake occurs, an unmanned aerial vehicle flies over the affected area and takes photos of the roads. The image data is then sent to a server, where the AI ​​analyzes the road for damage and obstacles. If a landslide is confirmed at a point on a major highway, the location is deemed "impassable" and displayed in red on a passability map. At the same time, the emotion engine analyzes the facial expressions and voices of users viewing the passability map, and if it detects anxiety, it also provides information on the locations of evacuation centers and relief supply distribution points. In this way, users can obtain appropriate routes and support information to evacuate quickly and safely.

[0772] Prompt Sentence Examples

[0773] For post-disaster traffic analysis, high-resolution image data is sent from an unmanned aerial vehicle to a server and road conditions are determined using a deep learning model. We would like to build a system that analyzes the user's facial expressions and voice and provides appropriate support information based on their emotions. First, we train an AI model to identify obstacles such as landslides from image data, then develop an emotion recognition model and customize the support information based on that. Please suggest an appropriate deep learning library and model structure.

[0774] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0775] Step 1:

[0776] The user uses an application on a dedicated device to refer to a map of the disaster-stricken area and specify the flight path and shooting points of the unmanned aerial vehicle. The input is the map data specified by the user, and the output is a flight plan for the unmanned aerial vehicle. In concrete terms, the user operates the map on the application, placing pins to set shooting points.

[0777] Step 2:

[0778] The terminal transmits the set flight plan to the unmanned aerial vehicle, which then automatically flies the designated area according to the received flight plan. The input is flight plan data, and the output is flight instructions for the unmanned aerial vehicle. The unmanned aerial vehicle flies along the specified route and periodically takes photos of the ground and roads.

[0779] Step 3:

[0780] The unmanned aerial vehicle sends the photographic data it captures to a server in real time. The input is the captured image data, and the output is the image data sent to the server. Specifically, the images captured by the unmanned aerial vehicle's camera are uploaded to the server via the communication function.

[0781] Step 4:

[0782] The server receives image data sent from the unmanned aerial vehicle and stores it in a database. The input is the image data received from the unmanned aerial vehicle, and the output is the image data stored in the database. Specifically, the server converts the data into an appropriate format and stores it in storage.

[0783] Step 5:

[0784] The server sequentially inputs the stored image data into the generative AI model and uses deep learning technology to analyze the road conditions. The stored image data is used as input, and the analysis results are obtained as output. Specifically, the image is input into the model using TensorFlow and Keras, and obstacles are detected and the condition is determined.

[0785] Step 6:

[0786] The server generates a passable / non-passable map based on the analysis results. The analysis results are input, and the passable / non-passable map is generated as output. Specifically, it uses a map generation algorithm to color-code passable roads in green, impassable roads in red, and roads requiring caution in yellow.

[0787] Step 7:

[0788] The server makes the generated passability map available to users via a web service or dedicated application. The generated passability map is the input, and the passability map provided to the user is generated as the output. Specifically, the map information is displayed on a web page using Django and updated in real time.

[0789] Step 8:

[0790] Users use a smartphone application to access real-time updated route availability maps and support information to plan evacuation routes and travel plans. A route availability map is available as input, and the user's confirmed evacuation route and support information are output. Specifically, a React Native application is used to enable users to easily obtain information.

[0791] Step 9:

[0792] When providing a passability map, the server analyzes the user's facial expressions and voice using an emotion recognition engine to recognize the user's emotions. The input is the user's facial expression image and voice data, and the output is the user's emotional state. Specifically, the server analyzes the user's emotions using OpenCV and a custom emotion recognition model.

[0793] Step 10:

[0794] The server provides additional support information, such as the location of evacuation shelters and information on the distribution of relief supplies, depending on the user's emotional state. The user's emotional state is input, and appropriate support information is provided as output. Specifically, the server selects appropriate support information based on the emotion recognition results and notifies the user.

[0795] Through the above processing steps, the system can quickly and accurately determine whether transportation infrastructure is passable during a disaster, and can also provide support information that takes into account the user's emotions.

[0796] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0797] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0798] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0799] [Third embodiment]

[0800] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0801] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0802] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0803] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0804] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0805] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0806] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0807] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0808] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0809] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0810] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0811] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0812] ---

[0813] The present invention is a system for quickly determining whether traffic infrastructure is passable during a disaster, and is configured as follows.

[0814] This system first uses an unmanned aerial vehicle (drone) equipped with a camera and communication functions to capture images of the ground surface. The unmanned aerial vehicle automatically flies over a designated area according to a flight plan, taking high-resolution photos of the ground surface and roads at regular intervals. The captured photo data is sent to a server in real time.

[0815] The server receives image data sent from the unmanned aerial vehicle and stores it in a database in a specific format. The received image data is analyzed sequentially and input into the generative AI. The generative AI then analyzes the image data using deep learning technology to automatically determine the condition of the terrain and roads. This allows it to accurately detect the extent of road damage and the presence or absence of obstacles, and obtain information on whether the road is passable.

[0816] Based on the analysis results, the server generates a passable / unpassable map. This map displays passable roads in a predetermined first color (e.g., green), impassable roads in a predetermined second color (e.g., red), and roads requiring caution in a predetermined third color (e.g., yellow). The generated passable / unpassable map is immediately provided to the user via a web service or dedicated application.

[0817] Next, the program processing of this system will be explained in natural language.

[0818] 1. Setting up a flight plan:

[0819] Using an application on a dedicated device, the user refers to a map of the disaster area and specifies the flight path and shooting points for the unmanned aerial vehicle.

[0820] 2. Automatic flight and photography of unmanned aerial vehicles:

[0821] The terminal transmits the set flight plan to the unmanned aerial vehicle.

[0822] The unmanned aerial vehicle follows a specified flight plan, automatically flies over a designated area, and periodically takes photographs of the ground and roads.

[0823] The unmanned aerial vehicle transmits the captured photo data to a server in real time.

[0824] 3. Receiving and storing data:

[0825] The server receives image data sent from the unmanned aerial vehicle and stores it in a database in a specific format.

[0826] 4. Image Analysis:

[0827] The server sequentially inputs the stored image data into the generative artificial intelligence, which then analyzes the road conditions using deep learning technology.

[0828] For example, if a landslide is visible in some images, the generative AI will determine that road is "impassable."

[0829] 5. Generate a passability map:

[0830] Based on the analysis results, the server generates a passability map that visually displays passability information on a map.

[0831] The road availability map shows passable roads in green, impassable roads in red, and roads requiring caution in yellow.

[0832] 6. Information provision:

[0833] The server publishes the generated passability map to web services or dedicated applications and provides it to users.

[0834] Users can use a web browser or dedicated application to access real-time updated road safety maps and plan evacuation routes and movements.

[0835] As a specific example, immediately after a disaster occurs, an unmanned aerial vehicle flies over the affected area and takes photos of the roads. The image data is sent to a server, and the artificial intelligence analyzes the road for damage and obstacles. For example, if a landslide is confirmed at a certain point on a major highway, the road will be determined to be "impassable" and displayed in red on the passability map. If the road is confirmed to be safe at other points, those parts will be displayed in green. In this way, the latest passability information is provided in real time, helping users to travel quickly and safely.

[0836] As described above, the present invention is a system that quickly and accurately determines whether transportation infrastructure is passable during a disaster, and efficiently supports the evacuation of disaster victims and rescue operations.

[0837] The processing flow will be explained below.

[0838] ---

[0839] Step 1:

[0840] Users use an application on a dedicated device to refer to a map of the disaster-affected area and specify the flight path and shooting points for the unmanned aerial vehicle (drone).

[0841] Step 2:

[0842] The terminal transmits the set flight plan to the unmanned aerial vehicle, and the unmanned aerial vehicle automatically flies in the designated area according to the received flight plan.

[0843] Step 3:

[0844] The drone periodically takes high-resolution photographs of the ground and roads along its flight path.

[0845] Step 4:

[0846] The unmanned aerial vehicle transmits the captured photo data to a server in real time.

[0847] Step 5:

[0848] The server receives the image data sent from the unmanned aerial vehicle and stores it in a database in a specific format.

[0849] Step 6:

[0850] The server sequentially inputs the stored image data into the generative artificial intelligence.

[0851] Step 7:

[0852] The server uses generative artificial intelligence to analyze the image data, using deep learning technology to automatically determine the terrain and road conditions.

[0853] Step 8:

[0854] The server then determines whether a road is passable based on the analysis results. For example, it may classify a point in the image where a landslide has been confirmed as "impassable."

[0855] Step 9:

[0856] The server generates a passability map based on the passability information.

[0857] Step 10:

[0858] The server makes the passability map available to users via a web service or dedicated application.

[0859] Step 11:

[0860] Users can access the road closure map using a web browser or a dedicated application, and create evacuation routes and travel plans based on information updated in real time.

[0861] ---

[0862] These are the specific processing steps of the program for this system. This flow enables rapid and accurate understanding of the state of transportation infrastructure during a disaster, and supports effective evacuation and relief efforts.

[0863] Example 1

[0864] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0865] With conventional systems, it is difficult to quickly and accurately determine whether roads are passable during a disaster. Specifically, there is a lack of means to analyze road conditions in real time and provide that information to users immediately. As a result, evacuation plans and rescue operations cannot be carried out quickly in an emergency, and there is a risk that the damage will worsen.

[0866] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0867] In this invention, the server includes a means for a user to set a flight plan for the unmanned aerial vehicle via a dedicated terminal, a means for the unmanned aerial vehicle to automatically fly according to the set flight plan and transmit captured data to the server in real time, and a means for presenting a passability map at startup based on the passability information generated by the server. This makes it possible to quickly and accurately determine whether roads are passable during a disaster and provide the information to the user in real time.

[0868] An "unmanned aerial vehicle" is a device equipped with a camera and communication capabilities that flies automatically according to a specified flight plan.

[0869] "Image data" refers to data that includes photographic information of the ground and roads taken by unmanned aerial vehicles.

[0870] "Analysis" is the process of determining the terrain and road conditions based on image data stored on the server.

[0871] "Generative AI" is an AI system that uses image data to analyze terrain and road conditions using deep learning technology.

[0872] A "flight plan" is information about the flight path and photographing points of an unmanned aerial vehicle that is set by the user via a dedicated terminal.

[0873] A "dedicated terminal" is a device that a user uses to set a flight plan for an unmanned aerial vehicle, and includes smartphones, tablets, etc.

[0874] The "server" is a computer system that receives and stores image data transmitted from unmanned aerial vehicles, then analyzes the data and provides the generated passable / unpassable map.

[0875] A "passable roads map" is a map that visually displays passable roads, impassable roads, and roads requiring caution, using different colors based on the analysis results.

[0876] "Real-time" means that image data captured by an unmanned aerial vehicle is sent to a server almost immediately, with an extremely short time lag for analysis and provision of information.

[0877] "User" refers to a person who uses a dedicated terminal to set up a flight plan for an unmanned aerial vehicle and view a passability map.

[0878] To specifically implement this invention, an unmanned aerial vehicle (drone), a dedicated terminal (smartphone, tablet, etc.), and a server are required. The purpose of this system is to quickly determine whether transportation infrastructure is passable during a disaster and provide users with information in real time.

[0879] System Overview

[0880] Hardware Configuration

[0881] Unmanned aerial vehicle (drone): Equipped with a camera and communication capabilities, it flies automatically according to a specified flight plan.

[0882] Dedicated terminal: A device used by a user to set up a flight plan for an unmanned aerial vehicle, including a smartphone or tablet.

[0883] Server: A computer system that receives, stores, and analyzes image data sent from unmanned aerial vehicles.

[0884] Software Configuration

[0885] Dedicated terminal application: An application that allows users to set up flight plans.

[0886] Server-side analysis software: Software that uses deep learning technology to analyze image data.

[0887] Web service / dedicated application: Software that provides the generated accessibility map to the user.

[0888] Processing flow

[0889] Setting up a flight plan

[0890] The user launches the application on the dedicated device and displays a map of the affected area. The user then specifies the flight path and shooting points for the unmanned aircraft. For example, the user might "set a flight path from the center of Sendai City to the main road in the east."

[0891] Automated flight and data transmission

[0892] The terminal sends the set flight plan to the unmanned aerial vehicle. The unmanned aerial vehicle automatically flies over the specified area according to the specified flight plan, periodically taking photos of the ground and roads. The captured photo data is sent to the server in real time. Example: The unmanned aerial vehicle "takes photos every 10 minutes while flying along the specified route, and immediately sends the image data to the server after taking the photos."

[0893] Receiving and storing data

[0894] The server receives image data sent from the unmanned aerial vehicle in real time and stores it in a database in a specific format. Example: The server "receives image data in real time and stores the image data in a database with date and location information."

[0895] Image analysis and generation of passability maps

[0896] The server sequentially inputs the stored image data into generative artificial intelligence (AI), which uses deep learning technology to analyze the condition of the road. The AI ​​analyzes the image data and determines whether the road is damaged or has any obstacles. For example, if a landslide is confirmed in some of the images, the AI ​​will determine that the road is "impassable." The server generates a passable / unpassable map based on the analysis results. The passable / unpassable map shows passable roads in green, impassable roads in red, and roads requiring caution in yellow. Example: The server "generates a map that shows areas where landslides have been confirmed in red, and other safe areas in green."

[0897] Providing information

[0898] The server publishes the generated route availability map to a web service or dedicated application and provides it to users. Users can use a web browser or dedicated application to access the route availability map, which is updated in real time, and plan evacuation routes and movements. Example: A user can "check the traffic conditions in real time and plan evacuation routes and movements."

[0899] Prompt Sentence Examples

[0900] "After a disaster occurs in central Sendai, please use a drone to photograph the traffic conditions of major roads in the eastern part of the city, send the data to a server for analysis, and generate a map of whether roads are passable or not."

[0901] As described above, this system quickly and accurately determines whether transportation infrastructure is passable during a disaster, and provides users with useful information in real time.

[0902] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0903] Step 1: Set up a flight plan

[0904] User:

[0905] The user launches the application on the dedicated device and displays a map of the disaster area.

[0906] The user specifies the flight path and shooting points of the unmanned aerial vehicle on the map.

[0907] Input: Map information of the disaster area, flight path and photo points

[0908] Output: Flight plan data for unmanned aerial vehicles

[0909] Specific actions: The user "tap the tablet screen to mark the shooting point of the unmanned aerial vehicle," "confirm the flight plan, and press the start button."

[0910] Step 2: Submit your flight plan

[0911] Device:

[0912] The configured flight plan is transmitted to the unmanned aerial vehicle.

[0913] Input: Flight plan data

[0914] Output: Flight plan sent to the unmanned aerial vehicle

[0915] Specific operation: The device "wirelessly transmits flight plan data to the unmanned aerial vehicle" and "displays a notification confirming receipt."

[0916] Step 3: Automated flight of the unmanned aerial vehicle

[0917] Unmanned Aerial Vehicles:

[0918] The unmanned aerial vehicle will automatically begin flying based on the received flight plan.

[0919] Input: Flight plan

[0920] Output: Captured image data

[0921] Specific operation: The unmanned aircraft "turns on its engines and ascends to a specified altitude," then "flies automatically along a set route, repeatedly taking photographs and transmitting data."

[0922] Step 4: Sending data

[0923] Unmanned Aerial Vehicles:

[0924] The captured photo data is sent to the server in real time.

[0925] Input: Captured image data

[0926] Output: Image data sent to the server

[0927] Specific operation: The unmanned aircraft "sends image data to the server immediately after taking the image."

[0928] Step 5: Receiving the data

[0929] server:

[0930] Receive image data transmitted from the unmanned aerial vehicle.

[0931] Input: Submitted image data

[0932] Output: Received image data

[0933] Specific operation: The server receives image data in real time.

[0934] Step 6: Save your data

[0935] server:

[0936] The received image data is stored in a database in a specific format.

[0937] Input: Received image data

[0938] Output: Image data stored in a database

[0939] Specific operation: The server "saves the image data in a database with date and location information."

[0940] Step 7: Selecting the image to analyze

[0941] server:

[0942] A new image is selected from the stored image data.

[0943] Input: Image data stored in a database

[0944] Output: Image data selected for analysis

[0945] Specific operation: The server selects the latest image for analysis.

[0946] Step 8: AI analysis of images

[0947] server:

[0948] The selected image data is input into the generative artificial intelligence (AI).

[0949] The AI ​​uses deep learning technology to analyze terrain and road conditions.

[0950] Input: Image data to be analyzed

[0951] Output: Analysis results (passability information)

[0952] Specific operation: The generative AI "analyzes the presence or absence of road damage and obstacles in the image."

[0953] Step 9: Generate a passable map

[0954] server:

[0955] Based on the analysis results, a passability map is generated that visually displays passability information on a map.

[0956] Input: Analysis results (passability information)

[0957] Output: Passability map

[0958] Specific operation: The server generates a map that shows passable roads in green and impassable roads in red.

[0959] Step 10: Timely updates

[0960] server:

[0961] The generated passability map is published to a web service or dedicated application.

[0962] Input: Passability map

[0963] Output: A public accessibility map for the user

[0964] Specific operation: The server "uploads the accessibility map to a user-accessible web portal."

[0965] Step 11: Real-time access

[0966] User:

[0967] Access real-time updated road availability maps using a web browser or dedicated application.

[0968] Input: Web browser or dedicated application

[0969] Output: Real-time road availability map

[0970] Specific operation: The user "checks traffic conditions in real time and plans evacuation routes and travel plans."

[0971] Through the above steps, this system quickly and accurately determines whether transportation infrastructure is passable during a disaster and provides users with useful information in real time.

[0972] (Application example 1)

[0973] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0974] During disasters, it is extremely important to quickly and accurately determine whether transportation infrastructure is passable. In particular, traffic information updated in real time is essential for autonomous vehicles to safely transport evacuees. However, with current technology, it is difficult to determine the extent of road damage and the presence of obstacles in real time and immediately reflect this information in the navigation system of autonomous vehicles. For this reason, improving the efficiency and ensuring safety of autonomous vehicle operations during disasters is a challenge.

[0975] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0976] In this invention, the server includes means for capturing images of the ground using an unmanned aerial vehicle equipped with a camera and communication capabilities, means for receiving and storing image data transmitted from the unmanned aerial vehicle, means for analyzing the stored image data and determining the condition of the terrain and roads, means for generating a passability map based on the passability information determined by the generating artificial intelligence, and means for providing the generated passability map to the navigation system of the autonomous vehicle. This enables the passability of transportation infrastructure in the event of a disaster to be quickly and accurately determined, enabling the autonomous vehicle to operate safely and efficiently.

[0977] An "unmanned aerial vehicle" is a device equipped with a camera and communication capabilities, which flies remotely or automatically and takes images of the earth's surface.

[0978] "Image data" is digital data that includes photographic information of the ground and roads taken by unmanned aerial vehicles.

[0979] "Generative AI" is an AI system that uses deep learning technology to analyze stored image data and automatically determine the condition of terrain and roads.

[0980] "Passability information" is information indicating whether a road is passable or not and whether there are any obstacles, obtained from the results of analysis by the generating artificial intelligence.

[0981] A "passable road map" is a map generated based on passable road information, which indicates passable roads in a predetermined color and is provided to the navigation system of an autonomous vehicle.

[0982] An "autonomous vehicle" is a vehicle that is controlled by a navigation system and can drive autonomously.

[0983] A "navigation system" is a system that uses information such as road availability maps to control the operation of autonomous vehicles and determine the optimal route to a destination.

[0984] This invention is a system for quickly determining whether transportation infrastructure is passable during disasters, and aims to provide real-time traffic information to the navigation systems of self-driving vehicles. This system is composed of unmanned aerial vehicles (drones), servers, generative artificial intelligence (AI), and self-driving vehicles.

[0985] First, the user sets up a flight plan for the unmanned aerial vehicle using a dedicated terminal. The flight plan targets the disaster-stricken area and specifies the route and photography points that the unmanned aerial vehicle will fly automatically. For example, it can be set to prioritize coverage of major highways and evacuation routes.

[0986] According to the set flight plan, the unmanned aerial vehicle automatically flies within a designated area and takes images of the ground and roads. The captured image data is sent from the unmanned aerial vehicle to a server in real time. The server stores the received image data in a specific format.

[0987] The saved image data is input into the generative AI and analyzed using deep learning technology. The generative AI automatically determines road conditions and changes in the terrain, and obtains passability information. For example, it determines the state of road damage and the presence of obstacles, and if it determines that a road is impassable, it analyzes the road as "impassable."

[0988] Next, the server generates a passable / unpassable map based on the analysis results obtained by the AI. On this map, passable roads are displayed in green, impassable roads in red, and roads requiring caution in yellow. The generated passable / unpassable map is provided in real time to the navigation system of the autonomous vehicle, which then uses this information to select the optimal route.

[0989] For example, if a major highway is deemed impassable due to a landslide, an autonomous vehicle will check the road availability map in real time and automatically select an alternative passable route. This information will also be presented to the driver and rescue teams, ensuring safe and efficient travel.

[0990] An example prompt might be: "Analyze traffic conditions of transportation infrastructure in real time during disasters and provide optimal route information for autonomous vehicles. Specifically, use a deep learning model to analyze road photos taken by a drone and detect landslides and road damage."

[0991] As a result, by using this system, it will be possible to quickly and accurately determine whether transportation infrastructure is passable during a disaster, enabling self-driving vehicles to operate safely and efficiently.

[0992] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0993] Step 1:

[0994] The user sets up a flight plan for the unmanned aerial vehicle using a dedicated terminal. The user refers to a map of the disaster area and specifies the route (input) that the unmanned aerial vehicle should take and the shooting points. This flight plan (output) is sent to the unmanned aerial vehicle.

[0995] Step 2:

[0996] The unmanned aerial vehicle flies automatically according to a set flight plan and periodically takes images of the ground and roads (specific operation). The image data taken by the unmanned aerial vehicle (input) is sent to a server in real time (output).

[0997] Step 3:

[0998] The server receives image data sent from the unmanned aerial vehicle (input) and stores it in a database in a specific format (output). The stored image data is then analyzed (specific operation).

[0999] Step 4:

[1000] The server inputs the stored image data into a generative AI model (generative AI model) (input) and analyzes it using deep learning technology (specific operation). The generative AI analyzes the image data and automatically determines the terrain and road conditions (data processing). The analysis results (output) include information on whether the road is passable.

[1001] Step 5:

[1002] The server generates a passable / non-passable map based on the analysis results obtained by the generation AI (specific operation). The input is the analysis results, and the output is a passable / non-passable map. On this map, passable roads are shown in green, impassable roads in red, and roads requiring caution in yellow.

[1003] Step 6:

[1004] The server provides the generated route availability map to the navigation system of the autonomous vehicle (output). The autonomous vehicle selects a safe and efficient route based on the route availability map (action) and starts moving to the destination (specific action).

[1005] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1006] ---

[1007] The present invention is a system that quickly determines whether traffic infrastructure is passable during a disaster, and further recognizes the user's emotions and provides support information.

[1008] System Overview

[1009] This system uses an unmanned aerial vehicle (drone) equipped with a camera and communication functions to capture images of the ground surface and send the image data to a server. The server stores the received image data and uses generative artificial intelligence (hereafter referred to as generative AI) to analyze the terrain and road conditions. Based on the analysis results, a passable / unpassable map is generated and provided to the user. The system also incorporates an emotion engine that recognizes the user's emotions and provides support information according to the user's emotional state.

[1010] Program processing

[1011] 1. Setting up a flight plan:

[1012] Using an application on a dedicated device, the user refers to a map of the disaster area and specifies the flight path and shooting points for the unmanned aerial vehicle.

[1013] 2. Automatic flight and photography of unmanned aerial vehicles:

[1014] The terminal transmits the set flight plan to the unmanned aerial vehicle, and the unmanned aerial vehicle automatically flies in the designated area according to the received flight plan.

[1015] The unmanned aerial vehicle periodically takes high-resolution photographs of the ground and roads along its flight path.

[1016] The captured photo data is sent to the server in real time.

[1017] 3. Receiving and storing data:

[1018] The server receives the image data sent from the unmanned aerial vehicle and stores it in a database.

[1019] 4. Image Analysis:

[1020] The server sequentially inputs the stored image data into the generating AI, which then analyzes the road conditions using deep learning technology.

[1021] For example, it can detect obstacles such as landslides in an image and determine that the location is "impassable."

[1022] 5. Generate a passability map:

[1023] The server determines passability information based on the analysis results and generates a passability map.

[1024] The road availability map shows passable roads in green, impassable roads in red, and roads requiring caution in yellow.

[1025] 6. Leveraging the Emotion Engine:

[1026] When providing a passability map, the server analyzes the user's facial expressions and voice using an emotion engine to recognize the user's emotions.

[1027] For example, if the system recognizes that a user is feeling anxious or stressed, it will provide not only a map of whether a route is passable, but also additional support information such as the location of evacuation shelters and information on the distribution of relief supplies.

[1028] 7. Information provision:

[1029] The server makes available to users road availability maps and additional assistance information via web services or dedicated applications.

[1030] Users can use a web browser or dedicated application to access real-time updated route availability maps and assistance information, and plan evacuation routes and travel plans.

[1031] Specific examples

[1032] For example, immediately after an earthquake occurs, an unmanned aerial vehicle flies over the affected area and takes photos of the roads. The image data is then sent to a server, where the AI ​​analyzes the road for damage and obstacles. If a landslide is confirmed at a point on a major highway, the location will be deemed "impassable" and displayed in red on a road access map.

[1033] At the same time, the emotion engine analyzes the facial expressions and voice of users viewing the map, and if it detects anxiety, it also provides information on the locations of evacuation shelters and relief supply distribution points. In this way, users can obtain appropriate routes and support information to evacuate quickly and safely.

[1034] As described above, the present invention is a system that quickly and accurately determines whether transportation infrastructure is passable during a disaster, and provides support information that takes the user's emotions into consideration, thereby effectively supporting the evacuation of disaster victims and rescue efforts.

[1035] The processing flow will be explained below.

[1036] ---

[1037] Step 1:

[1038] Users use an application on a dedicated device to refer to a map of the disaster-affected area and set the flight path and shooting points for the unmanned aerial vehicle (drone).

[1039] Step 2:

[1040] The terminal transmits the set flight plan to the unmanned aerial vehicle, and the unmanned aerial vehicle automatically flies in the designated area according to the received flight plan.

[1041] Step 3:

[1042] The drone periodically takes high-resolution photographs of the ground and roads along its flight path.

[1043] Step 4:

[1044] The unmanned aerial vehicle transmits the captured photo data to a server in real time.

[1045] Step 5:

[1046] The server receives the image data sent from the unmanned aerial vehicle and stores it in a database in a specific format.

[1047] Step 6:

[1048] The server sequentially inputs the stored image data into the generating AI, which then analyzes the road conditions using deep learning technology.

[1049] Step 7:

[1050] Based on the results of the analysis by the generation AI, the server determines the condition of the terrain and roads and determines whether each road is passable.

[1051] Step 8:

[1052] The server generates a passable road map based on the determined passable road information, showing passable roads in green, impassable roads in red, and roads requiring caution in yellow.

[1053] Step 9:

[1054] The server activates an emotion engine in the process of providing the user with a passable / unpassable map. The emotion engine analyzes the user's facial expressions and voice to recognize the user's emotions.

[1055] Step 10:

[1056] If the server determines that the user is feeling anxious or stressed based on the analysis results of the emotion engine, it will provide additional support information, such as a passable / unpassable road map, the location of evacuation shelters, and information on the distribution of relief supplies.

[1057] Step 11:

[1058] Using a web browser or dedicated application, users can access real-time updated route availability maps and assistance information to plan safe and appropriate evacuation routes and travel plans.

[1059] ---

[1060] The above are the specific processing steps for carrying out the present invention. These processing steps enable quick and accurate determination of whether an area is passable during a disaster, and provide appropriate support information according to the user's emotional state, thereby effectively supporting evacuation and rescue efforts.

[1061] Example 2

[1062] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1063] When a natural disaster occurs, it is difficult to quickly and accurately grasp information on whether the transportation infrastructure in the affected area is passable, which hinders the efficiency of evacuation and rescue efforts.In addition, support information that takes into account the emotional state of the victims is not provided, which increases the mental burden and hinders appropriate evacuation behavior.

[1064] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for capturing images of the ground surface using an unmanned aerial vehicle equipped with a camera and communication function, means for receiving and storing image data transmitted from the unmanned aerial vehicle, means for generating an AI generation system for analyzing the stored image data and determining the condition of the terrain and roads, means for generating a passability map based on the passability information determined by the AI ​​generation system, means for providing the generated passability map to the user, and means for analyzing the user's facial expressions and voice and providing support information based on the user's emotional state. This makes it possible to quickly and accurately provide passability information for disaster-stricken areas and to provide support information that takes the user's emotional state into consideration.

[1065] An "unmanned aerial vehicle equipped with a camera and communication capabilities" is a flying device with autonomous flight capabilities that is equipped with a camera and communication module for capturing image data and transmitting it wirelessly.

[1066] "Image data" refers to digital information such as photographs and videos taken by unmanned aerial vehicles of the conditions of the ground and roads.

[1067] "Generative AI" is a software system equipped with advanced machine learning technology that analyzes large amounts of image data and determines the condition of terrain and roads.

[1068] "Passability information" is digital information that indicates whether a road is passable or not, based on image data analyzed by the generating AI.

[1069] A "passability map" is a map for visually displaying passability information, and indicates passable roads, impassable roads, and roads requiring caution using different colors.

[1070] "Emotional state" refers to the emotional state of the user that is inferred based on facial expressions and voice analysis.

[1071] "Support information" is additional information that is useful in the event of a disaster, such as information on the location of evacuation shelters and distribution of relief supplies, provided according to the user's emotional state.

[1072] The present invention is a system that quickly determines whether traffic infrastructure is passable during a disaster, and further recognizes the user's emotions and provides support information. How the present invention is specifically implemented will be described below.

[1073] System Overview

[1074] The system of the present invention uses an unmanned aerial vehicle (drone) equipped with a camera and communication capabilities. The unmanned aerial vehicle takes high-resolution images of the ground and roads and transmits them to a server in real time. The server stores the received image data and analyzes the terrain and road conditions using a generative AI model. Based on the analysis results, a passability map is generated and provided to the user. The system also analyzes the user's facial expressions and voice to recognize their emotional state and provides support information based on the user's emotional state.

[1075] Hardware and software used

[1076] Unmanned aerial vehicle (drone): Equipped with a high-resolution camera and communication module.

[1077] Server: Equipped with a large database and high-performance processors to run generative AI models.

[1078] User device: A smartphone or tablet capable of running a web browser and dedicated applications.

[1079] Examples and prompts

[1080] For example, immediately after an earthquake, an unmanned aerial vehicle flies over the affected area and takes photographs of the condition of major roads and bridges. The captured image data is sent to a server in real time, where a generative AI model analyzes the road for damage and obstacles. For example, if a landslide is detected in the image, the location is determined to be "impassable" and displayed in red on a passability map.

[1081] Example prompt sentence:

[1082] Example prompt for analyzing the accessibility of transportation infrastructure:

[1083] Analyze road image data of disaster-stricken areas taken by unmanned aerial vehicles to determine whether roads are passable. Detect obstacles such as landslides and cracks in the roads, and create a passable map based on the analysis results.

[1084] Example prompt for user sentiment analysis:

[1085] Analyze the facial expressions and voice of users viewing the road availability map to detect their emotional state of anxiety or stress. Depending on their emotional state, provide them with additional support information, such as the location of evacuation shelters and information on the distribution of relief supplies.

[1086] When a user checks the passability map using a web browser or a dedicated application, the emotion engine analyzes the user's facial expressions and voice at the time and displays appropriate support information. For example, if the emotion engine detects the user's anxiety, it will add information about the location of evacuation shelters and relief supplies.

[1087] As described above, the present invention can quickly and accurately provide information on whether a disaster area is passable, and can provide support information that takes into account the user's emotional state, thereby effectively supporting evacuation and rescue operations.

[1088] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1089] Step 1: Set up a flight plan

[1090] User

[1091] The user launches the application on the dedicated device and displays a map of the disaster-affected area. Next, the user sets the flight path and shooting points for the drone on the map. For example, the user selects important areas such as major roads and bridges in the disaster area. Once the settings are complete, the user finalizes the flight plan and sends it from the device to the drone.

[1092] Input: Map of disaster area, flight route and photo points

[1093] Output: Flight plan data sent to unmanned aerial vehicles

[1094] Specific actions: Pointing and clicking on the map, pressing the flight plan confirmation button

[1095] Step 2: Automatic flight and photography of the unmanned aerial vehicle

[1096] unmanned aerial vehicle

[1097] The drone begins automatic flight based on the flight plan received from the device. It uses a GPS system to follow an accurate flight path, and when it reaches a designated shooting point, it takes photos of the ground and roads with a high-resolution camera. The captured image data is sent to a server in real time.

[1098] Input: Flight plan data

[1099] Output: The captured image data is sent to the server.

[1100] Specific operations: Obtaining location information using GPS, taking photos using a high-resolution camera, and transmitting data using a wireless communication module

[1101] Step 3: Receiving and storing data

[1102] server

[1103] The server receives image data sent from the unmanned aerial vehicle in real time and stores it in a database. The image data also contains metadata such as the date and time of the photo and GPS coordinates, so it is stored together with these.

[1104] Input: Captured image data (including metadata)

[1105] Output: Image data stored in a database

[1106] Specific operation: Receiving data via wireless communication and writing it to a database

[1107] Step 4: Image analysis

[1108] server

[1109] The server inputs the stored image data into a generative AI model and analyzes the images using deep learning technology. Specifically, it detects road damage and obstacles (e.g., landslides and fallen trees). Based on the results of this analysis, it determines whether the road is passable.

[1110] Input: Image data stored in a database

[1111] Output: Analyzed passability information (e.g. passable, not passable)

[1112] Specific operations: Data input to generative AI model, image analysis using deep learning

[1113] Step 5: Generate a passability map

[1114] server

[1115] The server generates a passable road map based on the results of image analysis, showing passable roads in green, impassable roads in red, and roads requiring caution in yellow.

[1116] Input: Parsed passability information

[1117] Output: Passability map

[1118] Specific operation: Visualization of passability information, color-coded display on the map

[1119] Step 6: Leverage your emotional engine

[1120] server

[1121] The server uses an emotion engine to analyze the user's facial expressions and voice as they browse the passability map. The emotion engine uses facial recognition software and tone analysis tools to determine the user's emotional state, and if it detects anxiety or stress, it provides additional support information.

[1122] Input: User's facial expression data, voice data

[1123] Output: Support information based on the user's emotional state

[1124] Specific operations: facial expression recognition and voice analysis, emotional state estimation, and provision of support information

[1125] Step 7: Provide information

[1126] server

[1127] The server provides users with road availability maps and additional assistance information via web services and dedicated applications. The information is updated in real time and can be used whenever needed.

[1128] User

[1129] Users can use a web browser or a dedicated application to check the passability map and support information, and plan evacuation routes and travel plans.

[1130] Input: Passability map, additional assistance information

[1131] Output: User viewing and use of information

[1132] Specific actions: Operating a web browser or application, planning an evacuation route

[1133] In this way, this system can quickly and accurately determine whether transportation infrastructure is passable during a disaster, and by providing support information that takes into account the user's emotional state, it can effectively support the evacuation of disaster victims and rescue efforts.

[1134] (Application example 2)

[1135] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1136] In the past, it was difficult to quickly and accurately determine whether transportation infrastructure was passable during disasters. Furthermore, when providing evacuation and support information, appropriate support was not provided that took into account the user's feelings. Therefore, there is a need for an information provision system that allows disaster victims to evacuate safely and receive appropriate support.

[1137] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1138] In this invention, the server includes means for capturing images of the ground surface using an unmanned aerial vehicle equipped with a camera and communication capabilities, means for receiving and storing image data transmitted from the unmanned aerial vehicle, artificial intelligence generation means for analyzing the stored image data and determining the condition of the terrain and roads, means for generating a passability map, means for providing the generated passability map, emotion recognition engine means for analyzing a user's facial expressions and voice and recognizing emotions, and means for providing support information based on the user's emotions. This makes it possible to quickly and accurately determine the passability of transportation infrastructure in the event of a disaster and to provide appropriate support information that takes the user's emotions into consideration.

[1139] A "camera" is a device that captures images or videos.

[1140] "Communication function" means the function of transmitting and receiving data wirelessly or via wire.

[1141] An "unmanned aerial vehicle" is an aircraft that flies remotely or autonomously to conduct observations and collect data.

[1142] "Image data" is digital information of still or moving images captured by a camera.

[1143] "Generative AI" refers to an AI technology that recognizes patterns based on large amounts of data and performs data analysis and judgments.

[1144] "Deep learning technology" is a technology in the field of artificial intelligence that uses multi-layer neural networks to analyze data with high precision.

[1145] A "passable roads map" displays information on a map in different colors, indicating which roads are passable, which roads are impassable, and which roads require caution.

[1146] An "emotion recognition engine" is a technology that analyzes a user's facial expressions and voice to identify their emotional state.

[1147] "Support information" refers to information for supporting users' evacuation and daily life in the event of a disaster.

[1148] This invention is a system that quickly determines whether traffic infrastructure is passable during a disaster, and further recognizes the user's emotions and provides support information. This system is configured as follows.

[1149] System Overview

[1150] This system uses an unmanned aerial vehicle equipped with a camera and communications capabilities to capture images of the ground surface and send the image data to a server. The server stores the received image data and uses generative artificial intelligence to analyze the terrain and road conditions. Based on the analysis results, a passable / unpassable map is generated and provided to the user. The system also incorporates an emotion engine that recognizes the user's emotions and provides support information according to the user's emotional state.

[1151] Hardware and software used

[1152] The system's main hardware consists of an unmanned aerial vehicle (drone) equipped with a camera and communication functions, a high-performance server, and a smartphone used by the user. The unmanned aerial vehicle is equipped with a high-resolution camera and communication functions that transmit image data to the server in real time.

[1153] The server uses Python, OpenCV, TensorFlow, Keras, and other deep learning libraries to analyze image data. The backend system uses Django, and the frontend uses React Native. OpenCV and a custom model are used for emotion recognition.

[1154] Data analysis and provision details

[1155] The server receives image data sent from the unmanned aerial vehicle and stores it in a database. The stored image data is then input into the artificial intelligence that generates it, which uses deep learning technology to analyze the terrain and road conditions. It detects obstacles such as landslides and damaged roads, classifies those areas as "impassable," and generates a passability map based on the results.

[1156] The generated road availability map shows passable roads in green, impassable roads in red, and roads requiring caution in yellow, and is updated in real time. Users can access the map and assistance information using a dedicated smartphone application.

[1157] The emotion recognition engine analyzes facial expressions and voice using the camera and microphone on the user's smartphone. For example, it uses OpenCV and a custom model to recognize emotions from the user's facial expressions and provides appropriate support information. For users who are feeling anxious or stressed, it provides information on evacuation shelters and relief supply distribution locations along with a map showing whether roads are passable.

[1158] Specific examples

[1159] For example, immediately after an earthquake occurs, an unmanned aerial vehicle flies over the affected area and takes photos of the roads. The image data is then sent to a server, where the AI ​​analyzes the road for damage and obstacles. If a landslide is confirmed at a point on a major highway, the location is deemed "impassable" and displayed in red on a passability map. At the same time, the emotion engine analyzes the facial expressions and voices of users viewing the passability map, and if it detects anxiety, it also provides information on the locations of evacuation centers and relief supply distribution points. In this way, users can obtain appropriate routes and support information to evacuate quickly and safely.

[1160] Prompt Sentence Examples

[1161] For post-disaster traffic analysis, high-resolution image data is sent from an unmanned aerial vehicle to a server and road conditions are determined using a deep learning model. We would like to build a system that analyzes the user's facial expressions and voice and provides appropriate support information based on their emotions. First, we train an AI model to identify obstacles such as landslides from image data, then develop an emotion recognition model and customize the support information based on that. Please suggest an appropriate deep learning library and model structure.

[1162] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1163] Step 1:

[1164] The user uses an application on a dedicated device to refer to a map of the disaster-stricken area and specify the flight path and shooting points of the unmanned aerial vehicle. The input is the map data specified by the user, and the output is a flight plan for the unmanned aerial vehicle. In concrete terms, the user operates the map on the application, placing pins to set shooting points.

[1165] Step 2:

[1166] The terminal transmits the set flight plan to the unmanned aerial vehicle, which then automatically flies the designated area according to the received flight plan. The input is flight plan data, and the output is flight instructions for the unmanned aerial vehicle. The unmanned aerial vehicle flies along the specified route and periodically takes photos of the ground and roads.

[1167] Step 3:

[1168] The unmanned aerial vehicle sends the photographic data it captures to a server in real time. The input is the captured image data, and the output is the image data sent to the server. Specifically, the images captured by the unmanned aerial vehicle's camera are uploaded to the server via the communication function.

[1169] Step 4:

[1170] The server receives image data sent from the unmanned aerial vehicle and stores it in a database. The input is the image data received from the unmanned aerial vehicle, and the output is the image data stored in the database. Specifically, the server converts the data into an appropriate format and stores it in storage.

[1171] Step 5:

[1172] The server sequentially inputs the stored image data into the generative AI model and uses deep learning technology to analyze the road conditions. The stored image data is used as input, and the analysis results are obtained as output. Specifically, the image is input into the model using TensorFlow and Keras, and obstacles are detected and the condition is determined.

[1173] Step 6:

[1174] The server generates a passable / non-passable map based on the analysis results. The analysis results are input, and the passable / non-passable map is generated as output. Specifically, it uses a map generation algorithm to color-code passable roads in green, impassable roads in red, and roads requiring caution in yellow.

[1175] Step 7:

[1176] The server makes the generated passability map available to users via a web service or dedicated application. The generated passability map is the input, and the passability map provided to the user is generated as the output. Specifically, the map information is displayed on a web page using Django and updated in real time.

[1177] Step 8:

[1178] Users use a smartphone application to access real-time updated route availability maps and support information to plan evacuation routes and travel plans. A route availability map is available as input, and the user's confirmed evacuation route and support information are output. Specifically, a React Native application is used to enable users to easily obtain information.

[1179] Step 9:

[1180] When providing a passability map, the server analyzes the user's facial expressions and voice using an emotion recognition engine to recognize the user's emotions. The input is the user's facial expression image and voice data, and the output is the user's emotional state. Specifically, the server analyzes the user's emotions using OpenCV and a custom emotion recognition model.

[1181] Step 10:

[1182] The server provides additional support information, such as the location of evacuation shelters and information on the distribution of relief supplies, depending on the user's emotional state. The user's emotional state is input, and appropriate support information is provided as output. Specifically, the server selects appropriate support information based on the emotion recognition results and notifies the user.

[1183] Through the above processing steps, the system can quickly and accurately determine whether transportation infrastructure is passable during a disaster, and can also provide support information that takes into account the user's emotions.

[1184] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1185] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1186] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1187] [Fourth embodiment]

[1188] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1189] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1190] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1191] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1192] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1193] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1194] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1195] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1196] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1197] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1198] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1199] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1200] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1201] ---

[1202] The present invention is a system for quickly determining whether traffic infrastructure is passable during a disaster, and is configured as follows.

[1203] This system first uses an unmanned aerial vehicle (drone) equipped with a camera and communication functions to capture images of the ground surface. The unmanned aerial vehicle automatically flies over a designated area according to a flight plan, taking high-resolution photos of the ground surface and roads at regular intervals. The captured photo data is sent to a server in real time.

[1204] The server receives image data sent from the unmanned aerial vehicle and stores it in a database in a specific format. The received image data is analyzed sequentially and input into the generative AI. The generative AI then analyzes the image data using deep learning technology to automatically determine the condition of the terrain and roads. This allows it to accurately detect the extent of road damage and the presence or absence of obstacles, and obtain information on whether the road is passable.

[1205] Based on the analysis results, the server generates a passable / unpassable map. This map displays passable roads in a predetermined first color (e.g., green), impassable roads in a predetermined second color (e.g., red), and roads requiring caution in a predetermined third color (e.g., yellow). The generated passable / unpassable map is immediately provided to the user via a web service or dedicated application.

[1206] Next, the program processing of this system will be explained in natural language.

[1207] 1. Setting up a flight plan:

[1208] Using an application on a dedicated device, the user refers to a map of the disaster area and specifies the flight path and shooting points for the unmanned aerial vehicle.

[1209] 2. Automatic flight and photography of unmanned aerial vehicles:

[1210] The terminal transmits the set flight plan to the unmanned aerial vehicle.

[1211] The unmanned aerial vehicle follows a specified flight plan, automatically flies over a designated area, and periodically takes photographs of the ground and roads.

[1212] The unmanned aerial vehicle transmits the captured photo data to a server in real time.

[1213] 3. Receiving and storing data:

[1214] The server receives image data sent from the unmanned aerial vehicle and stores it in a database in a specific format.

[1215] 4. Image Analysis:

[1216] The server sequentially inputs the stored image data into the generative artificial intelligence, which then analyzes the road conditions using deep learning technology.

[1217] For example, if a landslide is visible in some images, the generative AI will determine that road is "impassable."

[1218] 5. Generate a passability map:

[1219] Based on the analysis results, the server generates a passability map that visually displays passability information on a map.

[1220] The road availability map shows passable roads in green, impassable roads in red, and roads requiring caution in yellow.

[1221] 6. Information provision:

[1222] The server publishes the generated passability map to web services or dedicated applications and provides it to users.

[1223] Users can use a web browser or dedicated application to access real-time updated road safety maps and plan evacuation routes and movements.

[1224] As a specific example, immediately after a disaster occurs, an unmanned aerial vehicle flies over the affected area and takes photos of the roads. The image data is sent to a server, and the artificial intelligence analyzes the road for damage and obstacles. For example, if a landslide is confirmed at a certain point on a major highway, the road will be determined to be "impassable" and displayed in red on the passability map. If the road is confirmed to be safe at other points, those parts will be displayed in green. In this way, the latest passability information is provided in real time, helping users to travel quickly and safely.

[1225] As described above, the present invention is a system that quickly and accurately determines whether transportation infrastructure is passable during a disaster, and efficiently supports the evacuation of disaster victims and rescue operations.

[1226] The processing flow will be explained below.

[1227] ---

[1228] Step 1:

[1229] Users use an application on a dedicated device to refer to a map of the disaster-affected area and specify the flight path and shooting points for the unmanned aerial vehicle (drone).

[1230] Step 2:

[1231] The terminal transmits the set flight plan to the unmanned aerial vehicle, and the unmanned aerial vehicle automatically flies in the designated area according to the received flight plan.

[1232] Step 3:

[1233] The drone periodically takes high-resolution photographs of the ground and roads along its flight path.

[1234] Step 4:

[1235] The unmanned aerial vehicle transmits the captured photo data to a server in real time.

[1236] Step 5:

[1237] The server receives the image data sent from the unmanned aerial vehicle and stores it in a database in a specific format.

[1238] Step 6:

[1239] The server sequentially inputs the stored image data into the generative artificial intelligence.

[1240] Step 7:

[1241] The server uses generative artificial intelligence to analyze the image data, using deep learning technology to automatically determine the terrain and road conditions.

[1242] Step 8:

[1243] The server then determines whether a road is passable based on the analysis results. For example, it may classify a point in the image where a landslide has been confirmed as "impassable."

[1244] Step 9:

[1245] The server generates a passability map based on the passability information.

[1246] Step 10:

[1247] The server makes the passability map available to users via a web service or dedicated application.

[1248] Step 11:

[1249] Users can access the road closure map using a web browser or a dedicated application, and create evacuation routes and travel plans based on information updated in real time.

[1250] ---

[1251] These are the specific processing steps of the program for this system. This flow enables rapid and accurate understanding of the state of transportation infrastructure during a disaster, and supports effective evacuation and relief efforts.

[1252] Example 1

[1253] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1254] With conventional systems, it is difficult to quickly and accurately determine whether roads are passable during a disaster. Specifically, there is a lack of means to analyze road conditions in real time and provide that information to users immediately. As a result, evacuation plans and rescue operations cannot be carried out quickly in an emergency, and there is a risk that the damage will worsen.

[1255] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1256] In this invention, the server includes a means for a user to set a flight plan for the unmanned aerial vehicle via a dedicated terminal, a means for the unmanned aerial vehicle to automatically fly according to the set flight plan and transmit captured data to the server in real time, and a means for presenting a passability map at startup based on the passability information generated by the server. This makes it possible to quickly and accurately determine whether roads are passable during a disaster and provide the information to the user in real time.

[1257] An "unmanned aerial vehicle" is a device equipped with a camera and communication capabilities that flies automatically according to a specified flight plan.

[1258] "Image data" refers to data that includes photographic information of the ground and roads taken by unmanned aerial vehicles.

[1259] "Analysis" is the process of determining the terrain and road conditions based on image data stored on the server.

[1260] "Generative AI" is an AI system that uses image data to analyze terrain and road conditions using deep learning technology.

[1261] A "flight plan" is information about the flight path and photographing points of an unmanned aerial vehicle that is set by the user via a dedicated terminal.

[1262] A "dedicated terminal" is a device that a user uses to set a flight plan for an unmanned aerial vehicle, and includes smartphones, tablets, etc.

[1263] The "server" is a computer system that receives and stores image data transmitted from unmanned aerial vehicles, then analyzes the data and provides the generated passable / unpassable map.

[1264] A "passable roads map" is a map that visually displays passable roads, impassable roads, and roads requiring caution, using different colors based on the analysis results.

[1265] "Real-time" means that image data captured by an unmanned aerial vehicle is sent to a server almost immediately, with an extremely short time lag for analysis and provision of information.

[1266] "User" refers to a person who uses a dedicated terminal to set up a flight plan for an unmanned aerial vehicle and view a passability map.

[1267] To specifically implement this invention, an unmanned aerial vehicle (drone), a dedicated terminal (smartphone, tablet, etc.), and a server are required. The purpose of this system is to quickly determine whether transportation infrastructure is passable during a disaster and provide users with information in real time.

[1268] System Overview

[1269] Hardware Configuration

[1270] Unmanned aerial vehicle (drone): Equipped with a camera and communication capabilities, it flies automatically according to a specified flight plan.

[1271] Dedicated terminal: A device used by a user to set up a flight plan for an unmanned aerial vehicle, including a smartphone or tablet.

[1272] Server: A computer system that receives, stores, and analyzes image data sent from unmanned aerial vehicles.

[1273] Software Configuration

[1274] Dedicated terminal application: An application that allows users to set up flight plans.

[1275] Server-side analysis software: Software that uses deep learning technology to analyze image data.

[1276] Web service / dedicated application: Software that provides the generated accessibility map to the user.

[1277] Processing flow

[1278] Setting up a flight plan

[1279] The user launches the application on the dedicated device and displays a map of the affected area. The user then specifies the flight path and shooting points for the unmanned aircraft. For example, the user might "set a flight path from the center of Sendai City to the main road in the east."

[1280] Automated flight and data transmission

[1281] The terminal sends the set flight plan to the unmanned aerial vehicle. The unmanned aerial vehicle automatically flies over the specified area according to the specified flight plan, periodically taking photos of the ground and roads. The captured photo data is sent to the server in real time. Example: The unmanned aerial vehicle "takes photos every 10 minutes while flying along the specified route, and immediately sends the image data to the server after taking the photos."

[1282] Receiving and storing data

[1283] The server receives image data sent from the unmanned aerial vehicle in real time and stores it in a database in a specific format. Example: The server "receives image data in real time and stores the image data in a database with date and location information."

[1284] Image analysis and generation of passability maps

[1285] The server sequentially inputs the stored image data into generative artificial intelligence (AI), which uses deep learning technology to analyze the condition of the road. The AI ​​analyzes the image data and determines whether the road is damaged or has any obstacles. For example, if a landslide is confirmed in some of the images, the AI ​​will determine that the road is "impassable." The server generates a passable / unpassable map based on the analysis results. The passable / unpassable map shows passable roads in green, impassable roads in red, and roads requiring caution in yellow. Example: The server "generates a map that shows areas where landslides have been confirmed in red, and other safe areas in green."

[1286] Providing information

[1287] The server publishes the generated route availability map to a web service or dedicated application and provides it to users. Users can use a web browser or dedicated application to access the route availability map, which is updated in real time, and plan evacuation routes and movements. Example: A user can "check the traffic conditions in real time and plan evacuation routes and movements."

[1288] Prompt Sentence Examples

[1289] "After a disaster occurs in central Sendai, please use a drone to photograph the traffic conditions of major roads in the eastern part of the city, send the data to a server for analysis, and generate a map of whether roads are passable or not."

[1290] As described above, this system quickly and accurately determines whether transportation infrastructure is passable during a disaster, and provides users with useful information in real time.

[1291] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1292] Step 1: Set up a flight plan

[1293] User:

[1294] The user launches the application on the dedicated device and displays a map of the disaster area.

[1295] The user specifies the flight path and shooting points of the unmanned aerial vehicle on the map.

[1296] Input: Map information of the disaster area, flight path and photo points

[1297] Output: Flight plan data for unmanned aerial vehicles

[1298] Specific actions: The user "tap the tablet screen to mark the shooting point of the unmanned aerial vehicle," "confirm the flight plan, and press the start button."

[1299] Step 2: Submit your flight plan

[1300] Device:

[1301] The configured flight plan is transmitted to the unmanned aerial vehicle.

[1302] Input: Flight plan data

[1303] Output: Flight plan sent to the unmanned aerial vehicle

[1304] Specific operation: The device "wirelessly transmits flight plan data to the unmanned aerial vehicle" and "displays a notification confirming receipt."

[1305] Step 3: Automated flight of the unmanned aerial vehicle

[1306] Unmanned Aerial Vehicles:

[1307] The unmanned aerial vehicle will automatically begin flying based on the received flight plan.

[1308] Input: Flight plan

[1309] Output: Captured image data

[1310] Specific operation: The unmanned aircraft "turns on its engines and ascends to a specified altitude," then "flies automatically along a set route, repeatedly taking photographs and transmitting data."

[1311] Step 4: Sending data

[1312] Unmanned Aerial Vehicles:

[1313] The captured photo data is sent to the server in real time.

[1314] Input: Captured image data

[1315] Output: Image data sent to the server

[1316] Specific operation: The unmanned aircraft "sends image data to the server immediately after taking the image."

[1317] Step 5: Receiving the data

[1318] server:

[1319] Receive image data transmitted from the unmanned aerial vehicle.

[1320] Input: Submitted image data

[1321] Output: Received image data

[1322] Specific operation: The server receives image data in real time.

[1323] Step 6: Save your data

[1324] server:

[1325] The received image data is stored in a database in a specific format.

[1326] Input: Received image data

[1327] Output: Image data stored in a database

[1328] Specific operation: The server "saves the image data in a database with date and location information."

[1329] Step 7: Selecting the image to analyze

[1330] server:

[1331] A new image is selected from the stored image data.

[1332] Input: Image data stored in a database

[1333] Output: Image data selected for analysis

[1334] Specific operation: The server selects the latest image for analysis.

[1335] Step 8: AI analysis of images

[1336] server:

[1337] The selected image data is input into the generative artificial intelligence (AI).

[1338] The AI ​​uses deep learning technology to analyze terrain and road conditions.

[1339] Input: Image data to be analyzed

[1340] Output: Analysis results (passability information)

[1341] Specific operation: The generative AI "analyzes the presence or absence of road damage and obstacles in the image."

[1342] Step 9: Generate a passable map

[1343] server:

[1344] Based on the analysis results, a passability map is generated that visually displays passability information on a map.

[1345] Input: Analysis results (passability information)

[1346] Output: Passability map

[1347] Specific operation: The server generates a map that shows passable roads in green and impassable roads in red.

[1348] Step 10: Timely updates

[1349] server:

[1350] The generated passability map is published to a web service or dedicated application.

[1351] Input: Passability map

[1352] Output: A public accessibility map for the user

[1353] Specific operation: The server "uploads the accessibility map to a user-accessible web portal."

[1354] Step 11: Real-time access

[1355] User:

[1356] Access real-time updated road availability maps using a web browser or dedicated application.

[1357] Input: Web browser or dedicated application

[1358] Output: Real-time road availability map

[1359] Specific operation: The user "checks traffic conditions in real time and plans evacuation routes and travel plans."

[1360] Through the above steps, this system quickly and accurately determines whether transportation infrastructure is passable during a disaster and provides users with useful information in real time.

[1361] (Application example 1)

[1362] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1363] During disasters, it is extremely important to quickly and accurately determine whether transportation infrastructure is passable. In particular, traffic information updated in real time is essential for autonomous vehicles to safely transport evacuees. However, with current technology, it is difficult to determine the extent of road damage and the presence of obstacles in real time and immediately reflect this information in the navigation system of autonomous vehicles. For this reason, improving the efficiency and ensuring safety of autonomous vehicle operations during disasters is a challenge.

[1364] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1365] In this invention, the server includes means for capturing images of the ground using an unmanned aerial vehicle equipped with a camera and communication capabilities, means for receiving and storing image data transmitted from the unmanned aerial vehicle, means for analyzing the stored image data and determining the condition of the terrain and roads, means for generating a passability map based on the passability information determined by the generating artificial intelligence, and means for providing the generated passability map to the navigation system of the autonomous vehicle. This enables the passability of transportation infrastructure in the event of a disaster to be quickly and accurately determined, enabling the autonomous vehicle to operate safely and efficiently.

[1366] An "unmanned aerial vehicle" is a device equipped with a camera and communication capabilities, which flies remotely or automatically and takes images of the earth's surface.

[1367] "Image data" is digital data that includes photographic information of the ground and roads taken by unmanned aerial vehicles.

[1368] "Generative AI" is an AI system that uses deep learning technology to analyze stored image data and automatically determine the condition of terrain and roads.

[1369] "Passability information" is information indicating whether a road is passable or not and whether there are any obstacles, obtained from the results of analysis by the generating artificial intelligence.

[1370] A "passable road map" is a map generated based on passable road information, which indicates passable roads in a predetermined color and is provided to the navigation system of an autonomous vehicle.

[1371] An "autonomous vehicle" is a vehicle that is controlled by a navigation system and can drive autonomously.

[1372] A "navigation system" is a system that uses information such as road availability maps to control the operation of autonomous vehicles and determine the optimal route to a destination.

[1373] This invention is a system for quickly determining whether transportation infrastructure is passable during disasters, and aims to provide real-time traffic information to the navigation systems of self-driving vehicles. This system is composed of unmanned aerial vehicles (drones), servers, generative artificial intelligence (AI), and self-driving vehicles.

[1374] First, the user sets up a flight plan for the unmanned aerial vehicle using a dedicated terminal. The flight plan targets the disaster-stricken area and specifies the route and photography points that the unmanned aerial vehicle will fly automatically. For example, it can be set to prioritize coverage of major highways and evacuation routes.

[1375] According to the set flight plan, the unmanned aerial vehicle automatically flies within a designated area and takes images of the ground and roads. The captured image data is sent from the unmanned aerial vehicle to a server in real time. The server stores the received image data in a specific format.

[1376] The saved image data is input into the generative AI and analyzed using deep learning technology. The generative AI automatically determines road conditions and changes in the terrain, and obtains passability information. For example, it determines the state of road damage and the presence of obstacles, and if it determines that a road is impassable, it analyzes the road as "impassable."

[1377] Next, the server generates a passable / unpassable map based on the analysis results obtained by the AI. On this map, passable roads are displayed in green, impassable roads in red, and roads requiring caution in yellow. The generated passable / unpassable map is provided in real time to the navigation system of the autonomous vehicle, which then uses this information to select the optimal route.

[1378] For example, if a major highway is deemed impassable due to a landslide, an autonomous vehicle will check the road availability map in real time and automatically select an alternative passable route. This information will also be presented to the driver and rescue teams, ensuring safe and efficient travel.

[1379] An example prompt might be: "Analyze traffic conditions of transportation infrastructure in real time during disasters and provide optimal route information for autonomous vehicles. Specifically, use a deep learning model to analyze road photos taken by a drone and detect landslides and road damage."

[1380] As a result, by using this system, it will be possible to quickly and accurately determine whether transportation infrastructure is passable during a disaster, enabling self-driving vehicles to operate safely and efficiently.

[1381] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1382] Step 1:

[1383] The user sets up a flight plan for the unmanned aerial vehicle using a dedicated terminal. The user refers to a map of the disaster area and specifies the route (input) that the unmanned aerial vehicle should take and the shooting points. This flight plan (output) is sent to the unmanned aerial vehicle.

[1384] Step 2:

[1385] The unmanned aerial vehicle flies automatically according to a set flight plan and periodically takes images of the ground and roads (specific operation). The image data taken by the unmanned aerial vehicle (input) is sent to a server in real time (output).

[1386] Step 3:

[1387] The server receives image data sent from the unmanned aerial vehicle (input) and stores it in a database in a specific format (output). The stored image data is then analyzed (specific operation).

[1388] Step 4:

[1389] The server inputs the stored image data into a generative AI model (generative AI model) (input) and analyzes it using deep learning technology (specific operation). The generative AI analyzes the image data and automatically determines the terrain and road conditions (data processing). The analysis results (output) include information on whether the road is passable.

[1390] Step 5:

[1391] The server generates a passable / non-passable map based on the analysis results obtained by the generation AI (specific operation). The input is the analysis results, and the output is a passable / non-passable map. On this map, passable roads are shown in green, impassable roads in red, and roads requiring caution in yellow.

[1392] Step 6:

[1393] The server provides the generated route availability map to the navigation system of the autonomous vehicle (output). The autonomous vehicle selects a safe and efficient route based on the route availability map (action) and starts moving to the destination (specific action).

[1394] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1395] ---

[1396] The present invention is a system that quickly determines whether traffic infrastructure is passable during a disaster, and further recognizes the user's emotions and provides support information.

[1397] System Overview

[1398] This system uses an unmanned aerial vehicle (drone) equipped with a camera and communication functions to capture images of the ground surface and send the image data to a server. The server stores the received image data and uses generative artificial intelligence (hereafter referred to as generative AI) to analyze the terrain and road conditions. Based on the analysis results, a passable / unpassable map is generated and provided to the user. The system also incorporates an emotion engine that recognizes the user's emotions and provides support information according to the user's emotional state.

[1399] Program processing

[1400] 1. Setting up a flight plan:

[1401] Using an application on a dedicated device, the user refers to a map of the disaster area and specifies the flight path and shooting points for the unmanned aerial vehicle.

[1402] 2. Automatic flight and photography of unmanned aerial vehicles:

[1403] The terminal transmits the set flight plan to the unmanned aerial vehicle, and the unmanned aerial vehicle automatically flies in the designated area according to the received flight plan.

[1404] The unmanned aerial vehicle periodically takes high-resolution photographs of the ground and roads along its flight path.

[1405] The captured photo data is sent to the server in real time.

[1406] 3. Receiving and storing data:

[1407] The server receives the image data sent from the unmanned aerial vehicle and stores it in a database.

[1408] 4. Image Analysis:

[1409] The server sequentially inputs the stored image data into the generating AI, which then analyzes the road conditions using deep learning technology.

[1410] For example, it can detect obstacles such as landslides in an image and determine that the location is "impassable."

[1411] 5. Generate a passability map:

[1412] The server determines passability information based on the analysis results and generates a passability map.

[1413] The road availability map shows passable roads in green, impassable roads in red, and roads requiring caution in yellow.

[1414] 6. Leveraging the Emotion Engine:

[1415] When providing a passability map, the server analyzes the user's facial expressions and voice using an emotion engine to recognize the user's emotions.

[1416] For example, if the system recognizes that a user is feeling anxious or stressed, it will provide not only a map of whether a route is passable, but also additional support information such as the location of evacuation shelters and information on the distribution of relief supplies.

[1417] 7. Information provision:

[1418] The server makes available to users road availability maps and additional assistance information via web services or dedicated applications.

[1419] Users can use a web browser or dedicated application to access real-time updated route availability maps and assistance information, and plan evacuation routes and travel plans.

[1420] Specific examples

[1421] For example, immediately after an earthquake occurs, an unmanned aerial vehicle flies over the affected area and takes photos of the roads. The image data is then sent to a server, where the AI ​​analyzes the road for damage and obstacles. If a landslide is confirmed at a point on a major highway, the location will be deemed "impassable" and displayed in red on a road access map.

[1422] At the same time, the emotion engine analyzes the facial expressions and voice of users viewing the map, and if it detects anxiety, it also provides information on the locations of evacuation shelters and relief supply distribution points. In this way, users can obtain appropriate routes and support information to evacuate quickly and safely.

[1423] As described above, the present invention is a system that quickly and accurately determines whether transportation infrastructure is passable during a disaster, and provides support information that takes the user's emotions into consideration, thereby effectively supporting the evacuation of disaster victims and rescue efforts.

[1424] The processing flow will be explained below.

[1425] ---

[1426] Step 1:

[1427] Users use an application on a dedicated device to refer to a map of the disaster-affected area and set the flight path and shooting points for the unmanned aerial vehicle (drone).

[1428] Step 2:

[1429] The terminal transmits the set flight plan to the unmanned aerial vehicle, and the unmanned aerial vehicle automatically flies in the designated area according to the received flight plan.

[1430] Step 3:

[1431] The drone periodically takes high-resolution photographs of the ground and roads along its flight path.

[1432] Step 4:

[1433] The unmanned aerial vehicle transmits the captured photo data to a server in real time.

[1434] Step 5:

[1435] The server receives the image data sent from the unmanned aerial vehicle and stores it in a database in a specific format.

[1436] Step 6:

[1437] The server sequentially inputs the stored image data into the generating AI, which then analyzes the road conditions using deep learning technology.

[1438] Step 7:

[1439] Based on the results of the analysis by the generation AI, the server determines the condition of the terrain and roads and determines whether each road is passable.

[1440] Step 8:

[1441] The server generates a passable road map based on the determined passable road information, showing passable roads in green, impassable roads in red, and roads requiring caution in yellow.

[1442] Step 9:

[1443] The server activates an emotion engine in the process of providing the user with a passable / unpassable map. The emotion engine analyzes the user's facial expressions and voice to recognize the user's emotions.

[1444] Step 10:

[1445] If the server determines that the user is feeling anxious or stressed based on the analysis results of the emotion engine, it will provide additional support information, such as a passable / unpassable road map, the location of evacuation shelters, and information on the distribution of relief supplies.

[1446] Step 11:

[1447] Using a web browser or dedicated application, users can access real-time updated route availability maps and assistance information to plan safe and appropriate evacuation routes and travel plans.

[1448] ---

[1449] The above are the specific processing steps for carrying out the present invention. These processing steps enable quick and accurate determination of whether an area is passable during a disaster, and provide appropriate support information according to the user's emotional state, thereby effectively supporting evacuation and rescue efforts.

[1450] Example 2

[1451] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1452] When a natural disaster occurs, it is difficult to quickly and accurately grasp information on whether the transportation infrastructure in the affected area is passable, which hinders the efficiency of evacuation and rescue efforts.In addition, support information that takes into account the emotional state of the victims is not provided, which increases the mental burden and hinders appropriate evacuation behavior.

[1453] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for capturing images of the ground surface using an unmanned aerial vehicle equipped with a camera and communication function, means for receiving and storing image data transmitted from the unmanned aerial vehicle, means for generating an AI generation system for analyzing the stored image data and determining the condition of the terrain and roads, means for generating a passability map based on the passability information determined by the AI ​​generation system, means for providing the generated passability map to the user, and means for analyzing the user's facial expressions and voice and providing support information based on the user's emotional state. This makes it possible to quickly and accurately provide passability information for disaster-stricken areas and to provide support information that takes the user's emotional state into consideration.

[1454] An "unmanned aerial vehicle equipped with a camera and communication capabilities" is a flying device with autonomous flight capabilities that is equipped with a camera and communication module for capturing image data and transmitting it wirelessly.

[1455] "Image data" refers to digital information such as photographs and videos taken by unmanned aerial vehicles of the conditions of the ground and roads.

[1456] "Generative AI" is a software system equipped with advanced machine learning technology that analyzes large amounts of image data and determines the condition of terrain and roads.

[1457] "Passability information" is digital information that indicates whether a road is passable or not, based on image data analyzed by the generating AI.

[1458] A "passability map" is a map for visually displaying passability information, and indicates passable roads, impassable roads, and roads requiring caution using different colors.

[1459] "Emotional state" refers to the emotional state of the user that is inferred based on facial expressions and voice analysis.

[1460] "Support information" is additional information that is useful in the event of a disaster, such as information on the location of evacuation shelters and distribution of relief supplies, provided according to the user's emotional state.

[1461] The present invention is a system that quickly determines whether traffic infrastructure is passable during a disaster, and further recognizes the user's emotions and provides support information. How the present invention is specifically implemented will be described below.

[1462] System Overview

[1463] The system of the present invention uses an unmanned aerial vehicle (drone) equipped with a camera and communication capabilities. The unmanned aerial vehicle takes high-resolution images of the ground and roads and transmits them to a server in real time. The server stores the received image data and analyzes the terrain and road conditions using a generative AI model. Based on the analysis results, a passability map is generated and provided to the user. The system also analyzes the user's facial expressions and voice to recognize their emotional state and provides support information based on the user's emotional state.

[1464] Hardware and software used

[1465] Unmanned aerial vehicle (drone): Equipped with a high-resolution camera and communication module.

[1466] Server: Equipped with a large database and high-performance processors to run generative AI models.

[1467] User device: A smartphone or tablet capable of running a web browser and dedicated applications.

[1468] Examples and prompts

[1469] For example, immediately after an earthquake, an unmanned aerial vehicle flies over the affected area and takes photographs of the condition of major roads and bridges. The captured image data is sent to a server in real time, where a generative AI model analyzes the road for damage and obstacles. For example, if a landslide is detected in the image, the location is determined to be "impassable" and displayed in red on a passability map.

[1470] Example prompt sentence:

[1471] Example prompt for analyzing the accessibility of transportation infrastructure:

[1472] Analyze road image data of disaster-stricken areas taken by unmanned aerial vehicles to determine whether roads are passable. Detect obstacles such as landslides and cracks in the roads, and create a passable map based on the analysis results.

[1473] Example prompt for user sentiment analysis:

[1474] Analyze the facial expressions and voice of users viewing the road availability map to detect their emotional state of anxiety or stress. Depending on their emotional state, provide them with additional support information, such as the location of evacuation shelters and information on the distribution of relief supplies.

[1475] When a user checks the passability map using a web browser or a dedicated application, the emotion engine analyzes the user's facial expressions and voice at the time and displays appropriate support information. For example, if the emotion engine detects the user's anxiety, it will add information about the location of evacuation shelters and relief supplies.

[1476] As described above, the present invention can quickly and accurately provide information on whether a disaster area is passable, and can provide support information that takes into account the user's emotional state, thereby effectively supporting evacuation and rescue operations.

[1477] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1478] Step 1: Set up a flight plan

[1479] User

[1480] The user launches the application on the dedicated device and displays a map of the disaster-affected area. Next, the user sets the flight path and shooting points for the drone on the map. For example, the user selects important areas such as major roads and bridges in the disaster area. Once the settings are complete, the user finalizes the flight plan and sends it from the device to the drone.

[1481] Input: Map of disaster area, flight route and photo points

[1482] Output: Flight plan data sent to unmanned aerial vehicles

[1483] Specific actions: Pointing and clicking on the map, pressing the flight plan confirmation button

[1484] Step 2: Automatic flight and photography of the unmanned aerial vehicle

[1485] unmanned aerial vehicle

[1486] The drone begins automatic flight based on the flight plan received from the device. It uses a GPS system to follow an accurate flight path, and when it reaches a designated shooting point, it takes photos of the ground and roads with a high-resolution camera. The captured image data is sent to a server in real time.

[1487] Input: Flight plan data

[1488] Output: The captured image data is sent to the server.

[1489] Specific operations: Obtaining location information using GPS, taking photos using a high-resolution camera, and transmitting data using a wireless communication module

[1490] Step 3: Receiving and storing data

[1491] server

[1492] The server receives image data sent from the unmanned aerial vehicle in real time and stores it in a database. The image data also contains metadata such as the date and time of the photo and GPS coordinates, so it is stored together with these.

[1493] Input: Captured image data (including metadata)

[1494] Output: Image data stored in a database

[1495] Specific operation: Receiving data via wireless communication and writing it to a database

[1496] Step 4: Image analysis

[1497] server

[1498] The server inputs the stored image data into a generative AI model and analyzes the images using deep learning technology. Specifically, it detects road damage and obstacles (e.g., landslides and fallen trees). Based on the results of this analysis, it determines whether the road is passable.

[1499] Input: Image data stored in a database

[1500] Output: Analyzed passability information (e.g. passable, not passable)

[1501] Specific operations: Data input to generative AI model, image analysis using deep learning

[1502] Step 5: Generate a passability map

[1503] server

[1504] The server generates a passable road map based on the results of image analysis, showing passable roads in green, impassable roads in red, and roads requiring caution in yellow.

[1505] Input: Parsed passability information

[1506] Output: Passability map

[1507] Specific operation: Visualization of passability information, color-coded display on the map

[1508] Step 6: Leverage your emotional engine

[1509] server

[1510] The server uses an emotion engine to analyze the user's facial expressions and voice as they browse the passability map. The emotion engine uses facial recognition software and tone analysis tools to determine the user's emotional state, and if it detects anxiety or stress, it provides additional support information.

[1511] Input: User's facial expression data, voice data

[1512] Output: Support information based on the user's emotional state

[1513] Specific operations: facial expression recognition and voice analysis, emotional state estimation, and provision of support information

[1514] Step 7: Provide information

[1515] server

[1516] The server provides users with road availability maps and additional assistance information via web services and dedicated applications. The information is updated in real time and can be used whenever needed.

[1517] User

[1518] Users can use a web browser or a dedicated application to check the passability map and support information, and plan evacuation routes and travel plans.

[1519] Input: Passability map, additional assistance information

[1520] Output: User viewing and use of information

[1521] Specific actions: Operating a web browser or application, planning an evacuation route

[1522] In this way, this system can quickly and accurately determine whether transportation infrastructure is passable during a disaster, and by providing support information that takes into account the user's emotional state, it can effectively support the evacuation of disaster victims and rescue efforts.

[1523] (Application example 2)

[1524] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1525] In the past, it was difficult to quickly and accurately determine whether transportation infrastructure was passable during disasters. Furthermore, when providing evacuation and support information, appropriate support was not provided that took into account the user's feelings. Therefore, there is a need for an information provision system that allows disaster victims to evacuate safely and receive appropriate support.

[1526] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1527] In this invention, the server includes means for capturing images of the ground surface using an unmanned aerial vehicle equipped with a camera and communication capabilities, means for receiving and storing image data transmitted from the unmanned aerial vehicle, artificial intelligence generation means for analyzing the stored image data and determining the condition of the terrain and roads, means for generating a passability map, means for providing the generated passability map, emotion recognition engine means for analyzing a user's facial expressions and voice and recognizing emotions, and means for providing support information based on the user's emotions. This makes it possible to quickly and accurately determine the passability of transportation infrastructure in the event of a disaster and to provide appropriate support information that takes the user's emotions into consideration.

[1528] A "camera" is a device that captures images or videos.

[1529] "Communication function" means the function of transmitting and receiving data wirelessly or via wire.

[1530] An "unmanned aerial vehicle" is an aircraft that flies remotely or autonomously to conduct observations and collect data.

[1531] "Image data" is digital information of still or moving images captured by a camera.

[1532] "Generative AI" refers to an AI technology that recognizes patterns based on large amounts of data and performs data analysis and judgments.

[1533] "Deep learning technology" is a technology in the field of artificial intelligence that uses multi-layer neural networks to analyze data with high precision.

[1534] A "passable roads map" displays information on a map in different colors, indicating which roads are passable, which roads are impassable, and which roads require caution.

[1535] An "emotion recognition engine" is a technology that analyzes a user's facial expressions and voice to identify their emotional state.

[1536] "Support information" refers to information for supporting users' evacuation and daily life in the event of a disaster.

[1537] This invention is a system that quickly determines whether traffic infrastructure is passable during a disaster, and further recognizes the user's emotions and provides support information. This system is configured as follows.

[1538] System Overview

[1539] This system uses an unmanned aerial vehicle equipped with a camera and communications capabilities to capture images of the ground surface and send the image data to a server. The server stores the received image data and uses generative artificial intelligence to analyze the terrain and road conditions. Based on the analysis results, a passable / unpassable map is generated and provided to the user. The system also incorporates an emotion engine that recognizes the user's emotions and provides support information according to the user's emotional state.

[1540] Hardware and software used

[1541] The system's main hardware consists of an unmanned aerial vehicle (drone) equipped with a camera and communication functions, a high-performance server, and a smartphone used by the user. The unmanned aerial vehicle is equipped with a high-resolution camera and communication functions that transmit image data to the server in real time.

[1542] The server uses Python, OpenCV, TensorFlow, Keras, and other deep learning libraries to analyze image data. The backend system uses Django, and the frontend uses React Native. OpenCV and a custom model are used for emotion recognition.

[1543] Data analysis and provision details

[1544] The server receives image data sent from the unmanned aerial vehicle and stores it in a database. The stored image data is then input into the artificial intelligence that generates it, which uses deep learning technology to analyze the terrain and road conditions. It detects obstacles such as landslides and damaged roads, classifies those areas as "impassable," and generates a passability map based on the results.

[1545] The generated road availability map shows passable roads in green, impassable roads in red, and roads requiring caution in yellow, and is updated in real time. Users can access the map and assistance information using a dedicated smartphone application.

[1546] The emotion recognition engine analyzes facial expressions and voice using the camera and microphone on the user's smartphone. For example, it uses OpenCV and a custom model to recognize emotions from the user's facial expressions and provides appropriate support information. For users who are feeling anxious or stressed, it provides information on evacuation shelters and relief supply distribution locations along with a map showing whether roads are passable.

[1547] Specific examples

[1548] For example, immediately after an earthquake occurs, an unmanned aerial vehicle flies over the affected area and takes photos of the roads. The image data is then sent to a server, where the AI ​​analyzes the road for damage and obstacles. If a landslide is confirmed at a point on a major highway, the location is deemed "impassable" and displayed in red on a passability map. At the same time, the emotion engine analyzes the facial expressions and voices of users viewing the passability map, and if it detects anxiety, it also provides information on the locations of evacuation centers and relief supply distribution points. In this way, users can obtain appropriate routes and support information to evacuate quickly and safely.

[1549] Prompt Sentence Examples

[1550] For post-disaster traffic analysis, high-resolution image data is sent from an unmanned aerial vehicle to a server and road conditions are determined using a deep learning model. We would like to build a system that analyzes the user's facial expressions and voice and provides appropriate support information based on their emotions. First, we train an AI model to identify obstacles such as landslides from image data, then develop an emotion recognition model and customize the support information based on that. Please suggest an appropriate deep learning library and model structure.

[1551] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1552] Step 1:

[1553] The user uses an application on a dedicated device to refer to a map of the disaster-stricken area and specify the flight path and shooting points of the unmanned aerial vehicle. The input is the map data specified by the user, and the output is a flight plan for the unmanned aerial vehicle. In concrete terms, the user operates the map on the application, placing pins to set shooting points.

[1554] Step 2:

[1555] The terminal transmits the set flight plan to the unmanned aerial vehicle, which then automatically flies the designated area according to the received flight plan. The input is flight plan data, and the output is flight instructions for the unmanned aerial vehicle. The unmanned aerial vehicle flies along the specified route and periodically takes photos of the ground and roads.

[1556] Step 3:

[1557] The unmanned aerial vehicle sends the photographic data it captures to a server in real time. The input is the captured image data, and the output is the image data sent to the server. Specifically, the images captured by the unmanned aerial vehicle's camera are uploaded to the server via the communication function.

[1558] Step 4:

[1559] The server receives image data sent from the unmanned aerial vehicle and stores it in a database. The input is the image data received from the unmanned aerial vehicle, and the output is the image data stored in the database. Specifically, the server converts the data into an appropriate format and stores it in storage.

[1560] Step 5:

[1561] The server sequentially inputs the stored image data into the generative AI model and uses deep learning technology to analyze the road conditions. The stored image data is used as input, and the analysis results are obtained as output. Specifically, the image is input into the model using TensorFlow and Keras, and obstacles are detected and the condition is determined.

[1562] Step 6:

[1563] The server generates a passable / non-passable map based on the analysis results. The analysis results are input, and the passable / non-passable map is generated as output. Specifically, it uses a map generation algorithm to color-code passable roads in green, impassable roads in red, and roads requiring caution in yellow.

[1564] Step 7:

[1565] The server makes the generated passability map available to users via a web service or dedicated application. The generated passability map is the input, and the passability map provided to the user is generated as the output. Specifically, the map information is displayed on a web page using Django and updated in real time.

[1566] Step 8:

[1567] Users use a smartphone application to access real-time updated route availability maps and support information to plan evacuation routes and travel plans. A route availability map is available as input, and the user's confirmed evacuation route and support information are output. Specifically, a React Native application is used to enable users to easily obtain information.

[1568] Step 9:

[1569] When providing a passability map, the server analyzes the user's facial expressions and voice using an emotion recognition engine to recognize the user's emotions. The input is the user's facial expression image and voice data, and the output is the user's emotional state. Specifically, the server analyzes the user's emotions using OpenCV and a custom emotion recognition model.

[1570] Step 10:

[1571] The server provides additional support information, such as the location of evacuation shelters and information on the distribution of relief supplies, depending on the user's emotional state. The user's emotional state is input, and appropriate support information is provided as output. Specifically, the server selects appropriate support information based on the emotion recognition results and notifies the user.

[1572] Through the above processing steps, the system can quickly and accurately determine whether transportation infrastructure is passable during a disaster, and can also provide support information that takes into account the user's emotions.

[1573] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1574] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1575] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1576] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1577] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1578] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1579] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1580] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1581] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1582] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1583] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1584] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1585] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1586] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1587] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1588] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1589] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1590] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1591] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1592] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1593] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1594] The following is further disclosed regarding the above embodiment.

[1595] (Claim 1)

[1596] A means for capturing images of the ground surface using an unmanned aerial vehicle equipped with a camera and a communication function;

[1597] means for receiving and storing image data transmitted from the unmanned aerial vehicle;

[1598] A generating artificial intelligence means for analyzing the stored image data and determining the condition of the terrain and roads;

[1599] A means for generating a passability map based on the passability information determined by the generating artificial intelligence;

[1600] The system includes a means for providing the generated passability map.

[1601] (Claim 2)

[1602] The system of claim 1, wherein the generating artificial intelligence uses deep learning technology to determine the state of road damage and the presence or absence of obstacles from image data.

[1603] (Claim 3)

[1604] 2. The system according to claim 1, wherein the passable / unpassable map displays passable roads in a predetermined first color, impassable roads in a predetermined second color, and roads requiring caution in a predetermined third color.

[1605] "Example 1"

[1606] (Claim 1)

[1607] A means for capturing images of the ground surface using an unmanned aerial vehicle equipped with a camera and a communication function;

[1608] means for receiving and storing image data transmitted from the unmanned aerial vehicle;

[1609] A generating artificial intelligence means for analyzing the stored image data and determining the condition of the terrain and roads;

[1610] A means for a user to set a flight plan for the unmanned aerial vehicle via a dedicated terminal;

[1611] A means for automatically flying the unmanned aerial vehicle according to a set flight plan and transmitting captured data to a server in real time;

[1612] means for displaying a passability map based on passability information generated by the server at the time of startup;

[1613] a means for providing the generated passability map;

[1614] A system including:

[1615] (Claim 2)

[1616] The system of claim 1, wherein the generating artificial intelligence uses deep learning technology to determine the state of road damage and the presence or absence of obstacles from image data.

[1617] (Claim 3)

[1618] 2. The system according to claim 1, wherein the passable / unpassable map displays passable roads in a predetermined first color, impassable roads in a predetermined second color, and roads requiring caution in a predetermined third color.

[1619] "Application Example 1"

[1620] (Claim 1)

[1621] A means for capturing images of the ground surface using an unmanned aerial vehicle equipped with a camera and a communication function;

[1622] means for receiving and storing image data transmitted from the unmanned aerial vehicle;

[1623] A generating artificial intelligence means for analyzing the stored image data and determining the condition of the terrain and roads;

[1624] A means for generating a passability map based on the passability information determined by the generating artificial intelligence;

[1625] The system includes means for providing the generated passability map to a navigation system of an autonomous vehicle.

[1626] (Claim 2)

[1627] The system of claim 1, wherein the generating artificial intelligence uses deep learning technology to determine the state of road damage and the presence or absence of obstacles from image data.

[1628] (Claim 3)

[1629] 2. The system according to claim 1, wherein the passability map displays passable roads in a predetermined first color, impassable roads in a predetermined second color, and roads requiring caution in a predetermined third color, and the displayed passability map is provided to a display of the autonomous vehicle in real time.

[1630] "Example 2: Combining Emotion Engines"

[1631] (Claim 1)

[1632] A means for capturing images of the ground surface using an unmanned aerial vehicle equipped with a camera and a communication function;

[1633] means for receiving and storing image data transmitted from the unmanned aerial vehicle;

[1634] A generating artificial intelligence means for analyzing the stored image data and determining the condition of the terrain and roads;

[1635] A means for generating a passability map based on the passability information determined by the generating artificial intelligence;

[1636] a means for providing the generated passability map to a user;

[1637] A system that includes a means for analyzing a user's facial expressions and voice and providing support information based on their emotional state.

[1638] (Claim 2)

[1639] The system of claim 1, wherein the generating artificial intelligence uses deep learning technology to determine the state of road damage and the presence or absence of obstacles from image data.

[1640] (Claim 3)

[1641] 2. The system according to claim 1, wherein the passable / unpassable map displays passable roads in a predetermined first color, impassable roads in a predetermined second color, and roads requiring caution in a predetermined third color.

[1642] "Application example 2 when combining emotion engines"

[1643] (Claim 1)

[1644] A means for capturing images of the ground surface using an unmanned aerial vehicle equipped with a camera and a communication function;

[1645] means for receiving and storing image data transmitted from the unmanned aerial vehicle;

[1646] A generating artificial intelligence means for analyzing the stored image data and determining the condition of the terrain and roads;

[1647] A means for generating a passability map based on the passability information determined by the generating artificial intelligence;

[1648] a means for providing the generated passability map;

[1649] an emotion recognition engine means for analyzing a user's facial expression and voice and recognizing emotions;

[1650] A method for providing support information based on user's emotions

[1651] A system including:

[1652] (Claim 2)

[1653] The system of claim 1, wherein the generating artificial intelligence uses deep learning technology to determine the state of road damage and the presence or absence of obstacles from image data.

[1654] (Claim 3)

[1655] 2. The system according to claim 1, wherein the passable / unpassable map displays passable roads in a predetermined first color, impassable roads in a predetermined second color, and roads requiring caution in a predetermined third color. [Explanation of symbols]

[1656] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for capturing images of the ground surface using an unmanned aerial vehicle equipped with a camera and a communication function; means for receiving and storing image data transmitted from the unmanned aerial vehicle; A generating artificial intelligence means for analyzing the stored image data and determining the condition of the terrain and roads; A means for generating a passability map based on the passability information determined by the generating artificial intelligence; The system includes a means for providing the generated passability map.

2. The system according to claim 1, wherein the generating artificial intelligence uses deep learning technology to determine the state of road damage and the presence or absence of obstacles from image data.

3. 2. The system according to claim 1, wherein the passable roads map displays passable roads in a predetermined first color, impassable roads in a predetermined second color, and roads requiring caution in a predetermined third color.

Images