System

Abstract

A system is provided.SOLUTION: A system includes means for receiving flight information transmitted from a drone terminal and accumulating the flight information in a platform, means for training an air route generation AI based on the accumulated flight information, means for generating an optimal air route by the air route generation AI according to a delivery request and transmitting the air route to the drone terminal, means for monitoring the flight information in real time during flight of the drone terminal and adjusting the air route as necessary, means for generating and updating an air map based on the flight information and environmental data from the drone terminal, and means for notifying a user of a delivery state.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, in delivery systems using drones, it is difficult to generate efficient and safe flight routes and manage operations. This reduces the operational efficiency of drone delivery and makes it difficult to ensure delivery safety. In particular, there is a lack of means to appropriately avoid risks such as changes in weather, obstacles on the air route, and bird strikes, making drone accidents and delivery delays more likely to occur. Furthermore, the lack of real-time monitoring of drone status and updates to air route information makes it difficult to select the optimal delivery route. [Means for solving the problem]

[0005] The present invention includes a means for receiving flight information sent from a drone terminal and storing it on a platform. It also includes a means for training an air route generation AI based on the stored flight information, generating an optimal air route using the air route generation AI in response to a delivery request, and transmitting the optimal air route to the drone terminal. It also includes a means for monitoring the flight information in real time while the drone terminal is flying and adjusting the air route as necessary. The system provides a means for generating and updating an aerial map based on flight information and environmental data from the drone terminal, and includes a means for notifying users of delivery status, thereby providing an efficient and safe drone delivery system.

[0006] A "drone terminal" is an unmanned aerial vehicle capable of remote control or autonomous flight and capable of carrying designated cargo.

[0007] "Flight information" refers to data such as location, altitude, speed, and battery status acquired by the drone terminal during flight.

[0008] The "platform" is a centralized system for storing and analyzing data received from drone terminals.

[0009] "Air Route Generation AI" refers to an artificial intelligence algorithm for generating optimal flight routes based on accumulated flight information and related data.

[0010] "Delivery request" refers to a request made by a user to the system to transport a package to a specified location.

[0011] "Air route" refers to the route that the drone terminal will fly for delivery.

[0012] "Real-time monitoring" refers to the process of constantly monitoring data while the drone is flying and responding and adapting immediately.

[0013] "Environmental data" refers to data including weather information, obstacle information, and other external conditions that affect drone flight.

[0014] "Aeronautical map" refers to a map that shows geographical information and flight paths for the safe and effective flight of drones.

[0015] "User notification" refers to a communication method for providing delivery status and important information to users. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention relates to a system for realizing efficient and safe drone delivery, and detailed embodiments thereof will be described below.

[0038] The server has the function of receiving flight information sent from the drone terminal. During flight, the drone terminal sends flight information such as location, altitude, speed, and battery status to the server in real time. The server accumulates and stores this information on its own platform. This data is used to train the air route generation AI (described later) and to generate air routes in real time.

[0039] The server then uses the accumulated flight information to train the route generation AI, which learns relevant data such as past flight patterns, weather information, date and time, and past bird strikes to improve its ability to generate optimal flight paths, thereby providing more efficient and safer routes for future delivery requests.

[0040] When a user requests a drone delivery, the server accepts the request and generates the optimal route using route generation AI. The route generation AI considers parameters such as current weather, the specified delivery destination, and delivery time to select the most efficient route from multiple flight paths. The generated route information is sent to the drone terminal in real time.

[0041] During flight, the drone terminals periodically send flight information to the server. The server uses this information to monitor drone operations in real time and adjust air routes as necessary. The server also integrates the flight information and environmental data received from the drone terminals to generate and update a highly accurate aerial map. For example, newly discovered obstacles or sudden weather changes can be immediately reflected in the aerial map, ensuring that all drone terminals are flying based on the most up-to-date information.

[0042] In addition, the server provides users with real-time delivery status notifications, allowing them to receive information about the location of their delivery, estimated arrival time, and other important notifications via a smartphone app.

[0043] As a concrete example, when a user requests "drone delivery from their current address to their workplace" through a smartphone app, the request is sent to a server. The server uses route generation AI to generate the optimal flight route from their current address to their workplace and sends it to the drone terminal. The drone terminal then flies along this route and proceeds with the delivery. If an obstacle is detected during flight, the drone terminal sends that information to the server, which then updates the aerial map. The user can check the drone's current location and estimated arrival time in real time through the smartphone app.

[0044] In this way, the present invention enables efficient and safe drone delivery, while also improving user convenience.

[0045] The processing flow will be explained below.

[0046] Step 1:

[0047] Drone startup and initial setup

[0048] The device completes the initial setup and connects to the system, sending its identification information, initial position, altitude, and battery status to the server.

[0049] Step 2:

[0050] Real-time flight information transmission

[0051] During flight, the device periodically transmits flight information such as location, altitude, speed, and battery status to the server in real time.

[0052] Step 3:

[0053] Data accumulation

[0054] The server automatically stores the received flight information on its own platform, which stores the data necessary for subsequent analysis and route generation.

[0055] Step 4:

[0056] Air route generation AI training

[0057] The server trains the route generation AI based on accumulated flight information and data such as date and time, weather, and past bird strikes, allowing the route generation AI to learn and improve its algorithm for generating efficient and safe routes.

[0058] Step 5:

[0059] Accepting a delivery request

[0060] The user enters a delivery request through a smartphone app and sends it to the server.

[0061] The server analyzes the request content (delivery destination, desired time, etc.).

[0062] Step 6:

[0063] Air Route Generation

[0064] The server uses route generation AI to generate the optimal route based on the request.

[0065] The server takes the generated air route information and takes into account relevant constraints (weather, obstacles, etc.).

[0066] Step 7:

[0067] Sending air route information

[0068] The server transmits the generated air route information to the terminal (drone).

[0069] The terminal loads the received air route information into memory and prepares for flight.

[0070] Step 8:

[0071] Start of flight

[0072] The terminal starts the flight based on the received air route information.

[0073] The terminal periodically transmits flight information to the server.

[0074] Step 9:

[0075] Real-time monitoring and adjustment

[0076] The server monitors real-time data from the terminals during the flight.

[0077] The server analyzes flight information and environmental data and adjusts air routes as needed.

[0078] Step 10:

[0079] Generate and update aerial maps

[0080] The server generates and updates an aerial map based on flight information and environmental data from the drone.

[0081] Newly discovered obstacles and weather changes are instantly updated on the aerial map.

[0082] Step 11:

[0083] User Notifications

[0084] The server notifies the user of the delivery status in real time.

[0085] Users can use a smartphone app to check the current location and estimated arrival time of their delivery.

[0086] Step 12:

[0087] Delivery completed

[0088] The terminal arrives at the delivery destination and delivers the package safely.

[0089] The terminal notifies the server that the delivery is complete.

[0090] The server notifies the user that delivery has been completed.

[0091] Example 1

[0092] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0093] Drone delivery presents many challenges, including optimizing flight routes for efficiency, ensuring safety during flight, and understanding delivery status in real time. In particular, drone operation can be disrupted if the flight route is inappropriate or if unexpected obstacles appear during flight. Furthermore, when multiple drones are operating simultaneously, it is important to ensure they do not interfere with each other. Another problem is the lack of means to take rapid emergency measures when trouble occurs.

[0094] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0095] In this invention, the server includes means for receiving flight information transmitted from the drone terminal and storing it on the platform, means for training an air route generation AI based on the stored flight information, means for generating an optimal air route using the air route generation AI in response to a delivery request and transmitting it to the drone terminal, means for monitoring flight information in real time while the drone terminal is flying and adjusting the air route as necessary, means for generating and updating an aerial map based on flight information and environmental data from the drone terminal, means for notifying the user of the delivery status, means for taking emergency measures in the event of a problem based on flight information collected by the drone terminal, and means for multiple drone terminals to cooperate and share flight information to improve overall flight efficiency. This enables efficient and safe drone delivery and improves user convenience.

[0096] "Flight information" is a general term for a wide variety of data related to operation, such as location information, altitude, speed, and battery status obtained from drone terminals.

[0097] "Platform" refers to an integrated environment of servers and related software for receiving, storing, and managing flight information sent from drone terminals.

[0098] "Air Route Generation AI" is an AI (artificial intelligence) model that generates optimal flight routes based on accumulated flight information, weather data, past flight patterns, etc.

[0099] "Delivery Request" means a request for drone delivery made by a User through a smartphone app or other interface.

[0100] The "optimal route" refers to the flight route with the highest flight efficiency, selected based on the conditions specified by the route generation AI (delivery destination, weather, time of day, etc.).

[0101] "Monitoring" refers to the process of monitoring flight information transmitted by drone terminals in real time and adjusting operational status as necessary.

[0102] "Environmental data" is a general term for data related to the flight environment, such as weather information and the presence of obstacles, acquired by a drone terminal during flight.

[0103] "Aerial Map" refers to a high-precision aerial map generated by integrating flight information and environmental data.

[0104] "Emergency Plan" means the procedures and protocols for responding quickly if a problem occurs during flight.

[0105] "Improving flight efficiency" refers to the process of multiple drone devices working together and sharing flight information to minimize overall flight time and energy consumption.

[0106] The present invention relates to a system for realizing efficient and safe drone delivery, and detailed embodiments thereof will be described below.

[0107] Hardware and software used

[0108] 1. Server

[0109] Server: High-performance server (e.g. Dell PowerEdge R740)

[0110] Server software: Python, Django, MySQL (registered trademark)

[0111] Real-time data processing: Apache Kafka

[0112] 2. Drone terminal

[0113] Drone hardware: A typical high-performance drone (e.g., DJI Mavic 2)

[0114] Communication module: Standard internet communication module

[0115] Drone control application: Drone control SDK (e.g. DJI SDK)

[0116] 3. Air route generation AI

[0117] Deep learning framework: TENSORFLOW (registered trademark), PyTorch

[0118] 4. GIS Software

[0119] Aerial Map Generation Tool: Esri ArcGIS

[0120] Program processing

[0121] The server has the function of receiving flight information sent from the drone terminal. During flight, the drone terminal sends flight information such as location, altitude, speed, and battery status to the server in real time. The server stores this received information in a database. This database is operated using MySQL.

[0122] The server trains the route generation AI based on the accumulated flight information. The route generation AI builds an optimization model using Python, TensorFlow, and PyTorch to learn relevant data such as past flight patterns, weather information, date and time, and past bird strikes. This allows it to generate efficient and safe flight routes for future delivery requests.

[0123] When a user requests a drone delivery, the server accepts the request and generates the optimal air route using an air route generation AI. The request includes parameters such as the specified delivery destination and delivery time, and the server selects the optimal route taking this information into account. The generated air route information is then sent to the drone terminal.

[0124] During flight, the drones periodically send flight information to the server. The server monitors this information and adjusts flight routes as necessary. The server also integrates the flight information and environmental data received from the drones to generate and update a highly accurate aerial map. For example, newly discovered obstacles or sudden weather changes can be instantly reflected in the aerial map, ensuring that all drones fly based on the most up-to-date information.

[0125] Additionally, the server provides users with real-time delivery status notifications, allowing them to receive information about the location of their shipments, estimated arrival times, and other important notifications via a smartphone app.

[0126] Specific examples

[0127] When a user requests "drone delivery from their current address to their workplace" via a smartphone app, the request is sent to the server.

[0128] Example prompt sentence:

[0129] I would like to request drone delivery from my current address to my workplace. My current address is in Shibuya-ku, Tokyo, and my workplace is in Minato-ku, Tokyo. I would like the delivery time to be between 10:00 AM and 2:00 PM.

[0130] The server uses route generation AI to generate the optimal flight route from the current address to the workplace and sends it to the drone terminal. The drone terminal then flies along this route and proceeds with the delivery. If an obstacle is detected during flight, the drone terminal sends that information to the server, which then updates the aerial map. The user can check the drone's current location and estimated arrival time in real time via a smartphone app.

[0131] In this way, the present invention enables efficient and safe drone delivery, improving user convenience.

[0132] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0133] Step 1: Gather flight information

[0134] Subject: Device

[0135] Input: Location, altitude, speed, and battery status obtained from the drone's built-in sensors and GPS module.

[0136] How it works: The drone terminal reads the data obtained from these sensors and the GPS module at regular intervals.

[0137] Output: Retrieved flight information data.

[0138] Specific operation: The drone terminal obtains the current latitude and longitude, altitude (meters), speed (kilometers per hour), and battery status (percentage) and generates flight information data.

[0139] Step 2: Send flight information

[0140] Subject: Device

[0141] Input: Collected flight information.

[0142] Operation: The terminal sends the collected flight information to the server via the Internet communication module.

[0143] Output: Flight information sent to the server.

[0144] Specific operation: The drone terminal sends the latest flight information (location, altitude, speed, battery status) to the server as an HTTP request at regular intervals.

[0145] Step 3: Receiving and storing flight information

[0146] Subject: Server

[0147] Input: Flight information sent from the device.

[0148] Operation: The server stores the received flight information in a database.

[0149] Output: Flight information stored in a database.

[0150] What it does: The server receives the HTTP request, parses it, and stores flight information data in a MySQL database, including latitude, longitude, altitude, speed, battery status, and timestamps.

[0151] Step 4: Training the route generation AI

[0152] Subject: Server

[0153] Input: Past flight information, weather information, date and time, and past bird strike information stored in the database.

[0154] How it works: The server uses these data to train the route generation AI.

[0155] Output: A trained air route generation AI model.

[0156] How it works: The server uses TensorFlow and PyTorch to train a neural network based on the accumulated data, allowing the model to learn patterns to generate optimal flight paths.

[0157] Step 5: User Requests Delivery

[0158] Subject: User

[0159] Input: Delivery request from smartphone app (e.g. current address, delivery destination, delivery time).

[0160] How it works: The user enters a delivery request, which the app sends to the server.

[0161] Output: Delivery request information sent to the server.

[0162] Specific operation: The user uses the app to enter their current address (Shibuya-ku, Tokyo), workplace (Minato-ku, Tokyo), and desired delivery time (10:00 AM to 2:00 PM), and submits a delivery request.

[0163] Step 6: Generate optimal routes

[0164] Subject: Server

[0165] Input: A delivery request from the user and current weather information.

[0166] How it works: Uses route generation AI to generate optimal flight paths.

[0167] Output: Generated optimal air route information.

[0168] How it works: The server uses an AI model to input delivery request parameters (e.g., delivery destination, delivery time, weather) and selects the most efficient route from multiple candidate routes.

[0169] Step 7: Sending flight information to the terminal

[0170] Subject: Server

[0171] Input: Generated air route information.

[0172] Operation: The server transmits the generated air route information to the drone terminal in real time.

[0173] Output: Air route information received by the drone terminal.

[0174] Specific operation: The server sends the route information (starting point, intermediate points, and destination) selected by the route generation AI to the drone terminal as an HTTP request.

[0175] Step 8: In-flight monitoring

[0176] Subject: Server

[0177] Input: Flight information periodically sent from the drone terminal during flight.

[0178] How it works: The server monitors the drone in real time based on the flight information it receives and adjusts its flight path as needed.

[0179] Output: Adjusted air route information (if required).

[0180] Specific operation: The server receives the latest information from the drone terminal during flight, uses AI to recalculate the route based on the current situation, such as obstacles and weather changes, and sends the adjusted route information to the terminal in real time.

[0181] Step 9: Generate and update high-definition aerial maps

[0182] Subject: Server

[0183] Input: Flight information and environmental data from the drone device.

[0184] How it works: The server aggregates these data to generate and update aerial maps.

[0185] Output: Up-to-date aeronautical maps.

[0186] What happens: The server uses GIS software (e.g., Esri ArcGIS) to update the aerial map based on the received environmental data (e.g., newly discovered obstacles).

[0187] Step 10: Real-time delivery status notification

[0188] Subject: Server

[0189] Input: Real-time location information of the drone in flight.

[0190] Action: The server notifies the user of the delivery status.

[0191] Output: The user is notified of the location and estimated time of arrival of the delivery.

[0192] Specific operation: The server notifies the user of the drone's current location and estimated arrival time in real time via a smartphone app.

[0193] (Application example 1)

[0194] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0195] Conventional drone delivery systems have difficulty efficiently collecting and managing flight information, posing challenges in optimizing delivery routes and ensuring delivery safety. Furthermore, users have limited means of tracking delivery status in real time, making it difficult to provide accurate information on delivery arrival times. Technological solutions to address these challenges were needed.

[0196] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0197] In this invention, the server includes means for receiving flight information sent from the drone terminal and storing it on the platform, means for training an air route generation AI based on the stored flight information, means for generating an optimal air route using the air route generation AI in response to a delivery request and sending it to the drone terminal, means for monitoring flight information in real time while the drone terminal is flying and adjusting the air route as needed, means for generating and updating an aerial map based on flight information and environmental data from the drone terminal, means for notifying users of delivery status, and means for users to request drone delivery through a smartphone app and receive real-time tracking of deliveries and arrival time predictions. This enables efficient collection and management of flight information, optimizes delivery routes and ensures safety, and enables users to track delivery status in real time and receive accurate arrival predictions.

[0198] A "drone terminal" is an unmanned aerial vehicle that flies and transports objects or goods autonomously.

[0199] "Flight information" refers to data such as location, altitude, speed, and battery status acquired by a drone during flight.

[0200] A "platform" is a system infrastructure for accumulating, managing, and analyzing data.

[0201] "Air route generation AI" is an artificial intelligence that learns past flight patterns, weather information, date and time, obstacle information, etc., and autonomously generates the optimal flight route.

[0202] A "delivery request" is an instruction from a user requesting delivery by drone.

[0203] "Monitoring flight information in real time" refers to constantly monitoring the flight status of a drone while it is in flight.

[0204] "Adjusting the flight route" means instructing a drone in flight to take a new flight path as needed.

[0205] "Environmental data" refers to data related to the external environment, such as weather information and obstacle information obtained by a drone during flight.

[0206] An "aviation map" is map data that includes flight routes, obstacles, weather information, and so on.

[0207] "User" refers to a person who uses the drone delivery service.

[0208] A "smartphone app" is application software that runs on a smartphone and allows users to request drone delivery and check the delivery status.

[0209] "Real-time tracking" refers to instantly checking the location and status information during drone delivery.

[0210] "Arrival time prediction" refers to predicting and notifying the time it will take for a delivery to reach its destination.

[0211] The system of the present invention integrates multiple pieces of hardware and software to operate in order to realize efficient and safe drone delivery. Specific embodiments of the system will be described below.

[0212] Hardware:

[0213] Drone terminal: An unmanned aerial vehicle equipped with GPS, cameras, and sensors (for obstacle detection). This hardware is used to collect flight information and monitor flight status in real time.

[0214] Server: A computer that provides the computational resources for accumulating and analyzing flight information and running route generation AI.

[0215] User's smartphone: This is the device used to request drone delivery and perform real-time tracking.

[0216] software:

[0217] Route Generation AI: AI models built using TensorFlow and PyTorch perform flight path optimization, taking into account data such as historical flight patterns, weather data, time of day, and obstacle information.

[0218] Database: Flight information and user information are managed using a data management system such as PostgreSQL.

[0219] Smartphone application: An application that allows users to request drone delivery and track delivery status in real time. React Native and Flutter (registered trademark) are often used as front-end technologies.

[0220] Processing Description:

[0221] The server receives flight information sent from the drone terminal and stores it on the platform. The stored flight information is used to train the route generation AI. The route generation AI learns from past flight patterns, weather information, date and time, and obstacle information to improve its ability to generate optimal flight routes. When a user makes a delivery request through a smartphone app, the server uses the route generation AI to generate the optimal route and sends that information to the drone terminal.

[0222] More concrete examples:

[0223] For example, when a user requests food delivery from a restaurant to their home using a smartphone app, the request is sent to a server. The server uses route generation AI to generate the optimal flight path from the restaurant to their home and sends it to the drone terminal. The drone terminal flies along this path and proceeds with the delivery. If an obstacle is detected during flight, the drone terminal sends that information to the server, which then updates the aerial map. The user can check the drone's current location and estimated arrival time in real time using the smartphone app.

[0224] Example prompt sentence:

[0225] "Request a drone to deliver food to a specified address. Generate the safest and quickest route based on current weather and historical flight data."

[0226] In this way, the present invention can realize efficient and safe drone delivery, improving user convenience.

[0227] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0228] Step 1:

[0229] The server receives flight information (location, altitude, speed, battery status, etc.) sent from the drone terminal and accumulates the data on the platform. The server receives this input flight information and stores it appropriately in the database.

[0230] Step 2:

[0231] The server trains the route generation AI based on the accumulated flight information. Specifically, it receives data such as past flight patterns, weather information, date and time, and obstacle information as input, and updates the AI ​​model using TensorFlow and PyTorch to learn how to generate optimal flight routes.

[0232] Step 3:

[0233] The user makes a delivery request through a smartphone app. The user inputs the pickup and drop-off locations, and the request data is sent to the server.

[0234] Step 4:

[0235] The server receives delivery requests from users and generates the optimal flight route using route generation AI. The AI ​​receives the request data and current weather information as input, calculates the optimal route, and generates the route information.

[0236] Step 5:

[0237] The server then sends the generated optimal air route information to the drone terminal, which then begins flying along this route.

[0238] Step 6:

[0239] During flight, the drone terminal periodically transmits flight information to the server, which receives real-time flight information as input and monitors it, adjusting the flight path as needed.

[0240] Step 7:

[0241] If a drone detects an obstacle during flight, it sends that information to the server, which then updates the aerial map and recalculates the drone's route. The input is the detected obstacle information, and the output is the updated flight route.

[0242] Step 8:

[0243] The server notifies the user of the delivery status, including important information such as the delivery's location and estimated arrival time, and is communicated to the user in real time via a smartphone app.

[0244] Step 9:

[0245] Once the delivery is complete, the drone terminal transmits the final flight information to the server, which stores this new data in the platform and uses it for the next learning cycle, thereby improving the overall performance of the system.

[0246] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0247] The present invention relates to a system that recognizes a user's emotions and responds according to their state by combining an emotion engine with a drone-based delivery system. Detailed embodiments of the present invention will be described below.

[0248] The server has the function of receiving and storing flight information sent from the drone terminal. The drone terminal sends flight information such as location, altitude, speed, and battery status to the server in real time during flight. The server also trains the air route generation AI based on the accumulated flight information and improves the algorithm for generating efficient and safe air routes.

[0249] When a user requests drone delivery via a smartphone app, the server accepts the request and generates the optimal route using route generation AI. The generated route information is sent to the drone terminal in real time. During flight, the drone terminal periodically sends flight information to the server, which monitors the flight in real time and adjusts the route as necessary. The server also generates and updates an aerial map based on the flight information and environmental data.

[0250] A distinctive feature of this invention is the incorporation of an emotion engine. The emotion engine recognizes the user's emotional state and adjusts the system's behavior based on that information. Specifically, it analyzes the user's facial expressions, tone of voice, and interaction patterns to determine the user's emotions. For example, if the user is irritated, the server adjusts the content and timing of notifications and provides information to reduce the user's stress.

[0251] As a concrete example, when a user requests a drone delivery from their current address to their workplace, the information is sent to the server, and an optimal flight route is generated using air route generation AI. The generated air route information is then sent to the drone terminal, and the drone begins the delivery. Meanwhile, the server regularly monitors the user's emotional state through an emotion engine. If the user expresses irritation about a delivery delay, the server will use that information to adjust the content and timing of notifications, provide explanations to the user, and provide detailed progress reports to alleviate the user's frustration.

[0252] Furthermore, the emotion engine accumulates past emotion data and uses it for future responses, allowing it to learn the user's emotional patterns and provide more personalized responses.

[0253] In this way, by combining an emotion engine with a drone delivery system, the present invention can achieve efficient and safe delivery while improving the user experience.

[0254] The processing flow will be explained below.

[0255] Step 1:

[0256] Drone startup and initial setup

[0257] The device completes the initial setup and connects to the system, sending its identification information, initial position, altitude, and battery status to the server.

[0258] Step 2:

[0259] Real-time flight information transmission

[0260] During flight, the device periodically transmits flight information such as location, altitude, speed, and battery status to the server in real time.

[0261] Step 3:

[0262] Data accumulation

[0263] The server automatically stores the received flight information on a proprietary platform, which stores training data and data required for real-time route generation.

[0264] Step 4:

[0265] Air route generation AI training

[0266] The server trains the route generation AI based on accumulated flight information and data such as date and time, weather, and past bird strikes. The route generation AI learns and improves its algorithm to generate efficient and safe routes.

[0267] Step 5:

[0268] Accepting a delivery request

[0269] The user enters a delivery request through a smartphone app and sends it to the server.

[0270] The server analyzes the request content (delivery destination, desired time, etc.).

[0271] Step 6:

[0272] Air Route Generation

[0273] The server uses route generation AI to generate the optimal route based on the request.

[0274] The server takes the generated air route information and takes into account relevant constraints (weather, obstacles, etc.).

[0275] Step 7:

[0276] Sending air route information

[0277] The server transmits the generated air route information to the terminal (drone).

[0278] The terminal loads the received air route information into memory and prepares for flight.

[0279] Step 8:

[0280] Start of flight

[0281] The terminal starts the flight based on the received air route information.

[0282] The terminal periodically transmits flight information to the server.

[0283] Step 9:

[0284] Real-time monitoring and adjustment

[0285] The server monitors real-time data from the terminals during the flight.

[0286] The server analyzes flight information and environmental data and adjusts air routes as needed.

[0287] Step 10:

[0288] Generate and update aerial maps

[0289] The server generates and updates an aerial map based on flight information and environmental data from the drone.

[0290] Newly discovered obstacles and weather changes are instantly updated on the aerial map.

[0291] Step 11:

[0292] User Notifications

[0293] The server notifies the user of the delivery status in real time.

[0294] Users can use a smartphone app to check the current location and estimated arrival time of their delivery.

[0295] Step 12:

[0296] Emotion engine activation and emotion recognition

[0297] The server monitors the user's facial expressions, tone of voice, and interaction patterns through a smartphone or other interface.

[0298] The server runs an emotion engine that recognizes whether the user is feeling frustrated, happy, confused, etc.

[0299] Step 13:

[0300] Emotion-based notification adjustment

[0301] If the user expresses negative emotions such as irritation, the server adjusts the content and timing of the notification, for example adding detailed delivery progress information or an apology message.

[0302] If the user is satisfied, the server continues with the normal notification method.

[0303] Step 14:

[0304] Accumulating and learning emotional data

[0305] The server stores the user's emotional responses and uses them as data for future notifications and service improvements, allowing it to learn the user's emotional patterns and provide more personalized responses.

[0306] Step 15:

[0307] Delivery completed

[0308] The terminal arrives at the delivery destination and delivers the package safely.

[0309] The terminal notifies the server that the delivery is complete.

[0310] The server notifies the user that delivery has been completed.

[0311] Example 2

[0312] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0313] While conventional drone delivery systems have achieved a certain level of success in terms of delivery efficiency and safety, they lack consideration for the user's emotional state, limiting their ability to improve user experience. Furthermore, insufficient user response in the event of delivery delays or unexpected failures can result in increased user dissatisfaction. This has led to a decline in user satisfaction and left room for service improvement.

[0314] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0315] In this invention, the server includes means for receiving flight information transmitted from the drone terminal and storing it on the server, means for training an air route generation algorithm based on the stored flight information, means for generating an optimal air route using the air route generation algorithm in response to a delivery request and transmitting it to the drone terminal, means for monitoring the flight information in real time while the drone terminal is flying and adjusting the air route as necessary, means for generating and updating an air map based on the flight information and environmental data from the drone terminal, means including an emotion engine that recognizes user emotions and adjusts system behavior based on the user emotions, and means for notifying the user of delivery status and adjusting the content and timing of notification based on the user's emotional state. This enables flexible responses based on user emotions and efficient and safe delivery.

[0316] A "drone terminal" is an unmanned aerial vehicle that flies remotely or autonomously and delivers designated items.

[0317] "Flight information" refers to various flight-related data such as the drone terminal's location, altitude, speed, and battery status.

[0318] A "server" is a remote computer system that stores, processes, and analyzes data, and is a device that receives flight information from drone terminals and performs various functions based on that information.

[0319] An "air route generation algorithm" is an algorithm that uses artificial intelligence technology to calculate flight routes, and is designed to generate efficient and safe flight routes.

[0320] An "emotion engine" is a technology that analyzes a user's emotional state and adjusts the system's behavior based on that information. It is an engine that determines emotions by analyzing facial expressions and tone of voice.

[0321] A "delivery request" refers to a request made by a user via a smartphone app or other means to have a specific item delivered to a specified location.

[0322] An "aviation map" is a map generated based on flight information and environmental data, and provides information on the airspace, including the flight path of the drone terminal.

[0323] "Environmental Data" refers to data about external environmental conditions that affect flight, such as weather, temperature, and wind speed.

[0324] The present invention provides a system that recognizes the emotions of a user and responds according to that user's state by incorporating an emotion engine into a drone delivery system. Detailed embodiments of the present invention will be described below.

[0325] Hardware and software configuration

[0326] Drone terminal

[0327] Sensors: Equipped with a GPS sensor, gyro sensor, and battery monitoring system, it measures location, altitude, speed, and battery status in real time.

[0328] Communication module: The mobile communication module is used to send and receive data to and from the server in real time.

[0329] Control unit: Equipped with a control unit for flight control, it flies autonomously along a designated air route.

[0330] server

[0331] Database: The system has a database that stores and manages flight information received from drones.

[0332] Route Generation AI: Equipped with an AI model that trains route generation algorithms based on accumulated flight information to generate efficient and safe routes.

[0333] Emotion engine: Contains software that analyzes facial expressions and tone of voice to determine the user's emotional state.

[0334] System Operation

[0335] The server receives flight information sent from the drone terminal in real time and stores that information in a database. The stored data is used as training data for the air route generation AI. When a user sends a delivery request through a smartphone app, the server uses the air route generation AI to calculate the optimal flight path and sends that information to the drone terminal. The drone then begins flying autonomously along the specified air route.

[0336] Specific examples of processing

[0337] When a user requests a drone delivery from their current address to their workplace, the information is sent to the server. The server uses route generation AI to generate the optimal flight path and sends that route information to the drone terminal. The drone then begins the delivery and periodically sends flight information such as its location and battery status to the server during the flight.

[0338] Meanwhile, the server constantly monitors the user's emotional state through an emotion engine. For example, if the user expresses irritation about a delivery delay, the server can adjust the content and timing of notifications based on that information, and provide the user with detailed explanations and progress reports to alleviate the user's frustration.

[0339] Prompt Sentence Examples

[0340] "Please create the following drone delivery system model. When a user requests delivery via a smartphone app, the server accepts the request and generates the optimal flight route using route generation AI. During flight, the drone continuously sends flight information to the server, which monitors it in real time. It also uses an emotion engine to recognize the user's emotional state and adjusts the system's behavior based on that information."

[0341] In this way, the present invention combines an emotion engine with a drone delivery system to achieve efficient and safe delivery as well as an improved user experience.

[0342] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0343] Step 1: Request acceptance

[0344] Input: A user submits a delivery request from a smartphone app, including their current address, destination, and delivery details.

[0345] Processing: The user's request is sent to the server through the communication module.

[0346] Output: The server records the received request in a database and acknowledges receipt of the request.

[0347] Specific operation: The server analyzes the request data and stores the order ID, delivery item, origin and destination information in a database.

[0348] Step 2: Generate air routes

[0349] Input: The server receives the user's delivery request information.

[0350] Processing: The server runs a route generation algorithm to calculate an efficient and safe flight path. It uses a generative AI model to generate the optimal route based on the request data and accumulated historical flight information.

[0351] Output: Optimal flight path, estimated flight time, and predicted battery consumption data are generated.

[0352] Specific operation: The server sends the generated flight path to the drone terminal and starts the delivery plan.

[0353] Step 3: Send flight information

[0354] Input: The drone device receives location information, altitude, speed, and battery status collected from sensors during flight.

[0355] Processing: The drone terminal sends the collected flight information to the server via the communication module.

[0356] Output: The server monitors drone flight information in real time and stores it in a database.

[0357] Specific operation: The drone terminal collects data using GPS sensors, gyro sensors, and battery monitoring systems, and transmits the data to a server in real time.

[0358] Step 4: Real-time monitoring and air route adjustments

[0359] Input: Real-time flight information sent from the drone terminal.

[0360] Processing: The server analyzes the flight information and issues an alert if an anomaly is detected.

[0361] Output: New air routes are generated as needed and sent to the drone terminal.

[0362] Specific operation: The server checks whether the flight information is normal, and if the weather worsens or obstacles appear, it generates a new flight route and sends it to the drone terminal.

[0363] Step 5: Recognize and respond to emotional states

[0364] Input: Data on the user's facial expressions and tone of voice.

[0365] Processing: The server uses the emotion engine to analyze the user's emotional state. The emotion engine analyzes the received facial expressions and tone of voice and processes the data to determine the user's emotion.

[0366] Output: Notification content and delivery information are generated according to the user's emotional state.

[0367] Specific operation: The server uses an emotion engine to perform emotion analysis, such as "the user is irritated," and changes the content of the notification appropriately depending on the state.

[0368] Step 6: Accumulating and Learning Emotional Data

[0369] Input: User's historical emotion data.

[0370] Processing: The server stores past emotional data in the emotion engine database. Emotional patterns are learned based on this data.

[0371] Output: More personalized user-responsive data will be generated in the future.

[0372] Specific operation: The server accumulates emotional data, and the emotion engine uses this data to learn the user's emotional patterns, resulting in more precise and personalized responses to the user from the next time onwards.

[0373] (Application example 2)

[0374] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0375] Conventional drone delivery systems lack the flexibility to respond to the user's emotional state, and improvements to the user experience are desired. Furthermore, there is a lack of means to alleviate the user's anxiety and frustration during delivery, making it a challenge to improve user satisfaction.

[0376] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0377] In this invention, the server includes means for receiving flight information transmitted from the drone terminal and storing it on the platform, means for training an air route generation AI based on the stored flight information, means for generating an optimal air route using the air route generation AI in response to a delivery request and transmitting it to the drone terminal, means for monitoring flight information in real time while the drone terminal is flying and adjusting the air route as necessary, means for generating and updating an aerial map based on flight information and environmental data from the drone terminal, means for notifying the user of the delivery status, and means for combining an emotion engine that analyzes the user's emotional state and adjusts the content and timing of notifications based on that information. This enables flexible responses based on the user's emotional state, reducing the user's anxiety and irritation during delivery and improving user satisfaction.

[0378] A "drone terminal" is an unmanned aerial vehicle that flies through the air and delivers goods to designated locations.

[0379] "Flight information" refers to data such as the drone terminal's location, altitude, speed, and battery status.

[0380] The "platform" is a server or cloud-based data management system that receives and stores drone flight information.

[0381] "Air Route Generation AI" is an artificial intelligence algorithm for optimizing drone flight paths.

[0382] An "emotion engine" is an engine that has the function of analyzing emotions from the user's facial expressions, tone of voice, interaction patterns, etc., and adjusting the system's behavior based on that information.

[0383] "Environmental data" refers to data such as weather information and obstacle information that affect drone flight.

[0384] An "aviation map" is map data that includes route information and obstacle information for drone flight.

[0385] The present invention provides a system that flexibly responds to the emotional state of a user by combining an emotion engine with a drone-based delivery system. Detailed embodiments of the present invention will be described below.

[0386] The system mainly consists of the following components:

[0387] 1. Drone terminal

[0388] The drone terminal is an unmanned aerial vehicle that flies through the air and delivers goods to designated locations. The drone terminal transmits flight information such as location, altitude, speed, and battery status to a server in real time.

[0389] 2. Platform

[0390] The platform consists of a server and a cloud-based data management system that receives and stores flight information sent from drone terminals. Based on this information, the air route generation AI and emotion engine operate.

[0391] 3. Air route generation AI

[0392] Route generation AI is an artificial intelligence algorithm for optimizing drone flight paths. It is trained based on flight information and environmental data, such as the date and time, weather, and past bird strike information, to generate optimal routes.

[0393] 4. Emotion Engine

[0394] The emotion engine analyzes the user's facial expressions, tone of voice, and interaction patterns to determine their emotions, and adjusts the system's behavior based on this information. This engine is built using deep learning frameworks such as TensorFlow and Keras. It adjusts the content and timing of notifications to reduce user frustration and anxiety.

[0395] 5. Aerial map generation and update

[0396] The server generates and updates an aerial map based on flight information and environmental data from the drone terminals, and if new obstacles are detected, the information is also reflected on the map.

[0397] 6. Notification System

[0398] The server notifies the user of the delivery status. The content and timing of the notification are adjusted based on the user's emotional state analyzed by the emotion engine. This reduces the user's anxiety and frustration and improves satisfaction.

[0399] Hardware and Software Configuration

[0400] Server: Cloud-based data management system (Amazon Web Services, Microsoft® Azure®, etc.)

[0401] Drone terminal: Small unmanned aerial vehicle

[0402] Emotion recognition system: TensorFlow, Keras

[0403] Real-time communication: REST API and WebSocket

[0404] Specific examples

[0405] A user requests a food delivery drone using a smartphone app. The user's facial expressions and tone of voice are captured by a camera and microphone, and the data is analyzed by an emotion engine. Based on the analysis results, the content and timing of delivery notifications are adjusted. If the user shows signs of impatience, a message such as "Sorry for the wait. Delivery status: On its way." is displayed.

[0406] Prompt Sentence Examples

[0407] "Generate optimal delivery notification content based on the user's emotional state. Also, adjust the notification content in real time according to the delivery status."

[0408] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0409] Step 1:

[0410] A user requests drone delivery from a smartphone app. At this time, the app receives the user's delivery request data (shipping origin, delivery destination, order details, etc.) as input and sends it to the server. The server receives this data and stores it on the platform.

[0411] Step 2:

[0412] The server trains the route generation AI based on the accumulated flight information. It uses historical flight information and environmental data as input, and updates the model to generate optimal routes as output. This model is trained using an AI algorithm (e.g., TensorFlow or Keras).

[0413] Step 3:

[0414] The server generates the optimal flight route based on the user's delivery request using an AI route generation model. The inputs are the delivery origin, delivery destination, and real-time environmental data (weather, obstacle information, etc.), and the output is the optimal flight path. This route data is sent to the drone terminal.

[0415] Step 4:

[0416] The drone terminal will then begin delivery based on the received flight route. During flight, the drone terminal will continue to send real-time flight information, such as location, altitude, speed, and battery status, to the server. The server receives this data and stores it on the platform.

[0417] Step 5:

[0418] The server monitors the real-time flight information of the drone during flight and adjusts the flight path as needed. For example, if a new obstacle is detected, the flight path generation AI is reactivated, recalculating the optimal route and sending it to the drone. The obstacle information is used as input, and the updated flight path is obtained as output.

[0419] Step 6:

[0420] Based on the flight information and environmental data sent from the drone terminal, the server generates and updates an aerial map. It uses real-time flight information and environmental data as input and creates an updated aerial map as output. This map is used to generate future flight paths.

[0421] Step 7:

[0422] Analyze the user's emotional state. Emotional data such as facial expressions and tone of voice obtained from the user's smartphone app is input into the emotion engine. The emotion engine analyzes this data to determine the user's emotional state. For example, a Keras model can be used to analyze facial expression data and output emotional states such as irritation or anxiety.

[0423] Step 8:

[0424] The server adjusts the content and timing of notifications to the user based on the emotional state analyzed by the emotion engine. The emotional state and delivery status are used as inputs, and an appropriate notification message is generated as output to be sent to the user. For example, if the user is annoyed, a message detailing the progress of the delivery along with an apology is generated.

[0425] Step 9:

[0426] Finally, the server sends the generated notification message to the user's smartphone app, allowing the user to receive timely notifications about the delivery status, reducing user anxiety and frustration and improving overall satisfaction.

[0427] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0428] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0429] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0430] [Second embodiment]

[0431] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0432] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0433] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0434] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0435] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0436] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0437] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0438] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0439] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0440] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0441] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0442] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0443] The present invention relates to a system for realizing efficient and safe drone delivery, and detailed embodiments thereof will be described below.

[0444] The server has the function of receiving flight information sent from the drone terminal. During flight, the drone terminal sends flight information such as location, altitude, speed, and battery status to the server in real time. The server accumulates and stores this information on its own platform. This data is used to train the air route generation AI (described later) and to generate air routes in real time.

[0445] The server then uses the accumulated flight information to train the route generation AI, which learns relevant data such as past flight patterns, weather information, date and time, and past bird strikes to improve its ability to generate optimal flight paths, thereby providing more efficient and safer routes for future delivery requests.

[0446] When a user requests a drone delivery, the server accepts the request and generates the optimal route using route generation AI. The route generation AI considers parameters such as current weather, the specified delivery destination, and delivery time to select the most efficient route from multiple flight paths. The generated route information is sent to the drone terminal in real time.

[0447] During flight, the drone terminals periodically send flight information to the server. The server uses this information to monitor drone operations in real time and adjust air routes as necessary. The server also integrates the flight information and environmental data received from the drone terminals to generate and update a highly accurate aerial map. For example, newly discovered obstacles or sudden weather changes can be immediately reflected in the aerial map, ensuring that all drone terminals are flying based on the most up-to-date information.

[0448] In addition, the server provides users with real-time delivery status notifications, allowing them to receive information about the location of their delivery, estimated arrival time, and other important notifications via a smartphone app.

[0449] As a concrete example, when a user requests "drone delivery from their current address to their workplace" through a smartphone app, the request is sent to a server. The server uses route generation AI to generate the optimal flight route from their current address to their workplace and sends it to the drone terminal. The drone terminal then flies along this route and proceeds with the delivery. If an obstacle is detected during flight, the drone terminal sends that information to the server, which then updates the aerial map. The user can check the drone's current location and estimated arrival time in real time through the smartphone app.

[0450] In this way, the present invention enables efficient and safe drone delivery, while also improving user convenience.

[0451] The processing flow will be explained below.

[0452] Step 1:

[0453] Drone startup and initial setup

[0454] The device completes the initial setup and connects to the system, sending its identification information, initial position, altitude, and battery status to the server.

[0455] Step 2:

[0456] Real-time flight information transmission

[0457] During flight, the device periodically transmits flight information such as location, altitude, speed, and battery status to the server in real time.

[0458] Step 3:

[0459] Data accumulation

[0460] The server automatically stores the received flight information on its own platform, which stores the data necessary for subsequent analysis and route generation.

[0461] Step 4:

[0462] Air route generation AI training

[0463] The server trains the route generation AI based on accumulated flight information and data such as date and time, weather, and past bird strikes, allowing the route generation AI to learn and improve its algorithm for generating efficient and safe routes.

[0464] Step 5:

[0465] Accepting a delivery request

[0466] The user enters a delivery request through a smartphone app and sends it to the server.

[0467] The server analyzes the request content (delivery destination, desired time, etc.).

[0468] Step 6:

[0469] Air Route Generation

[0470] The server uses route generation AI to generate the optimal route based on the request.

[0471] The server takes the generated air route information and takes into account relevant constraints (weather, obstacles, etc.).

[0472] Step 7:

[0473] Sending air route information

[0474] The server transmits the generated air route information to the terminal (drone).

[0475] The terminal loads the received air route information into memory and prepares for flight.

[0476] Step 8:

[0477] Start of flight

[0478] The terminal starts the flight based on the received air route information.

[0479] The terminal periodically transmits flight information to the server.

[0480] Step 9:

[0481] Real-time monitoring and adjustment

[0482] The server monitors real-time data from the terminals during the flight.

[0483] The server analyzes flight information and environmental data and adjusts air routes as needed.

[0484] Step 10:

[0485] Generate and update aerial maps

[0486] The server generates and updates an aerial map based on flight information and environmental data from the drone.

[0487] Newly discovered obstacles and weather changes are instantly updated on the aerial map.

[0488] Step 11:

[0489] User Notifications

[0490] The server notifies the user of the delivery status in real time.

[0491] Users can use a smartphone app to check the current location and estimated arrival time of their delivery.

[0492] Step 12:

[0493] Delivery completed

[0494] The terminal arrives at the delivery destination and delivers the package safely.

[0495] The terminal notifies the server that the delivery is complete.

[0496] The server notifies the user that delivery has been completed.

[0497] Example 1

[0498] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0499] Drone delivery presents many challenges, including optimizing flight routes for efficiency, ensuring safety during flight, and understanding delivery status in real time. In particular, drone operation can be disrupted if the flight route is inappropriate or if unexpected obstacles appear during flight. Furthermore, when multiple drones are operating simultaneously, it is important to ensure they do not interfere with each other. Another problem is the lack of means to take rapid emergency measures when trouble occurs.

[0500] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0501] In this invention, the server includes means for receiving flight information transmitted from the drone terminal and storing it on the platform, means for training an air route generation AI based on the stored flight information, means for generating an optimal air route using the air route generation AI in response to a delivery request and transmitting it to the drone terminal, means for monitoring flight information in real time while the drone terminal is flying and adjusting the air route as necessary, means for generating and updating an aerial map based on flight information and environmental data from the drone terminal, means for notifying the user of the delivery status, means for taking emergency measures in the event of a problem based on flight information collected by the drone terminal, and means for multiple drone terminals to cooperate and share flight information to improve overall flight efficiency. This enables efficient and safe drone delivery and improves user convenience.

[0502] "Flight information" is a general term for a wide variety of data related to operation, such as location information, altitude, speed, and battery status obtained from drone terminals.

[0503] "Platform" refers to an integrated environment of servers and related software for receiving, storing, and managing flight information sent from drone terminals.

[0504] "Air Route Generation AI" is an AI (artificial intelligence) model that generates optimal flight routes based on accumulated flight information, weather data, past flight patterns, etc.

[0505] "Delivery Request" means a request for drone delivery made by a User through a smartphone app or other interface.

[0506] The "optimal route" refers to the flight route with the highest flight efficiency, selected based on the conditions specified by the route generation AI (delivery destination, weather, time of day, etc.).

[0507] "Monitoring" refers to the process of monitoring flight information transmitted by drone terminals in real time and adjusting operational status as necessary.

[0508] "Environmental data" is a general term for data related to the flight environment, such as weather information and the presence of obstacles, acquired by a drone terminal during flight.

[0509] "Aerial Map" refers to a high-precision aerial map generated by integrating flight information and environmental data.

[0510] "Emergency Plan" means the procedures and protocols for responding quickly if a problem occurs during flight.

[0511] "Improving flight efficiency" refers to the process of multiple drone devices working together and sharing flight information to minimize overall flight time and energy consumption.

[0512] The present invention relates to a system for realizing efficient and safe drone delivery, and detailed embodiments thereof will be described below.

[0513] Hardware and software used

[0514] 1. Server

[0515] Server: High-performance server (e.g. Dell PowerEdge R740)

[0516] Server software: Python, Django, MySQL

[0517] Real-time data processing: Apache Kafka

[0518] 2. Drone terminal

[0519] Drone hardware: A typical high-performance drone (e.g., DJI Mavic 2)

[0520] Communication module: Standard internet communication module

[0521] Drone control application: Drone control SDK (e.g. DJI SDK)

[0522] 3. Air route generation AI

[0523] Deep Learning Frameworks: TensorFlow, PyTorch

[0524] 4. GIS Software

[0525] Aerial Map Generation Tool: Esri ArcGIS

[0526] Program processing

[0527] The server has the function of receiving flight information sent from the drone terminal. During flight, the drone terminal sends flight information such as location, altitude, speed, and battery status to the server in real time. The server stores this received information in a database. This database is operated using MySQL.

[0528] The server trains the route generation AI based on the accumulated flight information. The route generation AI builds an optimization model using Python, TensorFlow, and PyTorch to learn relevant data such as past flight patterns, weather information, date and time, and past bird strikes. This allows it to generate efficient and safe flight routes for future delivery requests.

[0529] When a user requests a drone delivery, the server accepts the request and generates the optimal air route using an air route generation AI. The request includes parameters such as the specified delivery destination and delivery time, and the server selects the optimal route taking this information into account. The generated air route information is then sent to the drone terminal.

[0530] During flight, the drones periodically send flight information to the server. The server monitors this information and adjusts flight routes as necessary. The server also integrates the flight information and environmental data received from the drones to generate and update a highly accurate aerial map. For example, newly discovered obstacles or sudden weather changes can be instantly reflected in the aerial map, ensuring that all drones fly based on the most up-to-date information.

[0531] Additionally, the server provides users with real-time delivery status notifications, allowing them to receive information about the location of their shipments, estimated arrival times, and other important notifications via a smartphone app.

[0532] Specific examples

[0533] When a user requests "drone delivery from their current address to their workplace" via a smartphone app, the request is sent to the server.

[0534] Example prompt sentence:

[0535] I would like to request drone delivery from my current address to my workplace. My current address is in Shibuya-ku, Tokyo, and my workplace is in Minato-ku, Tokyo. I would like the delivery time to be between 10:00 AM and 2:00 PM.

[0536] The server uses route generation AI to generate the optimal flight route from the current address to the workplace and sends it to the drone terminal. The drone terminal then flies along this route and proceeds with the delivery. If an obstacle is detected during flight, the drone terminal sends that information to the server, which then updates the aerial map. The user can check the drone's current location and estimated arrival time in real time via a smartphone app.

[0537] In this way, the present invention enables efficient and safe drone delivery, improving user convenience.

[0538] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0539] Step 1: Gather flight information

[0540] Subject: Device

[0541] Input: Location, altitude, speed, and battery status obtained from the drone's built-in sensors and GPS module.

[0542] How it works: The drone terminal reads the data obtained from these sensors and the GPS module at regular intervals.

[0543] Output: Retrieved flight information data.

[0544] Specific operation: The drone terminal obtains the current latitude and longitude, altitude (meters), speed (kilometers per hour), and battery status (percentage) and generates flight information data.

[0545] Step 2: Send flight information

[0546] Subject: Device

[0547] Input: Collected flight information.

[0548] Operation: The terminal sends the collected flight information to the server via the Internet communication module.

[0549] Output: Flight information sent to the server.

[0550] Specific operation: The drone terminal sends the latest flight information (location, altitude, speed, battery status) to the server as an HTTP request at regular intervals.

[0551] Step 3: Receiving and storing flight information

[0552] Subject: Server

[0553] Input: Flight information sent from the device.

[0554] Operation: The server stores the received flight information in a database.

[0555] Output: Flight information stored in a database.

[0556] What it does: The server receives the HTTP request, parses it, and stores flight information data in a MySQL database, including latitude, longitude, altitude, speed, battery status, and timestamps.

[0557] Step 4: Training the route generation AI

[0558] Subject: Server

[0559] Input: Past flight information, weather information, date and time, and past bird strike information stored in the database.

[0560] How it works: The server uses these data to train the route generation AI.

[0561] Output: A trained air route generation AI model.

[0562] How it works: The server uses TensorFlow and PyTorch to train a neural network based on the accumulated data, allowing the model to learn patterns to generate optimal flight paths.

[0563] Step 5: User Requests Delivery

[0564] Subject: User

[0565] Input: Delivery request from smartphone app (e.g. current address, delivery destination, delivery time).

[0566] How it works: The user enters a delivery request, which the app sends to the server.

[0567] Output: Delivery request information sent to the server.

[0568] Specific operation: The user uses the app to enter their current address (Shibuya-ku, Tokyo), workplace (Minato-ku, Tokyo), and desired delivery time (10:00 AM to 2:00 PM), and submits a delivery request.

[0569] Step 6: Generate optimal routes

[0570] Subject: Server

[0571] Input: A delivery request from the user and current weather information.

[0572] How it works: Uses route generation AI to generate optimal flight paths.

[0573] Output: Generated optimal air route information.

[0574] How it works: The server uses an AI model to input delivery request parameters (e.g., delivery destination, delivery time, weather) and selects the most efficient route from multiple candidate routes.

[0575] Step 7: Sending flight information to the terminal

[0576] Subject: Server

[0577] Input: Generated air route information.

[0578] Operation: The server transmits the generated air route information to the drone terminal in real time.

[0579] Output: Air route information received by the drone terminal.

[0580] Specific operation: The server sends the route information (starting point, intermediate points, and destination) selected by the route generation AI to the drone terminal as an HTTP request.

[0581] Step 8: In-flight monitoring

[0582] Subject: Server

[0583] Input: Flight information periodically sent from the drone terminal during flight.

[0584] How it works: The server monitors the drone in real time based on the flight information it receives and adjusts its flight path as needed.

[0585] Output: Adjusted air route information (if required).

[0586] Specific operation: The server receives the latest information from the drone terminal during flight, uses AI to recalculate the route based on the current situation, such as obstacles and weather changes, and sends the adjusted route information to the terminal in real time.

[0587] Step 9: Generate and update high-definition aerial maps

[0588] Subject: Server

[0589] Input: Flight information and environmental data from the drone device.

[0590] How it works: The server aggregates these data to generate and update aerial maps.

[0591] Output: Up-to-date aeronautical maps.

[0592] What happens: The server uses GIS software (e.g., Esri ArcGIS) to update the aerial map based on the received environmental data (e.g., newly discovered obstacles).

[0593] Step 10: Real-time delivery status notification

[0594] Subject: Server

[0595] Input: Real-time location information of the drone in flight.

[0596] Action: The server notifies the user of the delivery status.

[0597] Output: The user is notified of the location and estimated time of arrival of the delivery.

[0598] Specific operation: The server notifies the user of the drone's current location and estimated arrival time in real time via a smartphone app.

[0599] (Application example 1)

[0600] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0601] Conventional drone delivery systems have difficulty efficiently collecting and managing flight information, posing challenges in optimizing delivery routes and ensuring delivery safety. Furthermore, users have limited means of tracking delivery status in real time, making it difficult to provide accurate information on delivery arrival times. Technological solutions to address these challenges were needed.

[0602] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0603] In this invention, the server includes means for receiving flight information sent from the drone terminal and storing it on the platform, means for training an air route generation AI based on the stored flight information, means for generating an optimal air route using the air route generation AI in response to a delivery request and sending it to the drone terminal, means for monitoring flight information in real time while the drone terminal is flying and adjusting the air route as needed, means for generating and updating an aerial map based on flight information and environmental data from the drone terminal, means for notifying users of delivery status, and means for users to request drone delivery through a smartphone app and receive real-time tracking of deliveries and arrival time predictions. This enables efficient collection and management of flight information, optimizes delivery routes and ensures safety, and enables users to track delivery status in real time and receive accurate arrival predictions.

[0604] A "drone terminal" is an unmanned aerial vehicle that flies and transports objects or goods autonomously.

[0605] "Flight information" refers to data such as location, altitude, speed, and battery status acquired by a drone during flight.

[0606] A "platform" is a system infrastructure for accumulating, managing, and analyzing data.

[0607] "Air route generation AI" is an artificial intelligence that learns past flight patterns, weather information, date and time, obstacle information, etc., and autonomously generates the optimal flight route.

[0608] A "delivery request" is an instruction from a user requesting delivery by drone.

[0609] "Monitoring flight information in real time" refers to constantly monitoring the flight status of a drone while it is in flight.

[0610] "Adjusting the flight route" means instructing a drone in flight to take a new flight path as needed.

[0611] "Environmental data" refers to data related to the external environment, such as weather information and obstacle information obtained by a drone during flight.

[0612] An "aviation map" is map data that includes flight routes, obstacles, weather information, and so on.

[0613] "User" refers to a person who uses the drone delivery service.

[0614] A "smartphone app" is application software that runs on a smartphone and allows users to request drone delivery and check the delivery status.

[0615] "Real-time tracking" refers to instantly checking the location and status information during drone delivery.

[0616] "Arrival time prediction" refers to predicting and notifying the time it will take for a delivery to reach its destination.

[0617] The system of the present invention integrates multiple pieces of hardware and software to operate in order to realize efficient and safe drone delivery. Specific embodiments of the system will be described below.

[0618] Hardware:

[0619] Drone terminal: An unmanned aerial vehicle equipped with GPS, cameras, and sensors (for obstacle detection). This hardware is used to collect flight information and monitor flight status in real time.

[0620] Server: A computer that provides the computational resources for accumulating and analyzing flight information and running route generation AI.

[0621] User's smartphone: This is the device used to request drone delivery and perform real-time tracking.

[0622] software:

[0623] Route Generation AI: AI models built using TensorFlow and PyTorch perform flight path optimization, taking into account data such as historical flight patterns, weather data, time of day, and obstacle information.

[0624] Database: Flight information and user information are managed using a data management system such as PostgreSQL.

[0625] Smartphone application: An application that allows users to request drone delivery and track the delivery status in real time. React Native and Flutter are often used as front-end technologies.

[0626] Processing Description:

[0627] The server receives flight information sent from the drone terminal and stores it on the platform. The stored flight information is used to train the route generation AI. The route generation AI learns from past flight patterns, weather information, date and time, and obstacle information to improve its ability to generate optimal flight routes. When a user makes a delivery request through a smartphone app, the server uses the route generation AI to generate the optimal route and sends that information to the drone terminal.

[0628] More concrete examples:

[0629] For example, when a user requests food delivery from a restaurant to their home using a smartphone app, the request is sent to a server. The server uses route generation AI to generate the optimal flight path from the restaurant to their home and sends it to the drone terminal. The drone terminal flies along this path and proceeds with the delivery. If an obstacle is detected during flight, the drone terminal sends that information to the server, which then updates the aerial map. The user can check the drone's current location and estimated arrival time in real time using the smartphone app.

[0630] Example prompt sentence:

[0631] "Request a drone to deliver food to a specified address. Generate the safest and quickest route based on current weather and historical flight data."

[0632] In this way, the present invention can realize efficient and safe drone delivery, improving user convenience.

[0633] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0634] Step 1:

[0635] The server receives flight information (location, altitude, speed, battery status, etc.) sent from the drone terminal and accumulates the data on the platform. The server receives this input flight information and stores it appropriately in the database.

[0636] Step 2:

[0637] The server trains the route generation AI based on the accumulated flight information. Specifically, it receives data such as past flight patterns, weather information, date and time, and obstacle information as input, and updates the AI ​​model using TensorFlow and PyTorch to learn how to generate optimal flight routes.

[0638] Step 3:

[0639] The user makes a delivery request through a smartphone app. The user inputs the pickup and drop-off locations, and the request data is sent to the server.

[0640] Step 4:

[0641] The server receives delivery requests from users and generates the optimal flight route using route generation AI. The AI ​​receives the request data and current weather information as input, calculates the optimal route, and generates the route information.

[0642] Step 5:

[0643] The server then sends the generated optimal air route information to the drone terminal, which then begins flying along this route.

[0644] Step 6:

[0645] During flight, the drone terminal periodically transmits flight information to the server, which receives real-time flight information as input and monitors it, adjusting the flight path as needed.

[0646] Step 7:

[0647] If a drone detects an obstacle during flight, it sends that information to the server, which then updates the aerial map and recalculates the drone's route. The input is the detected obstacle information, and the output is the updated flight route.

[0648] Step 8:

[0649] The server notifies the user of the delivery status, including important information such as the delivery's location and estimated arrival time, and is communicated to the user in real time via a smartphone app.

[0650] Step 9:

[0651] Once the delivery is complete, the drone terminal transmits the final flight information to the server, which stores this new data in the platform and uses it for the next learning cycle, thereby improving the overall performance of the system.

[0652] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0653] The present invention relates to a system that recognizes a user's emotions and responds according to their state by combining an emotion engine with a drone-based delivery system. Detailed embodiments of the present invention will be described below.

[0654] The server has the function of receiving and storing flight information sent from the drone terminal. The drone terminal sends flight information such as location, altitude, speed, and battery status to the server in real time during flight. The server also trains the air route generation AI based on the accumulated flight information and improves the algorithm for generating efficient and safe air routes.

[0655] When a user requests drone delivery via a smartphone app, the server accepts the request and generates the optimal route using route generation AI. The generated route information is sent to the drone terminal in real time. During flight, the drone terminal periodically sends flight information to the server, which monitors the flight in real time and adjusts the route as necessary. The server also generates and updates an aerial map based on the flight information and environmental data.

[0656] A distinctive feature of this invention is the incorporation of an emotion engine. The emotion engine recognizes the user's emotional state and adjusts the system's behavior based on that information. Specifically, it analyzes the user's facial expressions, tone of voice, and interaction patterns to determine the user's emotions. For example, if the user is irritated, the server adjusts the content and timing of notifications and provides information to reduce the user's stress.

[0657] As a concrete example, when a user requests a drone delivery from their current address to their workplace, the information is sent to the server, and an optimal flight route is generated using air route generation AI. The generated air route information is then sent to the drone terminal, and the drone begins the delivery. Meanwhile, the server regularly monitors the user's emotional state through an emotion engine. If the user expresses irritation about a delivery delay, the server will use that information to adjust the content and timing of notifications, provide explanations to the user, and provide detailed progress reports to alleviate the user's frustration.

[0658] Furthermore, the emotion engine accumulates past emotion data and uses it for future responses, allowing it to learn the user's emotional patterns and provide more personalized responses.

[0659] In this way, by combining an emotion engine with a drone delivery system, the present invention can achieve efficient and safe delivery while improving the user experience.

[0660] The processing flow will be explained below.

[0661] Step 1:

[0662] Drone startup and initial setup

[0663] The device completes the initial setup and connects to the system, sending its identification information, initial position, altitude, and battery status to the server.

[0664] Step 2:

[0665] Real-time flight information transmission

[0666] During flight, the device periodically transmits flight information such as location, altitude, speed, and battery status to the server in real time.

[0667] Step 3:

[0668] Data accumulation

[0669] The server automatically stores the received flight information on a proprietary platform, which stores training data and data required for real-time route generation.

[0670] Step 4:

[0671] Air route generation AI training

[0672] The server trains the route generation AI based on accumulated flight information and data such as date and time, weather, and past bird strikes. The route generation AI learns and improves its algorithm to generate efficient and safe routes.

[0673] Step 5:

[0674] Accepting a delivery request

[0675] The user enters a delivery request through a smartphone app and sends it to the server.

[0676] The server analyzes the request content (delivery destination, desired time, etc.).

[0677] Step 6:

[0678] Air Route Generation

[0679] The server uses route generation AI to generate the optimal route based on the request.

[0680] The server takes the generated air route information and takes into account relevant constraints (weather, obstacles, etc.).

[0681] Step 7:

[0682] Sending air route information

[0683] The server transmits the generated air route information to the terminal (drone).

[0684] The terminal loads the received air route information into memory and prepares for flight.

[0685] Step 8:

[0686] Start of flight

[0687] The terminal starts the flight based on the received air route information.

[0688] The terminal periodically transmits flight information to the server.

[0689] Step 9:

[0690] Real-time monitoring and adjustment

[0691] The server monitors real-time data from the terminals during the flight.

[0692] The server analyzes flight information and environmental data and adjusts air routes as needed.

[0693] Step 10:

[0694] Generate and update aerial maps

[0695] The server generates and updates an aerial map based on flight information and environmental data from the drone.

[0696] Newly discovered obstacles and weather changes are instantly updated on the aerial map.

[0697] Step 11:

[0698] User Notifications

[0699] The server notifies the user of the delivery status in real time.

[0700] Users can use a smartphone app to check the current location and estimated arrival time of their delivery.

[0701] Step 12:

[0702] Emotion engine activation and emotion recognition

[0703] The server monitors the user's facial expressions, tone of voice, and interaction patterns through a smartphone or other interface.

[0704] The server runs an emotion engine that recognizes whether the user is feeling frustrated, happy, confused, etc.

[0705] Step 13:

[0706] Emotion-based notification adjustment

[0707] If the user expresses negative emotions such as irritation, the server adjusts the content and timing of the notification, for example adding detailed delivery progress information or an apology message.

[0708] If the user is satisfied, the server continues with the normal notification method.

[0709] Step 14:

[0710] Accumulating and learning emotional data

[0711] The server stores the user's emotional responses and uses them as data for future notifications and service improvements, allowing it to learn the user's emotional patterns and provide more personalized responses.

[0712] Step 15:

[0713] Delivery completed

[0714] The terminal arrives at the delivery destination and delivers the package safely.

[0715] The terminal notifies the server that the delivery is complete.

[0716] The server notifies the user that delivery has been completed.

[0717] Example 2

[0718] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0719] While conventional drone delivery systems have achieved a certain level of success in terms of delivery efficiency and safety, they lack consideration for the user's emotional state, limiting their ability to improve user experience. Furthermore, insufficient user response in the event of delivery delays or unexpected failures can result in increased user dissatisfaction. This has led to a decline in user satisfaction and left room for service improvement.

[0720] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0721] In this invention, the server includes means for receiving flight information transmitted from the drone terminal and storing it on the server, means for training an air route generation algorithm based on the stored flight information, means for generating an optimal air route using the air route generation algorithm in response to a delivery request and transmitting it to the drone terminal, means for monitoring the flight information in real time while the drone terminal is flying and adjusting the air route as necessary, means for generating and updating an air map based on the flight information and environmental data from the drone terminal, means including an emotion engine that recognizes user emotions and adjusts system behavior based on the user emotions, and means for notifying the user of delivery status and adjusting the content and timing of notification based on the user's emotional state. This enables flexible responses based on user emotions and efficient and safe delivery.

[0722] A "drone terminal" is an unmanned aerial vehicle that flies remotely or autonomously and delivers designated items.

[0723] "Flight information" refers to various flight-related data such as the drone terminal's location, altitude, speed, and battery status.

[0724] A "server" is a remote computer system that stores, processes, and analyzes data, and is a device that receives flight information from drone terminals and performs various functions based on that information.

[0725] An "air route generation algorithm" is an algorithm that uses artificial intelligence technology to calculate flight routes, and is designed to generate efficient and safe flight routes.

[0726] An "emotion engine" is a technology that analyzes a user's emotional state and adjusts the system's behavior based on that information. It is an engine that determines emotions by analyzing facial expressions and tone of voice.

[0727] A "delivery request" refers to a request made by a user via a smartphone app or other means to have a specific item delivered to a specified location.

[0728] An "aviation map" is a map generated based on flight information and environmental data, and provides information on the airspace, including the flight path of the drone terminal.

[0729] "Environmental Data" refers to data about external environmental conditions that affect flight, such as weather, temperature, and wind speed.

[0730] The present invention provides a system that recognizes the emotions of a user and responds according to that user's state by incorporating an emotion engine into a drone delivery system. Detailed embodiments of the present invention will be described below.

[0731] Hardware and software configuration

[0732] Drone terminal

[0733] Sensors: Equipped with a GPS sensor, gyro sensor, and battery monitoring system, it measures location, altitude, speed, and battery status in real time.

[0734] Communication module: The mobile communication module is used to send and receive data to and from the server in real time.

[0735] Control unit: Equipped with a control unit for flight control, it flies autonomously along a designated air route.

[0736] server

[0737] Database: The system has a database that stores and manages flight information received from drones.

[0738] Route Generation AI: Equipped with an AI model that trains route generation algorithms based on accumulated flight information to generate efficient and safe routes.

[0739] Emotion engine: Contains software that analyzes facial expressions and tone of voice to determine the user's emotional state.

[0740] System Operation

[0741] The server receives flight information sent from the drone terminal in real time and stores that information in a database. The stored data is used as training data for the air route generation AI. When a user sends a delivery request through a smartphone app, the server uses the air route generation AI to calculate the optimal flight path and sends that information to the drone terminal. The drone then begins flying autonomously along the specified air route.

[0742] Specific examples of processing

[0743] When a user requests a drone delivery from their current address to their workplace, the information is sent to the server. The server uses route generation AI to generate the optimal flight path and sends that route information to the drone terminal. The drone then begins the delivery and periodically sends flight information such as its location and battery status to the server during the flight.

[0744] Meanwhile, the server constantly monitors the user's emotional state through an emotion engine. For example, if the user expresses irritation about a delivery delay, the server can adjust the content and timing of notifications based on that information, and provide the user with detailed explanations and progress reports to alleviate the user's frustration.

[0745] Prompt Sentence Examples

[0746] "Please create the following drone delivery system model. When a user requests delivery via a smartphone app, the server accepts the request and generates the optimal flight route using route generation AI. During flight, the drone continuously sends flight information to the server, which monitors it in real time. It also uses an emotion engine to recognize the user's emotional state and adjusts the system's behavior based on that information."

[0747] In this way, the present invention combines an emotion engine with a drone delivery system to achieve efficient and safe delivery as well as an improved user experience.

[0748] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0749] Step 1: Request acceptance

[0750] Input: A user submits a delivery request from a smartphone app, including their current address, destination, and delivery details.

[0751] Processing: The user's request is sent to the server through the communication module.

[0752] Output: The server records the received request in a database and acknowledges receipt of the request.

[0753] Specific operation: The server analyzes the request data and stores the order ID, delivery item, origin and destination information in a database.

[0754] Step 2: Generate air routes

[0755] Input: The server receives the user's delivery request information.

[0756] Processing: The server runs a route generation algorithm to calculate an efficient and safe flight path. It uses a generative AI model to generate the optimal route based on the request data and accumulated historical flight information.

[0757] Output: Optimal flight path, estimated flight time, and predicted battery consumption data are generated.

[0758] Specific operation: The server sends the generated flight path to the drone terminal and starts the delivery plan.

[0759] Step 3: Send flight information

[0760] Input: The drone device receives location information, altitude, speed, and battery status collected from sensors during flight.

[0761] Processing: The drone terminal sends the collected flight information to the server via the communication module.

[0762] Output: The server monitors drone flight information in real time and stores it in a database.

[0763] Specific operation: The drone terminal collects data using GPS sensors, gyro sensors, and battery monitoring systems, and transmits the data to a server in real time.

[0764] Step 4: Real-time monitoring and air route adjustments

[0765] Input: Real-time flight information sent from the drone terminal.

[0766] Processing: The server analyzes the flight information and issues an alert if an anomaly is detected.

[0767] Output: New air routes are generated as needed and sent to the drone terminal.

[0768] Specific operation: The server checks whether the flight information is normal, and if the weather worsens or obstacles appear, it generates a new flight route and sends it to the drone terminal.

[0769] Step 5: Recognize and respond to emotional states

[0770] Input: Data on the user's facial expressions and tone of voice.

[0771] Processing: The server uses the emotion engine to analyze the user's emotional state. The emotion engine analyzes the received facial expressions and tone of voice and processes the data to determine the user's emotion.

[0772] Output: Notification content and delivery information are generated according to the user's emotional state.

[0773] Specific operation: The server uses an emotion engine to perform emotion analysis, such as "the user is irritated," and changes the content of the notification appropriately depending on the state.

[0774] Step 6: Accumulating and Learning Emotional Data

[0775] Input: User's historical emotion data.

[0776] Processing: The server stores past emotional data in the emotion engine database. Emotional patterns are learned based on this data.

[0777] Output: More personalized user-responsive data will be generated in the future.

[0778] Specific operation: The server accumulates emotional data, and the emotion engine uses this data to learn the user's emotional patterns, resulting in more precise and personalized responses to the user from the next time onwards.

[0779] (Application example 2)

[0780] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0781] Conventional drone delivery systems lack the flexibility to respond to the user's emotional state, and improvements to the user experience are desired. Furthermore, there is a lack of means to alleviate the user's anxiety and frustration during delivery, making it a challenge to improve user satisfaction.

[0782] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0783] In this invention, the server includes means for receiving flight information transmitted from the drone terminal and storing it on the platform, means for training an air route generation AI based on the stored flight information, means for generating an optimal air route using the air route generation AI in response to a delivery request and transmitting it to the drone terminal, means for monitoring flight information in real time while the drone terminal is flying and adjusting the air route as necessary, means for generating and updating an aerial map based on flight information and environmental data from the drone terminal, means for notifying the user of the delivery status, and means for combining an emotion engine that analyzes the user's emotional state and adjusts the content and timing of notifications based on that information. This enables flexible responses based on the user's emotional state, reducing the user's anxiety and irritation during delivery and improving user satisfaction.

[0784] A "drone terminal" is an unmanned aerial vehicle that flies through the air and delivers goods to designated locations.

[0785] "Flight information" refers to data such as the drone terminal's location, altitude, speed, and battery status.

[0786] The "platform" is a server or cloud-based data management system that receives and stores drone flight information.

[0787] "Air Route Generation AI" is an artificial intelligence algorithm for optimizing drone flight paths.

[0788] An "emotion engine" is an engine that has the function of analyzing emotions from the user's facial expressions, tone of voice, interaction patterns, etc., and adjusting the system's behavior based on that information.

[0789] "Environmental data" refers to data such as weather information and obstacle information that affect drone flight.

[0790] An "aviation map" is map data that includes route information and obstacle information for drone flight.

[0791] The present invention provides a system that flexibly responds to the emotional state of a user by combining an emotion engine with a drone-based delivery system. Detailed embodiments of the present invention will be described below.

[0792] The system mainly consists of the following components:

[0793] 1. Drone terminal

[0794] The drone terminal is an unmanned aerial vehicle that flies through the air and delivers goods to designated locations. The drone terminal transmits flight information such as location, altitude, speed, and battery status to a server in real time.

[0795] 2. Platform

[0796] The platform consists of a server and a cloud-based data management system that receives and stores flight information sent from drone terminals. Based on this information, the air route generation AI and emotion engine operate.

[0797] 3. Air route generation AI

[0798] Route generation AI is an artificial intelligence algorithm for optimizing drone flight paths. It is trained based on flight information and environmental data, such as the date and time, weather, and past bird strike information, to generate optimal routes.

[0799] 4. Emotion Engine

[0800] The emotion engine analyzes the user's facial expressions, tone of voice, and interaction patterns to determine their emotions, and adjusts the system's behavior based on this information. This engine is built using deep learning frameworks such as TensorFlow and Keras. It adjusts the content and timing of notifications to reduce user frustration and anxiety.

[0801] 5. Aerial map generation and update

[0802] The server generates and updates an aerial map based on flight information and environmental data from the drone terminals, and if new obstacles are detected, the information is also reflected on the map.

[0803] 6. Notification System

[0804] The server notifies the user of the delivery status. The content and timing of the notification are adjusted based on the user's emotional state analyzed by the emotion engine. This reduces the user's anxiety and frustration and improves satisfaction.

[0805] Hardware and Software Configuration

[0806] Server: Cloud-based data management system (Amazon Web Services, Microsoft Azure, etc.)

[0807] Drone terminal: Small unmanned aerial vehicle

[0808] Emotion recognition system: TensorFlow, Keras

[0809] Real-time communication: REST API and WebSocket

[0810] Specific examples

[0811] A user requests a food delivery drone using a smartphone app. The user's facial expressions and tone of voice are captured by a camera and microphone, and the data is analyzed by an emotion engine. Based on the analysis results, the content and timing of delivery notifications are adjusted. If the user shows signs of impatience, a message such as "Sorry for the wait. Delivery status: On its way." is displayed.

[0812] Prompt Sentence Examples

[0813] "Generate optimal delivery notification content based on the user's emotional state. Also, adjust the notification content in real time according to the delivery status."

[0814] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0815] Step 1:

[0816] A user requests drone delivery from a smartphone app. At this time, the app receives the user's delivery request data (shipping origin, delivery destination, order details, etc.) as input and sends it to the server. The server receives this data and stores it on the platform.

[0817] Step 2:

[0818] The server trains the route generation AI based on the accumulated flight information. It uses historical flight information and environmental data as input, and updates the model to generate optimal routes as output. This model is trained using an AI algorithm (e.g., TensorFlow or Keras).

[0819] Step 3:

[0820] The server generates the optimal flight route based on the user's delivery request using an AI route generation model. The inputs are the delivery origin, delivery destination, and real-time environmental data (weather, obstacle information, etc.), and the output is the optimal flight path. This route data is sent to the drone terminal.

[0821] Step 4:

[0822] The drone terminal will then begin delivery based on the received flight route. During flight, the drone terminal will continue to send real-time flight information, such as location, altitude, speed, and battery status, to the server. The server receives this data and stores it on the platform.

[0823] Step 5:

[0824] The server monitors the real-time flight information of the drone during flight and adjusts the flight path as needed. For example, if a new obstacle is detected, the flight path generation AI is reactivated, recalculating the optimal route and sending it to the drone. The obstacle information is used as input, and the updated flight path is obtained as output.

[0825] Step 6:

[0826] Based on the flight information and environmental data sent from the drone terminal, the server generates and updates an aerial map. It uses real-time flight information and environmental data as input and creates an updated aerial map as output. This map is used to generate future flight paths.

[0827] Step 7:

[0828] Analyze the user's emotional state. Emotional data such as facial expressions and tone of voice obtained from the user's smartphone app is input into the emotion engine. The emotion engine analyzes this data to determine the user's emotional state. For example, a Keras model can be used to analyze facial expression data and output emotional states such as irritation or anxiety.

[0829] Step 8:

[0830] The server adjusts the content and timing of notifications to the user based on the emotional state analyzed by the emotion engine. The emotional state and delivery status are used as inputs, and an appropriate notification message is generated as output to be sent to the user. For example, if the user is annoyed, a message detailing the progress of the delivery along with an apology is generated.

[0831] Step 9:

[0832] Finally, the server sends the generated notification message to the user's smartphone app, allowing the user to receive timely notifications about the delivery status, reducing user anxiety and frustration and improving overall satisfaction.

[0833] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0834] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0835] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0836] [Third embodiment]

[0837] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0838] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0839] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0840] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0841] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0842] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0843] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0844] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0845] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0846] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0847] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0848] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0849] The present invention relates to a system for realizing efficient and safe drone delivery, and detailed embodiments thereof will be described below.

[0850] The server has the function of receiving flight information sent from the drone terminal. During flight, the drone terminal sends flight information such as location, altitude, speed, and battery status to the server in real time. The server accumulates and stores this information on its own platform. This data is used to train the air route generation AI (described later) and to generate air routes in real time.

[0851] The server then uses the accumulated flight information to train the route generation AI, which learns relevant data such as past flight patterns, weather information, date and time, and past bird strikes to improve its ability to generate optimal flight paths, thereby providing more efficient and safer routes for future delivery requests.

[0852] When a user requests a drone delivery, the server accepts the request and generates the optimal route using route generation AI. The route generation AI considers parameters such as current weather, the specified delivery destination, and delivery time to select the most efficient route from multiple flight paths. The generated route information is sent to the drone terminal in real time.

[0853] During flight, the drone terminals periodically send flight information to the server. The server uses this information to monitor drone operations in real time and adjust air routes as necessary. The server also integrates the flight information and environmental data received from the drone terminals to generate and update a highly accurate aerial map. For example, newly discovered obstacles or sudden weather changes can be immediately reflected in the aerial map, ensuring that all drone terminals are flying based on the most up-to-date information.

[0854] In addition, the server provides users with real-time delivery status notifications, allowing them to receive information about the location of their delivery, estimated arrival time, and other important notifications via a smartphone app.

[0855] As a concrete example, when a user requests "drone delivery from their current address to their workplace" through a smartphone app, the request is sent to a server. The server uses route generation AI to generate the optimal flight route from their current address to their workplace and sends it to the drone terminal. The drone terminal then flies along this route and proceeds with the delivery. If an obstacle is detected during flight, the drone terminal sends that information to the server, which then updates the aerial map. The user can check the drone's current location and estimated arrival time in real time through the smartphone app.

[0856] In this way, the present invention enables efficient and safe drone delivery, while also improving user convenience.

[0857] The processing flow will be explained below.

[0858] Step 1:

[0859] Drone startup and initial setup

[0860] The device completes the initial setup and connects to the system, sending its identification information, initial position, altitude, and battery status to the server.

[0861] Step 2:

[0862] Real-time flight information transmission

[0863] During flight, the device periodically transmits flight information such as location, altitude, speed, and battery status to the server in real time.

[0864] Step 3:

[0865] Data accumulation

[0866] The server automatically stores the received flight information on its own platform, which stores the data necessary for subsequent analysis and route generation.

[0867] Step 4:

[0868] Air route generation AI training

[0869] The server trains the route generation AI based on accumulated flight information and data such as date and time, weather, and past bird strikes, allowing the route generation AI to learn and improve its algorithm for generating efficient and safe routes.

[0870] Step 5:

[0871] Accepting a delivery request

[0872] The user enters a delivery request through a smartphone app and sends it to the server.

[0873] The server analyzes the request content (delivery destination, desired time, etc.).

[0874] Step 6:

[0875] Air Route Generation

[0876] The server uses route generation AI to generate the optimal route based on the request.

[0877] The server takes the generated air route information and takes into account relevant constraints (weather, obstacles, etc.).

[0878] Step 7:

[0879] Sending air route information

[0880] The server transmits the generated air route information to the terminal (drone).

[0881] The terminal loads the received air route information into memory and prepares for flight.

[0882] Step 8:

[0883] Start of flight

[0884] The terminal starts the flight based on the received air route information.

[0885] The terminal periodically transmits flight information to the server.

[0886] Step 9:

[0887] Real-time monitoring and adjustment

[0888] The server monitors real-time data from the terminals during the flight.

[0889] The server analyzes flight information and environmental data and adjusts air routes as needed.

[0890] Step 10:

[0891] Generate and update aerial maps

[0892] The server generates and updates an aerial map based on flight information and environmental data from the drone.

[0893] Newly discovered obstacles and weather changes are instantly updated on the aerial map.

[0894] Step 11:

[0895] User Notifications

[0896] The server notifies the user of the delivery status in real time.

[0897] Users can use a smartphone app to check the current location and estimated arrival time of their delivery.

[0898] Step 12:

[0899] Delivery completed

[0900] The terminal arrives at the delivery destination and delivers the package safely.

[0901] The terminal notifies the server that the delivery is complete.

[0902] The server notifies the user that delivery has been completed.

[0903] Example 1

[0904] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0905] Drone delivery presents many challenges, including optimizing flight routes for efficiency, ensuring safety during flight, and understanding delivery status in real time. In particular, drone operation can be disrupted if the flight route is inappropriate or if unexpected obstacles appear during flight. Furthermore, when multiple drones are operating simultaneously, it is important to ensure they do not interfere with each other. Another problem is the lack of means to take rapid emergency measures when trouble occurs.

[0906] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0907] In this invention, the server includes means for receiving flight information transmitted from the drone terminal and storing it on the platform, means for training an air route generation AI based on the stored flight information, means for generating an optimal air route using the air route generation AI in response to a delivery request and transmitting it to the drone terminal, means for monitoring flight information in real time while the drone terminal is flying and adjusting the air route as necessary, means for generating and updating an aerial map based on flight information and environmental data from the drone terminal, means for notifying the user of the delivery status, means for taking emergency measures in the event of a problem based on flight information collected by the drone terminal, and means for multiple drone terminals to cooperate and share flight information to improve overall flight efficiency. This enables efficient and safe drone delivery and improves user convenience.

[0908] "Flight information" is a general term for a wide variety of data related to operation, such as location information, altitude, speed, and battery status obtained from drone terminals.

[0909] "Platform" refers to an integrated environment of servers and related software for receiving, storing, and managing flight information sent from drone terminals.

[0910] "Air Route Generation AI" is an AI (artificial intelligence) model that generates optimal flight routes based on accumulated flight information, weather data, past flight patterns, etc.

[0911] "Delivery Request" means a request for drone delivery made by a User through a smartphone app or other interface.

[0912] The "optimal route" refers to the flight route with the highest flight efficiency, selected based on the conditions specified by the route generation AI (delivery destination, weather, time of day, etc.).

[0913] "Monitoring" refers to the process of monitoring flight information transmitted by drone terminals in real time and adjusting operational status as necessary.

[0914] "Environmental data" is a general term for data related to the flight environment, such as weather information and the presence of obstacles, acquired by a drone terminal during flight.

[0915] "Aerial Map" refers to a high-precision aerial map generated by integrating flight information and environmental data.

[0916] "Emergency Plan" means the procedures and protocols for responding quickly if a problem occurs during flight.

[0917] "Improving flight efficiency" refers to the process of multiple drone devices working together and sharing flight information to minimize overall flight time and energy consumption.

[0918] The present invention relates to a system for realizing efficient and safe drone delivery, and detailed embodiments thereof will be described below.

[0919] Hardware and software used

[0920] 1. Server

[0921] Server: High-performance server (e.g. Dell PowerEdge R740)

[0922] Server software: Python, Django, MySQL

[0923] Real-time data processing: Apache Kafka

[0924] 2. Drone terminal

[0925] Drone hardware: A typical high-performance drone (e.g., DJI Mavic 2)

[0926] Communication module: Standard internet communication module

[0927] Drone control application: Drone control SDK (e.g. DJI SDK)

[0928] 3. Air route generation AI

[0929] Deep Learning Frameworks: TensorFlow, PyTorch

[0930] 4. GIS Software

[0931] Aerial Map Generation Tool: Esri ArcGIS

[0932] Program processing

[0933] The server has the function of receiving flight information sent from the drone terminal. During flight, the drone terminal sends flight information such as location, altitude, speed, and battery status to the server in real time. The server stores this received information in a database. This database is operated using MySQL.

[0934] The server trains the route generation AI based on the accumulated flight information. The route generation AI builds an optimization model using Python, TensorFlow, and PyTorch to learn relevant data such as past flight patterns, weather information, date and time, and past bird strikes. This allows it to generate efficient and safe flight routes for future delivery requests.

[0935] When a user requests a drone delivery, the server accepts the request and generates the optimal air route using an air route generation AI. The request includes parameters such as the specified delivery destination and delivery time, and the server selects the optimal route taking this information into account. The generated air route information is then sent to the drone terminal.

[0936] During flight, the drones periodically send flight information to the server. The server monitors this information and adjusts flight routes as necessary. The server also integrates the flight information and environmental data received from the drones to generate and update a highly accurate aerial map. For example, newly discovered obstacles or sudden weather changes can be instantly reflected in the aerial map, ensuring that all drones fly based on the most up-to-date information.

[0937] Additionally, the server provides users with real-time delivery status notifications, allowing them to receive information about the location of their shipments, estimated arrival times, and other important notifications via a smartphone app.

[0938] Specific examples

[0939] When a user requests "drone delivery from their current address to their workplace" via a smartphone app, the request is sent to the server.

[0940] Example prompt sentence:

[0941] I would like to request drone delivery from my current address to my workplace. My current address is in Shibuya-ku, Tokyo, and my workplace is in Minato-ku, Tokyo. I would like the delivery time to be between 10:00 AM and 2:00 PM.

[0942] The server uses route generation AI to generate the optimal flight route from the current address to the workplace and sends it to the drone terminal. The drone terminal then flies along this route and proceeds with the delivery. If an obstacle is detected during flight, the drone terminal sends that information to the server, which then updates the aerial map. The user can check the drone's current location and estimated arrival time in real time via a smartphone app.

[0943] In this way, the present invention enables efficient and safe drone delivery, improving user convenience.

[0944] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0945] Step 1: Gather flight information

[0946] Subject: Device

[0947] Input: Location, altitude, speed, and battery status obtained from the drone's built-in sensors and GPS module.

[0948] How it works: The drone terminal reads the data obtained from these sensors and the GPS module at regular intervals.

[0949] Output: Retrieved flight information data.

[0950] Specific operation: The drone terminal obtains the current latitude and longitude, altitude (meters), speed (kilometers per hour), and battery status (percentage) and generates flight information data.

[0951] Step 2: Send flight information

[0952] Subject: Device

[0953] Input: Collected flight information.

[0954] Operation: The terminal sends the collected flight information to the server via the Internet communication module.

[0955] Output: Flight information sent to the server.

[0956] Specific operation: The drone terminal sends the latest flight information (location, altitude, speed, battery status) to the server as an HTTP request at regular intervals.

[0957] Step 3: Receiving and storing flight information

[0958] Subject: Server

[0959] Input: Flight information sent from the device.

[0960] Operation: The server stores the received flight information in a database.

[0961] Output: Flight information stored in a database.

[0962] What it does: The server receives the HTTP request, parses it, and stores flight information data in a MySQL database, including latitude, longitude, altitude, speed, battery status, and timestamps.

[0963] Step 4: Training the route generation AI

[0964] Subject: Server

[0965] Input: Past flight information, weather information, date and time, and past bird strike information stored in the database.

[0966] How it works: The server uses these data to train the route generation AI.

[0967] Output: A trained air route generation AI model.

[0968] How it works: The server uses TensorFlow and PyTorch to train a neural network based on the accumulated data, allowing the model to learn patterns to generate optimal flight paths.

[0969] Step 5: User Requests Delivery

[0970] Subject: User

[0971] Input: Delivery request from smartphone app (e.g. current address, delivery destination, delivery time).

[0972] How it works: The user enters a delivery request, which the app sends to the server.

[0973] Output: Delivery request information sent to the server.

[0974] Specific operation: The user uses the app to enter their current address (Shibuya-ku, Tokyo), workplace (Minato-ku, Tokyo), and desired delivery time (10:00 AM to 2:00 PM), and submits a delivery request.

[0975] Step 6: Generate optimal routes

[0976] Subject: Server

[0977] Input: A delivery request from the user and current weather information.

[0978] How it works: Uses route generation AI to generate optimal flight paths.

[0979] Output: Generated optimal air route information.

[0980] How it works: The server uses an AI model to input delivery request parameters (e.g., delivery destination, delivery time, weather) and selects the most efficient route from multiple candidate routes.

[0981] Step 7: Sending flight information to the terminal

[0982] Subject: Server

[0983] Input: Generated air route information.

[0984] Operation: The server transmits the generated air route information to the drone terminal in real time.

[0985] Output: Air route information received by the drone terminal.

[0986] Specific operation: The server sends the route information (starting point, intermediate points, and destination) selected by the route generation AI to the drone terminal as an HTTP request.

[0987] Step 8: In-flight monitoring

[0988] Subject: Server

[0989] Input: Flight information periodically sent from the drone terminal during flight.

[0990] How it works: The server monitors the drone in real time based on the flight information it receives and adjusts its flight path as needed.

[0991] Output: Adjusted air route information (if required).

[0992] Specific operation: The server receives the latest information from the drone terminal during flight, uses AI to recalculate the route based on the current situation, such as obstacles and weather changes, and sends the adjusted route information to the terminal in real time.

[0993] Step 9: Generate and update high-definition aerial maps

[0994] Subject: Server

[0995] Input: Flight information and environmental data from the drone device.

[0996] How it works: The server aggregates these data to generate and update aerial maps.

[0997] Output: Up-to-date aeronautical maps.

[0998] What happens: The server uses GIS software (e.g., Esri ArcGIS) to update the aerial map based on the received environmental data (e.g., newly discovered obstacles).

[0999] Step 10: Real-time delivery status notification

[1000] Subject: Server

[1001] Input: Real-time location information of the drone in flight.

[1002] Action: The server notifies the user of the delivery status.

[1003] Output: The user is notified of the location and estimated time of arrival of the delivery.

[1004] Specific operation: The server notifies the user of the drone's current location and estimated arrival time in real time via a smartphone app.

[1005] (Application example 1)

[1006] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1007] Conventional drone delivery systems have difficulty efficiently collecting and managing flight information, posing challenges in optimizing delivery routes and ensuring delivery safety. Furthermore, users have limited means of tracking delivery status in real time, making it difficult to provide accurate information on delivery arrival times. Technological solutions to address these challenges were needed.

[1008] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1009] In this invention, the server includes means for receiving flight information sent from the drone terminal and storing it on the platform, means for training an air route generation AI based on the stored flight information, means for generating an optimal air route using the air route generation AI in response to a delivery request and sending it to the drone terminal, means for monitoring flight information in real time while the drone terminal is flying and adjusting the air route as needed, means for generating and updating an aerial map based on flight information and environmental data from the drone terminal, means for notifying users of delivery status, and means for users to request drone delivery through a smartphone app and receive real-time tracking of deliveries and arrival time predictions. This enables efficient collection and management of flight information, optimizes delivery routes and ensures safety, and enables users to track delivery status in real time and receive accurate arrival predictions.

[1010] A "drone terminal" is an unmanned aerial vehicle that flies and transports objects or goods autonomously.

[1011] "Flight information" refers to data such as location, altitude, speed, and battery status acquired by a drone during flight.

[1012] A "platform" is a system infrastructure for accumulating, managing, and analyzing data.

[1013] "Air route generation AI" is an artificial intelligence that learns past flight patterns, weather information, date and time, obstacle information, etc., and autonomously generates the optimal flight route.

[1014] A "delivery request" is an instruction from a user requesting delivery by drone.

[1015] "Monitoring flight information in real time" refers to constantly monitoring the flight status of a drone while it is in flight.

[1016] "Adjusting the flight route" means instructing a drone in flight to take a new flight path as needed.

[1017] "Environmental data" refers to data related to the external environment, such as weather information and obstacle information obtained by a drone during flight.

[1018] An "aviation map" is map data that includes flight routes, obstacles, weather information, and so on.

[1019] "User" refers to a person who uses the drone delivery service.

[1020] A "smartphone app" is application software that runs on a smartphone and allows users to request drone delivery and check the delivery status.

[1021] "Real-time tracking" refers to instantly checking the location and status information during drone delivery.

[1022] "Arrival time prediction" refers to predicting and notifying the time it will take for a delivery to reach its destination.

[1023] The system of the present invention integrates multiple pieces of hardware and software to operate in order to realize efficient and safe drone delivery. Specific embodiments of the system will be described below.

[1024] Hardware:

[1025] Drone terminal: An unmanned aerial vehicle equipped with GPS, cameras, and sensors (for obstacle detection). This hardware is used to collect flight information and monitor flight status in real time.

[1026] Server: A computer that provides the computational resources for accumulating and analyzing flight information and running route generation AI.

[1027] User's smartphone: This is the device used to request drone delivery and perform real-time tracking.

[1028] software:

[1029] Route Generation AI: AI models built using TensorFlow and PyTorch perform flight path optimization, taking into account data such as historical flight patterns, weather data, time of day, and obstacle information.

[1030] Database: Flight information and user information are managed using a data management system such as PostgreSQL.

[1031] Smartphone application: An application that allows users to request drone delivery and track the delivery status in real time. React Native and Flutter are often used as front-end technologies.

[1032] Processing Description:

[1033] The server receives flight information sent from the drone terminal and stores it on the platform. The stored flight information is used to train the route generation AI. The route generation AI learns from past flight patterns, weather information, date and time, and obstacle information to improve its ability to generate optimal flight routes. When a user makes a delivery request through a smartphone app, the server uses the route generation AI to generate the optimal route and sends that information to the drone terminal.

[1034] More concrete examples:

[1035] For example, when a user requests food delivery from a restaurant to their home using a smartphone app, the request is sent to a server. The server uses route generation AI to generate the optimal flight path from the restaurant to their home and sends it to the drone terminal. The drone terminal flies along this path and proceeds with the delivery. If an obstacle is detected during flight, the drone terminal sends that information to the server, which then updates the aerial map. The user can check the drone's current location and estimated arrival time in real time using the smartphone app.

[1036] Example prompt sentence:

[1037] "Request a drone to deliver food to a specified address. Generate the safest and quickest route based on current weather and historical flight data."

[1038] In this way, the present invention can realize efficient and safe drone delivery, improving user convenience.

[1039] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1040] Step 1:

[1041] The server receives flight information (location, altitude, speed, battery status, etc.) sent from the drone terminal and accumulates the data on the platform. The server receives this input flight information and stores it appropriately in the database.

[1042] Step 2:

[1043] The server trains the route generation AI based on the accumulated flight information. Specifically, it receives data such as past flight patterns, weather information, date and time, and obstacle information as input, and updates the AI ​​model using TensorFlow and PyTorch to learn how to generate optimal flight routes.

[1044] Step 3:

[1045] The user makes a delivery request through a smartphone app. The user inputs the pickup and drop-off locations, and the request data is sent to the server.

[1046] Step 4:

[1047] The server receives delivery requests from users and generates the optimal flight route using route generation AI. The AI ​​receives the request data and current weather information as input, calculates the optimal route, and generates the route information.

[1048] Step 5:

[1049] The server then sends the generated optimal air route information to the drone terminal, which then begins flying along this route.

[1050] Step 6:

[1051] During flight, the drone terminal periodically transmits flight information to the server, which receives real-time flight information as input and monitors it, adjusting the flight path as needed.

[1052] Step 7:

[1053] If a drone detects an obstacle during flight, it sends that information to the server, which then updates the aerial map and recalculates the drone's route. The input is the detected obstacle information, and the output is the updated flight route.

[1054] Step 8:

[1055] The server notifies the user of the delivery status, including important information such as the delivery's location and estimated arrival time, and is communicated to the user in real time via a smartphone app.

[1056] Step 9:

[1057] Once the delivery is complete, the drone terminal transmits the final flight information to the server, which stores this new data in the platform and uses it for the next learning cycle, thereby improving the overall performance of the system.

[1058] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1059] The present invention relates to a system that recognizes a user's emotions and responds according to their state by combining an emotion engine with a drone-based delivery system. Detailed embodiments of the present invention will be described below.

[1060] The server has the function of receiving and storing flight information sent from the drone terminal. The drone terminal sends flight information such as location, altitude, speed, and battery status to the server in real time during flight. The server also trains the air route generation AI based on the accumulated flight information and improves the algorithm for generating efficient and safe air routes.

[1061] When a user requests drone delivery via a smartphone app, the server accepts the request and generates the optimal route using route generation AI. The generated route information is sent to the drone terminal in real time. During flight, the drone terminal periodically sends flight information to the server, which monitors the flight in real time and adjusts the route as necessary. The server also generates and updates an aerial map based on the flight information and environmental data.

[1062] A distinctive feature of this invention is the incorporation of an emotion engine. The emotion engine recognizes the user's emotional state and adjusts the system's behavior based on that information. Specifically, it analyzes the user's facial expressions, tone of voice, and interaction patterns to determine the user's emotions. For example, if the user is irritated, the server adjusts the content and timing of notifications and provides information to reduce the user's stress.

[1063] As a concrete example, when a user requests a drone delivery from their current address to their workplace, the information is sent to the server, and an optimal flight route is generated using air route generation AI. The generated air route information is then sent to the drone terminal, and the drone begins the delivery. Meanwhile, the server regularly monitors the user's emotional state through an emotion engine. If the user expresses irritation about a delivery delay, the server will use that information to adjust the content and timing of notifications, provide explanations to the user, and provide detailed progress reports to alleviate the user's frustration.

[1064] Furthermore, the emotion engine accumulates past emotion data and uses it for future responses, allowing it to learn the user's emotional patterns and provide more personalized responses.

[1065] In this way, by combining an emotion engine with a drone delivery system, the present invention can achieve efficient and safe delivery while improving the user experience.

[1066] The processing flow will be explained below.

[1067] Step 1:

[1068] Drone startup and initial setup

[1069] The device completes the initial setup and connects to the system, sending its identification information, initial position, altitude, and battery status to the server.

[1070] Step 2:

[1071] Real-time flight information transmission

[1072] During flight, the device periodically transmits flight information such as location, altitude, speed, and battery status to the server in real time.

[1073] Step 3:

[1074] Data accumulation

[1075] The server automatically stores the received flight information on a proprietary platform, which stores training data and data required for real-time route generation.

[1076] Step 4:

[1077] Air route generation AI training

[1078] The server trains the route generation AI based on accumulated flight information and data such as date and time, weather, and past bird strikes. The route generation AI learns and improves its algorithm to generate efficient and safe routes.

[1079] Step 5:

[1080] Accepting a delivery request

[1081] The user enters a delivery request through a smartphone app and sends it to the server.

[1082] The server analyzes the request content (delivery destination, desired time, etc.).

[1083] Step 6:

[1084] Air Route Generation

[1085] The server uses route generation AI to generate the optimal route based on the request.

[1086] The server takes the generated air route information and takes into account relevant constraints (weather, obstacles, etc.).

[1087] Step 7:

[1088] Sending air route information

[1089] The server transmits the generated air route information to the terminal (drone).

[1090] The terminal loads the received air route information into memory and prepares for flight.

[1091] Step 8:

[1092] Start of flight

[1093] The terminal starts the flight based on the received air route information.

[1094] The terminal periodically transmits flight information to the server.

[1095] Step 9:

[1096] Real-time monitoring and adjustment

[1097] The server monitors real-time data from the terminals during the flight.

[1098] The server analyzes flight information and environmental data and adjusts air routes as needed.

[1099] Step 10:

[1100] Generate and update aerial maps

[1101] The server generates and updates an aerial map based on flight information and environmental data from the drone.

[1102] Newly discovered obstacles and weather changes are instantly updated on the aerial map.

[1103] Step 11:

[1104] User Notifications

[1105] The server notifies the user of the delivery status in real time.

[1106] Users can use a smartphone app to check the current location and estimated arrival time of their delivery.

[1107] Step 12:

[1108] Emotion engine activation and emotion recognition

[1109] The server monitors the user's facial expressions, tone of voice, and interaction patterns through a smartphone or other interface.

[1110] The server runs an emotion engine that recognizes whether the user is feeling frustrated, happy, confused, etc.

[1111] Step 13:

[1112] Emotion-based notification adjustment

[1113] If the user expresses negative emotions such as irritation, the server adjusts the content and timing of the notification, for example adding detailed delivery progress information or an apology message.

[1114] If the user is satisfied, the server continues with the normal notification method.

[1115] Step 14:

[1116] Accumulating and learning emotional data

[1117] The server stores the user's emotional responses and uses them as data for future notifications and service improvements, allowing it to learn the user's emotional patterns and provide more personalized responses.

[1118] Step 15:

[1119] Delivery completed

[1120] The terminal arrives at the delivery destination and delivers the package safely.

[1121] The terminal notifies the server that the delivery is complete.

[1122] The server notifies the user that delivery has been completed.

[1123] Example 2

[1124] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1125] While conventional drone delivery systems have achieved a certain level of success in terms of delivery efficiency and safety, they lack consideration for the user's emotional state, limiting their ability to improve user experience. Furthermore, insufficient user response in the event of delivery delays or unexpected failures can result in increased user dissatisfaction. This has led to a decline in user satisfaction and left room for service improvement.

[1126] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1127] In this invention, the server includes means for receiving flight information transmitted from the drone terminal and storing it on the server, means for training an air route generation algorithm based on the stored flight information, means for generating an optimal air route using the air route generation algorithm in response to a delivery request and transmitting it to the drone terminal, means for monitoring the flight information in real time while the drone terminal is flying and adjusting the air route as necessary, means for generating and updating an air map based on the flight information and environmental data from the drone terminal, means including an emotion engine that recognizes user emotions and adjusts system behavior based on the user emotions, and means for notifying the user of delivery status and adjusting the content and timing of notification based on the user's emotional state. This enables flexible responses based on user emotions and efficient and safe delivery.

[1128] A "drone terminal" is an unmanned aerial vehicle that flies remotely or autonomously and delivers designated items.

[1129] "Flight information" refers to various flight-related data such as the drone terminal's location, altitude, speed, and battery status.

[1130] A "server" is a remote computer system that stores, processes, and analyzes data, and is a device that receives flight information from drone terminals and performs various functions based on that information.

[1131] An "air route generation algorithm" is an algorithm that uses artificial intelligence technology to calculate flight routes, and is designed to generate efficient and safe flight routes.

[1132] An "emotion engine" is a technology that analyzes a user's emotional state and adjusts the system's behavior based on that information. It is an engine that determines emotions by analyzing facial expressions and tone of voice.

[1133] A "delivery request" refers to a request made by a user via a smartphone app or other means to have a specific item delivered to a specified location.

[1134] An "aviation map" is a map generated based on flight information and environmental data, and provides information on the airspace, including the flight path of the drone terminal.

[1135] "Environmental Data" refers to data about external environmental conditions that affect flight, such as weather, temperature, and wind speed.

[1136] The present invention provides a system that recognizes the emotions of a user and responds according to that user's state by incorporating an emotion engine into a drone delivery system. Detailed embodiments of the present invention will be described below.

[1137] Hardware and software configuration

[1138] Drone terminal

[1139] Sensors: Equipped with a GPS sensor, gyro sensor, and battery monitoring system, it measures location, altitude, speed, and battery status in real time.

[1140] Communication module: The mobile communication module is used to send and receive data to and from the server in real time.

[1141] Control unit: Equipped with a control unit for flight control, it flies autonomously along a designated air route.

[1142] server

[1143] Database: The system has a database that stores and manages flight information received from drones.

[1144] Route Generation AI: Equipped with an AI model that trains route generation algorithms based on accumulated flight information to generate efficient and safe routes.

[1145] Emotion engine: Contains software that analyzes facial expressions and tone of voice to determine the user's emotional state.

[1146] System Operation

[1147] The server receives flight information sent from the drone terminal in real time and stores that information in a database. The stored data is used as training data for the air route generation AI. When a user sends a delivery request through a smartphone app, the server uses the air route generation AI to calculate the optimal flight path and sends that information to the drone terminal. The drone then begins flying autonomously along the specified air route.

[1148] Specific examples of processing

[1149] When a user requests a drone delivery from their current address to their workplace, the information is sent to the server. The server uses route generation AI to generate the optimal flight path and sends that route information to the drone terminal. The drone then begins the delivery and periodically sends flight information such as its location and battery status to the server during the flight.

[1150] Meanwhile, the server constantly monitors the user's emotional state through an emotion engine. For example, if the user expresses irritation about a delivery delay, the server can adjust the content and timing of notifications based on that information, and provide the user with detailed explanations and progress reports to alleviate the user's frustration.

[1151] Prompt Sentence Examples

[1152] "Please create the following drone delivery system model. When a user requests delivery via a smartphone app, the server accepts the request and generates the optimal flight route using route generation AI. During flight, the drone continuously sends flight information to the server, which monitors it in real time. It also uses an emotion engine to recognize the user's emotional state and adjusts the system's behavior based on that information."

[1153] In this way, the present invention combines an emotion engine with a drone delivery system to achieve efficient and safe delivery as well as an improved user experience.

[1154] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1155] Step 1: Request acceptance

[1156] Input: A user submits a delivery request from a smartphone app, including their current address, destination, and delivery details.

[1157] Processing: The user's request is sent to the server through the communication module.

[1158] Output: The server records the received request in a database and acknowledges receipt of the request.

[1159] Specific operation: The server analyzes the request data and stores the order ID, delivery item, origin and destination information in a database.

[1160] Step 2: Generate air routes

[1161] Input: The server receives the user's delivery request information.

[1162] Processing: The server runs a route generation algorithm to calculate an efficient and safe flight path. It uses a generative AI model to generate the optimal route based on the request data and accumulated historical flight information.

[1163] Output: Optimal flight path, estimated flight time, and predicted battery consumption data are generated.

[1164] Specific operation: The server sends the generated flight path to the drone terminal and starts the delivery plan.

[1165] Step 3: Send flight information

[1166] Input: The drone device receives location information, altitude, speed, and battery status collected from sensors during flight.

[1167] Processing: The drone terminal sends the collected flight information to the server via the communication module.

[1168] Output: The server monitors drone flight information in real time and stores it in a database.

[1169] Specific operation: The drone terminal collects data using GPS sensors, gyro sensors, and battery monitoring systems, and transmits the data to a server in real time.

[1170] Step 4: Real-time monitoring and air route adjustments

[1171] Input: Real-time flight information sent from the drone terminal.

[1172] Processing: The server analyzes the flight information and issues an alert if an anomaly is detected.

[1173] Output: New air routes are generated as needed and sent to the drone terminal.

[1174] Specific operation: The server checks whether the flight information is normal, and if the weather worsens or obstacles appear, it generates a new flight route and sends it to the drone terminal.

[1175] Step 5: Recognize and respond to emotional states

[1176] Input: Data on the user's facial expressions and tone of voice.

[1177] Processing: The server uses the emotion engine to analyze the user's emotional state. The emotion engine analyzes the received facial expressions and tone of voice and processes the data to determine the user's emotion.

[1178] Output: Notification content and delivery information are generated according to the user's emotional state.

[1179] Specific operation: The server uses an emotion engine to perform emotion analysis, such as "the user is irritated," and changes the content of the notification appropriately depending on the state.

[1180] Step 6: Accumulating and Learning Emotional Data

[1181] Input: User's historical emotion data.

[1182] Processing: The server stores past emotional data in the emotion engine database. Emotional patterns are learned based on this data.

[1183] Output: More personalized user-responsive data will be generated in the future.

[1184] Specific operation: The server accumulates emotional data, and the emotion engine uses this data to learn the user's emotional patterns, resulting in more precise and personalized responses to the user from the next time onwards.

[1185] (Application example 2)

[1186] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1187] Conventional drone delivery systems lack the flexibility to respond to the user's emotional state, and improvements to the user experience are desired. Furthermore, there is a lack of means to alleviate the user's anxiety and frustration during delivery, making it a challenge to improve user satisfaction.

[1188] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1189] In this invention, the server includes means for receiving flight information transmitted from the drone terminal and storing it on the platform, means for training an air route generation AI based on the stored flight information, means for generating an optimal air route using the air route generation AI in response to a delivery request and transmitting it to the drone terminal, means for monitoring flight information in real time while the drone terminal is flying and adjusting the air route as necessary, means for generating and updating an aerial map based on flight information and environmental data from the drone terminal, means for notifying the user of the delivery status, and means for combining an emotion engine that analyzes the user's emotional state and adjusts the content and timing of notifications based on that information. This enables flexible responses based on the user's emotional state, reducing the user's anxiety and irritation during delivery and improving user satisfaction.

[1190] A "drone terminal" is an unmanned aerial vehicle that flies through the air and delivers goods to designated locations.

[1191] "Flight information" refers to data such as the drone terminal's location, altitude, speed, and battery status.

[1192] The "platform" is a server or cloud-based data management system that receives and stores drone flight information.

[1193] "Air Route Generation AI" is an artificial intelligence algorithm for optimizing drone flight paths.

[1194] An "emotion engine" is an engine that has the function of analyzing emotions from the user's facial expressions, tone of voice, interaction patterns, etc., and adjusting the system's behavior based on that information.

[1195] "Environmental data" refers to data such as weather information and obstacle information that affect drone flight.

[1196] An "aviation map" is map data that includes route information and obstacle information for drone flight.

[1197] The present invention provides a system that flexibly responds to the emotional state of a user by combining an emotion engine with a drone-based delivery system. Detailed embodiments of the present invention will be described below.

[1198] The system mainly consists of the following components:

[1199] 1. Drone terminal

[1200] The drone terminal is an unmanned aerial vehicle that flies through the air and delivers goods to designated locations. The drone terminal transmits flight information such as location, altitude, speed, and battery status to a server in real time.

[1201] 2. Platform

[1202] The platform consists of a server and a cloud-based data management system that receives and stores flight information sent from drone terminals. Based on this information, the air route generation AI and emotion engine operate.

[1203] 3. Air route generation AI

[1204] Route generation AI is an artificial intelligence algorithm for optimizing drone flight paths. It is trained based on flight information and environmental data, such as the date and time, weather, and past bird strike information, to generate optimal routes.

[1205] 4. Emotion Engine

[1206] The emotion engine analyzes the user's facial expressions, tone of voice, and interaction patterns to determine their emotions, and adjusts the system's behavior based on this information. This engine is built using deep learning frameworks such as TensorFlow and Keras. It adjusts the content and timing of notifications to reduce user frustration and anxiety.

[1207] 5. Aerial map generation and update

[1208] The server generates and updates an aerial map based on flight information and environmental data from the drone terminals, and if new obstacles are detected, the information is also reflected on the map.

[1209] 6. Notification System

[1210] The server notifies the user of the delivery status. The content and timing of the notification are adjusted based on the user's emotional state analyzed by the emotion engine. This reduces the user's anxiety and frustration and improves satisfaction.

[1211] Hardware and Software Configuration

[1212] Server: Cloud-based data management system (Amazon Web Services, Microsoft Azure, etc.)

[1213] Drone terminal: Small unmanned aerial vehicle

[1214] Emotion recognition system: TensorFlow, Keras

[1215] Real-time communication: REST API and WebSocket

[1216] Specific examples

[1217] A user requests a food delivery drone using a smartphone app. The user's facial expressions and tone of voice are captured by a camera and microphone, and the data is analyzed by an emotion engine. Based on the analysis results, the content and timing of delivery notifications are adjusted. If the user shows signs of impatience, a message such as "Sorry for the wait. Delivery status: On its way." is displayed.

[1218] Prompt Sentence Examples

[1219] "Generate optimal delivery notification content based on the user's emotional state. Also, adjust the notification content in real time according to the delivery status."

[1220] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1221] Step 1:

[1222] A user requests drone delivery from a smartphone app. At this time, the app receives the user's delivery request data (shipping origin, delivery destination, order details, etc.) as input and sends it to the server. The server receives this data and stores it on the platform.

[1223] Step 2:

[1224] The server trains the route generation AI based on the accumulated flight information. It uses historical flight information and environmental data as input, and updates the model to generate optimal routes as output. This model is trained using an AI algorithm (e.g., TensorFlow or Keras).

[1225] Step 3:

[1226] The server generates the optimal flight route based on the user's delivery request using an AI route generation model. The inputs are the delivery origin, delivery destination, and real-time environmental data (weather, obstacle information, etc.), and the output is the optimal flight path. This route data is sent to the drone terminal.

[1227] Step 4:

[1228] The drone terminal will then begin delivery based on the received flight route. During flight, the drone terminal will continue to send real-time flight information, such as location, altitude, speed, and battery status, to the server. The server receives this data and stores it on the platform.

[1229] Step 5:

[1230] The server monitors the real-time flight information of the drone during flight and adjusts the flight path as needed. For example, if a new obstacle is detected, the flight path generation AI is reactivated, recalculating the optimal route and sending it to the drone. The obstacle information is used as input, and the updated flight path is obtained as output.

[1231] Step 6:

[1232] Based on the flight information and environmental data sent from the drone terminal, the server generates and updates an aerial map. It uses real-time flight information and environmental data as input and creates an updated aerial map as output. This map is used to generate future flight paths.

[1233] Step 7:

[1234] Analyze the user's emotional state. Emotional data such as facial expressions and tone of voice obtained from the user's smartphone app is input into the emotion engine. The emotion engine analyzes this data to determine the user's emotional state. For example, a Keras model can be used to analyze facial expression data and output emotional states such as irritation or anxiety.

[1235] Step 8:

[1236] The server adjusts the content and timing of notifications to the user based on the emotional state analyzed by the emotion engine. The emotional state and delivery status are used as inputs, and an appropriate notification message is generated as output to be sent to the user. For example, if the user is annoyed, a message detailing the progress of the delivery along with an apology is generated.

[1237] Step 9:

[1238] Finally, the server sends the generated notification message to the user's smartphone app, allowing the user to receive timely notifications about the delivery status, reducing user anxiety and frustration and improving overall satisfaction.

[1239] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1240] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1241] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1242] [Fourth embodiment]

[1243] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1244] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1245] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1246] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1247] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1248] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1249] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1250] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1251] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1252] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1253] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1254] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1255] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1256] The present invention relates to a system for realizing efficient and safe drone delivery, and detailed embodiments thereof will be described below.

[1257] The server has the function of receiving flight information sent from the drone terminal. During flight, the drone terminal sends flight information such as location, altitude, speed, and battery status to the server in real time. The server accumulates and stores this information on its own platform. This data is used to train the air route generation AI (described later) and to generate air routes in real time.

[1258] The server then uses the accumulated flight information to train the route generation AI, which learns relevant data such as past flight patterns, weather information, date and time, and past bird strikes to improve its ability to generate optimal flight paths, thereby providing more efficient and safer routes for future delivery requests.

[1259] When a user requests a drone delivery, the server accepts the request and generates the optimal route using route generation AI. The route generation AI considers parameters such as current weather, the specified delivery destination, and delivery time to select the most efficient route from multiple flight paths. The generated route information is sent to the drone terminal in real time.

[1260] During flight, the drone terminals periodically send flight information to the server. The server uses this information to monitor drone operations in real time and adjust air routes as necessary. The server also integrates the flight information and environmental data received from the drone terminals to generate and update a highly accurate aerial map. For example, newly discovered obstacles or sudden weather changes can be immediately reflected in the aerial map, ensuring that all drone terminals are flying based on the most up-to-date information.

[1261] In addition, the server provides users with real-time delivery status notifications, allowing them to receive information about the location of their delivery, estimated arrival time, and other important notifications via a smartphone app.

[1262] As a concrete example, when a user requests "drone delivery from their current address to their workplace" through a smartphone app, the request is sent to a server. The server uses route generation AI to generate the optimal flight route from their current address to their workplace and sends it to the drone terminal. The drone terminal then flies along this route and proceeds with the delivery. If an obstacle is detected during flight, the drone terminal sends that information to the server, which then updates the aerial map. The user can check the drone's current location and estimated arrival time in real time through the smartphone app.

[1263] In this way, the present invention enables efficient and safe drone delivery, while also improving user convenience.

[1264] The processing flow will be explained below.

[1265] Step 1:

[1266] Drone startup and initial setup

[1267] The device completes the initial setup and connects to the system, sending its identification information, initial position, altitude, and battery status to the server.

[1268] Step 2:

[1269] Real-time flight information transmission

[1270] During flight, the device periodically transmits flight information such as location, altitude, speed, and battery status to the server in real time.

[1271] Step 3:

[1272] Data accumulation

[1273] The server automatically stores the received flight information on its own platform, which stores the data necessary for subsequent analysis and route generation.

[1274] Step 4:

[1275] Air route generation AI training

[1276] The server trains the route generation AI based on accumulated flight information and data such as date and time, weather, and past bird strikes, allowing the route generation AI to learn and improve its algorithm for generating efficient and safe routes.

[1277] Step 5:

[1278] Accepting a delivery request

[1279] The user enters a delivery request through a smartphone app and sends it to the server.

[1280] The server analyzes the request content (delivery destination, desired time, etc.).

[1281] Step 6:

[1282] Air Route Generation

[1283] The server uses route generation AI to generate the optimal route based on the request.

[1284] The server takes the generated air route information and takes into account relevant constraints (weather, obstacles, etc.).

[1285] Step 7:

[1286] Sending air route information

[1287] The server transmits the generated air route information to the terminal (drone).

[1288] The terminal loads the received air route information into memory and prepares for flight.

[1289] Step 8:

[1290] Start of flight

[1291] The terminal starts the flight based on the received air route information.

[1292] The terminal periodically transmits flight information to the server.

[1293] Step 9:

[1294] Real-time monitoring and adjustment

[1295] The server monitors real-time data from the terminals during the flight.

[1296] The server analyzes flight information and environmental data and adjusts air routes as needed.

[1297] Step 10:

[1298] Generate and update aerial maps

[1299] The server generates and updates an aerial map based on flight information and environmental data from the drone.

[1300] Newly discovered obstacles and weather changes are instantly updated on the aerial map.

[1301] Step 11:

[1302] User Notifications

[1303] The server notifies the user of the delivery status in real time.

[1304] Users can use a smartphone app to check the current location and estimated arrival time of their delivery.

[1305] Step 12:

[1306] Delivery completed

[1307] The terminal arrives at the delivery destination and delivers the package safely.

[1308] The terminal notifies the server that the delivery is complete.

[1309] The server notifies the user that delivery has been completed.

[1310] Example 1

[1311] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1312] Drone delivery presents many challenges, including optimizing flight routes for efficiency, ensuring safety during flight, and understanding delivery status in real time. In particular, drone operation can be disrupted if the flight route is inappropriate or if unexpected obstacles appear during flight. Furthermore, when multiple drones are operating simultaneously, it is important to ensure they do not interfere with each other. Another problem is the lack of means to take rapid emergency measures when trouble occurs.

[1313] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1314] In this invention, the server includes means for receiving flight information transmitted from the drone terminal and storing it on the platform, means for training an air route generation AI based on the stored flight information, means for generating an optimal air route using the air route generation AI in response to a delivery request and transmitting it to the drone terminal, means for monitoring flight information in real time while the drone terminal is flying and adjusting the air route as necessary, means for generating and updating an aerial map based on flight information and environmental data from the drone terminal, means for notifying the user of the delivery status, means for taking emergency measures in the event of a problem based on flight information collected by the drone terminal, and means for multiple drone terminals to cooperate and share flight information to improve overall flight efficiency. This enables efficient and safe drone delivery and improves user convenience.

[1315] "Flight information" is a general term for a wide variety of data related to operation, such as location information, altitude, speed, and battery status obtained from drone terminals.

[1316] "Platform" refers to an integrated environment of servers and related software for receiving, storing, and managing flight information sent from drone terminals.

[1317] "Air Route Generation AI" is an AI (artificial intelligence) model that generates optimal flight routes based on accumulated flight information, weather data, past flight patterns, etc.

[1318] "Delivery Request" means a request for drone delivery made by a User through a smartphone app or other interface.

[1319] The "optimal route" refers to the flight route with the highest flight efficiency, selected based on the conditions specified by the route generation AI (delivery destination, weather, time of day, etc.).

[1320] "Monitoring" refers to the process of monitoring flight information transmitted by drone terminals in real time and adjusting operational status as necessary.

[1321] "Environmental data" is a general term for data related to the flight environment, such as weather information and the presence of obstacles, acquired by a drone terminal during flight.

[1322] "Aerial Map" refers to a high-precision aerial map generated by integrating flight information and environmental data.

[1323] "Emergency Plan" means the procedures and protocols for responding quickly if a problem occurs during flight.

[1324] "Improving flight efficiency" refers to the process of multiple drone devices working together and sharing flight information to minimize overall flight time and energy consumption.

[1325] The present invention relates to a system for realizing efficient and safe drone delivery, and detailed embodiments thereof will be described below.

[1326] Hardware and software used

[1327] 1. Server

[1328] Server: High-performance server (e.g. Dell PowerEdge R740)

[1329] Server software: Python, Django, MySQL

[1330] Real-time data processing: Apache Kafka

[1331] 2. Drone terminal

[1332] Drone hardware: A typical high-performance drone (e.g., DJI Mavic 2)

[1333] Communication module: Standard internet communication module

[1334] Drone control application: Drone control SDK (e.g. DJI SDK)

[1335] 3. Air route generation AI

[1336] Deep Learning Frameworks: TensorFlow, PyTorch

[1337] 4. GIS Software

[1338] Aerial Map Generation Tool: Esri ArcGIS

[1339] Program processing

[1340] The server has the function of receiving flight information sent from the drone terminal. During flight, the drone terminal sends flight information such as location, altitude, speed, and battery status to the server in real time. The server stores this received information in a database. This database is operated using MySQL.

[1341] The server trains the route generation AI based on the accumulated flight information. The route generation AI builds an optimization model using Python, TensorFlow, and PyTorch to learn relevant data such as past flight patterns, weather information, date and time, and past bird strikes. This allows it to generate efficient and safe flight routes for future delivery requests.

[1342] When a user requests a drone delivery, the server accepts the request and generates the optimal air route using an air route generation AI. The request includes parameters such as the specified delivery destination and delivery time, and the server selects the optimal route taking this information into account. The generated air route information is then sent to the drone terminal.

[1343] During flight, the drones periodically send flight information to the server. The server monitors this information and adjusts flight routes as necessary. The server also integrates the flight information and environmental data received from the drones to generate and update a highly accurate aerial map. For example, newly discovered obstacles or sudden weather changes can be instantly reflected in the aerial map, ensuring that all drones fly based on the most up-to-date information.

[1344] Additionally, the server provides users with real-time delivery status notifications, allowing them to receive information about the location of their shipments, estimated arrival times, and other important notifications via a smartphone app.

[1345] Specific examples

[1346] When a user requests "drone delivery from their current address to their workplace" via a smartphone app, the request is sent to the server.

[1347] Example prompt sentence:

[1348] I would like to request drone delivery from my current address to my workplace. My current address is in Shibuya-ku, Tokyo, and my workplace is in Minato-ku, Tokyo. I would like the delivery time to be between 10:00 AM and 2:00 PM.

[1349] The server uses route generation AI to generate the optimal flight route from the current address to the workplace and sends it to the drone terminal. The drone terminal then flies along this route and proceeds with the delivery. If an obstacle is detected during flight, the drone terminal sends that information to the server, which then updates the aerial map. The user can check the drone's current location and estimated arrival time in real time via a smartphone app.

[1350] In this way, the present invention enables efficient and safe drone delivery, improving user convenience.

[1351] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1352] Step 1: Gather flight information

[1353] Subject: Device

[1354] Input: Location, altitude, speed, and battery status obtained from the drone's built-in sensors and GPS module.

[1355] How it works: The drone terminal reads the data obtained from these sensors and the GPS module at regular intervals.

[1356] Output: Retrieved flight information data.

[1357] Specific operation: The drone terminal obtains the current latitude and longitude, altitude (meters), speed (kilometers per hour), and battery status (percentage) and generates flight information data.

[1358] Step 2: Send flight information

[1359] Subject: Device

[1360] Input: Collected flight information.

[1361] Operation: The terminal sends the collected flight information to the server via the Internet communication module.

[1362] Output: Flight information sent to the server.

[1363] Specific operation: The drone terminal sends the latest flight information (location, altitude, speed, battery status) to the server as an HTTP request at regular intervals.

[1364] Step 3: Receiving and storing flight information

[1365] Subject: Server

[1366] Input: Flight information sent from the device.

[1367] Operation: The server stores the received flight information in a database.

[1368] Output: Flight information stored in a database.

[1369] What it does: The server receives the HTTP request, parses it, and stores flight information data in a MySQL database, including latitude, longitude, altitude, speed, battery status, and timestamps.

[1370] Step 4: Training the route generation AI

[1371] Subject: Server

[1372] Input: Past flight information, weather information, date and time, and past bird strike information stored in the database.

[1373] How it works: The server uses these data to train the route generation AI.

[1374] Output: A trained air route generation AI model.

[1375] How it works: The server uses TensorFlow and PyTorch to train a neural network based on the accumulated data, allowing the model to learn patterns to generate optimal flight paths.

[1376] Step 5: User Requests Delivery

[1377] Subject: User

[1378] Input: Delivery request from smartphone app (e.g. current address, delivery destination, delivery time).

[1379] How it works: The user enters a delivery request, which the app sends to the server.

[1380] Output: Delivery request information sent to the server.

[1381] Specific operation: The user uses the app to enter their current address (Shibuya-ku, Tokyo), workplace (Minato-ku, Tokyo), and desired delivery time (10:00 AM to 2:00 PM), and submits a delivery request.

[1382] Step 6: Generate optimal routes

[1383] Subject: Server

[1384] Input: A delivery request from the user and current weather information.

[1385] How it works: Uses route generation AI to generate optimal flight paths.

[1386] Output: Generated optimal air route information.

[1387] How it works: The server uses an AI model to input delivery request parameters (e.g., delivery destination, delivery time, weather) and selects the most efficient route from multiple candidate routes.

[1388] Step 7: Sending flight information to the terminal

[1389] Subject: Server

[1390] Input: Generated air route information.

[1391] Operation: The server transmits the generated air route information to the drone terminal in real time.

[1392] Output: Air route information received by the drone terminal.

[1393] Specific operation: The server sends the route information (starting point, intermediate points, and destination) selected by the route generation AI to the drone terminal as an HTTP request.

[1394] Step 8: In-flight monitoring

[1395] Subject: Server

[1396] Input: Flight information periodically sent from the drone terminal during flight.

[1397] How it works: The server monitors the drone in real time based on the flight information it receives and adjusts its flight path as needed.

[1398] Output: Adjusted air route information (if required).

[1399] Specific operation: The server receives the latest information from the drone terminal during flight, uses AI to recalculate the route based on the current situation, such as obstacles and weather changes, and sends the adjusted route information to the terminal in real time.

[1400] Step 9: Generate and update high-definition aerial maps

[1401] Subject: Server

[1402] Input: Flight information and environmental data from the drone device.

[1403] How it works: The server aggregates these data to generate and update aerial maps.

[1404] Output: Up-to-date aeronautical maps.

[1405] What happens: The server uses GIS software (e.g., Esri ArcGIS) to update the aerial map based on the received environmental data (e.g., newly discovered obstacles).

[1406] Step 10: Real-time delivery status notification

[1407] Subject: Server

[1408] Input: Real-time location information of the drone in flight.

[1409] Action: The server notifies the user of the delivery status.

[1410] Output: The user is notified of the location and estimated time of arrival of the delivery.

[1411] Specific operation: The server notifies the user of the drone's current location and estimated arrival time in real time via a smartphone app.

[1412] (Application example 1)

[1413] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1414] Conventional drone delivery systems have difficulty efficiently collecting and managing flight information, posing challenges in optimizing delivery routes and ensuring delivery safety. Furthermore, users have limited means of tracking delivery status in real time, making it difficult to provide accurate information on delivery arrival times. Technological solutions to address these challenges were needed.

[1415] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1416] In this invention, the server includes means for receiving flight information sent from the drone terminal and storing it on the platform, means for training an air route generation AI based on the stored flight information, means for generating an optimal air route using the air route generation AI in response to a delivery request and sending it to the drone terminal, means for monitoring flight information in real time while the drone terminal is flying and adjusting the air route as needed, means for generating and updating an aerial map based on flight information and environmental data from the drone terminal, means for notifying users of delivery status, and means for users to request drone delivery through a smartphone app and receive real-time tracking of deliveries and arrival time predictions. This enables efficient collection and management of flight information, optimizes delivery routes and ensures safety, and enables users to track delivery status in real time and receive accurate arrival predictions.

[1417] A "drone terminal" is an unmanned aerial vehicle that flies and transports objects or goods autonomously.

[1418] "Flight information" refers to data such as location, altitude, speed, and battery status acquired by a drone during flight.

[1419] A "platform" is a system infrastructure for accumulating, managing, and analyzing data.

[1420] "Air route generation AI" is an artificial intelligence that learns past flight patterns, weather information, date and time, obstacle information, etc., and autonomously generates the optimal flight route.

[1421] A "delivery request" is an instruction from a user requesting delivery by drone.

[1422] "Monitoring flight information in real time" refers to constantly monitoring the flight status of a drone while it is in flight.

[1423] "Adjusting the flight route" means instructing a drone in flight to take a new flight path as needed.

[1424] "Environmental data" refers to data related to the external environment, such as weather information and obstacle information obtained by a drone during flight.

[1425] An "aviation map" is map data that includes flight routes, obstacles, weather information, and so on.

[1426] "User" refers to a person who uses the drone delivery service.

[1427] A "smartphone app" is application software that runs on a smartphone and allows users to request drone delivery and check the delivery status.

[1428] "Real-time tracking" refers to instantly checking the location and status information during drone delivery.

[1429] "Arrival time prediction" refers to predicting and notifying the time it will take for a delivery to reach its destination.

[1430] The system of the present invention integrates multiple pieces of hardware and software to operate in order to realize efficient and safe drone delivery. Specific embodiments of the system will be described below.

[1431] Hardware:

[1432] Drone terminal: An unmanned aerial vehicle equipped with GPS, cameras, and sensors (for obstacle detection). This hardware is used to collect flight information and monitor flight status in real time.

[1433] Server: A computer that provides the computational resources for accumulating and analyzing flight information and running route generation AI.

[1434] User's smartphone: This is the device used to request drone delivery and perform real-time tracking.

[1435] software:

[1436] Route Generation AI: AI models built using TensorFlow and PyTorch perform flight path optimization, taking into account data such as historical flight patterns, weather data, time of day, and obstacle information.

[1437] Database: Flight information and user information are managed using a data management system such as PostgreSQL.

[1438] Smartphone application: An application that allows users to request drone delivery and track the delivery status in real time. React Native and Flutter are often used as front-end technologies.

[1439] Processing Description:

[1440] The server receives flight information sent from the drone terminal and stores it on the platform. The stored flight information is used to train the route generation AI. The route generation AI learns from past flight patterns, weather information, date and time, and obstacle information to improve its ability to generate optimal flight routes. When a user makes a delivery request through a smartphone app, the server uses the route generation AI to generate the optimal route and sends that information to the drone terminal.

[1441] More concrete examples:

[1442] For example, when a user requests food delivery from a restaurant to their home using a smartphone app, the request is sent to a server. The server uses route generation AI to generate the optimal flight path from the restaurant to their home and sends it to the drone terminal. The drone terminal flies along this path and proceeds with the delivery. If an obstacle is detected during flight, the drone terminal sends that information to the server, which then updates the aerial map. The user can check the drone's current location and estimated arrival time in real time using the smartphone app.

[1443] Example prompt sentence:

[1444] "Request a drone to deliver food to a specified address. Generate the safest and quickest route based on current weather and historical flight data."

[1445] In this way, the present invention can realize efficient and safe drone delivery, improving user convenience.

[1446] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1447] Step 1:

[1448] The server receives flight information (location, altitude, speed, battery status, etc.) sent from the drone terminal and accumulates the data on the platform. The server receives this input flight information and stores it appropriately in the database.

[1449] Step 2:

[1450] The server trains the route generation AI based on the accumulated flight information. Specifically, it receives data such as past flight patterns, weather information, date and time, and obstacle information as input, and updates the AI ​​model using TensorFlow and PyTorch to learn how to generate optimal flight routes.

[1451] Step 3:

[1452] The user makes a delivery request through a smartphone app. The user inputs the pickup and drop-off locations, and the request data is sent to the server.

[1453] Step 4:

[1454] The server receives delivery requests from users and generates the optimal flight route using route generation AI. The AI ​​receives the request data and current weather information as input, calculates the optimal route, and generates the route information.

[1455] Step 5:

[1456] The server then sends the generated optimal air route information to the drone terminal, which then begins flying along this route.

[1457] Step 6:

[1458] During flight, the drone terminal periodically transmits flight information to the server, which receives real-time flight information as input and monitors it, adjusting the flight path as needed.

[1459] Step 7:

[1460] If a drone detects an obstacle during flight, it sends that information to the server, which then updates the aerial map and recalculates the drone's route. The input is the detected obstacle information, and the output is the updated flight route.

[1461] Step 8:

[1462] The server notifies the user of the delivery status, including important information such as the delivery's location and estimated arrival time, and is communicated to the user in real time via a smartphone app.

[1463] Step 9:

[1464] Once the delivery is complete, the drone terminal transmits the final flight information to the server, which stores this new data in the platform and uses it for the next learning cycle, thereby improving the overall performance of the system.

[1465] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1466] The present invention relates to a system that recognizes a user's emotions and responds according to their state by combining an emotion engine with a drone-based delivery system. Detailed embodiments of the present invention will be described below.

[1467] The server has the function of receiving and storing flight information sent from the drone terminal. The drone terminal sends flight information such as location, altitude, speed, and battery status to the server in real time during flight. The server also trains the air route generation AI based on the accumulated flight information and improves the algorithm for generating efficient and safe air routes.

[1468] When a user requests drone delivery via a smartphone app, the server accepts the request and generates the optimal route using route generation AI. The generated route information is sent to the drone terminal in real time. During flight, the drone terminal periodically sends flight information to the server, which monitors the flight in real time and adjusts the route as necessary. The server also generates and updates an aerial map based on the flight information and environmental data.

[1469] A distinctive feature of this invention is the incorporation of an emotion engine. The emotion engine recognizes the user's emotional state and adjusts the system's behavior based on that information. Specifically, it analyzes the user's facial expressions, tone of voice, and interaction patterns to determine the user's emotions. For example, if the user is irritated, the server adjusts the content and timing of notifications and provides information to reduce the user's stress.

[1470] As a concrete example, when a user requests a drone delivery from their current address to their workplace, the information is sent to the server, and an optimal flight route is generated using air route generation AI. The generated air route information is then sent to the drone terminal, and the drone begins the delivery. Meanwhile, the server regularly monitors the user's emotional state through an emotion engine. If the user expresses irritation about a delivery delay, the server will use that information to adjust the content and timing of notifications, provide explanations to the user, and provide detailed progress reports to alleviate the user's frustration.

[1471] Furthermore, the emotion engine accumulates past emotion data and uses it for future responses, allowing it to learn the user's emotional patterns and provide more personalized responses.

[1472] In this way, by combining an emotion engine with a drone delivery system, the present invention can achieve efficient and safe delivery while improving the user experience.

[1473] The processing flow will be explained below.

[1474] Step 1:

[1475] Drone startup and initial setup

[1476] The device completes the initial setup and connects to the system, sending its identification information, initial position, altitude, and battery status to the server.

[1477] Step 2:

[1478] Real-time flight information transmission

[1479] During flight, the device periodically transmits flight information such as location, altitude, speed, and battery status to the server in real time.

[1480] Step 3:

[1481] Data accumulation

[1482] The server automatically stores the received flight information on a proprietary platform, which stores training data and data required for real-time route generation.

[1483] Step 4:

[1484] Air route generation AI training

[1485] The server trains the route generation AI based on accumulated flight information and data such as date and time, weather, and past bird strikes. The route generation AI learns and improves its algorithm to generate efficient and safe routes.

[1486] Step 5:

[1487] Accepting a delivery request

[1488] The user enters a delivery request through a smartphone app and sends it to the server.

[1489] The server analyzes the request content (delivery destination, desired time, etc.).

[1490] Step 6:

[1491] Air Route Generation

[1492] The server uses route generation AI to generate the optimal route based on the request.

[1493] The server takes the generated air route information and takes into account relevant constraints (weather, obstacles, etc.).

[1494] Step 7:

[1495] Sending air route information

[1496] The server transmits the generated air route information to the terminal (drone).

[1497] The terminal loads the received air route information into memory and prepares for flight.

[1498] Step 8:

[1499] Start of flight

[1500] The terminal starts the flight based on the received air route information.

[1501] The terminal periodically transmits flight information to the server.

[1502] Step 9:

[1503] Real-time monitoring and adjustment

[1504] The server monitors real-time data from the terminals during the flight.

[1505] The server analyzes flight information and environmental data and adjusts air routes as needed.

[1506] Step 10:

[1507] Generate and update aerial maps

[1508] The server generates and updates an aerial map based on flight information and environmental data from the drone.

[1509] Newly discovered obstacles and weather changes are instantly updated on the aerial map.

[1510] Step 11:

[1511] User Notifications

[1512] The server notifies the user of the delivery status in real time.

[1513] Users can use a smartphone app to check the current location and estimated arrival time of their delivery.

[1514] Step 12:

[1515] Emotion engine activation and emotion recognition

[1516] The server monitors the user's facial expressions, tone of voice, and interaction patterns through a smartphone or other interface.

[1517] The server runs an emotion engine that recognizes whether the user is feeling frustrated, happy, confused, etc.

[1518] Step 13:

[1519] Emotion-based notification adjustment

[1520] If the user expresses negative emotions such as irritation, the server adjusts the content and timing of the notification, for example adding detailed delivery progress information or an apology message.

[1521] If the user is satisfied, the server continues with the normal notification method.

[1522] Step 14:

[1523] Accumulating and learning emotional data

[1524] The server stores the user's emotional responses and uses them as data for future notifications and service improvements, allowing it to learn the user's emotional patterns and provide more personalized responses.

[1525] Step 15:

[1526] Delivery completed

[1527] The terminal arrives at the delivery destination and delivers the package safely.

[1528] The terminal notifies the server that the delivery is complete.

[1529] The server notifies the user that delivery has been completed.

[1530] Example 2

[1531] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1532] While conventional drone delivery systems have achieved a certain level of success in terms of delivery efficiency and safety, they lack consideration for the user's emotional state, limiting their ability to improve user experience. Furthermore, insufficient user response in the event of delivery delays or unexpected failures can result in increased user dissatisfaction. This has led to a decline in user satisfaction and left room for service improvement.

[1533] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1534] In this invention, the server includes means for receiving flight information transmitted from the drone terminal and storing it on the server, means for training an air route generation algorithm based on the stored flight information, means for generating an optimal air route using the air route generation algorithm in response to a delivery request and transmitting it to the drone terminal, means for monitoring the flight information in real time while the drone terminal is flying and adjusting the air route as necessary, means for generating and updating an air map based on the flight information and environmental data from the drone terminal, means including an emotion engine that recognizes user emotions and adjusts system behavior based on the user emotions, and means for notifying the user of delivery status and adjusting the content and timing of notification based on the user's emotional state. This enables flexible responses based on user emotions and efficient and safe delivery.

[1535] A "drone terminal" is an unmanned aerial vehicle that flies remotely or autonomously and delivers designated items.

[1536] "Flight information" refers to various flight-related data such as the drone terminal's location, altitude, speed, and battery status.

[1537] A "server" is a remote computer system that stores, processes, and analyzes data, and is a device that receives flight information from drone terminals and performs various functions based on that information.

[1538] An "air route generation algorithm" is an algorithm that uses artificial intelligence technology to calculate flight routes, and is designed to generate efficient and safe flight routes.

[1539] An "emotion engine" is a technology that analyzes a user's emotional state and adjusts the system's behavior based on that information. It is an engine that determines emotions by analyzing facial expressions and tone of voice.

[1540] A "delivery request" refers to a request made by a user via a smartphone app or other means to have a specific item delivered to a specified location.

[1541] An "aviation map" is a map generated based on flight information and environmental data, and provides information on the airspace, including the flight path of the drone terminal.

[1542] "Environmental Data" refers to data about external environmental conditions that affect flight, such as weather, temperature, and wind speed.

[1543] The present invention provides a system that recognizes the emotions of a user and responds according to that user's state by incorporating an emotion engine into a drone delivery system. Detailed embodiments of the present invention will be described below.

[1544] Hardware and software configuration

[1545] Drone terminal

[1546] Sensors: Equipped with a GPS sensor, gyro sensor, and battery monitoring system, it measures location, altitude, speed, and battery status in real time.

[1547] Communication module: The mobile communication module is used to send and receive data to and from the server in real time.

[1548] Control unit: Equipped with a control unit for flight control, it flies autonomously along a designated air route.

[1549] server

[1550] Database: The system has a database that stores and manages flight information received from drones.

[1551] Route Generation AI: Equipped with an AI model that trains route generation algorithms based on accumulated flight information to generate efficient and safe routes.

[1552] Emotion engine: Contains software that analyzes facial expressions and tone of voice to determine the user's emotional state.

[1553] System Operation

[1554] The server receives flight information sent from the drone terminal in real time and stores that information in a database. The stored data is used as training data for the air route generation AI. When a user sends a delivery request through a smartphone app, the server uses the air route generation AI to calculate the optimal flight path and sends that information to the drone terminal. The drone then begins flying autonomously along the specified air route.

[1555] Specific examples of processing

[1556] When a user requests a drone delivery from their current address to their workplace, the information is sent to the server. The server uses route generation AI to generate the optimal flight path and sends that route information to the drone terminal. The drone then begins the delivery and periodically sends flight information such as its location and battery status to the server during the flight.

[1557] Meanwhile, the server constantly monitors the user's emotional state through an emotion engine. For example, if the user expresses irritation about a delivery delay, the server can adjust the content and timing of notifications based on that information, and provide the user with detailed explanations and progress reports to alleviate the user's frustration.

[1558] Prompt Sentence Examples

[1559] "Please create the following drone delivery system model. When a user requests delivery via a smartphone app, the server accepts the request and generates the optimal flight route using route generation AI. During flight, the drone continuously sends flight information to the server, which monitors it in real time. It also uses an emotion engine to recognize the user's emotional state and adjusts the system's behavior based on that information."

[1560] In this way, the present invention combines an emotion engine with a drone delivery system to achieve efficient and safe delivery as well as an improved user experience.

[1561] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1562] Step 1: Request acceptance

[1563] Input: A user submits a delivery request from a smartphone app, including their current address, destination, and delivery details.

[1564] Processing: The user's request is sent to the server through the communication module.

[1565] Output: The server records the received request in a database and acknowledges receipt of the request.

[1566] Specific operation: The server analyzes the request data and stores the order ID, delivery item, origin and destination information in a database.

[1567] Step 2: Generate air routes

[1568] Input: The server receives the user's delivery request information.

[1569] Processing: The server runs a route generation algorithm to calculate an efficient and safe flight path. It uses a generative AI model to generate the optimal route based on the request data and accumulated historical flight information.

[1570] Output: Optimal flight path, estimated flight time, and predicted battery consumption data are generated.

[1571] Specific operation: The server sends the generated flight path to the drone terminal and starts the delivery plan.

[1572] Step 3: Send flight information

[1573] Input: The drone device receives location information, altitude, speed, and battery status collected from sensors during flight.

[1574] Processing: The drone terminal sends the collected flight information to the server via the communication module.

[1575] Output: The server monitors drone flight information in real time and stores it in a database.

[1576] Specific operation: The drone terminal collects data using GPS sensors, gyro sensors, and battery monitoring systems, and transmits the data to a server in real time.

[1577] Step 4: Real-time monitoring and air route adjustments

[1578] Input: Real-time flight information sent from the drone terminal.

[1579] Processing: The server analyzes the flight information and issues an alert if an anomaly is detected.

[1580] Output: New air routes are generated as needed and sent to the drone terminal.

[1581] Specific operation: The server checks whether the flight information is normal, and if the weather worsens or obstacles appear, it generates a new flight route and sends it to the drone terminal.

[1582] Step 5: Recognize and respond to emotional states

[1583] Input: Data on the user's facial expressions and tone of voice.

[1584] Processing: The server uses the emotion engine to analyze the user's emotional state. The emotion engine analyzes the received facial expressions and tone of voice and processes the data to determine the user's emotion.

[1585] Output: Notification content and delivery information are generated according to the user's emotional state.

[1586] Specific operation: The server uses an emotion engine to perform emotion analysis, such as "the user is irritated," and changes the content of the notification appropriately depending on the state.

[1587] Step 6: Accumulating and Learning Emotional Data

[1588] Input: User's historical emotion data.

[1589] Processing: The server stores past emotional data in the emotion engine database. Emotional patterns are learned based on this data.

[1590] Output: More personalized user-responsive data will be generated in the future.

[1591] Specific operation: The server accumulates emotional data, and the emotion engine uses this data to learn the user's emotional patterns, resulting in more precise and personalized responses to the user from the next time onwards.

[1592] (Application example 2)

[1593] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1594] Conventional drone delivery systems lack the flexibility to respond to the user's emotional state, and improvements to the user experience are desired. Furthermore, there is a lack of means to alleviate the user's anxiety and frustration during delivery, making it a challenge to improve user satisfaction.

[1595] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1596] In this invention, the server includes means for receiving flight information transmitted from the drone terminal and storing it on the platform, means for training an air route generation AI based on the stored flight information, means for generating an optimal air route using the air route generation AI in response to a delivery request and transmitting it to the drone terminal, means for monitoring flight information in real time while the drone terminal is flying and adjusting the air route as necessary, means for generating and updating an aerial map based on flight information and environmental data from the drone terminal, means for notifying the user of the delivery status, and means for combining an emotion engine that analyzes the user's emotional state and adjusts the content and timing of notifications based on that information. This enables flexible responses based on the user's emotional state, reducing the user's anxiety and irritation during delivery and improving user satisfaction.

[1597] A "drone terminal" is an unmanned aerial vehicle that flies through the air and delivers goods to designated locations.

[1598] "Flight information" refers to data such as the drone terminal's location, altitude, speed, and battery status.

[1599] The "platform" is a server or cloud-based data management system that receives and stores drone flight information.

[1600] "Air Route Generation AI" is an artificial intelligence algorithm for optimizing drone flight paths.

[1601] An "emotion engine" is an engine that has the function of analyzing emotions from the user's facial expressions, tone of voice, interaction patterns, etc., and adjusting the system's behavior based on that information.

[1602] "Environmental data" refers to data such as weather information and obstacle information that affect drone flight.

[1603] An "aviation map" is map data that includes route information and obstacle information for drone flight.

[1604] The present invention provides a system that flexibly responds to the emotional state of a user by combining an emotion engine with a drone-based delivery system. Detailed embodiments of the present invention will be described below.

[1605] The system mainly consists of the following components:

[1606] 1. Drone terminal

[1607] The drone terminal is an unmanned aerial vehicle that flies through the air and delivers goods to designated locations. The drone terminal transmits flight information such as location, altitude, speed, and battery status to a server in real time.

[1608] 2. Platform

[1609] The platform consists of a server and a cloud-based data management system that receives and stores flight information sent from drone terminals. Based on this information, the air route generation AI and emotion engine operate.

[1610] 3. Air route generation AI

[1611] Route generation AI is an artificial intelligence algorithm for optimizing drone flight paths. It is trained based on flight information and environmental data, such as the date and time, weather, and past bird strike information, to generate optimal routes.

[1612] 4. Emotion Engine

[1613] The emotion engine analyzes the user's facial expressions, tone of voice, and interaction patterns to determine their emotions, and adjusts the system's behavior based on this information. This engine is built using deep learning frameworks such as TensorFlow and Keras. It adjusts the content and timing of notifications to reduce user frustration and anxiety.

[1614] 5. Aerial map generation and update

[1615] The server generates and updates an aerial map based on flight information and environmental data from the drone terminals, and if new obstacles are detected, the information is also reflected on the map.

[1616] 6. Notification System

[1617] The server notifies the user of the delivery status. The content and timing of the notification are adjusted based on the user's emotional state analyzed by the emotion engine. This reduces the user's anxiety and frustration and improves satisfaction.

[1618] Hardware and Software Configuration

[1619] Server: Cloud-based data management system (Amazon Web Services, Microsoft Azure, etc.)

[1620] Drone terminal: Small unmanned aerial vehicle

[1621] Emotion recognition system: TensorFlow, Keras

[1622] Real-time communication: REST API and WebSocket

[1623] Specific examples

[1624] A user requests a food delivery drone using a smartphone app. The user's facial expressions and tone of voice are captured by a camera and microphone, and the data is analyzed by an emotion engine. Based on the analysis results, the content and timing of delivery notifications are adjusted. If the user shows signs of impatience, a message such as "Sorry for the wait. Delivery status: On its way." is displayed.

[1625] Prompt Sentence Examples

[1626] "Generate optimal delivery notification content based on the user's emotional state. Also, adjust the notification content in real time according to the delivery status."

[1627] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1628] Step 1:

[1629] A user requests drone delivery from a smartphone app. At this time, the app receives the user's delivery request data (shipping origin, delivery destination, order details, etc.) as input and sends it to the server. The server receives this data and stores it on the platform.

[1630] Step 2:

[1631] The server trains the route generation AI based on the accumulated flight information. It uses historical flight information and environmental data as input, and updates the model to generate optimal routes as output. This model is trained using an AI algorithm (e.g., TensorFlow or Keras).

[1632] Step 3:

[1633] The server generates the optimal flight route based on the user's delivery request using an AI route generation model. The inputs are the delivery origin, delivery destination, and real-time environmental data (weather, obstacle information, etc.), and the output is the optimal flight path. This route data is sent to the drone terminal.

[1634] Step 4:

[1635] The drone terminal will then begin delivery based on the received flight route. During flight, the drone terminal will continue to send real-time flight information, such as location, altitude, speed, and battery status, to the server. The server receives this data and stores it on the platform.

[1636] Step 5:

[1637] The server monitors the real-time flight information of the drone during flight and adjusts the flight path as needed. For example, if a new obstacle is detected, the flight path generation AI is reactivated, recalculating the optimal route and sending it to the drone. The obstacle information is used as input, and the updated flight path is obtained as output.

[1638] Step 6:

[1639] Based on the flight information and environmental data sent from the drone terminal, the server generates and updates an aerial map. It uses real-time flight information and environmental data as input and creates an updated aerial map as output. This map is used to generate future flight paths.

[1640] Step 7:

[1641] Analyze the user's emotional state. Emotional data such as facial expressions and tone of voice obtained from the user's smartphone app is input into the emotion engine. The emotion engine analyzes this data to determine the user's emotional state. For example, a Keras model can be used to analyze facial expression data and output emotional states such as irritation or anxiety.

[1642] Step 8:

[1643] The server adjusts the content and timing of notifications to the user based on the emotional state analyzed by the emotion engine. The emotional state and delivery status are used as inputs, and an appropriate notification message is generated as output to be sent to the user. For example, if the user is annoyed, a message detailing the progress of the delivery along with an apology is generated.

[1644] Step 9:

[1645] Finally, the server sends the generated notification message to the user's smartphone app, allowing the user to receive timely notifications about the delivery status, reducing user anxiety and frustration and improving overall satisfaction.

[1646] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1647] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1648] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1649] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1650] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1651] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1652] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1653] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1654] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1655] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1656] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1657] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1658] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1659] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1660] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1661] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1662] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1663] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1664] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1665] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1666] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1667] The following is further disclosed regarding the above embodiment.

[1668] (Claim 1)

[1669] A means for receiving flight information transmitted from the drone terminal and storing it on the platform;

[1670] a means for training the route generation AI based on the accumulated flight information;

[1671] A means for generating the optimal air route using air route generation AI in response to a delivery request and transmitting the route to the drone terminal;

[1672] A means for monitoring flight information in real time while the drone terminal is flying and adjusting the air route as necessary;

[1673] means for generating and updating an aerial map based on flight information and environmental data from the drone terminal;

[1674] means for notifying a user of delivery status;

[1675] A system including:

[1676] (Claim 2)

[1677] The system according to claim 1, wherein the route generation AI learns information on date, time, weather, and past bird strikes.

[1678] (Claim 3)

[1679] 2. The system of claim 1, wherein when the drone terminal detects a new obstacle, the drone terminal transmits the information to the platform and updates the aerial map.

[1680] "Example 1"

[1681] (Claim 1)

[1682] A means for receiving flight information transmitted from the drone terminal and storing it on the platform;

[1683] a means for training the route generation AI based on the accumulated flight information;

[1684] A means for generating the optimal air route using air route generation AI in response to a delivery request and transmitting the route to the drone terminal;

[1685] A means for monitoring flight information in real time while the drone terminal is flying and adjusting the air route as necessary;

[1686] means for generating and updating an aerial map based on flight information and environmental data from the drone terminal;

[1687] means for notifying a user of delivery status;

[1688] Based on the flight information collected by the drone terminal, there are measures to take emergency measures in case of trouble,

[1689] A means for multiple drone terminals to work together to share flight information and improve overall flight efficiency;

[1690] A system including:

[1691] (Claim 2)

[1692] The system according to claim 1, wherein the route generation AI learns information on date, time, weather, and past bird strikes.

[1693] (Claim 3)

[1694] 2. The system of claim 1, wherein when the drone terminal detects a new obstacle, the drone terminal transmits the information to the platform and updates the aerial map.

[1695] "Application Example 1"

[1696] (Claim 1)

[1697] A means for receiving flight information transmitted from the drone terminal and storing it on the platform;

[1698] a means for training the route generation AI based on the accumulated flight information;

[1699] A means for generating the optimal air route using air route generation AI in response to a delivery request and transmitting the route to the drone terminal;

[1700] A means for monitoring flight information in real time while the drone terminal is flying and adjusting the air route as necessary;

[1701] means for generating and updating an aerial map based on flight information and environmental data from the drone terminal;

[1702] a means for notifying the user of the status of delivery;

[1703] A means for users to request drone delivery through a smartphone app and receive real-time tracking and arrival time estimates for their shipments;

[1704] A system including:

[1705] (Claim 2)

[1706] The system according to claim 1, wherein the route generation AI learns information on date, time, weather, and past bird strikes.

[1707] (Claim 3)

[1708] 2. The system of claim 1, wherein when the drone terminal detects a new obstacle, the drone terminal transmits the information to the platform and updates the aerial map.

[1709] "Example 2: Combining Emotion Engines"

[1710] (Claim 1)

[1711] A means for receiving flight information transmitted from the drone terminal and storing it in a server;

[1712] means for training an air route generation algorithm based on the accumulated flight information;

[1713] A means for generating an optimal air route using an air route generation algorithm in response to a delivery request and transmitting the generated route to a drone terminal;

[1714] A means for monitoring flight information in real time while the drone terminal is flying and adjusting the air route as necessary;

[1715] means for generating and updating an aerial map based on flight information and environmental data from the drone terminal;

[1716] means including an emotion engine for recognizing user emotions and adjusting system behavior based thereon;

[1717] means for notifying the user of delivery status and adjusting the content and timing of notification according to the user's emotional state;

[1718] A system including:

[1719] (Claim 2)

[1720] 10. The system of claim 1, wherein the route generation algorithm learns information about time, date, weather, and past obstacles.

[1721] (Claim 3)

[1722] The system of claim 1, wherein when the drone terminal detects a new obstacle, the information is sent to the server and the aerial map is updated.

[1723] "Application example 2 when combining emotion engines"

[1724] (Claim 1)

[1725] A means for receiving flight information transmitted from the drone terminal and storing it on the platform;

[1726] a means for training the route generation AI based on the accumulated flight information;

[1727] A means for generating the optimal air route using air route generation AI in response to a delivery request and transmitting the route to the drone terminal;

[1728] A means for monitoring flight information in real time while the drone terminal is flying and adjusting the air route as necessary;

[1729] means for generating and updating an aerial map based on flight information and environmental data from the drone terminal;

[1730] means for notifying a user of delivery status;

[1731] A means of combining an emotion engine that analyzes the user's emotional state and adjusts the content and timing of notifications based on that information;

[1732] A system including:

[1733] (Claim 2)

[1734] The system according to claim 1, wherein the route generation AI learns information on date, time, weather, and past bird strikes.

[1735] (Claim 3)

[1736] 2. The system of claim 1, wherein when the drone terminal detects a new obstacle, the drone terminal transmits the information to the platform and updates the aerial map. [Explanation of symbols]

[1737] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for receiving flight information transmitted from the drone terminal and storing it on the platform; a means for training the route generation AI based on the accumulated flight information; A means for generating the optimal air route using air route generation AI in response to a delivery request and transmitting the route to the drone terminal; A means for monitoring flight information in real time while the drone terminal is flying and adjusting the air route as necessary; means for generating and updating an aerial map based on flight information and environmental data from the drone terminal; means for notifying a user of delivery status; A system including:

2. The system according to claim 1, wherein the route generation AI learns information on date, time, weather, and past bird strikes.

3. The system of claim 1 , wherein when the drone terminal detects a new obstacle, the drone terminal transmits the information to the platform and updates the aerial map.

Images