system

Abstract

We provide the system. [Solution] Data collection means, Means for analyzing the collected data, Means for generating routes based on multiple means of transportation, A means of calculating the travel time for each route, A means of selecting the optimal route, A means of notifying the user of the optimal route, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In conventional transfer guidance systems, route proposals based on train and bus schedule information are mainstream, and it has been difficult to make comprehensive route proposals that combine transportation means with difficult time prediction, such as shared bicycles, taxis, and demand buses. For this reason, there has been a problem that users cannot select appropriate transportation means and cannot improve the efficiency of travel time.

Means for Solving the Problems

[0005] The present invention solves the above problems by providing a system that includes data collection means, means for analyzing the collected data, means for generating routes based on multiple means of transportation, means for calculating the travel time for each route, means for selecting the optimal route, and means for notifying the user of the optimal route. By collecting timetable data, real-time operation information, and location information, analyzing them using artificial intelligence, and supplementing missing timetable information, it is possible to provide users with efficient and accurate route suggestions.

[0006] "Data collection means" refers to devices or programs that collect timetable data for various modes of transportation, as well as operational records and location information for shared bicycles, taxis, on-demand buses, etc.

[0007] "Means of analysis" refers to a device or program that analyzes various collected data to identify missing information in timetables and predict real-time traffic conditions.

[0008] "Means for generating a route" refers to a device or program that combines multiple means of transportation to create the optimal route from the user's starting point to their destination.

[0009] "Means for calculating travel time" refers to a device or program that calculates the travel time for each segment of a route and then sums them up to calculate the total travel time.

[0010] "Means for selecting the optimal route" refers to a device or program that selects the most suitable route for the user based on evaluation criteria such as travel time, cost, and comfort for each calculated route.

[0011] "Means for notifying the user of the optimal route" refers to a device or program that provides detailed information about the selected optimal route to the user's terminal.

[0012] "Timetable data" refers to the operating schedule information provided by public transportation companies such as trains and buses.

[0013] "Real-time service information" refers to information that shows the current operating status of public transportation, shared bicycles, taxis, and other services.

[0014] "Artificial intelligence" refers to machine learning algorithms used for tasks such as analyzing large amounts of data, filling in missing information in timetables, and making predictions. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] The system of the present invention includes means for data collection, means for analysis, means for route generation, means for calculating travel time, means for selecting the optimal route, and means for notifying the user of the optimal route. This system proposes the optimal means of transportation by collecting and analyzing timetable data for various modes of transport, as well as operation records and location information for shared bicycles, taxis, on-demand buses, etc.

[0037] Program Processing Overview

[0038] Data acquisition methods

[0039] The server collects timetable data from various transportation services via APIs. It also acquires real-time operational information from railway and bus companies. In addition, it collects real-time location information for shared bicycles and taxis.

[0040] Data analysis means

[0041] Based on the collected data, the server uses AI algorithms to analyze it. This includes filling in missing information in timetables, predicting real-time traffic conditions, and estimating travel times for shared bicycles and taxis.

[0042] Route generation means

[0043] Based on the analysis results, the server generates route options that combine multiple modes of transportation. These routes include walking, shared bicycles, taxis, on-demand buses, trains, and buses.

[0044] Travel time calculation means

[0045] The travel time for each route is added up, and the average travel time is calculated by taking real-time traffic conditions into account. In addition, multiple different route and mode of transport options are calculated simultaneously.

[0046] Optimal route selection method

[0047] The server selects the optimal route from the generated route options based on evaluation criteria such as travel time, cost, and comfort. This results in the most efficient route being suggested to the user.

[0048] User notification means

[0049] The server notifies the terminal of the optimal route and estimated travel time. Based on the information provided, the user can choose the best route according to their preferences.

[0050] Specific example

[0051] For example, let's consider a scenario involving travel from Tokyo Station to Haneda Airport.

[0052] 1. The user enters "Tokyo Station" as the departure point, "Haneda Airport" as the destination, and "10:00" as the departure time into the terminal.

[0053] 2. The server collects data from railway companies, taxi companies, and shared bicycle providers and predicts the travel time for each route.

[0054] 3. The server generates the following route options:

[0055] Route A: Travel from Tokyo Station to Shinagawa Station by train → From Shinagawa Station, take the Keikyu Line to Haneda Airport.

[0056] Route B: Take a taxi from Tokyo Station to Hamamatsucho Station → Take the Tokyo Monorail from Hamamatsucho Station to Haneda Airport

[0057] Route C: Use a shared bicycle from Tokyo Station → Take a train from the nearest station to Haneda Airport.

[0058] 4. The server evaluates the travel time, cost, and comfort of each route option and selects the optimal route.

[0059] 5. The server notifies the terminal of the selected optimal route and estimated travel time.

[0060] 6. The user selects a route and begins to travel.

[0061] Through the above processing, the system of the present invention proposes efficient and accurate means of transportation and routes to the user.

[0062] The following describes the processing flow.

[0063] Step 1:

[0064] The user enters their departure point, destination, and desired departure or arrival time into the terminal. For example, they might enter "Tokyo Station" as the departure point, "Haneda Airport" as the destination, and "10:00" as the departure time.

[0065] Step 2:

[0066] The terminal receives input and sends that data to the server. The data sent includes the departure point, destination, and departure time.

[0067] Step 3:

[0068] The server uses APIs from various transportation providers to collect timetable data and real-time service information. This includes information from railway companies, bus companies, shared bicycle providers, taxi companies, and others. The collected data is stored in a database.

[0069] Step 4:

[0070] The server analyzes the collected data. Using AI algorithms, it fills in missing information in timetables and predicts real-time traffic conditions. It also estimates travel times for shared bicycles and taxis.

[0071] Step 5:

[0072] The server generates route options that combine multiple modes of transportation. These include walking, shared bicycles, taxis, on-demand buses, trains, and buses. For example, it can generate multiple routes to get from Tokyo Station to Haneda Airport.

[0073] Step 6:

[0074] The server calculates the travel time for each route. It then sums up the segmented travel times for each mode of transport to calculate the total travel time. It also evaluates delay risk, taking real-time traffic conditions into account.

[0075] Step 7:

[0076] The server selects the optimal route from the generated route options based on evaluation criteria such as travel time, cost, and comfort. Each evaluation criterion is weighted, and the AI ​​performs an overall scoring.

[0077] Step 8:

[0078] The server notifies the terminal of the optimal route selected by the server and the estimated travel time. The notification includes detailed information about each route, such as travel time, cost, and waiting time.

[0079] Step 9:

[0080] The user reviews the route options they are notified of and selects the best route according to their preferences. The selected route is then fed back to the server via the device.

[0081] Step 10:

[0082] The server uses user selection information to provide additional data to the AI ​​algorithm, improving the accuracy of future route generation and evaluation processes. This feedback loop enhances the overall system performance.

[0083] (Example 1)

[0084] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0085] Conventional transportation systems struggled to effectively combine multiple modes of transport to suggest the optimal route, and even more so to select routes that took real-time traffic conditions into account. As a result, users experienced inconvenience during their journeys and were unable to travel efficiently.

[0086] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0087] In this invention, the server includes means for collecting timetable data, real-time operation information, and location information; means for analyzing the collected data using artificial intelligence to fill in gaps in timetable information and predict real-time traffic conditions; and means for generating routes that combine multiple means of transportation. This enables users to travel efficiently and quickly.

[0088] "Timetable data" refers to information about the operating schedules of public transportation.

[0089] "Real-time service information" refers to information that shows the current location of transportation services and how much of a delay or progress they are making compared to their scheduled service.

[0090] "Location information" refers to information that indicates the current location of a particular mode of transportation (e.g., shared bicycles, taxis, etc.).

[0091] Artificial intelligence is a technology that enables computer systems to learn on their own through data analysis and pattern recognition, and to automatically perform specific tasks.

[0092] "Means of transportation" is a general term encompassing various methods of getting around, such as trains, buses, taxis, shared bicycles, and walking.

[0093] A "route" is a series of paths from a starting point to a destination.

[0094] "Travel time" refers to the time it takes to travel from a starting point to a destination using a specific route.

[0095] "Expenses" refer to the amount of money required to use a particular mode of transportation.

[0096] "Comfort" refers to the degree of ride quality and convenience experienced during travel.

[0097] A "user terminal" is an electronic device used by a user to receive information.

[0098] "Data collection means" refers to the functions and methods for obtaining necessary information from transportation systems.

[0099] "Analysis means" refers to the functions and methods for processing and analyzing collected data.

[0100] "Generation means" refers to functions and methods for creating routes that combine multiple modes of transportation based on analyzed data.

[0101] "Notification means" refers to functions and methods for informing the user's terminal of selected routes and other information.

[0102] The system of the present invention includes data collection means, analysis means, route generation means, travel time calculation means, optimal route selection means, and user notification means. Specific embodiments of each of these means are described below.

[0103] Data acquisition methods

[0104] The server collects timetable data, real-time service information, and location information from various transportation services via APIs. For example, it uses APIs from railway and bus companies to obtain timetables and service status in real time. It also collects location information from shared bicycles and taxis using the same method. This allows all necessary data to be managed centrally.

[0105] Data analysis means

[0106] The server analyzes the collected data using artificial intelligence (AI). Here, machine learning models and deep learning algorithms are utilized to fill in missing information in timetables, predict real-time traffic conditions, and estimate travel times for various modes of transportation. Specific software used includes TensorFlow and PyTorch.

[0107] Route generation means

[0108] The server generates a route combining multiple modes of transportation based on the analysis results. This route may include walking, trains, buses, shared bicycles, taxis, and on-demand buses. The combination of each mode of transportation will vary depending on the user's starting point and destination.

[0109] Travel time calculation means

[0110] The server adds up the travel times for each route and calculates the average travel time, taking real-time traffic conditions into account. For example, it calculates a route that takes 10 minutes on foot, 30 minutes by train, and 20 minutes by taxi, and then adjusts for traffic congestion to predict an accurate travel time.

[0111] Optimal route selection method

[0112] The server selects the optimal route from the generated routes based on evaluation criteria such as travel time, cost, and comfort. The optimal route is designed to maximize the user's travel efficiency, and the algorithm compares and selects routes using multiple evaluation criteria.

[0113] User notification means

[0114] The server notifies the user's device of the optimal route and estimated travel time. Based on this information, the user selects a route that suits their preferences and begins their journey. This notification is delivered via a mobile app or web application.

[0115] Specific example

[0116] Scenario: Travel from Tokyo Station to Haneda Airport

[0117] 1. The user enters the departure point "Tokyo Station", destination "Haneda Airport", and departure time "10:00" into the terminal.

[0118] 2. The server collects data from railway companies, taxi companies, and shared bicycle providers and predicts the travel time for each route.

[0119] 3. The server generates the following route options:

[0120] Route A: Travel from Tokyo Station to Shinagawa Station by train → From Shinagawa Station to Haneda Airport by train.

[0121] Route B: Take a taxi from Tokyo Station to Hamamatsucho Station → Take the monorail from Hamamatsucho Station to Haneda Airport

[0122] Route C: Use a shared bicycle from Tokyo Station → Take a train from the nearest station to Haneda Airport.

[0123] 4. The server evaluates the travel time, cost, and comfort of each route option and selects the optimal route.

[0124] 5. The server notifies the terminal of the optimal route and estimated travel time.

[0125] 6. The user selects a route and begins to travel.

[0126] Examples of prompts for generative AI models

[0127] "Please suggest the best route from Tokyo Station to Haneda Airport. Available modes of transport include train, bus, taxi, and shared bicycle."

[0128] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0129] Step 1:

[0130] The server uses transportation APIs to collect timetable data, real-time service information, and location information for shared bicycles and taxis.

[0131] Input: API endpoints for each transportation provider.

[0132] Processing: Call the API to retrieve timetable data, real-time service information, and location information.

[0133] Output: Collected raw data (timetables, service status, location information).

[0134] Step 2:

[0135] The server passes the collected data to an artificial intelligence (AI) algorithm for data analysis.

[0136] Input: Raw data collected in Step 1.

[0137] Processing: Machine learning models and deep learning algorithms are used to fill in missing information in timetables, predict real-time traffic conditions, and estimate travel times for shared bicycles and taxis.

[0138] Output: Interpolated timetable data, predicted traffic conditions, and estimated travel time.

[0139] Step 3:

[0140] The server generates a route that combines multiple modes of transportation based on the analysis results.

[0141] Input: Analysis result data generated in Step 2.

[0142] Processing: Create the optimal route option by combining various modes of transportation such as walking, trains, buses, shared bicycles, taxis, and on-demand buses.

[0143] Output: Multiple route options (e.g., Route A, Route B, Route C).

[0144] Step 4:

[0145] The server calculates the travel time for each route and then calculates the average travel time, taking real-time traffic conditions into account.

[0146] Input: Route options generated in Step 3.

[0147] Processing: The travel time for each mode of transport along the route is added together, and the average travel time is calculated taking real-time traffic conditions into account.

[0148] Output: Average travel time for each route.

[0149] Step 5:

[0150] The server selects the optimal route based on evaluation criteria such as travel time, cost, and comfort.

[0151] Input: Average travel time data for each route generated in Step 4.

[0152] Processing: Evaluate the travel time, cost, and comfort of each route, and select the optimal route overall.

[0153] Output: Optimal route.

[0154] Step 6:

[0155] The server notifies the terminal of the selected optimal route and estimated travel time.

[0156] Input: The optimal route selected in Step 5.

[0157] Processing: Convert the optimal route and estimated travel time into a data format for user notifications.

[0158] Output: Notification data sent to the user's terminal (optimal route, estimated travel time).

[0159] Step 7:

[0160] The device displays the notified optimal route and estimated travel time to the user.

[0161] Input: Notification data sent from Step 6.

[0162] Processing: Display notification data on the device's screen interface.

[0163] Output: Presenting the user with the optimal route and estimated travel time.

[0164] Step 8:

[0165] The user begins their journey to their destination based on the displayed optimal route and estimated travel time.

[0166] Input: The optimal route and estimated travel time displayed on the device.

[0167] Process: The user reviews the information and begins moving according to the selected route.

[0168] Output: Actual movement begins.

[0169] (Application Example 1)

[0170] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0171] Traditional food delivery systems did not effectively utilize real-time traffic and location information to optimize delivery routes, resulting in decreased delivery efficiency and longer delivery times. Furthermore, their inability to adapt to sudden changes in traffic conditions led to delivery delays and increased costs.

[0172] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0173] In this invention, the server includes means for collecting data, means for analyzing the collected data, means for generating routes based on multiple means of transportation, means for calculating the travel time for each route, means for selecting the optimal route, means for notifying the user of the optimal route, and means for generating efficient food delivery routes using real-time transportation data and location information and notifying delivery personnel of the optimal route. This significantly improves delivery efficiency, shortens delivery times, and enables rapid response to sudden changes in traffic conditions.

[0174] "Data collection methods" refer to means for acquiring real-time data and location information from various modes of transportation.

[0175] "Means for analyzing collected data" refers to methods for predicting and analyzing real-time traffic conditions based on collected data.

[0176] "Means for generating routes based on multiple modes of transportation" refers to methods for generating routes by combining multiple modes of transportation such as walking, cycling, taxis, buses, and trains.

[0177] "Means for calculating the travel time for each route" refers to the means for calculating the travel time for the generated routes.

[0178] "Means for selecting the optimal route" refers to methods for selecting the optimal route based on criteria such as travel time, cost, and comfort.

[0179] "Means for notifying the user of the optimal route" refers to the means of notifying the user of the selected optimal route.

[0180] "A means of generating efficient routes for food delivery using real-time data and location information from transportation systems and notifying delivery personnel of the optimal route" refers to a means of generating the optimal route for food delivery based on real-time data and location information and notifying delivery personnel of that route.

[0181] Modes for carrying out the invention

[0182] The present invention is a system that generates an optimal route using real-time transportation data and location information to improve the efficiency of food delivery and notifies delivery personnel of it. The embodiments thereof will be described in detail below.

[0183] System Configuration

[0184] The system of this invention consists of a server and a user terminal (the delivery person's smartphone).

[0185] 1. Data acquisition methods

[0186] The server uses APIs to collect real-time data and location information from various modes of transportation. Specific examples of APIs include bus data APIs, train data APIs, shared bicycle data APIs, and taxi data APIs.

[0187] 2. Data Analysis Methods

[0188] The server analyzes the collected real-time data using AI algorithms (e.g., TensorFlow or PyTorch) to predict traffic conditions. The analysis results are then used to generate routes.

[0189] 3. Route generation means

[0190] Based on the analyzed data, the server generates an efficient delivery route by combining multiple modes of transportation. This route includes walking, cycling, taxis, and public transport (buses, trains).

[0191] 4. Travel time calculation method

[0192] The server calculates the travel time for each generated route. This calculation takes into account real-time traffic conditions and forecast data.

[0193] 5. Optimal Route Selection Method

[0194] The server selects the optimal route from multiple options based on evaluation criteria such as travel time, cost, and comfort.

[0195] 6. User notification means

[0196] The server notifies the delivery person's terminal of the selected optimal route. Upon receiving the notification, the delivery person efficiently carries out deliveries according to the optimal route.

[0197] Hardware and software configuration to be used

[0198] Flask: A lightweight, Python-based web framework that handles communication between the server and the user's terminal, as well as data retrieval and analysis.

[0199] Requests library: A library for sending HTTP requests to various APIs.

[0200] AI algorithms (TensorFlow, PyTorch): Used in the data analysis portion to predict traffic conditions and impute missing data.

[0201] Specific example

[0202] For example, consider a case where a delivery person is making a delivery from Shinjuku Station to Roppongi Hills.

[0203] 1. Enter your starting point and destination.

[0204] The delivery person enters the starting point "Shinjuku Station" and the destination "Roppongi Hills" into a smartphone app.

[0205] 2. Collection of real-time data

[0206] The server collects real-time data on buses, trains, taxis, and shared bicycles.

[0207] 3. Analysis of traffic conditions

[0208] The server uses AI algorithms to analyze and predict traffic conditions based on the collected data.

[0209] 4. Route generation

[0210] The server generates multiple routes based on the analysis results. For example, a route using the subway, a route combining bicycles and trains, etc.

[0211] 5. Selection of the optimal route

[0212] The server selects the optimal route from the generated routes, taking into account travel time and comfort.

[0213] 6. Route notification

[0214] The optimal route is notified to the delivery person's smartphone. The delivery person follows that route and makes deliveries efficiently.

[0215] Example of a prompt

[0216] The starting point is Shinjuku Station, and the destination is Roppongi Hills. Please suggest the best route considering the current traffic conditions. Options include train, bus, taxi, bicycle, or walking. Please also provide the estimated travel time and cost for each route.

[0217] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0218] Step 1:

[0219] The user enters their departure point and destination into their device (smartphone). This input data is sent to the server. The entered data includes the departure point, destination, and departure time.

[0220] Step 2:

[0221] The server collects real-time data on transportation. Specifically, it uses various APIs for buses, trains, shared bicycles, taxis, etc., to obtain timetable data and real-time location information. This provides data on the current traffic situation.

[0222] Step 3:

[0223] The server analyzes the collected data and predicts real-time traffic conditions. This analysis uses AI algorithms (e.g., TensorFlow and PyTorch) to fill in missing information in timetables and predict traffic conditions. The input here is the collected real-time data, and the output is the analysis results.

[0224] Step 4:

[0225] The server generates a route combining multiple modes of transportation based on the analysis results. This route generation includes modes of transport such as walking, cycling, taxi, bus, and train. Specifically, it generates multiple route paths from the starting point to the destination and selects a mode of transport for each intermediate step. The input is the analysis results, and the output is multiple route options.

[0226] Step 5:

[0227] The server calculates the travel time for each route. This calculation takes into account real-time traffic conditions and forecast data, such as traffic congestion and bus delays. The input is the generated route options, and the output is the travel time for each route.

[0228] Step 6:

[0229] The server selects the optimal route from multiple options. It determines the best route based on evaluation criteria such as travel time, cost, and comfort. Specifically, it scores the evaluation criteria for each route and selects the route with the highest overall score. The input is evaluation data such as travel time and cost for each route, and the output is the optimal route.

[0230] Step 7:

[0231] The server notifies the user's terminal of the selected optimal route. The terminal displays the received route information to the delivery person. The delivery person uses this information to make deliveries along the optimal route. The input is the selected optimal route, and the output is the notification to the terminal.

[0232] As a concrete example of how it works, the user inputs the starting point "Shinjuku Station" and the destination "Roppongi Hills." The server collects real-time data and analyzes it using an AI algorithm. It generates multiple routes and calculates the travel time for each. It selects the optimal route and notifies the user's terminal. In this way, delivery drivers can perform deliveries efficiently.

[0233] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0234] The system of the present invention includes means for data collection, analysis, route generation, travel time calculation, optimal route selection, and notification of the optimal route to the user, as well as an emotion engine that recognizes the user's emotions. This system proposes the optimal mode of transportation by collecting and analyzing timetable data from various modes of transport, as well as operation records and location information from shared bicycles, taxis, and on-demand buses. Furthermore, the emotion engine can evaluate the user's emotional state and adjust the route based on that evaluation.

[0235] Program Processing Overview

[0236] Data acquisition methods

[0237] The server collects timetable data from various transportation services via APIs. It also acquires real-time operational information from railway and bus companies. In addition, it collects real-time location information for shared bicycles and taxis.

[0238] emotion recognition means

[0239] The device collects user voice input and facial recognition data and sends it to the emotion engine. The emotion engine analyzes this data and evaluates the user's emotional state (stress, relaxation, tension, etc.).

[0240] Data analysis means

[0241] Based on the collected data, the server uses AI algorithms to analyze it. This includes filling in missing information in timetables, predicting real-time traffic conditions, and estimating travel times for shared bicycles and taxis.

[0242] Route generation means

[0243] Based on the analysis results, the server generates route options that combine multiple modes of transportation. These include walking, shared bicycles, taxis, on-demand buses, trains, and regular buses.

[0244] Travel time calculation means

[0245] The travel time for each route is added up, and the average travel time is calculated by taking real-time traffic conditions into account. In addition, multiple different route and mode of transport options are calculated simultaneously.

[0246] Optimal route selection method

[0247] The server selects the optimal route from the generated route options, taking into account travel time, cost, comfort, and the user's emotional state. The evaluation criteria are weighted, and the AI ​​performs an overall scoring.

[0248] User notification means

[0249] The server notifies the terminal of the optimal route and estimated travel time. Based on the information provided, the user can choose the best route according to their preferences.

[0250] Specific example

[0251] For example, consider the following scenario of traveling from Tokyo Station to Haneda Airport.

[0252] 1. The user enters "Tokyo Station" as the departure point, "Haneda Airport" as the destination, and "10:00" as the departure time into the terminal. The terminal also collects the user's voice and facial recognition data and sends it to the emotion engine.

[0253] 2. The server collects data from railway companies, taxi companies, and shared bicycle providers and predicts the travel time for each route.

[0254] 3. The server generates the following route options:

[0255] Route A: Travel from Tokyo Station to Shinagawa Station by train → From Shinagawa Station, take the Keikyu Line to Haneda Airport.

[0256] Route B: Take a taxi from Tokyo Station to Hamamatsucho Station → Take the Tokyo Monorail from Hamamatsucho Station to Haneda Airport

[0257] Route C: Use a shared bicycle from Tokyo Station → Take a train from the nearest station to Haneda Airport.

[0258] 4. The emotion engine evaluates the user's emotional state (e.g., high stress) and prioritizes selecting routes that cause less stress.

[0259] 5. The server selects the optimal route and notifies the terminal of the estimated travel time.

[0260] 6. The user selects a route and begins to travel.

[0261] Through the above processing, the system of the present invention proposes efficient and accurate means of transportation and routes to the user, and further provides a comfortable travel experience based on the user's emotional state.

[0262] The following describes the processing flow.

[0263] Step 1:

[0264] The user enters their departure point, destination, and desired departure or arrival time into the terminal. For example, they might enter "Tokyo Station" as the departure point, "Haneda Airport" as the destination, and "10:00" as the departure time.

[0265] Step 2:

[0266] The terminal receives input and sends that data to the server. The data sent includes the departure point, destination, and departure time.

[0267] Step 3:

[0268] The device collects user voice input and facial recognition data. This uses the smartphone's microphone and camera. The collected data is sent to an emotion engine.

[0269] Step 4:

[0270] The server uses APIs from various transportation providers to collect timetable data and real-time service information. This includes information from railway companies, bus companies, shared bicycle providers, taxi companies, and others. The collected data is stored in a database.

[0271] Step 5:

[0272] The server analyzes the collected data. Using AI algorithms, it fills in missing information in timetables, predicts real-time traffic conditions, and estimates travel times for shared bicycles and taxis.

[0273] Step 6:

[0274] The emotion engine analyzes voice input and facial recognition data transmitted from the device to evaluate the user's emotional state. For example, it determines stress levels and relaxation levels from changes in the user's voice tone and facial expressions.

[0275] Step 7:

[0276] Based on the analysis results, the server generates route options that combine multiple means of transportation. This includes walking, shared cycling, taxis, demand buses, railways, buses, etc.

[0277] Step 8:

[0278] The server calculates the travel time for each route. The required time for each segment of each means of transportation is summed up to calculate the overall travel time. The delay risk considering real-time traffic conditions is also evaluated.

[0279] Step 9:

[0280] The server selects the optimal route from the generated route options, considering travel time, cost, comfort, and the user's emotional state. For example, if the user is feeling stressed, a more relaxing route is preferentially selected.

[0281] Step 10:

[0282] The server notifies the terminal of the selected optimal route and the predicted travel time. The content of the notification includes detailed information about each route, travel time, cost, waiting time, etc.

[0283] Step 11:

[0284] The user checks the notified route options and selects the optimal route according to their preferences. The data after selection is fed back to the server through the terminal.

[0285] Step 12:

[0286] Based on the user's selection information, the server reflects it as additional data in the AI algorithm for improving the accuracy of the next route generation and evaluation process. Through the feedback loop, the performance of the entire system is improved.

[0287] (Example 2)

[0288] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0289] Conventional transportation suggestion systems propose optimal routes based on transportation data, but they fail to consider the user's emotional state, making it difficult to reduce stress or improve comfort. Furthermore, they are unable to respond to missing timetable information or real-time changes in traffic conditions. As a result, the user's travel experience is not always optimal.

[0290] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0291] In this invention, the server includes means for collecting data from various modes of transportation, means for recognizing the user's emotional state, means for analyzing the collected data, means for generating routes based on multiple modes of transportation, means for calculating the travel time for each route, means for selecting the optimal route considering the user's emotional state, and means for notifying the user of the optimal route. This makes it possible to propose the optimal travel route considering the user's emotional state, thereby reducing stress and improving comfort. Furthermore, it becomes possible to fill in gaps in timetables based on real-time information and accurately predict travel times.

[0292] "Means of collecting data" refers to functions that acquire timetable data, real-time operation information, and location information from various transportation services.

[0293] "Means for recognizing the user's emotional state" refers to a function that collects user voice data and facial recognition data, analyzes it, and evaluates the user's emotional state.

[0294] "Means for analyzing collected data" refers to a function that uses AI algorithms to analyze collected traffic data, supplementing missing information in timetables and predicting real-time traffic conditions.

[0295] "Means for generating routes based on multiple modes of transportation" refers to a function that generates route options combining multiple modes of transportation, such as walking, shared bicycles, taxis, trains, and buses, based on analyzed data.

[0296] "Means for calculating travel time for each route" refers to a function that calculates the accurate travel time for each mode of transport by summing up the travel times for each generated route option and taking real-time traffic conditions into account.

[0297] "Means of selecting the optimal route while considering the user's emotional state" refers to a function that evaluates the user's emotional state (stress, relaxation, etc.) and, based on that, selects the optimal route while considering travel time, cost, and comfort.

[0298] "Means of notifying users of the optimal route" refers to a function that notifies the user's device of the selected optimal route and travel time, allowing the user to select the optimal route based on that information.

[0299] The system of this invention operates in a coordinated manner involving a server, a terminal, and a user. The following describes specific implementations of this system.

[0300] Data collection

[0301] The server implements APIs to collect necessary data from various modes of transportation. This includes timetable data from railway and bus companies, real-time service information, location data for shared bicycles, and real-time location data for taxis. This data is stored in the server's database and used for later analysis.

[0302] emotion recognition

[0303] The terminal has built-in sensors and an analysis engine for recognizing the user's emotional state. Voice input is obtained through the built-in microphone, and face recognition data is obtained through the built-in camera. These data are sent to the emotion engine to evaluate the user's emotional state (such as stress, relaxation, etc.).

[0304] Data Analysis

[0305] The server analyzes the collected traffic data using AI algorithms. Specifically, it complements the missing information in the timetable and predicts the real-time traffic situation. Machine learning frameworks such as Python and TensorFlow are used for this.

[0306] Route Generation

[0307] Based on the analysis results, the server generates route options that combine multiple means of transportation. Combinations of walking, shared bicycles, taxis, and public transportation (such as trains and buses) are considered.

[0308] Travel Time Calculation

[0309] For each route, the server calculates the travel time. This calculation also includes real-time traffic situation data. An algorithm that integrates the time required for each segment is used for the total calculation of travel time.

[0310] Optimal Route Selection

[0311] The server selects the optimal route considering the user's emotional state. Travel time, cost, and comfort are the evaluation criteria, and weighting is performed so that the route with the least stress based on the data from the emotion engine is prioritized.

[0312] User Notification

[0313] The server notifies the terminal of the optimal route and its travel time. Based on the notified route guidance, the user selects the optimal route according to their preferences and situation and starts moving.

[0314] Specific example

[0315] For example, consider the following scenario when thinking about travel from Tokyo Station to Haneda Airport.

[0316] 1. The user enters "Tokyo Station" as the departure point, "Haneda Airport" as the destination, and "10:00" as the departure time into the terminal. The terminal also collects the user's voice and facial recognition data and sends it to the emotion engine.

[0317] 2. The server collects data from transportation companies via APIs and stores timetable data and real-time location information in a database.

[0318] 3. The server uses AI algorithms to analyze the collected data, fill in missing information, and predict real-time traffic conditions.

[0319] 4. The server generates multiple route options, specifically the following possible routes:

[0320] Route A: Travel from Tokyo Station to Shinagawa Station by train → From Shinagawa Station, take the Keikyu Line to Haneda Airport.

[0321] Route B: Take a taxi from Tokyo Station to Hamamatsucho Station → Take the Tokyo Monorail from Hamamatsucho Station to Haneda Airport

[0322] Route C: Use a shared bicycle from Tokyo Station → Take a train from the nearest station to Haneda Airport.

[0323] 5. The server calculates the travel time for each route, taking real-time traffic data into consideration.

[0324] 6. The emotion engine evaluates the user's emotional state (e.g., high stress) and weights the routes that result in less stress.

[0325] 7. The server selects the optimal route and notifies the terminal along with the estimated travel time. The user then selects the most suitable route based on this information and begins their journey.

[0326] Example of a prompt

[0327] "I need to travel from Tokyo Station to Haneda Airport, departing at 10:00 AM. My current emotional state is high due to stress. Please suggest the best route."

[0328] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0329] Program processing flow

[0330] Step 1:

[0331] The server collects data from various modes of transportation. Specifically, it uses APIs from railway and bus companies to collect timetable data, real-time service information, and location information for shared bicycles and taxis. This data is sent to the server in JSON format and stored in a database.

[0332] Input: Timetable data, real-time service information, and location information from railway and bus company APIs.

[0333] Output: Traffic data stored in the database

[0334] Step 2:

[0335] The device collects user emotion data. Specifically, it acquires voice data using the device's built-in microphone and captures facial recognition data with its camera. This data is then sent to the emotion engine.

[0336] Input: User voice data, facial recognition data

[0337] Output: Data sent to the emotion engine

[0338] Step 3:

[0339] The server analyzes the collected traffic data using AI algorithms. Specifically, it uses machine learning frameworks such as Python and TensorFlow to fill in missing information in timetables and predict real-time traffic conditions.

[0340] Input: Traffic data stored in the database

[0341] Output: Timetable data with missing information filled in, real-time traffic forecast

[0342] Step 4:

[0343] The server generates route options that combine multiple modes of transportation based on the analysis results. Specifically, it uses an algorithm that generates routes that combine walking, shared bicycles, taxis, and public transport (trains, buses, etc.).

[0344] Input: Timetable data with missing data imputed, real-time traffic forecast.

[0345] Output: Multiple route options

[0346] Step 5:

[0347] The server calculates the travel time for each route. Specifically, for each route, it adds up the walking time, the time spent using shared bicycles, the time spent using taxis, and the time spent using public transportation, and calculates the total travel time taking into account real-time traffic conditions.

[0348] Input: Multiple route options, real-time traffic forecast

[0349] Output: Travel time for each route

[0350] Step 6:

[0351] The server selects the optimal route by considering the user's emotional state. Specifically, it uses data from the emotion engine to weight travel time, cost, and comfort as evaluation criteria.

[0352] Input: Travel time for each route, user sentiment data

[0353] Output: Optimal Route

[0354] Step 7:

[0355] The server notifies the terminal of the optimal route and its estimated travel time. The user then selects the route best suited to them based on the provided route guidance.

[0356] Input: Optimal route

[0357] Output: Route information notified to the terminal

[0358] (Application Example 2)

[0359] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0360] Conventional travel route suggestion systems provide the optimal route considering real-time traffic conditions and multiple modes of transport, but they fail to take into account the user's emotional state, which can lead to an uncomfortable travel experience. Furthermore, when selecting the optimal route with multiple travel options, the inability to adjust based on the user's emotions can result in stress and discomfort. Therefore, a challenge for conventional technologies is the ability to provide flexible route suggestions that take the user's emotional state into account.

[0361] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a data collection means, a means for analyzing the collected data, a means for generating a route based on multiple means of transportation, a means for calculating the travel time for each route, a means for selecting the optimal route, a means for notifying the user of the optimal route, a means for recognizing and evaluating the user's emotional state, and a means for adjusting the selected route based on the recognized emotional state. This makes it possible to propose an optimal route that takes the user's emotional state into consideration, allowing the user to travel more comfortably.

[0362] "Data collection methods" refer to means for obtaining timetable data for various modes of transportation, real-time operation information, and location information for shared bicycles and taxis.

[0363] "Means for analyzing collected data" refers to methods for predicting traffic conditions and travel times based on the acquired data.

[0364] "Means for generating routes based on multiple modes of transportation" refers to methods for constructing routes that combine various modes of transportation such as walking, shared bicycles, taxis, on-demand buses, trains, and buses.

[0365] "Means for calculating the travel time for each route" refers to means for calculating the time required to travel along each generated route.

[0366] "Means of selecting the optimal route" refers to methods of choosing the most appropriate route, taking into account travel time, cost, comfort, and the user's emotional state.

[0367] "Means of notifying users of the optimal route" refers to means of informing users of the selected optimal route and its estimated travel time.

[0368] "Means for recognizing and evaluating the user's emotional state" refers to a means of determining the emotional state using an emotion engine based on the user's voice input or facial recognition data.

[0369] "Means for adjusting the selected route based on recognized emotional states" refers to means for optimizing the selected route according to the user's emotional state, such as stress, relaxation, or tension.

[0370] The system in this invention includes means for collecting data, means for analyzing the collected data, means for generating routes based on multiple means of transportation, means for calculating the travel time for each route, means for selecting the optimal route, means for notifying the user of the optimal route, means for recognizing and evaluating the user's emotional state, and means for adjusting the selected route based on the recognized emotional state.

[0371] The server uses transportation APIs to collect timetable data and real-time service information, as well as location data for shared bicycles and taxis. This includes data from railway companies, bus companies, shared bicycle providers, and taxi companies. Specific software used includes the Python requests library and the pandas library for data analysis.

[0372] The device collects user voice input and facial recognition data and sends it to the emotion engine. The smartphone's camera and microphone are primarily used, and the EmotionEngine is employed to analyze emotions. Libraries such as OpenCV and dlib are used for user facial recognition.

[0373] The server uses artificial intelligence to fill in missing information in timetables based on collected data and predicts real-time traffic conditions. This allows it to estimate travel times for shared bicycles and taxis. Furthermore, the server generates route options combining multiple modes of transport and calculates the travel time for each route. This process considers various modes of transport, including walking, shared bicycles, taxis, on-demand buses, trains, and buses. TensorFlow and PyTorch are suitable AI algorithms.

[0374] When a user enters their current location and destination into the device, the device analyzes the user's emotional state and suggests a route accordingly, such as when stress levels are high. This involves EmotionEngine evaluating the emotional state, and the server selecting the optimal route considering the user's emotional state. A comprehensive scoring algorithm that takes emotional analysis data into account is used to optimize the route based on the emotional state.

[0375] For example, if a user wants to travel from Tokyo Station to Shibuya Station, and their current emotional state is assessed as relaxed, the optimal route will be selected as travel by train. In this case, the server will use the following prompt:

[0376] "Please suggest the optimal route from Tokyo Station to Shibuya Station. The user's emotional state is currently assessed as relaxed. Based on real-time traffic conditions and the location of the autonomous taxi, calculate the best travel option."

[0377] In this way, the system can take into account the user's emotional state and suggest the optimal mode of transportation and route. This allows the user to reach their destination comfortably and efficiently.

[0378] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0379] Step 1:

[0380] The user enters their starting point and destination into the device. The device also collects the user's facial recognition data and voice data using the smartphone's camera and microphone. The input data consists of current location information and destination information from GPS, while the collected data consists of facial images and voice data.

[0381] Step 2:

[0382] The device sends the collected facial recognition data and voice data to the emotion engine to determine the user's emotional state. The emotion engine performs facial expression analysis and voice tone analysis to evaluate the user's emotional state (e.g., stress, relaxation, tension). The input is facial recognition data and voice data, and the output is the evaluated emotional state.

[0383] Step 3:

[0384] The server collects timetable data, real-time service information, and location information for shared bicycles and taxis through APIs of various transportation services. The requests library is used to access the API endpoints of transportation services. The input is the API endpoint information of the transportation service, and the output is the collected timetable data and real-time service information.

[0385] Step 4:

[0386] The server analyzes the collected data, fills in missing information, and predicts real-time traffic conditions. This analysis uses artificial intelligence algorithms to fill in missing information in timetables and predict traffic. The input is various types of collected traffic data, and the output is the completed timetable data and predicted traffic conditions.

[0387] Step 5:

[0388] Based on the analysis results, the server generates route options combining multiple modes of transportation. This route generation uses an AI algorithm that considers various modes of transport, including walking, shared bicycles, taxis, on-demand buses, trains, and regular buses. Inputs are supplemented timetable data and predicted traffic conditions, while output is multiple route options.

[0389] Step 6:

[0390] The server calculates the travel time for each route and selects the optimal route, taking into account the user's emotional state. Using an evaluation algorithm, it scores travel time, cost, comfort, and emotional state, and selects the route with the highest score. The input is multiple route options and the user's emotional state, and the output is the optimal route.

[0391] Step 7:

[0392] The server notifies the terminal of the optimal route and estimated travel time. The terminal displays the notified information to the user and provides an interface for the user to make a selection. The input is the optimal route and estimated travel time, and the output is the display information that the user can confirm.

[0393] Step 8:

[0394] The user begins moving according to the notified optimal route. The device continuously acquires real-time location information and recalculates and notifies the user of the route as needed. The input is the user's current location information, and the output is the updated route information.

[0395] By following the processing steps described above, this invention realizes a system that provides an optimal travel route that takes into account the user's emotional state, enabling the user to reach their destination comfortably and efficiently.

[0396] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0397] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0398] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0399] [Second Embodiment]

[0400] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0401] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0402] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0403] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0404] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0405] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0406] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0407] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0408] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0409] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0410] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0411] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0412] The system of the present invention includes means for data collection, means for analysis, means for route generation, means for calculating travel time, means for selecting the optimal route, and means for notifying the user of the optimal route. This system proposes the optimal means of transportation by collecting and analyzing timetable data for various modes of transport, as well as operation records and location information for shared bicycles, taxis, on-demand buses, etc.

[0413] Program Processing Overview

[0414] Data acquisition methods

[0415] The server collects timetable data from various transportation services via APIs. It also acquires real-time operational information from railway and bus companies. In addition, it collects real-time location information for shared bicycles and taxis.

[0416] Data analysis means

[0417] Based on the collected data, the server uses AI algorithms to analyze it. This includes filling in missing information in timetables, predicting real-time traffic conditions, and estimating travel times for shared bicycles and taxis.

[0418] Route generation means

[0419] Based on the analysis results, the server generates route options that combine multiple modes of transportation. These routes include walking, shared bicycles, taxis, on-demand buses, trains, and buses.

[0420] Travel time calculation means

[0421] The travel time for each route is added up, and the average travel time is calculated by taking real-time traffic conditions into account. In addition, multiple different route and mode of transport options are calculated simultaneously.

[0422] Optimal route selection method

[0423] The server selects the optimal route from the generated route options based on evaluation criteria such as travel time, cost, and comfort. This results in the most efficient route being suggested to the user.

[0424] User notification means

[0425] The server notifies the terminal of the optimal route and estimated travel time. Based on the information provided, the user can choose the best route according to their preferences.

[0426] Specific example

[0427] For example, let's consider a scenario involving travel from Tokyo Station to Haneda Airport.

[0428] 1. The user enters "Tokyo Station" as the departure point, "Haneda Airport" as the destination, and "10:00" as the departure time into the terminal.

[0429] 2. The server collects data from railway companies, taxi companies, and shared bicycle providers and predicts the travel time for each route.

[0430] 3. The server generates the following route options:

[0431] Route A: Travel from Tokyo Station to Shinagawa Station by train → From Shinagawa Station, take the Keikyu Line to Haneda Airport.

[0432] Route B: Take a taxi from Tokyo Station to Hamamatsucho Station → Take the Tokyo Monorail from Hamamatsucho Station to Haneda Airport

[0433] Route C: Use a shared bicycle from Tokyo Station → Take a train from the nearest station to Haneda Airport.

[0434] 4. The server evaluates the travel time, cost, and comfort of each route option and selects the optimal route.

[0435] 5. The server notifies the terminal of the selected optimal route and estimated travel time.

[0436] 6. The user selects a route and begins to travel.

[0437] Through the above processing, the system of the present invention proposes efficient and accurate means of transportation and routes to the user.

[0438] The following describes the processing flow.

[0439] Step 1:

[0440] The user enters their departure point, destination, and desired departure or arrival time into the terminal. For example, they might enter "Tokyo Station" as the departure point, "Haneda Airport" as the destination, and "10:00" as the departure time.

[0441] Step 2:

[0442] The terminal receives input and sends that data to the server. The data sent includes the departure point, destination, and departure time.

[0443] Step 3:

[0444] The server uses APIs from various transportation providers to collect timetable data and real-time service information. This includes information from railway companies, bus companies, shared bicycle providers, taxi companies, and others. The collected data is stored in a database.

[0445] Step 4:

[0446] The server analyzes the collected data. Using AI algorithms, it fills in missing information in timetables and predicts real-time traffic conditions. It also estimates travel times for shared bicycles and taxis.

[0447] Step 5:

[0448] The server generates route options that combine multiple modes of transportation. These include walking, shared bicycles, taxis, on-demand buses, trains, and buses. For example, it can generate multiple routes to get from Tokyo Station to Haneda Airport.

[0449] Step 6:

[0450] The server calculates the travel time for each route. It then sums up the segmented travel times for each mode of transport to calculate the total travel time. It also evaluates delay risk, taking real-time traffic conditions into account.

[0451] Step 7:

[0452] The server selects the optimal route from the generated route options based on evaluation criteria such as travel time, cost, and comfort. Each evaluation criterion is weighted, and the AI ​​performs an overall scoring.

[0453] Step 8:

[0454] The server notifies the terminal of the optimal route selected by the server and the estimated travel time. The notification includes detailed information about each route, such as travel time, cost, and waiting time.

[0455] Step 9:

[0456] The user reviews the route options they are notified of and selects the best route according to their preferences. The selected route is then fed back to the server via the device.

[0457] Step 10:

[0458] The server uses user selection information to provide additional data to the AI ​​algorithm, improving the accuracy of future route generation and evaluation processes. This feedback loop enhances the overall system performance.

[0459] (Example 1)

[0460] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0461] Conventional transportation systems struggled to effectively combine multiple modes of transport to suggest the optimal route, and even more so to select routes that took real-time traffic conditions into account. As a result, users experienced inconvenience during their journeys and were unable to travel efficiently.

[0462] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0463] In this invention, the server includes means for collecting timetable data, real-time operation information, and location information; means for analyzing the collected data using artificial intelligence to fill in gaps in timetable information and predict real-time traffic conditions; and means for generating routes that combine multiple means of transportation. This enables users to travel efficiently and quickly.

[0464] "Timetable data" refers to information about the operating schedules of public transportation.

[0465] "Real-time service information" refers to information that shows the current location of transportation services and how much of a delay or progress they are making compared to their scheduled service.

[0466] "Location information" refers to information that indicates the current location of a particular mode of transportation (e.g., shared bicycles, taxis, etc.).

[0467] Artificial intelligence is a technology that enables computer systems to learn on their own through data analysis and pattern recognition, and to automatically perform specific tasks.

[0468] "Means of transportation" is a general term encompassing various methods of getting around, such as trains, buses, taxis, shared bicycles, and walking.

[0469] A "route" is a series of paths from a starting point to a destination.

[0470] "Travel time" refers to the time it takes to travel from a starting point to a destination using a specific route.

[0471] "Expenses" refer to the amount of money required to use a particular mode of transportation.

[0472] "Comfort" refers to the degree of ride quality and convenience experienced during travel.

[0473] A "user terminal" is an electronic device used by a user to receive information.

[0474] "Data collection means" refers to the functions and methods for obtaining necessary information from transportation systems.

[0475] "Analysis means" refers to the functions and methods for processing and analyzing collected data.

[0476] "Generation means" refers to functions and methods for creating routes that combine multiple modes of transportation based on analyzed data.

[0477] "Notification means" refers to functions and methods for informing the user's terminal of selected routes and other information.

[0478] The system of the present invention includes data collection means, analysis means, route generation means, travel time calculation means, optimal route selection means, and user notification means. Specific embodiments of each of these means are described below.

[0479] Data acquisition methods

[0480] The server collects timetable data, real-time service information, and location information from various transportation services via APIs. For example, it uses APIs from railway and bus companies to obtain timetables and service status in real time. It also collects location information from shared bicycles and taxis using the same method. This allows all necessary data to be managed centrally.

[0481] Data analysis means

[0482] The server analyzes the collected data using artificial intelligence (AI). Here, machine learning models and deep learning algorithms are utilized to fill in missing information in timetables, predict real-time traffic conditions, and estimate travel times for various modes of transportation. Specific software used includes TensorFlow and PyTorch.

[0483] Route generation means

[0484] The server generates a route combining multiple modes of transportation based on the analysis results. This route may include walking, trains, buses, shared bicycles, taxis, and on-demand buses. The combination of each mode of transportation will vary depending on the user's starting point and destination.

[0485] Travel time calculation means

[0486] The server adds up the travel times for each route and calculates the average travel time, taking real-time traffic conditions into account. For example, it calculates a route that takes 10 minutes on foot, 30 minutes by train, and 20 minutes by taxi, and then adjusts for traffic congestion to predict an accurate travel time.

[0487] Optimal route selection method

[0488] The server selects the optimal route from the generated routes based on evaluation criteria such as travel time, cost, and comfort. The optimal route is designed to maximize the user's travel efficiency, and the algorithm compares and selects routes using multiple evaluation criteria.

[0489] User notification means

[0490] The server notifies the user's device of the optimal route and estimated travel time. Based on this information, the user selects a route that suits their preferences and begins their journey. This notification is delivered via a mobile app or web application.

[0491] Specific example

[0492] Scenario: Travel from Tokyo Station to Haneda Airport

[0493] 1. The user enters the departure point "Tokyo Station", destination "Haneda Airport", and departure time "10:00" into the terminal.

[0494] 2. The server collects data from railway companies, taxi companies, and shared bicycle providers and predicts the travel time for each route.

[0495] 3. The server generates the following route options:

[0496] Route A: Travel from Tokyo Station to Shinagawa Station by train → From Shinagawa Station to Haneda Airport by train.

[0497] Route B: Take a taxi from Tokyo Station to Hamamatsucho Station → Take the monorail from Hamamatsucho Station to Haneda Airport

[0498] Route C: Use a shared bicycle from Tokyo Station → Take a train from the nearest station to Haneda Airport.

[0499] 4. The server evaluates the travel time, cost, and comfort of each route option and selects the optimal route.

[0500] 5. The server notifies the terminal of the optimal route and estimated travel time.

[0501] 6. The user selects a route and begins to travel.

[0502] Examples of prompts for generative AI models

[0503] "Please suggest the best route from Tokyo Station to Haneda Airport. Available modes of transport include train, bus, taxi, and shared bicycle."

[0504] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0505] Step 1:

[0506] The server uses transportation APIs to collect timetable data, real-time service information, and location information for shared bicycles and taxis.

[0507] Input: API endpoints for each transportation provider.

[0508] Processing: Call the API to retrieve timetable data, real-time service information, and location information.

[0509] Output: Collected raw data (timetables, service status, location information).

[0510] Step 2:

[0511] The server passes the collected data to an artificial intelligence (AI) algorithm for data analysis.

[0512] Input: Raw data collected in Step 1.

[0513] Processing: Machine learning models and deep learning algorithms are used to fill in missing information in timetables, predict real-time traffic conditions, and estimate travel times for shared bicycles and taxis.

[0514] Output: Interpolated timetable data, predicted traffic conditions, and estimated travel time.

[0515] Step 3:

[0516] The server generates a route that combines multiple modes of transportation based on the analysis results.

[0517] Input: Analysis result data generated in Step 2.

[0518] Processing: Create the optimal route option by combining various modes of transportation such as walking, trains, buses, shared bicycles, taxis, and on-demand buses.

[0519] Output: Multiple route options (e.g., Route A, Route B, Route C).

[0520] Step 4:

[0521] The server calculates the travel time for each route and then calculates the average travel time, taking real-time traffic conditions into account.

[0522] Input: Route options generated in Step 3.

[0523] Processing: The travel time for each mode of transport along the route is added together, and the average travel time is calculated taking real-time traffic conditions into account.

[0524] Output: Average travel time for each route.

[0525] Step 5:

[0526] The server selects the optimal route based on evaluation criteria such as travel time, cost, and comfort.

[0527] Input: Average travel time data for each route generated in Step 4.

[0528] Processing: Evaluate the travel time, cost, and comfort of each route, and select the optimal route overall.

[0529] Output: Optimal route.

[0530] Step 6:

[0531] The server notifies the terminal of the selected optimal route and estimated travel time.

[0532] Input: The optimal route selected in Step 5.

[0533] Processing: Convert the optimal route and estimated travel time into a data format for user notifications.

[0534] Output: Notification data sent to the user's terminal (optimal route, estimated travel time).

[0535] Step 7:

[0536] The device displays the notified optimal route and estimated travel time to the user.

[0537] Input: Notification data sent from Step 6.

[0538] Processing: Display notification data on the device's screen interface.

[0539] Output: Presenting the user with the optimal route and estimated travel time.

[0540] Step 8:

[0541] The user begins their journey to their destination based on the displayed optimal route and estimated travel time.

[0542] Input: The optimal route and estimated travel time displayed on the device.

[0543] Process: The user reviews the information and begins moving according to the selected route.

[0544] Output: Actual movement begins.

[0545] (Application Example 1)

[0546] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0547] Traditional food delivery systems did not effectively utilize real-time traffic and location information to optimize delivery routes, resulting in decreased delivery efficiency and longer delivery times. Furthermore, their inability to adapt to sudden changes in traffic conditions led to delivery delays and increased costs.

[0548] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0549] In this invention, the server includes means for collecting data, means for analyzing the collected data, means for generating routes based on multiple means of transportation, means for calculating the travel time for each route, means for selecting the optimal route, means for notifying the user of the optimal route, and means for generating efficient food delivery routes using real-time transportation data and location information and notifying delivery personnel of the optimal route. This significantly improves delivery efficiency, shortens delivery times, and enables rapid response to sudden changes in traffic conditions.

[0550] "Data collection methods" refer to means for acquiring real-time data and location information from various modes of transportation.

[0551] "Means for analyzing collected data" refers to methods for predicting and analyzing real-time traffic conditions based on collected data.

[0552] "Means for generating routes based on multiple modes of transportation" refers to methods for generating routes by combining multiple modes of transportation such as walking, cycling, taxis, buses, and trains.

[0553] "Means for calculating the travel time for each route" refers to the means for calculating the travel time for the generated routes.

[0554] "Means for selecting the optimal route" refers to methods for selecting the optimal route based on criteria such as travel time, cost, and comfort.

[0555] "Means for notifying the user of the optimal route" refers to the means of notifying the user of the selected optimal route.

[0556] "A means of generating efficient routes for food delivery using real-time data and location information from transportation systems and notifying delivery personnel of the optimal route" refers to a means of generating the optimal route for food delivery based on real-time data and location information and notifying delivery personnel of that route.

[0557] Modes for carrying out the invention

[0558] The present invention is a system that generates an optimal route using real-time transportation data and location information to improve the efficiency of food delivery and notifies delivery personnel of it. The embodiments thereof will be described in detail below.

[0559] System Configuration

[0560] The system of this invention consists of a server and a user terminal (the delivery person's smartphone).

[0561] 1. Data acquisition methods

[0562] The server uses APIs to collect real-time data and location information from various modes of transportation. Specific examples of APIs include bus data APIs, train data APIs, shared bicycle data APIs, and taxi data APIs.

[0563] 2. Data Analysis Methods

[0564] The server analyzes the collected real-time data using AI algorithms (e.g., TensorFlow or PyTorch) to predict traffic conditions. The analysis results are then used to generate routes.

[0565] 3. Route generation means

[0566] Based on the analyzed data, the server generates an efficient delivery route by combining multiple modes of transportation. This route includes walking, cycling, taxis, and public transport (buses, trains).

[0567] 4. Travel time calculation method

[0568] The server calculates the travel time for each generated route. This calculation takes into account real-time traffic conditions and forecast data.

[0569] 5. Optimal Route Selection Method

[0570] The server selects the optimal route from multiple options based on evaluation criteria such as travel time, cost, and comfort.

[0571] 6. User notification means

[0572] The server notifies the delivery person's terminal of the selected optimal route. Upon receiving the notification, the delivery person efficiently carries out deliveries according to the optimal route.

[0573] Hardware and software configuration to be used

[0574] Flask: A lightweight, Python-based web framework that handles communication between the server and the user's terminal, as well as data retrieval and analysis.

[0575] Requests library: A library for sending HTTP requests to various APIs.

[0576] AI algorithms (TensorFlow, PyTorch): Used in the data analysis portion to predict traffic conditions and impute missing data.

[0577] Specific example

[0578] For example, consider a case where a delivery person is making a delivery from Shinjuku Station to Roppongi Hills.

[0579] 1. Enter your starting point and destination.

[0580] The delivery person enters the starting point "Shinjuku Station" and the destination "Roppongi Hills" into a smartphone app.

[0581] 2. Collection of real-time data

[0582] The server collects real-time data on buses, trains, taxis, and shared bicycles.

[0583] 3. Analysis of traffic conditions

[0584] The server uses AI algorithms to analyze and predict traffic conditions based on the collected data.

[0585] 4. Route generation

[0586] The server generates multiple routes based on the analysis results. For example, a route using the subway, a route combining bicycles and trains, etc.

[0587] 5. Selection of the optimal route

[0588] The server selects the optimal route from the generated routes, taking into account travel time and comfort.

[0589] 6. Route notification

[0590] The optimal route is notified to the delivery person's smartphone. The delivery person follows that route and makes deliveries efficiently.

[0591] Example of a prompt

[0592] The starting point is Shinjuku Station, and the destination is Roppongi Hills. Please suggest the best route considering the current traffic conditions. Options include train, bus, taxi, bicycle, or walking. Please also provide the estimated travel time and cost for each route.

[0593] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0594] Step 1:

[0595] The user enters their departure point and destination into their device (smartphone). This input data is sent to the server. The entered data includes the departure point, destination, and departure time.

[0596] Step 2:

[0597] The server collects real-time data on transportation. Specifically, it uses various APIs for buses, trains, shared bicycles, taxis, etc., to obtain timetable data and real-time location information. This provides data on the current traffic situation.

[0598] Step 3:

[0599] The server analyzes the collected data and predicts real-time traffic conditions. This analysis uses AI algorithms (e.g., TensorFlow and PyTorch) to fill in missing information in timetables and predict traffic conditions. The input here is the collected real-time data, and the output is the analysis results.

[0600] Step 4:

[0601] The server generates a route combining multiple modes of transportation based on the analysis results. This route generation includes modes of transport such as walking, cycling, taxi, bus, and train. Specifically, it generates multiple route paths from the starting point to the destination and selects a mode of transport for each intermediate step. The input is the analysis results, and the output is multiple route options.

[0602] Step 5:

[0603] The server calculates the travel time for each route. This calculation takes into account real-time traffic conditions and forecast data, such as traffic congestion and bus delays. The input is the generated route options, and the output is the travel time for each route.

[0604] Step 6:

[0605] The server selects the optimal route from multiple options. It determines the best route based on evaluation criteria such as travel time, cost, and comfort. Specifically, it scores the evaluation criteria for each route and selects the route with the highest overall score. The input is evaluation data such as travel time and cost for each route, and the output is the optimal route.

[0606] Step 7:

[0607] The server notifies the user's terminal of the selected optimal route. The terminal displays the received route information to the delivery person. The delivery person uses this information to make deliveries along the optimal route. The input is the selected optimal route, and the output is the notification to the terminal.

[0608] As a concrete example of how it works, the user inputs the starting point "Shinjuku Station" and the destination "Roppongi Hills." The server collects real-time data and analyzes it using an AI algorithm. It generates multiple routes and calculates the travel time for each. It selects the optimal route and notifies the user's terminal. In this way, delivery drivers can perform deliveries efficiently.

[0609] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0610] The system of the present invention includes means for data collection, analysis, route generation, travel time calculation, optimal route selection, and notification of the optimal route to the user, as well as an emotion engine that recognizes the user's emotions. This system proposes the optimal mode of transportation by collecting and analyzing timetable data from various modes of transport, as well as operation records and location information from shared bicycles, taxis, and on-demand buses. Furthermore, the emotion engine can evaluate the user's emotional state and adjust the route based on that evaluation.

[0611] Program Processing Overview

[0612] Data acquisition methods

[0613] The server collects timetable data from various transportation services via APIs. It also acquires real-time operational information from railway and bus companies. In addition, it collects real-time location information for shared bicycles and taxis.

[0614] emotion recognition means

[0615] The device collects user voice input and facial recognition data and sends it to the emotion engine. The emotion engine analyzes this data and evaluates the user's emotional state (stress, relaxation, tension, etc.).

[0616] Data analysis means

[0617] Based on the collected data, the server uses AI algorithms to analyze it. This includes filling in missing information in timetables, predicting real-time traffic conditions, and estimating travel times for shared bicycles and taxis.

[0618] Route generation means

[0619] Based on the analysis results, the server generates route options that combine multiple modes of transportation. These include walking, shared bicycles, taxis, on-demand buses, trains, and regular buses.

[0620] Travel time calculation means

[0621] The travel time for each route is added up, and the average travel time is calculated by taking real-time traffic conditions into account. In addition, multiple different route and mode of transport options are calculated simultaneously.

[0622] Optimal route selection method

[0623] The server selects the optimal route from the generated route options, taking into account travel time, cost, comfort, and the user's emotional state. The evaluation criteria are weighted, and the AI ​​performs an overall scoring.

[0624] User notification means

[0625] The server notifies the terminal of the optimal route and estimated travel time. Based on the information provided, the user can choose the best route according to their preferences.

[0626] Specific example

[0627] For example, consider the following scenario of traveling from Tokyo Station to Haneda Airport.

[0628] 1. The user enters "Tokyo Station" as the departure point, "Haneda Airport" as the destination, and "10:00" as the departure time into the terminal. The terminal also collects the user's voice and facial recognition data and sends it to the emotion engine.

[0629] 2. The server collects data from railway companies, taxi companies, and shared bicycle providers and predicts the travel time for each route.

[0630] 3. The server generates the following route options:

[0631] Route A: Travel from Tokyo Station to Shinagawa Station by train → From Shinagawa Station, take the Keikyu Line to Haneda Airport.

[0632] Route B: Take a taxi from Tokyo Station to Hamamatsucho Station → Take the Tokyo Monorail from Hamamatsucho Station to Haneda Airport

[0633] Route C: Use a shared bicycle from Tokyo Station → Take a train from the nearest station to Haneda Airport.

[0634] 4. The emotion engine evaluates the user's emotional state (e.g., high stress) and prioritizes selecting routes that cause less stress.

[0635] 5. The server selects the optimal route and notifies the terminal of the estimated travel time.

[0636] 6. The user selects a route and begins to travel.

[0637] Through the above processing, the system of the present invention proposes efficient and accurate means of transportation and routes to the user, and further provides a comfortable travel experience based on the user's emotional state.

[0638] The following describes the processing flow.

[0639] Step 1:

[0640] The user enters their departure point, destination, and desired departure or arrival time into the terminal. For example, they might enter "Tokyo Station" as the departure point, "Haneda Airport" as the destination, and "10:00" as the departure time.

[0641] Step 2:

[0642] The terminal receives input and sends that data to the server. The data sent includes the departure point, destination, and departure time.

[0643] Step 3:

[0644] The device collects user voice input and facial recognition data. This uses the smartphone's microphone and camera. The collected data is sent to an emotion engine.

[0645] Step 4:

[0646] The server uses APIs from various transportation providers to collect timetable data and real-time service information. This includes information from railway companies, bus companies, shared bicycle providers, taxi companies, and others. The collected data is stored in a database.

[0647] Step 5:

[0648] The server analyzes the collected data. Using AI algorithms, it fills in missing information in timetables, predicts real-time traffic conditions, and estimates travel times for shared bicycles and taxis.

[0649] Step 6:

[0650] The emotion engine analyzes voice input and facial recognition data transmitted from the device to evaluate the user's emotional state. For example, it determines stress levels and relaxation levels from changes in the user's voice tone and facial expressions.

[0651] Step 7:

[0652] The server generates route options that combine multiple modes of transportation based on the analysis results. These include walking, shared bicycles, taxis, on-demand buses, trains, and buses.

[0653] Step 8:

[0654] The server calculates the travel time for each route. It then sums up the segmented travel times for each mode of transport to calculate the total travel time. It also evaluates delay risk, taking real-time traffic conditions into account.

[0655] Step 9:

[0656] The server selects the optimal route from the generated route options, taking into account travel time, cost, comfort, and the user's emotional state. For example, if the user is feeling stressed, it will prioritize a more relaxing route.

[0657] Step 10:

[0658] The server notifies the terminal of the optimal route selected by the server and the estimated travel time. The notification includes detailed information about each route, such as travel time, cost, and waiting time.

[0659] Step 11:

[0660] The user reviews the route options they are notified of and selects the best route according to their preferences. The selected route is then fed back to the server via the device.

[0661] Step 12:

[0662] The server uses user selection information to incorporate it into AI algorithms as additional data, improving the accuracy of future route generation and evaluation processes. This feedback loop improves the overall system performance.

[0663] (Example 2)

[0664] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0665] Conventional transportation suggestion systems propose optimal routes based on transportation data, but they fail to consider the user's emotional state, making it difficult to reduce stress or improve comfort. Furthermore, they are unable to respond to missing timetable information or real-time changes in traffic conditions. As a result, the user's travel experience is not always optimal.

[0666] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0667] In this invention, the server includes means for collecting data from various modes of transportation, means for recognizing the user's emotional state, means for analyzing the collected data, means for generating routes based on multiple modes of transportation, means for calculating the travel time for each route, means for selecting the optimal route considering the user's emotional state, and means for notifying the user of the optimal route. This makes it possible to propose the optimal travel route considering the user's emotional state, thereby reducing stress and improving comfort. Furthermore, it becomes possible to fill in gaps in timetables based on real-time information and accurately predict travel times.

[0668] "Means of collecting data" refers to functions that acquire timetable data, real-time operation information, and location information from various transportation services.

[0669] "Means for recognizing the user's emotional state" refers to a function that collects user voice data and facial recognition data, analyzes it, and evaluates the user's emotional state.

[0670] "Means for analyzing collected data" refers to a function that uses AI algorithms to analyze collected traffic data, supplementing missing information in timetables and predicting real-time traffic conditions.

[0671] "Means for generating routes based on multiple modes of transportation" refers to a function that generates route options combining multiple modes of transportation, such as walking, shared bicycles, taxis, trains, and buses, based on analyzed data.

[0672] "Means for calculating travel time for each route" refers to a function that calculates the accurate travel time for each mode of transport by summing up the travel times for each generated route option and taking real-time traffic conditions into account.

[0673] "Means of selecting the optimal route while considering the user's emotional state" refers to a function that evaluates the user's emotional state (stress, relaxation, etc.) and, based on that, selects the optimal route while considering travel time, cost, and comfort.

[0674] "Means of notifying users of the optimal route" refers to a function that notifies the user's device of the selected optimal route and travel time, allowing the user to select the optimal route based on that information.

[0675] The system of this invention operates in a coordinated manner involving a server, a terminal, and a user. The following describes specific implementations of this system.

[0676] Data collection

[0677] The server implements APIs to collect necessary data from various modes of transportation. This includes timetable data from railway and bus companies, real-time service information, location data for shared bicycles, and real-time location data for taxis. This data is stored in the server's database and used for later analysis.

[0678] emotion recognition

[0679] The device incorporates sensors and an analysis engine to recognize the user's emotional state. Voice input is acquired via the built-in microphone, and facial recognition data is acquired via the built-in camera. This data is sent to the emotion engine to evaluate the user's emotional state (stress, relaxation, etc.).

[0680] Data Analysis

[0681] The server analyzes the collected traffic data using AI algorithms. Specifically, it performs tasks such as filling in missing information in timetables and predicting real-time traffic conditions. Machine learning frameworks such as Python and TensorFlow are used for this purpose.

[0682] Route generation

[0683] The server generates route options that combine multiple modes of transportation based on the analysis results. Combinations of walking, shared bicycles, taxis, and public transport (trains, buses, etc.) are considered.

[0684] Travel time calculation

[0685] For each route, the server calculates the travel time. This calculation includes real-time traffic data. An algorithm is used to aggregate the travel times by combining the duration of each segment.

[0686] Optimal Route Selection

[0687] The server selects the optimal route while considering the user's emotional state. Travel time, cost, and comfort are evaluated, and weighting is applied based on data from the emotion engine to prioritize the least stressful route.

[0688] User notifications

[0689] The server notifies the terminal of the optimal route and estimated travel time. Based on the notified route guidance, the user selects the best route according to their preferences and circumstances and begins their journey.

[0690] Specific example

[0691] For example, consider the following scenario when thinking about travel from Tokyo Station to Haneda Airport.

[0692] 1. The user enters "Tokyo Station" as the departure point, "Haneda Airport" as the destination, and "10:00" as the departure time into the terminal. The terminal also collects the user's voice and facial recognition data and sends it to the emotion engine.

[0693] 2. The server collects data from transportation companies via APIs and stores timetable data and real-time location information in a database.

[0694] 3. The server uses AI algorithms to analyze the collected data, fill in missing information, and predict real-time traffic conditions.

[0695] 4. The server generates multiple route options, specifically the following possible routes:

[0696] Route A: Travel from Tokyo Station to Shinagawa Station by train → From Shinagawa Station, take the Keikyu Line to Haneda Airport.

[0697] Route B: Take a taxi from Tokyo Station to Hamamatsucho Station → Take the Tokyo Monorail from Hamamatsucho Station to Haneda Airport

[0698] Route C: Use a shared bicycle from Tokyo Station → Take a train from the nearest station to Haneda Airport.

[0699] 5. The server calculates the travel time for each route, taking real-time traffic data into consideration.

[0700] 6. The emotion engine evaluates the user's emotional state (e.g., high stress) and weights the routes that result in less stress.

[0701] 7. The server selects the optimal route and notifies the terminal along with the estimated travel time. The user then selects the most suitable route based on this information and begins their journey.

[0702] Example of a prompt

[0703] "I need to travel from Tokyo Station to Haneda Airport, departing at 10:00 AM. My current emotional state is high due to stress. Please suggest the best route."

[0704] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0705] Program processing flow

[0706] Step 1:

[0707] The server collects data from various modes of transportation. Specifically, it uses APIs from railway and bus companies to collect timetable data, real-time service information, and location information for shared bicycles and taxis. This data is sent to the server in JSON format and stored in a database.

[0708] Input: Timetable data, real-time service information, and location information from railway and bus company APIs.

[0709] Output: Traffic data stored in the database

[0710] Step 2:

[0711] The device collects user emotion data. Specifically, it acquires voice data using the device's built-in microphone and captures facial recognition data with its camera. This data is then sent to the emotion engine.

[0712] Input: User voice data, facial recognition data

[0713] Output: Data sent to the emotion engine

[0714] Step 3:

[0715] The server analyzes the collected traffic data using AI algorithms. Specifically, it uses machine learning frameworks such as Python and TensorFlow to fill in missing information in timetables and predict real-time traffic conditions.

[0716] Input: Traffic data stored in the database

[0717] Output: Timetable data with missing information filled in, real-time traffic forecast

[0718] Step 4:

[0719] The server generates route options that combine multiple modes of transportation based on the analysis results. Specifically, it uses an algorithm that generates routes that combine walking, shared bicycles, taxis, and public transport (trains, buses, etc.).

[0720] Input: Timetable data with missing data imputed, real-time traffic forecast.

[0721] Output: Multiple route options

[0722] Step 5:

[0723] The server calculates the travel time for each route. Specifically, for each route, it adds up the walking time, the time spent using shared bicycles, the time spent using taxis, and the time spent using public transportation, and calculates the total travel time taking into account real-time traffic conditions.

[0724] Input: Multiple route options, real-time traffic forecast

[0725] Output: Travel time for each route

[0726] Step 6:

[0727] The server selects the optimal route by considering the user's emotional state. Specifically, it uses data from the emotion engine to weight travel time, cost, and comfort as evaluation criteria.

[0728] Input: Travel time for each route, user sentiment data

[0729] Output: Optimal Route

[0730] Step 7:

[0731] The server notifies the terminal of the optimal route and its estimated travel time. The user then selects the route best suited to them based on the provided route guidance.

[0732] Input: Optimal route

[0733] Output: Route information notified to the terminal

[0734] (Application Example 2)

[0735] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0736] Conventional travel route suggestion systems provide the optimal route considering real-time traffic conditions and multiple modes of transport, but they fail to take into account the user's emotional state, which can lead to an uncomfortable travel experience. Furthermore, when selecting the optimal route with multiple travel options, the inability to adjust based on the user's emotions can result in stress and discomfort. Therefore, a challenge for conventional technologies is the ability to provide flexible route suggestions that take the user's emotional state into account.

[0737] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a data collection means, a means for analyzing the collected data, a means for generating a route based on multiple means of transportation, a means for calculating the travel time for each route, a means for selecting the optimal route, a means for notifying the user of the optimal route, a means for recognizing and evaluating the user's emotional state, and a means for adjusting the selected route based on the recognized emotional state. This makes it possible to propose an optimal route that takes the user's emotional state into consideration, allowing the user to travel more comfortably.

[0738] "Data collection methods" refer to means for obtaining timetable data for various modes of transportation, real-time operation information, and location information for shared bicycles and taxis.

[0739] "Means for analyzing collected data" refers to methods for predicting traffic conditions and travel times based on the acquired data.

[0740] "Means for generating routes based on multiple modes of transportation" refers to methods for constructing routes that combine various modes of transportation such as walking, shared bicycles, taxis, on-demand buses, trains, and buses.

[0741] "Means for calculating the travel time for each route" refers to means for calculating the time required to travel along each generated route.

[0742] "Means of selecting the optimal route" refers to methods of choosing the most appropriate route, taking into account travel time, cost, comfort, and the user's emotional state.

[0743] "Means of notifying users of the optimal route" refers to means of informing users of the selected optimal route and its estimated travel time.

[0744] "Means for recognizing and evaluating the user's emotional state" refers to a means of determining the emotional state using an emotion engine based on the user's voice input or facial recognition data.

[0745] "Means for adjusting the selected route based on recognized emotional states" refers to means for optimizing the selected route according to the user's emotional state, such as stress, relaxation, or tension.

[0746] The system in this invention includes means for collecting data, means for analyzing the collected data, means for generating routes based on multiple means of transportation, means for calculating the travel time for each route, means for selecting the optimal route, means for notifying the user of the optimal route, means for recognizing and evaluating the user's emotional state, and means for adjusting the selected route based on the recognized emotional state.

[0747] The server uses transportation APIs to collect timetable data and real-time service information, as well as location data for shared bicycles and taxis. This includes data from railway companies, bus companies, shared bicycle providers, and taxi companies. Specific software used includes the Python requests library and the pandas library for data analysis.

[0748] The device collects user voice input and facial recognition data and sends it to the emotion engine. The smartphone's camera and microphone are primarily used, and the EmotionEngine is employed to analyze emotions. Libraries such as OpenCV and dlib are used for user facial recognition.

[0749] The server uses artificial intelligence to fill in missing information in timetables based on collected data and predicts real-time traffic conditions. This allows it to estimate travel times for shared bicycles and taxis. Furthermore, the server generates route options combining multiple modes of transport and calculates the travel time for each route. This process considers various modes of transport, including walking, shared bicycles, taxis, on-demand buses, trains, and buses. TensorFlow and PyTorch are suitable AI algorithms.

[0750] When a user enters their current location and destination into the device, the device analyzes the user's emotional state and suggests a route accordingly, such as when stress levels are high. This involves EmotionEngine evaluating the emotional state, and the server selecting the optimal route considering the user's emotional state. A comprehensive scoring algorithm that takes emotional analysis data into account is used to optimize the route based on the emotional state.

[0751] For example, if a user wants to travel from Tokyo Station to Shibuya Station, and their current emotional state is assessed as relaxed, the optimal route will be selected as travel by train. In this case, the server will use the following prompt:

[0752] "Please suggest the optimal route from Tokyo Station to Shibuya Station. The user's emotional state is currently assessed as relaxed. Based on real-time traffic conditions and the location of the autonomous taxi, calculate the best travel option."

[0753] In this way, the system can take into account the user's emotional state and suggest the optimal mode of transportation and route. This allows the user to reach their destination comfortably and efficiently.

[0754] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0755] Step 1:

[0756] The user enters their starting point and destination into the device. The device also collects the user's facial recognition data and voice data using the smartphone's camera and microphone. The input data consists of current location information and destination information from GPS, while the collected data consists of facial images and voice data.

[0757] Step 2:

[0758] The device sends the collected facial recognition data and voice data to the emotion engine to determine the user's emotional state. The emotion engine performs facial expression analysis and voice tone analysis to evaluate the user's emotional state (e.g., stress, relaxation, tension). The input is facial recognition data and voice data, and the output is the evaluated emotional state.

[0759] Step 3:

[0760] The server collects timetable data, real-time service information, and location information for shared bicycles and taxis through APIs of various transportation services. The requests library is used to access the API endpoints of transportation services. The input is the API endpoint information of the transportation service, and the output is the collected timetable data and real-time service information.

[0761] Step 4:

[0762] The server analyzes the collected data, fills in missing information, and predicts real-time traffic conditions. This analysis uses artificial intelligence algorithms to fill in missing information in timetables and predict traffic. The input is various types of collected traffic data, and the output is the completed timetable data and predicted traffic conditions.

[0763] Step 5:

[0764] Based on the analysis results, the server generates route options combining multiple modes of transportation. This route generation uses an AI algorithm that considers various modes of transport, including walking, shared bicycles, taxis, on-demand buses, trains, and regular buses. Inputs are supplemented timetable data and predicted traffic conditions, while output is multiple route options.

[0765] Step 6:

[0766] The server calculates the travel time for each route and selects the optimal route, taking into account the user's emotional state. Using an evaluation algorithm, it scores travel time, cost, comfort, and emotional state, and selects the route with the highest score. The input is multiple route options and the user's emotional state, and the output is the optimal route.

[0767] Step 7:

[0768] The server notifies the terminal of the optimal route and estimated travel time. The terminal displays the notified information to the user and provides an interface for the user to make a selection. The input is the optimal route and estimated travel time, and the output is the display information that the user can confirm.

[0769] Step 8:

[0770] The user begins moving according to the notified optimal route. The device continuously acquires real-time location information and recalculates and notifies the user of the route as needed. The input is the user's current location information, and the output is the updated route information.

[0771] By following the processing steps described above, this invention realizes a system that provides an optimal travel route that takes into account the user's emotional state, enabling the user to reach their destination comfortably and efficiently.

[0772] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0773] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0774] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0775] [Third Embodiment]

[0776] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0777] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0778] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0779] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0780] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0781] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0782] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0783] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0784] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0785] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0786] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0787] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0788] The system of the present invention includes means for data collection, means for analysis, means for route generation, means for calculating travel time, means for selecting the optimal route, and means for notifying the user of the optimal route. This system proposes the optimal means of transportation by collecting and analyzing timetable data for various modes of transport, as well as operation records and location information for shared bicycles, taxis, on-demand buses, etc.

[0789] Program Processing Overview

[0790] Data acquisition methods

[0791] The server collects timetable data from various transportation services via APIs. It also acquires real-time operational information from railway and bus companies. In addition, it collects real-time location information for shared bicycles and taxis.

[0792] Data analysis means

[0793] Based on the collected data, the server uses AI algorithms to analyze it. This includes filling in missing information in timetables, predicting real-time traffic conditions, and estimating travel times for shared bicycles and taxis.

[0794] Route generation means

[0795] Based on the analysis results, the server generates route options that combine multiple modes of transportation. These routes include walking, shared bicycles, taxis, on-demand buses, trains, and buses.

[0796] Travel time calculation means

[0797] The travel time for each route is added up, and the average travel time is calculated by taking real-time traffic conditions into account. In addition, multiple different route and mode of transport options are calculated simultaneously.

[0798] Optimal route selection method

[0799] The server selects the optimal route from the generated route options based on evaluation criteria such as travel time, cost, and comfort. This results in the most efficient route being suggested to the user.

[0800] User notification means

[0801] The server notifies the terminal of the optimal route and estimated travel time. Based on the information provided, the user can choose the best route according to their preferences.

[0802] Specific example

[0803] For example, let's consider a scenario involving travel from Tokyo Station to Haneda Airport.

[0804] 1. The user enters "Tokyo Station" as the departure point, "Haneda Airport" as the destination, and "10:00" as the departure time into the terminal.

[0805] 2. The server collects data from railway companies, taxi companies, and shared bicycle providers and predicts the travel time for each route.

[0806] 3. The server generates the following route options:

[0807] Route A: Travel from Tokyo Station to Shinagawa Station by train → From Shinagawa Station, take the Keikyu Line to Haneda Airport.

[0808] Route B: Take a taxi from Tokyo Station to Hamamatsucho Station → Take the Tokyo Monorail from Hamamatsucho Station to Haneda Airport

[0809] Route C: Use a shared bicycle from Tokyo Station → Take a train from the nearest station to Haneda Airport.

[0810] 4. The server evaluates the travel time, cost, and comfort of each route option and selects the optimal route.

[0811] 5. The server notifies the terminal of the selected optimal route and estimated travel time.

[0812] 6. The user selects a route and begins to travel.

[0813] Through the above processing, the system of the present invention proposes efficient and accurate means of transportation and routes to the user.

[0814] The following describes the processing flow.

[0815] Step 1:

[0816] The user enters their departure point, destination, and desired departure or arrival time into the terminal. For example, they might enter "Tokyo Station" as the departure point, "Haneda Airport" as the destination, and "10:00" as the departure time.

[0817] Step 2:

[0818] The terminal receives input and sends that data to the server. The data sent includes the departure point, destination, and departure time.

[0819] Step 3:

[0820] The server uses APIs from various transportation providers to collect timetable data and real-time service information. This includes information from railway companies, bus companies, shared bicycle providers, taxi companies, and others. The collected data is stored in a database.

[0821] Step 4:

[0822] The server analyzes the collected data. Using AI algorithms, it fills in missing information in timetables and predicts real-time traffic conditions. It also estimates travel times for shared bicycles and taxis.

[0823] Step 5:

[0824] The server generates route options that combine multiple modes of transportation. These include walking, shared bicycles, taxis, on-demand buses, trains, and buses. For example, it can generate multiple routes to get from Tokyo Station to Haneda Airport.

[0825] Step 6:

[0826] The server calculates the travel time for each route. It then sums up the segmented travel times for each mode of transport to calculate the total travel time. It also evaluates delay risk, taking real-time traffic conditions into account.

[0827] Step 7:

[0828] The server selects the optimal route from the generated route options based on evaluation criteria such as travel time, cost, and comfort. Each evaluation criterion is weighted, and the AI ​​performs an overall scoring.

[0829] Step 8:

[0830] The server notifies the terminal of the optimal route selected by the server and the estimated travel time. The notification includes detailed information about each route, such as travel time, cost, and waiting time.

[0831] Step 9:

[0832] The user reviews the route options they are notified of and selects the best route according to their preferences. The selected route is then fed back to the server via the device.

[0833] Step 10:

[0834] The server uses user selection information to provide additional data to the AI ​​algorithm, improving the accuracy of future route generation and evaluation processes. This feedback loop enhances the overall system performance.

[0835] (Example 1)

[0836] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0837] Conventional transportation systems struggled to effectively combine multiple modes of transport to suggest the optimal route, and even more so to select routes that took real-time traffic conditions into account. As a result, users experienced inconvenience during their journeys and were unable to travel efficiently.

[0838] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0839] In this invention, the server includes means for collecting timetable data, real-time operation information, and location information; means for analyzing the collected data using artificial intelligence to fill in gaps in timetable information and predict real-time traffic conditions; and means for generating routes that combine multiple means of transportation. This enables users to travel efficiently and quickly.

[0840] "Timetable data" refers to information about the operating schedules of public transportation.

[0841] "Real-time service information" refers to information that shows the current location of transportation services and how much of a delay or progress they are making compared to their scheduled service.

[0842] "Location information" refers to information that indicates the current location of a particular mode of transportation (e.g., shared bicycles, taxis, etc.).

[0843] Artificial intelligence is a technology that enables computer systems to learn on their own through data analysis and pattern recognition, and to automatically perform specific tasks.

[0844] "Means of transportation" is a general term encompassing various methods of getting around, such as trains, buses, taxis, shared bicycles, and walking.

[0845] A "route" is a series of paths from a starting point to a destination.

[0846] "Travel time" refers to the time it takes to travel from a starting point to a destination using a specific route.

[0847] "Expenses" refer to the amount of money required to use a particular mode of transportation.

[0848] "Comfort" refers to the degree of ride quality and convenience experienced during travel.

[0849] A "user terminal" is an electronic device used by a user to receive information.

[0850] "Data collection means" refers to the functions and methods for obtaining necessary information from transportation systems.

[0851] "Analysis means" refers to the functions and methods for processing and analyzing collected data.

[0852] "Generation means" refers to functions and methods for creating routes that combine multiple modes of transportation based on analyzed data.

[0853] "Notification means" refers to functions and methods for informing the user's terminal of selected routes and other information.

[0854] The system of the present invention includes data collection means, analysis means, route generation means, travel time calculation means, optimal route selection means, and user notification means. Specific embodiments of each of these means are described below.

[0855] Data acquisition methods

[0856] The server collects timetable data, real-time service information, and location information from various transportation services via APIs. For example, it uses APIs from railway and bus companies to obtain timetables and service status in real time. It also collects location information from shared bicycles and taxis using the same method. This allows all necessary data to be managed centrally.

[0857] Data analysis means

[0858] The server analyzes the collected data using artificial intelligence (AI). Here, machine learning models and deep learning algorithms are utilized to fill in missing information in timetables, predict real-time traffic conditions, and estimate travel times for various modes of transportation. Specific software used includes TensorFlow and PyTorch.

[0859] Route generation means

[0860] The server generates a route combining multiple modes of transportation based on the analysis results. This route may include walking, trains, buses, shared bicycles, taxis, and on-demand buses. The combination of each mode of transportation will vary depending on the user's starting point and destination.

[0861] Travel time calculation means

[0862] The server adds up the travel times for each route and calculates the average travel time, taking real-time traffic conditions into account. For example, it calculates a route that takes 10 minutes on foot, 30 minutes by train, and 20 minutes by taxi, and then adjusts for traffic congestion to predict an accurate travel time.

[0863] Optimal route selection method

[0864] The server selects the optimal route from the generated routes based on evaluation criteria such as travel time, cost, and comfort. The optimal route is designed to maximize the user's travel efficiency, and the algorithm compares and selects routes using multiple evaluation criteria.

[0865] User notification means

[0866] The server notifies the user's device of the optimal route and estimated travel time. Based on this information, the user selects a route that suits their preferences and begins their journey. This notification is delivered via a mobile app or web application.

[0867] Specific example

[0868] Scenario: Travel from Tokyo Station to Haneda Airport

[0869] 1. The user enters the departure point "Tokyo Station", destination "Haneda Airport", and departure time "10:00" into the terminal.

[0870] 2. The server collects data from railway companies, taxi companies, and shared bicycle providers and predicts the travel time for each route.

[0871] 3. The server generates the following route options:

[0872] Route A: Travel from Tokyo Station to Shinagawa Station by train → From Shinagawa Station to Haneda Airport by train.

[0873] Route B: Take a taxi from Tokyo Station to Hamamatsucho Station → Take the monorail from Hamamatsucho Station to Haneda Airport

[0874] Route C: Use a shared bicycle from Tokyo Station → Take a train from the nearest station to Haneda Airport.

[0875] 4. The server evaluates the travel time, cost, and comfort of each route option and selects the optimal route.

[0876] 5. The server notifies the terminal of the optimal route and estimated travel time.

[0877] 6. The user selects a route and begins to travel.

[0878] Examples of prompts for generative AI models

[0879] "Please suggest the best route from Tokyo Station to Haneda Airport. Available modes of transport include train, bus, taxi, and shared bicycle."

[0880] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0881] Step 1:

[0882] The server uses transportation APIs to collect timetable data, real-time service information, and location information for shared bicycles and taxis.

[0883] Input: API endpoints for each transportation provider.

[0884] Processing: Call the API to retrieve timetable data, real-time service information, and location information.

[0885] Output: Collected raw data (timetables, service status, location information).

[0886] Step 2:

[0887] The server passes the collected data to an artificial intelligence (AI) algorithm for data analysis.

[0888] Input: Raw data collected in Step 1.

[0889] Processing: Machine learning models and deep learning algorithms are used to fill in missing information in timetables, predict real-time traffic conditions, and estimate travel times for shared bicycles and taxis.

[0890] Output: Interpolated timetable data, predicted traffic conditions, and estimated travel time.

[0891] Step 3:

[0892] The server generates a route that combines multiple modes of transportation based on the analysis results.

[0893] Input: Analysis result data generated in Step 2.

[0894] Processing: Create the optimal route option by combining various modes of transportation such as walking, trains, buses, shared bicycles, taxis, and on-demand buses.

[0895] Output: Multiple route options (e.g., Route A, Route B, Route C).

[0896] Step 4:

[0897] The server calculates the travel time for each route and then calculates the average travel time, taking real-time traffic conditions into account.

[0898] Input: Route options generated in Step 3.

[0899] Processing: The travel time for each mode of transport along the route is added together, and the average travel time is calculated taking real-time traffic conditions into account.

[0900] Output: Average travel time for each route.

[0901] Step 5:

[0902] The server selects the optimal route based on evaluation criteria such as travel time, cost, and comfort.

[0903] Input: Average travel time data for each route generated in Step 4.

[0904] Processing: Evaluate the travel time, cost, and comfort of each route, and select the optimal route overall.

[0905] Output: Optimal route.

[0906] Step 6:

[0907] The server notifies the terminal of the selected optimal route and estimated travel time.

[0908] Input: The optimal route selected in Step 5.

[0909] Processing: Convert the optimal route and estimated travel time into a data format for user notifications.

[0910] Output: Notification data sent to the user's terminal (optimal route, estimated travel time).

[0911] Step 7:

[0912] The device displays the notified optimal route and estimated travel time to the user.

[0913] Input: Notification data sent from Step 6.

[0914] Processing: Display notification data on the device's screen interface.

[0915] Output: Presenting the user with the optimal route and estimated travel time.

[0916] Step 8:

[0917] The user begins their journey to their destination based on the displayed optimal route and estimated travel time.

[0918] Input: The optimal route and estimated travel time displayed on the device.

[0919] Process: The user reviews the information and begins moving according to the selected route.

[0920] Output: Actual movement begins.

[0921] (Application Example 1)

[0922] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0923] Traditional food delivery systems did not effectively utilize real-time traffic and location information to optimize delivery routes, resulting in decreased delivery efficiency and longer delivery times. Furthermore, their inability to adapt to sudden changes in traffic conditions led to delivery delays and increased costs.

[0924] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0925] In this invention, the server includes means for collecting data, means for analyzing the collected data, means for generating routes based on multiple means of transportation, means for calculating the travel time for each route, means for selecting the optimal route, means for notifying the user of the optimal route, and means for generating efficient food delivery routes using real-time transportation data and location information and notifying delivery personnel of the optimal route. This significantly improves delivery efficiency, shortens delivery times, and enables rapid response to sudden changes in traffic conditions.

[0926] "Data collection methods" refer to means for acquiring real-time data and location information from various modes of transportation.

[0927] "Means for analyzing collected data" refers to methods for predicting and analyzing real-time traffic conditions based on collected data.

[0928] "Means for generating routes based on multiple modes of transportation" refers to methods for generating routes by combining multiple modes of transportation such as walking, cycling, taxis, buses, and trains.

[0929] "Means for calculating the travel time for each route" refers to the means for calculating the travel time for the generated routes.

[0930] "Means for selecting the optimal route" refers to methods for selecting the optimal route based on criteria such as travel time, cost, and comfort.

[0931] "Means for notifying the user of the optimal route" refers to the means of notifying the user of the selected optimal route.

[0932] "A means of generating efficient routes for food delivery using real-time data and location information from transportation systems and notifying delivery personnel of the optimal route" refers to a means of generating the optimal route for food delivery based on real-time data and location information and notifying delivery personnel of that route.

[0933] Modes for carrying out the invention

[0934] The present invention is a system that generates an optimal route using real-time transportation data and location information to improve the efficiency of food delivery and notifies delivery personnel of it. The embodiments thereof will be described in detail below.

[0935] System Configuration

[0936] The system of this invention consists of a server and a user terminal (the delivery person's smartphone).

[0937] 1. Data acquisition methods

[0938] The server uses APIs to collect real-time data and location information from various modes of transportation. Specific examples of APIs include bus data APIs, train data APIs, shared bicycle data APIs, and taxi data APIs.

[0939] 2. Data Analysis Methods

[0940] The server analyzes the collected real-time data using AI algorithms (e.g., TensorFlow or PyTorch) to predict traffic conditions. The analysis results are then used to generate routes.

[0941] 3. Route generation means

[0942] Based on the analyzed data, the server generates an efficient delivery route by combining multiple modes of transportation. This route includes walking, cycling, taxis, and public transport (buses, trains).

[0943] 4. Travel time calculation method

[0944] The server calculates the travel time for each generated route. This calculation takes into account real-time traffic conditions and forecast data.

[0945] 5. Optimal Route Selection Method

[0946] The server selects the optimal route from multiple options based on evaluation criteria such as travel time, cost, and comfort.

[0947] 6. User notification means

[0948] The server notifies the delivery person's terminal of the selected optimal route. Upon receiving the notification, the delivery person efficiently carries out deliveries according to the optimal route.

[0949] Hardware and software configuration to be used

[0950] Flask: A lightweight, Python-based web framework that handles communication between the server and the user's terminal, as well as data retrieval and analysis.

[0951] Requests library: A library for sending HTTP requests to various APIs.

[0952] AI algorithms (TensorFlow, PyTorch): Used in the data analysis portion to predict traffic conditions and impute missing data.

[0953] Specific example

[0954] For example, consider a case where a delivery person is making a delivery from Shinjuku Station to Roppongi Hills.

[0955] 1. Enter your starting point and destination.

[0956] The delivery person enters the starting point "Shinjuku Station" and the destination "Roppongi Hills" into a smartphone app.

[0957] 2. Collection of real-time data

[0958] The server collects real-time data on buses, trains, taxis, and shared bicycles.

[0959] 3. Analysis of traffic conditions

[0960] The server uses AI algorithms to analyze and predict traffic conditions based on the collected data.

[0961] 4. Route generation

[0962] The server generates multiple routes based on the analysis results. For example, a route using the subway, a route combining bicycles and trains, etc.

[0963] 5. Selection of the optimal route

[0964] The server selects the optimal route from the generated routes, taking into account travel time and comfort.

[0965] 6. Route notification

[0966] The optimal route is notified to the delivery person's smartphone. The delivery person follows that route and makes deliveries efficiently.

[0967] Example of a prompt

[0968] The starting point is Shinjuku Station, and the destination is Roppongi Hills. Please suggest the best route considering the current traffic conditions. Options include train, bus, taxi, bicycle, or walking. Please also provide the estimated travel time and cost for each route.

[0969] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0970] Step 1:

[0971] The user enters their departure point and destination into their device (smartphone). This input data is sent to the server. The entered data includes the departure point, destination, and departure time.

[0972] Step 2:

[0973] The server collects real-time data on transportation. Specifically, it uses various APIs for buses, trains, shared bicycles, taxis, etc., to obtain timetable data and real-time location information. This provides data on the current traffic situation.

[0974] Step 3:

[0975] The server analyzes the collected data and predicts real-time traffic conditions. This analysis uses AI algorithms (e.g., TensorFlow and PyTorch) to fill in missing information in timetables and predict traffic conditions. The input here is the collected real-time data, and the output is the analysis results.

[0976] Step 4:

[0977] The server generates a route combining multiple modes of transportation based on the analysis results. This route generation includes modes of transport such as walking, cycling, taxi, bus, and train. Specifically, it generates multiple route paths from the starting point to the destination and selects a mode of transport for each intermediate step. The input is the analysis results, and the output is multiple route options.

[0978] Step 5:

[0979] The server calculates the travel time for each route. This calculation takes into account real-time traffic conditions and forecast data, such as traffic congestion and bus delays. The input is the generated route options, and the output is the travel time for each route.

[0980] Step 6:

[0981] The server selects the optimal route from multiple options. It determines the best route based on evaluation criteria such as travel time, cost, and comfort. Specifically, it scores the evaluation criteria for each route and selects the route with the highest overall score. The input is evaluation data such as travel time and cost for each route, and the output is the optimal route.

[0982] Step 7:

[0983] The server notifies the user's terminal of the selected optimal route. The terminal displays the received route information to the delivery person. The delivery person uses this information to make deliveries along the optimal route. The input is the selected optimal route, and the output is the notification to the terminal.

[0984] As a concrete example of how it works, the user inputs the starting point "Shinjuku Station" and the destination "Roppongi Hills." The server collects real-time data and analyzes it using an AI algorithm. It generates multiple routes and calculates the travel time for each. It selects the optimal route and notifies the user's terminal. In this way, delivery drivers can perform deliveries efficiently.

[0985] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0986] The system of the present invention includes means for data collection, analysis, route generation, travel time calculation, optimal route selection, and notification of the optimal route to the user, as well as an emotion engine that recognizes the user's emotions. This system proposes the optimal mode of transportation by collecting and analyzing timetable data from various modes of transport, as well as operation records and location information from shared bicycles, taxis, and on-demand buses. Furthermore, the emotion engine can evaluate the user's emotional state and adjust the route based on that evaluation.

[0987] Program Processing Overview

[0988] Data acquisition methods

[0989] The server collects timetable data from various transportation services via APIs. It also acquires real-time operational information from railway and bus companies. In addition, it collects real-time location information for shared bicycles and taxis.

[0990] emotion recognition means

[0991] The device collects user voice input and facial recognition data and sends it to the emotion engine. The emotion engine analyzes this data and evaluates the user's emotional state (stress, relaxation, tension, etc.).

[0992] Data analysis means

[0993] Based on the collected data, the server uses AI algorithms to analyze it. This includes filling in missing information in timetables, predicting real-time traffic conditions, and estimating travel times for shared bicycles and taxis.

[0994] Route generation means

[0995] Based on the analysis results, the server generates route options that combine multiple modes of transportation. These include walking, shared bicycles, taxis, on-demand buses, trains, and regular buses.

[0996] Travel time calculation means

[0997] The travel time for each route is added up, and the average travel time is calculated by taking real-time traffic conditions into account. In addition, multiple different route and mode of transport options are calculated simultaneously.

[0998] Optimal route selection method

[0999] The server selects the optimal route from the generated route options, taking into account travel time, cost, comfort, and the user's emotional state. The evaluation criteria are weighted, and the AI ​​performs an overall scoring.

[1000] User notification means

[1001] The server notifies the terminal of the optimal route and estimated travel time. Based on the information provided, the user can choose the best route according to their preferences.

[1002] Specific example

[1003] For example, consider the following scenario of traveling from Tokyo Station to Haneda Airport.

[1004] 1. The user enters "Tokyo Station" as the departure point, "Haneda Airport" as the destination, and "10:00" as the departure time into the terminal. The terminal also collects the user's voice and facial recognition data and sends it to the emotion engine.

[1005] 2. The server collects data from railway companies, taxi companies, and shared bicycle providers and predicts the travel time for each route.

[1006] 3. The server generates the following route options:

[1007] Route A: Travel from Tokyo Station to Shinagawa Station by train → From Shinagawa Station, take the Keikyu Line to Haneda Airport.

[1008] Route B: Take a taxi from Tokyo Station to Hamamatsucho Station → Take the Tokyo Monorail from Hamamatsucho Station to Haneda Airport

[1009] Route C: Use a shared bicycle from Tokyo Station → Take a train from the nearest station to Haneda Airport.

[1010] 4. The emotion engine evaluates the user's emotional state (e.g., high stress) and prioritizes selecting routes that cause less stress.

[1011] 5. The server selects the optimal route and notifies the terminal of the estimated travel time.

[1012] 6. The user selects a route and begins to travel.

[1013] Through the above processing, the system of the present invention proposes efficient and accurate means of transportation and routes to the user, and further provides a comfortable travel experience based on the user's emotional state.

[1014] The following describes the processing flow.

[1015] Step 1:

[1016] The user enters their departure point, destination, and desired departure or arrival time into the terminal. For example, they might enter "Tokyo Station" as the departure point, "Haneda Airport" as the destination, and "10:00" as the departure time.

[1017] Step 2:

[1018] The terminal receives input and sends that data to the server. The data sent includes the departure point, destination, and departure time.

[1019] Step 3:

[1020] The device collects user voice input and facial recognition data. This uses the smartphone's microphone and camera. The collected data is sent to an emotion engine.

[1021] Step 4:

[1022] The server uses APIs from various transportation providers to collect timetable data and real-time service information. This includes information from railway companies, bus companies, shared bicycle providers, taxi companies, and others. The collected data is stored in a database.

[1023] Step 5:

[1024] The server analyzes the collected data. Using AI algorithms, it fills in missing information in timetables, predicts real-time traffic conditions, and estimates travel times for shared bicycles and taxis.

[1025] Step 6:

[1026] The emotion engine analyzes voice input and facial recognition data transmitted from the device to evaluate the user's emotional state. For example, it determines stress levels and relaxation levels from changes in the user's voice tone and facial expressions.

[1027] Step 7:

[1028] The server generates route options that combine multiple modes of transportation based on the analysis results. These include walking, shared bicycles, taxis, on-demand buses, trains, and buses.

[1029] Step 8:

[1030] The server calculates the travel time for each route. It then sums up the segmented travel times for each mode of transport to calculate the total travel time. It also evaluates delay risk, taking real-time traffic conditions into account.

[1031] Step 9:

[1032] The server selects the optimal route from the generated route options, taking into account travel time, cost, comfort, and the user's emotional state. For example, if the user is feeling stressed, it will prioritize a more relaxing route.

[1033] Step 10:

[1034] The server notifies the terminal of the optimal route selected by the server and the estimated travel time. The notification includes detailed information about each route, such as travel time, cost, and waiting time.

[1035] Step 11:

[1036] The user reviews the route options they are notified of and selects the best route according to their preferences. The selected route is then fed back to the server via the device.

[1037] Step 12:

[1038] The server uses user selection information to incorporate it into AI algorithms as additional data, improving the accuracy of future route generation and evaluation processes. This feedback loop improves the overall system performance.

[1039] (Example 2)

[1040] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1041] Conventional transportation suggestion systems propose optimal routes based on transportation data, but they fail to consider the user's emotional state, making it difficult to reduce stress or improve comfort. Furthermore, they are unable to respond to missing timetable information or real-time changes in traffic conditions. As a result, the user's travel experience is not always optimal.

[1042] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1043] In this invention, the server includes means for collecting data from various modes of transportation, means for recognizing the user's emotional state, means for analyzing the collected data, means for generating routes based on multiple modes of transportation, means for calculating the travel time for each route, means for selecting the optimal route considering the user's emotional state, and means for notifying the user of the optimal route. This makes it possible to propose the optimal travel route considering the user's emotional state, thereby reducing stress and improving comfort. Furthermore, it becomes possible to fill in gaps in timetables based on real-time information and accurately predict travel times.

[1044] "Means of collecting data" refers to functions that acquire timetable data, real-time operation information, and location information from various transportation services.

[1045] "Means for recognizing the user's emotional state" refers to a function that collects user voice data and facial recognition data, analyzes it, and evaluates the user's emotional state.

[1046] "Means for analyzing collected data" refers to a function that uses AI algorithms to analyze collected traffic data, supplementing missing information in timetables and predicting real-time traffic conditions.

[1047] "Means for generating routes based on multiple modes of transportation" refers to a function that generates route options combining multiple modes of transportation, such as walking, shared bicycles, taxis, trains, and buses, based on analyzed data.

[1048] "Means for calculating travel time for each route" refers to a function that calculates the accurate travel time for each mode of transport by summing up the travel times for each generated route option and taking real-time traffic conditions into account.

[1049] "Means of selecting the optimal route while considering the user's emotional state" refers to a function that evaluates the user's emotional state (stress, relaxation, etc.) and, based on that, selects the optimal route while considering travel time, cost, and comfort.

[1050] "Means of notifying users of the optimal route" refers to a function that notifies the user's device of the selected optimal route and travel time, allowing the user to select the optimal route based on that information.

[1051] The system of this invention operates in a coordinated manner involving a server, a terminal, and a user. The following describes specific implementations of this system.

[1052] Data collection

[1053] The server implements APIs to collect necessary data from various modes of transportation. This includes timetable data from railway and bus companies, real-time service information, location data for shared bicycles, and real-time location data for taxis. This data is stored in the server's database and used for later analysis.

[1054] emotion recognition

[1055] The device incorporates sensors and an analysis engine to recognize the user's emotional state. Voice input is acquired via the built-in microphone, and facial recognition data is acquired via the built-in camera. This data is sent to the emotion engine to evaluate the user's emotional state (stress, relaxation, etc.).

[1056] Data Analysis

[1057] The server analyzes the collected traffic data using AI algorithms. Specifically, it performs tasks such as filling in missing information in timetables and predicting real-time traffic conditions. Machine learning frameworks such as Python and TensorFlow are used for this purpose.

[1058] Route generation

[1059] The server generates route options that combine multiple modes of transportation based on the analysis results. Combinations of walking, shared bicycles, taxis, and public transport (trains, buses, etc.) are considered.

[1060] Travel time calculation

[1061] For each route, the server calculates the travel time. This calculation includes real-time traffic data. An algorithm is used to aggregate the travel times by combining the duration of each segment.

[1062] Optimal Route Selection

[1063] The server selects the optimal route while considering the user's emotional state. Travel time, cost, and comfort are evaluated, and weighting is applied based on data from the emotion engine to prioritize the least stressful route.

[1064] User notifications

[1065] The server notifies the terminal of the optimal route and estimated travel time. Based on the notified route guidance, the user selects the best route according to their preferences and circumstances and begins their journey.

[1066] Specific example

[1067] For example, consider the following scenario when thinking about travel from Tokyo Station to Haneda Airport.

[1068] 1. The user enters "Tokyo Station" as the departure point, "Haneda Airport" as the destination, and "10:00" as the departure time into the terminal. The terminal also collects the user's voice and facial recognition data and sends it to the emotion engine.

[1069] 2. The server collects data from transportation companies via APIs and stores timetable data and real-time location information in a database.

[1070] 3. The server uses AI algorithms to analyze the collected data, fill in missing information, and predict real-time traffic conditions.

[1071] 4. The server generates multiple route options, specifically the following possible routes:

[1072] Route A: Travel from Tokyo Station to Shinagawa Station by train → From Shinagawa Station, take the Keikyu Line to Haneda Airport.

[1073] Route B: Take a taxi from Tokyo Station to Hamamatsucho Station → Take the Tokyo Monorail from Hamamatsucho Station to Haneda Airport

[1074] Route C: Use a shared bicycle from Tokyo Station → Take a train from the nearest station to Haneda Airport.

[1075] 5. The server calculates the travel time for each route, taking real-time traffic data into consideration.

[1076] 6. The emotion engine evaluates the user's emotional state (e.g., high stress) and weights the routes that result in less stress.

[1077] 7. The server selects the optimal route and notifies the terminal along with the estimated travel time. The user then selects the most suitable route based on this information and begins their journey.

[1078] Example of a prompt

[1079] "I need to travel from Tokyo Station to Haneda Airport, departing at 10:00 AM. My current emotional state is high due to stress. Please suggest the best route."

[1080] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1081] Program processing flow

[1082] Step 1:

[1083] The server collects data from various modes of transportation. Specifically, it uses APIs from railway and bus companies to collect timetable data, real-time service information, and location information for shared bicycles and taxis. This data is sent to the server in JSON format and stored in a database.

[1084] Input: Timetable data, real-time service information, and location information from railway and bus company APIs.

[1085] Output: Traffic data stored in the database

[1086] Step 2:

[1087] The device collects user emotion data. Specifically, it acquires voice data using the device's built-in microphone and captures facial recognition data with its camera. This data is then sent to the emotion engine.

[1088] Input: User voice data, facial recognition data

[1089] Output: Data sent to the emotion engine

[1090] Step 3:

[1091] The server analyzes the collected traffic data using AI algorithms. Specifically, it uses machine learning frameworks such as Python and TensorFlow to fill in missing information in timetables and predict real-time traffic conditions.

[1092] Input: Traffic data stored in the database

[1093] Output: Timetable data with missing information filled in, real-time traffic forecast

[1094] Step 4:

[1095] The server generates route options that combine multiple modes of transportation based on the analysis results. Specifically, it uses an algorithm that generates routes that combine walking, shared bicycles, taxis, and public transport (trains, buses, etc.).

[1096] Input: Timetable data with missing data imputed, real-time traffic forecast.

[1097] Output: Multiple route options

[1098] Step 5:

[1099] The server calculates the travel time for each route. Specifically, for each route, it adds up the walking time, the time spent using shared bicycles, the time spent using taxis, and the time spent using public transportation, and calculates the total travel time taking into account real-time traffic conditions.

[1100] Input: Multiple route options, real-time traffic forecast

[1101] Output: Travel time for each route

[1102] Step 6:

[1103] The server selects the optimal route by considering the user's emotional state. Specifically, it uses data from the emotion engine to weight travel time, cost, and comfort as evaluation criteria.

[1104] Input: Travel time for each route, user sentiment data

[1105] Output: Optimal Route

[1106] Step 7:

[1107] The server notifies the terminal of the optimal route and its estimated travel time. The user then selects the route best suited to them based on the provided route guidance.

[1108] Input: Optimal route

[1109] Output: Route information notified to the terminal

[1110] (Application Example 2)

[1111] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1112] Conventional travel route suggestion systems provide the optimal route considering real-time traffic conditions and multiple modes of transport, but they fail to take into account the user's emotional state, which can lead to an uncomfortable travel experience. Furthermore, when selecting the optimal route with multiple travel options, the inability to adjust based on the user's emotions can result in stress and discomfort. Therefore, a challenge for conventional technologies is the ability to provide flexible route suggestions that take the user's emotional state into account.

[1113] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a data collection means, a means for analyzing the collected data, a means for generating a route based on multiple means of transportation, a means for calculating the travel time for each route, a means for selecting the optimal route, a means for notifying the user of the optimal route, a means for recognizing and evaluating the user's emotional state, and a means for adjusting the selected route based on the recognized emotional state. This makes it possible to propose an optimal route that takes the user's emotional state into consideration, allowing the user to travel more comfortably.

[1114] "Data collection methods" refer to means for obtaining timetable data for various modes of transportation, real-time operation information, and location information for shared bicycles and taxis.

[1115] "Means for analyzing collected data" refers to methods for predicting traffic conditions and travel times based on the acquired data.

[1116] "Means for generating routes based on multiple modes of transportation" refers to methods for constructing routes that combine various modes of transportation such as walking, shared bicycles, taxis, on-demand buses, trains, and buses.

[1117] "Means for calculating the travel time for each route" refers to means for calculating the time required to travel along each generated route.

[1118] "Means of selecting the optimal route" refers to methods of choosing the most appropriate route, taking into account travel time, cost, comfort, and the user's emotional state.

[1119] "Means of notifying users of the optimal route" refers to means of informing users of the selected optimal route and its estimated travel time.

[1120] "Means for recognizing and evaluating the user's emotional state" refers to a means of determining the emotional state using an emotion engine based on the user's voice input or facial recognition data.

[1121] "Means for adjusting the selected route based on recognized emotional states" refers to means for optimizing the selected route according to the user's emotional state, such as stress, relaxation, or tension.

[1122] The system in this invention includes means for collecting data, means for analyzing the collected data, means for generating routes based on multiple means of transportation, means for calculating the travel time for each route, means for selecting the optimal route, means for notifying the user of the optimal route, means for recognizing and evaluating the user's emotional state, and means for adjusting the selected route based on the recognized emotional state.

[1123] The server uses transportation APIs to collect timetable data and real-time service information, as well as location data for shared bicycles and taxis. This includes data from railway companies, bus companies, shared bicycle providers, and taxi companies. Specific software used includes the Python requests library and the pandas library for data analysis.

[1124] The device collects user voice input and facial recognition data and sends it to the emotion engine. The smartphone's camera and microphone are primarily used, and the EmotionEngine is employed to analyze emotions. Libraries such as OpenCV and dlib are used for user facial recognition.

[1125] The server uses artificial intelligence to fill in missing information in timetables based on collected data and predicts real-time traffic conditions. This allows it to estimate travel times for shared bicycles and taxis. Furthermore, the server generates route options combining multiple modes of transport and calculates the travel time for each route. This process considers various modes of transport, including walking, shared bicycles, taxis, on-demand buses, trains, and buses. TensorFlow and PyTorch are suitable AI algorithms.

[1126] When a user enters their current location and destination into the device, the device analyzes the user's emotional state and suggests a route accordingly, such as when stress levels are high. This involves EmotionEngine evaluating the emotional state, and the server selecting the optimal route considering the user's emotional state. A comprehensive scoring algorithm that takes emotional analysis data into account is used to optimize the route based on the emotional state.

[1127] For example, if a user wants to travel from Tokyo Station to Shibuya Station, and their current emotional state is assessed as relaxed, the optimal route will be selected as travel by train. In this case, the server will use the following prompt:

[1128] "Please suggest the optimal route from Tokyo Station to Shibuya Station. The user's emotional state is currently assessed as relaxed. Based on real-time traffic conditions and the location of the autonomous taxi, calculate the best travel option."

[1129] In this way, the system can take into account the user's emotional state and suggest the optimal mode of transportation and route. This allows the user to reach their destination comfortably and efficiently.

[1130] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1131] Step 1:

[1132] The user enters their starting point and destination into the device. The device also collects the user's facial recognition data and voice data using the smartphone's camera and microphone. The input data consists of current location information and destination information from GPS, while the collected data consists of facial images and voice data.

[1133] Step 2:

[1134] The device sends the collected facial recognition data and voice data to the emotion engine to determine the user's emotional state. The emotion engine performs facial expression analysis and voice tone analysis to evaluate the user's emotional state (e.g., stress, relaxation, tension). The input is facial recognition data and voice data, and the output is the evaluated emotional state.

[1135] Step 3:

[1136] The server collects timetable data, real-time service information, and location information for shared bicycles and taxis through APIs of various transportation services. The requests library is used to access the API endpoints of transportation services. The input is the API endpoint information of the transportation service, and the output is the collected timetable data and real-time service information.

[1137] Step 4:

[1138] The server analyzes the collected data, fills in missing information, and predicts real-time traffic conditions. This analysis uses artificial intelligence algorithms to fill in missing information in timetables and predict traffic. The input is various types of collected traffic data, and the output is the completed timetable data and predicted traffic conditions.

[1139] Step 5:

[1140] Based on the analysis results, the server generates route options combining multiple modes of transportation. This route generation uses an AI algorithm that considers various modes of transport, including walking, shared bicycles, taxis, on-demand buses, trains, and regular buses. Inputs are supplemented timetable data and predicted traffic conditions, while output is multiple route options.

[1141] Step 6:

[1142] The server calculates the travel time for each route and selects the optimal route, taking into account the user's emotional state. Using an evaluation algorithm, it scores travel time, cost, comfort, and emotional state, and selects the route with the highest score. The input is multiple route options and the user's emotional state, and the output is the optimal route.

[1143] Step 7:

[1144] The server notifies the terminal of the optimal route and estimated travel time. The terminal displays the notified information to the user and provides an interface for the user to make a selection. The input is the optimal route and estimated travel time, and the output is the display information that the user can confirm.

[1145] Step 8:

[1146] The user begins moving according to the notified optimal route. The device continuously acquires real-time location information and recalculates and notifies the user of the route as needed. The input is the user's current location information, and the output is the updated route information.

[1147] By following the processing steps described above, this invention realizes a system that provides an optimal travel route that takes into account the user's emotional state, enabling the user to reach their destination comfortably and efficiently.

[1148] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1149] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1150] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1151] [Fourth Embodiment]

[1152] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1153] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1154] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1155] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1156] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1157] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1158] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1159] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1160] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1161] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1162] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1163] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1164] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1165] The system of the present invention includes means for data collection, means for analysis, means for route generation, means for calculating travel time, means for selecting the optimal route, and means for notifying the user of the optimal route. This system proposes the optimal means of transportation by collecting and analyzing timetable data for various modes of transport, as well as operation records and location information for shared bicycles, taxis, on-demand buses, etc.

[1166] Program Processing Overview

[1167] Data acquisition methods

[1168] The server collects timetable data from various transportation services via APIs. It also acquires real-time operational information from railway and bus companies. In addition, it collects real-time location information for shared bicycles and taxis.

[1169] Data analysis means

[1170] Based on the collected data, the server uses AI algorithms to analyze it. This includes filling in missing information in timetables, predicting real-time traffic conditions, and estimating travel times for shared bicycles and taxis.

[1171] Route generation means

[1172] Based on the analysis results, the server generates route options that combine multiple modes of transportation. These routes include walking, shared bicycles, taxis, on-demand buses, trains, and buses.

[1173] Travel time calculation means

[1174] The travel time for each route is added up, and the average travel time is calculated by taking real-time traffic conditions into account. In addition, multiple different route and mode of transport options are calculated simultaneously.

[1175] Optimal route selection method

[1176] The server selects the optimal route from the generated route options based on evaluation criteria such as travel time, cost, and comfort. This results in the most efficient route being suggested to the user.

[1177] User notification means

[1178] The server notifies the terminal of the optimal route and estimated travel time. Based on the information provided, the user can choose the best route according to their preferences.

[1179] Specific example

[1180] For example, let's consider a scenario involving travel from Tokyo Station to Haneda Airport.

[1181] 1. The user enters "Tokyo Station" as the departure point, "Haneda Airport" as the destination, and "10:00" as the departure time into the terminal.

[1182] 2. The server collects data from railway companies, taxi companies, and shared bicycle providers and predicts the travel time for each route.

[1183] 3. The server generates the following route options:

[1184] Route A: Travel from Tokyo Station to Shinagawa Station by train → From Shinagawa Station, take the Keikyu Line to Haneda Airport.

[1185] Route B: Take a taxi from Tokyo Station to Hamamatsucho Station → Take the Tokyo Monorail from Hamamatsucho Station to Haneda Airport

[1186] Route C: Use a shared bicycle from Tokyo Station → Take a train from the nearest station to Haneda Airport.

[1187] 4. The server evaluates the travel time, cost, and comfort of each route option and selects the optimal route.

[1188] 5. The server notifies the terminal of the selected optimal route and estimated travel time.

[1189] 6. The user selects a route and begins to travel.

[1190] Through the above processing, the system of the present invention proposes efficient and accurate means of transportation and routes to the user.

[1191] The following describes the processing flow.

[1192] Step 1:

[1193] The user enters their departure point, destination, and desired departure or arrival time into the terminal. For example, they might enter "Tokyo Station" as the departure point, "Haneda Airport" as the destination, and "10:00" as the departure time.

[1194] Step 2:

[1195] The terminal receives input and sends that data to the server. The data sent includes the departure point, destination, and departure time.

[1196] Step 3:

[1197] The server uses APIs from various transportation providers to collect timetable data and real-time service information. This includes information from railway companies, bus companies, shared bicycle providers, taxi companies, and others. The collected data is stored in a database.

[1198] Step 4:

[1199] The server analyzes the collected data. Using AI algorithms, it fills in missing information in timetables and predicts real-time traffic conditions. It also estimates travel times for shared bicycles and taxis.

[1200] Step 5:

[1201] The server generates route options that combine multiple modes of transportation. These include walking, shared bicycles, taxis, on-demand buses, trains, and buses. For example, it can generate multiple routes to get from Tokyo Station to Haneda Airport.

[1202] Step 6:

[1203] The server calculates the travel time for each route. It then sums up the segmented travel times for each mode of transport to calculate the total travel time. It also evaluates delay risk, taking real-time traffic conditions into account.

[1204] Step 7:

[1205] The server selects the optimal route from the generated route options based on evaluation criteria such as travel time, cost, and comfort. Each evaluation criterion is weighted, and the AI ​​performs an overall scoring.

[1206] Step 8:

[1207] The server notifies the terminal of the optimal route selected by the server and the estimated travel time. The notification includes detailed information about each route, such as travel time, cost, and waiting time.

[1208] Step 9:

[1209] The user reviews the route options they are notified of and selects the best route according to their preferences. The selected route is then fed back to the server via the device.

[1210] Step 10:

[1211] The server uses user selection information to provide additional data to the AI ​​algorithm, improving the accuracy of future route generation and evaluation processes. This feedback loop enhances the overall system performance.

[1212] (Example 1)

[1213] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1214] Conventional transportation systems struggled to effectively combine multiple modes of transport to suggest the optimal route, and even more so to select routes that took real-time traffic conditions into account. As a result, users experienced inconvenience during their journeys and were unable to travel efficiently.

[1215] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1216] In this invention, the server includes means for collecting timetable data, real-time operation information, and location information; means for analyzing the collected data using artificial intelligence to fill in gaps in timetable information and predict real-time traffic conditions; and means for generating routes that combine multiple means of transportation. This enables users to travel efficiently and quickly.

[1217] "Timetable data" refers to information about the operating schedules of public transportation.

[1218] "Real-time service information" refers to information that shows the current location of transportation services and how much of a delay or progress they are making compared to their scheduled service.

[1219] "Location information" refers to information that indicates the current location of a particular mode of transportation (e.g., shared bicycles, taxis, etc.).

[1220] Artificial intelligence is a technology that enables computer systems to learn on their own through data analysis and pattern recognition, and to automatically perform specific tasks.

[1221] "Means of transportation" is a general term encompassing various methods of getting around, such as trains, buses, taxis, shared bicycles, and walking.

[1222] A "route" is a series of paths from a starting point to a destination.

[1223] "Travel time" refers to the time it takes to travel from a starting point to a destination using a specific route.

[1224] "Expenses" refer to the amount of money required to use a particular mode of transportation.

[1225] "Comfort" refers to the degree of ride quality and convenience experienced during travel.

[1226] A "user terminal" is an electronic device used by a user to receive information.

[1227] "Data collection means" refers to the functions and methods for obtaining necessary information from transportation systems.

[1228] "Analysis means" refers to the functions and methods for processing and analyzing collected data.

[1229] "Generation means" refers to functions and methods for creating routes that combine multiple modes of transportation based on analyzed data.

[1230] "Notification means" refers to functions and methods for informing the user's terminal of selected routes and other information.

[1231] The system of the present invention includes data collection means, analysis means, route generation means, travel time calculation means, optimal route selection means, and user notification means. Specific embodiments of each of these means are described below.

[1232] Data acquisition methods

[1233] The server collects timetable data, real-time service information, and location information from various transportation services via APIs. For example, it uses APIs from railway and bus companies to obtain timetables and service status in real time. It also collects location information from shared bicycles and taxis using the same method. This allows all necessary data to be managed centrally.

[1234] Data analysis means

[1235] The server analyzes the collected data using artificial intelligence (AI). Here, machine learning models and deep learning algorithms are utilized to fill in missing information in timetables, predict real-time traffic conditions, and estimate travel times for various modes of transportation. Specific software used includes TensorFlow and PyTorch.

[1236] Route generation means

[1237] The server generates a route combining multiple modes of transportation based on the analysis results. This route may include walking, trains, buses, shared bicycles, taxis, and on-demand buses. The combination of each mode of transportation will vary depending on the user's starting point and destination.

[1238] Travel time calculation means

[1239] The server adds up the travel times for each route and calculates the average travel time, taking real-time traffic conditions into account. For example, it calculates a route that takes 10 minutes on foot, 30 minutes by train, and 20 minutes by taxi, and then adjusts for traffic congestion to predict an accurate travel time.

[1240] Optimal route selection method

[1241] The server selects the optimal route from the generated routes based on evaluation criteria such as travel time, cost, and comfort. The optimal route is designed to maximize the user's travel efficiency, and the algorithm compares and selects routes using multiple evaluation criteria.

[1242] User notification means

[1243] The server notifies the user's device of the optimal route and estimated travel time. Based on this information, the user selects a route that suits their preferences and begins their journey. This notification is delivered via a mobile app or web application.

[1244] Specific example

[1245] Scenario: Travel from Tokyo Station to Haneda Airport

[1246] 1. The user enters the departure point "Tokyo Station", destination "Haneda Airport", and departure time "10:00" into the terminal.

[1247] 2. The server collects data from railway companies, taxi companies, and shared bicycle providers and predicts the travel time for each route.

[1248] 3. The server generates the following route options:

[1249] Route A: Travel from Tokyo Station to Shinagawa Station by train → From Shinagawa Station to Haneda Airport by train.

[1250] Route B: Take a taxi from Tokyo Station to Hamamatsucho Station → Take the monorail from Hamamatsucho Station to Haneda Airport

[1251] Route C: Use a shared bicycle from Tokyo Station → Take a train from the nearest station to Haneda Airport.

[1252] 4. The server evaluates the travel time, cost, and comfort of each route option and selects the optimal route.

[1253] 5. The server notifies the terminal of the optimal route and estimated travel time.

[1254] 6. The user selects a route and begins to travel.

[1255] Examples of prompts for generative AI models

[1256] "Please suggest the best route from Tokyo Station to Haneda Airport. Available modes of transport include train, bus, taxi, and shared bicycle."

[1257] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1258] Step 1:

[1259] The server uses transportation APIs to collect timetable data, real-time service information, and location information for shared bicycles and taxis.

[1260] Input: API endpoints for each transportation provider.

[1261] Processing: Call the API to retrieve timetable data, real-time service information, and location information.

[1262] Output: Collected raw data (timetables, service status, location information).

[1263] Step 2:

[1264] The server passes the collected data to an artificial intelligence (AI) algorithm for data analysis.

[1265] Input: Raw data collected in Step 1.

[1266] Processing: Machine learning models and deep learning algorithms are used to fill in missing information in timetables, predict real-time traffic conditions, and estimate travel times for shared bicycles and taxis.

[1267] Output: Interpolated timetable data, predicted traffic conditions, and estimated travel time.

[1268] Step 3:

[1269] The server generates a route that combines multiple modes of transportation based on the analysis results.

[1270] Input: Analysis result data generated in Step 2.

[1271] Processing: Create the optimal route option by combining various modes of transportation such as walking, trains, buses, shared bicycles, taxis, and on-demand buses.

[1272] Output: Multiple route options (e.g., Route A, Route B, Route C).

[1273] Step 4:

[1274] The server calculates the travel time for each route and then calculates the average travel time, taking real-time traffic conditions into account.

[1275] Input: Route options generated in Step 3.

[1276] Processing: The travel time for each mode of transport along the route is added together, and the average travel time is calculated taking real-time traffic conditions into account.

[1277] Output: Average travel time for each route.

[1278] Step 5:

[1279] The server selects the optimal route based on evaluation criteria such as travel time, cost, and comfort.

[1280] Input: Average travel time data for each route generated in Step 4.

[1281] Processing: Evaluate the travel time, cost, and comfort of each route, and select the optimal route overall.

[1282] Output: Optimal route.

[1283] Step 6:

[1284] The server notifies the terminal of the selected optimal route and estimated travel time.

[1285] Input: The optimal route selected in Step 5.

[1286] Processing: Convert the optimal route and estimated travel time into a data format for user notifications.

[1287] Output: Notification data sent to the user's terminal (optimal route, estimated travel time).

[1288] Step 7:

[1289] The device displays the notified optimal route and estimated travel time to the user.

[1290] Input: Notification data sent from Step 6.

[1291] Processing: Display notification data on the device's screen interface.

[1292] Output: Presenting the user with the optimal route and estimated travel time.

[1293] Step 8:

[1294] The user begins their journey to their destination based on the displayed optimal route and estimated travel time.

[1295] Input: The optimal route and estimated travel time displayed on the device.

[1296] Process: The user reviews the information and begins moving according to the selected route.

[1297] Output: Actual movement begins.

[1298] (Application Example 1)

[1299] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1300] Traditional food delivery systems did not effectively utilize real-time traffic and location information to optimize delivery routes, resulting in decreased delivery efficiency and longer delivery times. Furthermore, their inability to adapt to sudden changes in traffic conditions led to delivery delays and increased costs.

[1301] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1302] In this invention, the server includes means for collecting data, means for analyzing the collected data, means for generating routes based on multiple means of transportation, means for calculating the travel time for each route, means for selecting the optimal route, means for notifying the user of the optimal route, and means for generating efficient food delivery routes using real-time transportation data and location information and notifying delivery personnel of the optimal route. This significantly improves delivery efficiency, shortens delivery times, and enables rapid response to sudden changes in traffic conditions.

[1303] "Data collection methods" refer to means for acquiring real-time data and location information from various modes of transportation.

[1304] "Means for analyzing collected data" refers to methods for predicting and analyzing real-time traffic conditions based on collected data.

[1305] "Means for generating routes based on multiple modes of transportation" refers to methods for generating routes by combining multiple modes of transportation such as walking, cycling, taxis, buses, and trains.

[1306] "Means for calculating the travel time for each route" refers to the means for calculating the travel time for the generated routes.

[1307] "Means for selecting the optimal route" refers to methods for selecting the optimal route based on criteria such as travel time, cost, and comfort.

[1308] "Means for notifying the user of the optimal route" refers to the means of notifying the user of the selected optimal route.

[1309] "A means of generating efficient routes for food delivery using real-time data and location information from transportation systems and notifying delivery personnel of the optimal route" refers to a means of generating the optimal route for food delivery based on real-time data and location information and notifying delivery personnel of that route.

[1310] Modes for carrying out the invention

[1311] The present invention is a system that generates an optimal route using real-time transportation data and location information to improve the efficiency of food delivery and notifies delivery personnel of it. The embodiments thereof will be described in detail below.

[1312] System Configuration

[1313] The system of this invention consists of a server and a user terminal (the delivery person's smartphone).

[1314] 1. Data acquisition methods

[1315] The server uses APIs to collect real-time data and location information from various modes of transportation. Specific examples of APIs include bus data APIs, train data APIs, shared bicycle data APIs, and taxi data APIs.

[1316] 2. Data Analysis Methods

[1317] The server analyzes the collected real-time data using AI algorithms (e.g., TensorFlow or PyTorch) to predict traffic conditions. The analysis results are then used to generate routes.

[1318] 3. Route generation means

[1319] Based on the analyzed data, the server generates an efficient delivery route by combining multiple modes of transportation. This route includes walking, cycling, taxis, and public transport (buses, trains).

[1320] 4. Travel time calculation method

[1321] The server calculates the travel time for each generated route. This calculation takes into account real-time traffic conditions and forecast data.

[1322] 5. Optimal Route Selection Method

[1323] The server selects the optimal route from multiple options based on evaluation criteria such as travel time, cost, and comfort.

[1324] 6. User notification means

[1325] The server notifies the delivery person's terminal of the selected optimal route. Upon receiving the notification, the delivery person efficiently carries out deliveries according to the optimal route.

[1326] Hardware and software configuration to be used

[1327] Flask: A lightweight, Python-based web framework that handles communication between the server and the user's terminal, as well as data retrieval and analysis.

[1328] Requests library: A library for sending HTTP requests to various APIs.

[1329] AI algorithms (TensorFlow, PyTorch): Used in the data analysis portion to predict traffic conditions and impute missing data.

[1330] Specific example

[1331] For example, consider a case where a delivery person is making a delivery from Shinjuku Station to Roppongi Hills.

[1332] 1. Enter your starting point and destination.

[1333] The delivery person enters the starting point "Shinjuku Station" and the destination "Roppongi Hills" into a smartphone app.

[1334] 2. Collection of real-time data

[1335] The server collects real-time data on buses, trains, taxis, and shared bicycles.

[1336] 3. Analysis of traffic conditions

[1337] The server uses AI algorithms to analyze and predict traffic conditions based on the collected data.

[1338] 4. Route generation

[1339] The server generates multiple routes based on the analysis results. For example, a route using the subway, a route combining bicycles and trains, etc.

[1340] 5. Selection of the optimal route

[1341] The server selects the optimal route from the generated routes, taking into account travel time and comfort.

[1342] 6. Route notification

[1343] The optimal route is notified to the delivery person's smartphone. The delivery person follows that route and makes deliveries efficiently.

[1344] Example of a prompt

[1345] The starting point is Shinjuku Station, and the destination is Roppongi Hills. Please suggest the best route considering the current traffic conditions. Options include train, bus, taxi, bicycle, or walking. Please also provide the estimated travel time and cost for each route.

[1346] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1347] Step 1:

[1348] The user enters their departure point and destination into their device (smartphone). This input data is sent to the server. The entered data includes the departure point, destination, and departure time.

[1349] Step 2:

[1350] The server collects real-time data on transportation. Specifically, it uses various APIs for buses, trains, shared bicycles, taxis, etc., to obtain timetable data and real-time location information. This provides data on the current traffic situation.

[1351] Step 3:

[1352] The server analyzes the collected data and predicts real-time traffic conditions. This analysis uses AI algorithms (e.g., TensorFlow and PyTorch) to fill in missing information in timetables and predict traffic conditions. The input here is the collected real-time data, and the output is the analysis results.

[1353] Step 4:

[1354] The server generates a route combining multiple modes of transportation based on the analysis results. This route generation includes modes of transport such as walking, cycling, taxi, bus, and train. Specifically, it generates multiple route paths from the starting point to the destination and selects a mode of transport for each intermediate step. The input is the analysis results, and the output is multiple route options.

[1355] Step 5:

[1356] The server calculates the travel time for each route. This calculation takes into account real-time traffic conditions and forecast data, such as traffic congestion and bus delays. The input is the generated route options, and the output is the travel time for each route.

[1357] Step 6:

[1358] The server selects the optimal route from multiple options. It determines the best route based on evaluation criteria such as travel time, cost, and comfort. Specifically, it scores the evaluation criteria for each route and selects the route with the highest overall score. The input is evaluation data such as travel time and cost for each route, and the output is the optimal route.

[1359] Step 7:

[1360] The server notifies the user's terminal of the selected optimal route. The terminal displays the received route information to the delivery person. The delivery person uses this information to make deliveries along the optimal route. The input is the selected optimal route, and the output is the notification to the terminal.

[1361] As a concrete example of how it works, the user inputs the starting point "Shinjuku Station" and the destination "Roppongi Hills." The server collects real-time data and analyzes it using an AI algorithm. It generates multiple routes and calculates the travel time for each. It selects the optimal route and notifies the user's terminal. In this way, delivery drivers can perform deliveries efficiently.

[1362] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1363] The system of the present invention includes means for data collection, analysis, route generation, travel time calculation, optimal route selection, and notification of the optimal route to the user, as well as an emotion engine that recognizes the user's emotions. This system proposes the optimal mode of transportation by collecting and analyzing timetable data from various modes of transport, as well as operation records and location information from shared bicycles, taxis, and on-demand buses. Furthermore, the emotion engine can evaluate the user's emotional state and adjust the route based on that evaluation.

[1364] Program Processing Overview

[1365] Data acquisition methods

[1366] The server collects timetable data from various transportation services via APIs. It also acquires real-time operational information from railway and bus companies. In addition, it collects real-time location information for shared bicycles and taxis.

[1367] emotion recognition means

[1368] The device collects user voice input and facial recognition data and sends it to the emotion engine. The emotion engine analyzes this data and evaluates the user's emotional state (stress, relaxation, tension, etc.).

[1369] Data analysis means

[1370] Based on the collected data, the server uses AI algorithms to analyze it. This includes filling in missing information in timetables, predicting real-time traffic conditions, and estimating travel times for shared bicycles and taxis.

[1371] Route generation means

[1372] Based on the analysis results, the server generates route options that combine multiple modes of transportation. These include walking, shared bicycles, taxis, on-demand buses, trains, and regular buses.

[1373] Travel time calculation means

[1374] The travel time for each route is added up, and the average travel time is calculated by taking real-time traffic conditions into account. In addition, multiple different route and mode of transport options are calculated simultaneously.

[1375] Optimal route selection method

[1376] The server selects the optimal route from the generated route options, taking into account travel time, cost, comfort, and the user's emotional state. The evaluation criteria are weighted, and the AI ​​performs an overall scoring.

[1377] User notification means

[1378] The server notifies the terminal of the optimal route and estimated travel time. Based on the information provided, the user can choose the best route according to their preferences.

[1379] Specific example

[1380] For example, consider the following scenario of traveling from Tokyo Station to Haneda Airport.

[1381] 1. The user enters "Tokyo Station" as the departure point, "Haneda Airport" as the destination, and "10:00" as the departure time into the terminal. The terminal also collects the user's voice and facial recognition data and sends it to the emotion engine.

[1382] 2. The server collects data from railway companies, taxi companies, and shared bicycle providers and predicts the travel time for each route.

[1383] 3. The server generates the following route options:

[1384] Route A: Travel from Tokyo Station to Shinagawa Station by train → From Shinagawa Station, take the Keikyu Line to Haneda Airport.

[1385] Route B: Take a taxi from Tokyo Station to Hamamatsucho Station → Take the Tokyo Monorail from Hamamatsucho Station to Haneda Airport

[1386] Route C: Use a shared bicycle from Tokyo Station → Take a train from the nearest station to Haneda Airport.

[1387] 4. The emotion engine evaluates the user's emotional state (e.g., high stress) and prioritizes selecting routes that cause less stress.

[1388] 5. The server selects the optimal route and notifies the terminal of the estimated travel time.

[1389] 6. The user selects a route and begins to travel.

[1390] Through the above processing, the system of the present invention proposes efficient and accurate means of transportation and routes to the user, and further provides a comfortable travel experience based on the user's emotional state.

[1391] The following describes the processing flow.

[1392] Step 1:

[1393] The user enters their departure point, destination, and desired departure or arrival time into the terminal. For example, they might enter "Tokyo Station" as the departure point, "Haneda Airport" as the destination, and "10:00" as the departure time.

[1394] Step 2:

[1395] The terminal receives input and sends that data to the server. The data sent includes the departure point, destination, and departure time.

[1396] Step 3:

[1397] The device collects user voice input and facial recognition data. This uses the smartphone's microphone and camera. The collected data is sent to an emotion engine.

[1398] Step 4:

[1399] The server uses APIs from various transportation providers to collect timetable data and real-time service information. This includes information from railway companies, bus companies, shared bicycle providers, taxi companies, and others. The collected data is stored in a database.

[1400] Step 5:

[1401] The server analyzes the collected data. Using AI algorithms, it fills in missing information in timetables, predicts real-time traffic conditions, and estimates travel times for shared bicycles and taxis.

[1402] Step 6:

[1403] The emotion engine analyzes voice input and facial recognition data transmitted from the device to evaluate the user's emotional state. For example, it determines stress levels and relaxation levels from changes in the user's voice tone and facial expressions.

[1404] Step 7:

[1405] The server generates route options that combine multiple modes of transportation based on the analysis results. These include walking, shared bicycles, taxis, on-demand buses, trains, and buses.

[1406] Step 8:

[1407] The server calculates the travel time for each route. It then sums up the segmented travel times for each mode of transport to calculate the total travel time. It also evaluates delay risk, taking real-time traffic conditions into account.

[1408] Step 9:

[1409] The server selects the optimal route from the generated route options, taking into account travel time, cost, comfort, and the user's emotional state. For example, if the user is feeling stressed, it will prioritize a more relaxing route.

[1410] Step 10:

[1411] The server notifies the terminal of the optimal route selected by the server and the estimated travel time. The notification includes detailed information about each route, such as travel time, cost, and waiting time.

[1412] Step 11:

[1413] The user reviews the route options they are notified of and selects the best route according to their preferences. The selected route is then fed back to the server via the device.

[1414] Step 12:

[1415] The server uses user selection information to incorporate it into AI algorithms as additional data, improving the accuracy of future route generation and evaluation processes. This feedback loop improves the overall system performance.

[1416] (Example 2)

[1417] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1418] Conventional transportation suggestion systems propose optimal routes based on transportation data, but they fail to consider the user's emotional state, making it difficult to reduce stress or improve comfort. Furthermore, they are unable to respond to missing timetable information or real-time changes in traffic conditions. As a result, the user's travel experience is not always optimal.

[1419] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1420] In this invention, the server includes means for collecting data from various modes of transportation, means for recognizing the user's emotional state, means for analyzing the collected data, means for generating routes based on multiple modes of transportation, means for calculating the travel time for each route, means for selecting the optimal route considering the user's emotional state, and means for notifying the user of the optimal route. This makes it possible to propose the optimal travel route considering the user's emotional state, thereby reducing stress and improving comfort. Furthermore, it becomes possible to fill in gaps in timetables based on real-time information and accurately predict travel times.

[1421] "Means of collecting data" refers to functions that acquire timetable data, real-time operation information, and location information from various transportation services.

[1422] "Means for recognizing the user's emotional state" refers to a function that collects user voice data and facial recognition data, analyzes it, and evaluates the user's emotional state.

[1423] "Means for analyzing collected data" refers to a function that uses AI algorithms to analyze collected traffic data, supplementing missing information in timetables and predicting real-time traffic conditions.

[1424] "Means for generating routes based on multiple modes of transportation" refers to a function that generates route options combining multiple modes of transportation, such as walking, shared bicycles, taxis, trains, and buses, based on analyzed data.

[1425] "Means for calculating travel time for each route" refers to a function that calculates the accurate travel time for each mode of transport by summing up the travel times for each generated route option and taking real-time traffic conditions into account.

[1426] "Means of selecting the optimal route while considering the user's emotional state" refers to a function that evaluates the user's emotional state (stress, relaxation, etc.) and, based on that, selects the optimal route while considering travel time, cost, and comfort.

[1427] "Means of notifying users of the optimal route" refers to a function that notifies the user's device of the selected optimal route and travel time, allowing the user to select the optimal route based on that information.

[1428] The system of this invention operates in a coordinated manner involving a server, a terminal, and a user. The following describes specific implementations of this system.

[1429] Data collection

[1430] The server implements APIs to collect necessary data from various modes of transportation. This includes timetable data from railway and bus companies, real-time service information, location data for shared bicycles, and real-time location data for taxis. This data is stored in the server's database and used for later analysis.

[1431] emotion recognition

[1432] The device incorporates sensors and an analysis engine to recognize the user's emotional state. Voice input is acquired via the built-in microphone, and facial recognition data is acquired via the built-in camera. This data is sent to the emotion engine to evaluate the user's emotional state (stress, relaxation, etc.).

[1433] Data Analysis

[1434] The server analyzes the collected traffic data using AI algorithms. Specifically, it performs tasks such as filling in missing information in timetables and predicting real-time traffic conditions. Machine learning frameworks such as Python and TensorFlow are used for this purpose.

[1435] Route generation

[1436] The server generates route options that combine multiple modes of transportation based on the analysis results. Combinations of walking, shared bicycles, taxis, and public transport (trains, buses, etc.) are considered.

[1437] Travel time calculation

[1438] For each route, the server calculates the travel time. This calculation includes real-time traffic data. An algorithm is used to aggregate the travel times by combining the duration of each segment.

[1439] Optimal Route Selection

[1440] The server selects the optimal route while considering the user's emotional state. Travel time, cost, and comfort are evaluated, and weighting is applied based on data from the emotion engine to prioritize the least stressful route.

[1441] User notifications

[1442] The server notifies the terminal of the optimal route and estimated travel time. Based on the notified route guidance, the user selects the best route according to their preferences and circumstances and begins their journey.

[1443] Specific example

[1444] For example, consider the following scenario when thinking about travel from Tokyo Station to Haneda Airport.

[1445] 1. The user enters "Tokyo Station" as the departure point, "Haneda Airport" as the destination, and "10:00" as the departure time into the terminal. The terminal also collects the user's voice and facial recognition data and sends it to the emotion engine.

[1446] 2. The server collects data from transportation companies via APIs and stores timetable data and real-time location information in a database.

[1447] 3. The server uses AI algorithms to analyze the collected data, fill in missing information, and predict real-time traffic conditions.

[1448] 4. The server generates multiple route options, specifically the following possible routes:

[1449] Route A: Travel from Tokyo Station to Shinagawa Station by train → From Shinagawa Station, take the Keikyu Line to Haneda Airport.

[1450] Route B: Take a taxi from Tokyo Station to Hamamatsucho Station → Take the Tokyo Monorail from Hamamatsucho Station to Haneda Airport

[1451] Route C: Use a shared bicycle from Tokyo Station → Take a train from the nearest station to Haneda Airport.

[1452] 5. The server calculates the travel time for each route, taking real-time traffic data into consideration.

[1453] 6. The emotion engine evaluates the user's emotional state (e.g., high stress) and weights the routes that result in less stress.

[1454] 7. The server selects the optimal route and notifies the terminal along with the estimated travel time. The user then selects the most suitable route based on this information and begins their journey.

[1455] Example of a prompt

[1456] "I need to travel from Tokyo Station to Haneda Airport, departing at 10:00 AM. My current emotional state is high due to stress. Please suggest the best route."

[1457] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1458] Program processing flow

[1459] Step 1:

[1460] The server collects data from various modes of transportation. Specifically, it uses APIs from railway and bus companies to collect timetable data, real-time service information, and location information for shared bicycles and taxis. This data is sent to the server in JSON format and stored in a database.

[1461] Input: Timetable data, real-time service information, and location information from railway and bus company APIs.

[1462] Output: Traffic data stored in the database

[1463] Step 2:

[1464] The device collects user emotion data. Specifically, it acquires voice data using the device's built-in microphone and captures facial recognition data with its camera. This data is then sent to the emotion engine.

[1465] Input: User voice data, facial recognition data

[1466] Output: Data sent to the emotion engine

[1467] Step 3:

[1468] The server analyzes the collected traffic data using AI algorithms. Specifically, it uses machine learning frameworks such as Python and TensorFlow to fill in missing information in timetables and predict real-time traffic conditions.

[1469] Input: Traffic data stored in the database

[1470] Output: Timetable data with missing information filled in, real-time traffic forecast

[1471] Step 4:

[1472] The server generates route options that combine multiple modes of transportation based on the analysis results. Specifically, it uses an algorithm that generates routes that combine walking, shared bicycles, taxis, and public transport (trains, buses, etc.).

[1473] Input: Timetable data with missing data imputed, real-time traffic forecast.

[1474] Output: Multiple route options

[1475] Step 5:

[1476] The server calculates the travel time for each route. Specifically, for each route, it adds up the walking time, the time spent using shared bicycles, the time spent using taxis, and the time spent using public transportation, and calculates the total travel time taking into account real-time traffic conditions.

[1477] Input: Multiple route options, real-time traffic forecast

[1478] Output: Travel time for each route

[1479] Step 6:

[1480] The server selects the optimal route by considering the user's emotional state. Specifically, it uses data from the emotion engine to weight travel time, cost, and comfort as evaluation criteria.

[1481] Input: Travel time for each route, user sentiment data

[1482] Output: Optimal Route

[1483] Step 7:

[1484] The server notifies the terminal of the optimal route and its estimated travel time. The user then selects the route best suited to them based on the provided route guidance.

[1485] Input: Optimal route

[1486] Output: Route information notified to the terminal

[1487] (Application Example 2)

[1488] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1489] Conventional travel route suggestion systems provide the optimal route considering real-time traffic conditions and multiple modes of transport, but they fail to take into account the user's emotional state, which can lead to an uncomfortable travel experience. Furthermore, when selecting the optimal route with multiple travel options, the inability to adjust based on the user's emotions can result in stress and discomfort. Therefore, a challenge for conventional technologies is the ability to provide flexible route suggestions that take the user's emotional state into account.

[1490] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a data collection means, a means for analyzing the collected data, a means for generating a route based on multiple means of transportation, a means for calculating the travel time for each route, a means for selecting the optimal route, a means for notifying the user of the optimal route, a means for recognizing and evaluating the user's emotional state, and a means for adjusting the selected route based on the recognized emotional state. This makes it possible to propose an optimal route that takes the user's emotional state into consideration, allowing the user to travel more comfortably.

[1491] "Data collection methods" refer to means for obtaining timetable data for various modes of transportation, real-time operation information, and location information for shared bicycles and taxis.

[1492] "Means for analyzing collected data" refers to methods for predicting traffic conditions and travel times based on the acquired data.

[1493] "Means for generating routes based on multiple modes of transportation" refers to methods for constructing routes that combine various modes of transportation such as walking, shared bicycles, taxis, on-demand buses, trains, and buses.

[1494] "Means for calculating the travel time for each route" refers to means for calculating the time required to travel along each generated route.

[1495] "Means of selecting the optimal route" refers to methods of choosing the most appropriate route, taking into account travel time, cost, comfort, and the user's emotional state.

[1496] "Means of notifying users of the optimal route" refers to means of informing users of the selected optimal route and its estimated travel time.

[1497] "Means for recognizing and evaluating the user's emotional state" refers to a means of determining the emotional state using an emotion engine based on the user's voice input or facial recognition data.

[1498] "Means for adjusting the selected route based on recognized emotional states" refers to means for optimizing the selected route according to the user's emotional state, such as stress, relaxation, or tension.

[1499] The system in this invention includes means for collecting data, means for analyzing the collected data, means for generating routes based on multiple means of transportation, means for calculating the travel time for each route, means for selecting the optimal route, means for notifying the user of the optimal route, means for recognizing and evaluating the user's emotional state, and means for adjusting the selected route based on the recognized emotional state.

[1500] The server uses transportation APIs to collect timetable data and real-time service information, as well as location data for shared bicycles and taxis. This includes data from railway companies, bus companies, shared bicycle providers, and taxi companies. Specific software used includes the Python requests library and the pandas library for data analysis.

[1501] The device collects user voice input and facial recognition data and sends it to the emotion engine. The smartphone's camera and microphone are primarily used, and the EmotionEngine is employed to analyze emotions. Libraries such as OpenCV and dlib are used for user facial recognition.

[1502] The server uses artificial intelligence to fill in missing information in timetables based on collected data and predicts real-time traffic conditions. This allows it to estimate travel times for shared bicycles and taxis. Furthermore, the server generates route options combining multiple modes of transport and calculates the travel time for each route. This process considers various modes of transport, including walking, shared bicycles, taxis, on-demand buses, trains, and buses. TensorFlow and PyTorch are suitable AI algorithms.

[1503] When a user enters their current location and destination into the device, the device analyzes the user's emotional state and suggests a route accordingly, such as when stress levels are high. This involves EmotionEngine evaluating the emotional state, and the server selecting the optimal route considering the user's emotional state. A comprehensive scoring algorithm that takes emotional analysis data into account is used to optimize the route based on the emotional state.

[1504] For example, if a user wants to travel from Tokyo Station to Shibuya Station, and their current emotional state is assessed as relaxed, the optimal route will be selected as travel by train. In this case, the server will use the following prompt:

[1505] "Please suggest the optimal route from Tokyo Station to Shibuya Station. The user's emotional state is currently assessed as relaxed. Based on real-time traffic conditions and the location of the autonomous taxi, calculate the best travel option."

[1506] In this way, the system can take into account the user's emotional state and suggest the optimal mode of transportation and route. This allows the user to reach their destination comfortably and efficiently.

[1507] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1508] Step 1:

[1509] The user enters their starting point and destination into the device. The device also collects the user's facial recognition data and voice data using the smartphone's camera and microphone. The input data consists of current location information and destination information from GPS, while the collected data consists of facial images and voice data.

[1510] Step 2:

[1511] The device sends the collected facial recognition data and voice data to the emotion engine to determine the user's emotional state. The emotion engine performs facial expression analysis and voice tone analysis to evaluate the user's emotional state (e.g., stress, relaxation, tension). The input is facial recognition data and voice data, and the output is the evaluated emotional state.

[1512] Step 3:

[1513] The server collects timetable data, real-time service information, and location information for shared bicycles and taxis through APIs of various transportation services. The requests library is used to access the API endpoints of transportation services. The input is the API endpoint information of the transportation service, and the output is the collected timetable data and real-time service information.

[1514] Step 4:

[1515] The server analyzes the collected data, fills in missing information, and predicts real-time traffic conditions. This analysis uses artificial intelligence algorithms to fill in missing information in timetables and predict traffic. The input is various types of collected traffic data, and the output is the completed timetable data and predicted traffic conditions.

[1516] Step 5:

[1517] Based on the analysis results, the server generates route options combining multiple modes of transportation. This route generation uses an AI algorithm that considers various modes of transport, including walking, shared bicycles, taxis, on-demand buses, trains, and regular buses. Inputs are supplemented timetable data and predicted traffic conditions, while output is multiple route options.

[1518] Step 6:

[1519] The server calculates the travel time for each route and selects the optimal route, taking into account the user's emotional state. Using an evaluation algorithm, it scores travel time, cost, comfort, and emotional state, and selects the route with the highest score. The input is multiple route options and the user's emotional state, and the output is the optimal route.

[1520] Step 7:

[1521] The server notifies the terminal of the optimal route and estimated travel time. The terminal displays the notified information to the user and provides an interface for the user to make a selection. The input is the optimal route and estimated travel time, and the output is the display information that the user can confirm.

[1522] Step 8:

[1523] The user begins moving according to the notified optimal route. The device continuously acquires real-time location information and recalculates and notifies the user of the route as needed. The input is the user's current location information, and the output is the updated route information.

[1524] By following the processing steps described above, this invention realizes a system that provides an optimal travel route that takes into account the user's emotional state, enabling the user to reach their destination comfortably and efficiently.

[1525] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1526] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1527] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1528] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1529] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1530] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1531] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1532] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1533] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1534] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1535] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1536] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1537] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1538] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1539] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1540] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1541] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1542] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1543] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1544] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1545] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1546] The following is further disclosed regarding the embodiments described above.

[1547] (Claim 1)

[1548] Data collection means,

[1549] Means for analyzing the collected data,

[1550] Means for generating routes based on multiple means of transportation,

[1551] A means of calculating the travel time for each route,

[1552] A means of selecting the optimal route,

[1553] A means of notifying the user of the optimal route,

[1554] A system that includes this.

[1555] (Claim 2)

[1556] The system according to claim 1 for collecting timetable data, real-time operation information, and location information.

[1557] (Claim 3)

[1558] The system according to claim 1, which uses artificial intelligence to analyze collected data and fill in gaps in timetable information.

[1559] "Example 1"

[1560] (Claim 1)

[1561] A means for collecting timetable data, real-time operation information, and location information,

[1562] A method for analyzing collected data using artificial intelligence to fill in gaps in timetable information and predict real-time traffic conditions,

[1563] A means for generating a route that combines multiple means of transportation,

[1564] A method for calculating travel time for each route and calculating travel time considering real-time traffic conditions,

[1565] A method for selecting the optimal route based on evaluation criteria such as travel time, cost, and comfort,

[1566] A means of notifying the user terminal of the optimal route and estimated travel time,

[1567] A system that includes this.

[1568] (Claim 2)

[1569] The system according to claim 1, which collects data from railway companies, bus companies, shared bicycle providers and taxi companies.

[1570] (Claim 3)

[1571] The system according to claim 1, which uses artificial intelligence to analyze collected data, estimates the time required for each mode of transport, and selects the optimal route.

[1572] "Application Example 1"

[1573] (Claim 1)

[1574] Data collection means,

[1575] Means for analyzing the collected data,

[1576] Means for generating routes based on multiple means of transportation,

[1577] A means of calculating the travel time for each route,

[1578] A means of selecting the optimal route,

[1579] A means of notifying the user of the optimal route,

[1580] A means for generating efficient food delivery routes using real-time data and location information from transportation systems and notifying delivery personnel of the optimal route,

[1581] A system that includes this.

[1582] (Claim 2)

[1583] The system according to claim 1 for collecting timetable data, real-time operation information, and location information.

[1584] (Claim 3)

[1585] The system according to claim 1, which uses artificial intelligence to analyze collected data and fill in gaps in timetable information.

[1586] "Example 2 of combining an emotion engine"

[1587] (Claim 1)

[1588] Means of collecting data from various modes of transportation,

[1589] A means of recognizing the user's emotional state,

[1590] Means for analyzing the collected data,

[1591] Means for generating routes based on multiple means of transportation,

[1592] A means of calculating the travel time for each route,

[1593] A method for selecting the optimal route while considering the user's emotional state,

[1594] A means of notifying the user of the optimal route,

[1595] A system that includes this.

[1596] (Claim 2)

[1597] The system according to claim 1, which collects timetable data, real-time operation information, and location information, and adjusts the route based on the user's emotional state.

[1598] (Claim 3)

[1599] The system according to claim 1, which uses artificial intelligence to analyze collected data, fills in gaps in timetable information, and assigns weights based on the user's emotional state.

[1600] "Application example 2 when combining with an emotional engine"

[1601] (Claim 1)

[1602] Data collection means,

[1603] Means for analyzing the collected data,

[1604] Means for generating routes based on multiple means of transportation,

[1605] A means of calculating the travel time for each route,

[1606] A means of selecting the optimal route,

[1607] A means of notifying the user of the optimal route,

[1608] Means for recognizing and evaluating the user's emotional state,

[1609] A means of adjusting the selected route based on the recognized emotional state,

[1610] A system that includes this.

[1611] (Claim 2)

[1612] The system according to claim 1 for collecting timetable data, real-time operation information, and location information.

[1613] (Claim 3)

[1614] The system according to claim 1, which uses artificial intelligence to analyze collected data, fills in gaps in timetable information, and further analyzes the user's emotional state to adjust the travel route. [Explanation of Symbols]

[1615] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Data collection means, Means for analyzing the collected data, Means for generating routes based on multiple means of transportation, A means of calculating the travel time for each route, A means of selecting the optimal route, A means of notifying the user of the optimal route, A system that includes this.

2. The system according to claim 1 for collecting timetable data, real-time operation information, and location information.

3. The system according to claim 1, which uses artificial intelligence to analyze collected data and fill in gaps in timetable information.

Images